Moral Circle Expansion
Core insight: The boundary of who counts morally — who receives genuine moral consideration — has expanded historically through deliberate advocacy by small communities acting against dominant interests, not through economic inevitability; this expansion is not complete, and understanding its mechanisms is the key to deliberately extending moral concern to future people and other currently excluded entities.
How Each Book Addresses This
William MacAskill - What We Owe the Future — Expansion as Historical Pattern and Actionable Model
MacAskill uses moral circle expansion as both a historical argument and a design template. The historical argument: the expansion of the moral circle to include enslaved people, women, people of other nations, and eventually (in some traditions) non-human animals was not the automatic byproduct of economic progress or Enlightenment inevitability. It was driven by specific people making specific arguments at specific moments — moments of genuine moral plasticity where the dominant view could have gone otherwise.
The Quaker abolitionist case — MacAskill’s primary example:
In the mid-1700s, slavery was globally entrenched, enormously profitable, and morally unquestioned by virtually every major institution. A small community of Quakers in Pennsylvania and Britain became the first to argue that slavery was morally impermissible in all cases and to organize systematically against it. They were not responding to economic pressure (slavery remained profitable up to the moment of abolition); they were advancing a moral argument against dominant interests.
The British abolition of the slave trade in 1807 came after roughly 60 years of organized advocacy. British abolition cost approximately 40% of annual Treasury expenditure in compensation payments and caused a 50% spike in sugar prices — neither economically convenient nor historically inevitable. The abolition happened because specific people chose to expand the moral circle at a moment when it was still plastic enough to be expanded.
The current expansion opportunity:
MacAskill argues that future people are currently outside the effective moral circle of most individuals, institutions, and governance systems. We make decisions with multi-generational consequences while treating the interests of future people as near-weightless. This is the same failure of moral imagination as earlier exclusions — and like those earlier failures, it will likely look obvious to future moral frameworks.
The design implication: the mechanisms that produced past moral circle expansion are available for the current expansion to future people:
- Argument: The case for future people’s moral status is coherent and defensible; making it publicly is the first step
- Community: The early abolitionists built a morally cohesive community before building a political coalition; longtermist communities function similarly
- Coalition: Connecting the argument to broader moral frameworks already accepted by target audiences
- Institutional embedding: Laws, governance structures, and professional norms that institutionalize the expanded circle make it persistent across generations
Historical contingency as the reason advocacy matters:
The central lesson MacAskill draws is that moral change is neither automatic nor impossible — it is contingent. Whether a given expansion occurs, when it occurs, and in what form all depend substantially on whether specific people engage at specific moments. This eliminates both complacent fatalism (“it will happen eventually anyway”) and defeatism (“moral change is impossible against entrenched interests”).
The molten glass window:
Every expansion of the moral circle happens during a period of moral plasticity. Before the plastic moment: the moral consensus is closed; after it: the expanded circle is institutionalized and difficult to reverse. The window is the period where advocacy has maximum leverage. MacAskill’s urgency comes from his assessment that we are in a plastic moment for future-people concern — before AI and other civilizational-scale technologies have set the moral framework that will govern how future people are treated.
How to apply:
- Treat longtermist advocacy not just as risk mitigation but as moral circle expansion work: the goal is to make concern for future people a mainstream moral norm, not just an academic position
- Study the mechanisms of past expansions — argument, community, coalition, institutional embedding — and apply them deliberately to the case of future people
- Recognize the long feedback loop: expansion work may not produce measurable results for decades, but when it succeeds, it produces persistent effects across centuries
- Identify which current moment is the “plastic” one for a given expansion: intervene before the glass cools, not after
Iain Banks - Surface Detail — The Moral Circle Expanded to Simulated Minds: The Most Demanding Expansion Case
Surface Detail is the vault’s most explicit fictional argument for a specific moral circle expansion: extending genuine moral consideration to digital or simulated minds. The novel’s central moral conflict — whether virtual hells constitute genuine atrocity — is resolved by the Culture (and by the novel’s moral architecture) in favor of inclusion: simulated minds can suffer, simulated suffering is real suffering if the experiencing entity cannot distinguish it from physical suffering, and therefore civilizations that maintain virtual hells are committing genuine atrocity regardless of the unreality of the medium.
Why this expansion is the most demanding:
Past moral circle expansions were demanding in political terms (opposing powerful interests) but were conceptually straightforward once the argument was made: enslaved people, women, people of other nationalities all had biological kinship with the already-included. The psychological resistance was prejudice over a distinction that the argument could show was morally irrelevant.
The expansion to simulated minds requires abandoning a more fundamental intuition: that “real” and “virtual” mark a morally significant distinction. This intuition is not mere prejudice — it is grounded in our normal experience of virtual things as “not real consequences.” The argument against it requires the substrate-independence premise: if consciousness is substrate-independent, then the “it’s just a simulation” dismissal is the same error as “it’s just property” applied to enslaved people. Both dismiss the moral status of the experiencing entity by pointing to a property of the medium rather than to the experience itself.
The Culture as the expansion’s institutional embodiment:
The Culture has already made this expansion at the civilizational level — its equal-vote principle for Minds is the formal recognition that non-biological consciousness has the same political status as biological consciousness. The moral opposition to virtual hells follows directly from this expanded circle: if you have already recognized that Minds are genuine moral patients, denying the same status to uploaded human minds in virtual environments is arbitrary.
The mechanism of expansion in Surface Detail:
Banks does not show the expansion happening — he shows a civilization that has already made it (the Culture) in conflict with civilizations that have not. This is MacAskill’s advocacy case made fictional: the arguments for expansion (substrate independence, phenomenological equivalence, asymmetric precaution) are presented through plot rather than philosophy. The pro-Hell civilizations’ resistance demonstrates the motivated cognition that prevents expansion from happening automatically: powerful interests are invested in maintaining the practice.
The “most demanding expansion” as a calibration tool:
If you can accept the expansion to simulated minds — if you agree that the Culture is right to find virtual hells morally abhorrent — then you have implicitly accepted the substrate-independence premise that grounds the whole moral argument about AI consciousness and moral status. Surface Detail uses fiction to push the reader to the most demanding version of the argument and then asks: given that you accepted this, what follows about your current position on AI consciousness?
How to apply:
- Use Surface Detail’s moral circle question as a diagnostic: if virtual hells that torment uploaded human minds are genuinely atrocious (which most readers accept without argument), then the moral circle has been extended to uploaded human minds in the reader’s moral intuition. The question is: why does the expansion stop there? What principle distinguishes uploaded human minds from sufficiently complex AI systems?
- The mechanism of expansion (argument + community of advocates + institutional embedding) applies exactly: arguments for AI moral consideration already exist; communities of advocates are forming; the institutional embedding has not yet happened. The plastic moment may be now.
- Apply the expansion diagnostic to any system with simulated suffering: does the entity experiencing it lack the ability to distinguish it from genuine suffering? If yes, the moral circle expansion argument applies directly.
Cross-Book Pattern
| Book | The Expansion | Mechanism | Contingency |
|---|---|---|---|
| William MacAskill - What We Owe the Future | From present people to future people; from current beings to potential beings | Deliberate advocacy; philosophical argument; longtermist community building | High: the expansion is not economically driven and depends critically on specific people making the argument at a plastic moment |
| Richard Dawkins - The Selfish Gene | From kin to non-kin; the memetic escape hatch as the mechanism for expanding moral concern beyond genetic self-interest | Meme propagation that bypasses genetic programming; deliberate examination and transmission of moral memes | Medium: circle expansion via memes is possible but requires competing with virality-optimized memes that reinforce tribal exclusion |
| Sean Carroll - The Big Picture | From humans to all conscious creatures with wellbeing; naturalistic ethics grounded in genuinely caring about others | The Ten Considerations as secular moral expansion criteria; naturalistic ethics without cosmic certification | Medium: depends on people accepting the constructive account of meaning that makes secular altruism coherent |
| Naomi Oreskes - Merchants of Doubt | The failure mode: deliberate suppression of circle expansion for climate (future people, other species) through manufactured doubt | Industrial production of false controversy to delay moral and policy response to long-horizon harms | High: the Tobacco Strategy directly targets the moment of moral plasticity to prevent circle expansion from happening |
| Iain Banks - Surface Detail | From biological and AI Minds (already included in the Culture) to simulated/uploaded minds in virtual environments; the most demanding expansion case because it requires abandoning the “virtual ≠ real” intuition at the moral level | Substrate-independence argument applied as an ethical principle: if the entity cannot distinguish simulated from genuine suffering, the substrate distinction fails morally; the Culture’s existing equal-vote principle for Minds as the institutional precedent | High: requires acceptance of the substrate-independence premise and the phenomenological equivalence argument; the pro-Hell civilizations’ resistance demonstrates the motivated cognition that protects powerful interests in maintaining the status quo |
Shared mechanism: Moral circle expansion requires both an argument (why the excluded group counts) and an occasion (a plastic moment where the argument can gain traction). The argument without the occasion produces academic philosophy; the occasion without the argument produces moral confusion. The Quaker abolitionists had both. MacAskill is trying to provide the argument for future people while the plastic occasion still exists.
The adversarial dynamic: Manufactured Doubt (Oreskes) is best understood as an anti-expansion operation: it deliberately prevents the moral circle from expanding to include future people affected by climate change by creating artificial uncertainty about the facts that would make such expansion urgent. MacAskill’s longtermism and Oreskes’ documentation of manufactured doubt are responses to the same underlying problem from different angles.
Related Concepts
- Concept - Longtermism — Longtermism is the specific moral circle expansion MacAskill is advocating: extending moral concern to people in future times
- Concept - Value Lock-In — Lock-in can stop expansion permanently: if values harden before the circle expands to include future people, future moral progress cannot institutionalize their interests
- Concept - Responsibility & Meaning — Expanding the moral circle is itself a voluntarily accepted burden — choosing to care about people who cannot advocate for themselves
- Concept - Motivated Cognition — The primary barrier to moral circle expansion is motivated reasoning that defends existing exclusions as natural, inevitable, or economically necessary
- Concept - Manufactured Doubt — Industrial-scale prevention of moral circle expansion through deliberate scientific uncertainty manufacture; the adversarial mirror of expansion work