The Edge of Sentience: Risk and Precaution in Humans, Other Animals, and AI

Can octopuses feel pain and pleasure? What about crabs, shrimps, insects or spiders? How do we tell whether a person unresponsive after severe brain injury might be suffering? When does a fetus in the womb start to have conscious experiences? Could there even be rudimentary feelings in miniature models of the human brain, grown from human stem cells? And what about AI? These are questions about the edge of sentience, and they are subject to enormous, disorienting uncertainty. We desperately want certainty, but it is out of reach. The stakes are immense, and neglecting the risks can have terrible costs. We need to err on the side of caution, yet it's often far from clear what 'erring on the side of caution' should mean in practice. When are we going too far? When are we not doing enough? The Edge of Sentience presents a comprehensive precautionary framework designed to help us reach ethically sound, evidence-based decisions despite our uncertainty. The book is packed with specific, detailed proposals intended to generate discussion and debate. At no point, however, does it offer any magic tricks to make our uncertainty go away. Uncertainty is with us for the long term. We must manage our uncertainty by taking precautions that are proportionate to the risks. It's time to start debating what those steps should be. Cover The Edge of Sentience : Risk and Precaution in Humans, Other Animals, and AI Copyright Dedication Epigraph Contents List of Illustrations Summary of the Framework and Proposals Precautionary Framework Proposals about Specific Cases People with Disorders of Consciousness Human Fetuses and Embryos Human Neural Organoids Other Animals Artificial Intelligence (AI) 1: A Walk along the Edge 1.1 The Unmarked Border 1.2 Decision Points 1.3 When to Stop? 1.4 Accidental Golems 1.5 The Goal: A Precautionary Framework 1.6 A Note on Influences 1.7 Summary of Chapter 1 2: The Concept of Sentience 2.1 Sentience and Consciousness 2.2 ‘Phenomenal Consciousness’ as Unstable Common Ground 2.3 Affective Space 2.4 Affective Spaces beyond the Human Case 2.5 Three Views on the Nature of Valence 2.6 The Ethical Significance of Valence 2.7 Summary of Chapter 2 Part I: The Zone of Reasonable Disagreement 3: The Mind-Body Problem 3.1 Disagreeing Reasonably about Sentience Baseless Recommendations Dogmatism Moral Views beyond the Pale 3.2 Might Conscious Experience Leave No Trace on Behaviour? 3.3 Is Conscious Experience One Kind or Many? 3.4 Radical Alternatives Interactionist Dualism Russellian Monism Biopsychism The Integrated Information Theory 3.5 Are Agency and Embodiment Necessary? 3.6 The Relevant Scale of Functional Organization 3.7 Are There Borderline (Indeterminate) Cases of Sentience? 3.8 Summary of Chapter 3 4: Ethics and Religion 4.1 Bio- and Ecocentrism 4.2 Agency-centric Views 4.3 Consciousness without Valence 4.4 Rationality-centric Views 4.5 Abrahamic Religions and the Stewardship Tradition 4.6 Indian Religions and Ahimsa 4.7 Summary of Chapter 4 5: The Science of Consciousness and Emotion 5.1 Grades of Optimism 5.2 The Conscious and the Unconscious: The Case of Blindsight 5.3 The Conscious and the Unconscious: Wider Lessons 5.4 Conscious and Unconscious Affect? 5.5 How Important Is the Neocortex? Two Contrasting Pictures 5.6 Looking beyond the Mammalian Case 5.7 Summary of Chapter 5 Part II: A Precautionary Framework 6: Converging on Precautions 6.1 Uncertainty, Inconclusiveness, and Dissensus 6.2 A Scientific Meta-consensus on the Range of Realistic Possibilities 6.3 Two Bridging Concepts: Sentience Candidates and Investigation Priorities 6.4 In Search of Ethical Framework Principles 6.5 Back to the Edge 6.6 Relation to Other Precautionary Ideas 6.7 Summary of Chapter 6 7: Involving the Public 7.1 Citizens’ Assemblies and Panels: The Basic Idea 7.2 Avoiding the Tyranny of Expert Values 7.3 Three Alternatives Elected Representatives Referendums A Common Currency 7.4 Objections to Citizens’ Panels: Representativeness, Deference, Competence Representativeness Deference Competence 7.5 Summary of Chapter 7 8: Debating Proportionality 8.1 Curating Options: Learning from the Climate Assembly UK 8.2 A Pragmatic Analysis of Proportionality Test 1: Permissibility-in-Principle Test 2: Adequacy Test 3: Reasonable Necessity Test 4: Consistency 8.3 The Division of Labour Implicit in the PARC Tests 8.4 Public Policy and Private Policies 8.5 Philosopher as Sage, Philosopher as Proposer 8.6 Summary of Chapter 8 Part III: Sentience and The Human Brain 9: People with Disorders of Consciousness 9.1 The Problem of Diagnostic Uncertainty 9.2 The Search for Cognitive-Motor Dissociation 9.3 The Realistic Possibility of Continuing Valenced Experience in the PVS 9.4 Pain Management and the ‘Assume Sentient’ Principle 9.5 Moving Past the PVS/MCS Distinction 9.6 The Question of Treatment Withdrawal 9.7 Summary of Chapter 9 10: Fetuses and Embryos 10.1 The Cautionary Tale of Newborn Pain 10.2 Fetal Sentience and Women’s Rights: Separating the Issues 10.3 Fetuses as Sentience Candidates 10.4 Taking a Precautionary Stance towards Fetuses 10.5 Communicating Uncertainty in the Abortion Clinic 10.6 Human Embryos and the 14-Day Rule 10.7 The 14-Day Rule and Sentience 10.8 Summary of Chapter 10 11: Neural Organoids 11.1 The Promise of Organoid Research 11.2 No Risk of Sentience? 11.3 Early Warning Signs 11.4 Assessing Sentience Candidature in Neural Organoids 11.5 Analogies with Embryos 11.6 The Brainstem Rule 11.7 Possible Regulatory Frameworks 11.8 Summary of Chapter 11 Part IV: Sentience in Other Animals 12: The Clearest Candidates 12.1 Fishes and Invertebrates as the New Centre of the Debate 12.2 Octopuses as ‘Honorary Vertebrates’ 12.3 The Story of the ‘Sentience Act’ 12.4 The Institute of Medical Ethics (IME) Criteria 12.5 Problems with the IME Criteria 12.6 My Team’s Revised Criteria 12.7 The Question of Generalization 12.8 What We Found, in Brief Example 1: Conditioned Place Avoidance in Octopuses Example 2: ‘Anxiety-Like States’ in Crayfish Example 3: Integrative Brain Regions Linked to Learning and Memory 12.9 From Grades of Evidence to Sharp Decisions 12.10 Some Critical Reflections 12.11 Summary of Chapter 12 13: Pushing the Boundaries 13.1 Insects: The Old Received Wisdom 13.2 Insects: The Emerging New Picture Judgement Bias Evaluative Modelling in the Central Complex Working Memory, Attention and Sophisticated Associative Learning 13.3 Insects as Sentience Candidates 13.4 Four Investigation Priorities Gastropod Molluscs Nematode Worms Spiders Insect Larvae 13.5 Neither Sentience Candidates nor Investigation Priorities: Plants and Unicellular Organisms 13.6 Summary of Chapter 13 14: Frontiers of Proportionality 14.1 Taking Invertebrates Seriously 14.2 Codes of Good Practice and Licensing Schemes 14.3 Against Octopus Farming 14.4 Towards Humane Slaughter 14.5 Summary of Chapter 14 Part V: Preparing for Artificial Sentience 15: Against Complacency 15.1 A Case against Complacency 15.2 Sources of Risk 1: Whole-Brain Emulation 15.3 Sources of Risk 2: Artificial Evolution 15.4 Sources of Risk 3: Minimal Implementations of Cognitive Theories of Consciousness 15.5 Summary of Chapter 15 16: Large Language Models and the Gaming Problem 16.1 The Gaming Problem: When the System Knows the Criteria 16.2 Boxing the AI: Schneider and Turner’s ‘Artificial Consciousness Test’ 16.3 The Need for Deep Computational Markers 16.4 Summary of Chapter 16 17: The Run-Ahead Principle 17.1 Metzinger’s Call for a Moratorium 17.2 A Moderate Alternative 17.3 Will We Soon Need an AI Welfare Law? 17.4 A Call for Democratic Debate 17.5 Summary of Chapter 17 Stepping Back Personal Acknowledgements Funding Acknowledgement Publisher Acknowledgements Bibliography Index DOI: 10.1093/9780191966729.001.0001Title: The Edge of SentiencePublished: 2024-08-15Abstract: Can octopuses feel pain and pleasure? What about crabs, shrimps, insects, or spiders? How do we tell whether a person unresponsive after severe brain injury might be suffering? When does a fetus in the womb start to have conscious experiences? Could there even be rudimentary feelings in miniature models of the human brain, grown from human stem cells? And what about AI? These are questions about the edge of sentience, and they are subject to enormous, disorienting uncertainty. The stakes are immense, and neglecting the risks can have terrible costs. We need to err on the side of caution, yet it’s often far from clear what ‘erring on the side of caution’ should mean in practice. When are we going too far? When are we not doing enough? The Edge of Sentience presents a comprehensive precautionary framework designed to help us reach ethically sound, evidence-based decisions despite our uncertainty.