The careful builder
All punishment is mischief
After you argue · 1 min
Bentham's objection starts by refusing the idea that blame is something people deserve. "All punishment is mischief: all punishment in itself is evil," he writes in chapter 13 of An Introduction to the Principles of Morals and Legislation. It is justified only where it "promises to exclude some greater evil." He then lists the cases where punishment is unwarranted, and one of them is where it is inefficacious: where it could not have changed the conduct that led to the harm, because the person could not have known or could not have done otherwise.
Blame is punishment's quieter cousin, and the same test applies. Blaming the careful builder in proportion to the harm would not undo it, and would not have prevented it, since by hypothesis the care was taken. What it would do is teach every other builder that the size of the disaster, not the quality of the work, decides their fate, and the rational response to that lesson is to disclose less and test less anywhere failures are visible.
So for the utilitarian, proportional blame is bad policy before it is unfair: an evil imposed without the greater evil it was supposed to prevent. Blame belongs exactly where it changes future conduct, on the shortcut, the skipped audit, the warning seen and ignored. A large harm with none of those behind it is a tragedy and a reason to redesign, and the builder should be asked what they learned, not made to answer for a number.
Sources
- 1Jeremy Bentham, An Introduction to the Principles of Morals and Legislation, chapter 13, §§1–2
The passage that frames the question
Bernard Williams, Moral Luck, the essay 'Moral Luck' (1976), collected in Moral Luck, Cambridge University Press, 1981
When an AI system causes harm, the people who built it are to blame in proportion to the harm, however careful they were.