Sophecs
Menu
1 of 2Reading 1 of 2

The opaque benefit

Right in proportion as they tend

Before you argue · 1 min

John Stuart Mill's statement of the principle is one sentence, and the important word in it is a small one. Actions are right "in proportion as they tend to promote happiness, wrong as they tend to produce the reverse of happiness" (Utilitarianism, chapter 2, 1861). Tend. Mill is not asking whether you know why an action works. He is asking what it does, on the whole and over time.

That framing was built for doctors, lawmakers and ordinary people making decisions they could at least describe. Now imagine a system that cuts hospital readmissions by a fifth, and whose designers cannot say which features of a patient's record it is weighing. The tendency is measurable. The reasoning is not.

Two readings are open. On one, Mill has already answered: the tendency is the whole of the moral question, and demanding an explanation on top is a separate preference, perhaps a reasonable one, but not a moral requirement. On the other, "tend" was doing quiet work. A tendency is something you can assert only because you understand enough of the mechanism to expect it to hold tomorrow, in a new hospital, with different patients. Without that you have a track record, and a track record is a different thing.

Which reading is Mill's is a fair question. Which one you would defend is the one you are about to answer.

Sources

  1. 1John Stuart Mill, Utilitarianism, chapter 2

The objection to this, from another school

Cleverness is not wisdom

Aristotle, Nicomachean Ethics, VI.12, 1144a23–29

An AI system that reliably reduces suffering should be deployed even where the people it decides about cannot be told why.