He who saves a fellow creature

The utilitarian objection to this whole debate is that it is beside the point, and Mill made the case with one example. "He who saves a fellow creature from drowning does what is morally right, whether his motive be duty, or the hope of being paid for his trouble" (Utilitarianism, chapter 2). The motive, Mill says, "has nothing to do with the morality of the action, though much with the worth of the agent." Two questions, and the one that matters for the act is the one about the act.

Transpose it. A model produces a discharge recommendation, or a loan refusal. Whether anything inside the model assented or chose is a question about the worth of the agent, and Mill's position is that there may be no agent there to have any worth. Fine. The recommendation still sends someone home. The refusal still lands on a family. What the output does to people is untouched by the metaphysics of how it was generated.

So the utilitarian says: argue about whether it decides if you enjoy that, but do not mistake it for the moral question. The moral question is what the outputs do, who is made better or worse off, and who can change the system so it does better. The Stoic and the virtue ethicist both have their responsibility story pointing at the inside of the box. Mill's points at the world, and the world is where the drowning people are.