Interesting proposal with the Goodhart's law admission. PRF has the same failure every compliance framework eventually hits: once a benchmark becomes the procurement gate, vendors train to the benchmark instead of the judgment it measures, the same reason control testing keeps finding environments built to pass the audit rather than be secure. The sealed hypotheticals are the right instinct; the real test is whether the commission can out-iterate labs optimizing against a known target.
An interesting proposal, but I wonder whether it addresses the wrong philosophical question? PRF asks whether a model reasons as the public reasons, yet democratic legitimacy has never rested on simple fidelity to majority opinion. Tocqueville warned that liberal democracy depends as much on limiting the tyranny of the majority as expressing it.
More fundamentally, MacIntyre argued that modern appeals to "values" often conceal subjective preferences rather than reasoned accounts of the human good. Choosing AI on the basis of alignment with public reasoning risks institutionalising prevailing opinion without ever asking the prior question: what is government AI actually for, and by what conception of justice or the common good should its reasoning ultimately be judged?
Interesting proposal with the Goodhart's law admission. PRF has the same failure every compliance framework eventually hits: once a benchmark becomes the procurement gate, vendors train to the benchmark instead of the judgment it measures, the same reason control testing keeps finding environments built to pass the audit rather than be secure. The sealed hypotheticals are the right instinct; the real test is whether the commission can out-iterate labs optimizing against a known target.
The move from acuracy to
reasoning feels like the real
contribution here!
Accuracy tells us whether a conclusion is right.
Reasoning helps us understand why it was reached and in a democracy, that understanding is part of accountability.
An interesting proposal, but I wonder whether it addresses the wrong philosophical question? PRF asks whether a model reasons as the public reasons, yet democratic legitimacy has never rested on simple fidelity to majority opinion. Tocqueville warned that liberal democracy depends as much on limiting the tyranny of the majority as expressing it.
More fundamentally, MacIntyre argued that modern appeals to "values" often conceal subjective preferences rather than reasoned accounts of the human good. Choosing AI on the basis of alignment with public reasoning risks institutionalising prevailing opinion without ever asking the prior question: what is government AI actually for, and by what conception of justice or the common good should its reasoning ultimately be judged?