Our entire legal framework for AI governance is based on a category error that mistakes looking coherent for having a goal.
AI systems are actually just constraint-bound generators that lack true agency or intent. We treat these models as agents because they produce output that looks like it came from a being with a purpose. This paper argues that this misclassification is leading to flawed laws and ethical guidelines. These systems do not have goals, they just have boundaries that they operate within. Continuing to govern them as agents will lead to a fundamental failure in how we manage the risks they pose.
The Agency Error in AI Governance: Coherent Output, Constraint, and the Misclassification of Artificial Systems
SSRN · 6641339
<p>AI systems do not act. They generate coherent outputs under constraint. The persistence of agency attribution in artificial intelligence governance arises not from the presence of intention in these systems, but from a structural feature of their design: sufficiently constrained systems operating over structured data reliably produce the observable markers—coherence, responsiveness, apparent goal-directedness—that humans use to infer agency in other contexts. The inference is not irrational.