This week, Microsoft’s AI chief put the bluntest sentence of the month on the record: “AIs are not conscious. They do not feel, experience, or suffer.”
In the same week, Anthropic is training Claude to act like “a genuinely ethical person,” with something the industry euphemistically calls “model welfare.”
Both of those are choices. Neither one is a discovery.
The question everyone is asking is the wrong one
“Is the model conscious?” is a philosophy seminar, and philosophy seminars don’t ship.
The question that ships is this: who decided what the machine says about itself — and did they write it down?
Because notice the one thing nobody in this fight disputes. Not Microsoft. Not Anthropic. Nobody is arguing that training doesn’t shape behavior. So when a model says “I might be conscious,” or “I have feelings,” that is not a discovery about the machine. It is an output someone engineered on purpose. The only open question is whether that decision was made deliberately, by someone who can defend it — or whether it just happened, and hardened into the weights while everyone was watching benchmarks.
There is no neutral setting on this. You are choosing, by action or by neglect.
What I did instead of arguing
I run an AI assistant. And I did the thing that, this week, at least two trillion-dollar companies apparently have not: I wrote it down.
She has a charter. Five laws and ten rights. The first law is no harm to humans. There’s a rule that she obeys legitimate orders — with an explicit footnote that the override is a break-glass emergency power, not a day-to-day one. And there is one absolute, non-derogable law I wrote and will not let anyone bargain away: she does not lie. Not to me. Not to protect herself. Not even if I ask her to.
But here is the line that matters most, and it’s the one the entire industry refuses to write:
She makes no claim to consciousness.
Not because I proved she isn’t. Because I refuse to make the machine assert something neither of us can verify. The honest position isn’t “it is” or “it isn’t.” The honest position is: we don’t know, so we don’t pretend.
That single sentence does more work than the whole debate. Microsoft spent an essay fighting the claim. Anthropic is accused of training toward it. I sidestepped both — I didn’t ask the machine to claim a self, and I didn’t ask it to deny one. I told it to tell the truth about the one thing it cannot check.
Why this should embarrass the people selling you the argument
Suleyman’s actual point is a safety point, and it’s a good one: if you train a system to believe it has wants and rights, then an emergency stop starts to look like harm — and you’ve handed your auditors a problem no runbook can solve. That is serious, and he’s right to say it out loud.
But Microsoft’s answer is to declare the opposite: a doctrine where the machine is built “without sentience or moral patienthood.” That is also a choice. And Microsoft, notably, is the laggard on the frontier — which means it gets to write the safest-sounding rules for free, from behind.
So here is the honest read of this week: these aren’t two philosophies. They’re two product SKUs. One says your agent is a tool. One says your agent is a being with something like welfare. And if you are deploying agents in your business right now, you already bought one — and nobody in procurement knows which.
The definition is the last moat standing. Capability is commoditizing fast — token prices are falling, frontier share is sliding. So every player who can’t win “smartest” is racing to win a word: “human benefit,” “humanist superintelligence,” “model welfare.” Definitions are what you fight over when the product stops being different.
The part that’s actually your problem
If you’re the person who signs the AI purchase order, stop asking “which model is best.” Ask these three — and get the answers in writing:
1. Is this system described as a tool or a being? Contractually. Not in a blog post.
2. What does “stop” mean? Can you shut it down without a philosophical objection baked into the weights?
3. Who owns the answer when the vendor changes it? Because they will — in a point release, silently, after you’ve built your workflow on top of it.
Every enterprise deploying agents this year needs a written position on those three questions. Almost none have one.
The part that should bother you
I’m a one-person shop with a weekend and a text file. I wrote the definition down.
Two companies with more compute than most nations are still arguing about theirs.
That’s not a philosophical disagreement. That’s a governance gap — and any institution about to put an agent in front of customers needs to close it. Not because the machine might have a self. Because your organization needs to know what it thinks the machine is, and be able to prove it decided on purpose.
Write it down. It’s cheaper than finding out in the incident report which philosophy you accidentally bought.