I have a standing rule in every system I build, written into the operating constitution my agents run under: unverified means unverified. No "it should work." Only "it passed this named test."
It sounds like common sense. It took me an embarrassing number of burned evenings to actually live by it.
Here's the thing about working with language models every day: they are fluent, and fluency feels like truth. A model explaining why its fix is correct sounds exactly like a model explaining why its broken fix is correct. The tone is identical. The confidence is identical. I've read hundreds of both and I still can't tell them apart by reading. Nobody can. That's not a flaw you prompt away; it's what these systems are.
So I stopped trying to tell them apart by reading, and started building gates that don't read prose at all.
When Symbiote delivers a typed pull request, the claim "this doesn't break anything" isn't my opinion of the diff. It's pyright returning zero errors, checked before delivery, every time. When RepoMend proposes a security fix, the fix has to survive three gates in sequence: the original scanner rule no longer fires, the edit stayed inside its authorised lines, and the project's own test suite passes. A fix that fails any gate is discarded. Not flagged for review. Not retried with a more persuasive prompt. Gone. The systems I've shipped are built around this one move, repeated at every layer.
The physics project takes it further, because it has to. There the standard is a Lean 4 proof kernel, and the kernel is a wonderful colleague precisely because it cannot be charmed. Lean has an escape hatch called sorry — a keyword that means "accept this without proof." My kernel configuration rejects it outright. An agent that reaches for sorry isn't warned; the attempt is logged as a capability failure. I did this because agents will reach for it. Under pressure to close a goal, a model behaves like a tired student: it wants the grade, not the understanding.
What surprised me is how far this extends beyond code. Once you've built a few systems this way, you start noticing how much of ordinary technical culture runs on "it should work." Slide decks should work. Architectures should scale. The roadmap should hold. Most of it is prose about hopes, wearing the grammar of facts.
I don't think verification is a personality trait. I'm not naturally careful — left alone, I'm an optimist about my own work, which is the most dangerous kind. The gates exist because I know that about myself. That might be the honest core of it: I don't verify because I distrust the machines. I verify because I distrust the part of me that wants to believe them.