The failure mode is not “wrong answers”
Wrong answers are easy.
The real failure mode is output that:
- sounds right,
- fits the story we want to hear,
- is wrong in one subtle assumption,
- and gets copied into documentation or used in a decision.
That’s what makes AI different from most tools. Not that it is “smart”.
It is generative.
And it is confident.
So reliability has to come from the people and the process around it.
A real session as a reference point
We recently ran an AI-assisted product concept development session and documented the workflow.
It was not a science project. It was normal product work.
The numbers make the point:
- 100 turns
- ~4 hours active time
- €1.76 total cost
The output was also not “a nice conversation.” It was durable work:
- Full product concept developed (internal feature)
- 6 documentation pages updated
- 4 new skill files created, 8 existing skills improved
- Cross-repo knowledge propagated to 4 repositories
This is where the “people in the lead” part shows up: those outputs didn’t happen because we trusted AI.
They happened because we kept ownership with people and used AI inside a structured loop.
Leadership is a workflow
“People in the loop” is often used as a vague comfort phrase.
What actually works is specific:
- People define what is allowed (constraints and guardrails).
- People define what is true (ground truth sources).
- People decide what matters (priorities).
- People verify outputs before they become artifacts.
- People capture corrections so the next run starts smarter.
That’s not a new skill. Good teams always did this—through reviews, retros, playbooks, tests, and automation.
AI just makes the failure modes we already know much cheaper and much faster.
One extra pattern helps here: you can also use AI to review AI output — but only as a first-pass review to catch obvious gaps or missing counter-arguments. The final review and decision still stays with people.
Pattern 1: questions first, answers one by one
The fastest way to get misleading output is to let the system run with missing context.
So early in the session, the explicit rule was: Don’t assume anything. Ask everything.
That generated 10–20 questions.
This is where many people get impatient.
But if the system stops asking questions too early, it is not being efficient. It is running on assumptions.
So we answered open questions one by one.
That forces precision, and it keeps ownership where it belongs: with the people who understand the domain.
Pattern 2: stay strong against premature convergence (and keep options open)
A black box will always try to converge quickly.
It wants to produce a neat plan, a clean conclusion, and one best option.
But real product work often needs the opposite: keep options open until there is enough detail to choose correctly.
That’s why it matters to stay strong against early AI suggestions and to keep decisions open until the underlying assumptions are explicit.
And once you do want to compare options, AI can do the heavy lifting in parallel: generating two alternatives side-by-side is not “waste” — it’s often the fastest way to force a real comparison and improve decision quality.
Pattern 3: treat documentation as a reliability mechanism
Documentation is not a “nice to have” in AI-assisted workflows.
It is how you prevent the next run (or the next person) from reinventing reality.
In the session, the documentation work didn’t happen at the end as an afterthought.
It was a cascade:
- Review existing docs for staleness
- Update what’s wrong
- Add missing context info
- Make issues explicit
This is also why API-driven documentation systems matter: it must be easy to turn outputs into durable artifacts.
The only thing that should compound is the organization’s knowledge
Some people say “train the AI.”
That framing blurs the line between people and machines.
A safer framing is:
- People correct.
- Teams standardize.
- The organization accumulates reusable context.
- AI runs inside those standards.
In our session, every correction became a permanent update:
- terminology fixes
- formatting rules
- document hygiene
- voicing rules (e.g. express uncertain items as questions)
That is what makes the next session better.
Not because the machine became a person.
Because the process became sharper.
Closing: clear boundaries make AI usable
When people humanize AI, they hand it something it cannot earn: trust.
When people treat AI as a tool, they keep the correct posture:
- verify,
- review,
- log,
- correct,
- and keep ownership with people – the experts.
This is the practical path to scaling AI: not by chasing models, but by keeping people in charge of the workflow that surrounds the tool.
by Dominik Go
