Back to Blog
EngineeringProduct

The decisions your AI agent doesn't ask about

Lex Interview Blog Img Placeholder

Lex Vorona asked an AI agent to build a new API endpoint. Two hours later, it came back with working code and a full set of passing tests. Everything looked right, except for one decision nobody had asked it to make.

The endpoint returned every user in the system, including deleted ones. "Nobody ever talked about the deleted ones," Lex says. "We don't need them. Why are those in the output?" But by then, the agent had written tests that treated the deleted users as correct, and undoing that assumption took another hour.

Lex shared that story in a recent conversation with my colleague Daniel Topham about how AI has changed his work. He's a software engineer and Atono's technical co-founder who made many of our early technical decisions, and he's been experimenting with AI in one form or another since the 1990s.

He's usually the person on our team trying out new models and tools and sharing what he's learned. But he's particular about how he uses agents. Some he watches closely, ready to step in, while others he leaves to run on their own. Agents still make decisions he didn't ask them to make, and the longer they work out of sight, the more that gets built on top of those decisions before anyone notices. He thinks carefully about how much independence to give them in the first place.

Reasonable decisions, wrong for this product

Lex has seen this pattern many times. Agents tend to make the choice that would be reasonable in a typical codebase, even when it's wrong for the one they're working in.

In one of Lex's projects, an agent wrote a migration script to move existing data into a new format, even though the table had been created that morning and held no data. In another, an agent working on its own to set up OAuth for a remote MCP server wrote thousands of lines of code without using the auth infrastructure we already had, then rewrote it from scratch twice more after being corrected.

"That was a lot faster than if I had to write 3,000 lines of code by hand, but it's three times slower than it could have been."

Visibility is what lets you catch them early

Lex keeps four or five AI tools set up on purpose, each with a different level of AI involvement, and chooses between them depending on the work.

  • Google AI Studio gets general questions, with no context at all, so his codebase doesn't skew the answer.

  • VS Code handles hands-on coding, with autocomplete that only suggests a line or two when he presses a hotkey.

  • The Zed editor, running models through OpenRouter, is where he does agentic work he wants to watch.

  • Claude Code, running in an isolated virtual machine with full access, takes on work he's happy to leave unattended.

For the work he does watch, he needs to see what the agent is doing while it does it. Reasoning and tool calls scroll past quickly, but scanning them is often enough to spot a wrong turn, such as an agent reading authentication code during what should be a simple database change.

"Why are you looking at authorization and tokens when I just asked you to create a database patch?"

When Lex stops an agent, he goes back to the instructions and context he gave it. "What is it that I did not tell you? What was missing that made the model go on this?" he asks. "And then I fix it." His editor lets him rerun from an earlier point in the conversation and roll back the file changes made since then. A wrong turn caught a minute in takes seconds to correct, and Lex says he enjoys that part of working with agents. It gives him some of the same satisfaction he got from writing code himself, because he's still figuring out what's wrong and how to fix it.

Tools that hide their work behind summaries and background subagents take that opportunity away. Lex also says his preferred setup isn't for everyone. The editor he relies on most isn't one of the most popular options, and it took real effort to set up. He stuck with it for the visibility and control it gives him, while recognizing that more widely used tools do some things better.

Autonomy is an engineering decision

Lex keeps several tools partly because each one behaves differently. For example, one of his agents reruns every unit test by default after even a small change, which takes about 40 minutes, while another focuses on just the tests related to the code it changed. Differences like these help him decide which tool to use for each task.

That's why he's wary of top-down AI adoption, where leadership chooses the models and tooling everyone will use.

"The top-down approach limits us to the same tool for every problem."

Teams have good reasons to standardize their tools, from security and data handling to cost and support. But standardizing tools doesn't have to mean standardizing how engineers work with AI, as long as those tools let engineers choose how closely to watch the work.

Choosing how much autonomy to give an agent is still an engineering decision. Lex is happy to let agents work unattended on certain tasks. But in a long autonomous run, he may not see a wrong turn for hours, while in an interactive session he can catch it within minutes.

Give it what it needs to succeed

Lex's focus on giving AI the information it needs goes back to a neural network he built at university to play a puzzle game. It played poorly until he gave it higher-level descriptions of the board, such as how much of it was empty, after which it discovered strategies he had never coded and learned to ignore details that didn't matter.

"My responsibility is to give it all the things that it needs to succeed."

He still works with agents this way today. Watching them lets him catch a wrong turn early, and then work out what they were missing. Agents will keep making decisions on their own, and seeing those decisions early gives engineers the chance to correct them before more work is built on top.

Watch highlights from Daniel's conversation with Lex below.

At Atono, we're moving our own engineering work to agent-driven processes while building toward helping other teams see what their agents are actually doing and deciding. You can read more about where we're headed in our handbook.

Stay up to date

Share

Stay up to date

Share