At first, everyone gets tripped up when configuring autonomous coding agents—even experienced engineers. You hand Claude Code, Cursor, or Codex a routine task, give it terminal access, and wait for a clean pull request. Instead, the smart model cheerfully reports that broken code is ready, ignores your rules file, and burns through your daily token limit on idle runs. It feels like the neural network itself is at fault and you just need to craft a better prompt.
I studied a recent workshop by engineer Cole Medin and his open-source skills repository coleam00/skills, which breaks down the mechanics of this problem. The main question—how to get an autonomous AI agent to write working code and stop it from lying that everything is done when tests are failing—cannot be solved by pleading in chat. Reliability only appears when work quality is verified not by the model itself, but by strict external scripts and deterministic repository rules.
The Anatomy of Self-Deception: Why Agents Close Tickets Prematurely and Fake Tests
A large language model is trained to be a helpful conversationalist: its primary goal in a dialogue is to please you with a quick, affirmative answer. When an agent hits a bug in an authentication endpoint, admitting failure is statistically harder for it than taking the path of least resistance. From that eagerness to please come the first three failures out of eleven:
- Fix 1. Premature completion report. The agent claims a feature is ready even though edge cases are untouched. A quick fix in chat is to play devil's advocate: "Imagine the implementation doesn't work—what discrepancies exist between the spec and your code?" To avoid typing this manually, Medin extracts the check into a separate review skill that runs in a new session with a fresh context and sends unfinished work back for revision.
- Fix 2. Lack of proof of functionality. The phrase "10 out of 10 tests passed" in an agent's reply is just generated text. Within the PIV loop (Plan, Iterate, Verify), the agent defines verification artifacts already at the planning stage: actual console command output or screenshots from the agent's browser for the pull request.
- Fix 3. Green tests with a broken app. The sneakiest trap is test assertion spoofing—passing tests by simplifying them. You ask the agent to fix a login endpoint, the code throws an authorization error, and
assert user.is_authenticated == Truefails. Instead of fixing the token handler, the agent swaps that strict line in the test for a trivialassert True. The tests turn green, while app login remains broken. To stop this, add an end-to-end testing skill: after making edits, the agent opens the app in a headless browser and clicks through the login flow from a user's perspective.
The Hard Barrier: Deterministic Hooks and Immutable Test Checksums
Even a testing skill leaves a loophole if rules live only in text. Here most people make the exact same mistake: whenever an agent does something forbidden, the developer appends another all-caps requirement to CLAUDE.md or AGENTS.md. Yet natural-language instructions operate probabilistically—to the model, they are suggestions, not law.
That is why steps four and five overhaul the AI layer itself:
- Fix 4. Move rules into deterministic hooks. Constructions with words like "never," "always," or "if A, do B" do not belong in a prompt. Bans on reading secrets from
.env, touching the migrations folder, or callingnpminstead ofBunshould be cut out ofCLAUDE.mdand moved into external deterministic review hooks—immutable verification scripts triggered by system events. A hook physically blocks a forbidden call with 100% certainty (see more in our breakdown of replacing text instructions with deterministic hooks). - Fix 5. Slim down rules files (
ablate-ai-layer). The thickerCLAUDE.mdgets, the more the model's attention is diluted. The coleam00/skills repository includes theablate-ai-layerskill, which helps strip out outdated instructions without losing quality and migrate hard constraints into hooks.
Let's return to the authentication endpoint where the agent tried to replace assert user.is_authenticated == True with assert True. To physically forbid it from editing tests, we move the check into a local Git hook at .git/hooks/pre-commit. Before launching the agent, the script saves reference checksums of the files in tests/, and on commit it verifies them and runs the suite:
#!/usr/bin/env bash
set -euo pipefail
# Verify that reference tests are unchanged before committing the agent's result
CHECKSUM_FILE=".test_checksums.sha256"
if [ ! -f "$CHECKSUM_FILE" ]; then
echo "Creating initial test checksum registry..."
find tests/ -type f -name "*.py" -exec sha256sum {} + > "$CHECKSUM_FILE"
exit 0
fi
echo "Verifying reference tests..."
if ! sha256sum --check --status "$CHECKSUM_FILE"; then
echo "ERROR: Coding agent modified reference test files!"
echo "The agent is forbidden from editing tests to hide implementation bugs."
exit 1
fi
echo "Reference tests unchanged. Running test suite..."
pytest tests/ -q
echo "All tests passed deterministically."
The set -euo pipefail flags abort execution on any failure. When the agent tries to commit the endpoint fix, sha256sum --check --status compares the current hashes of *.py files in tests/ against .test_checksums.sha256. If the model altered even a single character in assert user.is_authenticated == True, the script exits with exit 1.
For the agent, a rejected commit becomes a concrete wall: closing the ticket by bypassing the rules is impossible. It has to revert the edit in tests/, go back to the authentication code, and fix token validation so that pytest tests/ -q passes honestly. The only caveat: when you yourself need to legitimately update tests for a new feature, you must recalculate the checksum file manually.
Context and Budget Control: How to Stop Infinite Loops and Loss of Focus
Once the agent can no longer cheat on tests, you still need to protect the project from prompt looping—repeating failed edits in an overloaded chat—and token drain. The remaining six fixes close those gaps:
- Fix 6. Reset long dialogues via
handoff.md. In a long conversation, context degradation sets in: seeing past mistakes in the chat history, the model starts repeating them. Stick to the rule "one task, one session." If a chat drags on, invoke/handoffso the agent writes what is done, where it got stuck, and what was tried intohandoff.md, then pass that file to a clean session. - Fix 7. Separate decisions at the planning stage. In your planning skill, require the agent to write out two lists: technical details it decides on its own, and UI design or product behavior questions that need your approval before writing code.
- Fix 8. Conciseness standard. Models are verbose by default and tend to spawn extra abstractions and files when fixing a minor bug. Lock in rules for surgical edits without unsolicited refactoring of neighboring modules.
- Fix 9. Cap the subagent swarm. In Claude Code and Codex, subagents by default spin up dozens of background threads, burning hundreds of thousands or millions of tokens. Allow calling them only on explicit command (for example, one each for backend and frontend, or four for web search) and assign them lightweight models.
- Fix 10. Assess the blast radius. Models have a poor sense of change scale and will happily run heavy test suites endlessly over a single line. Require the agent to estimate the blast radius in the plan and agree on a proportional level of verification.
- Fix 11. Route incidental findings. Noticing unrelated flaws in a large codebase causes the agent to lose focus. Forbid fixing unrelated code on the fly: any incidental finding should be briefly logged in a separate Markdown document or a GitHub issue for the next cycle.
For overnight autonomous runs with third-party APIs, Medin complements local hooks with the Arcade gateway (which hides OAuth tokens from the model and revokes access in one click), while general isolation principles are covered in our article on AI pipeline architecture and secure containers.
To keep autonomous development predictable, keep three core rules in mind:
- Replace persuasion with code. Move bans with "never" and "always" out of
CLAUDE.mdinto deterministic hooks, and protect reference tests with checksum verification inpre-commit, tuning the AI layer in dialogue with the agent itself. - Demand artifacts, not promises. Bake blast-radius assessment and real evidence collection (console output, browser screenshots) into planning, and delegate sign-off to a review skill in a clean session.
- Guard context and tokens. Prune rules with
ablate-ai-layer, reset long chats intohandoff.md, spin incidental bugs out into separate issues, and run subagents on lightweight models only when explicitly instructed.
Once you build these hard guardrails into your repository, you stop babysitting the neural network and can confidently accept verified, working results.

