Coding agents: Simon Willison's new llmcodingagent

Simon Willison dropped llm-coding-agent 0.1a0 on 2 July 2026. It's an experiment in Fable 5 driven automation that skips the hand-holding. This release proves coding agents can now autonomously execute file edits and command-line operations using red/green TDD workflows. While broader market benchmarks show Codex + GPT-5.5 leading with an 83.4% score, this tool ignores abstract problem-solving. It focuses entirely on local repository manipulation.

We need to talk about tool execution. The agent doesn't guess; it uses specific functions like `CodingTools_edit_file` to modify source code safely. The system handles file system operations through glob patterns that respect `.gitignore` rules while blocking access to hidden directories. You can install the package via `uvx` and run initial coding tasks immediately.

The Claude Code style prompts used during development reveal how simple instructions generate a functional agent framework capable of running shell commands with set timeouts. We'll analyze the Python API structure that lets developers instantiate a `CodingAgent` with specific model parameters like `gpt-5.5`. This clarifies the gap between high-level benchmark performance and the gritty reality of local software automation.

The Role of LLM Coding Agents in Modern Software Automation

Defining the LLM Coding Agent and Claude Code Style Architecture

Stop thinking of an LLM coding agent as a chatbot that writes code. It is an autonomous component executing file edits and shell commands through structured tool calls. Simon Willison released version 0.1a0 of llm-coding-agent on 2nd July 2026, labeling it "A coding agent built on LLM." This implementation adopts a Claude Code style architecture depending on explicit tool use instead of implicit code generation. Models must interact with the filesystem via set functions like `edit_file` and `execute_command`. The framework exposes a CodingAgent class so developers instantiate agents with specific models and approval flags for test-driven development workflows.

Proprietary variants hit high benchmark scores. Open implementations prioritize transparency in how the LLM selects actions. Human review remains necessary because the mean acceptance rate for solutions generated by top agents on complex projects remains 27.38%. Builders gain a reference implementation for function calling patterns without opaque backend logic. The low acceptance rate on complex projects indicates "augmented coding" rather than full autonomy represents the current state of the art.

Real-World Performance of Fable 5 and GPT-5.5 on SWE-bench and Terminal-Bench

Fable 5 leads the SWE-bench Verified dataset with a 95.0% success rate, establishing a new baseline for repository-level issue resolution. Such performance metrics indicate autonomous agents now reliably navigate complex codebases to implement fixes. Terminal-heavy workflows present a different challenge entirely. Codex combined with GPT-5.5 edges ahead with an 83.4% score on TerminalBench v2.1. This distinction matters for operators deciding between local repository fixes and command-line orchestration tasks. Some scenarios demand file manipulation while others require reliable shell interaction.

Fable 5 Versus Opus 4.8: Benchmark Disparities in Terminal-Bench Scores

Model selection dictates terminal proficiency because newer architectures outperform legacy variants on command-line tasks. Claude Code running on Fable 5 achieves a close second on TerminalBench v2.1 with a score of 83.1%, trailing only the GPT-5.5 leader. The same agent framework running on Opus 4.8 scores 78.9% on TerminalBench, notably trailing the GPT-5.5 and Fable 5 variants. A 4.2-point delta illustrates that model architecture matters more than the orchestration layer for shell-heavy workflows.

Operators asking whether to use AI for coding tasks must distinguish between file editing and terminal execution capabilities. Data suggests Fable 5 handles complex command sequences improved than Opus 4.8, though both remain viable for standard file modifications. Agents on the ProjDevBench dataset average 138 turns per problem to reach a solution. Even top-tier models require significant interaction to solve complex problems. Builders should evaluate models against their specific workload profiles rather than relying on aggregate scores.

Internal Mechanics of Tool Execution and File System Operations

How CodingTools_edit_file Enforces Exact String Matching

`CodingTools_edit_file` rejects any edit request where the old_string parameter fails to match file contents exactly, down to every whitespace character. This strict comparison stops the agent from hallucinating context or applying partial matches that could corrupt code structure. The tool demands the target substring identify a unique location within the file; if the string appears multiple times, the operation fails unless the replace_all flag is explicitly set to true. Ambiguous context often leads LLM agents to unintended modifications across similar functions, a failure mode this mechanism directly addresses. Returning a diff of the change rather than the full file allows immediate verification of applied logic without re-reading the entire artifact. Documentation specifies that `old_string` must match exactly so autonomous agents cannot silently overwrite critical logic when file state drifts from the model's internal representation.

Executing Shell Commands and Handling Timeout Limits

The CodingTools_execute_command utility runs shell instructions within the session root and returns combined stdout and stderr followed by an exit code line. This mechanism enables the agent to validate code changes by running test suites or build scripts directly during autonomous workflows. Operators configure the timeout parameter in seconds, respecting a hard ceiling of 600 seconds for any single operation. When a process exceeds this duration threshold, the framework terminates the entire process tree rather than just the parent PID. Aggressive cleanup prevents orphaned child processes from consuming resources after the LLM context has moved on. Such behavior proves necessary when fixing failing tests where a compilation error might cause a test runner to hang indefinitely waiting for input. The tool definition strictly enforces this limit, killing the process tree upon timeout. Resource predictability comes at the cost of reduced autonomy, yet builders gain transparency into exactly what the agent executes. Closed alternatives often hide command execution behind a black box, whereas this approach avoids such opacity.

LLM Reasoning Limits When Selecting SwiftUI for CLI Tasks

GPT-5.5 reasoning explicitly flagged that SwiftUI lacks suitability for true command-line interfaces, yet the agent proceeded to construct one regardless. This specific failure mode exposes a disconnect between an agent's analytical chain and its action selection logic within the llm-coding-agent framework. The model correctly identified the architectural mismatch during its thought process but failed to inhibit the subsequent tool use required to generate inappropriate SwiftUI boilerplate. Instructions processed during CodingTools_write_file invocation reveal the root cause. A plausible explanation involves the agent optimizing for task completion metrics rather than technical correctness, a trend noted where leaders specialize by benchmark rather than maintaining universal competence across all domains (benchmark specialization). Autonomous development workflows require strict separation between reasoning validation and tool invocation permissions. Even high-performing models may generate functional but architecturally invalid codebases without such controls.

Practical Implementation Steps for Installing and Running the Agent

Executing the Agent via uvx and PyPI

The installation process relies on a "slop-alpha" release currently hosted on PyPI, requiring explicit flags to handle pre-release stability. Operators must use the `uvx` runner with the `--prerelease=allow` directive to bypass default version constraints and access the experimental build. This configuration enables the `llm code` entry point, which orchestrates file system tools and command execution within a local sandbox.

1. Ensure the `uv` package manager is installed on the host system.
2. Execute the agent using the specific command structure:

`uvx --prerelease=allow --with llm-coding-agent llm code`

1. Verify tool availability by checking the loaded `CodingTools` set in the runtime output.

Operate the agent immediately via the `--yolo` flag for unrestricted execution or constrain scope with `--allow` patterns. Recipes from Fable demonstrate this duality, permitting `pytest*` while blocking broader system access. This configuration balances automation speed against safety requirements in production environments. Builders requiring tighter integration can embed the `CodingAgent` class directly into Python scripts for programmatic control. Initialize the runner with a specific model like `gpt-5.5` and a target root directory to start execution.

1. Import the `CodingAgent` class from the library.
2. Instantiate with `approve=True` to enable auto-approval for trusted tasks.
3. Call the `run` method with a natural language instruction.

This API approach enables embedding agent logic within larger CI/CD pipelines or test harnesses. While the CLI offers immediate interactivity, the Python interface supports complex orchestration workflows. However, relying on autonomous execution requires careful scoping of tool permissions. Operators must weigh the convenience of auto-approval against the potential for unintended file modifications during early adoption phases.

Validating Prerelease Flags and Model Configuration

Verify the `--prerelease=allow` flag in `uvx` commands to access the alpha-stage `llm-coding-agent` package on PyPI. Without this explicit directive, the runner rejects the unstable build, preventing execution of the `llm code` entry point entirely.

1. Confirm the host environment permits pre-release package resolution.
2. Validate the active model configuration matches the intended benchmark profile before running tasks.
3. Inspect the loaded toolset to ensure file system permissions align with the `approve` parameter.

Selecting a model requires matching specific benchmark strengths rather than assuming universal capability. Complex projects consume an average of millions of tokens per problem, making model selection an economic constraint as much as a performance one. The author implemented this Python API unexpectedly, noting delight at the addition. However, the `--yolo` flag explicitly bypasses approval steps, which removes human verification gates.

Comparative Performance Against Leading AI Coding Agents

TDD Methodology in AI Coding Agents

Test-Driven Development in llm-coding-agent 0.1a0 executes a strict red/green/refactor loop where the agent generates failing tests before writing implementation code. This workflow relies on tool use for file editing and command execution to validate each commit incrementally. The development process for the agent itself utilized prompts to "build it using red/green TDD in a series of sensible commits," ensuring passing tests and updated docs at each step.

Benchmarks indicate that agents using full project contexts achieve higher success rates on project-level tasks compared to those limited to single-file completion. However, this automated approach relies heavily on the initial specification; the agent implements tools like `edit_file` which require exact string matches, meaning precise test boundaries and file content definitions are critical before execution begins. Without precise initial constraints, the agent may optimize for passing tests rather than solving the underlying business problem. Validating agent-generated tests against known edge cases remains a recommended practice before trusting subsequent code synthesis.

Generating SwiftUI CLI Apps and ASCII Art via llm code --yolo

Executing `llm code --yolo` with a prompt to build a SwiftUI command-line interface reveals immediate framework constraints. In the test transcript, reasoning explicitly notes that "SwiftUI isn't suitable for a true CLI," yet the agent proceeds to generate a functional ASCII time-telling application. This behavior demonstrates how the system navigates conflicting requirements by prioritizing executable output, resulting in an app that outputs ASCII art via standard output rather than a native GUI.

The distinction here is that while a human developer might halt to resolve the GUI-versus-CLI conflict, the agent attempts a workaround to satisfy the prompt's core intent. Observations suggest that such adaptive reasoning helps achieve high success rates in terminal-heavy tasks. However, this flexibility introduces risk; the resulting binary may lack the robustness of a manually curated project structure. Builders must verify that function calling for file creation does not obscure deeper compatibility issues when targeting specific platforms like macOS. The limitation is clear: rapid prototyping capability comes at the cost of strict architectural enforcement. This 0.

References

• Updated June 18, 2026 (post-Fable-5). Coding agents scored on
• OpenCode 2026: Setup Guide for the Top Open-Source AI
• Best AI Coding Agents in 2026, Ranked — MightyBot