Atlas works with pytest through the shell, because pytest has no official MCP server. Atlas shells out with a machine-readable reporter, writes the failing test first, and implements until the suite is green. Install pytest-json-report, then have Atlas run pytest -q --json-report --json-report-file=report.json and read failures straight from JSON rather than from scraped terminal text.
pytest has no MCP server, so Atlas uses a reporter
pytest has no official MCP server in 2026, so Atlas runs it through the shell with a machine-readable reporter. That is not a workaround. A reporter is how pytest has always exposed results to other programs, and structured results are the only interface an AI coding agent actually needs from a test runner.
Atlas is a terminal-native agent, which means the pytest CLI is already inside its normal working surface. What it needs from pytest is not a new protocol but a result it can parse without ambiguity. Human-facing pytest output is designed for eyes: dots, colored tracebacks, a summary line. An agent scraping that text will eventually misread a traceback or lose a failure in the noise. A machine-readable reporter removes the guesswork entirely, handing Atlas the failing node ID, the outcome, and the traceback as fields.
Structured failures with pytest-json-report
Install pytest-json-report, then have Atlas run pytest -q --json-report --json-report-file=report.json and read failures straight from JSON. Those 3 flags write the whole run to report.json, so Atlas opens 1 file and knows exactly which tests failed and why, with no terminal scraping involved.
Read the command left to right. The -q flag keeps pytest quiet on stdout, since the agent is not reading stdout anyway. The --json-report flag activates the pytest-json-report plugin. The --json-report-file=report.json flag names the output file. What lands in report.json is a full structured record of the run, including each test's node ID, its outcome, and the traceback for anything that failed. Atlas reads that file, forms a hypothesis about the failure, edits the code, and runs pytest again. Every iteration is grounded in data rather than in an approximate reading of console output.
Trimming the payload with --json-report-omit
Trim the payload with --json-report-omit=keywords,streams so a large suite does not flood the context window. Those 2 sections, per-test markers and captured stdout, carry a great deal Atlas will never read, and every byte competes for the same limited context budget.
Context is the scarce resource in any agent loop, and a bloated report spends it on nothing. The keywords section records pytest markers per test, and the streams section captures the captured stdout and stderr of every test, including the ones that passed. Omitting both with --json-report-omit=keywords,streams leaves exactly what matters, which is the outcomes and the tracebacks for failures. On a small suite this hardly matters. On a large one it is the difference between Atlas reading the report and Atlas running out of room to reason about it.
Streaming results with pytest-reportlog
For streaming results, pytest-reportlog emits JSON Lines: run pytest --report-log=out.jsonl. Instead of 1 JSON document written at the end of the run, out.jsonl gains 1 JSON object per line as the run progresses, which is the format to reach for when the suite is slow.
The difference between pytest-json-report and pytest-reportlog is when the data arrives. pytest-json-report writes report.json once the run has finished, which is fine for a fast suite. pytest-reportlog appends a line to out.jsonl as each result comes in, so failures are readable while the rest of the suite is still running. On a long suite that matters, because Atlas can start reasoning about the first failure without waiting for the last test to finish. JSON Lines is also trivially chunked, so Atlas can read the tail of out.jsonl rather than loading the entire file.
Allowing 'pytest *' under permission.bash
Set 'pytest *' to allow under permission.bash so the loop runs without a prompt on every iteration. Atlas asks before every shell command by default in 2026, which is right for most commands and wrong for the 1 command a test-driven loop runs dozens of times per session.
Scope the permission to the command pattern rather than turning off shell approval entirely. Mapping 'pytest *' to allow under permission.bash in atlas.json means Atlas can run the suite as often as the loop demands, while any other shell command still stops and waits for you. Running tests is safe, repeatable, and read-only with respect to your source code, which is exactly the profile of a command worth allowing without a prompt. Everything with real consequences stays behind the approval gate where it belongs.
Red, green, refactor: write the failing test first
Ask Atlas to write the failing test first, watch it fail, and only then implement until pytest is green. Those 3 steps, red then green then refactor, are the only way to know that the test you just wrote can actually fail, which is what makes a passing suite mean something.
An agent that writes an implementation and a test in the same breath will produce a test that passes, and you will have no idea whether it would ever fail. Inverting the order fixes that. Atlas writes the failing test, runs pytest -q --json-report --json-report-file=report.json, and confirms in the JSON that the test failed for the reason it was supposed to fail. Only then does it implement. The red state is evidence, and the green state after it is a real result rather than a tautology. Review the diff of both the test and the implementation before either lands.
Setup
- 01Skip the MCP search: pytest has no official MCP server, so Atlas runs it through the shell with a machine-readable reporter.
- 02Install pytest-json-report, then have Atlas run pytest -q --json-report --json-report-file=report.json and read failures straight from JSON.
- 03Trim the payload with --json-report-omit=keywords,streams so a large suite does not flood the context window.
- 04For streaming results, use pytest-reportlog, which emits JSON Lines: pytest --report-log=out.jsonl
- 05Set 'pytest *' to allow under permission.bash so the loop runs without a prompt on every iteration.
- 06Ask Atlas to write the failing test first, watch it fail, and only then implement until pytest is green.
Frequently asked questions
- is there a pytest MCP server
- No. pytest has no official MCP server, so Atlas shells out with a machine-readable reporter and reads failures from JSON instead of scraping terminal output.
- how do I get JSON output from pytest for an AI agent
- Install pytest-json-report, then run pytest -q --json-report --json-report-file=report.json. Atlas reads the failing node IDs and tracebacks straight from report.json.
- how do I stop a large pytest report from flooding the context window
- Trim the payload with --json-report-omit=keywords,streams. That drops per-test markers and captured stdout, leaving the outcomes and tracebacks Atlas actually reads.
- how do I stream pytest results as JSON Lines
- Use pytest-reportlog and run pytest --report-log=out.jsonl. It appends one JSON object per line as the run progresses, which suits a slow suite.
- how do I stop Atlas asking permission before every pytest run
- Set 'pytest *' to allow under permission.bash in atlas.json. Only that command pattern is allowed, so every other shell command still prompts.
- can an AI agent do test driven development with pytest
- Yes. Ask Atlas to write the failing test first, watch it fail, and only then implement until pytest is green, so the red state proves the test can actually fail.
- why should an AI agent not scrape pytest console output
- Because human-facing pytest output is designed for eyes, and scraping it eventually misreads a traceback. A machine-readable reporter gives Atlas the node ID, outcome, and traceback as fields.
Try Atlas in your terminal
The terminal-native AI coding agent. Free core, single binary.
Install AtlasRelated guides
Atlas with DeepSeek Chat: 384,000 Token Output at $0.28 per Mtok in 2026
Run Atlas on DeepSeek Chat in 2026. DeepSeek's non-reasoning endpoint gives 1M tokens (1,000,000) of context at $0.14 per Mtok input, $0.28 per Mtok output.
Atlas with Qwen3 235B-A22B: Flagship Sparse Reasoning in 2026
Qwen3 235B-A22B in Atlas: 235B total parameters, 22B active per token, $0.70 per Mtok input and $2.80 per Mtok output, 128K tokens (131,072) of context.
Atlas for COBOL: Copybooks, PIC Clauses, and GnuCOBOL in 2026
Atlas is a terminal-native AI coding agent for COBOL in 2026. It reads your divisions and copybooks, explains a paragraph before touching it, and compiles with cobc under GnuCOBOL.
Atlas for React Native: Terminal-Native AI Coding Across the Native Boundary in 2026
Atlas is a terminal-native AI coding agent for React Native in 2026. Work across the New Architecture, native modules, and platform-specific files with diff-first review.
Atlas with DeepCoder 14B (Ollama): RL-Tuned for First-Attempt Diffs in 2026
DeepCoder 14B (Ollama) is a 9.0GB RL-tuned coder with a 128K context, built for first-attempt correctness. Free (self-hosted). Atlas setup and tradeoffs for 2026.
Atlas with Gemini 3.5 Flash: The May 2026 Latency Tier That Keeps 1M Context
Gemini 3.5 Flash in Atlas: the May 2026 release keeps the full 1,048,576 token window at $1.50 / $9 per Mtok, with reasoning and tool calling enabled.
Atlas with Gemma 4 E4B (Ollama): the Default Gemma 4 Tag in 2026
Gemma 4 E4B (Ollama) is the :latest tag of Google's newest Gemma line: 9.6GB, 128K tokens (131,072), Free (self-hosted), with qat, mlx, mxfp8 and nvfp4 quants.
Atlas with Qwen3.5 Plus: A Million-Token Window for $0.40 per Mtok in 2026
Qwen3.5 Plus gives Atlas a 1M tokens (1,000,000) context window at $0.40 per Mtok input and $2.40 per Mtok output. What the million tokens buy, and what closed weights cost.