Atlas runs a Python suite with pytest through its bash tool and turns the red output into a prioritized list of distinct root causes. Because a full pytest run produces far more output than any model should read, Atlas truncates at 2000 lines or 50 KB, writes the complete log to a retained file, and tells you the path, so triage happens against the whole log instead of a lossy tail. Atlas then greps that log to group failures by cause rather than by test name, records one todowrite entry per cause with status pending, and fixes them one at a time with edit, re-running only the affected pytest node IDs in between.
How do you triage a wall of pytest failures with an AI agent?
Atlas triages pytest failures by grouping them by root cause, not by test name. A Python suite that reports 143 failures rarely has 143 bugs; it usually has three, and one of them is a fixture in conftest.py. Atlas greps the saved pytest log to find the three, then records each as a todowrite entry.
The instinct when pytest prints a screen of red is to open the first failure and start fixing. That is the slow path in Python, because Python failures cluster hard. A single broken fixture in conftest.py fails every test in the package that requests it. One renamed model field fails every Django test that touches the ORM. A dependency resolved differently by uv fails every module that imports it, and the failures show up as collection errors before a single assertion runs. Atlas attacks the cluster instead of the leaves. It greps the log for the exception type and the traceback's deepest repeated frame, and the 143 failures collapse into three named causes: an ImportError from a missing extra, an AttributeError from the renamed field, and a fixture that now returns None. Three todowrite entries, three fixes, one green suite.
How does Atlas handle pytest output that is too long to read?
Atlas's bash tool truncates output at 2000 lines or 50 KB, writes the complete log to a retained file, and prints the path in an output truncated header. A pytest run over a large Django or FastAPI suite blows past that limit routinely, so Atlas reads the saved file rather than a lossy tail.
Context windows and pytest tracebacks are a bad match. A single failing test in a FastAPI service can emit forty lines of traceback through the dependency injection stack, and a hundred of those fills any model's working memory with noise. Atlas's answer is not to shrink the output; it is to keep all of it somewhere addressable. The bash tool caps what comes back inline at 2000 lines or 50 KB, saves the complete log to a file, and names the path in the truncation header. Atlas then uses read on that file to pull the sections that matter and grep to count occurrences across the entire run. Counting matters: knowing that AttributeError appears 96 times and ImportError 41 times tells you which cause to fix first, and no tail of the console output would have told you that. Pass a generous timeout in milliseconds to the bash tool so a slow pytest suite is not killed mid-run.
What pytest commands does Atlas actually run in a Python repo?
Atlas runs the real Python commands, not abstractions. In 2026 the full sweep is pytest through the bash tool with a generous timeout. Re-runs are narrowed with pytest -k or an exact node ID such as tests/test_orders.py::test_refund_partial, and formatting after a fix is ruff format on the files that changed.
A useful triage session in a Python repo looks like this in the transcript. First, pytest over the whole suite, run by the bash tool with a millisecond timeout large enough for a suite that hits a database. Second, read on the saved log path from the truncation header. Third, grep over that log for Error and for the repeated frames, which is how the distinct causes surface. Fourth, one todowrite entry per cause, status pending. Fifth, edit against the production module, then a narrow re-run: pytest -k on the affected selector, or the exact node ID, so the feedback loop is seconds instead of minutes. Only when the narrow runs are green does the full pytest go again. Dependencies get resolved with uv when a failure turns out to be an environment problem rather than a code problem, and ruff format tidies the files Atlas touched before you review the diff.
How does Atlas keep track of multiple Python test failures at once?
Atlas records one todowrite entry per distinct root cause, each with status pending, so a triage session across 143 pytest failures does not lose the third cause while fixing the first. The todo list is the working memory of the session and survives across turns.
Multi-cause triage is where agents usually drift. The model fixes the ImportError, the suite gets less red, momentum takes over, and the AttributeError cluster quietly never gets addressed because it fell out of context ten turns ago. Atlas's todowrite list exists to stop exactly that. Each distinct cause becomes an entry with status pending, and the entries stay visible while the fixes proceed one at a time with edit. In a Python repo the entries read concretely: fix the None-returning fixture in tests/conftest.py, restore the renamed field in app/models/order.py, pin the missing extra so uv resolves it. When the last entry flips off pending, the full pytest run is the proof, not the vibe. Atlas also snapshots file changes as git patches, so a fix that made things worse can be diffed and rolled back without losing the rest of the session's work.
Is it safe to let Atlas run pytest and edit Python code unattended?
In 2026, every Atlas tool call is permission-gated against allow, ask, and deny rules before it runs, which includes the bash tool that invokes pytest and the edit tool that changes app/models/order.py. Atlas also computes a unified diff for every file edit and surfaces it for approval before writing.
Running a Python test suite is not a harmless action. A pytest run can hit a real database, a test fixture can call an external service, and a careless bash command in a repo with a management script can do considerably worse. Atlas puts every one of those behind the same permission model: allow, ask, and deny rules evaluated before the tool call executes. Teams commonly allow pytest and ruff format outright, ask on anything invoking uv, and deny destructive shell patterns entirely. Edits are separate and stricter: Atlas computes a unified diff for every file edit and surfaces it for approval before writing, so the fix to the fixture in tests/conftest.py is something you read before it exists on disk. Combined with git patch snapshots for rollback, a triage session that touches a dozen Python files stays reversible from start to finish.
Step by step
- 01Run the full suite with the bash tool: pytest, with a generous timeout in milliseconds so a slow Django or FastAPI suite is not killed mid-run.
- 02If the bash tool reports output truncated at 2000 lines or 50 KB, read the log file named in the truncation header to see the complete pytest output.
- 03Grep the saved log for exception types and repeated traceback frames to group the failures by root cause instead of by test name.
- 04Separate collection errors from assertion failures; a pytest collection error usually means an import or an environment problem that uv can resolve, not a code bug.
- 05Record one todowrite entry per distinct cause, each with status pending, so a fifth cause is not forgotten while the first is being fixed.
- 06Fix the causes one at a time with edit, approving the unified diff Atlas surfaces for each Python file before it is written.
- 07Re-run only the affected tests between fixes via bash, using pytest -k or the exact node ID such as tests/test_orders.py::test_refund_partial.
- 08Run ruff format on the changed files, then run the full pytest suite once more to confirm every todowrite entry is genuinely closed.
Frequently asked questions
- How do I get an AI agent to triage hundreds of failing pytest tests?
- Have it group failures by root cause. Atlas runs pytest through bash, saves the full log when output exceeds 2000 lines or 50 KB, greps that log for exception types and repeated traceback frames, and records one todowrite entry per distinct cause with status pending.
- What happens when pytest output is too long for the model's context window?
- Atlas's bash tool truncates the inline output at 2000 lines or 50 KB, writes the complete log to a retained file, and prints the path in an output truncated header. Atlas then reads and greps that file, so no failure is lost to a truncated tail.
- Can Atlas tell the difference between a pytest collection error and a test failure?
- Yes, and the distinction drives the fix. A pytest collection error usually points at an import or environment problem you resolve with uv, while an assertion failure points at code. Atlas separates them when grouping the saved log and files them as separate todowrite entries.
- Does Atlas re-run the whole pytest suite after every fix?
- No. Atlas re-runs only the affected tests between changes through bash, using pytest -k or an exact node ID such as tests/test_orders.py::test_refund_partial, and runs the full suite once at the end to confirm every todowrite entry is closed.
- How does Atlas avoid forgetting a failing test while fixing another one?
- Atlas records one todowrite entry per distinct root cause with status pending. The list persists across turns, so fixing the ImportError cluster does not push the AttributeError cluster out of context before it is addressed.
- Is it safe to let Atlas run pytest against a repo with a real database?
- Every Atlas tool call is permission-gated against allow, ask, and deny rules before it runs, including the bash tool that invokes pytest. Approve pytest and ruff format, ask on uv, and deny destructive shell patterns to keep an unattended run bounded.
- Can I undo the Python fixes Atlas made during triage?
- Yes. Atlas snapshots file changes as git patches so edits can be diffed and rolled back, and Atlas computes a unified diff for every file edit and surfaces it for approval before writing to app/models/order.py or tests/conftest.py.
Try Atlas in your terminal
The terminal-native AI coding agent. Free core, single binary.
Install AtlasRelated guides
Run the Test Suite and Triage the Failures with Atlas in 2026
How to triage a failing test suite with Atlas in 2026: bash truncates at 2000 lines or 50 KB and saves the full log, then grep groups failures by root cause.
Atlas for Python in 2026
Atlas is a terminal-native AI coding agent for Python in 2026. Run it in a repo with a pyproject.toml or requirements.txt and review every diff before it lands.
Refactor a legacy module in Python with Atlas (2026)
Refactor a legacy Python module with Atlas in 2026: enumerate callsites with lsp findReferences, restructure with apply_patch, and prove behavior with pytest.
Onboard to an Unfamiliar Python Codebase with Atlas in 2026
Onboard to an unfamiliar Python codebase in 2026. Atlas uses codebase_search, glob, read, and lsp to map a pyproject.toml repo without opening every module.
Research a Third-Party API Before Integrating It in Python with Atlas in 2026
Research a third-party API before integrating it in Python in 2026. Atlas uses websearch and webfetch to pull live docs, then writes against real signatures.
Document a Python Module With a README Using Atlas (2026)
Atlas writes Python docs from source, not memory: lsp documentSymbol lists the real exports, read supplies behavior, and every code sample is proven with pytest.
Upgrade Python Dependencies and Fix Breakage with Atlas in 2026
In 2026, Python developers use Atlas to upgrade dependencies like Django or FastAPI, automatically fixing compile and test failures with `uv`, `pytest`, and `ruff format`.
Automate GitHub Issue and Pull Request Triage in Python with Atlas (2026)
Wire the atlas github command into a Python repo's workflow in 2026. Require MODEL in provider/model form, gate on write permission, and verify with pytest and ruff format.