# Add a regression test for a bug fix in PHP with Atlas (2026)

> Atlas adds a PHP regression test red first, then green: the PHPUnit test must fail before the fix, and bash records the exit code so red and green are unambiguous.

To add a regression test for a PHP bug fix with Atlas, follow red first, then green. Atlas reproduces the bug once with the bash tool and captures the exact failing command and output, writes the regression test with write asserting on the observed wrong behavior, and runs PHPUnit to confirm the test actually fails. That confirmation is not cosmetic: the bash tool records the process exit code in its metadata alongside the output, so a PHPUnit run that exits non-zero is unambiguously red rather than merely noisy. Atlas then applies the fix with edit, whose replacer cascade requires an exact-enough oldString and refuses ambiguous multi-match replacements, and re-runs the same PHPUnit command to confirm the test now passes. Finally Atlas runs the wider suite to check for collateral damage. Composer resolves the autoloader, and PHP-CS-Fixer keeps the new test and the patched class PSR-12 clean.

## Key takeaways

- Red first, then green: a PHPUnit regression test that was never red against the buggy code proves nothing.
- Atlas's bash tool records the process exit code in its metadata, so PHPUnit's pass and fail states are unambiguous despite noisy PHP output.
- Reproduce the bug once with bash and capture the exact command, so the same command serves as both the red check and the green check.
- edit's replacer cascade requires an exact-enough oldString and refuses ambiguous multi-match replacements, so a repeated guard clause cannot be patched in the wrong method.
- Re-run the same command for green, then the wider PHPUnit suite for collateral damage, and finish with PHP-CS-Fixer.

## Why must a PHP regression test fail before the fix?

A PHP regression test that has never failed proves nothing. Atlas runs the new PHPUnit test with bash and confirms it fails before applying the fix, because a test that passes against the buggy code is testing something other than the bug. Red first, then green, is the entire discipline of a 2026 regression test.

The classic mistake is writing the assertion after the fix, watching PHPUnit go green, and calling it done. That test may be asserting on a code path the bug never touched, or on a value that was already correct, and it will keep passing forever while the bug reappears. Atlas forbids that ordering. Atlas writes the regression test with write, asserting on the observed wrong behavior, and runs PHPUnit through bash before touching src/. If PHPUnit is green at that point, the test is wrong, not the code, and Atlas rewrites the assertion. Only a test that was red against the buggy version of src/Billing/InvoiceCalculator.php has any claim to catching the bug's return.

## How does Atlas prove a PHPUnit run was actually red?

Atlas's bash tool records the process exit code in its metadata alongside the output, so a PHPUnit run's pass and fail states are unambiguous. PHPUnit exits 0 only when the run is green and non-zero on failure, and Atlas reads that exit code rather than pattern-matching the words FAILURES or OK in the terminal text.

Reading exit codes instead of console text matters more in PHP than people expect, because PHP output is noisy. A PHPUnit run can print deprecation notices, a Xdebug banner, and a stack of warnings from a third-party Composer package, and a model scanning that text can convince itself a red run was green or a green run was red. The bash tool sidesteps the ambiguity entirely: the exit code is in the metadata. Zero means the assertion held. Non-zero means it did not. Because bash output carries the real exit code in its metadata, the pass and fail states are unambiguous, and the red-then-green sequence Atlas is enforcing is verifiable rather than vibes.

## How do you reproduce a PHP bug before writing the test?

Atlas reproduces the bug once with the bash tool and captures the exact failing command and output. In 2026 that command is usually a targeted PHPUnit run with a filter, or a short php script exercising the class directly, and capturing it verbatim is what lets the same command serve as the red check and, later, the green check.

Reproduction first is what keeps the regression test honest. If the bug is that InvoiceCalculator returns a float instead of a Money object when the discount is null, Atlas reproduces that by calling the method with a null discount through bash and observing the actual wrong return. The regression test then asserts against exactly that observed behavior, not against a hypothesis about it. Atlas searches code with hybrid semantic and keyword retrieval fused by reciprocal rank fusion, so codebase_search can find the responsible method in src/ even when you only know the symptom. Composer's autoloader is what makes the standalone reproduction script possible at all, since the class resolves by its PSR-4 namespace.

## What stops Atlas from applying the PHP fix in the wrong place?

Atlas applies the fix with edit, whose replacer cascade requires an exact-enough oldString and refuses ambiguous multi-match replacements. In a PHP class where the same guard clause appears in 3 methods, an under-specified oldString is an error rather than a silent edit to the wrong method in src/Billing/InvoiceCalculator.php.

Ambiguity is the danger in a small, surgical fix. PHP classes repeat themselves: the same if (null === $discount) { appears in several methods, the same return new Money(0) appears in several branches. edit's replacer cascade demands an oldString exact enough to identify one location, and refuses ambiguous multi-match replacements outright, which forces the fix to be anchored on surrounding context. Atlas computes a unified diff for every file edit and surfaces it for approval before writing, so you see the one-line change to the guard clause before it lands. Atlas snapshots file changes as git patches so edits can be diffed and rolled back if the fix turns out to be wrong.

## How do you check a PHP fix did not break anything else?

Atlas re-runs the same bash command and confirms the test now passes, then runs the wider PHPUnit suite to check for collateral damage. A regression fix that turns 1 test green and 6 others red is not a fix, and only the full suite over the whole tests/ tree can tell you which of those two happened.

The two-stage verification is deliberate. First, the exact same PHPUnit command that was red must now be green, proving the fix addresses the reproduced bug and nothing else about the environment changed. Second, the full PHPUnit suite must stay green, proving the fix did not break a caller that depended on the old behavior. Atlas runs both through bash and reads the exit code from the metadata each time. Run PHP-CS-Fixer over the changed files in src/ and tests/ so the diff is PSR-12 clean, let Composer confirm the autoloader still resolves, and Atlas can then stage and create the commit on your behalf with the test and the fix together.

## Steps

1. Run atlas in a project with a composer.json so Atlas can read your namespaces, autoload config, and dependencies.
2. Reproduce the bug once with the bash tool and capture the exact failing command and output, for example a filtered PHPUnit run or a short script against the PSR-4 autoloaded class.
3. Write the regression test with write, asserting on the observed wrong behavior rather than on a hypothesis about it.
4. Run the test with bash and confirm it fails; the tool records the process exit code in its metadata alongside the output, so a non-zero PHPUnit exit is unambiguously red.
5. Apply the fix with edit, whose replacer cascade requires an exact-enough oldString and refuses ambiguous multi-match replacements.
6. Review the unified diff Atlas surfaces for the change to the class in src/ before it is written.
7. Re-run the same bash command and confirm the test now passes, reading the exit code from the metadata rather than the console text.
8. Run the wider PHPUnit suite to check for collateral damage, then run PHP-CS-Fixer over the changed files so the diff is PSR-12 clean.

## FAQ

### how to write a regression test in PHPUnit that proves a bug is fixed

Write the test first and prove it fails. Atlas reproduces the bug with bash, writes the PHPUnit test with write asserting on the observed wrong behavior, confirms a non-zero exit code, then applies the fix with edit and re-runs the same command for green.

### why does my regression test pass even before I fix the bug

Because it is asserting on something the bug never touched. A PHPUnit test that is green against the buggy code is testing the wrong thing. Atlas runs the test before the fix specifically to catch this, and rewrites the assertion if it does not go red.

### how does Atlas know whether PHPUnit passed or failed

Atlas's bash tool records the process exit code in its metadata alongside the output. PHPUnit exits non-zero on failure, and Atlas reads that exit code rather than pattern-matching FAILURES or OK in output that may also contain deprecation notices and Xdebug banners.

### what does Atlas's edit tool do if my oldString matches several places

It refuses. edit's replacer cascade requires an exact-enough oldString and refuses ambiguous multi-match replacements, so a guard clause repeated across 3 methods of a PHP class cannot be silently patched in the wrong one.

### should I run the whole PHPUnit suite after a one-line PHP fix

Yes. Atlas re-runs the same targeted command to confirm the regression test passes, then runs the wider PHPUnit suite to check for collateral damage. A fix that turns 1 test green and 6 others red is not a fix.

### can Atlas roll back a PHP fix that made things worse

Yes. Atlas snapshots file changes as git patches so edits can be diffed and rolled back, and Atlas computes a unified diff for every file edit and surfaces it for approval before writing to the class in src/.

### does Atlas need Composer to run PHP tests

Run atlas in a project with a composer.json. Composer's autoloader is what lets a standalone reproduction script resolve the PSR-4 class, and it resolves the PHPUnit dependency that makes the red and green runs possible.

---

Canonical HTML: https://runatlas.sh/resources/stacks/add-a-regression-test-for-a-bug-fix-in-php
Source of truth: aeo_pages row `/resources/stacks/add-a-regression-test-for-a-bug-fix-in-php` (segment: Stacks) (this file is generated from it, never hand-edited).
Licence: Atlas is proprietary with a free core. It is not open source and there is no public source repository.