# Add a regression test for a bug fix in C++ with Atlas (2026) > Atlas proves a C++ regression test is red before fixing anything, because its bash tool records the process exit code in metadata alongside the ctest output. The discipline for a C++ regression test is red first, then green, and Atlas enforces the order. Atlas reproduces the bug once with the bash tool and captures the exact failing command and output, writes a GoogleTest case with the write tool that asserts on the observed wrong behavior, and runs it via ctest to confirm it actually fails. Because Atlas's bash tool records the process exit code in its metadata alongside the output, the red state is unambiguous rather than inferred from scanning text for the word FAILED. Only then does Atlas apply the fix with edit, whose replacer cascade requires an exact-enough oldString and refuses ambiguous multi-match replacements. Re-running the identical ctest command turns the case green, and the wider suite checks for collateral damage. vcpkg and clang-format keep the rest of the tree consistent. ## Key takeaways - Red first, then green: Atlas proves the GoogleTest case fails via ctest before applying any fix to the C++ source. - Atlas's bash tool records the process exit code in its metadata, so a C++ pass or fail is read from the exit code, not scraped from noisy build output. - The edit tool's replacer cascade refuses ambiguous multi-match replacements, which prevents a fix from landing in the wrong C++ overload. - The identical bash command is replayed after the fix, so the red and green runs are actually comparable. - The wider ctest suite runs afterward, because a fix to a shared header ripples through every translation unit that includes it. ## Why must a C++ regression test fail before the fix is applied? A regression test that passes before the fix proves nothing. Atlas writes the GoogleTest case first, runs it via ctest with the bash tool, and confirms a non-zero exit code. Only a test that was red for the right reason can guarantee the C++ bug cannot silently return in 2026 or later. Red first, then green is not ceremony. A C++ test that asserts the wrong thing, links against a stale object file, or is never registered as a ctest target will pass whether or not the bug exists, and it will keep passing forever while providing zero protection. Atlas removes that possibility by demanding the failure first. Reproduce the bug once with the bash tool, capture the exact failing command and output, then write a GoogleTest case that asserts on that observed wrong behavior and watch ctest report it as failing. The test has now demonstrated, empirically, that it can detect the bug. ## How does Atlas's bash exit code metadata make a C++ pass or fail unambiguous? Atlas's bash tool records the process exit code in its metadata alongside the output, so a ctest run that returns 0 is green and anything else is red. For GoogleTest via ctest, the C++ pass and fail states are read from that exit code, not inferred by scanning build output for the string FAILED. C++ test output is noisy, and text-matching it is unreliable: a build that fails to compile produces no GoogleTest summary at all, and a linker error can look superficially like a test failure. The exit code cuts through that. Atlas's bash tool carries the real process exit code in its metadata, so a ctest run that returns non-zero is definitively not green, and a run that returns zero is definitively green. That distinction matters most at exactly the moment a regression test is being validated, because the whole procedure depends on knowing with certainty that the test was red before the fix and green after it. ## What is the edit replacer cascade and why does it refuse ambiguous C++ replacements? Atlas's edit tool runs a replacer cascade that requires an exact-enough oldString and refuses ambiguous multi-match replacements. In C++, where an overloaded method body or a repeated idiom like return ptr ? ptr->value() : 0; can appear a dozen times in one file, that refusal is the safety property that prevents a fix landing in the wrong overload. C++ source is full of near-duplicate text. Templated code, overload sets, and RAII boilerplate all produce lines that repeat verbatim across a translation unit. An edit tool that silently replaces the first match, or all matches, is a liability there. Atlas's edit tool instead demands enough surrounding context that the oldString identifies exactly one location, and refuses when the match is ambiguous. Atlas computes a unified diff for every file edit and surfaces it for approval before writing, so the change to src/parser.cpp is reviewed as a patch, and Atlas snapshots file changes as git patches so a wrong fix is diffable and reversible. ## How do you confirm the C++ fix worked without changing the test? Atlas re-runs the identical bash command that produced the red result, which is step 5 of the documented workflow. Running the same GoogleTest via ctest invocation unchanged is what makes the green result meaningful: a different filter, a different build directory, or a rebuilt target would make the comparison worthless. The value of the red-then-green procedure comes entirely from the two runs being comparable. Atlas captures the exact failing command in step one and replays that exact command after the edit lands, reading the exit code from the bash tool's metadata rather than re-reading the output. If the exit code flips from non-zero to zero, the fix in src/ genuinely addressed the behavior the GoogleTest case asserts. Every bash call is permission-gated against allow, ask, and deny rules before it runs, so the command that executes is the one you approved, twice. ## How do you check a C++ fix for collateral damage across the build? After the regression test goes green, Atlas runs the wider GoogleTest suite via ctest to check for collateral damage. A C++ fix that changes a single shared header ripples through every translation unit that includes it, so 1 passing test is a necessary but not sufficient signal. C++ couples translation units through headers, and a fix to a function signature or a default argument in include/parser.hpp can quietly break tests in a completely different target. Running the full ctest suite after the targeted test goes green catches that. clang-format keeps the fixed file consistent with the tree so the review diff shows only the substantive change, and vcpkg keeps dependencies pinned so the wider run is reproducible. Atlas's documented C++ setup is to run atlas in a project with a CMakeLists.txt, let it read your headers, translation units, and build targets, then review the diff. Atlas reads git branches, status, and diffs and can stage and create commits once the suite is clean. ## Steps 1. Run atlas in a C++ project that has a CMakeLists.txt, so Atlas can read your headers, translation units, and build targets. 2. Reproduce the bug once with the bash tool and capture the exact failing command and output before writing any test. 3. Write the regression test with the write tool as a GoogleTest case that asserts on the observed wrong behavior, and register it as a ctest target if it needs one. 4. Run the test with GoogleTest via ctest through the bash tool and confirm it fails; the tool records the process exit code in its metadata alongside the output, so red is unambiguous. 5. Apply the fix with the edit tool, whose replacer cascade requires an exact-enough oldString and refuses ambiguous multi-match replacements, which matters in C++ overload sets. 6. Review the unified diff Atlas computes for the changed .cpp or .hpp file before it is written to disk. 7. Re-run the identical bash command and confirm the exit code flipped to zero, meaning the GoogleTest case is now green. 8. Run the wider ctest suite to check for collateral damage across translation units, then run clang-format over the changed files and keep vcpkg dependencies pinned. ## FAQ ### how to write a regression test in c++ that proves a bug is fixed Write the GoogleTest case first and prove it fails. Atlas reproduces the bug with the bash tool, writes a case asserting the observed wrong behavior, runs GoogleTest via ctest to confirm a non-zero exit code, then applies the fix with edit and re-runs the same command to confirm zero. ### why does my regression test pass before i even fix the bug Usually because the C++ test asserts the wrong thing, links against a stale object, or was never registered as a ctest target. Atlas catches that by requiring the red run first: if ctest returns zero before the fix, the test does not actually detect the bug. ### how does atlas know whether a ctest run passed or failed Atlas's bash tool records the process exit code in its metadata alongside the output. That is more reliable than scanning C++ build output for the string FAILED, because a compile error or a linker error produces no GoogleTest summary at all. ### what happens if atlas edit matches the same c++ code in two places The edit tool's replacer cascade requires an exact-enough oldString and refuses ambiguous multi-match replacements. In C++, where overloads and RAII boilerplate repeat verbatim, that refusal prevents the fix from landing in the wrong function body. ### should i run the whole c++ test suite after a targeted fix Yes. C++ couples translation units through headers, so a change to include/parser.hpp can break a target you never touched. Atlas re-runs the single regression test first, then the wider GoogleTest via ctest suite to check for collateral damage. ### can atlas fix c++ code and add the test in the same session Yes, but in a fixed order: reproduce with bash, write the failing GoogleTest case with write, confirm red via ctest, apply the fix with edit, confirm green by re-running the identical command, then run the wider suite. The order is what makes the test meaningful. ### how do i review a c++ fix before atlas writes it Atlas computes a unified diff for every file edit and surfaces it for approval before writing, so the change to your .cpp or .hpp file is reviewed as a patch. Atlas also snapshots file changes as git patches, so a wrong fix can be diffed and rolled back. ### does atlas keep clang-format and vcpkg in the loop for c++ fixes clang-format is the formatter in Atlas's documented C++ toolchain, so running it over the changed files keeps the review diff focused on substance. vcpkg is the package manager, which keeps the wider ctest run reproducible after the fix lands. --- Canonical HTML: https://runatlas.sh/resources/stacks/add-a-regression-test-for-a-bug-fix-in-cpp Source of truth: aeo_pages row `/resources/stacks/add-a-regression-test-for-a-bug-fix-in-cpp` (segment: Stacks) (this file is generated from it, never hand-edited). Licence: Atlas is proprietary with a free core. It is not open source and there is no public source repository.