Recently, Codex promotions have been unprecedented, with quota resets happening one after another. I went from daily calculations of whether I had enough quota to daily calculations of how to spend it all.
After all, if you use less quota today, you lose out when it resets tomorrow. Not to mention, I have a reset card that is about to expire.
If you are facing the same problem and looking for a task that can run automatically with a simple Prompt without much intervention—allowing you to spend your quota while doing useful work—then you might want to learn from my approach: letting Sol Ultra automatically fix bugs in your project.
Automated Bug Fixing Strategy
Model Configuration
First, the model configuration:
- Codex app
- Model: 5.6 Sol Ultra
- Goal mode enabled
Sol is the most expensive model in the GPT 5.6 series. Ultra not only has the strongest reasoning intensity but also activates as many Multi-Agents as possible to solve complex problems.
The Codex app is used to control the browser via a Chrome extension, which is the most powerful weapon for automated bug fixing. I will introduce how to use it next.
Goal mode is enabled to prevent task interruption, as I usually leave it running overnight. However, Sol Ultra itself has an extremely strong will to complete tasks; Goal mode is just double insurance.
Of course, to enable Codex to automatically discover, analyze, and fix problems, you also need some Prompt techniques.
How to Uncover Hidden Bugs
From a user's perspective, explore various project functions through the browser entry point to uncover and fix hidden bugs.
Start with non-obvious scenarios that are not easily discovered by routine manual verification to increase the probability of finding bugs. For example:
- Check for error requests or duplicate requests in the browser F12 DevTools.
- Check for errors in service logs.
- Look at inconspicuous UI issues or edge cases.
During the exploration process, consider cross-functional scenarios where different features might interact or affect each other, as these are more likely to hide complex issues.Technique 1: Explore from the browser entry point using a user perspective
Many automated tests start from API endpoints, which is suitable for deterministic, programmatic calls. However, since we are not short on money (enough Tokens) or time (running overnight), exploring from the browser entry point allows the Agent to explore more freely and stay closer to real usage scenarios (discovering real bugs).
Being able to explore this way is thanks to Codex’s powerful Chrome extension, which can easily control the browser and reuse my login state within it.
Technique 2: Consider edge cases and cross-functional scenarios
The main product paths and common scenarios rarely have problems because people use them frequently. Real issues often hide in scenarios that are rarely used or easily overlooked. Therefore, I specifically guide the Agent to explore in this direction. This makes it easier to find major bugs rather than trivial ones.
During the Codex run, I also noticed its initial strategy was conservative, analyzing module by module. So, I specifically added the “cross-functional scenarios” prompt. After all, the interaction between multiple scenarios is less likely to be encountered in daily use and more likely to hide latent bugs.
How to Fix Bugs
My strategy is to let Codex discover and fix bugs on the spot.
Some people might think that discovery and fixing should be separated: let one Agent explore and record a batch of issues first, then let another Agent fix them based on those issues.
However, this method has a huge flaw: how do you judge if the discovering Agent and the fixing Agent have consistent logic? How do you ensure the fix doesn’t introduce new problems?
I believe this is a major pain point for many: I just want to use a single Prompt to let the Agent help me spend some quota and make the project code better. If I have to spend time manually analyzing each bug, it defeats the purpose.
So my approach is:
- Let the Agent score the “Certainty of the Fix.” Only if the score is above 80% will it perform an automatic fix. This skips architectural decisions or product design issues, preventing the Agent from making large-scale changes that could spiral out of control.
- Fix problems immediately upon discovery to ensure the most coherent thought process.
- Require the Agent to immediately re-run the reproduction path after a fix to verify the result on the spot. This prevents the Agent from hallucinating and “thinking it has fixed the problem.”
With this combination of tactics and the power of 5.6 Sol, you can basically trust the results of the Agent’s automated fixes without needing extensive manual inspection.
Summary
I ran this on a project for about 13 hours. Sol Ultra worked diligently, launching a massive number of Subagents and fixing a total of 48 bugs.
This number far exceeded my expectations. Most of the bugs were actual issues, not minor things like typos or copy-editing. Moreover, I only spent some Codex quota (which would have expired and gone to waste anyway).
Friends who are worried about having too much quota might want to give this a try.
Appendix: Full Prompt
## Goal
From a user's perspective, explore various project functions through the browser entry point to uncover and fix hidden bugs.
## How to Uncover Hidden Bugs
Start with non-obvious scenarios that are not easily discovered by routine manual verification to increase the probability of finding bugs. For example:
- Check for error requests or duplicate requests in the browser F12 DevTools.
- Check for errors in service logs.
- Look at inconspicuous UI issues or edge cases.
During the exploration process, consider cross-functional scenarios where different features might interact or affect each other, as these are more likely to hide complex issues.
## How to Explore
Start the service locally and verify by operating the browser.
The primary exploration methods should be code reading + browser operations.
Allowed operations:
- Browser control
- Querying logs and database records
Disallowed operations:
- Deleting or modifying database data
- Directly calling APIs to construct test scenarios (all bugs must be reproducible through browser operations)
## Fix Method
Fix bugs immediately upon discovery. You must perform regression testing through the same reproduction path to verify the fix.
Use bugs.md to record the bug discovery and fixing process.
Score the certainty of the fix for discovered bugs. If the certainty is below 80%, do not fix it; move it to open-bugs.md and only record the reproduction path, expected results, and actual results. Regardless of whether the bug is fixed, record the fix certainty score for each bug.
## Stop Conditions
Satisfy one of the following:
- Time exceeds 10 hours
- One full round of your planned exploration is completed, and all bugs are fixed