In my experience, the LLM usually proposes a solution, but it’s a crapshoot whether it works well. And plenty of bugfixes go well beyond a few lines, especially in older code.
I use this shit on a daily basis at work. They work well, as long as you’re choosing the right models, making the right engineering decisions, and have a good set of baseline rules.
Do you have a guide on which baseline rules are good? I use Claude Opus 5.5 at work and while it’s definitely more helpful than not having it, it also does some incredibly baffling things that a human would realize immediately are unproductive.
“Baseline” is subjective, so I’d recommend just start a global rules file and steer it away from the behaviors you don’t like. Don’t be afraid of quizzing the agent itself on why it did that thing, and what’s a good bit of language to add to the rules file to prevent it from happening.
Also, for specific tasks, skill files are really useful, and LLMs can write the whole thing, based on what you did the hard way in a session. Same thing for triggering a skill file and then ask it if there are any useful improvements to make to the skills file in the future, at the end of the session.
Most vulnerability patches are a few lines of code. Finding it is the hard part.
Even then, the AI session that found it probably already has a solution to fix it.
In my experience, the LLM usually proposes a solution, but it’s a crapshoot whether it works well. And plenty of bugfixes go well beyond a few lines, especially in older code.
I use this shit on a daily basis at work. They work well, as long as you’re choosing the right models, making the right engineering decisions, and have a good set of baseline rules.
Do you have a guide on which baseline rules are good? I use Claude Opus 5.5 at work and while it’s definitely more helpful than not having it, it also does some incredibly baffling things that a human would realize immediately are unproductive.
“Baseline” is subjective, so I’d recommend just start a global rules file and steer it away from the behaviors you don’t like. Don’t be afraid of quizzing the agent itself on why it did that thing, and what’s a good bit of language to add to the rules file to prevent it from happening.
Also, for specific tasks, skill files are really useful, and LLMs can write the whole thing, based on what you did the hard way in a session. Same thing for triggering a skill file and then ask it if there are any useful improvements to make to the skills file in the future, at the end of the session.