Blogmark

I'm looking for some details on how harnesses edit code. There are high-level details in this post about code/file editing being part of the tool use architecture.

First, in "Tool Exposure":

Tool exposure: The harness declares the tools the model is allowed to call: reading files (readfile), editing code (replacestringinfile or applypatch), running terminal commands (runinterminal), searching the codebase (semanticsearch), and many more.

Then, in "Tool Execution":

Tool execution: When the model requests a tool to be run (using JSON like {"name": "runinterminal", "arguments": {"command": "npm test"}}), the harness is the one that validates the arguments, runs the tool, handles errors, formats the result, and feeds it back in the next iteration. For example, if the model asks to edit a file, the harness writes the diff. If the model asks to run a shell command, the harness is what spawns the process, captures output, and relays it.