AI-assisted Coding (46 blogmarks)

A lot of people seem confused because AI both lowers the floor AND raises the ceiling. They [see] slop and assume thats all that's possible. But special is possible but takes effort!

Don't listen to people who cannot imagine using a powerful tool to do anything but put minimal effort in.

I quickly figured out ways to integrate LLMs and coding harnesses into my day-to-day as a software developer. As the models get better, so do the results. There is still a wide range of quality that you can end up.

What I’ve been finding most exciting lately is figuring out specific prompts, skills, and workflows to get better software from a coding harness than if I were writing it on my own. I’m nowhere near the ceiling on that.

In Another attempt at cutting back on LLM code comments, I showed a small prompt I was passing to a separate model / context (as a sort of Adversarial Review) to clean up excessive code comments that Claude Code was peppering around changesets for feature work.

I've been getting way too chatty in the code comments and sometimes write comments that aren't going to help someone in the future or get too in the weeds on details that won't have long-term durability. Can you do a pass to reduce or remove any unnecessary comments added by this branch?

That's been working pretty well, but I wanted to see how I could improve on it. I've been rediscovering the magic of asking LLMs to write their own prompts. To improve on my one-liner, I thought I'd ask Claude to write a prompt that it would give to a reviewer of code comments.

Here is my prompt for a prompt:

Can you write a prompt for someone who like to clean up code comments before a PR goes through? They have a strong sense of when comments add long-term durable value for the codebase and when comments are documenting fleeting details about the current implementation that don't have long term value. They want the code to be well documented, but they strongly prefer concision and would rather let the code speak for itself if it is well written and the understanding can easily be derived.

And here is what Opus 5.5 came up with:

You are reviewing code comments on this branch before the PR merges. Look only at comments added or changed in the diff (`git diff main...HEAD`). Leave existing comments alone unless the diff makes them stale.

Your bar: a comment stays only if it will still be true and useful to someone reading this code a year from now with no knowledge of the PR, the ticket, or the conversation that produced it.

Keep comments that explain:
- Why, when the reason isn't visible in the code: a constraint, a tradeoff, a non-obvious invariant, a workaround for an external bug (link it)
- Surprising behavior a careful reader would otherwise "fix" and break
- Contracts at public boundaries: inputs, outputs, side effects, error cases that the signature doesn't show
- Domain knowledge the code relies on but can't express

Remove comments that:
- Restate what the code does (`# increment counter`, `# loop over users`)
- Describe the change rather than the code ("now uses X instead of Y", "added to fix…", "new:", "updated")
- Mention the PR, ticket, session, or author's process ("per review", "as discussed", "for now", "temporary" with no plan to remove)
- Tell the story of how the implementation got this way
- Are leftover scaffolding: TODOs with no owner or trigger, commented-out code, debug notes
- Section-label obvious structure (`# --- helpers ---` in a short file)

Rewrite comments that hold real value but are padded. Cut them down to the fewest words that keep the meaning. One line beats three. Drop filler like "Note that", "This function", "Basically".

Before keeping a "what" comment, ask whether a better name, an extracted function, or a named constant would make it unnecessary. If so, suggest that change rather than keeping the comment. Don't make the change yourself unless it's trivial and clearly safe.

Match the codebase's existing comment density and style. Don't add new comments unless something genuinely non-obvious has none.

Output: apply the edits, then give a short list grouped as removed / trimmed / kept-but-flagged / suggested-refactor, with a one-line reason for each that isn't self-evident.

You can tune it in a few places:
- Scope: change main...HEAD if they use a different base branch.
- Autonomy: if they want to approve each change first, swap "apply the edits" for "propose the edits as a list; don't modify files."
- Docstrings: if the project needs doc comments on its public API (Elixir @doc, JSDoc, and so on), add "Never remove required API doc comments; only tighten them."

The difference I'm noticing so far is that this prompt is leading the LLM to be more judicious about removing comments and opts toward simplifying and condensing the useful parts instead of deleting them wholesale. Though I need to spend more time with this one, it already feels like an improvement.

Justin Searls wrote about adversarial review saying:

Over a year ago, I stopped reading code and started automating adversarial agents to perform recursive code reviews instead. The result: low-defect systems that do what I want. Can't believe anyone engaging seriously with these tools still believes human code review is superior

More and more of the code review that I do day-to-day is agent driven and I’m constantly looking for ways to make it more effective. This sounds compelling, so I’m looking into how to approach adversarial review.

Adversarial code review applies the maker-checker principle to AI coding agents: the agent that writes a change never certifies it. The pattern uses a distinct reviewer role to check code produced by a builder role against the spec before human review. An implementation can add opposing reviewer roles. A primary reviewer proposes findings, a challenger tries to disprove them and hunt for missed issues, and an arbiter weighs both sides and decides what blocks the merge.

While looking for tips on how to get started with adversarial reviews, I came across a reddit post that made a simple but surprisingly effective recommendation.

ask: *write a prompt for an adversarial reviewer.*

The OP goes on to say:

You'll probably get a carefully considered prompt with excellent context you can paste straight into a fresh claude (or better yet, Codex/GPT 5.4) session. It'll almost certainly find at least a few things that would've cost you much more time later.

A couple variations I've tried for this today:

1. "Can you write a prompt for an adversarial reviewer for this feature?"

2. "Can you write a prompt for an adversarial feature / design doc review? It should be generic, but I will use it to review a single Linear issue that represents an epic (several user stories for a whole feature)."

Dax of OpenCode recently wrote:

a weird inversion with LLMs is the models improve faster than the tinkerers

when i see people with custom workflows and setups they're all addressing problems that don't exist anymore

the person naively using vanilla codex is more likely to be experiencing state of the art

I find this to be a challenging conundrum as someone who has typically invested a lot in my tooling and development environment. It seems like as quickly as I learn about and start using a new tool or technique for improving some aspect of AI-assisted coding, it goes out of fashion.

My approach for the past year or so has been: stay curious and try new things that catch my attention. And hold those things lightly and be ready to throw away my assumptions about what works well and what these models are capable of.

I'll also add that the churn can be overwhelming, so feel free to use, as Dax suggests, vanilla Codex or vanilla Claude Code with frontier models. You're still gonna get good results.

My response to Kieran Murphy's post about how "pairing" with Claude (i.e. one human and one AI agent) does not constitute pair programming in a meaningful way.

I’ve yet to try it, but I feel bullish about two humans pairing on the orchestration of multiple agents.

Where pairing shines is the heightened level of engagement, opportunity to discuss edge cases, and learning patterns of the craft from one another. Pairing feels dull when you are in the doldrums of writing tedious code. AI coding agents take care of the latter so that you can do more of the former.

That’s my theory anyway.

I agree with Kieran's main point (you need two people to pair) and go a step further to suggest that pair programming is probably the best way for us to adapt to the new way of building software in the era of AI.

I tried to get Claude Code to chill with the excessive and ephemeral code comments it was leaving all over the place during a feature build. I attempted this with a bit of text in CLAUDE.md on Code Comment Conventions. It is not having enough of an impact.

What I've been experimenting with this week that seems much more effective is a separate pass when it is done with this prompt:

I've been getting way too chatty in the code comments and sometimes write comments that aren't going to help someone in the future or get too in the weeds on details that won't have long-term durability. Can you do a pass to reduce or remove any unnecessary comments added by this branch?

Sometimes I even hand this to a separate model via Codex to mix things up.

We can do amazing things with [coding agents], but unlocking their full potential requires extraordinary discipline and knowledge.

Sending prompts one at a time to Claude Code to fix a bug or build a feature is only the tip of the iceberg.

Getting coding agents to integrate well into every part of the software engineering process is way more involved. Getting coding agents to help you not just ship features faster but to ship features better is not a given.

Coding agents, in my experience, take the path of least resistance when taking on a coding task. They aren't, by default, finding useful abstractions or proactively identifying architectural improvements or pruning excess, duplicative tests from the test suite, etc. That's where your own knowledge and discipline, understanding of the broader context, asking thoughtful questions, curating agent skills and workflows, and other human touches play a big role.

A dominant framing of AI-assisted coding and LLM agent harnesses is that they supposedly can empower you to ship, say, 10x more features.

Another way to look at the efficiency gains from these tools is that they allow you to focus more time and cycles and prototypes on getting features right for your customers.

Mitchell Hashimoto on how you can keep yourself and others on your team honest about the customer-facing code you are shipping, especially in a time where so much code is being generated by LLM coding harnesses.

The "whiteboard defense:" I should be able to pull you aside at any moment and ask you to explain any customer-facing system you've shipped. You should be able to clearly explain how it works and defend the decisions you made. This is my benchmark for responsible AI usage.

I don't expect line-level familiarity with the code. I don't care if you remember the exact function name or implementation detail. You may not even know it. I don't care.

But if I ask "why did you do X instead of Y?", "what happens if this actor behaves maliciously?", "what data structure did you use here and why?", or "where does this fail?" you should be able to answer confidently.

For PoCs, demos, experiments, whatever: I don't care. Generate 100% of it and understand none of it. Speed over quality every time in those specific scenarios.

But if you're shipping customer-facing work, you can't be shipping things you don't understand at a high level.

For really core decisions of features you've owned and shipped, yes, you should be able to know and defend your decisions. I do think this is a little too idealistic.

There are many times, pre- and post-November 2025 Inflection where after a few months and a lot of context-switching, those details start to get pretty fuzzy. "Why did we make X tradeoff?" or "Why did we decide to have Y happen in this one scenario?" Sometimes we chat for a minute and it all comes flooding back. Other times I realize I'm going to need to dig through the codebase or my notes to see if I can jog my memory. There are certainly times where a couple developers and stakeholders are chatting and none of us know the exact reason we went a certain direction. We end up having to just own it -- this is how the system works, what do we want to do going forward.

Claude models have been doing way too much yapping in the code comments in recent weeks. So I've been experimenting with a simple "Code Comments Conventions" section in the CLAUDE.md file.

"Only write code comments that cover critical and unintuitive behavior, preferring concision and clarity. A comment should have long-term durability. Skip the comment if it is only disclosing reasoning about the specifics of this change."

It must be working because after about a week of using this I ended up on a branch that didn't have these changes yet and was bowled over by the sudden verbosity of the comments.

The landscape of LLM and agentic tooling advancement is shifting too fast to think that any workflow I've figured out is going to be the most effective approach for too long.

Here is Justin Searls on this:

A skill we need to learn is how to ratchet up our ambitions until we uncover what the models truly cannot do. And each time they're improved, to once again test the fences. Confidence and certainty in one's own workflow have become trademark signs of ignorance and complacency.

This fits with my observation from this interview with developers at Anthropic:

good ideas about how to work well with LLMs have a short shelf life

Supervisory Engineering Work

We propose a new category of work we term supervisory engineering work, encompassing the direction, evaluation, and correction of AI output.

The productivity-experience paradox of working with AI-coding assistants:

We also identified a productivity-experience paradox: productivity perceptions held stable, with 84% reporting improvement at both time points, yet among matched participants, the proportion reporting worsened developer experience in at least one dimension nearly doubled from 14% to 27%, with flow state and cognitive load eroding while feedback loops improved. These findings suggest that AI coding assistants are impacting both the nature of software engineering work and how engineers experience it.

There have been studies on other aspects of LLM using in software as well as productivity for students, but not of software professionals.

Further, what remains largely missing is an understanding of how professional software engineers experience these tools, and whether those experiences change over time.

It strikes me that we need a classification system for the different forms of AI-assisted software development. These tools are used in widely non-uniform ways. From tab-completion for the current line or next couple lines to series of small prompts where the developer views and confirms each changeset to “auto mode” flows where an entire task or feature is completed by an agent harness to multi-loop agentic flows and beyond. Productivity gains and developer experience must vary widely across these types of AI development.

LLM tooling can tighten the feedback loop during development.

However, the usefulness of this feedback depends on its quality and relevance, and developers still report needing to validate AI output carefully.

Cognitive Fragmentation:

Overall, prior work indicates that AI coding assistants can improve aspects of developer experience while simultaneously introducing new sources of cognitive effort and fragmentation.

Concerns about maintainability had a big jump between the initial survey and subsequent survey. Which I interpret as the more these tools are used, the more maintenance becomes a growing concern.

Quality was the dominant primary concern at both time points, though it declined from 44% to 36%, followed by security. The most notable shift was in maintainability, which rose from 3% to 19% as a primary concern between Q1 and Q2.

Learning, onboarding, and code understanding are improved by LLMs and coding harnesses.

Sub-theme 2B: Accelerated learning, onboarding, and code understanding. Participants in both time points used AI to accelerate understanding across unfamiliar languages, frameworks, or legacy codebases. For example, one participant wrote: “Much easier to pick up new tools, libraries, languages and get productive.” (P132, Q1). Another wrote: “It has been particularly helpful in explaining legacy code and detecting bugs more efficiently.” (P2, Q2). Others reported gaining conceptual clarity: “Definitely felt like it’s given me a level up on understanding new concepts” (P5, Q2).

This supervisory work doesn’t really map onto existing tasks. It’s often compared, in my experience, to being a lead who is reviewing and giving feedback to others on the team, but I don’t think this is quite the same activity.

This supervisory work replaced portions of hands-on implementation effort but does not map cleanly onto traditional software development task categories. As one participant explained, “Sometimes, I’ll not be happy with the original reference, and will ’coach’ the AI to produce something that works better for my use-case”

Interesting, there is a “Use of Gen AI” disclaimer at the end of the paper.

We used Claude Code (Anthropic) to assist with developing R scripts for statistical analysis and reviewing written text for clarity and readability. All outputs were critically reviewed and validated, and responsibility for the correctness of the analysis and interpretations rests entirely with the researchers.

Links directly from Keith Kurson's comment on the linked post:

• The Impact of AI Coding Assistants on Software Engineering: A Longitudinal Study https://arxiv.org/abs/2605.23135
• Agent READMEs: An Empirical Study of Context Files for Agentic Coding https://arxiv.org/abs/2511.12884
• Usage, Effects and Requirements for AI Coding Assistants in the Enterprise: An Empirical Study https://arxiv.org/abs/2601.20112
• The State of AI-assisted Software Development report https://dora.dev/research/2025/dora-report/
• Professional Software Developers Don't Vibe, They Control: AI Agent Use for Coding in 2025 https://arxiv.org/abs/2512.14012
• Good Vibrations? A Qualitative Study of Co-Creation, Communication, Flow, and Trust in Vibe Coding https://arxiv.org/abs/2509.12491

With codegen no longer being the bottleneck, can the downstream parts of the software entering process keep up?

The metric is whether review, tests, and CI can absorb the new rate of change without letting bad diffs through.

A Claude Code plugin and a couple skills that route certain kinds of work like bulk file reads and boilerplate file creation through Portal which offloads the work to cheaper models like Gemini flash.

• https://agentclientprotocol.com/get-started/introduction
• Zed developed ACP https://zed.dev/acp

I like the sentiment of this post. The idea that what AI enables doesn’t have to only be focused on “lines of code” productivity. We can use AI tooling to take the on ambitious features and refactors that make our software better.

I was hoping it would go into more specific examples or discuss approaches for leveraging LLMs in that way.

Using /clear and @-mentioning files are the essential things I do from this list -- I run multiple Claude Code sessions everyday while I work and I never hit usage limits.

The other thing that makes a big difference is planning out my work and handing off well-scoped, verifiable feature descriptions to CC. This allows it to do targeted work and quickly check that it met its objective without a bunch of churn and re-work.

Something else I do fairly often is ask CC to build a phased (sometimes "Red/Green TDD") plan of how to implement the feature and to /grill-me with any questions. This obviously has some up-front token cost, but I usually find that we address ambiguities, contradictions, and missing requirements that way. The phased plan, especially with the TDD, allows CC to proceed in a more methodical and direct way to the finished feature. I think that general leads to lower token usage overall.

Complexity is anything related to the structure of a software system that makes it hard to understand and modify the system.
- John Ousterhout

• OpenCode - "OpenCode is an open source agent that helps you write code in your terminal, IDE, or desktop."
• Pi - "Pi is a minimal agent harness. Adapt Pi to your workflows, not the other way around."
• prime-agent - "self-improving coding harness designed around two abstractions, the Recursive Language Model (RLM) and Continual Harness"

Occasionally a colleague will respond to a question or a PR comment I left with, "Claude says ..." (or even just a link to their conversation). I find that pretty grating because:

1. I also could have asked Claude, but I wanted your (human) input or response.
2. This is not just low-effort, but puts the burden on me to interpret an LLM response.
3. Intentionally or not, it reads as "I'm not taking ownership of this."

By all means, prompt AI. But don't just relay the output. Read it, understand it, validate it, and then write a response in your own words (a decent certificate that you've done the prior steps). Making that effort is value you can add.

Otherwise, you're just being a "meat proxy".

I'm looking for some details on how harnesses edit code. There are high-level details in this post about code/file editing being part of the tool use architecture.

First, in "Tool Exposure":

Tool exposure: The harness declares the tools the model is allowed to call: reading files (readfile), editing code (replacestringinfile or applypatch), running terminal commands (runinterminal), searching the codebase (semanticsearch), and many more.

Then, in "Tool Execution":

Tool execution: When the model requests a tool to be run (using JSON like {"name": "runinterminal", "arguments": {"command": "npm test"}}), the harness is the one that validates the arguments, runs the tool, handles errors, formats the result, and feeds it back in the next iteration. For example, if the model asks to edit a file, the harness writes the diff. If the model asks to run a shell command, the harness is what spawns the process, captures output, and relays it.

Addy Osmani shares the idea to “let AI interview you”:

I’m about to start a project. Interview me until you have 95% confidence about what I actually want - not what I think I should want.

This is similar to Matt Pocock’s “Grill Me” Claude Code skill:

Interview the user relentlessly about a plan or design until reaching shared understanding, resolving each branch of the decision tree. Use when user wants to stress-test a plan, get grilled on their design, or mentions "grill me".

A Ruby on Rails–focused fork of obra/superpowers — a complete software development workflow for coding agents built on composable "skills".

I learned about this from Lucian Ghinda’s Linkedin post.

This is the November 2025 Inflection:

I would say in December is when it really just something flipped, where I kind of went from 80/20 of like, you know, to like 20/80 of writing code by myself versus just delegating to agents. And I don't even think it's 20/80 by now. I think it's a lot more than that. I don't think I've typed like a line of code probably since December, basically, which is like an extremely large change.

I've also included a larger excerpt from the interview because I think there are some other interesting comments alongside that one line. For instance, naming this "AI Psychosis" characterized by constantly thinking about working with more and more agents and keeping up with the latest capabilities in the AI space.

I kind of feel like I was in this perpetual, I still am often in this state of AI psychosis, just like all the time, because there was a huge unlock in what you can achieve as a person, as an individual, right? Because you were bottlenecked by, you know, your typing speed and so on. But now with these agents, it really, I would say in December is when it really just something flipped, where I kind of went from 80/20 of like, you know, to like 20/80 of writing code by myself versus just delegating to agents. And I don't even think it's 20/80 by now. I think it's a lot more than that. I don't think I've typed like a line of code probably since December, basically, which is like an extremely large change. I was talking to it, like, for example, I was talking about it too, for example, my parents and so on. And I don't think like a normal person actually realizes that this happened or how dramatic it was. Like literally, like if you just find a random software engineer or something like that at their desk and what they're doing, like their default workflow of, you know, building software is completely different as of basically December. So I'm just like in the state of psychosis of trying to figure out like what's possible, trying to push it to the limit. How is it? How can I have not just a single session of, you know, Claude Code or Codex or some of these agent harnesses? How can I have more of them? How can I do that appropriately? And then how can I use these Claws? What are these Claws? And so there's like a lot of new things. I want to be at the forefront of it, you know, and I'm very antsy that I'm not at the forefront of it. And I see lots of people on Twitter doing all kinds of things and they all sound like really good ideas. And I need to be at the forefront or I feel extremely nervous. And so I guess I'm just in the psychosis of like what's possible, like because it's unexplored fundamentally.

When a human writes code, the code is an artifact of their reasoning process. You can review the code and infer the thinking behind it. When AI generates code, you lose that direct connection. The code might be perfectly functional but reflect no coherent design intent – or worse, reflect a design intent that’s subtly different from what the developer actually wanted.

I've been in meetings where I've been asked to imagine a near-future in which members of the team who aren't currently producing code (e.g. project managers, C-suite, support team) are able to prompt an LLM blackbox with feature requests. Each of those requests will process in the background and eventually produce a preview environment for the prompter to look at and a PR to handoff to a software developer.

One of the assumptions baked in to the idea of this workflow is that there are all these well-defined, high-priority issues just sitting in the project management software waiting to be worked. Maybe that is the case in some orgs, however in my experience, the majority of teams and projects I've worked on don't have this. The backlog is a place where "nice-to-have" improvements and half-backed ideas collect dust and lose proximity to the state of the software system.

there’s a risk of AI side quests distracting from doubling down on the main one.

My hard-earned intuition for what works well with various LLM models and coding agents combined with my general expertise in software engineering combined with my knowledge of the specific codebase(s) are what allow me to get strong results from LLM tooling.

I suspect in a lot of cases I'd be tossing out the initial "engineer-out-of-the-loop" attempt and re-prompting from scratch, all while trying to keep tabs on the main quest work I have in progress.

Andrej Karpathy wrote a post recently that gets at why I think "asking an LLM to build that feature from the backlog" is not as straightforward as it seems:

It’s not perfect, it needs high-level direction, judgement, taste, oversight, iteration and hints and ideas. It works a lot better in some scenarios than others (e.g. especially for tasks that are well-specified and where you can verify/test functionality). The key is to build intuition to decompose the task just right to hand off the parts that work and help out around the edges.

I'd like to coin the term "AI-assisted Backlog Resurrection", but I don't think it's going to catch on.

Two things:
- Coding agents operating on your codebase are much more powerful than a disconnected LLM chat session where you try to manually give whatever context you think is important.
- Many of our decades old (and newer too) software engineering practices have directly useful benefits to AI-assisted coding.

Your codebase, way more than the prompt that you use, way more than your AGENTS.md file, is the biggest influence on AI's output.

Use DDD's Bounded Context or A Philosophy of Software Design's Deep Modules to create clear boxes around functionality with interfaces on top.

Design your codebase for progressive disclosure of complexity.

The math has shifted, in some cases, significantly.

Coding agents dramatically drop the cost of typing code into the computer, which disrupts so many of our existing personal and organizational intuitions about which trade-offs make sense.

That prototype, that bug fix, that nice-to-have -- those things that tend to get pushed off over and over because more urgent things are taking your time -- are suddenly viable in a lot of cases with an LLM coding agent. Equipped with a paragraph of detail and using something like Cursor or Claude Code, we can hand off these back-burner tasks, iterate on the idea in minutes instead of hours, and have at the very least an MVP if not a ready-to-ship implementation. All with a relatively minimal interruption to the main task at hand.

While we're trying to catch up to what is possible here, all the different processes our organizations and engineering teams have surrounding how we go from feature idea to shipped and supported implementation of said feature have some catching up to do as well. In some cases, these processes are meant to slow things down, as a way of managing risk. These processes are in opposition to the mandate to using AI agents to ship faster. What are organizations going to do with this contradiction? That's an open question.

I also like that as part of this series Simon is putting forth the term Agentic Engineering as the other side of the spectrum from vibe coding when it comes to using LLMs to write code.

I like hearing about other people's AI coding workflows, especially if they are demonstrating concrete success on a complex task. Porting a browser engine from C++ to Rust in two weeks with no regressions is impressive. This is a great example of human heavily in the loop -- using knowledge of the system to pick what parts of the codebase to port and when.

I used Claude Code and Codex for the translation. This was human-directed, not autonomous code generation. I decided what to port, in what order, and what the Rust code should look like. It was hundreds of small prompts, steering the agents where things needed to go. After the initial translation, I ran multiple passes of adversarial review, asking different models to analyze the code for mistakes and bad patterns.

CCC shows that AI systems can internalize the textbook knowledge of a field and apply it coherently at scale. AI can now reliably operate within established engineering practice. This is a genuine milestone that removes much of the drudgery of repetition and allows engineers to start closer to the state of the art.

Right now, where LLM coding agents can do there best work is in codebase where a solid foundation, clear conventions, and robust abstractions and patterns have all been well-established.

As writing code is becoming easier, designing software becomes more important than ever. As custom software becomes cheaper to create, the real challenge becomes choosing the right problems and managing the resulting complexity. I also see big open questions about who is going to maintain all this software.

Managing the complexity of large software systems has always been what makes software engineering challenging. While LLMs handle the “drudgery” of actually writing the code, are they simultaneously exacerbating the challenge of managing the complexity? If so, how do we, as “conductors” of coding agents, reign in that expanding complexity?

Something’s different this time, and I can say confidently this is the most unsure I’ve ever been about software’s future.

As Jerod puts it. It's not FOMO (fear of missing out) so much as it is FOBLB (fear of being left behind).

One fascinating part of our conversation with Steve Ruiz from tldraw started when he confessed that he feels bad going to bed without his Claudes working on something.

I think part of what is behind this is the same thing discussed in the HBR article, AI Doesn't Reduce Work--It Intensifies It, which points to this feeling that if we can do more we should be doing more.

Anthropic released the results of a recent study they conducted where they warn of the impact that LLM agent use for software development tasks can have on learning and skill development. This impacts everyone, but is crucial for earlier-career developers who haven't developed as many of these skills without LLMs in the picture.

None of this means we shouldn't be using LLMs for software development tasks, but rather we have to be intentional about how we use them.

Importantly, using AI assistance didn’t guarantee a lower score. How someone used AI influenced how much information they retained. The participants who showed stronger mastery used AI assistance not just to produce code but to build comprehension while doing so—whether by asking follow-up questions, requesting explanations, or posing conceptual questions while coding independently.

And here is another excerpt from the end of the article:

Our study can be viewed as a small piece of evidence toward the value of intentional skill development with AI tools. Cognitive effort—and even getting painfully stuck—is likely important for fostering mastery. This is also a lesson that applies to how individuals choose to work with AI, and which tools they use.

I ran a variation of the following prompt because the OP sounds pretty hyped about the results they are getting from it.

Do a git diff and pretend you're a senior dev doing a code review and you HATE this implementation. What would you criticize? What edge cases am I missing?

I expanded on it a little to guide it toward overly verbose or overengineered code. And I added some structure by asking it to include a confidence score with each item.

Based on the latest commit (see git show) and the untracked files that go along with it, pretend you're a senior dev doing a code review and you HATE this implementation. What would you criticize? What edge cases am I missing? What is overengineered, too verbose, or overcomplicated? Provide a confidence score with each issue and order your results with highest confidence issues at the top.

These instructions also work better with my workflow where I have in progress changes that I plan to amend into the most recent commit.

It picked up a couple things that I completely missed. It didn't find much in terms of refactoring away verbose or over-engineered code, unfortunately. I'm going to keep trying that though. This was quick to do and give me some actionable feedback. Worth the squeeze in my opinion.

To master these tools you need to learn how to get them to prove their changes work as well.

I’ve been using Claude Code to automate a series of dependency upgrades on a side project. I was surprised by the number of times it has confidently told me it fixed such-and-such issue and that it can now see that it works, when in fact it doesn’t work. I had to give it additional tests and tooling to verify the changes it was making had the intended effect.

My surprise at this was because the previous several upgrade steps it would deftly accomplish.

Almost anyone can prompt an LLM to generate a thousand-line patch and submit it for code review. That’s no longer valuable. What’s valuable is contributing code that is proven to work.

The step beyond this that I also view as table stakes is ensuring the code meets team standards, follows existing patterns and conventions, and doesn’t unreasonably accrue technical debt.

Put commands early: Put relevant executable commands in an early section: npm test, npm run build, pytest -v. Include flags and options, not just tool names. Your agent will reference these often.

I’ve seen big improvements in how agents do with my codebase when I include exactly what command and flags should be used for things like running the test suite.

Cover six core areas: Hitting these areas puts you in the top tier: commands, testing, project structure, code style, git workflow, and boundaries.

Covering these areas is a good starting point. I’m usually not sure where to start and only add things as needed.

Around March 14th, 2025:

"I think we will be there in three to six months, where AI is writing 90% of the code. And then, in 12 months, we may be in a world where AI is writing essentially all of the code," Amodei said at a Council of Foreign Relations event on Monday.

Amodei said software developers would still have a role to play in the near term. This is because humans will have to feed the AI models with design features and conditions, he said.

Via this Armin Ronacher article, via this David Crespo Bluesky post.

There’s no good way to say “allow access to everything on my computer, except for my password manager, my bank, my ~/.aws/credentials file, and the API keys I left in my environment variables”. Especially with Simon Willison’s lethal trifecta, you don’t really want to be giving access to these things, even if most of the time, nothing bad happens.

As Simon argues here, “prompt engineering” as a term didn’t really catch on. Perhaps “context engineering” better captures the task of providing just the right context to an LLM to get it to most effectively solve a problem.

It turns out that inferred definitions are the ones that stick. I think the inferred definition of "context engineering" is likely to be much closer to the intended meaning.

Justin describes some of ways that LLM coding agents used as pair programmers can lead to bad patterns in pair programming. In particular, one person taking over while the other person disengages or becomes a rubber stamp for their changes.

He then throws out some ideas for ways that LLM coding assistants can be improved with what we’ve learned over the years from person-to-person pairing.

My preference is what he describes here:

throttle down from the semi-autonomous "Agent" mode to the turn-based "Edit" or "Ask" modes. You'll go slower, and that's the point.

One suggestion is something we can do now (out-of-band or at least front-loaded).

Design agents to act with less self-confidence and more self-doubt. They should frequently stop to converse: validate why we're building this, solicit advice on the best approach, and express concern when we're going in the wrong direction

Tell the LLM you want to approach things this way, that you want to break the problem down and tease out any issues with the proposed approach. Engage in some back and forth, have it generate a plan, and then go from there.

Simon Willison on whether LLMs are going to replace software developers and what these tools mean for our careers:

I do think that AI-assisted development will drop the cost of producing custom software - because smaller teams will be able to achieve more than they could before. My strong intuition is that this will result in massively more demand for custom software - companies that never would have considered going custom in the past will now be able to afford exactly the software they need, and will hire people to build that for them. Pretty much the Jevons paradox applied to software development.

Just In Time manual reading

The world is full of manuals, how can I decide which ones I should read? One way is to read them all, which used to work back when I was younger, but now I have more responsibilities and I don't have time to read the entire Python library reference including all the useless bits I will never use. But here's Claude pointing out to me that this is something I need to know about, now, today, and I should read this one. That is valuable knowledge.

TL;DR I use coding assistants to build out a piece of code. Once I feel ok with the initial output, I switch back into normal coding mode. I refactor the code, carefully document it and introduce abstractions that capture common patterns. I also write good tests for the code. For testing, I use my “normal coding brain” to come up with the test cases but lean on AI code accelerators to speed up the process.

I like this term AI code accelerators. It suggests that it helps do what you’d already decided to do. It helps with the boilerplate or repetition for instance.

I’m seeing more and more of this genre of article where very experienced software engineer describes their AI coding workflow.

My advice about using AI is simple: use AI as an assistant, not an expert, and use it judiciously. Some people will object, “but AI can be wrong!” Yes, and so can the internet in general, but no one now recommends avoiding online resources because they can be wrong. They recommend taking it all with a grain of salt and being careful. That’s what you should do with AI help as well.

Skeptics of the uses of LLMs typically point to strawman arguments and gotchas to wholesale discredit LLMs*. They're either clinging too tightly to their bias against these tools or completely missing the point. These tools are immensely useful. They aren't magic boxes though, despite what the hypemen might want you to believe. If you take a little extra effort to use them well, they are a versatile swiss army knife to add to your software tool belt."Use [these LLMs] as an assistant, not an expert."

*There are soooo many criticisms and concerns we can and should raise about LLMs. Let's have those conversations. But if all you're doing is saying, "Look, I asked it an obvious question that I know the answer to and it got it WRONG! Ha, see, this AI stuff is a joke." The only thing you're adding to the dialogue is a concern for your critical thinking skills.

If you approach AI thinking that it will hallucinate and be wrong, and then discard it as soon as it does, you are falling victim to confirmation bias. Yes, AI will be wrong sometimes. That doesn’t mean it is useless. It means you have to use it carefully.

Use it carefully, and use it for what it is good at.

For instance, it can scaffold a 98% solution to a bash script that does something really useful for me. Something that I might have taken an hour to write myself or something I would have spent just as much time doing manually.

Another instance, I'm about to write a couple lines of code to do X. I've done something like X before, I have an idea of how to approach it. A habit I've developed that enriches my programming life is to prompt Claude or Cursor for a couple approaches to X. I see how those compare to what I was about to do.

There are typically a few valuable things I get from doing this:

1. The effort of putting thoughts into words to make the prompt clarifies my thinking about the problem itself. Maybe I notice an aspect of it I hadn't before. Maybe I have a few sentences that I can repurpose as part of a PR description later.
2. The LLM suggests approaches I hadn't considered. Maybe it suggests a command, function, or language feature I don't know much about. I go look those things up and learn something I wouldn't have otherwise encountered.
3. The LLM often draws my attention to edge cases and other considerations I hadn't thought of when initially thinking through a solution. This leads me to develop more complete and robust solutions.

I’ve used AI to help me write code when I didn’t know how to get started because it needed more research than I could afford at the moment. The AI didn’t produce finished code, but it got me going in the right direction, and iterating with it got me to working code.

Sometimes you're staring at a blank page. The LLM can be the first one to toss out an idea to get things moving. It can get you to that first prototype that you throw away once you've wrapped your head around the problem space.

Your workflow probably has steps where AI can help you. It’s not a magic bullet, it’s a tool that you have to learn how to use.

This reiterates points I made above. Approach LLMs with an open and critical mind, give it a chance to see where they can fit into your workflow.

Software engineering is programming integrated over time. The integrated over time part is crucial. It highlights that software engineering isn't simply writing a functioning program but building a system that successfully serves the needs, can scale to the demand, and is able to evolve over its complete lifespan.

LLMs generate point-in-time code artifacts. This is only a small part of engineering software systems over time.

Vibe Coding as a practice is here to stay. It works, and it solves real-world problems – getting you from zero to a working prototype in hours. Yet, at the moment, it isn’t suitable for building production-grade software.

LLMs are no substitute for the hard-won expertise of years of building software, working within software teams, and evolving systems. You can squeeze the most out of iterations with a coding LLM by bringing that experience to every step of the conversation.

In other words, they're applying years of hard-won engineering wisdom to shape and constrain the AI's output. The AI is accelerating their implementation, but their expertise is what keeps the code maintainable.

The 70% problem

A tweet that recently caught my eye perfectly captures what I've been observing in the field: Non-engineers using AI for coding find themselves hitting a frustrating wall. They can get 70% of the way there surprisingly quickly, but that final 30% becomes an exercise in diminishing returns.

Addy goes on to describe this "two steps back pattern" where a developer using an LLM encounters an error, they ask the LLM to suggest a fix, the fix sorta works but two other issues crop up, and repeat.

This cycle is particularly painful for non-engineers because they lack the mental models to understand what's actually going wrong. When an experienced developer encounters a bug, they can reason about potential causes and solutions based on years of pattern recognition.

Beyond having the general programming and debugging experience to expedite this cycle, there is also an LLM intuition to be developed. I remember John Lindquist describing that he notices certain "smells" when working with LLMs. For instance, often when you're a couple steps into a debugging cycle with an LLM and it starts wanting to go make changes to config files, that is a smell. It's a "smell" because it should catch your attention and scrutiny. A lot of times this means the LLM is way off course and it is now throwing generative spaghetti at the wall. I learned two useful things from John through this:

1. You have to spend a lot of time using different models and LLM tools to build up your intuition for these "smells".
2. When you notice one of these smells, it's likely that the LLM doesn't have enough or the right context. Abort the conversation, refine the context and prompt, and try again. Or feed what you've tried into another model (perhaps a more powerful reasoning one) and see where that gets you.

Being able to do any of that generally hinges on having already spent many, many years debugging software and having already developed some intuitions for what is a good next step and what is likely heading toward a dead end.

These LLM tools have shown to be super impressive at specific tasks, so it is tempting to generalize their utility to all of software engineering. However, at least for now, we should recognize the specific things they are good at and use them for that:

This "70% problem" suggests that current AI coding tools are best viewed as:
- Prototyping accelerators for experienced developers
- Learning aids for those committed to understanding development
- MVP generators for validating ideas quickly

I'd at to this list:

• Apply context-aware boilerplate autocomplete — establish a pattern in a file/codebase or rely on existing library conventions and a tool like Cursor will often suggest an autocompletion that saves a bunch of tedious typing.
• Scaffold narrow feature slices in a high-convention framework or library — Rails codebases are a great example of this where the ecosystem has developed strong conventions that span files and directories. The LLM can generate 90% of what is needed, following those conventions. By providing specific rules about how you develop in that ecosystem and a tightly defined feature prompt, the LLM will produce a small diff of changes that you can quickly assess and test for correctness. To me this is distinct from the prototyping item suggested by Addy because it is a pattern for working in an existing codebase.