Ethan Mollick (1 blogmarks)

AI is weird, there is no instruction manual

AI is weird. No one actually knows the full range of capabilities of the most advanced Large Language Models, like GPT-4. No one really knows the best ways to use them, or the conditions under which they fail. There is no instruction manual. On some tasks AI is immensely powerful, and on others it fails completely or subtly. And, unless you use AI a lot, you won’t know which is which.

Here are observations of lower-performers going from no-AI to using AI:

We also found something else interesting, an effect that is increasingly apparent in other studies of AI: it works as a skill leveler. The consultants who scored the worst when we assessed them at the start of the experiment had the biggest jump in their performance, 43%, when they got to use AI. The top consultants still got a boost, but less of one.

I’m not sure “skill leveler” is the right term here. Something like “speed-booster” or “efficiency enhancer” seems more apt to me. When I think about programmers using LLMs and agent harnesses, they aren’t usually writing a new level of code, but rather producing a similar quality of code at much faster speeds.

This paragraph from Mollick’s book “Co-Intelligence” sums up these concepts well:

Using Al as a co-intelligence, as I did while writing is where Al is the most valuable. Figure out a way to do this yourself if you can. As a starting point, follow the first principle (invite AI to everything) until you start to learn the shape of the Jagged Frontier in your work. This will let you know what the Al can do and what it can't. Then start working like a Centaur. Give the tasks that you hate but can easily check (like writing meaningless reports or low-priority emails) to the AI and see whether it improves your life. You will likely start to transition naturally into Cyborg usage, as you find the Al indispensable in overcoming small barriers and helping with tricky tasks. At that point, you have found a co-intelligence.