AI makes it easier to look finished than to be finished

Working with frontier AI capabilities on new products and even on this website, I’ve been struck by how quickly an idea can become something that looks objectively ready to launch. There’s a real excitement in seeing it take shape, especially when it would once have taken weeks.

I think that speed can also make it harder to see what’s missing.

The journey from an idea to a working product used to expose questions along the way. You made decisions, encountered difficulties and gradually built an understanding of how the thing worked. When much of that journey is compressed, a convincing result can arrive before that understanding has caught up.

Experience helps, but I don’t think it makes anyone immune. Even after years of building products before these tools existed, I can get caught up in the magic of seeing something come to life.

AI is also opening up product building to people who haven’t worked this way before. That’s exciting. Someone can now bring an idea to life without having encountered the discipline of defining what they’re trying to learn or deciding what would count as evidence. They may be learning how to build, how to test and how to interpret what happens all at once.

The gaps become more difficult to untangle once an end user starts using it. If they struggle, is the problem the interface, the implementation, or an assumption about what they needed? Without a clear hypothesis, it can be hard to know what their experience is telling you.

AI lets us build and test sooner. That’s valuable. The hypothesis we’re trying to evaluate still requires thinking.

What are we expecting this version to help someone do, and what would we need to observe to know whether it works?