We Thought We Were Programming AI — We Were Just Talking to It
Why prompt engineering worked… until it didn’t
Photo by Rafiee Artist on Unsplash
There was a time when it genuinely felt like we had discovered a new kind of programming.
You didn’t write functions. You wrote prompts. And somehow, that was enough.
You could take a vague instruction, refine it slightly, add a few examples, and watch the output transform in ways that felt almost deterministic. A better prompt led to a better result. A clearer instruction reduced errors. If something went wrong, you didn’t debug the system — you rewrote the sentence.
It felt intuitive, almost deceptively so. Language became the interface, and for the first time, programming didn’t look like code. It looked like conversation.
For a while, that illusion held remarkably well.
The moment control started slipping
The shift didn’t happen all at once. It crept in slowly, almost invisibly.
A prompt that worked perfectly yesterday would behave slightly differently today. The system handled familiar cases well, but struggled with variations you hadn’t explicitly covered. You would adjust the wording, add more constraints, maybe include another example — and it would improve, but never quite stabilize.
At first, this felt like a skill issue. Maybe the prompt wasn’t precise enough. Maybe the instruction could be clearer. And so the natural response was to push further in the same direction: more detail, more structure, more control through language.
But something subtle was changing underneath.
The more you tried to tighten control through prompts, the more fragile the system became. Fixing one case would introduce another failure somewhere else. The prompt grew longer, more specific, more defensive — yet the system never became truly reliable.
It was still responsive. It was still powerful. But it was no longer predictable.
And that was the first real signal that something deeper was off.
The part we weren’t looking at
What made prompt engineering feel like programming was feedback. You changed the input, and the output changed accordingly. That feedback loop gave the impression of control.
But that control was always partial.
Because the model was never responding to the prompt alone.
It was responding to the prompt plus everything else it was given at that moment. And as systems became more complex, that “everything else” started to matter more than the prompt itself.
A slight change in retrieved information could shift the entire answer. Missing a single piece of context could invalidate an otherwise correct reasoning chain. Two identical prompts could produce different outputs simply because the underlying data was different.
The prompt didn’t define the system.
It sat on top of it.
When language stopped being enough
This became impossible to ignore once AI systems moved beyond simple interactions.
The moment you asked a system to do more than answer a single question — to retrieve data, to reason across steps, to interact with tools — you were no longer dealing with a prompt-driven setup. You were dealing with a process.
And processes need structure.
They need a way to decide what information enters at each step, how that information is transformed, and how it flows through the system over time. Language alone is not precise enough to handle that.
You can tell a model to “use the most relevant data,” but you cannot guarantee what it will consider relevant. You can instruct it to “be accurate,” but you cannot ensure it has access to the right facts. You can ask it to “think step by step,” but you cannot control what each step is based on.
At some point, you stop trying to control behavior through instructions.
You start controlling what the system sees.
The quiet realization
This is where the real shift happened, though most people didn’t name it at the time.
The quality of the system wasn’t improving because of better prompts.
It was improving when the right information was present, and failing when it wasn’t.
That realization changes how you look at everything.
You stop asking whether the prompt is well-written. You start asking whether the system is providing the model with the right view of reality. You begin to notice that many so-called “hallucinations” are actually consistent responses to inconsistent context.
The model isn’t making things up.
It is doing its best with what it has.
Prompting as an interface, not a foundation
Looking back, prompt engineering was never the system itself. It was the first interface we had to interact with something we didn’t fully understand yet.
It gave us a way to guide behavior without building infrastructure. It allowed rapid experimentation, quick iteration, and a sense of progress that felt immediate and tangible.
But it also masked where the real complexity lived.
Language made it easy to believe we were in control, even when we weren’t. It blurred the boundary between intention and execution. And for simple use cases, that was enough.
For anything more complex, it wasn’t.
Why it had to evolve
As soon as systems required memory, multi-step reasoning, or interaction with external data, the limitations of prompting became structural.
You cannot manage evolving state through static instructions. You cannot guarantee consistency when the system’s inputs change in ways the prompt cannot account for. You cannot build reliable behavior on top of a layer that is inherently soft and interpretive.
So the center of gravity moved.
Not away from models, and not away from intelligence, but away from language as the primary control mechanism.
The system needed something more concrete.
It needed a way to decide, consistently and explicitly, what information should shape each decision.
Reframing the beginning
In hindsight, 2024 was not really the era of prompt engineering.
It was the era before we had better abstractions.
We were using language because it was the only tool available. It worked just well enough to give us momentum, but not well enough to scale into reliable systems.
What we experienced as progress was real. But what we thought we were building was slightly off.
We believed we were programming intelligence.
In reality, we were just learning how to talk to it.
Closing
The story of AI systems over the past few years is often told as a progression from simple to complex.
But there is another way to see it.
We started at the surface, with prompts, because that was the only layer we could touch. As systems grew, we were forced to look deeper — into context, into execution, into coordination.
Each step didn’t add complexity.
It revealed it.
And once you see that clearly, prompt engineering stops looking like the foundation of AI systems.
It becomes what it always was.
The first conversation we had before we understood what we were actually building.