1. Awe, Promise, and the Quiet Doubt
I never planned to dive into AI. It wasn’t a strategic career move or a response to market signals. It was curiosity of the kind that doesn’t leave you alone.
Large Language Models were the trigger. When GPT-3 and its successors began generating coherent text, writing code, holding conversations, something shifted. Not just “this is impressive,” but “I need to understand what’s happening here.” That’s where it started.
But I didn’t want to just use LLMs. I wanted to understand them. Surely not in every microscopic detail, but in principle. What patterns do they capture? How do they simulate reasoning? Where are the boundaries between prediction and understanding?
To get there, I decided not to start with transformers (I think this would be impossible), but with the foundations. Over the course of three years, I worked through the core disciplines:
- Classical machine learning with scikit-learn,
- Deep learning and neural networks,
- Computer vision, reinforcement learning, and generative models,
- As well as symbolic and rule-based approaches: the forms of AI that existed long before language models dominated the conversation.
It was demanding, but it reshaped how I think about intelligence in machines. It also gave me a broader view of AI than what most current narratives center around. Not better, just different, and in many ways, deeper.
With this perspective, I returned to LLMs. But now the questions had changed. I wasn’t asking “What can this tool do?” but “What assumptions is it making? What does it not know? What limitations are structural, not just technical?”
At the same time, I began to feel increasingly restless in my role as a software engineer. My work with enterprise systems, digitalizing business processes using .NET and Azure, was technically solid, but intellectually flat. I wasn’t solving problems that required learning or pattern recognition. I was orchestrating logic. Clean, but static.
So I tried to shift. My goal was to enter the applied AI space: the kind grounded in signal, measurement, and constraint. Computer vision, anomaly detection, recommendation systems. Areas where AI isn’t a buzzword, but a tool with clear success criteria.
But there were no opportunities. At least not for someone trying to transition into the field. And then, like many others, I heard the same advice repeated in various forms:
“Look into LLMs. That’s where everything is happening.”
So I did. I explored, built prototypes, studied the architecture of modern agents, learned how retrieval works, and tried to create end-to-end solutions. And what I found was not what I expected.
Instead of a pathway to real-world impact, I encountered layers of abstraction, tools built on tools, and promises that often failed quiet tests of reliability.
I found awe. But I also found illusion.
And behind the surface, a quiet doubt began to grow.
“If these models are so brilliant, why do so many projects still fall apart in the real world?”
2. AI ≠ LLMs: A Misleading Equation
Somewhere between 2022 and 2024, the term “AI” was quietly redefined in the public mind. Not by researchers, but by headlines, pitch decks, and viral demos. What once referred to a wide spectrum of methods, from symbolic logic to reinforcement learning, was now almost exclusively associated with Large Language Models.
AI became GPT. AI became chat.
But the truth is: AI was here long before that. It lived in search algorithms, optimization techniques, game-playing agents, decision trees, and expert systems. It existed in autonomous vehicles and factory robotics. It powered recommendation engines, fraud detectors, and medical image analysis.
LLMs are a recent and truly impressive development within the larger AI field. But they are not the field itself. They are, quite literally, language models. They learn statistical patterns in sequences of text. Their primary function is prediction: given a context, what is the most likely next word, token, or output?
Yes, the results often feel intelligent. Sometimes, even unsettlingly so. But this is not intelligence in the broad sense. It’s linguistic fluency shaped by vast data and dense networks. It doesn’t involve goals, world models, or genuine understanding.
And yet, the confusion is understandable. LLMs are the first form of AI most people have ever interacted with directly. They talk back. They explain. They write code. They respond in human terms. That makes them feel more real than the quiet algorithms running in the background of every logistics system or recommendation engine.
This shift in perception isn’t just semantic. It has consequences. When we mistake LLMs for general-purpose AI, we begin to expect the wrong things. We assume reasoning where there is only association. We expect autonomy where there is only looping logic. We build workflows and products on top of these assumptions and then wonder why they break.
This isn’t a call to downplay LLMs. On the contrary, their capabilities are extraordinary. But to treat them as the entirety of AI is like mistaking language for thought itself. They intersect, but they are not the same.
Understanding this difference was essential for what came next. Because when I tried to apply LLMs to a real-world use case: one that involved structure, knowledge, and action, I ran headfirst into the limits.
And that’s where the Smart Navigator story begins.
3. The Smart Navigator: Where Promise Met Practice
After months of study and prototyping, I was ready to build something concrete. Something useful. Not another chatbot, not another demo. A tool grounded in a real use case. I called it Smart Navigator.
The idea was simple in principle. Take a dense, technical knowledge base (in this case, aviation maintenance manuals from the FAA) and make that knowledge accessible through natural language. Let mechanics, engineers, or trainees ask questions in plain English and receive accurate, helpful responses. No searching across dozens of PDFs, no hunting through indexes. Just answers.
It was the kind of problem that LLM-based systems seemed made for. The stack was familiar: document parsing, text chunking, embedding, vector search, retrieval-augmented generation. I added tool use, memory, and response formatting. I tuned the prompts and structured the outputs.
At first, the results were encouraging. The system could locate sections across multiple documents, synthesize fragments, and return well-written answers. It handled synonyms, paraphrased questions, and even some domain-specific jargon.
But then the cracks appeared.
The underlying documents of the manual were rich in structure: tables, diagrams, references, conditional logic. These were not blog posts or Wikipedia entries. They were technical documents written with precision, often referring to other sections, parts, or procedures. And here, the LLM began to struggle.
Parsing failed silently in edge cases. Important details were lost in formatting. References became untraceable. The summaries started drifting. Not wildly, but just enough to make a mechanic hesitate.
And that hesitation matters. In regulated fields like aviation, trust is not optional. Precision is not a nice-to-have. An answer that is mostly correct is still wrong.
Worse still, the system had no real sense of what it didn’t know. It could hallucinate plausible answers when retrieval failed, and do so with confidence. It couldn’t verify, it couldn’t trace back reliably, and it couldn’t flag uncertainty in a meaningful way.
I found myself manually curating answers, rewriting prompts, and checking outputs line by line. The very thing the tool was supposed to automate became dependent on human oversight and this not only occasionally, but structurally.
This was the moment it clicked: I wasn’t building a navigator. I was building a mirage. The illusion of understanding, backed by a system that could simulate insight without grounding in fact.
The problem wasn’t my implementation. It was the architecture itself. LLMs can simulate reasoning, but they don’t reason. They can use tools, but they don’t understand what the tools do. They can summarize, but not verify.
And yet, everywhere I looked, similar systems were being marketed as solutions. Knowledge assistants. Autonomous agents. AI copilots that claimed to replace tasks without acknowledging the oversight required to keep them safe and useful.
I realized that Smart Navigator didn’t fail as a project. It succeeded in showing me where the current limits are. And it pointed to a deeper issue that is not just technical, but economic.
Because the problem wasn’t only in the model. It was in the story being told around it.
“LLMs can imitate understanding. But they don’t verify it. And I still had to.”
4. Why the System Wants More Than the Tech Can Give
After working on Smart Navigator, I began to notice a pattern. It wasn’t just that LLMs had limitations. That was expected. What stood out was how consistently those limitations were ignored, downplayed, or glossed over and that especially when money was involved.
Across industries, the appetite for LLM-based solutions was growing rapidly. Internal meetings, startup pitches, keynote talks. All spoke of autonomy, efficiency, disruption. Words like “copilot,” “agent,” and “workflow automation” appeared everywhere. The promise was not just better tools, but fewer humans.
That is the real engine behind the hype: not curiosity, but cost-cutting. LLMs are being positioned as a lever to eliminate labor and maximize margin. It’s not about augmenting human judgment, but replacing it. And this even when the models clearly aren’t ready.
This isn’t a fringe phenomenon. It’s systemic. Business leaders, under pressure to innovate, are being sold the vision of near-instant productivity gains. Replace your analysts with chatbots. Let an AI agent file reports, schedule tasks, interpret logs, summarize legal briefs. No mention of hallucination rates, data privacy risks, or domain specificity. No mention of oversight cost. Just automation, at scale.
But here’s the uncomfortable truth: the ROI of LLM-based systems is still largely unproven.
Training your own model? Economically infeasible for most.
Hosting open models? Costly in infrastructure and expertise.
Using APIs? Priced by token, unpredictable at scale, and locked into black-box providers.
Building agents? Brittle chains of function calls that often break under real-world complexity.
Even when systems work technically, they often fail economically.
You end up with tools that require constant human supervision, frequent re-prompting, and expensive retries without delivering reliable savings. And yet, teams keep building, not because they’ve run the numbers, but because the narrative is too attractive to question.
The irony is that many of these efforts quietly shift work back to the humans. Users end up doing the verification, catching hallucinations, rewording queries, validating summaries. We are not removing friction. We are displacing it, and pretending it’s innovation.
I don’t blame the engineers. I don’t even blame the models. The pressure comes from above. From a system that wants more than the tech can give. It demands clean margins, scalable automation, and fast returns. LLMs, dressed up as agents, are the next shiny solution.
But a prediction engine pretending to be an autonomous worker is still a prediction engine. And calling it “agentic” doesn’t change that.
“The models are brilliant. The story told about them is what’s broken.”
5. The Future of Work Isn’t Autonomous. It’s Augmented.
There’s a kind of desperation running through the current AI discourse. A race not just to innovate, but to eliminate. To cut the human out of the loop. Not because the technology is ready, but because the profit margins demand it.
We are watching a global effort to reduce labor to a line item. LLMs and so-called “agentic AI” have been cast in the role of replacement technology, whether they can handle it or not. And for now, they cannot.
Not because they aren’t impressive. They definitively are. But because they lack the grounding, memory, and judgment that real work requires. They simulate competence, but they don’t possess it. They output solutions, but they don’t know if they’ve solved anything.
And still, the drive continues. Replace support teams with bots. Replace analysts with dashboards. Replace writers, developers, researchers. The goal isn’t augmentation. It’s subtraction.
There’s something tragic about that.
Because what LLMs have already given us is extraordinary.
They allow us to think faster. To express better. To explore ideas more freely.
They’re not a threat to human productivity. They’re an invitation to expand it.
When used well, they act as intelligent mirrors. They help clarify what we already know, and sometimes reveal patterns we hadn’t noticed. They can co-write, co-reason, co-translate. They are tools for amplification, not erasure.
The real promise of this technology is not that it makes us obsolete. It’s that it can make us more powerful: intellectually, creatively, operationally. But only if we let it support us instead of trying to replace us.
Yes, it’s possible that one day these systems will reach a level where full automation becomes viable. Perhaps they will reason, plan, verify, and act with the nuance required for independence. But we are not there. Not even close.
LLMs and agents, as they exist today, are in their infancy. Fragile. Incomplete. Fascinating, but not yet trustworthy on their own.
And so we stand at a fork in the road.
One path leads to hollow automation. Systems that pretend to think, while quietly shifting the burden of correctness onto the user.
The other path leads to augmentation, to tools that extend our reach, enhance our judgment, and preserve what is uniquely human in the loop.
The second path is not only more realistic. It’s also more dignified.
We should take it.
“Not man versus machine: but man with machine, navigating complexity together.”
Interlude: Hype Has a Pattern
We’ve seen this before.
A new technology arrives. It sparks genuine excitement. Early adopters build incredible things. But then something shifts. The focus moves from exploration to extraction. From capability to narrative. From what it is to what it can be sold as.
Enter the CEOs, the CTOs, the investors. And suddenly, the technology isn’t just promising. No it is nothing less than revolutionary. It’s a new era. A turning point. A moment in history. Slide decks multiply. Markets adjust. Entire departments are told to “adopt AI” before anyone even asks what for.
And the public? It follows. Not because people are naïve, but because the story is persuasive. No one wants to miss the next wave. No one wants to be left behind. Everyone feels the pressure to jump on board. Because just in case, this time the hype is maybe real.
But beneath the surface, the pattern repeats.
We saw it in the dot-com bubble. We saw it in crypto. We saw it in the housing market before 2008. Grand narratives fueled by money, speed, and the promise of transformation. Until the assumptions break, the numbers don’t add up, and reality reasserts itself.
AI is now riding the same curve.
The difference is that this time, the stakes feel higher. Because what’s being promised isn’t just a shift in infrastructure or finance. It’s the replacement of human labor itself. The automation of cognition. The phasing out of people.
And that makes the story even harder to resist, and more dangerous to believe uncritically.
The models are improving. No doubt about it. But the narrative is running ahead of the facts. Again.
The question isn’t whether this hype will deflate. It’s when, and how much damage will be done before it does.
6. Let’s Stop Chasing Illusions
There is something seductive about the idea of full automation.
It promises clarity, efficiency, scale. It suggests that the messy, unpredictable nature of human work can be replaced by systems that run on pure logic and language. That we can remove friction without losing fidelity. That we can offload our effort and still keep the outcome.
But that vision, as it stands today, is an illusion.
Most current LLM systems are not autonomous. They are guided sequences. Fragile chains of prompts, tools, and memory hacks. They simulate planning, but do not plan. They speak fluently, but do not know. They act with structure, but without awareness.
And yet, we keep pushing them into roles they cannot fulfill.
We frame them as copilots, but expect them to fly the plane alone.
We design them as assistants, but treat them as replacements.
And when they fail, we call it a use-case mismatch instead of what it often is: a design built on wishful thinking.
This is not a critique of the technology. It is a call for a different posture toward it.
We can acknowledge the brilliance of these systems without pretending they are more than they are. We can integrate them where they make sense, and avoid pretending they are ready for full autonomy when they clearly are not.
Let’s stop imagining that intelligence can be abstracted away from context, from responsibility, from judgment. Let’s stop acting as if the cost of error is always someone else’s problem. Let’s stop building systems that look right on the surface, but depend on invisible labor underneath.
Because the longer we chase the illusion of autonomous AI, the longer we delay building tools that actually help people. The longer we pretend that cognition can be outsourced, the more we devalue the kind of work that matters most. This is the work that still requires thought, care, and presence.
There is no shame in a tool that needs a human. There is only shame in pretending otherwise.
7. Reclaiming the Real Magic
When I first started working with LLMs, I was amazed.
Not just by what they could do, but by the possibilities they opened up. Writing, summarizing, coding, explaining things in ways that actually made sense. It felt like something important had shifted.
And even now, after months of building, testing, and running into limitations, I still think they’re incredible tools. They’ve helped me think more clearly, work faster, and explore ideas that would have taken much longer without them.
But over time, I also came to see something else.
The real value of these models isn’t that they replace us. It’s that they support us. They extend our thinking. They help with structure, with momentum, with clarity. They’re not decision-makers or autonomous workers. They’re amplifiers. Helpers.
That should be enough.
But instead, the bigger story being told is all about automation. Removing people. Cutting labor. Replacing jobs. And I think that’s a mistake. Not just because the tech isn’t ready, but because it misses the point.
We’ve been handed something powerful, and instead of asking how it can help people do better work, we’re asking how to get rid of the people. That’s not just shortsighted, it’s wasteful.
A better future is possible. One where LLMs are used to make human work more meaningful, not less. Where they take care of the repetitive stuff, and leave the thinking and judgment to us. Where they make hard tasks easier, and help us grow into more capable versions of ourselves.
That’s the direction I want to keep moving in.
And I hope others will too.
“I don’t want less from this technology. I want honesty about what it is and space to build what it can truly become.”
Have you tried building with LLMs and hit a wall?
I’d love to hear your reflections but not on the demos, but on the real work.”
Leave a Reply
You must be logged in to post a comment.