In traditional software engineering, "truth" is binary. A variable is either true or false. A database query returns a record or it doesn't. But as I transitioned from Industrial Engineering to building AI products like Illari, I realized that in the world of Large Language Models (LLMs), truth is not binary—it is probabilistic.
This shift isn't just an engineering challenge; it is an epistemological one.
Currently, I inhabit two distinct worlds. By day, I am a Technical Product Manager orchestrating RAG (Retrieval-Augmented Generation) systems. By night, I am a Philosophy undergraduate at the University of Navarra. While these paths seem divergent, they converge on the single most critical problem in AI development today: Steerability.
How do we force a probabilistic system to adhere to deterministic constraints? How do we align "model belief" with "ground truth"? Here is what shipping a production RAG system taught me about the epistemology of AI agents.
1. RAG is an Epistemological Architecture
When we build a RAG system, we are essentially trying to solve the "Justified True Belief" problem for a machine. A base model has beliefs (weights trained on the internet), but it lacks justification for specific, private contexts.
With Illari, my goal was to ground the model in a corpus of 1,300+ proprietary documents. We chose a stack of LangChain, Qdrant, and Postgres to act as the "knowledge layer." But I quickly learned that retrieving context isn't enough; the model must know how to use it.
The Hallucination Trap: Early on, the model would confidently answer questions based on its pre-training rather than the retrieved context.
The Fix: We had to engineer "epistemic humility" into the system prompts and temperature settings, effectively teaching the model to say "I don't know" when the retrieved vectors didn't contain the answer.
For a Product Manager, this redefines the roadmap. We aren't just shipping features; we are shipping verifiability.
2. Latency as the Friction of Thought
In philosophy, we study the immediacy of intuition. In AI, we study Time-to-First-Token (TTFT).
Intelligence that takes 10 seconds to respond feels like incompetence. With Illari, we faced a massive trade-off between the depth of reasoning (context window size) and the speed of response. Through rigorous optimization of our vector search and embedding strategies, we achieved a ~1.2s latency.
Why does this matter for Steerability? Because if an agent is too slow, users will bypass it. They will revert to unsafe, unmanaged workflows. Speed is not just a UX metric; it is a safety feature. It ensures users stay within the "steered" environment we have built.
3. Ethics Must Be Hard-Coded (RBAC & NIST)
Anthropic’s research on "Constitutional AI" resonates with me because it treats safety as an inherent property of the system, not an afterthought. I applied this same principle to Illari.
We didn't rely on the model's "good will" to protect data. We implemented Role-Based Access Control (RBAC) at the vector store level. If a user doesn't have the "Manager" role, the vectors containing sensitive financial data simply do not exist for the model. Furthermore, we aligned our risk matrix with NIST AI RMF and EU AI Act principles. We added PII scrubbing layers before the prompt ever reached the LLM.
True steerability means the model cannot misbehave because the architecture precludes it. It is the difference between asking a child not to touch a hot stove and turning the stove off.
4. The Future: Agentic Coding & Claude Code
As we move toward agents that can write and execute code—like Claude Code—the stakes get higher. An agent that hallucinates a sentence is annoying; an agent that hallucinates a DROP TABLE command is catastrophic.
My experience building autonomous agents with n8n and Cloud Functions (which achieved a 90% auto-resolution rate) showed me that the future of development tools lies in interpretable action. We need interfaces that allow developers to "audit the thought process" of the AI before it executes.
Conclusion
I build to understand the machine. I study philosophy to understand the intent. The next generation of AI products won't be defined by raw power, but by how well we can steer that power toward human-aligned goals.
That is the product I want to build.
By Diego Andres Salazar