<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Gabriel Ferrari | Pragmatic AI Engineering]]></title><description><![CDATA[Gabriel Ferrari | Pragmatic AI Engineering]]></description><link>https://gabrielferrari.hashnode.dev</link><generator>RSS for Node</generator><lastBuildDate>Tue, 03 Nov 2026 19:28:03 GMT</lastBuildDate><atom:link href="https://gabrielferrari.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Code Review for AI-Generated Changes Needs Better Triage]]></title><description><![CDATA[AI can generate a patch faster than a developer can understand its consequences. That changes the work, but it does not remove the need to review the code.]]></description><link>https://gabrielferrari.hashnode.dev/code-review-for-ai-generated-changes-needs-better-triage</link><guid isPermaLink="true">https://gabrielferrari.hashnode.dev/code-review-for-ai-generated-changes-needs-better-triage</guid><dc:creator><![CDATA[Gabriel Ferrari]]></dc:creator><pubDate>Sat, 03 Oct 2026 21:53:06 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6aa8aa00ab9d3c9320651c23/2b9ebc45-c34b-4397-a17a-60753e59751e.jpg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>AI can generate a patch faster than a developer can understand its consequences. That changes the work, but it does not remove the need to review the code.</p>
<p>The problem is especially familiar in solo projects: I may guide an implementation and then be the person responsible for reviewing it. Knowing what I intended to build is useful context, but it can also make me less likely to question my assumptions. After a long implementation session, I still need to inspect the result with enough attention to catch security issues, broken business rules, and unexpected changes.</p>
<p>That concern is part of a broader engineering conversation. Martin Fowler distinguishes agentic programming, where a developer reviews the agent’s work, from “vibe coding,” where the code may not be inspected. In either case, someone remains accountable for how the software behaves. <a href="https://martinfowler.com/bliki/AgenticProgramming.html">Fowler, “Agentic Programming”</a></p>
<h2>The review burden is becoming more visible</h2>
<p>In Sonar’s 2026 State of Code survey, 38% of 1,149 respondents said reviewing AI-generated code took more effort than reviewing code written by colleagues; 48% said they always checked AI-assisted code before committing. These are self-reported survey results from a code-quality vendor, not an independent measurement of review time, but they suggest the verification burden is a practical concern beyond my own workflow. <a href="https://www.sonarsource.com/state-of-code-developer-survey-report.pdf">Sonar, <em>State of Code Developer Survey 2026</em></a></p>
<p>Practitioners have described the same tension. Thoughtworks’ Birgitta Böckeler recommends carefully reviewing generated code and pausing when an agentic coding session becomes overwhelming. Her examples include misunderstood requirements, unnecessary complexity, and changes that could create maintenance costs later. <a href="https://www.martinfowler.com/articles/exploring-gen-ai/13-role-of-developer-skills.html">Böckeler, “The role of developer skills in agentic coding”</a></p>
<p>In a separate migration case, Thoughtworks’ Rahul Ramesh describes code review as a significant part of the work, even when generation itself was fast. He notes that an AI reviewer may identify syntax or common practices, while intent and business rules still need human confirmation. That distinction matters: code can be locally plausible and still violate what the system is supposed to do. <a href="https://www.martinfowler.com/articles/research-review-rebuild.html">Ramesh, “Research, Review, Rebuild”</a></p>
<p>There is also research on cognitive bias and decision fatigue in code review. Jetzen and co-authors designed prototypes to address these concerns, but describe their work as an early step, not proof that a particular tool eliminates bias or fatigue. The specific case of a developer reviewing their own AI-assisted change deserves careful framing: it is a real concern in my experience, not a universal result established by that paper. <a href="https://arxiv.org/abs/2407.01407">Jetzen et al., “Towards debiasing code review support”</a></p>
<h2>The goal is to direct human attention</h2>
<p>A review skill should help answer a narrower question than “Is this code good?” It should help the reviewer see which parts of a change deserve closer inspection, why they matter, and what context is still missing.</p>
<p>The <a href="https://github.com/gabrielolferrari/ai-sandbox/blob/main/skills/ai-code-review-triage/SKILL.md">AI Code Review Triage skill</a> follows that approach. It starts by establishing the intended behavior and review scope, then traces consequential changes through relevant code paths. It prioritizes risks such as authorization boundaries, core business rules, data integrity, failure handling, and concurrency when those categories apply.</p>
<p>For a candidate finding, the skill asks for evidence, a trigger, a concrete consequence, and a focused way to verify it. If a claim depends on a business rule that is not in the available context, it should be presented as a question for the domain owner rather than as a confirmed defect.</p>
<p>Google’s AutoCommenter research offers a useful precedent for scoped AI assistance in code review: the system focuses on programming-language practices and was evaluated at industrial scale. That is different from claiming that a model can infer a system’s business intent or approve a change. <a href="https://research.google/pubs/ai-assisted-assessment-of-coding-practices-in-industrial-code-review/">Google Research, “AI-assisted Assessment of Coding Practices in Industrial Code Review”</a></p>
<h2>A second reader, not an approval bot</h2>
<p>When I review my own change, I already know the story I meant to implement. I want the skill to push me back toward the evidence in the code: what changed, which paths it affects, what assumptions it relies on, and what still requires domain judgment.</p>
<p>That can help organize the review, but it cannot guarantee that every issue will be found. A narrow, evidence-based report can still miss a risk when the relevant context is absent. A long list of generic warnings can also consume attention instead of focusing it. The skill therefore avoids treating “no actionable findings” as approval and calls out validation gaps separately.</p>
<p>A useful triage step should complement tests, static analysis, security scanning, and human review. It is most valuable when it helps a reviewer spend limited time on consequential behavior rather than repeating generic style advice.</p>
<h2>Keeping review practice sharp</h2>
<p>Reviewing real changes is where triage matters. But staying sharp also means practicing on code that is unfamiliar and low stakes. I built <a href="https://gabrielolferrari.github.io/one-diff-a-day/">One Diff a Day</a> for that: <em>a daily review game with a quick diff for low-energy days and a deeper challenge when I have more time.</em></p>
<p>That deliberate exposure is a complement to review triage, the skill focuses attention on a real change, while the game keeps the habit of reading unfamiliar code active.</p>
<p>--</p>
<p>The broader trade-off is straightforward. Faster code generation can increase the amount of code that needs to be understood. Triage may make the review more focused, but the reviewer remains responsible for security boundaries, domain rules, and whether the change is fit to merge. The skill’s job is to make those questions harder to overlook.</p>
]]></content:encoded></item><item><title><![CDATA[State Management in Multi-Agent Systems: Bridging Probabilistic LLMs and Deterministic Code]]></title><description><![CDATA[The rapid integration of LLMs into the Software Development Life Cycle has introduced a fascinating architectural challenge. For years, classical software…]]></description><link>https://gabrielferrari.hashnode.dev/state-management-in-multi-agent-systems-bridging-probabilistic-llms-and-deterministic-code</link><guid isPermaLink="true">https://gabrielferrari.hashnode.dev/state-management-in-multi-agent-systems-bridging-probabilistic-llms-and-deterministic-code</guid><dc:creator><![CDATA[Gabriel Ferrari]]></dc:creator><pubDate>Tue, 15 Sep 2026 02:32:39 GMT</pubDate><content:encoded><![CDATA[<p>The rapid integration of LLMs into the Software Development Life Cycle has introduced a fascinating architectural challenge. For years, classical software engineering has relied on absolute determinism: state is sacred, contracts are strict, and mathematics is exact. Generative AI, however, introduces a probabilistic layer to our systems.</p>
<p>To explore the boundaries between these two paradigms, I built a local multi-agent experiment. The premise was straightforward: two local models (Llama 3.2 and Qwen 2.5) running via Ollama, tasked with negotiating a shared, finite compute budget. Agent A would request resources for workloads, and Agent B would act as the governor, approving or denying requests based on a shared state.</p>
<p>While this sandbox is intentionally simplified, the underlying challenge mirrors complex enterprise scenarios. The market is currently flooded with proposals to deploy AI agents for autonomous budget allocation, dynamic pricing models, analyzing sales pipeline probability, or even estimating development complexity to assign story points. In all these real-world use cases, the AI eventually hits a hard wall: it must interact with finite resources, strict business constraints, and external systems.</p>
<h3>The Illusion of State in Conversational AI</h3>
<p>The initial approach relied on natural language to manage the transaction. The orchestrator injected the current budget into the prompt, and the models conversed to reach an agreement.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6aa8aa00ab9d3c9320651c23/011a6031-201a-43f3-bb58-2d72668ee1d1.png" alt="" style="display:block;margin:0 auto" />

<p>At first glance, it looks like a success. The math (100 - 40 = 60) is correct. However, this is where the illusion of state breaks down in AI engineering.</p>
<p>The LLM generated the correct text, but the deterministic system (our Python orchestrator) is blind to conversational output. Because there was no data contract, the Python script could not parse the "60 units" to update our underlying database (the <code>state.json</code> file).</p>
<p>The Single Source of Truth (<code>state.json</code>) remained unchanged at <code>allocated_budget: 0</code>. When Round 2 started, the orchestrator unknowingly injected the full 100 budget back into the prompt. The models, trying to conciliate the chat history with the new conflicting system prompt, began to lose track of reality.</p>
<p>Treating conversational output as a transactional engine is a critical anti-pattern. You cannot persist state in a chat log.</p>
<pre><code class="language-json">{
  "total_budget": 100,
  "allocated_budget": 0,
  "turn": 4,
  "max_turns": 3,
  "status": "IN_PROGRESS"
}
</code></pre>
<h3>Forcing Contracts: The Structured Output Pattern</h3>
<p>To stabilize the system, the LLM had to be constrained. Instead of a free-flowing text generator, it needed to output predictable data structures.</p>
<p>By enforcing a Strict Structured Output (JSON) on Agent B, the architecture shifted responsibilities. The AI was now solely responsible for the <em>reasoning</em> evaluating the strategic merit of Agent A's request based on context. The deterministic orchestrator took back control of the state.</p>
<p>Once Agent B outputted a valid JSON containing <code>"approved_units"</code>, the orchestration layer parsed the data, executed the exact mathematical subtraction, updated the state file on disk, and enforced a Circuit Breaker pattern to prevent any hallucinated over-allocation.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6aa8aa00ab9d3c9320651c23/2f69cd1f-635d-42d3-8166-09f0cc95a826.png" alt="" style="display:block;margin:0 auto" />

<pre><code class="language-json">{
  "total_budget": 100,
  "allocated_budget": 22,
  "turn": 4,
  "max_turns": 3,
  "status": "IN_PROGRESS"
}
</code></pre>
<h3>The Pragmatic Architect's Dilemma</h3>
<p>The structured approach successfully stabilized the multi-agent system. The JSON contracts held, the state mutated correctly, and the experiment was a technical success. But it inevitably leads to a necessary architectural provocation: <em>Why use Generative AI for this in the first place?</em></p>
<p>If the core business rule is simply "approve if requested amount is less than or equal to balance," a fundamental <code>if</code> statement in Java, C#, or Ruby executes in microseconds, costs zero API tokens, and has a 0% hallucination rate.</p>
<p>The true value of AI in this context is not managing the state, but evaluating the unstructured variables. If Agent A requests 80% of the remaining budget, the LLM should be utilized to analyze the <em>justification</em> for that request against the current operational context, not to do the math.</p>
<p>As we evolve towards Agentic Workflows, the role of a Solutions Architect is not to replace traditional engineering with AI. It is knowing exactly where the probabilistic reasoning of an LLM ends, and where the unforgiving determinism of classical software engineering must begin.</p>
]]></content:encoded></item></channel></rss>