Replay-Verified Is Not Production-Safe: Where the Agentic Database's Autonomy Ladder Breaks
TL;DR: While the concept of an agentic database promises to automate manual DBA tasks through a three-tier autonomy ladder, relying on replay-verification to justify "unattended action" is operationally risky. Replay proves answer-equivalence on historical traffic but fails to account for secondary production side effects like lock contention, cache invalidation, and replication lag.
The database remains the most conservative component of the modern stack, and for good reason. While we have automated deployments for application code and infrastructure, the database still requires high-touch human intervention. This friction is at the heart of the recent discourse surrounding the agentic database.
Engineering leaders are currently circulating a provocative argument: databases are stuck being passive storage engines when they should be active agents. The thesis, most notably championed by Balnojan on the Finish Slime Substack, suggests that the "moat" for database companies is no longer just features, but permission—the degree of autonomy they are granted to act on behalf of the user.
The article everyone in database land is passing around
The central argument in favor of the agentic database is compelling: a database should not just wait for an EXPLAIN ANALYZE command to tell you a query is slow. It should observe the slowness, hypothesize a fix (like a new index), and ideally, implement it.
According to Balnojan in "Agentic databases aren't agentic," the current state of "AI in DBs" is mostly just glorified chat interfaces. We are essentially building better manuals rather than better pilots. The vision presented is a database that climbs an "autonomy ladder" to reclaim the time engineers spend on performance tuning, schema migrations, and vacuuming.
In this model, the database evolves from a tool to a teammate. We should "steelman" this position: the passive database is a waste of compute and human cognitive load. If a system has the telemetry to see a looming OOM (Out Of Memory) error or a table bloat issue, forcing a human to log in at 2:00 a.m. to run a manual VACUUM FULL is a failure of system design.
The ladder: act, propose, ask
The proposed framework for granting this autonomy relies on a three-tier ladder designed to manage risk while maximizing speed:
- Act: The agent identifies a change where correctness can be proven. It executes the change autonomously and notifies the team.
- Propose: The agent identifies a change that alters behavior or where the outcome is probabilistic. It drafts a plan for a human to approve.
- Ask: The agent identifies a bottleneck but lacks the "meaning" or business context to solve it. It asks the human for direction.
The fulcrum of this entire strategy is the "Act" lane. To safely allow an agentic database to act without a human in the loop, the system must prove that the action is safe. The suggested mechanism for this proof is fork-and-replay: spin up a thin clone of the database, apply the change, and replay recent production traffic to ensure the results are identical.
What replay actually proves
At first glance, fork-and-replay looks like the gold standard of verification. If the agent wants to add an index or rewrite an expensive query, and the replayed traffic shows a 20% speedup with 0% error deviation, why not let it rip?
The problem is that "replay-verified" is often conflated with "production-safe." These are not the same thing. Replay verification demonstrates answer-equivalence under a recorded slice of time. It proves that for the queries replayed, the output did not change. However, it fails to account for the chaotic, stateful nature of a live production environment.
Key Takeaway: Replay-verification proves that a change worked yesterday, but it cannot guarantee that the secondary operational side effects won't break production tomorrow.
Here is an incomplete list of what replay-verification typically misses:
- Lock Contention: Adding an index
CONCURRENTLYis safer, but it still requires two table scans and can hang on long-running transactions. A replay on a quiet fork won't capture the lock-ordering deadlocks that occur in a high-concurrency primary. - Plan Flips on Unrelated Queries: The "Halloween Problem" or general optimizer shifts mean that a new index might make Query A faster but cause the optimizer to choose a disastrous nested loop for Query B, which wasn't part of the replay sample.
- Write Amplification: An index speeds up reads but slows down every
INSERT,UPDATE, andDELETE. On a write-heavy path, the cumulative latency increase can tip a system into a retry-storm. - Cache Warmup and Buffer Pool Churn: Replaying traffic on a fresh fork doesn't simulate the impact of blowing out the LRU cache on the production primary.
- Replication Lag: A change that is "fast" on a standalone fork can generate massive amounts of WAL (Write Ahead Log) traffic, causing replicas to fall behind and breaking read-after-write consistency for the application.
- Disk Headroom: Replays often happen on thin-provisioned clones. They don't always account for the physical disk space required for a 500GB index build on a volume that is already at 85% capacity.
'Provably correct' meets production
Consider the example of the 2:14 a.m. index creation. In the autonomy ladder model, the agent sees a slow query, forks the DB, verifies the index works, and applies it.
But imagine the production database is currently undergoing a massive data import or a heavy ETL job that the replay window missed. The "provably correct" index creation starts, hits a metadata lock, and queues up all subsequent queries behind it. Within minutes, the application's connection pool is exhausted. The agent was "correct" about the query optimization, but it was "wrong" about the operational context.
In this scenario, the "correctness" is narrow (mathematical/logical), while the "failure" is broad (architectural/operational). This is where the autonomy ladder's "Act" lane starts to break down in real-world agentic databases. When the cost of a false positive is a site-wide outage, the threshold for "proven" must be impossibly high.
The unit of trust is the reviewable change
If the "Act" lane is narrower than we hope, how do we actually get the benefits of an agentic database without the risk?
The answer lies in shifting the focus from the autonomy boundary (who has permission to click the button) to the reviewable unit (what is actually being changed).
A reviewable change is:
- Diffable: It clearly shows exactly what will be modified.
- Revertible: It includes an automated rollback plan that has also been verified.
- Evidence-Attached: It doesn't just say "trust me"; it provides the execution plans, the replay logs, and the observed metrics from the fork.
When a change is packaged this way, the "Propose" lane becomes incredibly fast. A human doesn't need to spend two hours investigating the issue; they spend two minutes reviewing the evidence provided by the agent and then approve the deployment. This maintains the human-in-the-loop safety net while removing the investigative bottleneck.
Actually, a patch file that a human reviews and applies via an existing CI/CD or migration pipeline is superior to an agent with direct write access. It ensures that the change is recorded in the git history, follows standard deployment windows, and is visible to the entire engineering team.
The market already voted
We have seen this play out before with earlier iterations of "autonomous" database tools.
According to various industry post-mortems, OtterTune—one of the most promising early players in AI-driven database tuning—eventually shut down in 2024. While there are many reasons for a startup's sunset, a recurring theme in the "auto-tuning" space is the difficulty of gaining sufficient trust for unattended write access to production.
In contrast, the tools that have gained massive adoption follow the "Propose-then-Apply" model. PlanetScale, for example, uses "Deploy Requests." Even though their underlying technology (Vitess) is capable of sophisticated schema management, they treat database changes like code reviews. According to PlanetScale’s documentation, this flow ensures safety, team awareness, and zero-downtime migrations without requiring the system to be fully "autonomous."
Similarly, internal engineering teams at companies like OpenAI use agents (often powered by Codex or GPT-4o) to monitor for schema drift or performance regressions. These agents don't typically "fix" the problem themselves; they draft the fix, attach the evidence, and open a PR.
Where we agree completely
While we argue that the "Act" lane is thinner than Balnojan suggests, his core insight regarding trust remains the path forward. Autonomy must be earned per change class.
You might grant an agent the right to "Act" on a very specific, low-risk class of changes—perhaps updating statistics or killing a query that has exceeded a 5-minute timeout. Every time a human reviews and accepts a proposal in the "Propose" lane, that becomes training data for the agent.
Over time, if an agent proposes a specific type of index 100 times, and 100 times the engineer approves it and it succeeds, the "Propose" boundary for that specific action can eventually shift to "Act." Trust is the ledger of reviewed-and-accepted diffs.
The revised ladder
To make agentic databases or any production AI agent truly safe for the enterprise, we propose a slight adjustment to the autonomy ladder:
- Propose everything by default: Treat every action as a draft. Provide the full evidence, the "why," and the "how to undo."
- Act only on earned, monitored, and revertible tracks: Move a class of change to "Act" only after a documented track record of success. Even then, keep it on a "short leash" with automated rollbacks if production metrics (like replication lag) deviate.
- Ask where meaning begins: Continue to use humans for high-level strategy and business logic.
The goal of the agentic database shouldn't be to remove the human, but to provide the human with a high-fidelity "operator's assistant" that does all the legwork.
At Operate, we've built our platform around this "Propose" philosophy. Our agents investigate incidents across the entire stack—code, databases, and infra—but they don't have direct write access to your production environment. Instead, they produce a verified patch and an evidence-backed explanation, allowing you to open the PR with confidence. By focusing on the reviewable unit rather than total autonomy, you get the speed of an agent with the safety of your existing engineering standards.
Sources & further reading
- According to Balnojan in "Agentic databases aren't agentic," the real moat for future databases is the degree of permission users grant the system to act.
- PlanetScale's documentation on "Deploy Requests" highlights the industry's preference for a review-based workflow for database changes.
- The 2024 shutdown of OtterTune serves as a data point for the challenges faced by "below the line" automated tuning systems.
- The concept of "Act, Propose, Ask" is explored in depth in the July 16, 2026, post on the Finish Slime Substack.
- For more on the limitations of verification, see "Recovery Is Not Convergence."
The transition to agentic databases is inevitable, but it won't happen because we finally trusted a "black box" to write to our disks. It will happen because we developed agents that can argue for their changes so clearly that the human "approval" becomes the easiest part of the day.