Everyone Is Building an Ops Copilot: Notes From the DIY Wave
Practitioners are increasingly choosing to build rather than buy when it comes to AI operations, favoring local-first, self-hosted LLM stacks for production troubleshooting. This shift highlights a growing demand for data privacy and auditability, setting a new bar for what commercial AI SRE platforms must provide to remain competitive.
The traditional build vs buy software debate is undergoing a fundamental shift as Large Language Models (LLMs) move from "cool demo" to "operational necessity." This past week, a distinct pattern emerged across the engineering community: senior practitioners are no longer waiting for vendors to solve AI operations. Instead, they are assembling their own "Ops Copilots" using local-first tools and self-hosted models.
From database engineers running StarRocks and Ollama on their laptops to Kubernetes veterans proposing new application interfaces for agents, the DIY wave is here. But this isn't just about engineers wanting to play with new toys. It is a calculated response to the specific constraints of production environments: security, latency, and the need for absolute control over operational data.
The pattern: five posts, one week, same instinct
In a single seven-day window, five separate signals converged on the same conclusion: the "Buy" option for AI operations is currently failing to meet the requirements of sophisticated engineering teams.
- The Milhouse Launch: Ryan Perry introduced Milhouse, a local-first observability and ops feedback platform. It ties together deployments, errors, alerts, and AI agent sessions, all while keeping the data under the user's control.
- The Local Data-Ops Stack: A prominent database engineer shared their setup for a fully local Kubernetes + Ollama + StarRocks + Postgres stack. The goal? Explicit validation of operational insights without a single byte of sensitive schema data leaving the network.
- Brian Grant’s "Application Interfaces": The Kubernetes co-lead argued that if AI agents are going to manage infrastructure, applications need discoverable management interfaces—not just scraped dashboards.
- The "Plan-Act-Verify" Loop: A Microsoft engineer demonstrated agent loops over live operational data, emphasizing human approval gates as a non-negotiable requirement for AI in production.
- The AIOps Reality Check: Alex Ponomarev’s analysis of the current AIOps landscape noted that while the promise is high, practitioners are retreating to custom-built "agents" that do one thing well rather than broad, black-box platforms.
Key Takeaway: The DIY wave is a requirements document: if you build, budget for verification and maintenance as first-class costs; if you buy, hold vendors to the DIY bar of being self-hosted, read-only, and auditable.
Why engineers build: control, data locality, and distrust of cloud AI
The primary driver behind the build vs buy software decision in AI operations is trust. When choosing a tool for root cause analysis (RCA), an engineer is effectively choosing who to trust with their most sensitive telemetry.
Cloud-based AI vendors often require streaming logs, metrics, and traces to their proprietary models. For a CTO or VP of Engineering, this is a non-starter for three reasons:
- Data Locality: Regulatory requirements (GDPR, SOC2, HIPAA) make it excruciatingly difficult to ship production logs—which often contain accidental PII—to a third-party LLM provider.
- Context Loss: A generic cloud model doesn't understand your internal service mesh or the specific quirks of your legacy monolith. Engineers build because they can "feed" the local model their specific documentation and architectural hurdles.
- The "Black Box" Problem: If a SaaS AI tells you to restart a database, you want to know exactly why. DIY builders prioritize explicit evidence—showing the exact log line or metric spike that led to the conclusion.
What the builders get right: self-hosting and approval gates
The DIY community is building with a level of rigor that many commercial "Copilots" lack. According to Ryan Perry’s introduction of Milhouse, the focus on a "local-first" experience ensures that the tool is available even during a total network partitions—the exact time you need an ops tool most.
Builders are also implementing human approval gates by default. Instead of letting an AI agent execute a "Plan" (like scaling a cluster), the DIY tools demoed by Microsoft engineers focus on the "Plan" and "Verify" steps, leaving the "Act" to a human. This "Read-Only" nature by design is a safety feature, not a limitation.
Furthermore, these homegrown tools are increasingly using explicit validation. Rather than trusting a single LLM's hallucination, builders are using secondary models or deterministic scripts to check the agent's work. This multi-layered approach to reliability is often missing from polished SaaS interfaces that prioritize a slick UI over verifiable accuracy.
Where DIY ops copilots stall
Despite the momentum, internal DIY projects often hit a ceiling. When evaluating the build vs buy software trade-off, engineering leaders must account for the hidden costs of success.
1. Verification: The "Who guards the guards?" problem
Most DIY tools rely on a single-model self-review. If the same model that proposed a root cause is also asked to verify it, it will often reinforce its own hallucinations. Implementing a truly independent verification pipeline—where a separate, specialized model audits the work of the primary agent—is technically complex and resource-intensive to build from scratch.
2. The maintenance tax: New unplanned work
An internal AI ops tool is, itself, a production service. It requires monitoring, patching, and model fine-tuning. According to an example from Notion's blog on building observability into dead-letter queues, internal tooling requires constant evolution to stay relevant. Often, the "Ops Copilot" becomes a new source of toil for the SRE team, negating the very efficiency it was meant to create.
3. The Bus Factor
Internal platforms frequently die when their primary author moves to another team or company. These tools are often built on specific "tribal knowledge" that isn't documented. When the builder leaves, the tool becomes a "zombie" platform—too critical to turn off, but too opaque to update.
Brian Grant's point: agents need management interfaces
A critical insight from Kubernetes co-lead Brian Grant is that our current infrastructure isn't designed for AI. We’ve spent years building dashboards for humans to read. When we build a DIY AI tool, we often force the AI to "scrape" these dashboards or "read" logs like a human would.
Grant argues that for AI operations to truly scale, applications must expose discoverable management interfaces. This means structured data that an agent can query directly—like a specialized API for system health and intent—rather than forcing the agent to guess based on unstructured log patterns. For a DIY team, building these interfaces across an entire microservices architecture is a multi-year undertaking.
A build-vs-buy framework for AI ops tooling
If you are a VP of Engineering or a Staff Engineer deciding your path, use this evaluation checklist. If you cannot check at least four of the "Buy" boxes, you should probably allow your team to continue their DIY efforts.
| Feature | The DIY Path | The "Buy" Requirement |
|---|---|---|
| Privacy | Data never leaves the VPC | Self-hosted or Private Link deployment |
| Safety | Human-in-the-loop by default | Read-only access; drafts PRs/commands |
| Reliability | Tailored to your specific stack | Independent verification of AI findings |
| Maintenance | High (Internal eng team) | Low (Handled by vendor) |
| Auditability | Full trace of AI "thought process" | Evidence-based (Shows log/metric links) |
| Context | Uses internal docs/repos | Integrated with GitHub/GitLab and Telemetry |
What buying has to prove: matching the DIY bar
The market for AI operations is crowded, but the DIY wave shows that many solutions miss the mark. To win over a team that is already building a "Local-LLM-StarRocks-Ollama" stack, a commercial tool must prove it respects the same virtues: it must be self-hosted, read-only, and auditable.
The "Buy" option only makes sense when it delivers the sophistication of a multi-agent system that a single engineer can't build in their spare time, without sacrificing the privacy of the local stack.
Why this matters now
The urgency of this build vs buy software decision is driven by the "AI Toil" paradox: as we adopt AI to write code faster, we create more services and more complexity, which in turn creates more operational toil. Engineering leaders can't afford to let their best SREs spend 40% of their time maintaining an internal AI tool, but they also can't afford the security risk of a generic cloud-AI copilot.
Finding a middle ground—a professional platform that installs into your environment and acts with the caution of a senior engineer—is the next frontier for production stability.
Sources & further reading
- According to Ryan Perry, the launch of Milhouse signals a move toward local-first observability that integrates AI sessions with live telemetry.
- Brian Grant (Kubernetes co-lead) notes in his essay on Application Interfaces that we must move beyond scraping dashboards if we want AI agents to effectively operate software.
- The Microsoft engineering team's research on "Plan-Act-Verify" loops suggests that human approval gates are the primary mechanism for building trust in AI operations.
- Alex Ponomarev highlights that the AIOps landscape is shifting toward specialized agents rather than general-purpose assistants.
- Internal tooling examples from Notion demonstrate the high maintenance cost of building specialized observability logic in-house.
Moving Beyond the DIY Ceiling
Operate exists because the DIY bar is the correct one: it is designed to be self-hosted, read-only, and fully auditable. While many homegrown copilots rely on a single model's intuition, Operate utilizes a specialized four-agent pipeline to investigate incidents.
Our architecture ensures that the AI's conclusions are verified by a different model than the one that proposed the root cause—addressing the "verification gap" where most DIY projects stall. Operate doesn't ask for write access to your production environment; it investigates your telemetry, finds the evidence, and drafts a fix as a PR. A human operator always makes the final call, ensuring that the benefits of an advanced AI SRE platform never come at the cost of control or privacy.