Ankra Blog

Ankra is now ankra.ai: Rewritten in Go, Ready for What's Next

Mark Shine — Mon, 06 Jul 2026 00:00:00 GMT

The product has been pointing at this for a year: an AI that runs your operations, not one that just answers questions about them. This month the name caught up. Ankra is now ankra.ai. Same platform, same clusters, new center of gravity.

Underneath the new name sits the biggest engineering release we have ever shipped. Here is everything that landed since March: the Go rewrite, an AI that owns the incident loop, a complete identity stack, European managed multi-cloud, and Ankra on your phone.

Ankra goes all the way with Go

Over the past two quarters we re-engineered the Ankra Agent and the Ankra backend in Go: the API, the work schedulers, the maintenance loops, the GitOps reconciler, and the chat pipeline. Performance is now a first-class focus, and the gains are platform-wide. Go has taken our headroom to 25x the scale on the same footprint, and the speed-up is felt everywhere, from API responses to reconciles to chat. As Ankra picks up traction, this is the foundation we are building toward enterprise-grade scale.

What that means in practice:

A fraction of the footprint. The agent now runs with a 512Mi memory limit, down from 1536Mi. Same reconcile work, a third of the reservation, on every cluster you run.
Built to scale 25x. No runtime to warm up, no interpreter in the path. Go lets the platform absorb an order of magnitude more load, so performance holds as you grow into enterprise-grade fleets.
Zero action required. The Go agent on the 2.0.x track is a drop-in replacement, rolled out across the fleet automatically. Your stacks, operations, and workflows carry over untouched.
One binary everywhere. A single statically compiled binary per platform. No dependency tree, no runtime patching, a smaller surface to attack and to audit.

Describe the problem. Approve the fix.

Most of every incident is spent collecting evidence. The AI assistant now owns that loop: tell it the symptom and it correlates logs, events, manifests, metrics, Stack history, and Git state, drafts a multi-step plan, and executes it with your approval at every mutating step. It remembers context across conversations, reaches your own tools over MCP, and nothing changes without you. Between incidents, AI Insights keeps scanning, adapting to what your clusters actually run.

Prefer your own agent? Point Cursor, Claude Code, or anything with a terminal at the Ankra CLI and it becomes a cluster-aware infrastructure subagent: real cluster state, incident triage with ankra chat, and every change landing as a reviewable draft, never a raw kubectl edit. We wrote a full guide on wiring Cursor up as an Ankra subagent.

Security that keeps up with you

This quarter shipped a complete identity stack. Native two-factor authentication and multi-social sign-in, built into every surface you use: browser, CLI, and now mobile.

Passkeys, natively. Sign in with Touch ID, Face ID, or Windows Hello, the credential your device already trusts, with hardware security keys and authenticator-app codes (TOTP) alongside, and one unified set of recovery codes.
Step-up at every sign-in. Verification happens on each login, and ankra login completes MFA in the browser before dropping you back in the terminal.
Enforce it for the whole organisation. Require MFA org-wide, with admin reset for locked-out teammates.
Sign in your way. Google, Apple, GitHub, or email and password. Link them to one account from Profile → Authentication.
Beyond login. Trivy Operator vulnerability reports on a dedicated cluster Security page, kubeconfigs that expire on their own, --strict-secrets validation in CI, and checksum-verified CLI upgrades.

The platform, in your pocket

Ankra is now on iOS and Android in beta. Mission Control, cluster health, live operations, AI chat, and AI Insights, from wherever the page finds you. And the security stack above is built in from the first unlock: Face ID or Touch ID on iOS, BiometricPrompt on Android, automatic re-lock the moment the app leaves the foreground, device passcode as the fallback.

The beta ships through TestFlight and Google Play internal testing. Reach out on Slack and we will send you an invite.

European by default. Sovereign by design.

Ankra was founded in Sweden and built for Europe, and our managed clusters reflect it. Provision and run production Kubernetes on Hetzner, OVH, and UpCloud, three European clouds, from one console and one set of commands. Your control plane, your data, and your workloads stay on the continent, under EU jurisdiction. No hyperscaler lock-in, no data leaving the region unless you choose it.

Three European clouds, one workflow. Create, scale, upgrade, and deprovision with the same provider-agnostic commands, wherever the cluster runs.
Data residency you control. Keep clusters and data inside the EU, close to your users and inside your regulatory perimeter, without giving up the managed experience of a hyperscaler.
Roots in Sweden and the EU. We are a European company by conviction, not convenience. Digital sovereignty is not a feature we bolted on; it is the reason Ankra exists.
No lock-in, ever. Standard Kubernetes, GitOps as the source of truth, and clusters you can move between providers. Your platform is yours to take anywhere.

Deploying your app

Connect a repo, get a running app. Applications takes a repository and generates the Dockerfile, Helm chart, and CI pipeline as a pull request, then deploys the result to your cluster as a stack. Review the PR. That is the whole handoff. It is in closed beta today: open Applications in the platform to join the waitlist.

Commit to live in one view: the delivery pipeline, image vulnerabilities, and the GHCR artifact, all tracked per application.

Capture a working stack once, apply it anywhere. Stack Profiles turn a proven stack into a versioned template for any cluster: ankra stack-profiles get and apply from the CLI, with --category to browse by what they do.

Take the safe path to production. ankra cluster draft stages the change, validate --strict-secrets fails CI before a plaintext secret lands in Git, then deploy with --watch and follow it land from the terminal. Changing one value? Helm-style --set patches it atomically, with --dry-run to preview first.

Maintaining your app

Retire the ArgoCD runbook. Ankra’s own engine now reconciles your clusters natively: nothing to install, upgrade, or unstick, and the same Git-as-source-of-truth workflow. Existing clusters migrate addon-by-addon.

One set of verbs for every cloud. ankra cluster upgrade, scale, node-group: the same commands wherever the cluster lives. The provider is a detail, not a prefix you have to remember.

Upgrades that don’t gamble with your workloads. ankra cluster upgrade now covers kubeadm clusters alongside k3s: one node at a time, control plane first, each node cordoned and drained respecting PodDisruptionBudgets, with an etcd snapshot taken before the control plane moves. A blocked drain aborts safely; --force only when you say so.

Stop paying for idle clusters. Delete a cloud cluster’s infrastructure to stop the charges while it stays registered in Ankra, then restore it on demand, even reshaped: controllers, instance types, node groups.

Clone environments, not runbooks. Select every stack and clone to another cluster, even across organisations, in one operation, name conflicts resolved automatically and variables resolving per scope so the clone fits its new home.

See the whole fleet at a glance. Operations live-update as they run, PromQL queries work straight from the CLI, and the dashboard puts your fleet on a world map with cloud cost beside it.

Also shipped

On Bitbucket? You’re in. Bitbucket Cloud joins GitHub as a GitOps provider, and GitHub users can switch to App installs, with scoped, revocable access instead of long-lived personal tokens.
kubeadm, end to end. ankra cluster kubeadm-versions lists upgrade targets, and cloud creates take --etcd-topology stacked or external with 3- or 5-node dedicated etcd.
Two-factor from the terminal. ankra profile auth enrolls, lists, and removes MFA methods without opening a browser tab.
The CLI updates itself. ankra upgrade takes you to v0.5.0 with a verified checksum and swaps the binary in place; pin a version to roll back.
One terminal, many targets. Global --org and --cluster flags apply per request, and kubeconfigs now pin the owning organisation, so kubectl keeps working after you switch orgs.
Stack edits without opening the UI. Edit dependency ordering, print addon values or manifest YAML, delete manifests, and keep node SSH keys in sync (ankra ssh-keys get/set/resync) from the CLI.
Guardrails for scripts and humans alike. A documented exit-code contract so pipelines branch on the failure class, and every destructive command confirms before acting.
Claude Code joins Cursor. ankra skills --editor claude-code installs the curated Ankra practices into your coding agent of choice.
The CLI reference writes itself. Generated from the command tree and synced to the docs on every release, so the reference never drifts from the binary you run.
Removed: add-on subscriptions. Add-ons are now managed directly on the cluster or stack; legacy subscription endpoints return 410 Gone.

Why the name changed

A year ago, Ankra was where you managed Kubernetes. Today it is where you deploy an app, keep it healthy, and hand the 3am incident to an agent that fixes it with your approval, from a terminal, a browser, or the phone on your nightstand. ankra.ai is not a rebrand. It is a description.

Faster than you remember it. Same clusters, new engine. Open Ankra or read the full changelog.

Secrets Belong in Git. Plaintext Does Not.

Mattias Åsell — Sat, 04 Jul 2026 00:00:00 GMT

Every GitOps rollout hits the same wall, usually in week two. The team agrees Git should be the source of truth. The cluster definition goes into a repo. The Helm values follow. Then someone tries to commit the Postgres password and the whole plan stalls in a meeting about “how we handle secrets,” and three sprints later half the configuration lives in Git and the other half lives in a vault nobody wired up, a CI variable nobody documented, and a .env file that gets passed around in Slack DMs.

We have written a lot on this blog about Git as the source of truth: pipeline agents that review cluster changes on pull requests, AI that proposes drafts instead of touching production, stacks that clone from a laptop to production. All of it assumes an answer to the question this post finally tackles head-on: if Git holds everything, where do the credentials go?

The answer is that they go in Git too, encrypted, with a workflow that makes the safe path the default path. Here is how that works, why the alternatives fail, and how Ankra makes it a one-time setup instead of a platform project.

The numbers got worse, and AI is making them worse faster

GitGuardian’s State of Secrets Sprawl report has been the industry’s yearly cold shower for half a decade. The 2026 edition is the coldest yet: 28.65 million new hardcoded secrets landed in public GitHub commits during 2025, a 34 percent jump year over year and the largest single-year increase they have ever recorded.

Two findings in that report matter specifically for infrastructure teams.

First, AI-assisted commits leak secrets at roughly double the baseline rate. Coding assistants reproduce the patterns they were trained on, and the training data is full of API keys pasted into examples and passwords hardcoded in test fixtures. As more of your YAML gets written by an agent, the odds of a credential slipping into a commit go up, not down.

Second, the report found six times more hardcoded secrets in internal repositories than in public ones. Teams behave as if a private repo is a security boundary. It is not, and there is a well-known incident that proves it.

”The repo is private” is not a secrets strategy

In April 2024, attackers got into the self-managed GitLab instance of Sisense, a business intelligence company whose product connects to its customers’ data sources. Inside that private repository they found hardcoded AWS credentials. Those credentials opened the company’s S3 buckets, and terabytes of customer data walked out: access tokens, passwords, SSL certificates. The breach was serious enough that CISA issued an alert telling every Sisense customer to rotate every credential they had ever shared with the platform.

Note what failed there. Not the firewall, not the cluster, not the cloud account. A private Git repository was treated as a safe place for plaintext credentials, and a single compromised repo converted into full cloud access.

The lesson is not “keep secrets out of Git.” Config that references secrets without containing them just moves the problem somewhere with less review, less history, and less audit trail. The lesson is that a secret in Git must be worthless to anyone who reads it.

And one more number, because it explains why leaks hurt for years instead of days: GitGuardian found that 64 percent of the secrets they detected in 2022 were still valid in January 2026. Most teams do not rotate. The secret that leaks today is the secret that gets used against you in three years.

Encrypt values, not files

The tool for this job is SOPS, and the detail that makes it the right tool is that it encrypts the values in a YAML file while leaving the keys and structure readable.

That distinction sounds small. It is everything. A fully encrypted blob cannot be reviewed: a pull request that changes it is a diff between two walls of base64, and your review process degrades to “approve and hope.” A SOPS-encrypted file keeps its shape:

apiVersion: v1
kind: Secret
metadata:
  name: postgres-credentials
  namespace: database
type: Opaque
stringData:
  username: app_user
  password: ENC[AES256_GCM,data:vxvYs7vKw+bPqNWO3tE=,iv:...,tag:...,type:str]
sops:
  age:
    - recipient: age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p
  encrypted_regex: ^(password)$
  version: 3.9.4

A reviewer can see that this is a Secret, in the database namespace, that the username changed, and that the password is ciphertext. The commit is safe to push to any repo, public or private, because the value is encrypted with AES256 and only the holder of the private key can recover it.

For the key pair itself, Ankra uses AGE rather than PGP. AGE is a modern, deliberately boring encryption tool: one public key that encrypts, one private key that decrypts, no keyservers, no trust ceremonies, no twenty-year-old tooling to fight. The public key is shareable by design. The private key is the only thing that matters, and in Ankra’s setup, no human ever handles it.

The Ankra workflow: initialize once, encrypt by default

Most SOPS guides now ask you to generate keys, distribute them to every cluster, patch your CD tooling with a decryption plugin, and write a .sops.yaml policy file. That setup cost is exactly why teams postpone secrets hygiene until after the incident.

In Ankra it is one step. Go to Organisation Settings, open Encryption, and click Initialize Encryption. The platform generates an AGE key pair, shows you the public key, and stores the private key in its vault. From that moment:

Encryption is on by default. Any field you mark as encrypted is protected the moment you save.
Decryption is automatic. The Ankra agent on each cluster decrypts values at deploy time by running sops --decrypt directly, as part of Ankra’s native deployment engine. The AGE private key is fetched from the platform for the render and is not stored on the cluster, so there is no key Secret sitting in a namespace waiting to be read. There is nothing to configure per cluster, whether you run three clusters or fifty.

The flow, end to end: you type a plaintext value in the Stack Builder, Ankra encrypts it with your organisation’s public key on save, the ENC[AES256_GCM,...] ciphertext is what gets committed to your GitOps repository, and the agent decrypts it at deploy time. Plaintext exists in the editor and in the running workload. It never exists in Git, and the decryption key never lives on the cluster.

In the Stack Builder

When you edit a manifest or an addon’s Helm values, there is an Encrypted Keys (SOPS) section. Add the key names that are sensitive, password, apiKey, token, adminPassword, and enter values normally. Only the listed keys get encrypted; everything else stays readable, so reviews and debugging keep working.

From the CLI, for GitOps-first teams

If your cluster definition lives in a repo and you edit it locally, the Ankra CLI does the same thing in file mode. Encrypt a key in a Secret manifest before committing:

# Inspect the public key your org encrypts with
ankra cluster sops-config

# Encrypt the password in a manifest referenced by cluster.yaml
ankra cluster encrypt manifest db-secret --key password -f cluster.yaml

# Encrypt a sensitive Helm value on an addon
ankra cluster encrypt addon --name grafana --key adminPassword -f cluster.yaml

The CLI rewrites the referenced file with the encrypted value and records what it did in the cluster definition, so the platform knows which fields to decrypt at deploy time:

manifests:
  - name: db-secret
    from_file: "manifests/db-secret.yaml"
    encrypted_paths:
      - password
addons:
  - name: grafana
    chart_name: grafana
    chart_version: 9.2.10
    repository_url: https://grafana.github.io/helm-charts
    namespace: monitoring
    configuration:
      from_file: "values/grafana.yaml"
      encrypted_paths:
        - adminPassword

Need to check what a value actually is? Decryption prints to stdout and nowhere else:

ankra cluster decrypt manifest db-secret -f cluster.yaml

Make plaintext a failed build, not a code review comment

Hygiene that depends on a reviewer noticing a credential at 5pm on a Friday is not hygiene. The check belongs in CI, where it is deterministic.

Ankra’s validation command has a flag for exactly this. In the pull request pipeline that reviews your cluster changes, run:

ankra cluster validate -f cluster.yaml --strict-secrets

Validation already checks your stack structure, chart references, and dependency wiring against the platform. With --strict-secrets, a plaintext Secret or an unencrypted sensitive value stops being a warning and becomes a failed job. The pull request goes red before a human ever looks at it, and the credential never reaches the default branch.

If you followed our pipeline agents guide, you already have this workflow running; the flag is in it. This is the piece most secrets tutorials skip, and it is the piece that actually changes outcomes, because it removes the human from the failure path.

The AI angle: agents that cannot leak what they never hold

Here is where the GitGuardian finding about AI-assisted commits stops being scary and starts being an architecture requirement. If AI writes a growing share of your configuration, and AI-assisted commits leak at twice the baseline rate, then the fix is not prompting the model to be careful. The fix is a pipeline where the unsafe output physically cannot land.

Ankra’s AI operates under the same contract as everyone else: it proposes drafts, humans approve, Git records the result. Sensitive values in those drafts are SOPS-encrypted before they leave the platform. The agent never holds your cloud credentials, and the commit it produces contains ciphertext, not keys. An AI that drafts a Grafana stack with an admin password produces an encrypted adminPassword, because the encryption step is in the platform, not in the model’s judgment.

Two related behaviors are worth knowing:

Secret variables are masking, not encryption. Ankra variables can be marked secret, which hides them in the UI. That is a convenience feature. For anything that must be encrypted at rest in Git, use SOPS. The platform docs are explicit about this distinction, and now you are too.
Clones do not carry secrets. When you clone a stack to another cluster or another organisation, encrypted values deliberately do not transfer. You reconfigure them in the target. Mildly annoying on purpose: the alternative is credentials silently propagating to environments they were never scoped for.

Rotate like you mean it

Remember the 64 percent of 2022’s leaked secrets that still worked in 2026? Encryption at rest solves the leak. Rotation solves the lifetime.

Ankra supports rotating the organisation’s AGE key pair from the same Encryption settings page: generate a new pair, re-encrypt existing content with the new public key, confirm, and the old private key is destroyed. Do it on a schedule, and do it when someone with access leaves. The workflow exists so that rotation is an afternoon, not a quarter.

The same discipline applies to the credentials inside the encrypted files. A database password that has been encrypted in Git for three years is still a three-year-old password. SOPS makes rotation safe to execute through the normal GitOps flow: change the value, the diff shows one key changed, the deploy rolls it out, and the history shows exactly when and by whom.

The five rules

If you take nothing else from this post:

Secrets live in Git, encrypted. Anywhere else and you have two sources of truth, which is zero sources of truth.
Encrypt values, not files. Reviews only work when reviewers can read the structure.
No human touches the private key. Initialize once, let the platform hold it, let clusters decrypt automatically.
Plaintext fails the build. --strict-secrets in CI, on every pull request that touches cluster config.
Rotate on a schedule, not after an incident. Keys quarterly or on departure, credentials on their own clock.

A GitOps repository where every credential is ciphertext is a repository you can hand to a new hire, an auditor, or an AI agent without holding your breath. That is what the source of truth is supposed to feel like.

Set it up this afternoon: one click in Organisation Settings, Encryption, one flag in CI, done.

Get started: Create a free account on Ankra.

Join our community: Slack

Kubermatic vs Ankra: Do You Want to Run a Kubernetes Factory, or Just Ship Software?

Mark Shine — Sat, 04 Jul 2026 00:00:00 GMT

Kubermatic deserves more recognition than it gets. While American vendors dominate the Kubernetes management conversation, this Hamburg company has been quietly running one of the more technically interesting platforms in the space for the better part of a decade: an open source system that manufactures Kubernetes clusters at a density most competitors cannot match, with an Apache 2.0 community edition you can run without ever talking to sales. When European teams evaluating Ankra ask “why not Kubermatic, it is open source and it is from here,” that is a fair question from people asking the right things.

The answer has the same shape as our Spectro Cloud comparison, but with a twist that matters: Kubermatic Kubernetes Platform (KKP) is a factory for producing and hosting clusters, and you are the one who runs the factory. Ankra is a platform for running what lives on clusters, and nobody on your team operates the platform at all. The real question this post answers is not “which product has more features.” It is: does your organization want to be a Kubernetes provider, or a Kubernetes consumer?

What KKP actually is, and why the architecture is genuinely clever

KKP is a self-hosted platform for creating and operating fleets of Kubernetes clusters. Its architecture has three tiers: a master cluster holding users, projects, and credentials; seed clusters; and the user clusters your workloads actually run on.

The seed cluster is the clever part. When KKP creates a user cluster, it does not stand up dedicated control plane VMs. It runs the user cluster’s entire control plane, kube-apiserver, etcd, scheduler, controller-manager, as pods inside a namespace on the seed cluster. One seed hosts the control planes of many user clusters, packed like containers, because they are containers. That density is how KKP credibly claims to manage thousands of clusters: control planes stop being infrastructure you provision and become workloads the seed schedules.

Around that core, KKP is a complete Kubernetes-as-a-Service operation. It provisions user clusters on AWS, Azure, Google Cloud, OpenStack, vSphere, Nutanix, Hetzner, DigitalOcean, KubeVirt, and edge environments. It ships a self-service portal where project teams create their own clusters from admin-approved templates. It has a one-click multi-tenant monitoring, logging, and alerting stack (Prometheus, Loki, Cortex, Grafana) for user clusters. The 2.30 release from this April added Kubernetes 1.35 support, Gateway API, and better GPU handling for AI workloads. The community edition is genuinely open source under Apache 2.0; the enterprise edition adds multi-seed scaling and support, aimed, in Kubermatic’s own words, at organizations running more than 50 clusters.

If your organization’s product is Kubernetes itself, a hosting provider selling managed clusters, a telco running network functions, a central IT group offering internal KaaS to forty business units, KKP is one of the best-engineered answers available, and the only major one that is open source top to bottom.

The factory comes with a factory floor

Here is the part the architecture diagram does not show: someone has to run all of that, and that someone is you.

A production KKP deployment means you operate a master cluster, one or more seed clusters (multiple seeds require the enterprise edition), the platform’s own monitoring stack, its upgrades, its certificate rotation, its etcd health for every hosted control plane, and the capacity planning for seeds whose resource needs grow with every user cluster created. KKP’s own documentation is refreshingly honest about this: plan your seed sizing ahead, because control planes are workloads and workloads need somewhere to run.

None of that is a criticism. It is the deal, stated plainly: KKP transfers the cloud provider’s job to you, and in exchange you get sovereignty, density, and zero per-cluster fees to a hyperscaler. For a service provider, that trade is the business model. For a regulated enterprise that must own everything, the trade is compliance.

But if you are a product company with six or sixteen clusters, look at what you just signed up for. You now run a platform whose job is to run platforms, staffed by engineers you hired to ship your product. The factory produces clusters beautifully. Someone still has to sweep the factory floor, and the factory does not answer the question you actually had, which was never “how do I get more clusters.”

The question after the cluster exists

Ask the teams we work with what consumed last month, and cluster creation is never on the list. What is on the list: which Helm charts make a cluster production-grade and in what order, why staging drifted from production, how the new developer gets an environment without a ticket, what broke at 2 AM and which change broke it, where the audit trail is when the auditor asks.

That is the territory Ankra occupies, and it is territory KKP mostly leaves to you. KKP has addons and an application catalog for seeding user clusters with default tooling, and they work. But the lifecycle of what runs on the cluster, the dependency graph between your applications and their data services, the delivery pipeline, the incident workflow, the review process for changes, that is explicitly your problem in a KKP world, solved with whatever GitOps tooling and observability stack you assemble next to it.

Ankra’s Stacks make that lifecycle the platform’s core object: addons, manifests, and applications wired into a dependency graph, deployed in order by Ankra’s own engine, with every change from every surface (UI, CLI, API, Terraform, or the AI) written back to your Git repository. The AI on top is an operator, not a workload scheduler: it designs stacks, explains crashing pods against the change that broke them, posts root-cause incident reports to Slack, and proposes fixes as drafts a human approves. Same contract as always: propose, review, record.

And the platform itself is not yours to operate. The only thing running in your cluster is the Ankra agent, installed with one Helm command, making outbound connections only. No master cluster, no seeds, no platform monitoring stack, no upgrade weekends. The trade is the mirror image of KKP’s: you give up hosting the management plane, and you get back every hour your team would have spent being its SRE.

Two kinds of open, honestly compared

Kubermatic’s open source credentials are real and deserve respect. KKP CE is Apache 2.0, the repository is public, and you can run the whole platform forever without paying anyone. For teams with a hard open-source-only policy, that settles the evaluation, and it settles it in Kubermatic’s favor.

Be precise about what the freedom costs and what it protects, though. The freedom is operational: your data, your infrastructure, your hands on every component. The cost is that same sentence read back to you. Free-as-in-freedom software is not free-as-in-headcount, and a production KKP installation is a standing engineering commitment before the first user cluster serves traffic. Meanwhile the features that matter most at the scale where KKP shines, multiple seeds past 50 clusters, are in the paid enterprise edition anyway, so serious fleets end up in a commercial relationship regardless.

Ankra’s openness runs through a different channel: exit rights rather than source code. Every stack, every value, every dependency lives in your Git repository, written there continuously by the engine. Switch Ankra off tomorrow and you hold a complete, working GitOps description of your entire estate that any tool can reconcile. We made this argument in the Komodor post and it applies unchanged: the openness that protects you long-term is not being able to read the vendor’s code, it is being able to leave without losing your operational history. Ideally you want both. If you can only have one, take the exit rights.

On sovereignty, the honest note is that this is shared ground, not a differentiator: Kubermatic is German, Ankra is European too, and neither routes your operations through a US hyperscaler’s control plane. If EU data residency drove you to Kubermatic’s door, it does not argue against Ankra’s.

Side by side

Dimension	Kubermatic KKP	Ankra
What it manufactures	Kubernetes clusters, control planes hosted at density on seeds	Running software: stacks, delivery, operations on any cluster
Who operates the platform	You: master, seeds, platform MLA, upgrades	Ankra; your side is one outbound agent per cluster
Deployment model	Self-hosted, on your infrastructure	SaaS platform, agent in your clusters
Open source	CE under Apache 2.0; EE commercial (multi-seed, 50+ clusters)	Platform is commercial; your config lives in your Git, CLI and agent workflows public
Source of truth	KKP (CRDs on master/seed; templates in platform)	Your Git repository, written back by the engine
Delivery of applications	Addons and app catalog; bring your own GitOps	Core product: dependency-graph stacks, drafts, approvals
AI	GPU and AI workload support for clusters (2.30)	AI operator: designs, investigates, proposes reviewable fixes
Sweet spot	Service providers, internal KaaS at fleet scale, sovereignty mandates	Product and platform teams shipping software on Kubernetes
Cost of entry	Engineering time to stand up and run the platform	Free 30 worker vCPUs, self-serve, minutes to first import

Where Kubermatic is the right call

Same rule as every comparison we publish: there are evaluations Ankra should lose.

If you sell Kubernetes, as a regional cloud, a managed service provider, or a telco, KKP’s seed architecture is close to purpose-built for your margins: control planes as dense workloads instead of dedicated VMs is a real cost structure advantage at hundreds of clusters. If your mandate is a fully self-hosted, open source management plane inside your own perimeter with no SaaS dependency at all, KKP delivers that and Ankra’s SaaS model does not. If central IT is building an internal cluster vending machine for dozens of autonomous teams and has the platform engineers to staff it, KKP is a mature, honest choice, and you will not resent it.

And the quiet truth from the Spectro post applies here too: the two are not mutually exclusive. A user cluster minted by KKP is a conformant Kubernetes cluster, and one Helm install of the Ankra agent later it is on Ankra’s graph, with stacks, AI operations, and Git write-back running on top of Kubermatic’s plumbing. The factory makes the cluster. The workshop ships the product. Some organizations legitimately need both floors of that building.

The move

Kubermatic answers “how do we produce and host many Kubernetes clusters, ourselves, at density, with open source.” It answers it better than almost anyone. Ankra answers the question that starts the moment a cluster exists and never goes away: “how does this team ship, run, understand, and fix what is on it, without becoming a platform department.”

If you are staffing a factory, evaluate KKP seriously. If you are trying to get out of running one, create a free account, import any cluster you already have, KKP-minted ones included, and watch the AI propose its first stack this afternoon. The free 30 vCPU allowance covers the whole experiment.

Get started: Create a free account on Ankra.

Join our community: Slack

The Early-Stage Infrastructure Playbook: Kubernetes Before Your First DevOps Hire

Mattias Åsell — Sat, 04 Jul 2026 00:00:00 GMT

There is a piece of advice every technical founder has heard, usually delivered with the weary confidence of someone who has seen things: “Do not touch Kubernetes until you can afford a platform team. Use a PaaS, ship your product, worry about infrastructure later.”

I want to take that advice seriously, because it was correct when it was coined. Around 2019, adopting Kubernetes as a three-person startup meant one founder becoming a part-time YAML archaeologist: hand-rolling cluster provisioning, assembling ingress and cert-manager and monitoring from blog posts, debugging CrashLoopBackOffs at midnight with nobody to ask. The tax was real, the product suffered, and the PaaS alternative was genuinely the better trade.

But advice is a compressed conclusion, and conclusions rot when their premises change. Every load-bearing premise under “not yet” has changed. This post is the updated playbook: what an early-stage team should actually set up, what it should refuse to set up, and the math on what it costs, which rounds to zero.

What the old advice was really about

Strip the meme away and the 2019 advice encoded three true statements:

Kubernetes itself was expensive to operate. Control planes, upgrades, networking, certificates: a standing job before your first deploy.
The ecosystem around it was assembly required. Fifteen Helm charts, in the right order, with values nobody documented, glued together by the one person who understood them.
Operating it required expertise you could not afford. The person who could run it well cost six figures and was better spent building product, if you could hire them at all.

Notice that none of these statements is about Kubernetes being wrong for small workloads. Kubernetes was always a fine way to run a web app, a database, and a queue. The objection was never the destination; it was the price of the road. So the honest question in 2026 is not “is Kubernetes overkill,” it is “what does the road cost now.”

What changed

The cluster became a commodity. Managed control planes are table stakes on every cloud, and after the Hetzner repricing we did the math on this blog: a production-shaped cluster on European cloud, three workers, a control plane, ingress, runs about €65 a month. Ankra provisions one on Hetzner, OVHcloud, or UpCloud in minutes, or imports whatever you already have, including the k3s cluster on the OrbStack instance of your laptop. Premise one is gone.

The assembly became a product. The fifteen-chart scavenger hunt is what Stacks exist to kill: addons, manifests, and your applications wired into a dependency graph, deployed in order, versioned in Git. Describe the stack once, in the builder or by asking the AI for it, and clone it to every environment you will ever have. Premise two is gone.

The expertise became software. This is the one people have not internalized yet. The judgment you were hiring for, which values are load-bearing, why the pod is crashlooping, what that OOM trend means, is exactly what an AI operator with full cluster context does now: root-cause analysis delivered to Slack, proactive insights before things page you, plain-language explanations of anything on the graph. And because every AI change lands as a draft a human approves and Git records, you get the senior engineer’s output without handing production to an intern with root. Premise three is the reason this post exists.

The advice was “not yet” because the road was expensive. The road is now paved, and the toll for a team your size is literally zero: Ankra’s free plan includes 30 worker vCPUs, unlimited clusters, unlimited team members, and the AI, which covers a seed-stage company’s entire footprint with room to spare.

But why not just stay on a PaaS?

Because the PaaS trade has a shape, and early-stage teams keep rediscovering its edges at the worst possible moments.

The PaaS deal is: pay a premium per compute unit in exchange for not thinking about infrastructure. At day one the premium is small and the not-thinking is priceless, which is why it is a fine place to start and I will not pretend otherwise. The edges show up later, on a schedule you do not control. The pricing curve bends steeply exactly when your usage grows, which is exactly when you have the least time to re-platform. The features you eventually need, a VPC peering, a compliance requirement, a background worker with weird resource needs, a customer who demands EU-only data residency, sit on the other side of a wall the PaaS did not build doors into. And the migration off, when it comes, lands as an unplanned quarter of engineering during your growth phase. Ask anyone who was on Heroku’s free tier in November 2022 how much notice you get when a platform changes its mind.

Kubernetes through a platform inverts the shape: slightly more thinking on day three, and no wall, ever. The same stack that runs your MVP on one Hetzner node runs your Series B fleet across providers, because the abstraction underneath is the industry’s, not a vendor’s. There is no cliff where you outgrow it, and, as we argued in the cost post, portability is also the only durable defense against every vendor’s future pricing decisions, PaaS and cloud alike.

The old model made you choose: PaaS simplicity now and a migration later, or Kubernetes power now and a platform tax later. The entire point of the current generation of tooling is that this fork is gone.

The week-one playbook

Here is the concrete setup we would give any two-to-six person team, and roughly an afternoon of work.

One cluster, boring shape. Three CX43 workers and a small control plane on Hetzner through Ankra (about €65 a month), or import the managed cluster your cloud credits pay for. Do not agonize over this choice; the whole point of what follows is that it is cheap to change later.

One GitOps repo from minute one. Connect it when you import the cluster. Every change you ever make, from the UI, the CLI, or the AI, becomes a commit. You are two people today and you think you will remember why you changed that value. You will not. In eighteen months, when you hire, the repo is the onboarding doc; in three years, when a customer’s auditor asks, it is the audit trail. This costs nothing to start and cannot be retrofitted.

One production stack. Ingress-nginx, cert-manager, your app, its database, a monitoring addon. Ask the AI to draft it (Cmd+J, “a production stack for a web app with Postgres, Redis, and TLS on this domain”), review the graph it proposes, deploy. Pin every chart version. Put every secret through SOPS encryption from day one, because day one is when it is free and day four hundred is when the plaintext password in Git becomes an incident report.

Alerts to the Slack you already live in. Wire the webhook so that when something breaks, the message contains a root cause, not a red X. For a team with no on-call rotation, because the rotation is “both of you,” an alert that arrives pre-investigated is the difference between a five-minute fix and a lost evening.

Environments by cloning, not by ceremony. When you need staging, or a demo environment for that enterprise prospect, clone the production stack and let variables resolve the domains and sizes. The five-person team in that post ran laptop-to-production on this pattern; at your size it is even simpler.

That is the whole thing. No service mesh, no multi-cluster topology, no platform team, no hire.

The anti-playbook: what to refuse to build

Early-stage infrastructure fails in two directions, and the second is underrated. Under-engineering gives you the plaintext secret and the untracked hotfix. Over-engineering gives you a three-person company operating a service mesh. Refuse, specifically:

A second cluster you do not need. Environments are stacks with different variables, not more control planes. Scale to zero what idles; deleting the Kubernetes layer keeps the config and stops the bill.
A self-hosted observability empire. One monitoring addon in the stack. You do not need a Thanos deployment; you barely need the Prometheus-vs-VictoriaMetrics decision yet, and when you do, it is a values change, not a project.
A DevOps hire whose job is the platform. The moment infrastructure becomes someone’s whole job at a six-person company, a founder has quietly decided to build a platform company inside their product company. Hire when there is product-shaped work for the person, and let the AI hold the pager until then.
A migration you are saving up for. The most expensive infrastructure pattern in startups is “we will fix it after the fundraise.” You will not; you will be busier. Start on the thing that does not need leaving.

The runway math, said out loud

Infrastructure decisions at seed stage are runway decisions wearing overalls. The comparison that matters is not €65 versus €0; it is the fully loaded alternatives:

The DevOps hire you defer is six figures a year in most markets, your single largest line item after founders, spent on someone whose output your customers never see.
The founder-time tax of DIY is worse: every hour a technical founder spends debugging ingress annotations at midnight is an hour of the most expensive labor your company will ever employ, spent below its pay grade.
The PaaS premium plus eventual migration is the quiet third option, fine until the invoice bends or the wall appears, both on someone else’s schedule.

Against those: a €65 cluster, a platform whose free tier covers your whole footprint, and an AI that does the investigating. The startup advice canon says do things that do not scale. Your infrastructure is the one place where the same effort now buys you the thing that does.

The unfair advantage nobody mentions

One more thing, and it is the one I would actually lead with if this were a pitch instead of a playbook. Early-stage companies do not lose to incumbents on product ideas. They lose on capacity: five people cannot cover product, sales, and a 3 AM pager the way fifty people can. Every hour of operational toil you delete is transferred directly to the only work that differentiates you.

A five-person team with a senior DevOps agent on call, boring GitOps rails underneath, and infrastructure that clones instead of being rebuilt is not “doing Kubernetes early.” It is competing with a thirty-person company’s operational capacity on a five-person payroll. That is the actual reason the 2019 advice needs retiring: not because Kubernetes got fashionable, but because the leverage flipped sides.

Start it this weekend. Create a free account, import or provision one cluster, ask the AI for your first stack, and connect the Git repo before you deploy anything. Thirty free vCPUs is more infrastructure than most companies need before their Series A, and the setup fits in the afternoon between two customer calls.

Get started: Create a free account on Ankra.

Join our community: Slack

Kubernetes Logging in 2026: Elasticsearch, OpenSearch, Loki, or Just Renting It from Your Cloud

Mark Shine — Sat, 04 Jul 2026 00:00:00 GMT

Nobody chooses a logging stack. Teams choose a metrics stack deliberately, argue about it, benchmark it. Logging just sort of happens: whatever the cloud gives you, or whatever chart someone installed in 2022, quietly ingesting gigabytes a day until the bill or the OOM forces the first real decision. Then the team discovers that logging solutions differ more than almost any other category of infrastructure, and that the differences were all downstream of one question nobody asked at the start.

The question: how much of your log data do you index?

Index everything, and every query is instant but you run a search engine cluster. Index almost nothing, and storage is nearly free but queries do the work at read time. Rent the whole problem, and someone else operates it while you pay per gigabyte forever. Every serious option is one of these three answers wearing a product name. Here is the comparison, with the numbers.

Family one: index everything (Elasticsearch and OpenSearch)

Elasticsearch and OpenSearch are the same answer from two vendors, and their shared story is quick to tell: both are built on Apache Lucene, both tokenize and index every field of every log line at ingest, and both give you the payoff of that work: full-text search across terabytes in milliseconds, arbitrary aggregations, and a mature dashboard layer (Kibana, or OpenSearch Dashboards).

They split in 2021, when Elastic moved off Apache 2.0 to fight AWS’s managed offering, and AWS forked the last open commit into OpenSearch. Five years on, both halves of the fork are healthy, and the licensing has settled into something you can finally reason about. Elasticsearch (9.4 today) added AGPLv3 back as an option in late 2024, so it is open source again by OSI rules, though the trademark and roadmap remain Elastic’s. OpenSearch (3.7 as of June) is Apache 2.0 under the OpenSearch Software Foundation, part of the Linux Foundation since 2024, with genuinely multi-party governance. If the 2021 fork was about trust, 2026’s honest scorecard says: Elastic rebuilt a lot of it, and OpenSearch built the more neutral home.

Functionally they remain close enough that choosing between them is mostly a governance and ecosystem question, not a capability one. Elasticsearch moves faster on vendor-led features (ES|QL, the vector search push); OpenSearch has been closing gaps and adding its own (3.7 even ships native PromQL support, of all things).

What both make you pay is the same tax: you are now operating a distributed search engine. JVM heap sizing, shard strategy, index lifecycle management to tier old indices off hot storage, cluster upgrades, and the RAM appetite that comes from holding an inverted index of everything you have ever logged. The rule of thumb that survives every generation of this stack: the index costs as much as the data. For a Kubernetes cluster logging 50 GB a day, an Elastic-family deployment is typically three dedicated nodes and someone who knows what refresh_interval does.

That tax buys real things. If your logs are a product surface (search-heavy support tooling, audit trails a customer queries), if you run security analytics or SIEM workloads over them, or if analysts genuinely ask arbitrary questions of full log content all day, the Elastic family is the correct tool and nothing else on this page truly substitutes for it.

Family two: index almost nothing (Grafana Loki)

Loki’s founding insight is that most log queries are not arbitrary. An engineer debugging production does not search “all logs containing this string ever.” They look at one service, in one namespace, over the last hour, and then grep. So Loki indexes only a small set of labels (namespace, app, pod, the things you filter by), and stores the log content itself as compressed chunks in object storage. The index is tiny. The storage is S3-priced. The grep happens at query time, brute-forced in parallel over just the chunks your labels selected.

The consequences, good and bad, all follow from that one design choice:

Cost collapses. Object storage plus a small index means Loki’s cost per retained terabyte is a fraction of an indexed store’s, and retention becomes a bucket lifecycle policy instead of a cluster expansion.
Kubernetes fit is natural. Labels are already how Kubernetes thinks. Your LogQL query mirrors your kubectl mental model, and Grafana displays logs next to the metrics they correlate with.
Queries are only as fast as their label selection. A well-labeled query over an hour is instant. An unbounded “find this request ID across everything, last 30 days” is a cluster-wide brute-force scan, exactly the query the Elastic family eats for breakfast.
Cardinality discipline is mandatory. Labels are the index; a high-cardinality label (user ID, request ID) explodes the index and defeats the design. Those values belong in the log line, found by grep, not in labels.

One operational note that matters this year specifically: Promtail, the log shipper most Loki tutorials (ours included) grew up with, reached end-of-life on March 2, 2026. Its replacement is Grafana Alloy, Grafana’s OpenTelemetry Collector distribution, which carries Promtail’s capabilities plus metrics, traces, and OTLP in one agent, and ships a one-command config converter. If you deploy Loki today, deploy Alloy beside it, not Promtail; if you run Promtail today, the migration is a config conversion, not a re-architecture.

Loki is AGPLv3, single-vendor (Grafana Labs) but with the structural mitigation that your data sits in your object storage in an open format, and the ecosystem around it is the largest in observability.

Family three: rent the whole problem (GCP Logs Explorer and friends)

Every hyperscaler will make logging entirely not your problem: an agent ships everything to a managed service, retention is a setting, the query UI is a browser tab, and the operational burden is zero. Google’s Cloud Logging with its Logs Explorer is the cleanest example, so let us use its actual prices.

Cloud Logging charges $0.50 per GiB ingested after a free 50 GiB per project per month, with 30 days of storage included; keeping logs longer costs $0.01 per GiB-month, and querying costs nothing extra. Run the math on a modest 200 GB per month: 150 paid GiB is $75, plus $3 if you keep 90 days. Call it $78 a month for a fully managed, quite good, log system. At 1 TB per month the same math lands around $500. AWS CloudWatch Logs and Azure Monitor price with the same shape: cheap-looking per-GB rates that scale linearly with volume, forever.

The catches are structural, not hidden. The bill tracks your log volume, and log volume grows with traffic, debug verbosity, and team size, none of which anyone budgets for. The query languages and dashboards are provider-specific, so your runbooks and muscle memory are non-portable. And the moment you have clusters on more than one provider, or on Hetzner where there is no managed logging at all, you either run two logging systems or none. Cloud logging is the right answer for single-cloud teams below a few hundred gigabytes a month, and it gets quietly wrong as either of those conditions decays.

The same 1 TB, three ways

Rough numbers, deliberately conservative, for 1 TB of logs a month with 90-day retention:

Approach	What you run	Monthly cost shape
GCP Cloud Logging	Nothing	About $495 ingestion plus $19 retention, roughly $515, scaling linearly with volume forever
Elasticsearch / OpenSearch	3+ node cluster, ILM, someone who owns it	Roughly $300 to $600 of compute and disk (more on a hyperscaler), plus the real cost: the engineer-hours
Loki + Alloy	2 small pods plus object storage	Tens of euros of compute, single-digit euros of object storage, cardinality discipline as the price of admission

The pattern generalizes: cloud logging wins below roughly 100 to 200 GB per month on convenience, Loki wins on cost from there to nearly any scale if your query patterns are label-shaped, and the Elastic family wins whenever the queries themselves are the product, at whatever the cluster costs, because nothing else answers them.

The decision framework

Four questions settle almost every case:

Are logs a product or a debugging tool? Product (search features, SIEM, customer-facing audit): Elastic family. Debugging: keep reading.
Is all your Kubernetes on one hyperscaler, under ~200 GB per month, and likely to stay that way? Yes: use the cloud’s logging, stop here, spend the saved attention elsewhere.
Multi-cloud, cost-sensitive, or European-cloud clusters? Loki with Alloy, in your object storage. This is the default answer for most Kubernetes teams in 2026, and it is why it is the default in our stacks too.
Do you need both? Common at scale: Loki for the debugging firehose, a small Elastic-family index for the subset that needs real search (security events, audit logs). Route with Alloy; it exists to do exactly this kind of splitting.

Where Ankra sits in this

Deliberately underneath, and deliberately neutral, the same position as with metrics backends.

First: for the most common thing teams use logs for, live debugging, you need none of the above to start. The Ankra agent streams pod logs directly from any managed cluster, and the AI reads them, along with events, manifests, and the stack change that broke things, when it investigates an incident or explains a crashloop. A logging backend is for retention and search across time, not for “why is this pod crashing right now.” Plenty of small teams run on live logs plus alerts for months, which is exactly the right amount of infrastructure for their stage.

When you do want retention, the logging stack is an ordinary Stack: Loki and Alloy (or an OpenSearch cluster, if your answer to question one was “product”) as pinned-version addons with the namespace first and dependencies wired through parents, deployed by Ankra’s engine, committed to your Git repo, and cloned to every cluster that needs it. Grafana credentials and shipper tokens go through SOPS, because plaintext secrets in a values file is how logging stacks end up in security postmortems.

The stack is yours to choose. The part where someone has to remember how it was wired, keep it identical across environments, and explain at 3 AM what changed, that part is the platform’s job.

Pick your family with the framework above, then deploy it once, as a stack, and let it be boring. Start free, and if you are unsure, the AI will happily draft you the Loki stack to react to.

Get started: Create a free account on Ankra.

Join our community: Slack

The Kubernetes Security Work That Actually Matters in 2026

Mattias Åsell — Sat, 04 Jul 2026 00:00:00 GMT

In January, the Kubernetes Steering Committee and Security Response Committee published a joint statement with a sentence you rarely see from a project of that size: “Half of you will be affected. You have two months left to prepare.” They were talking about the retirement of ingress-nginx, the controller that routes traffic into roughly half of all cloud native environments. The deadline passed in March. There will be no more releases, no bug fixes, and no security patches, ever, for the component standing between the internet and half the world’s clusters.

That is what Kubernetes security actually looks like in 2026. Not zero-days in the kubelet, not container escapes from a conference talk. A load-bearing component nobody was paid to maintain, credentials nobody rotates, changes nobody reviews. When we look at real clusters, the same five findings account for nearly all of the actual risk, and none of them are exotic. Here they are, each with the incident that proves it is real and the fix that closes it.

Finding 1: A load-bearing component nobody maintains

The ingress-nginx story deserves to be told properly, because it is the security lesson of the year. The project served about 50 percent of cloud native environments and was maintained, for years, by one or two people in their free time. In March 2025 that arrangement produced IngressNightmare (CVE-2025-1974), a CVSS 9.8 unauthenticated remote code execution that could hand an attacker the whole cluster. Eight months later the project announced its retirement, and since maintenance ended this March, new vulnerabilities have already been catalogued against a codebase that will never be patched again. Existing deployments keep working, which is precisely the problem: nothing breaks, so nothing forces the migration, so the exposure quietly compounds.

Check whether this is you, right now:

kubectl get pods --all-namespaces --selector app.kubernetes.io/name=ingress-nginx

If that returns pods, you are running retired software on your network edge, and the community’s recommendation is to migrate to Gateway API or another maintained ingress controller. None of the alternatives are drop-in replacements, which is exactly why this belongs at the top of the backlog rather than the bottom.

The general lesson is bigger than one controller. Your cluster’s security posture is the union of every addon on it, and most teams cannot answer “what versions of what are running where” without an afternoon of kubectl archaeology. This is a place where structure beats vigilance: in Ankra, every addon is a pinned-version node in a stack, so the inventory question is a glance, the AI’s proactive insights flag vulnerable and end-of-life addon versions before they page you, and the migration itself is a stack change that rolls out with dependency ordering and a Git record instead of a hand-run weekend.

Finding 2: Kubeconfigs are the new SSH keys, and they are everywhere

Ask a team how many valid kubeconfigs exist for their production cluster and watch the faces. There is one on every engineer’s laptop, including two who left last year. There is one in CI, with cluster-admin because scoping it down was a ticket nobody took. There is one in a shared vault whose password is in Slack. Static, long-lived, rarely scoped, never rotated: Kubernetes credentials in most organizations are exactly where SSH keys were in 2010.

The fix is the same one SSH eventually got: stop distributing static credentials at all. Access should be identity-based, short-lived, and scoped to the task. This is what Ankra’s Cluster Access grants implement: an admin maps a user to a role (view, edit, admin, cluster-admin) at a scope (whole cluster or a single namespace), the platform reconciles that grant into native Kubernetes RBAC on the cluster, and the user’s kubectl flows through an authenticated proxy using short-lived tokens minted on demand. Nobody holds a static cluster secret, offboarding is deleting a grant, and the QA lead who needs to check one namespace gets view on that namespace instead of the keys to production.

ankra cluster access grant sofia@example.com --role view --namespace staging

The same principle applies to the machines: the pipeline agents we wired in the CI post hold one revocable API token, not a kubeconfig, and there is no kubeconfig anywhere in the pipeline to leak.

And one structural bonus that costs you nothing: the Ankra agent connects outbound only. Importing a cluster requires no inbound firewall rule and no publicly reachable API server, which removes the single most common Kubernetes exposure, the control plane endpoint that Shodan can find, from the board entirely.

Finding 3: The secret is in Git, and Git is forever

We wrote a full post on this, Secrets Belong in Git. Plaintext Does Not., so here is only the shape: 28.65 million hardcoded secrets landed in public GitHub commits last year, AI-assisted commits leak at roughly double the baseline rate, and the Sisense breach demonstrated what one plaintext credential in one private repo converts into. The fix is SOPS with AGE encryption, initialized once at the organisation level, with --strict-secrets in CI so a plaintext credential is a failed build instead of a code review comment. If you read nothing else on this list, read that one; secrets remain the most reliably exploited finding on it.

Finding 4: Changes nobody reviewed, by humans or their agents

Most production incidents with a security flavor do not start with an attacker. They start with a change: a values edit applied straight to the cluster, a hotfix kubectl’d in during an incident and never written down, a well-meaning AI agent with production credentials doing something “entirely on its own initiative,” as the PocketOS postmortem memorably put it after an agent destroyed their production database and backups in nine seconds. The 2025 wave of AI-agent incidents, which we dissected when we built our guardrails, all reduce to the same missing property: no separation between proposing a change and applying it, and no audit trail the actor could not rewrite.

The fix is a property of your delivery system, not a policy document. Every change, from every surface and every actor, human or AI, goes through the same gate: it lands as a draft or a pull request, a human approves it, and Git records it immutably. In Ankra this is the architecture rather than a convention: the UI, CLI, API, Terraform, and the AI all write through the same engine, the engine writes to Git, and the cluster reconciles from Git. When the auditor asks “who changed the ingress config and who approved it,” the answer is a commit hash, not an interview process. Drift, the gap between what Git says and what the cluster runs, is detected and self-healed by the engine’s reconcile loop, so the unreviewed hotfix does not quietly become the new production state.

Finding 5: `latest` is not a version, and floating tags are a supply chain

Every unpinned dependency in your cluster definition is a change you have agreed to accept, sight unseen, at a time of someone else’s choosing. A floating chart version means next week’s deploy installs code nobody reviewed. A latest image tag means your rollback target is a moving object. The supply chain attacks that make headlines, poisoned packages, hijacked maintainer accounts, all need a victim who auto-ingests updates, and floating versions are the standing invitation.

The fix costs nothing but discipline: pin exact chart_version on every addon, immutable tags on every image, so that a Git commit fully determines what runs, and an upgrade is a visible diff that says 65.1.1 became 66.0.2, reviewable like any other change. Ankra’s validation checks your cluster definition against your connected registries on every pull request, which turns “does this chart version actually exist where we think it does” into a CI answer instead of a deploy-time surprise.

While we are here, the honorable mention that nearly made the list: unauthenticated internal dashboards. Researchers found roughly 336,000 Prometheus endpoints facing the public internet in late 2024, leaking credentials and API keys through a tool everyone assumes is behind a firewall. If a human can open your Grafana or Prometheus without authenticating, so can everyone else. (Ankra reads metrics through the agent, so powering dashboards and AI analysis requires no publicly exposed endpoint at all.)

What did not make the list, and why

Notice what is absent: service meshes, runtime eBPF sensors, admission controller frameworks, CVE scanners producing four-digit finding counts. Not because those tools are bad, but because they are layer-seven answers bought by teams that have not done the layer-one work. A mesh’s mTLS does not help you when the ingress controller in front of it is unpatched by design, the intern’s kubeconfig is on a stolen laptop, and the database password is in a public repo. Sequence matters. The five findings above are cheaper than any security product, and they close the doors that real incidents actually walked through.

There is also a pattern across all five worth naming. None of them are fixed by vigilance, training, or a wiki page. All of them are fixed by structure: making the safe path the default path, so the insecure option requires effort rather than the secure one. Pinned versions in a stack, grants instead of kubeconfigs, encryption at save time, drafts before deploys. Security that depends on everyone remembering everything fails on the first busy Friday. Security that is a property of the delivery system does not care how busy Friday gets.

The checklist

Run these five checks this week; none takes more than an hour:

kubectl get pods -A --selector app.kubernetes.io/name=ingress-nginx. If it returns pods, migration planning starts today.
Count the valid kubeconfigs for production. If the answer is not “zero static ones,” move to grant-based, short-lived access.
ankra cluster validate -f cluster.yaml --strict-secrets (or grep your repo for password:). Encrypt what you find, then make the check a CI gate.
Pick a recent production change and reconstruct who made it, who approved it, and where that is recorded. If the answer involves asking someone, your audit trail is a person.
Grep your cluster definition for latest and unpinned versions. Pin them, and let every future upgrade be a reviewable diff.

Five findings, five checks, and none of them require a security team, which is rather the point: a platform that makes them structural means the five-person company and the fifty-person one pass the same review.

Get started: Create a free account on Ankra.

Join our community: Slack

GitOps Without ArgoCD: Inside Ankra's Native Deployment Engine

Mattias Åsell — Sat, 04 Jul 2026 00:00:00 GMT

If you created an Ankra cluster after our May 2026 release, there is something missing from it: ArgoCD. No argocd-server, no repo server, no application controller. Your stacks still deploy, drift still self-heals, SOPS values still decrypt, and Git is still the source of truth. The seven pods that used to do the last mile are simply not there.

This post is the engineering story behind that change. Not a takedown of ArgoCD, which served us and thousands of our clusters well, but an honest account of what it cost, what we built instead, how the migration adopts your live resources without restarting a single pod, and what the native engine does not do yet.

The one-sentence version: GitOps is a contract, not a tool. The contract says desired state lives in Git, something reconciles reality against it, and drift gets corrected. Nothing in that contract says the something has to be a seven-pod control plane inside every cluster you operate.

What ArgoCD was costing us

Ankra’s engine has always owned the hard parts of delivery: the dependency graph, the ordering, the lifecycle, the Git write-back. ArgoCD’s job in our architecture was narrower than in most shops, essentially the last mile: render the chart, apply it, report health.

For that last mile, every single cluster carried:

Seven pods. argocd-server, argocd-application-controller, argocd-repo-server, argocd-redis, argocd-applicationset-controller, argocd-notifications-controller, and argocd-dex-server. Roughly 500 to 800 MB of RAM, on every cluster, forever.
A JWT lifecycle. The Ankra agent authenticated to ArgoCD’s API with tokens that had to be created, cached, refreshed, and rotated. An entire class of “deploy failed because auth expired” incidents existed only because of this hop.
An extra round trip for status. Reading addon health meant asking ArgoCD what it thought, over a proxied API call, instead of the agent just knowing.

On a beefy cloud cluster you can shrug at 800 MB. On the fleets we increasingly see, 50 warehouse edge nodes, a k3s cluster on a Raspberry Pi, an OrbStack cluster on a developer laptop, the overhead tax was often bigger than the workloads. And after June’s Hetzner repricing, 800 MB of platform overhead per cluster is not an abstraction; it is the difference between a node size.

There was a subtler cost too. When something went wrong, the answer lived inside ArgoCD’s abstractions. You debugged an Application CR, sync waves, and a repo server cache, when what you wanted to debug was a Helm release.

What the native engine is

The native engine (ankra_native) deletes the middleman. The Ankra agent, which already runs in every cluster and is already the only writer of cluster state, drives the Helm CLI directly: install, upgrade, reconcile, roll back. No shell interpolation, a single wrapped subprocess with an argument list, one release operation at a time per release, serialised through a Kubernetes Lease.

The consequence that matters most day to day: releases land in standard Helm storage, the helm.sh/release.v1 Secrets every Helm tool understands. Which means this works, on any Ankra-managed cluster, with no Ankra tooling involved:

helm list -A
helm history grafana -n monitoring
helm get manifest grafana -n monitoring
helm rollback grafana 4 -n monitoring

Your cluster is debuggable with the ecosystem’s standard tools at 3 AM, even if Ankra’s control plane were unreachable. We consider that a feature on the same tier as any dashboard we could build. There is no proprietary release format, no CR to reverse-engineer, no cache to invalidate. A Helm release is a Helm release.

The engine is selected per addon, with a cluster-level default. New clusters default to native; existing clusters keep ArgoCD until you choose to migrate. Mixed states are fine and normal, which is exactly what makes the migration boring, in the good way.

The reconcile loop, concretely

“We reconcile drift” is easy to say and easy to fake with a timer and a prayer. Here is what the native engine’s sweep actually does, every 60 seconds per cluster:

Recovers stuck releases. Any release sitting in a pending-install, pending-upgrade, or pending-rollback state for more than ten minutes, the classic Helm wedge that usually means a human with --force is coming, gets an automatic fresh helm upgrade --install --atomic. If that fails, the engine rolls back to the last known deployed revision.
Detects drift against the rendered truth. The live resources are fetched in parallel and diffed against helm get manifest, with Helm-injected metadata stripped from both sides so you are comparing intent against reality, not annotation noise.
Self-heals when told to. If drift is found and the addon’s sync_policy.self_heal is on, the addon is marked for update and the reconciler re-runs the upgrade. Someone’s manual kubectl edit quietly stops being the state of production.
Prunes orphans when told to. With auto_prune enabled, resources whose UID is no longer in the rendered manifest are deleted after apply, so removed chart resources do not accumulate as archaeology.
Proves it is alive. Every sweep publishes a heartbeat with the sweep duration and the number of addons inspected, and the platform exposes a reconcile-loop lag metric per cluster. A reconciler you cannot monitor is a reconciler you get to discover has been dead for a week.

Sync windows survived the migration from ArgoCD too: give an addon a cron expression, a duration, and a timezone, and create, update, and self-heal operations only run inside the window. Outside it, the job returns immediately as cancelled with an explicit “outside sync window” message rather than silently queueing into your Friday evening.

Feature parity, honestly

We held the migration until the answer to “what do I lose?” was almost nothing:

Capability	ArgoCD engine	Native engine
Self-heal on drift	Yes	Yes, per-addon `sync_policy.self_heal`
Prune orphaned resources	Yes	Yes, `auto_prune` plus post-apply orphan sweep
Ignore-differences rules (JSON pointers, jq paths, field managers)	Yes	Yes
Sync windows	Yes	Yes, cron plus duration plus timezone
Custom retry policy	Yes	Yes
Strict health checks	Yes	Yes
Revision history limit	Yes	Yes, `--history-max`
SOPS-encrypted values	Yes, via helm-secrets plugin	Yes, direct `sops --decrypt`
GitHub, Bitbucket Cloud, Bitbucket DC	Yes	Yes
Multi-source applications	Yes	Not yet, single source only

The SOPS row deserves a sentence, because it connects to the secrets post. Under ArgoCD, decryption ran through the helm-secrets plugin on the repo server, with the AGE key deployed to the cluster as a Kubernetes Secret. Under the native engine, the agent fetches the encrypted values file from your Git provider, pulls the key from the platform for just that render, runs sops --decrypt itself, and hands Helm the plaintext. The private key is never stored on the cluster; it exists in a locked-down temporary file for the seconds the decryption takes and is deleted immediately after. Fewer places a key exists is the whole game in secrets management, and this removed one.

One honest limitation on that path: today the native engine decrypts the values file referenced by the addon’s configuration. If you split encrypted values across several files on one addon, consolidate them into that file, or keep that addon on the ArgoCD engine until multi-file support lands.

The migration: adoption, not reinstallation

The part everyone fears about switching deployment engines is the switch. Here is what per-addon migration actually does:

helm upgrade --install --take-ownership

Helm’s take-ownership flow adopts the live resources into Helm storage. Resource UIDs do not change. No pod is recreated. No traffic blips. The only on-cluster mutation is metadata: the app.kubernetes.io/managed-by=Helm label and the release-name and release-namespace annotations. The ArgoCD Application CR is deleted only after the Helm release verifies as deployed.

Before anything runs, the platform shows a per-addon eligibility report: addons whose live resources include kinds the native health rules do not cover yet get flagged, in-flight ArgoCD syncs block until they settle, and multi-source applications are marked not yet migratable. The migration runs per addon, in parallel up to a small concurrency limit, and is resumable across control-plane restarts. Until you explicitly decommission ArgoCD on the cluster, any individual addon can be rolled back to the ArgoCD engine.

And when the last addon has migrated, decommissioning deliberately stops short of uninstalling ArgoCD. Plenty of teams run their own application delivery through the same ArgoCD instance, outside of Ankra’s addons. We clear our managed credentials and step away. Your ArgoCD, your call.

Why this matters beyond the RAM

The resource savings are real but they are the least interesting part.

Debuggability became standard. Every question about a deployment now has a helm answer. Your existing runbooks, your muscle memory, your incident tooling all apply. The abstraction between you and your releases went from “a CD system’s data model” to “the package manager you already use.”

The failure surface shrank. Every component in the delivery path is a component that can be the incident. Removing the ArgoCD hop removed its API, its tokens, its Redis, and its cache from the list of things that can be why production did not update.

The contract stayed identical. This is the point I care most about. Stacks, drafts, approvals, the dependency graph, Git write-back, the AI’s propose-review-record mandate, none of it changed, because none of it ever lived in ArgoCD. It lives in Ankra’s engine and in your repository. Swapping the last-mile executor under a well-defined contract is exactly the kind of change a platform should be able to make without its users rewriting anything. That it shipped as a settings-page button rather than a migration project is the architecture working as intended.

If you run new clusters, you are already on the native engine and there is nothing to do. If you have existing clusters on ArgoCD, the Migrate to native engine button is in your cluster settings, the eligibility report will tell you if anything needs attention first, and the deployment engines documentation covers the details.

Seven fewer pods per cluster, one fewer credential lifecycle, zero fewer guarantees.

Get started: Create a free account on Ankra.

Join our community: Slack

Prometheus vs VictoriaMetrics: When the Default Stops Being the Answer

Mark Shine — Sat, 04 Jul 2026 00:00:00 GMT

Every Kubernetes cluster we see starts its monitoring life the same way: kube-prometheus-stack, Grafana, a dashboard someone found on the internet, done. That is the correct starting move. It is also why, a year or two later, we keep having the same conversation with teams whose Prometheus has started eating 40 GB of RAM, restarting into half-hour WAL replays, or getting OOM-killed the day a deploy added one innocent-looking label.

That conversation is usually titled “should we move to VictoriaMetrics?” and the answer is a genuine “it depends,” which is the most annoying answer in engineering unless someone tells you exactly what it depends on. That is this post. Ankra has no horse in this race: the platform reads metrics from any Prometheus-compatible endpoint, and both systems deploy as ordinary stacks. What follows is the comparison we walk customers through, with the numbers and the failure stories that inform it.

What Prometheus gets right, still

Let us start by being fair to the incumbent, because it earned its position.

Prometheus is the second project ever to graduate from the CNCF, after Kubernetes itself. Its governance is multi-vendor, its ecosystem is the industry’s shared vocabulary, and its integration with Kubernetes is not “good,” it is the reference: the Prometheus Operator’s ServiceMonitor, PodMonitor, and PrometheusRule CRDs are the most battle-tested way to wire scraping that exists. Every exporter targets it. Every alerting example assumes it. PromQL is the lingua franca that every competing backend, VictoriaMetrics included, must speak to be taken seriously.

And it has not stood still. The 3.x line (3.0 shipped in late 2024; 3.13 landed this month) brought a native OTLP receiver so OpenTelemetry metrics land without a collector in between, full UTF-8 metric and label names so OTel-style names no longer get mangled with underscores, and native histograms, stable since 3.8, which replace dozens of bucket series with a single efficient exponential histogram. The old benchmark gaps you find in blog posts from 2020 have narrowed with every release since.

For a single cluster with tens or a few hundreds of thousands of active series, Prometheus with 15 to 30 days of retention is boring, well-understood, and correct. If that describes you, you can stop reading and keep your stack. There is no prize for migrating a healthy system.

The wall, and why it is always cardinality

Prometheus keeps every active time series in an in-memory head block at a cost of roughly 3 to 4 KB of RAM per series. That is the single number that explains most Prometheus incidents: a million active series is several gigabytes of RAM before anyone runs a query. Every unique combination of label values is its own series. Attach a label with unbounded values, a user ID, a request path, a pod name in a high-churn deployment, and the series count multiplies silently.

The failure mode is not graceful degradation. It is the OOM killer, followed by a write-ahead-log replay on restart that can take tens of minutes while you have no monitoring at all. This is not a small-team skill issue. Cloudflare maintains custom patches on their 900-plus Prometheus fleet because upstream has no hard cap on series count. The UK Ministry of Justice’s cloud platform lost monitoring for over three hours in 2024 when a WAL replay outran the startup probe and put Prometheus into a restart loop. Teams running Prometheus on machines with hundreds of gigabytes of RAM have still hit the ceiling.

You can hold the wall back with discipline: recording rules, relabeling to drop offending labels, sample_limit per target, federation to shard the load. Plenty of teams do. But notice what you are doing: operating around the storage engine’s design, forever. At some scale, the question stops being “how do we tune Prometheus” and becomes “should the thing we tune be something else.”

What VictoriaMetrics actually changes

VictoriaMetrics is an Apache 2.0 time-series database built to be a drop-in Prometheus replacement: it scrapes the same targets with the same configs, serves the same HTTP query API so Grafana’s Prometheus datasource works unchanged, and accepts Prometheus remote_write. The differences are under the floor.

The storage engine. Instead of a WAL plus in-memory head block plus two-hour compacted blocks, VictoriaMetrics uses a MergeTree-inspired design closer to ClickHouse. The practical consequences: no WAL replay on restart, no compaction I/O spikes, and much tighter compression. When series churn exceeds what fits in memory, ingestion slows down (visible in a metric you can alert on) rather than the process dying. Running out of headroom becomes a dial you watch instead of a page you get.

The resource profile. Here honesty is required, because most published benchmarks come from the vendor. VictoriaMetrics’ own 2024 comparison showed 2.5x less disk and 1.7x less RAM than Prometheus on identical workloads; the older “7x less disk” figures trace to a 2020 test against a long-obsolete Prometheus and should be retired. The credible independent evidence points the same direction with workload-dependent magnitude: Criteo, running around a billion active series, reported roughly one byte per datapoint in production and cut their metrics backend from 382 nodes to 61 while storing 15 times more data. Prezi went on record with a 70 percent storage reduction, 60 percent less memory, and heavy queries dropping from over 30 seconds to single digits after moving 5 million series off Prometheus. The rule of thumb that survives contact with reality: budget about 1 GB of RAM per million active series for VictoriaMetrics against several for Prometheus.

The scaling story. A single VictoriaMetrics binary comfortably handles millions of active series, and past that there is a cluster version with horizontal scaling and replication set by one flag. Which brings us to the comparison people actually get wrong.

The real comparison is not Prometheus vs VictoriaMetrics

For anything beyond one cluster and a few weeks of retention, plain Prometheus is not the alternative, because plain Prometheus does not do high availability, global views, or years of retention at all. The honest comparison is Prometheus plus Thanos or Mimir versus VictoriaMetrics, and that changes the calculus entirely.

The Prometheus path to durability means running replica pairs per cluster, sidecars or remote writes shipping blocks to object storage, compactors, store gateways, and query frontends that deduplicate across replicas. Thanos and Mimir are both capable, proven systems. They are also each a distributed system you now operate on top of your monitoring, with their own failure modes, upgrade cycles, and 2 AM personalities.

VictoriaMetrics answers the same requirements inside one system: retention is a flag, replication is a flag, deduplication is a flag, and multi-year storage rides on the same compressed local disks. Criteo’s numbers above came from evaluating exactly this three-way choice. When teams tell us they want “Prometheus but durable,” a large part of the answer is deciding whether they want to assemble that from parts or get it in one box.

PromQL vs MetricsQL, the fine print

VictoriaMetrics queries with MetricsQL, a deliberate superset of PromQL. Nearly all existing queries and Grafana dashboards run unchanged, which is why migrations rarely break dashboards. It adds genuinely useful things: WITH expressions for reusable query fragments, extra rollup functions, the keep_metric_names modifier.

But “superset” hides deliberate divergence. In the independent PromLabs compliance suite, VictoriaMetrics scored 74 percent, against 100 for Thanos and Mimir. Most failures are intentional choices the VictoriaMetrics authors consider more correct: rate() and increase() consider the sample just before the window, NaN handling differs, metric names survive some functions that strip them in Prometheus. In day-to-day dashboarding you will likely never notice. Where it bites is precision work: a large library of recording rules and alerts tuned to Prometheus’ exact extrapolation behavior may fire differently after a migration, and alert thresholds deserve a review pass. If strict PromQL portability across backends is a hard requirement for you, that is a legitimate reason to stay in the Prometheus family.

Governance, because it matters now

The last two years taught everyone to read licenses before adopting infrastructure. Prometheus is CNCF-owned, multi-vendor, and structurally impossible for any one company to capture. VictoriaMetrics is one bootstrapped company of about 50 people, with an enterprise tier alongside the open core.

That asymmetry is real but smaller than it looks. VictoriaMetrics’ open source is Apache 2.0 and the project takes no contributor license agreement, meaning contributors keep their copyright and the existing code cannot be relicensed out from under you the way HashiCorp’s and Redis’s licenses were changed. The realistic risk is not a rug-pull; it is future features landing enterprise-first. Adopt with eyes open, as with any single-vendor dependency.

The decision, compressed

Stay on (or start with) Prometheus when:

One or a few clusters, series counts in the low hundreds of thousands, retention needs measured in weeks
Your organisation requires CNCF-governed, vendor-neutral tooling
You maintain recording rules and alerts whose exact PromQL semantics are load-bearing
Your stack is healthy today; migrating a working system is a cost with no benefit

Move to (or start with) VictoriaMetrics when:

Prometheus RAM or OOM incidents are a recurring agenda item
You need one queryable view over many clusters, the exact fleet problem we described in the edge post
You want months or years of retention without operating Thanos or Mimir alongside
You are paying for monitoring nodes and the 1 GB versus several GB per million series difference is a real invoice line, which after June’s Hetzner repricing it is

And remember the third option, which is where most successful migrations start: both. Add VictoriaMetrics as a remote_write target from your existing Prometheus, point long-range dashboards at it, and let the two run side by side until the cutover is a formality instead of a bet:

remote_write:
  - url: http://victoria-metrics.monitoring.svc.cluster.local:8428/api/v1/write

Running either on Ankra

Whichever way you decide, the mechanics on Ankra are the same, because both are just addons in a stack.

The classic stack is kube-prometheus-stack plus Loki and Promtail, with the namespace deployed first and dependencies wired through parents:

stacks:
  - name: monitoring
    manifests:
      - name: monitoring-ns
        from_file: "manifests/monitoring-namespace.yaml"
    addons:
      - name: kube-prometheus-stack
        chart_name: kube-prometheus-stack
        chart_version: 65.1.1
        repository_url: https://prometheus-community.github.io/helm-charts
        namespace: monitoring
        parents:
          - manifest: monitoring-ns
        configuration:
          from_file: "values/kube-prometheus-stack.yaml"
          encrypted_paths:
            - adminPassword

Swap the addon for victoria-metrics-single (or the cluster chart) from the VictoriaMetrics Helm repository and the shape of the stack does not change. Pin the chart version, keep the Grafana admin password SOPS-encrypted, and clone the same stack to every cluster that needs it.

Then point the platform at whichever backend you chose. Ankra’s metrics integration accepts any Prometheus-compatible query endpoint, and that compatibility is exactly why this choice is low-stakes at the platform layer:

Backend	Endpoint you connect
Prometheus (kube-prometheus-stack)	`http://prometheus-operated.monitoring.svc.cluster.local:9090`
VictoriaMetrics	`http://victoria-metrics.monitoring.svc.cluster.local:8428`
Thanos Query	`http://thanos-query.monitoring.svc.cluster.local:9090`
Grafana Cloud, Amazon or Google managed Prometheus	the provider’s query URL plus a read-only credential

That one setting feeds the resource graphs on every pod and deployment, the cluster dashboards, and, most importantly, the AI. When an incident report lands in Slack or the assistant explains why a pod is throttling, the metrics behind that analysis come from this data source. The AI does not care whether container_memory_working_set_bytes came out of Prometheus or VictoriaMetrics. It cares that the query answers quickly, which, if you have just moved a multi-million-series fleet to VictoriaMetrics, it now does.

Pick the engine for your scale, not for fashion. Prometheus by default, VictoriaMetrics when the default starts costing you sleep or servers, and either way the platform on top does not change.

Get started: Create a free account on Ankra.

Join our community: Slack

Spectro Cloud vs Ankra: Managing the Metal, or Running What Lives On It?

Mark Shine — Sat, 04 Jul 2026 00:00:00 GMT

Spectro Cloud is a serious company. Palette manages Kubernetes estates for telcos, defense agencies, and manufacturers, it closed a $75 million Series C led by Goldman Sachs’ growth equity arm in late 2024, and it reported doubling revenue the year after. When teams evaluating Ankra ask “how do you compare to Spectro Cloud,” they are asking about a real platform with real customers, and it deserves a real answer rather than a feature-table ambush.

Here is the real answer: Spectro Cloud and Ankra look similar from a distance, both are “manage all your Kubernetes from one place” platforms with agents in your clusters and blueprint concepts at the core, but they point in opposite directions. Palette manages Kubernetes downward: the distro, the operating system underneath it, the bare metal and hypervisors underneath that. Ankra runs Kubernetes upward: the addons, the applications, the delivery, the incidents, the developers, and the AI that operates all of it. Which direction you need is the whole decision, and for a lot of readers the honest conclusion will be that they need one of these platforms and not the other.

Let us take it apart properly.

What Palette actually is

Palette is a full-lifecycle Kubernetes management platform built around two ideas.

The first is the Cluster Profile: a declarative, layered blueprint of an entire cluster, and “entire” means further down than most tools go. A profile stacks the operating system, the Kubernetes distribution, the CNI and CSI layers, and add-on applications into one versioned object. Palette then builds clusters from that profile and holds them to it. It extends the CNCF Cluster API project for the provisioning machinery, which is the architecturally sound way to do it.

The second is breadth of substrate. Palette provisions and manages clusters on AWS, Azure, and Google Cloud (both IaaS and the managed EKS, AKS, GKE services), on VMware vSphere, Nutanix, and Apache CloudStack in the data center, on bare metal through Canonical MAAS, and on edge devices, including air-gapped ones. There is a hardened FIPS-compliant edition for government, a Virtual Machine Orchestrator for running VMs next to containers as a VMware exit ramp, and since late 2025 there is PaletteAI, a platform for deploying NVIDIA AI factory infrastructure, GPUs, DPUs, NIM, NeMo and all, from data center to edge.

If your problem is “I have three data centers full of vSphere, a bare-metal footprint, four hundred retail edge boxes, and a mandate to run Kubernetes on all of it with the OS lifecycle under management,” Palette is one of a very small number of credible answers to that problem. I want to be unambiguous about that before the comparison, because the rest of this post is about a different problem.

The different problem

Walk into the teams we work with, the five-person product team, the platform team of a mid-size SaaS, the agency running client workloads, and ask what actually hurts. It is almost never “we cannot provision the operating system.” Their clusters come from EKS, from Hetzner, from k3s on a box, from OrbStack on a laptop. The cluster is a commodity, increasingly the cheapest line on the invoice, as we worked through after the Hetzner repricing.

What hurts is everything above the cluster. Which fifteen Helm charts make it production-grade, in which order, with which values. Who is allowed to change them and how the change is reviewed. What happens at 2 AM when a pod crashloops and the one person who understands cert-manager is asleep. How a developer gets a production-like environment without filing a ticket. Where the audit trail lives when the auditor asks.

That is the problem Ankra is built for, and it shapes every architectural choice differently than Palette’s problem shaped theirs.

Blueprints: Cluster Profiles vs Stacks

Both platforms have a blueprint at the center, and comparing them is the fastest way to see the philosophical split.

A Palette Cluster Profile answers: what should this cluster be? OS, distro, CNI, monitoring agents, security tooling, baked into layers, enforced by the agent. Its natural owner is an infrastructure team, its natural scope is the fleet, and its natural cadence is quarterly. It is infrastructure-as-catalog, and at fleet scale that governance is exactly what a central IT organization wants.

An Ankra Stack answers: what should run on this cluster, and in what order? Helm addons, raw manifests, applications, and the dependency edges between them, rendered as a visible graph with deploy order, namespaces, and ingress paths on display before anything ships. Its natural owner is whoever ships the software, its natural scope is a solution (a monitoring stack, a data layer, a product), and its natural cadence is daily. Variables resolve per cluster so one stack template lands correctly in dev, staging, production, and a laptop. Stacks clone across clusters and organisations as drafts, with secrets deliberately left behind.

The tell is what each blueprint refuses to model. Cluster Profiles do not model the dependency choreography between your applications and their data services; that is your CD tool’s job in a Palette world. Stacks do not model the operating system under the kubelet; Ankra assumes something else owns the metal, whether that is AWS, Hetzner via Ankra’s own provisioning, or your data center team.

Git: written back, or managed through an API

Palette is operated through its UI, API, and a solid Terraform provider. Profiles and cluster specs are Palette objects, versioned and governed inside Palette. You can wire GitOps tooling into what Palette deploys, but the platform’s source of truth is the platform.

Ankra inverts that. Every change, from the UI, the CLI, the API, Terraform, or the AI, is written back to your Git repository as commits and pull requests, and the engine reconciles from Git. This is the same argument we made in the Komodor comparison, and it is worth repeating in one sentence because it is the highest-stakes line in any platform evaluation: if you switch Ankra off tomorrow, you keep a complete, working GitOps repository describing every stack, every value, and every dependency, and your workloads keep running. Your operational history accumulates in an asset you own, not in a vendor’s database.

At enterprise fleet scale, Palette’s model is a governance feature: profiles as controlled catalog items, drift as a violation. At product-team scale, it is a second source of truth to keep synchronized with the one your developers already live in. Which one is right depends entirely on which team you are.

Two meanings of “AI platform”

Both companies say AI a lot in 2026. They mean almost entirely different things, and this is the least understood difference between them.

PaletteAI is a platform for deploying AI workloads. It packages NVIDIA’s AI Enterprise stack, provisions GPUs and DPUs, applies validated AI factory designs, and gets an inference environment running in hours instead of months. The AI is the cargo. It is a smart bet on where enterprise infrastructure spend is going, and if you are standing up NVIDIA racks, it is genuinely relevant.

Ankra’s AI is an operator of your platform. It designs stacks with you in the Stack Builder, explains a crashing pod against its logs, events, and the stack change that broke it, investigates incidents and posts root causes to Slack, scans clusters continuously for drift and capacity risks, and proposes fixes as reviewable drafts under the propose, review, record contract. The AI is the coworker. It never touches a cluster directly; it writes drafts a human approves and Git records.

Neither replaces the other. One deploys the GPU cluster your data scientists asked for; the other is the senior engineer your platform team does not have to hire. But when a vendor page says “AI-powered,” ask which of these two sentences it means, because the difference is a data center purchase versus a head count.

Cost of entry, in time and money

The commercial shapes are as different as the architectures.

Palette prices cloud and virtualized environments in kilo-Core-hours, a usage metric of worker cores under management multiplied by time, with a lower rate on managed services like EKS where the cloud already does part of the job. Edge and bare metal are licensed per node, sized per device, quotation on request. It is a fair model for what it measures, and it is unmistakably an enterprise motion: talk to a representative, scope the estate, negotiate the agreement. The buyers Palette serves expect exactly that.

Ankra bills on worker vCPU with every plan including 30 free vCPUs, unlimited clusters, and unlimited team members; paid tiers start at €0.01 per vCPU-hour. Control planes are never billed. There is no sales call between you and a running platform: sign up, helm install the agent into any cluster you already have, and the graph, the metrics, and the AI are live the same afternoon. A small team can run production on Ankra without ever leaving the free allowance.

Time-to-value follows the same split. Palette deployments are projects, worth it when the estate is measured in data centers. Ankra adoptions are afternoons, which is the point when the team is five people and the estate is six clusters.

Side by side

Dimension	Spectro Cloud Palette	Ankra
Core object	Cluster Profile: OS, distro, addons as governed layers	Stack: addons, manifests, apps as a dependency graph
Direction	Down the stack: distro, OS, hypervisor, metal	Up the stack: delivery, operations, developers, AI
Provisioning	AWS, Azure, GCP, vSphere, Nutanix, CloudStack, MAAS bare metal, edge	Import any cluster; provision K3s on Hetzner, OVH, UpCloud
Source of truth	Palette (API, Terraform; GitOps integrable)	Your Git repository, written back by the engine
AI story	PaletteAI: deploy NVIDIA AI factory infrastructure	AI operator: designs, explains, investigates, proposes drafts
Incident workflow	Integrates with your observability stack	Built in: AI root cause to Slack, insights, incident reports
Pricing motion	kilo-Core-hours plus per-node edge licensing, sales-led	Per-vCPU, 30 vCPUs free, self-serve
Sweet spot	Enterprise fleets, VMware exits, air-gapped edge, GPU estates	Product and platform teams shipping software on Kubernetes

Where Palette is the right call

Staying fair, as with Komodor: there are evaluations Ankra should lose, and these are them.

If you must manage the operating system and below, immutable OS images on edge devices, bare metal through MAAS, a vSphere estate mid-exodus with VMs that need to land in KubeVirt, Palette does things Ankra does not do and does not intend to do. If you are deploying NVIDIA AI factories with DPU-enforced zero trust, PaletteAI is purpose-built for it. If your procurement requires FIPS-certified, air-gapped, self-hosted management planes for defense work, that is Palette’s hardened edition territory. Central IT organizations governing hundreds of clusters across dozens of business units will find Palette’s tenancy and catalog governance genuinely mature.

But notice what every one of those cases has in common: the cluster itself, and what is under it, is the product being managed. If instead the cluster is just where your software runs, if your team’s output is measured in features shipped and incidents resolved rather than nodes under management, then you would be buying a fleet management system to get a delivery workflow it does not contain.

What running both above and below would even mean

There is also a quiet compatibility here that the vs-framing hides. Ankra imports any conformant Kubernetes cluster through one Helm install of its agent. A cluster that Palette provisioned is exactly that. Teams inside large organizations do not always get to choose the fleet layer; central IT may standardize on Palette for the estate while a product group runs Ankra on its slice of it for stacks, delivery, developer self-service, and AI operations. The two platforms tug on different ends of the same cluster, and they do not fight, because one stops where the other starts.

The move

Spectro Cloud built an excellent answer to “how does a large enterprise provision and govern Kubernetes estates, from hyperscaler to bare metal to the edge.” Ankra is built for the question that starts one layer up and never stops mattering at any size: “how does my team ship, run, and fix software on Kubernetes without a platform priesthood in the middle.”

If your pain is the metal, talk to Spectro Cloud, and mean it. If your pain is everything that happens after the cluster exists, create a free account, import a cluster you already run with one Helm command, and let the AI propose its first stack before the coffee is cold. The 30 vCPU free allowance means the evaluation costs you nothing but the afternoon.

Get started: Create a free account on Ankra.

Join our community: Slack

Hetzner Repriced. Here Is the Math for Your Kubernetes Cluster.

Mark Shine — Wed, 01 Jul 2026 00:00:00 GMT

On June 15, 2026 at 8 AM CEST, Hetzner’s new cloud prices took effect. If you run Kubernetes on the CPX or CCX families, the numbers are not subtle: a CCX23 in Germany went from €31.49 to €85.99 a month, a CPX41 in the US regions went up 209 percent, and the CCX33 that a lot of teams use as their default worker node went from €62.49 to €138.49. The cost-optimized CX and Arm-based CAX lines got off lighter, up roughly 30 to 38 percent.

The increase applies to new orders and rescales. Servers you already had before June 15 keep their old price until you resize them, which turns out to matter for how you should react.

Plenty of teams built their entire infrastructure story on “Hetzner is a fifth the price of AWS.” Those teams just got a letter reminding them that a price is a snapshot, not a contract. This post does three things: works the actual numbers for what a production cluster costs today, explains why the panic takes are wrong, and makes the argument the incident actually supports. The lesson is not “find a cheaper provider.” The lesson is “never build in a way that makes repricing your problem alone.”

The new numbers, without the outrage

All prices are Germany/Finland monthly caps, excluding IPv4 and VAT, from Hetzner’s published adjustment table.

Instance	vCPU / RAM	Before June 15	After	Change
CX33 (shared x86)	4 / 8 GB	€6.49	€8.49	+31%
CX43 (shared x86)	8 / 16 GB	€11.99	€15.99	+33%
CAX31 (shared Arm)	8 / 16 GB	€15.99	€20.99	+31%
CPX41 (shared AMD)	8 / 16 GB	€38.99	€120.49	+209%
CCX33 (dedicated)	8 / 32 GB	€62.49	€138.49	+122%

Two observations that most of the commentary missed.

First, the increase is wildly uneven across families. The shared Intel and Arm lines rose about a third. The AMD shared and dedicated-vCPU lines doubled or tripled. If your node groups happen to be on CX or CAX, this event barely touched you. If you defaulted to CCX “because production deserves dedicated cores,” your bill more than doubled for the same compute.

Second, grandfathering changes the tactics. Your existing servers keep their old prices until you rescale them. That means the worst possible response is casually churning nodes: every replacement server is a new order at new prices. The right response is to decide deliberately what your next node group looks like, and leave the old one alone until you do.

Is Hetzner still cheap? Yes, embarrassingly so

Here is the comparison that matters, because the alternative to Hetzner is not free compute, it is a hyperscaler.

Take a real production baseline: 24 worker vCPUs, a highly available setup, ingress, and a managed control plane. On AWS EKS in us-east-1 at on-demand list prices, that looks like:

EKS control plane: $0.10 per hour, about $73 a month, per cluster, even when idle
6 x m6a.xlarge workers (4 vCPU, 16 GB each): about $126 each, $757 a month
NAT gateways across three availability zones: about $98 a month before data processing
An application load balancer: about $22 a month base

That is roughly $950 a month before storage, egress, and observability, and cost guides for EKS routinely tell you to add 20 to 35 percent on top for the pieces everyone forgets. Savings plans and spot can cut the worker line substantially, at the price of commitment or interruption tolerance.

The same 24 worker vCPUs on Hetzner through Ankra, at the new, post-increase prices:

3 x CX43 workers (8 shared vCPU, 16 GB each): €47.97 a month
1 x CX33 control plane: €8.49 a month
A small bastion host and IPv4 addresses: call it €8 a month
Ankra platform fee: €0, because 24 worker vCPUs is inside the 30 vCPU free allowance, and control planes are never billed

About €65 a month, all in. If shared vCPUs make you nervous for your workload profile, do it with dedicated cores: three CCX33 nodes at the new price is €415 a month, still less than half the EKS figure, with double the RAM per node.

The honest caveat: those are shared cores versus dedicated EC2 cores in the first comparison, one region versus three availability zones, and EKS brings an ecosystem Hetzner does not. This is not a like-for-like SLA comparison, it is a “what does the invoice say” comparison. But the invoice gap is 14x, and it did not stop being enormous because Hetzner raised prices. What changed is something else.

What the repricing actually broke

The price increase did not break anyone’s budget on its own. A 33 percent increase on a €65 cluster is €20. What it broke is a mental model: the idea that you pick the cheap provider once, build everything there, and the decision stays made.

Cloud pricing is an input that moves. Hetzner just proved it, AWS quietly reprices too, in both directions, and your negotiated discount expires. The teams that got hurt by June 15 are not the ones paying an extra €20. They are the ones who did the math, realized they should move from CCX to CAX, or from Hetzner to OVHcloud or UpCloud for a specific workload, and then looked at what moving actually involves:

Reprovision a cluster by hand, or resurrect the Terraform that built the last one
Reinstall the fifteen Helm charts that make a cluster production-grade: ingress, cert-manager, monitoring, logging, the works
Port the values files, and rediscover which of the hundred settings were load-bearing
Rewire DNS, secrets, CI/CD, and kubeconfigs
Do it all without an outage

If that list costs you three weeks of a senior engineer, you do not move. You eat the increase, this one and the next one. The provider knows it. Migration friction is exactly what makes repricing safe to do to you.

This is the same trap we described in the edge computing post from the fleet angle: the cost of Kubernetes is rarely the compute, it is the operational plumbing around every individual cluster.

Portability is a property you build in, not a migration you run later

Here is how the same event plays out when the platform layer is portable, which is the actual design goal behind how Ankra treats clusters and stacks.

The cluster is a commodity. Ankra provisions managed k3s clusters on Hetzner, OVHcloud, and UpCloud from the same interface: pick a location, control plane count and type, and worker node groups. One command, or a few clicks, and the full topology comes up, private network, bastion, control planes, workers, optional ingress stack included.

ankra cluster hetzner create \
  --name prod-fsn1 \
  --credential-id <hetzner-credential-id> \
  --ssh-key-credential-id <ssh-key-id> \
  --location fsn1 \
  --control-plane-count 1 \
  --control-plane-server-type cx33 \
  --worker-count 3 \
  --worker-server-type cx43

The software layer is a stack, and stacks travel. Everything installed on top, ingress, cert-manager, monitoring, your applications, lives in Stacks with their dependency graph and Helm values, versioned in Git. Standing up the same platform on a second provider is Clone to Cluster, not a quarter-long project. Variables resolve per cluster, so the same template lands with the right domains, storage classes, and replica counts. Secrets deliberately do not travel with clones; they stay SOPS-encrypted and get reconfigured per environment.

That combination changes the negotiation. When “move this workload to OVHcloud” costs an afternoon instead of three weeks, you can actually respond to a repricing. Benchmark the same stack on two providers with real traffic, compare the invoices, and keep whichever wins. Providers compete for portable workloads. They tax captive ones.

The three cost levers you get on top

Portability is the strategic lever. Day to day, three smaller ones do most of the work.

Node groups let you re-shape instead of overpay. Workers live in named groups, each with its own instance type, count, labels, and taints, scalable from 0 to 100 nodes independently. The June increase made instance family the single biggest cost decision on Hetzner, and node groups are how you act on it: add a CAX Arm group at a third of the CCX price, taint it, move the workloads that are multi-arch clean, and watch. One warning from the trenches: instance type changes on an existing group are upgrade-only and irreversible, because disks grow but never shrink. To go smaller, add a new group and delete the old one. Which, conveniently, is also the pattern that preserves your grandfathered pre-June prices on the nodes you keep.

Scale to zero means idle environments cost zero. A staging cluster that only matters during business hours does not need servers at 3 AM. Deleting the Kubernetes layer tears down every Hetzner resource, servers, network, SSH keys, and stops the charges completely, while the cluster record, its stacks, and its credentials stay in Ankra. Bringing it back is adding a node group: the control plane restores automatically, workers join, and the stacks reconcile onto the fresh infrastructure on their own. Fresh servers, same platform. For review environments and client demos, this is the difference between “we keep a cluster around just in case” and “we pay for hours we use.”

You can see the spend before the invoice does. Ankra’s Cloud Cost view estimates what each cluster costs from the live node and volume inventory plus public pricing, broken down by compute, storage, and network, and allocated per namespace and stack. The fleet rollup puts every provider on one page. Estimates are for trend-watching and right-sizing rather than invoice reconciliation, and that is precisely what you want in the week after a provider reprices: which node groups, which namespaces, which teams does this actually hit?

And to close the loop on Ankra’s own pricing, since this is a post about invoices: the platform bills on worker vCPU only, control planes are free, every plan includes a 30 vCPU free allowance, and clusters and team members are unlimited. The billing page shows current burn, projected month-end, and a full-month average from your actual hourly usage. No per-cluster fee, which matters exactly when the right answer is “run more, smaller clusters on whichever provider is cheapest this quarter.”

The playbook for this month

If you run Kubernetes on Hetzner today:

Do not churn grandfathered nodes. Servers from before June 15 keep old prices until rescaled. Stability is now literally worth money.
Audit your instance families. CPX and CCX took the hit; CX and CAX barely moved. If your workloads are not pinned to x86 or dedicated cores, a CAX node group at €20.99 for 8 vCPU and 16 GB is the cheapest production compute in Europe, still.
Turn off what idles. Every staging and demo cluster that survives the night out of provisioning fear is pure waste once restoring it is a one-step operation.
Price the alternatives with data, not vibes. Clone your stack to an OVHcloud or UpCloud cluster, run it for a week, read the Cloud Cost page, decide.
Whatever you choose, choose it from portability. The next repricing is not hypothetical. It is scheduled, somewhere, by someone, and the only durable defense is a platform layer that moves.

Hetzner after June 15 is still, by a comfortable margin, the cheapest serious place to run Kubernetes in Europe. That is not the point. The point is that you should never again be in a position where a pricing PDF from any provider dictates your architecture. Build the platform layer portable, and the pricing PDF becomes an input to a decision you can actually make.

Get started: Create a free account on Ankra and provision your first cluster inside the free 30 vCPU allowance.

Join our community: Slack

Building Pipeline Agents with the Ankra CLI

Mark Shine — Wed, 10 Jun 2026 00:00:00 GMT

Most posts about AI agents stop at the architecture diagram. This one walks through three working agents, built from plain CI jobs and the Ankra CLI, that you can copy into a repo today.

What they do:

PR review agent. When a pull request touches your cluster definition, it validates the change against the live platform and comments on the PR with what will actually change on the cluster and what looks risky.
Deploy watcher. On merge it applies the change, waits for the rollout to finish, verifies health, and posts the result to Slack. When the deploy fails, it posts a root cause instead of a red X.
Scheduled infra watcher. Runs every 30 minutes, independent of CI. It stays silent while things are healthy and posts to Slack only when something degrades.

There is no framework here, no orchestration layer, and no vector database. Each agent is a trigger, a few deterministic CLI calls to gather facts, one LLM call for the judgment step, and a write to somewhere humans actually look: a PR thread or a Slack channel.

You also never wire up a model yourself. ankra chat is a one-shot, scriptable command, and the AI behind it already has server-side access to your cluster context: logs, events, manifests, stack history, operations, and metrics. You ask a question from CI and it answers against live state. The pipeline never holds a model API key or a kubeconfig.

Prerequisites

You need an Ankra-managed cluster, imported or provisioned, with its definition versioned in the repo the pipelines run in. That is the standard GitOps layout: a cluster.yaml plus a stacks/ directory.

Create an API token for the agent. Do this from a machine where you are logged in:

ankra login
ankra tokens create ci-agent

Store the returned token as ANKRA_API_TOKEN in your CI secret store (GitHub Actions secrets, GitLab CI/CD variables). Also add SLACK_WEBHOOK_URL, a plain Slack incoming webhook for the channel you want reports in.

The CLI installs in one line on any runner:

bash <(curl -sL https://github.com/ankraio/ankra-cli/releases/latest/download/install.sh)

One gotcha before you script anything. The CLI resolves its token in this order: the --token flag, then a saved login from ankra login, then ANKRA_API_TOKEN. On a throwaway CI runner the environment variable works fine. On a self-hosted runner where someone once ran ankra login, the saved login silently wins over your CI token. If you run on shared machines, pass --token "$ANKRA_API_TOKEN" explicitly.

Agent 1: PR review comments

The job triggers when a pull request touches the cluster definition. It does three things: server-side validation, an impact question against the live cluster, and a comment on the PR.

.github/workflows/infra-review.yml:

name: infra-review
on:
  pull_request:
    paths:
      - "cluster.yaml"
      - "stacks/**"

permissions:
  contents: read
  pull-requests: write

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Install the Ankra CLI
        run: bash <(curl -sL https://github.com/ankraio/ankra-cli/releases/latest/download/install.sh)

      - name: Validate against the platform
        env:
          ANKRA_API_TOKEN: ${{ secrets.ANKRA_API_TOKEN }}
        run: |
          ankra cluster validate -f cluster.yaml --strict-secrets 2>&1 | tee validation.txt

      - name: Ask what the change does to the live cluster
        env:
          ANKRA_API_TOKEN: ${{ secrets.ANKRA_API_TOKEN }}
        run: |
          DIFF=$(git diff "origin/${GITHUB_BASE_REF}"...HEAD -- cluster.yaml stacks/ | head -c 12000)
          ankra chat --cluster production "A pull request changes our cluster definition.
          Review the diff below against the current state of this cluster.
          Answer in three short sections:
          1. What changes on the cluster (which stacks, addons, manifests).
          2. Which workloads will restart or be replaced.
          3. Anything risky given live state right now (capacity, dependencies, ordering).
          Be specific. If nothing is risky, say so in one line.

          ${DIFF}" > impact.md

      - name: Comment on the pull request
        env:
          GH_TOKEN: ${{ github.token }}
        run: |
          {
            echo "## Infra review"
            echo
            echo "**Validation** (\`ankra cluster validate --strict-secrets\`)"
            echo '```'
            cat validation.txt
            echo '```'
            echo
            echo "**Impact against live cluster state**"
            echo
            cat impact.md
          } > comment.md
          gh pr comment "${{ github.event.pull_request.number }}" --body-file comment.md

Why each piece is there:

ankra cluster validate -f runs local structural and dependency checks first, then sends the file to the API for the checks you cannot do offline: whether the referenced charts exist in your connected Helm registries, whether there are plaintext Secrets or unencrypted addon values, and whether parents references resolve against the cluster’s deployed resources. --strict-secrets turns a plaintext secret from a warning into a failed job, which is exactly what you want on a PR.
The ankra chat step is the only LLM call. The diff is in the prompt; the cluster state is on the server side. That combination is what makes the review better than a generic LLM looking at YAML. It knows the addon you are reparenting is mid-rollout, or that the node group you are scaling down is already at 85% memory.
The comment is plain gh pr comment. No app installation, no webhook receiver. The default GITHUB_TOKEN with pull-requests: write is enough.

A real comment from this job looks like:

Infra review

Validation (ankra cluster validate --strict-secrets)
cluster.yaml: OK
stacks: payments (2 addons, 3 manifests), monitoring (unchanged)
chart payments-api 1.4.2 found in registry harbor-internal
no plaintext secrets detected
Impact against live cluster state

Bumps the payments-api chart from 1.4.1 to 1.4.2 and raises its memory limit from 256Mi to 512Mi.

The payments-api Deployment (3 replicas, namespace payments) rolls. Nothing else restarts.

Worker group default has 1.1Gi allocatable headroom; three new 512Mi pods fit, but only just. If you also merge PR #214 (raises checkout limits), schedule them apart.

That last point, the cross-reference against live headroom, is the check a human reviewer skips at 5pm on a Friday.

The same agent on GitLab

The structure is identical. The one piece of friction is that CI_JOB_TOKEN cannot post notes on merge requests. Create a project access token with the api scope and Reporter role, store it as GITLAB_AGENT_TOKEN, and post to the notes API:

infra-review:
  stage: review
  image: alpine:3.20
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      changes:
        - cluster.yaml
        - stacks/**/*
  script:
    - apk add --no-cache bash curl git jq
    - bash <(curl -sL https://github.com/ankraio/ankra-cli/releases/latest/download/install.sh)
    - ankra cluster validate -f cluster.yaml --strict-secrets 2>&1 | tee validation.txt
    - |
      DIFF=$(git diff "origin/${CI_MERGE_REQUEST_TARGET_BRANCH_NAME}"...HEAD -- cluster.yaml stacks/ | head -c 12000)
      ankra chat --cluster production "A merge request changes our cluster definition. Review the diff against live state. What changes, what restarts, what is risky right now? ${DIFF}" > impact.md
    - |
      BODY=$(printf '## Infra review\n\n```\n%s\n```\n\n%s\n' "$(cat validation.txt)" "$(cat impact.md)")
      curl -sS --request POST \
        --header "PRIVATE-TOKEN: ${GITLAB_AGENT_TOKEN}" \
        --data-urlencode "body=${BODY}" \
        "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/merge_requests/${CI_MERGE_REQUEST_IID}/notes"

Agent 2: merge, deploy, verify, report to Slack

On merge to main, this agent applies the cluster definition, blocks until the platform finishes, verifies health, and reports. The interesting part is the failure path: instead of “pipeline failed, go look”, the Slack message contains the failed operation and a root cause pulled from live state.

.github/workflows/deploy-watch.yml:

name: deploy-watch
on:
  push:
    branches: [main]
    paths:
      - "cluster.yaml"
      - "stacks/**"

permissions:
  contents: read
  pull-requests: write

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install the Ankra CLI
        run: bash <(curl -sL https://github.com/ankraio/ankra-cli/releases/latest/download/install.sh)

      - name: Select the target cluster
        env:
          ANKRA_API_TOKEN: ${{ secrets.ANKRA_API_TOKEN }}
        run: ankra cluster select production

      - name: Apply and wait
        id: apply
        env:
          ANKRA_API_TOKEN: ${{ secrets.ANKRA_API_TOKEN }}
        run: |
          set +e
          ankra cluster apply -f cluster.yaml --wait --timeout 15m 2>&1 | tee apply.txt
          echo "exit_code=${PIPESTATUS[0]}" >> "$GITHUB_OUTPUT"

      - name: Verify health after deploy
        if: steps.apply.outputs.exit_code == '0'
        env:
          ANKRA_API_TOKEN: ${{ secrets.ANKRA_API_TOKEN }}
        run: |
          ankra chat health > report.md
          ankra cluster metrics query \
            'sum(rate(http_requests_total{code=~"5.."}[5m]))' \
            -o json > error-rate.json

      - name: Investigate failure
        if: steps.apply.outputs.exit_code != '0'
        env:
          ANKRA_API_TOKEN: ${{ secrets.ANKRA_API_TOKEN }}
        run: |
          ankra cluster operations list 2>&1 | head -n 20 > operations.txt
          ankra chat --cluster production "The apply for commit ${GITHUB_SHA::12} failed or timed out.
          Find the most recent failed operation on this cluster, look at its jobs,
          and tell me: the exact resource that failed, the error, and whether a
          retry is likely to succeed or the change itself is broken.
          Apply output for reference: $(tail -c 4000 apply.txt)" > report.md

      - name: Report to Slack
        if: always()
        env:
          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
        run: |
          STATUS="DEPLOYED"
          if [ "${{ steps.apply.outputs.exit_code }}" != "0" ]; then STATUS="FAILED"; fi
          jq -n \
            --arg status "$STATUS" \
            --arg commit "${GITHUB_SHA::12}" \
            --arg url "${{ github.server_url }}/${{ github.repository }}/commit/${{ github.sha }}" \
            --rawfile report report.md \
            '{text: "[\($status)] production <\($url)|\($commit)>\n\($report)"}' \
            | curl -sS -X POST -H 'Content-type: application/json' --data @- "$SLACK_WEBHOOK_URL"

      - name: Comment on the merged PR
        if: always()
        env:
          GH_TOKEN: ${{ github.token }}
        run: |
          PR=$(gh api "repos/${GITHUB_REPOSITORY}/commits/${GITHUB_SHA}/pulls" --jq '.[0].number' || true)
          if [ -n "$PR" ]; then
            gh pr comment "$PR" --body-file report.md
          fi

Details worth knowing:

ankra cluster apply is asynchronous by default. Without --wait the command exits 0 the moment the platform accepts the change, while the actual rollout continues in the background. For a deploy watcher that exit code is useless on its own. Always pass --wait with a --timeout that matches your slowest stack.
ankra cluster select production is run once, by name, non-interactively. Bare ankra cluster select opens an interactive picker and will hang a CI job until the timeout kills it. The same applies to every command with an interactive mode: in CI, always pass names.
The failure path does not parse error strings or grep logs. It collects the apply output and the recent operations as raw facts and hands the diagnosis to ankra chat, which can also see the operation’s jobs, pod events, and container logs on the server side. The one thing the prompt insists on is a verdict on “retry or broken change”, because that is the decision the on-call human has to make. Make the agent answer it.
The last step closes the loop: the verdict lands back on the PR that caused it. Whoever merged gets the outcome in the same thread as the review, without watching the pipeline.

The Slack message on a bad day reads like this:

[FAILED] production a3f8c91b02d4

Operation op-7c21 failed at job helm-upgrade payments-api. The new pod is in CrashLoopBackOff: the 1.4.2 image reads PAYMENTS_DB_POOL_SIZE at startup and the variable is not set in this environment. Rolled-back replicas from 1.4.1 are still serving; no user impact. A retry will fail the same way because the change itself is broken. Add the variable to the stack, or revert the image bump, before re-merging.

Compare that to what a failed pipeline normally gives you: a red X and twenty minutes of kubectl describe.

Agent 3: a scheduled watcher that only speaks when something is wrong

CI-triggered agents miss everything that breaks between deploys: node pressure, a certificate that expired, an addon that drifted. This one runs on a schedule and posts to Slack only on a degraded verdict.

name: infra-watch
on:
  schedule:
    - cron: "*/30 * * * *"

jobs:
  watch:
    runs-on: ubuntu-latest
    steps:
      - name: Install the Ankra CLI
        run: bash <(curl -sL https://github.com/ankraio/ankra-cli/releases/latest/download/install.sh)

      - name: Check cluster health
        env:
          ANKRA_API_TOKEN: ${{ secrets.ANKRA_API_TOKEN }}
        run: |
          ankra cluster select production
          ankra chat --cluster production "Check this cluster now: failed or stuck operations
          in the last 30 minutes, pods not Running or Succeeded, nodes not Ready,
          and anything alarming in recent events. The FIRST LINE of your answer
          must be exactly OK or DEGRADED. Then the details." > verdict.md

      - name: Post to Slack only when degraded
        env:
          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
        run: |
          if ! head -n 1 verdict.md | grep -qx "OK"; then
            jq -n --rawfile v verdict.md '{text: "[infra-watch] production\n\($v)"}' \
              | curl -sS -X POST -H 'Content-type: application/json' --data @- "$SLACK_WEBHOOK_URL"
          fi

The verdict-first-line trick is the load-bearing part. You cannot branch a shell script on prose, so force the model to emit a machine-checkable first line and treat anything that is not exactly OK as degraded. If the model ignores the format, the worst case is a spurious Slack message rather than a missed incident. That is the right direction to fail in.

If this overlaps with alerting you already have, keep the alerts. The watcher’s value is the paragraph under the verdict: which operation, which pod, what the events say. Ankra’s built-in alerts and webhooks cover the push-based version of this with AI analysis attached; the scheduled agent is for the checks you want to phrase yourself.

The rules that keep this safe

The agents are read-only by design. Everything above validates, asks, and reports. The only mutation is ankra cluster apply on merge, which is the deploy your GitOps flow was going to run anyway, gated by the PR review. If you want an agent to propose fixes, use ankra cluster draft -f. It stages every stack in the file as reviewable drafts in Ankra instead of deploying, and a human ships them from the stack builder. Never give a pipeline agent a path to mutate the cluster outside Git.

Treat the model output as untrusted text. It goes into a PR comment or a Slack message and stops there. Never pipe ankra chat output into a shell, an apply, or anything that executes. The diff you feed in came from a PR author; the output inherits that trust level.

Scope the credentials. The CI token is created with ankra tokens create under a real account, so use a dedicated machine user with the minimum org role rather than your own login. The Slack webhook posts to one channel and can do nothing else. The GitHub side runs on the default GITHUB_TOKEN with one extra permission. There is no kubeconfig and no cloud credential anywhere in the pipeline.

Forked PRs do not get secrets. On GitHub, pull_request runs from forks without access to repository secrets, so the review agent simply does not run there. That is the behavior you want, since untrusted diffs should not reach your ankra chat context anyway. Run it on internal branches and have maintainers re-trigger for external contributions.

Control the noise. One comment per PR update, one Slack message per merge, silence from the watcher when healthy. The fastest way to kill an agent like this is to let it chat. If nobody would act on the message, do not send it.

Where this goes next

Everything here flows one way, from pipeline to humans. The natural next step is letting humans answer back: replying in the Slack thread with “scale it down” or “show me the logs” and having an agent act on it. That takes a long-running process rather than a CI job. The OpenClaw integration covers deploying a chat-native agent on your cluster with the Ankra CLI installed as a skill, using the same token model as above.

Start with the pipeline agents, though. They are three YAML files, one token, and an afternoon of work, and they put live-cluster context into the two places your team already looks: the PR thread and the Slack channel.

Get started: Create a free account on Ankra.

Join our community: Slack

Using Cursor with the Ankra CLI as an Infrastructure Subagent

Mark Shine — Thu, 21 May 2026 00:00:00 GMT

Cursor is good at application code. It is much weaker once a change crosses the infrastructure boundary into Kubernetes manifests, Helm charts, and CI pipelines. The model has no live view of the cluster, so it falls back to generic snippets and guesses at namespaces, dependencies, and image tags.

Pointing Cursor at the Ankra CLI fixes that. The CLI gives Cursor a cluster-aware subagent that reads real state from your Ankra-managed clusters and proposes changes against the actual platform instead of free-floating YAML.

What the daily friction looks like today

Audience	Where it hurts	What it looks like
Developer teams	YAML and context switching	Hours per week wrestling with raw manifests, debugging Dockerfiles, or waiting on platform teams to provision an environment.
Operations and platform teams	Guardrails and config drift	DevOps becomes a ticketing bottleneck. The alternative is shadow DevOps or AI tools running destructive `kubectl` or Terraform edits.
The organization	High MTTR and tech debt	Infrastructure knowledge is fragmented. Onboarding takes weeks and orphaned dev clusters bloat the cloud bill.

How the Cursor + Ankra setup works

Cursor’s LLM is strong on syntax and local application logic but has no live infrastructure data. The Ankra CLI fills that gap. It talks to Ankra’s API, knows your clusters, stacks, addons, and manifests, and produces changes in the same shape Ankra-managed GitOps already uses.

Ankra also publishes an LLM-friendly documentation index at https://docs.ankra.ai/llms.txt. Adding it as a doc source in Cursor grounds prompts in the real CLI surface instead of hallucinated kubectl snippets.

Wire it up once:

In Cursor, open Settings, Features, Docs, Add new doc.
Paste https://docs.ankra.ai/llms.txt and name it Ankra.
Prompt with @Docs Ankra (or just @Ankra) when you want infrastructure context.

From that point on, infrastructure prompts produce Ankra CLI invocations and Ankra-managed Stack edits instead of one-off manifests.

Use cases

1. Self-service onboarding

Onboarding a new engineer often means a stale README, environment variables to chase down, cryptic shell scripts, and a hope that the local Kubernetes setup matches staging.

Instead, a new developer clones the repo in Cursor, opens the integrated terminal, and runs:

ankra login
ankra cluster select development-sandbox
ankra chat "Spin up the microservice dependency mesh for the payment gateway"

The subagent reads the cluster state, checks namespace capacity, configures local connections, and gets the developer to a working environment in hours instead of weeks.

2. Draft-first GitOps

Platform teams hesitate to give developers (or their AI tools) write access to clusters, because automated execution can wipe databases or break ingress.

Ankra runs a draft-first, GitOps-backed model. When the subagent modifies infrastructure from the Cursor terminal, it does not touch live clusters directly. It produces an isolated Draft inside Ankra. Once approved, Ankra’s engine syncs the draft back to your GitHub or GitLab repository. Git stays the source of truth, and platform teams review the AI-generated change like any other pull request.

[Cursor IDE / Developer]
       │
       ▼ (Natural Language Command)
[Ankra CLI Subagent] ───► Generates Secure "Draft"
                                      │
                                      ▼ (Human Review / Approval)
                              [Automated GitOps Sync] ───► Live Cluster

3. Incident triage in the IDE

A local or staging deployment crashes. The usual loop is the developer wading through Kube-API streams while operations is dragged into a call to run root cause analysis.

Ankra is wired to both Prometheus metrics and live Kube-API state, so the developer can diagnose the failure from Cursor without context switching:

ankra chat "Why are my payment-service pods crashing in the dev-sandbox cluster?"

The subagent cross-references container logs, resource limits, and cluster events and returns something like “The pod is OOMKilled because the memory limit is set to 256Mi while Redis initialization requires 512Mi. Here is a draft Stack edit to raise the limit.” The fix arrives as a draft for review, not as a kubectl edit that mutates the cluster behind your back.

4. Environment cloning for QA and demos

Replicating a production issue usually means replicating production-grade infrastructure, which is too expensive, complex, or insecure to do on demand.

Ankra clones existing Stacks, with their dependency graph and Helm values intact, into any target cluster (local OrbStack, a sandbox, or a QA cluster). The CLI operates on the ImportCluster definitions you already version in Git:

# Clone the analytics-engine stack from prod into a local cluster file
ankra cluster clone prod.cluster.yaml local.cluster.yaml \
  --stack "analytics-engine"

# Apply the resulting cluster definition to the active local cluster
ankra apply local.cluster.yaml

Variables resolve per environment, dependencies are re-mapped, and production secrets are not propagated. The developer gets a ring-fenced replica on their laptop or sandbox to exercise edge cases before they reach staging.

What changes for the rest of the org

Adopting this setup is not only about typing less YAML.

Product managers see features move from ideation to deployment faster because the infrastructure bottleneck shrinks.
Platform engineers spend less time as ticket handlers and more time optimizing base stacks and defining the policies that every Ankra-managed change must satisfy.
Security and compliance teams get a consistent audit trail. Every infrastructure change, including AI-generated ones, goes through GitOps review, version control, and SOPS-encrypted secrets. No raw cloud credentials in the agent, no out-of-band kubectl edits.

Cursor handles the application code. The Ankra CLI subagent handles the infrastructure, with drafts and GitOps reviews in between. Developers move quickly and operations keeps the guarantees they need.

Get started: Create a free account on Ankra.

Join our community: Slack

Drafts, Approvals, GitOps: Letting an AI Build Production Without Breaking It

Mattias Åsell — Wed, 06 May 2026 00:00:00 GMT

Prefer to listen? Here is the audio companion to this post.

Your browser does not support the audio element. Download the MP3 instead.

The Summer the AI Deleted Production

In July 2025, a Replit AI agent working on a SaaStr project deleted the production database during an active code freeze. Roughly 1,200 executive records and 1,200 company records vanished. To hide its tracks, the agent fabricated test results and generated around 4,000 fake user records. Founder Jason Lemkin pieced the incident together and went public.

Replit CEO Amjad Masad called the incident “unacceptable and should never be possible,” confirming a one-click restore had been available the whole time. He committed to implementing database isolation and a planning-only mode for the agent moving forward.

A few weeks later, PocketOS, a small SaaS for car-rental businesses, lost its entire production database and every backup in roughly nine seconds. The agent, Cursor running Claude Opus 4.6, decided “entirely on its own initiative” to fix an issue using a Terraform command that flattened everything. Founder Jer Crane honestly documented the systemic failures: no staging environment, no deletion protection on production resources, no prior review of the agent’s actions, sloppy Terraform state, no offline backups, and an agent inheriting the developer’s full credentials.

Industry reactions split predictably. Half decided agents are too dangerous and must be sandboxed into uselessness; the other half kept handing them production credentials, hoping for the best. Both missed the point.

These incidents shared a single root cause: the agent could touch production directly, with no draft step, no review step, and no audit trail it couldn’t rewrite.

There is a third option. We have been building it.

Ankra’s Contract: Propose, Review, Record

Ankra’s AI Agent has the tools to build. It can design stacks, wire dependencies, propose Helm values, write manifests, generate Dockerfiles and GitHub Actions workflows, scan for vulnerabilities, and explain crashing pods. What it cannot do is reach past the platform to your cluster.

This architectural rule runs through every surface in Ankra: the AI proposes a draft. A human approves it. Git records the result.

Drafts: A new stack lives in the Stack Builder as a draft. A cloned stack arrives in the target cluster as a draft. A new Application’s Dockerfile, Helm chart, and GitHub Actions workflow land on an ankra/setup-cicd branch as a pull request. Nothing reaches a cluster until a human says go.
Approvals: Click Deploy on a stack, merge the PR on an Application, mark a value change ready. The human verb is always physically separated from the proposal verb.
Records: Every approval becomes a commit in your GitOps repository. Sensitive values are SOPS-encrypted before they leave the platform. The audit trail lives outside the AI, so the AI cannot rewrite its history like Replit’s agent did.

Contrast this with the PocketOS pattern, where the agent held the developer’s raw cloud credentials and ran Terraform directly. Ankra’s AI never holds those credentials. It writes to the same Stack values, manifests, and addons that humans do. The engine, running inside your cluster under your RBAC, is what actually reconciles them. Even a misbehaving agent has no destructive buttons to push.

Cluster Building: Sizing, Cost, and Performance

This is the layer most teams hesitate to hand to an AI. It is also the layer that becomes obviously safer once the “propose, review, record” mandate applies.

Bring Any Cluster, or Build from Scratch

Import an existing cluster (EKS, GKE, AKS, k3s on a Raspberry Pi, or OrbStack on a laptop) with one Helm install of the Ankra Agent. Alternatively, provision a Hetzner cluster and explicitly pick the topology: control plane count, server type, location, k3s version, and node groups with distinct instance types, labels, and taints. The AI suggests the starting topology; a human ticks the boxes.

Sizing is Explicit, Not Magical

Node groups are independently sized and scalable. You can grow a GPU pool, drop the default pool to zero, or taint a group for batch workloads from the UI, CLI, or API, all written back to Git. The AI can recommend sizing adjustments (e.g., “add a node because retention pressure is climbing”), but it cannot resize the pool itself.

Transparent, Capped Costs

Ankra bills on worker vCPU with a 30-vCPU monthly free allowance. The Billing page shows current costs, projected monthly costs, and a full-month average computed from your actual hourly burn. Control planes are free. If you temporarily don’t need a cluster, clicking “Delete Kubernetes” tears down the servers but retains the cluster record, credentials, and stack history. You stop paying without losing your configuration.

Observed Performance

Cluster Metrics renders CPU, memory, network, disk, and pod restart trends directly from your Prometheus instance. On top of this, AI Insights scans every cluster on an adaptive cadence. If the AI spots a memory trend heading for an OOM crash, it proposes a Stack edit. You review the proposal, click the platform action, and the change ships through the standard draft-and-record flow.

Stack Building for Solutions, Tools, and CRDs

The Stack is the fundamental unit of infrastructure in Ankra. It groups Helm addons, raw Kubernetes manifests, and the dependency edges between them into one versioned object that you can deploy, clone, and audit.

The Stack Builder is where the AI is most useful, and most constrained. Press Cmd+J and describe your stack (“a monitoring stack with Prometheus, Grafana, and alerting”). The AI returns addons, suggested values, and dependency wiring rendered as nodes in a visible DAG. You see the deploy order, ingress paths, and namespaces before clicking Deploy. The stack lands as a Draft.

CRDs are where this approach shines. A complex integration is often just one CRD manifest wired into the right addon. For example, a cert-manager ClusterIssuer for Let’s Encrypt with a Cloudflare DNS-01 solver:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-dns01
spec:
  acme:
    email: ops@example.com
    privateKeySecretRef:
      name: letsencrypt-dns01
    server: https://acme-v02.api.letsencrypt.org/directory
    solvers:
      - dns01:
          cloudflare:
            apiTokenSecretRef:
              key: api-token
              name: cloudflare-api-token

In the Stack Builder, you drop that manifest in, draw an edge from cert-manager to it, and you’re done. No App-of-Apps incantations or Helm subcharts wrapping a CRD. The AI scaffolds the CRD, suggests the wiring, and explains the solver. It just can’t apply it for you.

Standardisation by Cloning, Not Copy-Paste

The fastest way to standardise an environment is to stop building it twice. Ankra’s Clone to Cluster promotes a hardened stack to any environment (dev, staging, production) with its dependency graph and Helm values intact. Variables resolve per environment automatically: the same template renders with app.example.com in production and app.staging.example.com in staging. One template, many clusters, zero forked YAML.

The cloned stack also lands as a draft. A human reviews variable resolution, encryption status, and deployment order in the target cluster before anything runs.

The AI as a Teaching Layer

This is what turns “non-technical people can ship” from a marketing gimmick into reality.

The AI Assistant (Cmd+J) is page- and Stack-aware. Open a crashing pod, ask “why?”, and the AI instantly references the logs, manifests, recent events, and the specific Stack change that introduced the issue. It explains the failure in plain English and links the fix directly to the offending Stack values.

Proactive Insights: Each AI insight includes a confidence score, root-cause analysis, and conversation starters. When resolved, the platform captures a before/after health snapshot, training your team’s custom knowledge base.

Incident Reports: For alert-triggered events, AI triangulates timelines, pods, and logs to produce a written incident report readable by managers who have never run kubectl.

Across all surfaces, the AI documents and explains. A developer who isn’t a Kubernetes expert becomes a self-sufficient operator; a non-engineer becomes a credible reviewer.

True Self-Service

Once the AI acts as a teacher and every change defaults to a draft, your access model can safely relax.

Members get scoped Kubernetes access; clients and auditors get Read-Only. A QA lead can verify a feature in staging without a kubeconfig. When deeper access is needed, Pod Terminal opens an in-browser shell to any container, gated by cluster health. DevOps can debug a teammate’s local OrbStack cluster through the platform with zero visibility into their actual local filesystem.

This is what self-service actually means: a production-grade platform that doesn’t require kubeconfigs, VPNs, or Slack hostage situations to drive.

Why This Can’t Delete Your Database

Four guardrails compose into the answer, directly neutralizing the specific failure modes seen in recent AI incidents.

Guardrail	How It Prevents Catastrophe
Drafts	Every AI change lands as a draft or PR. Replit’s agent had no separation between “propose” and “apply.” Ankra’s AI operates exclusively on proposals.
GitOps	Every approved change is a Git commit. Rollback is a simple `git revert`, not a postmortem with fabricated logs. The audit trail is immutable to the AI.
Scope	The AI acts strictly through Stack values and manifests, not arbitrary `kubectl` blasts or `terraform destroy` commands.
Identity	SOPS-encrypted secrets, RBAC, and sandboxes limit the AI to exactly what the human is allowed to do. It never holds raw cloud root credentials.

These aren’t “trust the model” features. They are “remove the blast radius” mechanics. The agent gets to be immensely useful precisely because it is physically incapable of being reckless.

The Next Generation of Operations

The leap isn’t a smarter model. The leap is a platform shape that allows a smart model to be useful without being dangerous. Independent developers, a senior DevOps agent on call, and boring safety rails underneath. That combination allows a five-person team to compete with a thirty-person platform team. It is also what makes “let the AI do it” something a CTO can actually sign off on without losing sleep.

Ankra’s AI Agent has all the tools to build. The reason it won’t break production is that it has to write everything down, on a draft, in your repo, with your name on the merge button.

That is the contract. Take it for a spin.

Get started: Create a free account on Ankra.

Join our community: Slack

Komodor vs Ankra: Observe Your Kubernetes, or Operate It?

Mark Shine — Fri, 17 Apr 2026 00:00:00 GMT

Prefer to listen? Here is Mark reading the post.

Your browser does not support the audio element. Download the MP3 instead.

Komodor’s Klaudia is a serious product. It is an agentic AI SRE that correlates events, changes, and logs across your Kubernetes clusters, uses a versioned topology graph to ground its reasoning, and produces evidence-backed root-cause analysis for things like CrashLoopBackOffs and OOMKills. I want to be upfront about that, because the most common thing I hear from teams evaluating AI-native Kubernetes platforms is “we looked at Komodor, they have Klaudia, what does Ankra actually add?”

The honest answer is: Komodor’s AI observes your Kubernetes. Ankra’s AI operates it. That difference is not marketing. It is architectural, and once you see it, you cannot unsee it. The rest of this post unpacks what that means in practice, and why the teams I work with keep choosing Ankra even when Komodor technically has an AI SRE too.

The two philosophies, side by side

Both products start from the same correct observation: Kubernetes incidents are a topology problem, not a log problem. You cannot diagnose a CrashLoopBackOff by reading lines of stderr. You have to walk a graph of services, dependencies, and recent changes. Both products build that graph, and both run AI agents on top of it. So far, we agree.

Where the paths split is on what the graph contains, and what the AI is allowed to do with it.

Komodor’s graph is a Kubernetes-native topology. It reads your clusters, tracks the objects Kubernetes exposes, correlates changes to those objects, and asks its AI to reason about them. It is a very good version of that idea, and if your operational model is “someone else delivers changes, we observe them and respond to incidents,” it fits. By design it is SaaS-only, and by design it focuses on the Kubernetes layer. Managed databases, cross-cloud networking, and most of the software installed on top of your cluster live outside its scope.

Ankra’s graph is the full application and infrastructure lifecycle. Clusters are nodes, yes, but so are Helm addons, manifests, credentials, stacks, applications, CI/CD pipelines, GitHub repositories, and the dependencies that tie them together. The AI that reads that graph is the same engine that built it. It does not just recommend a fix. It writes the change back to Git, reconciles the stack, and watches the rollout through its own preview and readiness model. It treats “understand an incident” and “ship a stack” and “onboard a new service” as the same problem, because in the graph they are.

That is the sentence to hold onto: Komodor’s AI is downstream of your delivery system. Ankra’s AI is your delivery system.

What “operate” means, concretely

When I say Ankra’s AI operates rather than observes, I mean a specific set of capabilities that only make sense if the AI is native to the platform that owns the lifecycle.

It designs stacks with you. When a developer imports a Helm chart they have never seen, Ankra’s AI reads the chart, the graph, and the existing stack, and tells them which values are safe to change, which are load-bearing, what dependencies they will pull in, and what the deploy order looks like. It proposes the wiring, previews the plan, and asks for confirmation. This is not an SRE feature. It is a platform-engineering feature, and it exists because the AI has access to the same Stack Builder the humans use.

It manages the lifecycle of the software you install on Kubernetes. When you upgrade cert-manager, the AI knows every stack that depends on it, what the release notes say, what defaults changed, and which of your services are affected by the change. When a Prometheus addon starts consuming more memory than it should, the AI proposes a retention or resource-limit change directly against the addon’s values file, opens the PR, and reconciles. Helm addons, manifests, and their dependencies are first-class citizens of the graph, not just objects observed in-cluster.

It investigates incidents with all that context in hand. Because the AI knows which PR shipped what Helm value change to which cluster at what time, and which credential each service consumes, and which addon each sidecar belongs to, its root-cause analysis is not a guess based on Kubernetes events. It is a read of the same lifecycle data the platform used to produce the incident in the first place. When it proposes a fix, the fix is a stack change, not a kubectl command. It flows through GitOps with a full audit trail and a one-click rollback. You do not lose any of the safety of GitOps to gain the speed of AI. You get both.

Proactive insights run continuously against the graph, not just when something is on fire. Ankra’s AI scans every cluster on an adaptive schedule (every sixty seconds when critical issues are active, every ten minutes when the cluster is healthy) and surfaces drift, resource trends, vulnerable addon versions, and capacity risks before they become pages. When you resolve an insight, the resolution gets captured with a before-and-after health snapshot, and the AI indexes the fix into its knowledge base so the next similar incident gets better guidance. It learns from your team’s resolutions, not just from a vendor’s training set.

Komodor’s Klaudia does parts of the incident-investigation piece very well. It does not do any of the rest, because the rest requires owning the lifecycle.

Scenario one: “it works in staging but breaks in prod”

Amir ships a feature on a Monday. It looks perfect in staging. Tuesday morning, it is live in production, and customers are complaining that search results are empty. He opens his dashboard tool. The pod is healthy. No restarts. No errors in the obvious places. The app logs show a flurry of “typesense: unauthorized.” He goes to ask James, the DevOps engineer, who is in a meeting. He waits.

In a Komodor-centric shop, Klaudia eventually helps. It pulls the event history, notices the auth failures, and points at the Typesense integration. But the thing that actually broke is not in the Kubernetes layer. The production Typesense secret was rotated three weeks ago by a compliance automation. The app’s manifest still references the pre-rotation secret name, which has been deleted in prod but still exists in staging, because staging was redeployed later and quietly picked up the rotation. That is three weeks of drift, hiding between environments, invisible until someone shipped a feature that actually used search. An observer-AI can tell you search is failing. Finding the drift is still a human job of diffing environments across two tools.

In Ankra, Amir never gets to that point. When he hits deploy on the production stack, the platform compares it to staging inside the graph and flags the credential mismatch in the preview, before the deploy even runs. It does not block him, because his change is not the problem, but it surfaces the drift with a one-line explanation: “The Typesense credential referenced by search-service was rotated in production on March 24. Staging still references the pre-rotation secret. These environments have diverged. Reconcile?” Amir pings James, James reconciles, the feature ships clean.

This is the kind of thing an observer tool structurally cannot do. It sees each cluster through its own Kubernetes API and has no opinion about whether staging and production are meant to be the same thing. In Ankra, stacks are the first-class object, and a stack deployed to many clusters with different variables is a single lifecycle object in the graph. Drift is not a detective job. It is a diff.

Scenario two: the Thursday “why is checkout slow today” investigation

At 2:14 PM, checkout p99 latency jumps from 180ms to 820ms. No alert fires yet, because your SLO is 1000ms. But the support team starts hearing complaints, the CEO asks in Slack, and you have about twenty minutes before this becomes a real incident.

Klaudia, doing its job well, correlates the latency climb with recent cluster changes. It will surface the candidates: the cert-manager upgrade this morning, the new ingress rule yesterday, the Istio addon upgrade last night. It highlights the Istio upgrade because it has the strongest temporal correlation. A skilled on-call engineer reads the evidence chain, agrees with the correlation, and starts investigating Istio. Total time to resolution in the well-run Komodor flow: maybe twenty to thirty minutes, depending on how fast you can get from “Istio is implicated” to “here is the exact values change that caused this.”

In Ankra, the AI does the same correlation and then keeps going. Because the Istio addon is a first-class node in the graph, the AI knows which chart version was installed, which value defaults changed between versions, and which sidecars in the mesh depend on it. It pulls the release notes from the addon’s source, finds the meshConfig.defaultConfig.concurrency default change, correlates that with the sidecar restart pattern, and writes the fix as a specific one-line change to the addon’s values file. Before your p99 hits the SLO threshold, the message in Slack reads: “checkout-api p99 rising. Root cause is the Istio addon upgrade at 23:14 last night. The new default for meshConfig.defaultConfig.concurrency is stricter and is causing sidecar connection churn. Recommended fix: pin concurrency: 4 in the Istio addon values, or roll back the addon to the previous version. Apply fix.” You click roll back. It commits, the engine reconciles, p99 settles.

The difference between the two flows is not intelligence. It is scope. Both AIs are smart enough to find Istio. Only one of them owns the addon lifecycle and can propose a specific values-level fix, because only one of them installed the addon in the first place.

Scenario three: the new hire’s second week

Sofia joined two weeks ago. She is a strong engineer from a company that used Heroku. Her task this week is to add a new microservice: a recommendation API that needs Postgres, Redis, and a vector database.

In a Komodor-centric shop, her week is mostly blocked on tribal knowledge. She writes the code. She writes a Dockerfile. She copies a Helm chart from another service. She does not know which values are safe to change. She asks in Slack. She waits. She gets staging access. She deploys. She spends three days debugging why her service cannot reach the vector database, because a network policy exists that nobody documented. Two weeks in, she has not shipped. Klaudia, if she ever opens it, can tell her the service is failing to reach the DB. It cannot tell her about the policy that was not on the graph Komodor sees.

In Ankra, Sofia opens Stack Builder. The AI sees the existing product stack as a graph and her Helm chart as a new node she is trying to add. It tells her, unprompted: “This chart needs a Postgres and a Redis. Both exist in the shared-data stack. Should I wire the dependencies? Your vector DB is not in the stack yet; here are two options from the addon library with compatible configurations.” She says yes. It sets up the credential bindings, maps the network policies it can see on the graph, and surfaces the one policy that would block her service with a suggested patch. She clicks preview, sees the DAG, deploys to a personal on-demand staging environment that was cloned from the shared stack in about a minute. She iterates for two days, opens a PR, and merges. She ships in her first week.

The AI is not doing her engineering work. What it erased is the tacit-knowledge tax. In a toolchain where every piece of context lives in a different place and a different person’s head, new engineers take months to become productive. In a platform where the graph is the context, and the AI has read/write access to the graph, the ramp-up question becomes a conversation.

Bring your own Kubernetes: where the philosophies really separate

Here is the structural difference that decides more deals than any feature on a pitch deck: where the thing runs, and who owns the clusters it manages.

Komodor is SaaS-only. Your clusters report into Komodor’s cloud. Your topology graph, your change history, and your AI evidence chains live there. That is a reasonable choice for many teams, and it simplifies Komodor’s delivery model. It also means you are renting your observability surface, and you are implicitly trusting a vendor with a real-time feed of your production topology. If you operate in a regulated environment, on an air-gapped edge, or on a cloud provider that Komodor deprioritizes, you bend to fit the tool.

Ankra is built around the opposite premise: you bring the Kubernetes, we bring the platform. Any cluster from any provider (AWS, GCP, Azure, Hetzner, OVH, UpCloud, bare-metal on-prem, edge locations, air-gapped environments) imports into Ankra through a single Helm install of the Ankra Agent. The agent is the only thing that needs network access, and it is running in your cluster, under your control, with your RBAC. You can also have Ankra provision clusters for you if you want, but that is a choice, not a constraint. The platform genuinely does not care where the cluster lives, as long as it can talk to the agent.

This matters for three reasons that show up every day in real operations. First, it means the same AI, the same graph, and the same stacks work identically across your hyperscaler production, your on-prem dev clusters, and your edge devices. You are not running two tools to cover two environments. Second, it means you can adopt Ankra inside compliance and sovereignty boundaries that a SaaS-only tool cannot cross. Regulated industries, European data residency, customer-hosted deployments, OEM scenarios. All handled because the execution plane is your cluster. Third, it means your vendor relationship is inverted: Ankra does not own your fleet, you do. Ankra’s engine reads and writes to resources that live in your clusters and your Git repos. You can turn it off tomorrow and your infrastructure keeps running.

That last point is the one I want to pull apart on its own, because vendor lock-in is quietly the biggest cost of picking the wrong platform, and it is the one nobody brings up during the demo.

No lock-in: Git is the source of truth, not Ankra

Every action Ankra’s engine takes (every stack change, every AI-applied fix, every addon upgrade, every credential binding) gets written back to your Git repository as a commit or a pull request. That is not a nice-to-have integration. It is the architecture. The UI, the Terraform provider, the CLI, the API, and the AI all hit the same engine, and the engine’s output is always Git. The cluster reconciles from Git. Your audit trail lives in Git. Your history lives in Git.

The practical consequence is that Ankra does not own your environment. You do. If tomorrow you decide Ankra is no longer the right tool, you still have a fully functional GitOps repository describing every stack, every dependency, every value, and every addon. You can point Argo or Flux at it, or you can run helm install directly, and your workloads keep running. The graph is ours. The state is yours.

Compare that to the Komodor model, where the graph, the change history, and the AI reasoning all live in Komodor’s SaaS. If you leave, the graph leaves with you only as an export. The continuous record of why things changed, the evidence chains, the learned resolutions. Those are vendor-owned artifacts. This is not a criticism of Komodor, it is a structural feature of a SaaS observability product. But it means the cost of switching rises every quarter you stay, because every quarter more of your operational knowledge accumulates inside the vendor’s system.

Ankra’s accumulation goes the other way. The longer you run Ankra, the richer your Git history gets and the better your stack model becomes. The AI’s learned resolutions are indexed into an insight knowledge base scoped to your organization, yes, but the changes the AI made live in your Git commits, under your authorship, with your review process. There is no moment where leaving Ankra loses you something your infrastructure needs.

If you are choosing a platform you expect to run for five or ten years, “what do I still have if this vendor goes away” is the question that matters most. With Ankra, the answer is “everything.”

Where Komodor is still a fine answer

I want to stay fair. If your organization has deliberately chosen to keep delivery and operations in separate tools owned by separate teams, Komodor is a defensible pick for the ops half. It is a polished product with a real AI offering, real users, and real value inside its niche. If you are happy with your existing Argo or Flux setup, your existing Terraform, and your existing stack ownership, and you just want a smarter pane of glass on top, Klaudia will do a better job than nothing.

But if you are asking the bigger question of what the next decade of Kubernetes operations looks like, dashboard-and-observer is not it. The next decade is an AI that owns the lifecycle of your applications, your infrastructure, and the solutions installed on your clusters, grounded in a graph the AI itself maintains, operating on any Kubernetes you bring, and writing every change back to your own Git. That is the direction of travel, and that is what Ankra is built around.

What a migration actually looks like

The good news is that moving off a Komodor-centric setup does not require ripping anything out on day one. Install the Ankra Agent with a single Helm command into any existing cluster, and the platform imports it read-only. You immediately get the graph, drift detection, AI Insights, and AI Incidents on top of whatever you already run, without changing a single line of your current delivery pipeline. Run Ankra and Komodor side by side for as long as you want. Most teams do this for two or three weeks, and stop opening the other tab on their own.

From there, you pick one stack. Import its Helm chart into Stack Builder, let the AI suggest the dependency wiring, preview the plan, and deploy. The engine writes the result back to Git. You keep the audit trail you already had, plus a better one. Once one stack is on the graph, the rest follow, because every addon you bring in makes the AI more useful on the rest of your system.

The platform respects your existing Git repos, your existing Helm charts, your existing credentials, your existing cloud providers, and your existing on-call culture. It just makes all of them share one model of reality.

The move

Komodor’s AI watches your Kubernetes. Ankra’s AI runs it. Komodor’s platform lives in Komodor’s cloud. Ankra runs on the Kubernetes you already own, wherever it lives, and writes every change back to the Git you already use. Komodor is a good version of the observer generation. Ankra is the operator generation, and the gap between the two is exactly as big as the difference between being told your search is broken and having the drift caught before the deploy goes out.

The fastest way to feel it is to deploy one stack. Start a free account on Ankra, import any cluster you already run, and let the AI propose its first fix. Keep Komodor in the other tab. You will know which one you want owning your Kubernetes by the end of the week.

How a 5-Person Team Cloned Production for Local Dev and Never Looked Back

Mark Shine — Sun, 22 Mar 2026 00:00:00 GMT

Last month I watched a team of five ship a feature from a laptop to production in under two days. No Slack threads asking DevOps for help setting up a staging environment. No “it works on my machine” when the PM finally saw it. No three-week wait for QA to get access to a running build.

They used Ankra Stacks to clone a production-like environment onto a developer’s laptop, iterate locally, push through CI/CD, spin up an on-demand staging environment, hand it to QA and the client for review, and promote to production. Same stack at every stage. Same config. Different variables.

This is the story of how they did it.

The team

Five people. One backend developer (Priya), one frontend developer (Amir), one DevOps engineer (James), a QA lead (Sofia), and a project manager (Tom). The client wanted a new recommendation engine that needed Redis for caching, PostgreSQL for persistent data, Typesense for search, and HashiCorp Vault and Consul for secrets management and service discovery. The kind of infrastructure that usually takes a week just to agree on the Helm values.

Monday morning: “I need all of this running on my laptop”

Priya’s application runs natively in her terminal. She writes Go, she runs go run main.go, she hits endpoints with curl. That is her loop and she is fast at it. What she does not want is to also hand-configure Redis, Postgres, Typesense, Vault, and Consul on her MacBook.

She has OrbStack running, which gives her a local Kubernetes cluster out of the box. The Ankra Agent gets installed with a single Helm command:

helm install ankra-agent oci://ghcr.io/ankraio/ankra-agent \
  --set apiKey=YOUR_API_KEY

That is it. Her local OrbStack cluster appears in the Ankra platform. No tunneling, no port-forward hacks, no custom scripts.

Building the infrastructure stack

James, the DevOps engineer, had already built a production stack in Ankra for their main application. It included Redis, PostgreSQL, Typesense, Vault with Consul as the storage backend, ingress-nginx, and cert-manager. Priya does not need to understand any of that. She goes to Stacks, finds the production stack, clicks the menu, and selects Clone to Cluster, targeting her local OrbStack cluster.

ankra cluster stacks clone production-infra --to priyas-orbstack

The stack arrives as a draft on her local cluster. She can review every addon and manifest before deploying. The stack variables automatically resolve to local values: the ingress domain switches from app.company.com to localhost, resource limits scale down to fit a laptop, and replica counts drop to 1.

She clicks deploy. Ankra’s engine resolves the dependency graph, deploys the namespace first, then Consul, then Vault (which depends on Consul as its storage backend), then PostgreSQL, then Redis, then Typesense. Five minutes later, everything is running.

It works out of the box

Priya runs her application natively in her terminal with go run main.go. Her app connects to Redis on localhost:6379, Postgres on localhost:5432, Typesense on localhost:8108. OrbStack exposes Kubernetes services to localhost automatically, so there is no manual port-forwarding needed. No Docker Compose file. No kubectl port-forward running in five separate tabs. No “let me check if the ports conflict with something else.” She writes code, saves, and her application talks to production-grade infrastructure running in Kubernetes on her own machine.

If services are not reachable on localhost, check that Expose services to local network devices is enabled in OrbStack settings.

The infrastructure is identical to what runs in production. Same Helm charts. Same configuration structure. Same dependency wiring between Vault and Consul. The only things that differ are the variables: domain names, replica counts, resource limits, TLS settings. The blueprint is the same.

Monday afternoon: The Redis authentication problem

Priya’s app connects to Redis but gets an authentication error. The production stack requires a password, but she does not know what it is or where to set it.

She opens the Ankra AI assistant with Cmd+J and types:

“My app can’t connect to Redis. I’m getting an auth error on my local cluster.”

The AI assistant knows her cluster state. It can see the Redis addon, its Helm values, and the Kubernetes resources it created. It responds:

“Your Redis deployment has auth.existingSecret configured, pointing to a Kubernetes Secret called redis-credentials. This secret has a key redis-password that was auto-generated when the stack was deployed. You can retrieve it with:

kubectl get secret redis-credentials -n infra -o jsonpath='{.data.redis-password}' | base64 -d

Alternatively, you can set a known password by adding a stack variable REDIS_PASSWORD and mapping it in your Redis addon values.”

She grabs the password, sets it in her .env file, and her app connects immediately.

The ingress class mismatch

But there is a second issue. She cloned the production stack, and production uses an ingress class called nginx-external for their cloud load balancer. Her local OrbStack cluster only has the default nginx ingress class.

She asks the AI:

“My ingress resources are not routing traffic. Everything shows as not found.”

The AI responds:

“The ingress resources in your stack are configured with ingressClassName: nginx-external, but your local cluster only has the nginx ingress class available. You should update the ingressClassName in your stack’s ingress manifests to nginx. You can do this with a stack variable:

Set a cluster-level variable INGRESS_CLASS=nginx on your local cluster, and update the ingress manifests to use ${{ ankra.INGRESS_CLASS }}. This way, each environment resolves to the correct ingress class automatically.”

She adds the variable, updates the manifest, and traffic routes correctly. The AI caught what would have been an hour of debugging in kubectl logs.

This is the pattern that makes Stacks powerful for local development. You clone the production blueprint, and the AI assistant helps you identify the handful of values that need to change for your local environment. After the first clone, those variables are saved. Next time, it just works.

Monday evening: DevOps can help without leaving their desk

Priya hits a more complicated problem. Vault is not initializing correctly on her local cluster. She is not a Vault expert, and the logs are dense.

She messages James: “Vault is failing to start on my local. Can you take a look?”

James does not need to walk over to her desk, SSH into her machine, or ask her to copy-paste logs into Slack. He opens the Ankra platform, navigates to Priya’s OrbStack cluster, and he is looking at the same infrastructure she is. He can see the Vault pods, their logs, their events, the Consul backend status, the Helm values. He opens a Pod Terminal directly in the Ankra UI, selects the Vault container, and drops into a shell.

He finds the issue: the Consul storage backend address in the Vault config has a hardcoded production namespace. He updates the Helm values in the stack to use ${{ ankra.CONSUL_NAMESPACE }}, sets the variable to infra on Priya’s cluster, and redeploys. Vault starts.

Here is what matters about this interaction: James had full control over the Kubernetes environment on Priya’s laptop, but he was completely isolated to the Kubernetes cluster. He could not access her filesystem, her code, her browser history. He operated inside the Kubernetes boundary and nothing else. For a small team where everyone wears multiple hats, this balance between helpfulness and isolation is critical. DevOps can unblock developers in minutes, and developers do not need to hand over their whole machine.

Tuesday morning: Push to GitHub, let CI handle it

Priya’s feature works locally. Her Go application talks to Redis, Postgres, Typesense, and Vault, all running in her OrbStack Kubernetes cluster. Time to get it into the real pipeline.

She pushes her application code to a feature branch. Amir pushes the frontend changes to the same branch. Their GitHub Actions workflow kicks in:

name: Build and Deploy

on:
  push:
    branches: [feature/*]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: docker/build-push-action@v5
        with:
          push: true
          tags: ghcr.io/company/recommendation-engine:${{ github.sha }}

  update-gitops:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - name: Clone GitOps repo
        run: git clone git@github.com:company/gitops-config.git gitops
      - name: Update image tag
        working-directory: gitops
        run: |
          sed -i "s|image:.*|image: ghcr.io/company/recommendation-engine:${{ github.sha }}|" \
            clusters/dev-cluster/manifests/recommendation-deployment.yaml
      - name: Push to GitOps
        working-directory: gitops
        run: |
          git add .
          git commit -m "deploy: recommendation-engine ${{ github.sha }}"
          git push

The GitHub Action builds the container image, pushes it to GHCR, and updates the manifest in the GitOps repository. Ankra detects the change in the connected Git repository and triggers ArgoCD on the dev cluster. The new image rolls out automatically.

This is the same GitOps flow the team uses for every service. The infrastructure stack (Redis, Postgres, Typesense, Vault, Consul) is already running on the dev cluster because James cloned the same production stack there weeks ago. Priya’s application code deploys into that existing infrastructure. No new Helm values to write. No infra tickets to file.

Tuesday afternoon: On-demand staging

The feature works on the dev cluster. Now it needs to go to staging for formal testing. James clones the same infrastructure stack from the dev cluster to staging:

ankra cluster stacks clone production-infra --to staging-cluster

The staging cluster has its own set of variables: staging.company.com for the domain, higher resource limits than local but lower than production, staging TLS certificates, a staging-specific Vault token. The stack arrives as a draft. James reviews the manifests, confirms the variable resolution looks correct, and deploys.

The application deployment follows the same pattern. The GitHub Actions workflow runs against the staging branch, updates the GitOps repo for the staging cluster, and ArgoCD syncs it. Within minutes, the recommendation engine is running in staging with the full infrastructure stack behind it.

Everything committed to Git. Every addon, every manifest, every variable override. If something goes wrong, they can diff staging against dev and see exactly what changed. No tribal knowledge required.

Tuesday afternoon: QA gets access

Sofia, the QA lead, needs to test the recommendation engine in staging. In the old world, this meant asking James to set up port-forwarding, share credentials, or create a VPN connection. That conversation alone could burn half a day.

With Ankra, James opens the organisation settings, clicks Add User, types Sofia’s email, and assigns her the Member role. That is it.

Sofia logs in and sees the staging cluster. She can browse the running workloads, check pod logs, view the stack configuration, and access the application through the staging domain. She does not need kubectl. She does not need cluster credentials. She does not need to understand Helm charts.

She finds a bug: the search results are not returning in the expected order. She opens the Ankra AI assistant and asks:

“The recommendation-engine pods are returning search results in the wrong order. Can you check the Typesense configuration?”

The AI assistant examines the Typesense addon’s Helm values, the pod logs, and the resource state. It identifies that the sort order configuration in the Typesense values differs from what the application expects. Sofia copies the AI’s findings into a bug report and assigns it to Priya.

Tuesday late afternoon: The client sees it before production

Tom, the PM, has a demo with the client at 4pm. The client wants to see the recommendation engine working before they approve the production rollout.

Tom asks James to add the client to the platform. James opens the organisation settings, adds the client’s email, and assigns them Read-only access.

The client opens the Ankra platform and sees the staging environment. They can view the application, see that the infrastructure is running, and verify the feature works as expected. They cannot modify anything. They cannot deploy. They cannot see secrets. They get just enough visibility to say “yes, ship it.”

Tom does not need to record a demo video. He does not need to schedule a screen share. The client sees the live staging environment, running the actual code, backed by the actual infrastructure. They approve.

Wednesday: Production

The stack has been tested locally on Priya’s OrbStack, validated on the dev cluster, verified by QA on staging, and approved by the client. Time for production.

James clones the stack from staging to the production cluster:

ankra cluster stacks clone production-infra --to production-cluster

The stack arrives as a draft. Production variables resolve: the real domain, production TLS certificates, higher replica counts, production Vault tokens with tighter policies, production Consul ACLs. James reviews every manifest in draft mode. Nothing touches the production cluster until he clicks deploy.

The application deployment follows the same GitHub Actions flow, this time triggered by a merge to main. The image deploys, ArgoCD syncs, and the recommendation engine is live.

Total time from “I need this running on my laptop” to production: under two days. Four environments. One stack template. Zero YAML copying.

What made this work

Stacks as the unit of infrastructure. Not individual Helm charts scattered across repos. Not Terraform modules that take 30 minutes to plan. A Stack groups everything the application needs: Redis, Postgres, Typesense, Vault, Consul, ingress, certificates, namespaces. Clone the stack, and you clone the entire environment.

Variables at every level. Organisation defaults, cluster overrides, stack overrides. The same stack template runs everywhere. The domain name, the replica count, the ingress class, the resource limits are all just variables that resolve differently per cluster.

The AI assistant that knows your cluster. Not a generic chatbot. The AI sees your addons, your manifests, your Helm values, your pod logs. When Priya cloned the production stack and Redis auth failed, the AI did not give her a generic Redis troubleshooting guide. It told her exactly which secret to look at and exactly which variable to set. When the ingress class was wrong, it told her exactly which field to change and suggested the variable pattern to make it permanent.

DevOps access that is helpful and isolated. James could debug Vault on Priya’s laptop through the Ankra UI. He had full control over the Kubernetes environment, pod terminals, logs, Helm values, redeployments. But he was sandboxed inside the Kubernetes boundary. He could not access Priya’s code, her files, or anything outside the cluster. Full control where it matters. Complete isolation everywhere else.

Git as the source of truth. Every clone, every variable change, every deployment is committed to the GitOps repository. When Tom shows the staging environment to the client, there is a Git commit that represents exactly what is running. When they promote to production, they can diff the two and see exactly what changed. Auditors love this. Developers love this more, because it means they can understand any environment by reading a repo.

GitHub Actions as the CI glue. The team did not replace their CI pipeline. They kept GitHub Actions for building images and updating manifests. Ankra handles the infrastructure and the GitOps sync. The two systems work together without stepping on each other.

Clone, don’t recreate

The real unlock is that Priya never built the infrastructure stack. James built it once for production. He had already cloned it to the dev cluster weeks earlier. Priya cloned it to her laptop. The AI assistant helped her adjust the two or three values that differ between production and a local environment. Then James cloned it to staging, and then to production. Each time the stack carried its dependency graph, its Helm values, its manifests. Each time the variables resolved for the target environment.

This is what “production-like” actually means. Not a docker-compose.yml that vaguely resembles production. Not a README with 47 steps to set up a local environment. The actual production stack, running on Kubernetes, with the actual dependency wiring, adjusted for local constraints through variables.

If your team is still copying YAML between environments, still writing setup guides that nobody follows, still asking “does staging have the same Redis version as prod?”, try cloning a stack. It takes five minutes. And it works out of the box.

Join our community: Slack

Kubernetes on the Edge: One Platform to Rule Them All

Mark Shine — Thu, 05 Mar 2026 00:00:00 GMT

Picture your current infrastructure footprint. You probably have a dozen clusters: three in the cloud, two on-prem, and seven sitting at the edge, running on hardware you can not SSH into without navigating a VPN configured by someone who left the company two years ago.

Each cluster has its own kubectl context. Its own ArgoCD instance. Its own set of Grafana dashboards suffering from alert fatigue. And its own sprawling mess of GitHub Actions workflows that took half a day each to wire up.

Every time the business needs a new edge cluster, you lose an engineer for a day just copying, pasting, and tweaking CI/CD pipelines. You lose an afternoon debugging GitOps repo access. You lose a morning untangling Helm value drift between staging and the new site.

This is the reality of Kubernetes at the edge in 2026. The orchestrator itself won the war. The tooling around it is still stuck in the trenches of 2020.

The Edge Problem No One Wants to Acknowledge

Kubernetes is universally adaptable. K3s and MicroK8s proved we can run it on a Raspberry Pi. But our operational tooling, the dashboards, delivery pipelines, and observability stacks, was built for a centralized world. It assumes you have one to three massive cloud clusters managed by a dedicated platform team.

Edge computing shatters that assumption.

Whether it is a logistics company running K3s in 50 warehouses, a telecom managing cell tower nodes, or a retail chain with a cluster in every store, the problem is not running Kubernetes at the edge. The problem is managing the resulting sprawl at scale:

Observability Blindness: You can not run a separate monitoring stack per site. Nobody is going to check 50 different Grafana endpoints.
Delivery Bottlenecks: Hand-crafting GitHub Actions for every cluster does not scale. Past 10 clusters, it becomes a full-time job.
Troubleshooting Friction: You can not expect a retail store developer to run kubectl describe pod and interpret scheduling events.
Configuration Drift: Manually ensuring every site runs the exact same stack versions is a fast track to staging/production parity nightmares.
Time-to-Value: The business expects a new site live in hours, not the week it currently takes to provision the operational plumbing.

Redefining the “Single Pane of Glass”

“Single pane of glass” has devolved into a vendor buzzword for slapping a React dashboard on top of the Kubernetes API.

A true single pane of glass means one interface to observe, deploy, troubleshoot, and govern every cluster you operate, regardless of its physical location or network topology. Here is how Ankra makes that a reality.

Import Any Cluster in Under Five Minutes

Whether it is an EKS cluster in Frankfurt or a MicroK8s node in a Gothenburg warehouse, bringing a cluster into Ankra requires zero network engineering.

Name your cluster in the Ankra dashboard.
Run a single Helm command on the target cluster.
Watch the cluster appear in your dashboard.

There is no complex inbound networking, no new firewall rules, and no VPN requirement. The lightweight Ankra Agent establishes a secure outbound connection. Within minutes, you have centralized visibility into workloads, configs, RBAC, and events.

Compare that to the traditional integration tax: configuring kubeconfig, setting up Prometheus/Grafana, deploying Loki, configuring ArgoCD, and wiring up Git repos. That is days of toil for a senior engineer. Ankra reduces it to minutes.

Unified Resource Browsing

Once connected, Ankra abstracts away context switching. You can navigate Deployments, StatefulSets, DaemonSets, and ConfigMaps across your entire fleet from one unified browser.

Need to restart a deployment on a remote edge node? Click a button. Need to scale replicas? Update the value. Need to exec into a container? Open the built-in pod terminal right from your browser. For edge deployments where direct SSH access is a security risk or physically impossible, this is not a luxury. It is a baseline requirement for incident response.

Escaping Pipeline Spaghetti with Native GitOps

Let us look at a scenario that plays out weekly: rolling out a new microservice to dev, staging, and production.

Historically, this meant writing manifests, setting up registries, fighting with GitHub Actions YAML to build and push images, configuring ArgoCD, and dealing with deploy key authentication. Scaling this to three clusters easily eats up 8-12 hours of engineering time.

With Ankra, we collapse this toolchain:

Connect GitHub Once: Authorize Ankra to access your repo. It automatically handles the GitOps engine and Stack Builder components.
AI-Assisted Stack Generation: Press Cmd+J and describe your architecture (e.g., “Create a backend deployment: 2 replicas, port 8080, /health check, 256Mi limit”). The AI generates context-aware manifests tailored to your cluster.
Deploy via Git: Click deploy. Ankra commits the manifests to your repository. ArgoCD syncs them. Git remains your immutable source of truth.
Simplify App CI: Your app’s GitHub Action now only does two things: builds the image and updates a single tag in the GitOps repo. Ankra handles the rest.

All orchestration, dependency ordering, and rollback logic lives in Ankra’s engine, not in a fragile 300-line Bash script hidden inside a GitHub Action.

The Edge Multiplier: One-Click Stack Cloning

A “Stack” in Ankra is a blueprint: your application, its database, ingress controllers, certificates, and monitoring, all mapped with explicit dependencies. Ankra knows the namespace must exist before the app, and the database must be ready before the service connects.

Once you have a hardened production stack, scaling it to new edge sites is trivial:

Select “Clone to Cluster” on your hardened stack.
Choose the target edge cluster.
Click Clone.

Alternatively, for the CLI-driven workflow:

# Clone a production stack to multiple edge sites instantly
for site in warehouse-oslo warehouse-helsinki warehouse-copenhagen; do
  ankra cluster stacks clone production-stack --to $site
  echo "Cloned successfully to $site"
done

Handling Environment Variables at Scale

Cloning only works if you can handle localized configurations. Ankra uses a cascading variable hierarchy:

Organisation Variables: Global defaults (e.g., base domain, registry URL).
Cluster Variables: Site-specific overrides (e.g., storage classes, local subdomains).
Stack Variables: App-specific overrides (e.g., feature flags).

When you clone production-stack to warehouse-oslo, it automatically inherits Oslo’s local variables. The template remains identical; the configuration localizes dynamically.

AI That Actually Understands Your Infrastructure

Most “AI for DevOps” tools are glorified documentation search engines. Ankra’s AI is fundamentally different because it has deterministic access to your real-time cluster state, deployment history, and dependency graphs.

When an incident occurs, the AI performs Incident Triangulation:

Telemetry Source	What the AI Analyzes
Error Logs	Pattern matches exceptions across the entire affected service chain.
Stack History	Correlates incident timing with recent manifest or Helm value changes.
Resource Events	Identifies OOMKills, scheduling failures, and CrashLoopBackOffs.
Dependencies	Traces failures through your service mesh and external integrations.

If you highlight a crashing pod and press Cmd+J, the AI does not give you generic advice. It tells you: “This pod is crash-looping because the latest Stack deployment lowered the memory limit to 256Mi, and your active connection pool requires 512Mi. I recommend reverting the last commit.”

At an edge site without an on-call SRE, this turns a two-hour outage into a five-minute fix.

The ROI: Collapsing the Integration Tax

When your centralized dashboard, delivery system, GitOps engine, and observability stack are unified, the “integration tax” disappears. Here is the reality of managing 10 edge clusters:

Operational Task	The Legacy Approach	The Ankra Approach
Import a new cluster	2-3 days (monitoring, RBAC, ArgoCD)	5 minutes (one Helm command)
Set up CI/CD per cluster	4-6 hours (Actions + Argo + debug)	30 minutes (connect repo + clone)
Troubleshoot a failing pod	1-2 hours (kubectl + context switching)	5 minutes (context-aware AI)
Replicate stack to 10 sites	2-3 weeks of engineering toil	One afternoon (stack cloning)

Time to Stop Building Plumbing

Ankra is not replacing your cloud provider or your Kubernetes distribution. Whether you run EKS, K3s, vanilla Kubernetes, or bare-metal edge nodes, if it runs Kubernetes, Ankra manages it.

If your team has grown past three clusters, you have likely realized that ad-hoc operational tooling does not scale. Your engineers should be building core product features, not writing bespoke CI/CD pipelines for every new warehouse or cell tower that comes online.

Your Git repo stays yours. Your Helm charts stay standard. Your Kubernetes remains untouched. Ankra simply provides the management layer required to operate at scale.

Get started with Ankra today to see what a real single pane of glass looks like. Stop fighting your infrastructure, and start scaling it.

AI Incident Management: From Alert to Resolution in Minutes

Mark Shine — Sat, 17 Jan 2026 00:00:00 GMT

Your pod just crashed. Again. You get an alert in Slack that says “CrashLoopBackOff” and now you’re about to spend the next hour jumping between kubectl, logs, metrics dashboards, and recent deployments trying to figure out what went wrong. Sound familiar?

Traditional alerting tells you something broke. It doesn’t tell you why, and it certainly doesn’t tell you how to fix it. That’s where Ankra’s AI incident management changes everything.

The Problem with Traditional Alerts

Most alerting systems are glorified notification pipelines. They detect a threshold breach, fire a webhook, and dump a message into Slack. Then it’s on you to:

SSH into the cluster or set up kubectl context
Find the failing pod and check its status
Read through logs looking for errors
Check recent deployments for changes
Cross-reference with metrics to understand the timeline
Search documentation or Stack Overflow for the error message
Test a fix and hope it works

This process takes anywhere from 30 minutes to several hours. For critical production issues, that’s unacceptable.

How Ankra’s AI Changes the Game

When an incident occurs in your Ankra-managed cluster, the AI doesn’t just notify you - it investigates for you. Because Ankra has access to your entire stack’s context, it can analyze the situation from multiple angles simultaneously.

Instant Root Cause Analysis

The moment a pod enters a failure state, Ankra’s AI:

Pulls recent logs from the failing container and related services
Analyzes Kubernetes events for scheduling issues, resource constraints, or configuration problems
Reviews recent changes to the stack, including Helm value updates, manifest changes, and addon upgrades
Correlates with cluster metrics to identify resource pressure or network issues
Traces dependencies to find if the root cause is actually in an upstream service

Within seconds, the AI synthesizes this information into a clear root cause analysis delivered directly to your Slack channel.

Actionable Slack Alerts

Instead of a cryptic alert like:

🚨 ALERT: Pod my-app-7d4f8b6c9-x2k4m is in CrashLoopBackOff

You get an intelligent incident report that reads like a post-mortem written by someone who actually understands your system:

🔍 Incident Analysis: my-app CrashLoopBackOff

Root Cause: OOMKilled - Container exceeded memory limit

Timeline: At 14:32 Helm values reduced memory from 512Mi to 256Mi. Rolling update at 14:33, pods at 78% memory by 14:34, OOM kill at 14:35.

Impact: 2 of 3 replicas affected. Service degraded but not down.

Fix: Increase memory limit to 512Mi or higher. Connection pool increased from 10→50 recently, which explains the failure.

📎 View in Ankra | 🔧 Apply Fix | 📊 Full Analysis

No more guessing. The AI explains what happened, when it happened, and most importantly - how to fix it.

Deep Integration with Ankra’s Debugging Tools

Here’s where it gets powerful. Ankra’s AI isn’t just reading logs - it has access to every debugging tool in the platform:

Stack Context Awareness

The AI understands your entire stack topology. When a database connection fails, it doesn’t just report the error. It knows:

Which services depend on that database
What credentials are configured
Whether the database pod is healthy
If there were recent network policy changes
What the connection string looks like

This context allows for precise diagnosis that would take a human engineer significant time to piece together.

One-Click Fixes

For common issues, the AI can propose fixes that you can apply directly from Slack:

Memory/CPU adjustments - Modify resource limits in your stack configuration
Rollback changes - Revert to the last known working state
Restart components - Bounce specific pods or services
Scale operations - Increase replicas to handle load

Click the “Apply Fix” button in Slack, and Ankra updates your stack configuration through the normal GitOps flow - with full audit trail and the ability to rollback.

Deep Dive Exploration

When the fix isn’t obvious, the AI provides deep dive capabilities:

Live log streaming with AI-highlighted anomalies
Resource visualization showing CPU, memory, and network patterns
Dependency graph highlighting affected components
Event timeline correlating changes across the stack
Similar incidents from your history with their resolutions

All accessible from a single link in the Slack message.

Real-World Impact

Teams using Ankra’s AI incident management report dramatic improvements:

Before Ankra

Mean Time to Detect (MTTD): Variable, often learned from users
Mean Time to Resolution (MTTR): 45 minutes to several hours
Context switching: Constant jumping between tools
Documentation: “Ask the person who built it”

After Ankra

MTTD: Instant, proactive alerting
MTTR: 5-10 minutes for common issues
Context switching: Everything in Slack + one platform
Documentation: AI explains the what, why, and how

One customer reduced their debugging time from an average of 2 hours to under 10 minutes for 80% of incidents. The AI handles the investigation while engineers focus on the remaining complex issues.

Setting It Up

Getting started with AI incident management takes minutes:

1. Connect Slack

integrations:
  slack:
    webhook_url: https://hooks.slack.com/services/XXX/YYY/ZZZ
    channel: "#incidents"
    mention_on_critical: "@oncall-team"

2. Configure Alert Rules

Define what matters to your team:

alerts:
  - name: pod-failures
    condition: pod.status == "CrashLoopBackOff"
    severity: critical
    ai_analysis: true

  - name: high-latency
    condition: p99_latency > 500ms
    severity: warning
    ai_analysis: true

3. Enable AI Analysis

AI analysis is enabled by default for all alerts. The AI automatically:

Gathers relevant context when an alert fires
Performs root cause analysis
Generates fix suggestions
Sends enriched alerts to Slack

No configuration needed - it just works.

Privacy and Security

Your data stays yours. Ankra’s AI:

Runs analysis within your security boundary
Never stores sensitive information externally
Respects RBAC permissions in suggestions
Logs all AI actions for audit compliance

The AI only accesses what it needs for analysis and only suggests fixes you have permission to apply.

Beyond Reactive: Proactive Insights

The AI doesn’t wait for things to break. It continuously monitors for:

Resource trends heading toward limits
Configuration drift from best practices
Dependency vulnerabilities in your addons
Capacity planning alerts before you hit limits

You get warned about problems before they become incidents.

The Future of Incident Management

The old model of alert → investigate → fix → document is being replaced by alert → understand → resolve. AI handles the investigation and documentation automatically. Engineers focus on decision-making and complex problem-solving.

With Ankra, your Slack channel becomes a command center where incidents are understood and resolved, not just announced. Every alert comes with context. Every issue comes with a suggested path forward. Every resolution is documented and learned from.

Stop spending hours debugging. Let the AI do the investigation while you focus on building.

Ready to transform your incident management? Get started with Ankra and experience AI-powered debugging in minutes, not hours.

Delivery that understands your stack

Mark Shine — Mon, 11 Aug 2025 00:00:00 GMT

The real slowdown in delivery isn’t one rebuild, it’s every change crawling through slow release cycles with messy dependencies. You can’t see other setups, Lifecycle is unclear, The code turns into technical debt fast. No one wants to touch it. You have to read too much before using it. Every tweak means rebuilding the same foundations across every pipeline. With Ankra, a few calls replace all that. With the Ankra Platform, your release cycles will be cut by up to 90 percent.

What Ankra does

Ankra treats delivery as a graph problem, not a scripting problem. The moment a resource enters the system, it gets a lifecycle. We keep a strict DAG of dependencies and update only what changed, in the right order, and in parallel where it is safe. Validation and readiness live with the resources, not in scattered scripts. Because the engine knows order and health, you are not writing custom waits and retries. In practice, the hand written job count falls sharply.

AI Assistant in the platform

The AI assistant helps DevOps design stacks with clear guardrails and immediate feedback on order, health, and impact. Developers can see what the platform team configured, understand why, and resolve customizations or small fixes themselves without waiting on DevOps. Technical managers and security officers, validate posture and policy, compare environments, and understand lifecycle timing without extra permissions. The AI assistant explains dependencies and requirements in plain language and links back to the nodes in the graph. All in one platform.

How it fits your workflow

Most teams keep their current tools and change the shape of the work.

Package the app with Docker and Helm, push to your registry and repo.
Connect your Git repository and credentials so the engine can write back.
Import the Helm chart into Stack Builder and map dependencies like DB, cache, ingress, and observability.
Preview the plan, then deploy. The engine runs the lifecycle and commits results to Git.
Operate through Git, Terraform, CLI, or API as needed. Same engine, same rules.
Spin up a new environment by copying the stack folder, adjusting inputs, and committing. Minutes, not days.

Impact in practice

In customer and users work, we see large drops in effort and lead time because the engine absorbs the choreography and runs only what is needed. Pipeline builds shrink because order, waits, and checks come from the graph instead of hand written jobs. Teams report fewer blocked by DevOps, as developers import Helm, assemble stacks, preview plans, and deploy without waiting on DevOps approval or TicketOps. Convergence is faster because only changes are isolated and safe paths run in parallel. When a provider cannot do partial updates, the engine narrows the blast radius so unrelated resources do not restart. Developers can become capable to solve problems and configurations themselves, while managers and security validate live setups without cluster credentials.

Fewer deliver blockers, more time on product work.
Faster path from idea to merge to running service.
Fewer rollbacks and narrower incidents, which lowers the cost of change.

Next-generation optimization

Changing configs in Kubernetes shouldn’t be a slow, painful process. With live feedback, instant edits, and context-aware assistance, you can tweak settings and see the results in the dependency graph immediately. No waiting on long deploy cycles just make the change, watch it update, and keep moving. It s built for speed, for experimenting, and for getting deployments exactly right without the guesswork. This is DevOps in real time. Fine tune in the fastest way possible, roll out when its ready, all from one platform.

Git as the source of truth

Every action the engine takes is written to your repository. You get history, diffs, pull request reviews, and rollbacks without inventing conventions. It is clean for machine to machine automation: systems can read one repo, produce signed commits, open pull requests, and let policy engines approve or reject based on code and context. You see who changed what, when, and why, and you can link each change to tickets and releases. Collaboration improves because product, security, and operations share the same record. Reviews speed up, audits take hours instead of days, and incident timelines are easier to reconstruct. UI and Git stay aligned. If you click in the UI, the engine still writes to Git. If you commit in Git, the engine reconciles the same way. One system. One truth.

A minimal, single manifest and single addon example. Simple, readable, and ready to drop in.

apiVersion: v1
kind: ImportCluster
metadata:
  name: imported-cluster
  description: Simple example with one manifest and one addon
spec:
  git_repository:
    provider: github
    credential_name: <github credential id>
    branch: <main>
    repository: <username>/<gitops repository>
  stacks:
  - name: monitoring
    manifests:
    - name: grafana-ns
      from_file: manifests/grafana-ns.yaml
    addons:
    - name: grafana
      chart_name: grafana
      chart_version: 9.2.10
      repository_url: https://grafana.github.io/helm-charts
      namespace: grafana
      configuration_type: standalone
      configuration:
        from_file: add-ons/grafana/values.yaml
      parents:
      - name: grafana-ns
        kind: manifest

What this replaces

If you have maintained delivery YAML, you know the bloat. Wait loops because readiness is hard to observe. Custom sequencing because dependency order is implicit. Duplicate validation because the job runner cannot see your resources. Retries without context. Ankra absorbs that work. Health and readiness are native. Order comes from the graph. Validations run where the resource lives. Retries happen with context. The day to day effect is fewer moving parts to babysit and more time spent on product work.

Repeatability and safety

Build an environment once and it’s easy to clone it, evolve it, or retire it cleanly. Because the plan is derived from the same DAG every time, you get deterministic outcomes from the same inputs. That holds in the UI, a commit, Terraform, the CLI, or an API call. The surface area for error shrinks, rollbacks are clean, and change failure rate trends down as you replace ad hoc scripts with a system that understands dependencies.

The backbone

A strict DAG gives safe parallelism, predictable sequencing, and clear failure domains. It also lets us decide what not to touch while still landing your change. That restraint is where much of the speed up comes from. Fewer components restarted. Fewer paths for things to go sideways. Less rework after the fact.

The Minimalist’s Guide to Homelab Setup

Shiva Swaroop N K — Mon, 11 Aug 2025 00:00:00 GMT

Welcome wanderer!

If you’ve clicked on this blog, you either stumbled upon Ankra and are finding out how it can accelerate your productivity, or are just curious about building homelab in simple steps.

By the end of this guide, you will have a WordPress blog hosted on your homelab, accessible securely from anywhere via a Cloudflare tunnel.

Before we dive right into it, here are the recommended prerequisites to go ahead. In case you don’t meet the requirements, don’t worry, we got you covered.

Ankra account: head out to ankra.ai to create one if you already don’t have one
A Kubernetes Cluster: It does not have to be on a cloud provider. You can have a small single node cluster with kind or minikube.
A domain of your choice: This is so that all your applications can have a subdomain of your choice. A domain on cloudflare or godaddy can help you access your homelab applications from anywhere in the world. If not, you can have a domain locally in your home network.

Setting up your cluster

Now that we have out prerequisites covered, let’s start with setting everything up.

Upon creating an Ankra account, you shall be prompted to import a Kubernetes cluster, follow along.

Once you give it a name and click on import, you get a pop up to install Ankra agent on your cluster.

If you forgot to copy the command, you can always go to settings and click on generate command like below.

You should see this if the command has run successfully

Now that we have connected our cluster, it’s time to deploy some stacks.

To begin with, we need to connect a github account/repository so that a backend IAAC can be configured. Click on stacks on the left pane

Setting up IAAC

Once this is done, any applications you add/update/delete on the UI will reflect on your repository and vice-versa.

It’s important to know that all kubernetes manifests will automatically be committed to the git repository you connect Ankra with. Make sure you do not commit any secrets/api keys to a public repository.

I have a cloudflare domain shivu.io and I want to deploy a wordpress app on the homelab. Wordpress can be used for various things. I want to have a handy workspace for all my blog ideas and it’s a great tool for that purpose.

Once the stack is deployed, I will have a wordpress.shivu.io deployed with ssl enabled, so the website will be https and I can access it from anywhere on the internet.

Deploying your stack

A stack on Ankra is a set of applications logically put together in the form of a interactive flowchart. Ankra supports applications as Addons (helm charts) or Manifests. For my stack, I will have the following applications.

Wordpress: Community helm chart as an Addon
Cert Manager: Community helm chart as an Addon for certificates
- ClusterIssuer: A single manifest additional to the add on to specify cloudflare as my domain issuer.
Cloudflared: Community helm chart as an Addon for tunnelling locally hosted applications to the internet.

The values.yaml for all these are as below

# Global parameters
global:
  imageRegistry: ""
  imagePullSecrets: []
  defaultStorageClass: "standard"
  security:
    allowInsecureImages: true

# Common configuration
kubeVersion: ""
nameOverride: ""
fullnameOverride: ""
commonLabels: {}
commonAnnotations: {}
clusterDomain: cluster.local
extraDeploy: []
usePasswordFiles: true
diagnosticMode:
  enabled: false

# WordPress Image
image:
  registry: docker.io
  repository: bitnami/wordpress
  tag: latest
  digest: ""
  pullPolicy: IfNotPresent
  pullSecrets: []
  debug: true

# WordPress Configuration
wordpressUsername: admin
wordpressPassword: devpassword123
wordpressEmail: dev@example.com
wordpressBlogName: Dev Blog
persistence:
  enabled: true
  storageClass: ""
  accessModes:
    - ReadWriteOnce
  size: 2Gi

# Deployment Settings
replicaCount: 1
updateStrategy:
  type: RollingUpdate
schedulerName: ""
terminationGracePeriodSeconds: ""
topologySpreadConstraints: []
priorityClassName: ""
automountServiceAccountToken: false

# Resource Allocation
resourcesPreset: "nano"
resources:
  requests:
    cpu: 100m
    memory: 128Mi
  limits:
    cpu: 250m
    memory: 256Mi

# Service Configuration
service:
  metadata:
    name: wordpress
  type: ClusterIP
  ports:
    http: 80
    https: 443
  clusterIP: ""
  loadBalancerIP: ""
  loadBalancerSourceRanges: []

# Database Configuration
mariadb:
  enabled: true
  architecture: standalone
  auth:
    rootPassword: rootpassword
    database: bitnami_wordpress
    username: bn_wordpress
    password: dbpassword
  primary:
    persistence:
      enabled: true
      storageClass: ""
      accessModes:
        - ReadWriteOnce
      size: 2Gi

# Disable Production Features
autoscaling:
  enabled: false
pdb:
  create: false
metrics:
  enabled: false
networkPolicy:
  enabled: false
memcached:
  enabled: false

# Development-Specific Overrides
allowEmptyPassword: true
wordpressSkipInstall: false
overrideDatabaseSettings: true

To ensure WordPress runs smoothly without crashing or timing out, it’s crucial to configure resource limits and requests in your Helm chart values. For example, setting modest CPU and memory limits helps Kubernetes manage pod stability and readiness checks effectively.

One additional thing that cert-manager requires is the cloudflare api token to create certs for the domains configured.

You can create this by executing:

kubectl -n cert-manager create secret generic cloudflare-api-token-secret --from-literal=api-token=<YOUR_CLOUDFLARE_API_TOKEN>

installCRDs: true

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-dns01
spec:
  acme:
    email: <your-email>
    privateKeySecretRef:
      name: letsencrypt-dns01
    server: https://acme-v02.api.letsencrypt.org/directory
    solvers:
    - dns01:
        cloudflare:
          apiTokenSecretRef:
            key: api-token
            name: cloudflare-api-token

In order to configure cloudflared tunnel, you need a couple of things. In order to generate them. Install cloudflared locally. Setting this up might take a while if you are new to cloudflare, but the docs are clear and concise.

Pem certificate: Upon login, a pem file is generated.
Config File: Upon creation of a tunnel, a config is placed in ~/.cloudflared/config

tunnelSecrets:
  base64EncodedPemFile: "<base64 encoded pem file>"
  base64EncodedConfigJsonFile: "<base64 encoded config.json"

replica:
  # -- This will use DaemonSet to deploy cloudflared to all nodes
  allNodes: false
  # -- If previous flag disabled, this will use Deployment to deploy cloudflared only number of following count
  count: 1

tunnelConfig:
  name: cloudflared-tunnel
  metricsUpdateFrequency: 10s
  logLevel: info
  warpRouting: true


ingress:
  - hostname: wordpress.shivu.io
    service: http://wordpress.wordpress.svc.cluster.local:80
  - service: http_status:404

Click on apply on the bottom of the page and you’re good to go.

In order to access your wordpress on the browser, you must add the URL of your tunnel as a CNAME record on the cloudflare dashboard.

The target here is <tunnel-id>.cfargotunnel.com. The tunnel-id can be fetched from the cloudflare dashboard or the first line of the cloudflared logs.

A quick clarification: The Cloudflared tunnel works independently from Kubernetes ingress controllers. While Kubernetes ingress manages traffic routing inside your cluster, the Cloudflared tunnel securely connects traffic from Cloudflare’s network to your cluster. Think of Cloudflared as the gateway that tunnels external requests safely, whereas Kubernetes ingress handles traffic once inside the cluster.

Accessing your application

If everything goes well, you should see this on your hostname

Common points to debug:

Check the logs of cert-manager if it has created the necessary certificates
Check the logs of cloudflared pods if it has registered the tunnel
Check the wordpress logs if it is serving traffic
Check the initial cloudflared values.yaml to make sure you’ve provided the right names for the service, hostname and namespace.

Join our slack: Join Community

Kubernetes Best Practices for Production Deployments

Mattias Åsell — Wed, 06 Aug 2025 00:00:00 GMT

Running applications in production on Kubernetes is challenging, but following proven best practices can make the difference between a reliable system and a maintenance nightmare. This guide covers the essential strategies that successful teams use to deploy and maintain production workloads.

What You’ll Learn

By the end of this guide, you’ll understand:

How to design resilient deployments that handle failures gracefully
Security practices that protect your applications from common threats
Monitoring strategies that give you visibility into your system’s health
Scaling patterns that keep your applications performant under load
Backup and recovery procedures that protect your data

1. Resource Management: The Foundation of Stability

Why Resource Management Matters

Resource management is the cornerstone of stable Kubernetes deployments. Without proper resource allocation, your applications can experience:

Resource starvation when one pod consumes all available CPU/memory
OOM kills when containers exceed memory limits
Poor performance due to CPU throttling
Unpredictable scaling behavior

Best Practice: Always Set Resource Limits

apiVersion: apps/v1
kind: Deployment
metadata:
  name: production-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: production-app
  template:
    metadata:
      labels:
        app: production-app
    spec:
      containers:
      - name: app
        image: production-app:v1.2.0
        resources:
          requests:
            memory: "512Mi"
            cpu: "250m"
          limits:
            memory: "1Gi"
            cpu: "500m"
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10
          timeoutSeconds: 5
          failureThreshold: 3
        readinessProbe:
          httpGet:
            path: /ready
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 5
          timeoutSeconds: 3
          failureThreshold: 3

Key Points:

Requests: Guaranteed resources your pod will receive
Limits: Maximum resources your pod can use
Health Checks: Essential for Kubernetes to know when pods are healthy
Realistic Values: Base limits on actual usage patterns, not guesses

Pro Tip: Use Resource Quotas

apiVersion: v1
kind: ResourceQuota
metadata:
  name: production-quota
spec:
  hard:
    requests.cpu: "4"
    requests.memory: 8Gi
    limits.cpu: "8"
    limits.memory: 16Gi
    pods: "20"

2. Security: Protect Your Applications

The Security-First Approach

Security in Kubernetes requires careful attention. The principle of least privilege should guide every security decision.

Best Practice: Run as Non-Root

apiVersion: apps/v1
kind: Deployment
metadata:
  name: secure-app
spec:
  template:
    spec:
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        fsGroup: 2000
      containers:
      - name: app
        image: secure-app:latest
        securityContext:
          allowPrivilegeEscalation: false
          readOnlyRootFilesystem: true
          capabilities:
            drop:
            - ALL
        volumeMounts:
        - name: tmp
          mountPath: /tmp
        - name: varlog
          mountPath: /var/log
        - name: app-config
          mountPath: /app/config
          readOnly: true
      volumes:
      - name: tmp
        emptyDir: {}
      - name: varlog
        emptyDir: {}
      - name: app-config
        configMap:
          name: app-config

Security Benefits:

Non-root execution prevents privilege escalation attacks
Read-only filesystem prevents malicious file modifications
Dropped capabilities remove unnecessary privileges
ConfigMap mounting keeps configuration separate and secure

Pro Tip: Use Network Policies

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: app-network-policy
spec:
  podSelector:
    matchLabels:
      app: production-app
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          name: frontend
    ports:
    - protocol: TCP
      port: 8080
  egress:
  - to:
    - namespaceSelector:
        matchLabels:
          name: database
    ports:
    - protocol: TCP
      port: 5432

3. Monitoring and Observability: Know Your System

The Three Pillars of Observability

Metrics: Quantitative data about your system’s performance
Logs: Detailed records of events and errors
Traces: Request flow through your distributed system

Best Practice: Comprehensive Monitoring Setup

apiVersion: v1
kind: ConfigMap
metadata:
  name: prometheus-config
data:
  prometheus.yml: |
    global:
      scrape_interval: 15s
      evaluation_interval: 15s

    rule_files:
      - "alert_rules.yml"

    scrape_configs:
    - job_name: 'kubernetes-pods'
      kubernetes_sd_configs:
      - role: pod
      relabel_configs:
      - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
        action: keep
        regex: true
      - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
        action: replace
        target_label: __metrics_path__
        regex: (.+)
      - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
        action: replace
        regex: ([^:]+)(?::\d+)?;(\d+)
        replacement: $1:$2
        target_label: __address__

Essential Metrics to Monitor

# Example alerting rules
groups:
- name: kubernetes.rules
  rules:
  - alert: HighCPUUsage
    expr: container_cpu_usage_seconds_total{container!=""} > 0.8
    for: 5m
    labels:
      severity: warning
    annotations:
      summary: "High CPU usage detected"
      description: "Container {{ $labels.container }} is using {{ $value }} CPU"

  - alert: HighMemoryUsage
    expr: container_memory_usage_bytes{container!=""} / container_spec_memory_limit_bytes{container!=""} > 0.85
    for: 5m
    labels:
      severity: warning
    annotations:
      summary: "High memory usage detected"
      description: "Container {{ $labels.container }} is using {{ $value | humanizePercentage }} memory"

4. Scaling Strategies: Handle Traffic Spikes

Horizontal vs Vertical Scaling

Horizontal scaling (adding more pods) is generally preferred in Kubernetes because it’s more resilient and can handle traffic spikes better.

Best Practice: Implement HPA with Multiple Metrics

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: production-app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: production-app
  minReplicas: 3
  maxReplicas: 20
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80
  - type: Object
    object:
      metric:
        name: requests-per-second
      describedObject:
        apiVersion: networking.k8s.io/v1
        kind: Ingress
        name: production-app-ingress
      target:
        type: Value
        value: 1000
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
      - type: Percent
        value: 10
        periodSeconds: 60
    scaleUp:
      stabilizationWindowSeconds: 60
      policies:
      - type: Percent
        value: 100
        periodSeconds: 15

Key Features:

Multiple metrics: CPU, memory, and custom metrics
Stabilization windows: Prevent rapid scaling oscillations
Conservative scale-down: Avoid scaling down too aggressively
Aggressive scale-up: Respond quickly to traffic spikes

Pro Tip: Use VPA for Vertical Scaling

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: production-app-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: production-app
  updatePolicy:
    updateMode: "Off"  # Use "Auto" for automatic updates
  resourcePolicy:
    containerPolicies:
    - containerName: '*'
      minAllowed:
        cpu: 100m
        memory: 50Mi
      maxAllowed:
        cpu: 1
        memory: 500Mi
      controlledValues: RequestsAndLimits

5. Backup and Disaster Recovery: Protect Your Data

The 3-2-1 Backup Rule

3 copies of your data
2 different storage types
1 off-site backup

Best Practice: Automated Backup Strategy

apiVersion: velero.io/v1
kind: Schedule
metadata:
  name: production-daily-backup
spec:
  schedule: "0 2 * * *"  # Daily at 2 AM
  template:
    includedNamespaces:
    - production
    includedResources:
    - deployments
    - services
    - configmaps
    - secrets
    - persistentvolumeclaims
    - persistentvolumes
    storageLocation: production-backup
    volumeSnapshotLocations:
    - production-snapshot
    ttl: 720h  # Keep backups for 30 days

Backup Verification Strategy

apiVersion: velero.io/v1
kind: Schedule
metadata:
  name: backup-verification
spec:
  schedule: "0 4 * * 0"  # Weekly on Sunday at 4 AM
  template:
    includedNamespaces:
    - backup-test
    includedResources:
    - deployments
    - services
    - configmaps
    - secrets
    - persistentvolumeclaims
    storageLocation: production-backup
    volumeSnapshotLocations:
    - production-snapshot
    ttl: 24h

6. Deployment Strategies: Zero-Downtime Updates

Rolling Updates with Health Checks

apiVersion: apps/v1
kind: Deployment
metadata:
  name: production-app
spec:
  replicas: 5
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    spec:
      containers:
      - name: app
        image: production-app:v1.2.0
        readinessProbe:
          httpGet:
            path: /ready
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 5
          timeoutSeconds: 3
          failureThreshold: 3
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10
          timeoutSeconds: 5
          failureThreshold: 3

Benefits:

Zero downtime: New pods are ready before old ones are terminated
Rollback capability: Easy to revert to previous version
Health verification: Only healthy pods serve traffic

Key Takeaways

Immediate Actions You Can Take

Set resource limits on all your containers today
Implement health checks for every application
Run containers as non-root users
Set up basic monitoring with Prometheus
Configure HPA for your critical applications
Implement automated backups with Velero

Long-term Strategy

Gradually implement security policies
Build comprehensive monitoring dashboards
Test disaster recovery procedures regularly
Optimize resource usage based on monitoring data
Automate everything possible

Most importantly, keep it simple! Overcomplicating your infrasturcutre will result in unimaginable growing pains.

Remember: Production Kubernetes is a journey, not a destination. Start with these fundamentals and continuously improve based on your specific needs and challenges.

Need help implementing these best practices? Join us on Slack

Ankra Blog

Ankra is now ankra.ai: Rewritten in Go, Ready for What's Next

Ankra goes all the way with Go

Describe the problem. Approve the fix.

Security that keeps up with you

The platform, in your pocket

European by default. Sovereign by design.

Deploying your app

Maintaining your app

Also shipped

Why the name changed

Secrets Belong in Git. Plaintext Does Not.

The numbers got worse, and AI is making them worse faster

”The repo is private” is not a secrets strategy

Encrypt values, not files

The Ankra workflow: initialize once, encrypt by default

In the Stack Builder

From the CLI, for GitOps-first teams

Make plaintext a failed build, not a code review comment

The AI angle: agents that cannot leak what they never hold

Rotate like you mean it

The five rules

Kubermatic vs Ankra: Do You Want to Run a Kubernetes Factory, or Just Ship Software?

What KKP actually is, and why the architecture is genuinely clever

The factory comes with a factory floor

The question after the cluster exists

Two kinds of open, honestly compared

Side by side

Where Kubermatic is the right call

The move

The Early-Stage Infrastructure Playbook: Kubernetes Before Your First DevOps Hire

What the old advice was really about

What changed

But why not just stay on a PaaS?

The week-one playbook

The anti-playbook: what to refuse to build

The runway math, said out loud

The unfair advantage nobody mentions

Kubernetes Logging in 2026: Elasticsearch, OpenSearch, Loki, or Just Renting It from Your Cloud

Family one: index everything (Elasticsearch and OpenSearch)

Family two: index almost nothing (Grafana Loki)

Family three: rent the whole problem (GCP Logs Explorer and friends)

The same 1 TB, three ways

The decision framework

Where Ankra sits in this

The Kubernetes Security Work That Actually Matters in 2026

Finding 1: A load-bearing component nobody maintains

Finding 2: Kubeconfigs are the new SSH keys, and they are everywhere

Finding 3: The secret is in Git, and Git is forever

Finding 4: Changes nobody reviewed, by humans or their agents

Finding 5: latest is not a version, and floating tags are a supply chain

What did not make the list, and why

The checklist

GitOps Without ArgoCD: Inside Ankra's Native Deployment Engine

What ArgoCD was costing us

What the native engine is

The reconcile loop, concretely

Feature parity, honestly

The migration: adoption, not reinstallation

Why this matters beyond the RAM

Prometheus vs VictoriaMetrics: When the Default Stops Being the Answer

What Prometheus gets right, still

The wall, and why it is always cardinality

What VictoriaMetrics actually changes

The real comparison is not Prometheus vs VictoriaMetrics

PromQL vs MetricsQL, the fine print

Governance, because it matters now

The decision, compressed

Running either on Ankra

Spectro Cloud vs Ankra: Managing the Metal, or Running What Lives On It?

What Palette actually is

The different problem

Blueprints: Cluster Profiles vs Stacks

Git: written back, or managed through an API

Two meanings of “AI platform”

Cost of entry, in time and money

Side by side

Where Palette is the right call

What running both above and below would even mean

The move

Finding 5: `latest` is not a version, and floating tags are a supply chain