feat: observe installed Zitadel instances through native references

[patrickleet]

AuthStack currently learns an installed Zitadel instance ID through a one-shot Python Job and a ConfigMap. Consumers then copy that ID into domain resources. This creates extra RBAC and lifecycle/retry wiring for metadata that provider-upjet-zitadel v0.3.0 can now observe directly.

This replaces the Job, ConfigMap, ServiceAccount and RBAC with a read-only namespaced Instance and an ordering Usage. Helm continues to own the installed service and database. The observer reads credentials and transport settings from an existing ProviderConfig; secrets never pass through composition templates or XR status.

For example, a platform stack installs Zitadel once and separate preview namespaces register trusted domains. Those namespaces can now reference the installed instance by resource name instead of copying its numeric ID:

apiVersion: hops.ops.com.ai/v1alpha1
kind: AuthStack
metadata:
  name: identity
  namespace: platform
spec:
  # Keep the existing installation/database/bootstrap configuration.
  instanceDiscovery:
    enabled: true
    providerConfigRef:
      name: zitadel-admin
      kind: ClusterProviderConfig

After successful observation, status includes the existing instanceId and the additive instanceRef: {name: identity-instance, namespace: platform}. A consumer can use:

apiVersion: instance.zitadel.m.crossplane.io/v1alpha1
kind: TrustedDomain
metadata:
  name: browser
  namespace: preview
spec:
  forProvider:
    instanceIdRef:
      name: identity-instance
      namespace: platform
      policy: {resolve: Always}
    domain: preview.example.com
  providerConfigRef:
    name: zitadel-admin
    kind: ClusterProviderConfig

The provider also supports instanceIdSelector; name references are the simplest default. The ProviderConfig must target this installation, since AuthStack cannot establish ownership of an arbitrary external endpoint. For first install, wait for Helm to generate the PAT, publish it using PushSecret, establish the ProviderConfig through ESO, and then wait for AuthStack. Waiting for AuthStack before creating the observer credentials would introduce a dependency cycle.

Compatibility:

• Discovery remains disabled by default. Existing consumers that do not enable it retain their installation behavior. This introduces no localhost, plaintext transport or secret-backend defaults.
• The opt-in discovery API changes: replace internalURL, allowInsecureHTTP, caCertSecretRef and image with providerConfigRef, and configure transport there. Enabled old configurations without the new reference are rejected. The package now requires provider-upjet-zitadel >=v0.3.0; coordinate an existing provider pin before upgrading.
• status.instanceId is preserved; status.instanceRef is additive. Failed, stale or deleting observations clear the published metadata and explicitly make AuthStack Not Ready, even if the managed resource retains an older Ready condition. The observer continues rendering through those failures.
• The observer binds the first observed instance ID. Intentionally replacing the installation requires recreating the observer; deleting an observation does not delete Zitadel. A Usage orders observation deletion before Helm deletion.

Validation:

• 33 existing composition tests passed, plus six new observation/migration tests (including failed sync, stale generation and deleting observations). Both PR and main CI run the new suite and validate the same 16 example/observed-state cases as the local Makefile.
• All 16 examples rendered and passed schema validation: 61 resources, zero missing schemas or validation failures. Fixed local Makefile validation to load generated dependency metadata, serialize builders that modify shared caches, and propagate pipeline failures.
• Source-installed the configuration on kind-hops with released provider v0.3.0. The composed observer and AuthStack became Ready with the original instance ID.
• Live name and label-selector consumers waited for missing credentials, recovered when their ProviderConfig appeared, and survived a subsequent dependency failure with unchanged IDs/UIDs. AuthStack itself became Not Ready and cleared its published ID on a missing ProviderConfig, then recovered at its current generation.
• During native adoption, all 54 pre-existing Zitadel managed-resource IDs/UIDs and four existing credential fingerprints were unchanged. On restarting the baseline GitOps watcher, it pruned two retired manual user proofs from its persisted inventory; they were recreated with new IDs outside deploy paths. The other 52 resources and all existing credentials remained unchanged. Disposable observations/domain/application resources were deleted, and the previous configuration/provider were restored. No cloud rollout was performed.

CI at d83edde: all 16 validation cases, composition tests, native-observation tests and the existing E2E job passed in run 37067598051. Preview-package publication is still running.

To repeat the configuration checks:

make test
python3 -m pip install PyYAML==6.0.3
make test-security
make validate:all

For a live trial, start the local control plane with hops local up, retain a working Zitadel installation and its ProviderConfig, then install this checkout with hops config install --path /path/to/auth-stack --context kind-hops --cluster-provider kind --docker-provider dory. Upgrade the local Zitadel provider to v0.3.0 before enabling observation. The example above shows the changed fields; keep your existing installation/bootstrap spec.

[coderabbitai]

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

• Configuration used: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: 620b0ba4-1da6-47e9-aa26-7e1c5ece4176

📥 Commits

Reviewing files that changed from the base of the PR and between d853006 and d83edde.

📒 Files selected for processing (13)

• .github/workflows/on-pr.yaml
• .github/workflows/on-push-main.yaml
• Makefile
• README.md
• apis/authstacks/definition.yaml
• examples/authstacks/with-instance.yaml
• functions/render/010-state-status.yaml.gotmpl
• functions/render/210-instance-discovery.yaml.gotmpl
• functions/render/999-status.yaml.gotmpl
• tests/security/test_discovery.py
• tests/security/test_instance_observation.py
• tests/test-render/main.k
• upbound.yaml

 _________________________________________________________________
< Granted, I'm not human, but I still know when your code is bad. >
 -----------------------------------------------------------------
  \
   \   (\__/)
       (•ㅅ•)
       / 　 づ

✨ Finishing Touches

📝 Generate docstrings

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Published Crossplane Package

The following Crossplane package was published as part of this PR:

Package: ghcr.io/hops-ops/auth-stack:pr-32-1d56a922fc6b94d10a458e461d9c4f8338a1b103

View Package