Repository navigation
[HWORKS-3565] Upgrade Kueue from the bundled 0.12.2 to 0.18.1 installed outside the Hopsworks namespace, migrating existing queues and CRDs - #708
Closed
jimdowling wants to merge 5 commits into
Conversation
jimdowling
marked this pull request as ready for review
October 21, 2026 21:04
jimdowling
force-pushed
the
HWORKS-3565
branch
from
October 22, 2026 07:19
d8ed650 to
6dd2eea
Compare
…ed outside the Hopsworks namespace, migrating existing queues and CRDs https://hopsworks.atlassian.net/browse/HWORKS-3565 Every release from 4.3 bundles Kueue 0.12.2 in the Hopsworks release namespace. From 5.3 Hopsworks can use a Kueue installed as its own release in kueue-system instead (hw-kueue.kueue.enabled=false), and the chart ships files/migrate-to-external.sh to move a running cluster to Kueue 0.18.1 while keeping every Kueue object. The docs add an operator page for that move: why helm upgrade alone cannot do it, the script's phases, what to prepare (snapshot, maintenance window, tools, airgapped images, admin API key), the command, what changes for queues and Hopsworks afterwards (no topologyName on flavors, parentName on Cohorts, queues managed with kubectl, Hopsworks-namespace jobs queued, the queue utilisation view failing until the backend reads Cohorts at v1beta1) and how to verify. The hw-kueue values page links to it. Signed-off-by: Jim Dowling <jim@logicalclocks.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ed outside the Hopsworks namespace, migrating existing queues and CRDs https://hopsworks.atlassian.net/browse/HWORKS-3565 Clusters whose releases Flux manages. Flux reconciles a HelmRelease back to Git, so the helm upgrades in remove-bundled, install-kueue and enable-hopsworks would be reverted on a Flux-managed cluster. logicalclocks.github.io: the Kueue operator page gains a section on clusters managed by Flux (what to commit at each phase, what the script waits for, the queue export) and lists export-queues. Signed-off-by: Jim Dowling <jim@logicalclocks.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ed outside the Hopsworks namespace, migrating existing queues and CRDs https://hopsworks.atlassian.net/browse/HWORKS-3565 Step-by-step instructions for clusters managed by Flux. logicalclocks.github.io: the Flux section of the Kueue operator page becomes a runbook for --gitops: what to have before starting (Flux 2.3 or later, Git access, the Helm release name and namespace, where the HelmRelease values live, a terminal that survives a dropped connection), then one step per change to commit, with the command, the HelmRelease YAML, the flux reconcile command that applies it at once and what the script waits for, then where the queue manifests go (a Kustomization depending on the Kueue one) and how to resume after a timeout. Signed-off-by: Jim Dowling <jim@logicalclocks.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jimdowling
force-pushed
the
HWORKS-3565
branch
from
October 22, 2026 17:05
73b63c4 to
cb82f4e
Compare
…ed outside the Hopsworks namespace, migrating existing queues and CRDs https://hopsworks.atlassian.net/browse/HWORKS-3565 Review comments on logicalclocks/hopsworks-helm#2599. logicalclocks.github.io: the Kueue operator page runs the migration script with bash in every command, because helm package stores it without the execute bit and a direct call fails with permission denied. Signed-off-by: Jim Dowling <jim@logicalclocks.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Upstream published the Helm upgrade guide here (HWORKS-3580) and moved the Kafka 4 notes under a new Upgrades section of the admin nav. Kueue outside the Hopsworks namespace moves into that section, and the upgrade guide's 5.2.x to 5.3.x section gets an entry for it that links the runbook, replacing the UPGRADE.md section the chart no longer ships. Signed-off-by: Jim Dowling <jim@logicalclocks.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Superseded: the external-Kueue install page moves into #709 (HWORKS-3579); the migration runbook goes to branch-5.1. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
https://hopsworks.atlassian.net/browse/HWORKS-3565
Summary
Operator documentation for moving a Hopsworks cluster from the Kueue 0.12.2 bundled in the Hopsworks release to Kueue 0.18.1 installed as its own release in
kueue-system, as implemented by logicalclocks/hopsworks-helm#2599 (hw-kueue.kueue.enabled=falseandcharts/hw-kueue/files/migrate-to-external.sh).docs/setup_installation/admin/kueue_external.md, in the nav after the Kafka 4 upgrade notes:helm upgradealone cannot do the move;topologyNameon flavors,parentNameon Cohorts, queues managed with kubectl, Hopsworks-namespace jobs queued, the queue utilisation view failing until the backend reads Cohorts atv1beta1;hw-kueuevalues page links to it.Testing
uv run hopsworks-docs markdownlint: clean.uv run hopsworks-docs check(strict build): passes.Depends on logicalclocks/hopsworks-helm#2599: merge after it, since the page documents the script and value it adds.
🤖 Generated with Claude Code