ROX-37670: test container rebuilds happen during nightlies by ovalenti · Pull Request #2209 · stackrox/fact

ovalenti · 2026-10-22T09:39:13Z

Description

Test containers use the list of files modified from a reference to determine whether they should be rebuilt. In the case of 'schedule' GHA, this detection returns "all modified", which triggers the rebuild.

We propose to add a specific test that prevents rebuild when the event_name is 'schedule'.

For this PR to be merged will require that the test containers be fixed. So work on this is required as well. This work could arguably be done as a preliminary PR in order to conserve the separation of concerns ; but it is also likely overkill, so we will start with just one PR having both fixes.

Checklist

Patch has a change log entry OR does not need one.
Investigated and inspected CI test results
Updated documentation accordingly

Automated testing

Added unit tests
Added integration tests
Added regression tests

If any of these don't apply, please comment below.

Testing Performed

Summary by CodeRabbit

Tests
- Editor test containers now use CentOS Stream 10, with updated Neovim and Vim packages. Editor checks continue to cover file creation, editing, and file-event behavior in the updated environment.
Chores
- Scheduled workflow runs no longer build container images when the QA tag changes or a rebuild label is present. Other event types retain the existing image-build conditions.

coderabbitai · 2026-10-22T09:39:21Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Source mode: Auto
Repository: stackrox/fact
Branch: main
File: .coderabbit.yml
Commit: 77fbaeb
Layers: Repository YAML
Review profile: CHILL
Plan: Enterprise
Run ID: a8999094-b78f-47a4-b8ce-e5eb1d5b6e83

📥 Commits

Reviewing files that changed from the base of the PR and between 77fbaeb and 4d9dbbc.

📒 Files selected for processing: 11

.github/workflows/test-containers-build.yml
tests/containers/QA_TAG
tests/containers/base/Containerfile
tests/containers/base/Makefile
tests/containers/editors/Containerfile
tests/containers/fedora/Containerfile
tests/test_editors/commons.py
tests/test_editors/test_nvim.py
tests/test_editors/test_sed.py
tests/test_editors/test_vi.py
tests/test_editors/test_vim.py

💤 Files with no reviewable changes: 1

tests/containers/fedora/Containerfile

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

📝 Walkthrough

The editor test containers now use CentOS Stream 10-based images. The build workflow excludes scheduled events from enabling the image build output. Editor test fixtures and touch-process expectations are updated for the new environment.

Changes: Editor test container migration

Layer / File(s)	Summary
Container image definitions and build tags `.github/workflows/test-containers-build.yml`, `tests/containers/QA_TAG`, `tests/containers/base/*`, `tests/containers/editors/Containerfile`, `tests/containers/fedora/Containerfile`	The QA tag changes to `0.2.0`. The base and editor container definitions use digest-pinned CentOS Stream 10 images. The editor image installs specified Neovim and Vim versions. The build output excludes scheduled events, and the Fedora Containerfile is removed.
Editor test fixtures and process expectations `tests/test_editors/commons.py`, `tests/test_editors/test_nvim.py`, `tests/test_editors/test_sed.py`, `tests/test_editors/test_vi.py`, `tests/test_editors/test_vim.py`	Editor tests use the base image fixture instead of the Fedora fixture. Updated touch-process expectations identify `/usr/bin/coreutils`; expected event sequences remain unchanged.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Suggested reviewers: molter73

Merge Risk: ⚪ Minimal · up to 4d9db

Scheduled runs no longer trigger the test-container rebuild through this workflow. No actionable merge-blocking issue was found; normal checks can proceed.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 12 functions across 5 files. (5 skipped: 5…	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Title check		The title clearly identifies the scheduled test-container rebuild behavior addressed by the pull request.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Description check		The description follows the required structure and explains the scheduled-event rebuild issue and container updates. The Testing Performed section is empty, and the unchecked automated-testing items a…

Full details: Description check

Explanation

The description follows the required structure and explains the scheduled-event rebuild issue and container updates. The Testing Performed section is empty, and the unchecked automated-testing items are not clarified, but the description is otherwise complete and relevant.

Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 12 functions across 5 files. (5 skipped: 5 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

✨ Finishing Touches: 💡 1

📝 Generate docstrings: 💡

Commit to this branch
Create a new PR

🧪 Generate unit tests (beta)

Commit to this branch
Create a new PR

_{Comment @coderabbitai help to get the list of available commands.}

codecov-commenter · 2026-10-22T09:41:52Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 33.45%. Comparing base (77fbaeb) to head (4d9dbbc).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2209   +/-   ##
=======================================
  Coverage   33.45%   33.45%           
=======================================
  Files          22       22           
  Lines        3623     3623           
  Branches     3623     3623           
=======================================
  Hits         1212     1212           
  Misses       2402     2402           
  Partials        9        9

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This leads to a closer and (hopefully) more stable version of the existing fedora images we were previously using.

ovalenti · 2026-10-22T15:34:49Z

@Molter73 's changes look good to me

This is a follow up to #2209, we need to backport those changes and we need to ensure release branches will not accidentally overwrite images built in the main branch. A new `exists` make target is added to all test containers and a global check ensures at least one of the images is missing before triggering the rebuild. On normal operation, new versions of the images will be built by the main branch and be available once the QA_TAG bump is backported to release branches. In this case the paths-filter action will determine a rebuild is needed, but the new check will skip building the images. If the images are in the process of being built on the main branch and some of them are not yet pushed, the backported commit will push the images again but they will have been built in such a small span of time the resulting images will pretty much be the same. On PRs, if the containers need to be rebuilt a unique tag is generated and the step will always return the images don't exist.

) * chore: validate test images don't exist on remote before building This is a follow up to #2209, we need to backport those changes and we need to ensure release branches will not accidentally overwrite images built in the main branch. A new `exists` make target is added to all test containers and a global check ensures at least one of the images is missing before triggering the rebuild. On normal operation, new versions of the images will be built by the main branch and be available once the QA_TAG bump is backported to release branches. In this case the paths-filter action will determine a rebuild is needed, but the new check will skip building the images. If the images are in the process of being built on the main branch and some of them are not yet pushed, the backported commit will push the images again but they will have been built in such a small span of time the resulting images will pretty much be the same. On PRs, if the containers need to be rebuilt a unique tag is generated and the step will always return the images don't exist. * Global FACT_QA_TAG and portable redirections * Don't conflate network/registry errors with manifest does not exist errors

* ROX-36983: version containers used in tests (#1789) After a change in the Vi/Vim versioned used by Fedora broke our integration tests, we have decided to tag and version all images used in our tests. This patch adds a CI job for building and pushing the images when `tests/containers/QA_TAG` is updated, much like we do in stackrox/collector. It also modifies the existing tests from always building the images to: * Try to find the image locally. * Attempt to pull it if it is not found. * If unable to pulled and allowed to build locally, do that. * ROX-37670: test container rebuilds happen during nightlies (#2209) * Never rebuild test containers on scheduled builds * Bump test container version * Use debian:trixie as the base distro for 'editors' * Use centos images + pin image versions and packages used This leads to a closer and (hopefully) more stable version of the existing fedora images we were previously using. --------- Co-authored-by: Mauro Ezequiel Moltrasio <mmoltras@redhat.com> * chore: validate test images don't exist on remote before building (#2214) * chore: validate test images don't exist on remote before building This is a follow up to #2209, we need to backport those changes and we need to ensure release branches will not accidentally overwrite images built in the main branch. A new `exists` make target is added to all test containers and a global check ensures at least one of the images is missing before triggering the rebuild. On normal operation, new versions of the images will be built by the main branch and be available once the QA_TAG bump is backported to release branches. In this case the paths-filter action will determine a rebuild is needed, but the new check will skip building the images. If the images are in the process of being built on the main branch and some of them are not yet pushed, the backported commit will push the images again but they will have been built in such a small span of time the resulting images will pretty much be the same. On PRs, if the containers need to be rebuilt a unique tag is generated and the step will always return the images don't exist. * Global FACT_QA_TAG and portable redirections * Don't conflate network/registry errors with manifest does not exist errors --------- Co-authored-by: ovalenti <ovalenti@redhat.com>

ovalenti added 3 commits October 21, 2026 16:41

Never rebuild test containers on scheduled builds

897e42f

Bump test container version

db56e5b

Use debian:trixie as the base distro for 'editors'

14b16f8

ovalenti self-assigned this Oct 22, 2026

ovalenti assigned Molter73 and unassigned ovalenti Oct 22, 2026

Molter73 force-pushed the ovalenti/spurious_test_container_rebuilds branch from 2dc30b0 to 5956046 Compare October 22, 2026 13:22

Use centos images + pin image versions and packages used

4d9dbbc

This leads to a closer and (hopefully) more stable version of the existing fedora images we were previously using.

Molter73 force-pushed the ovalenti/spurious_test_container_rebuilds branch from 5956046 to 4d9dbbc Compare October 22, 2026 14:05

Molter73 marked this pull request as ready for review October 22, 2026 14:33

Molter73 requested a review from a team as a code owner October 22, 2026 14:33

Stringy approved these changes Oct 22, 2026

View reviewed changes

Molter73 merged commit e105a4d into main Oct 23, 2026
108 of 113 checks passed

Molter73 deleted the ovalenti/spurious_test_container_rebuilds branch October 23, 2026 08:45

Molter73 mentioned this pull request Oct 23, 2026

Backport test container versioning to release branches #2213

Open

Molter73 mentioned this pull request Oct 23, 2026

chore: validate test images don't exist on remote before building #2214

Merged

5 of 9 tasks

This was referenced Oct 26, 2026

Fix test containers used on release-0.4 #2225

Merged

Fix test containers used on release-0.3 #2226

Merged

Navigation Menu

Repository navigation

ROX-37670: test container rebuilds happen during nightlies - #2209

ROX-37670: test container rebuilds happen during nightlies#2209
Molter73 merged 4 commits into
mainfrom
ovalenti/spurious_test_container_rebuilds

ovalenti commented Oct 22, 2026 •

edited by Molter73

Loading

Uh oh!

coderabbitai Bot commented Oct 22, 2026 •

edited

Loading

Uh oh!

codecov-commenter commented Oct 22, 2026 •

edited

Loading

Uh oh!

ovalenti commented Oct 22, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

ovalenti commented Oct 22, 2026 • edited by Molter73 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Testing Performed

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-commenter commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ovalenti commented Oct 22, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ovalenti commented Oct 22, 2026 •

edited by Molter73

Loading

coderabbitai Bot commented Oct 22, 2026 •

edited

Loading

codecov-commenter commented Oct 22, 2026 •

edited

Loading