CFL restarts: don't overwrite the restart checkpoint on the first step

[sbryngelson]

Summary

In CFL mode (cfl_adap_dt / cfl_const_dt), a restart can overwrite its own restart checkpoint with the state one step later. @thierrydaoud spotted the symptom in #1926.

Mechanism

p_main.fpp saved when abs(mod(mytime, t_save)) < dt .or. mytime >= t_stop. A restart sets mytime = t_save*n_start exactly. After the first step, mod(n_start*t_save + dt, t_save) should equal dt, but because of round-off it comes out within an ulp of dt and falls below it about half the time. The save then fires and s_save_data writes index int(mytime/t_save) = n_start, so checkpoint n_start is replaced by the state after one step.

The cause is round-off, not a change in dt. The dt used in the check is the step's own dt, because s_compute_dt runs only at the start of s_perform_time_step.

Fix

Track the last saved index, starting at n_start. Save when int(mytime/t_save) moves past it, or at t_stop. This is the same expression s_save_data uses for the file index, so each index is written once and the restart checkpoint is never written again.

• Fresh runs save at the same steps as before: the first step at or past each k*t_save.
• The final save at t_stop is unchanged.
• If int(mytime/t_save) rounds down just after a crossing, the save comes one step late with the correct index. The old check would have written the previous index in that case.
• post_process only reads int(t_stop/t_save) and does not use the mod pattern. No other code uses mod(mytime, ...).

Reproduction (Frontier, CCE CPU, --no-mpi)

I modified examples/1D_sodshocktube/case.py to use cfl_adap_dt = T, t_save = 0.01, t_stop = 0.06, parallel_io = F, format = 1. I ran it fresh, then restarted it with n_start = 1..4, resetting p_all to the fresh run's output before each restart.

At one overwrite, the debug output showed mod(mytime, t_save) = 5.05147325735151587E-04 < dt = 5.05147325735151695E-04.

	cfl_target 0.45	cfl_target 0.4
master: checkpoint n_start overwritten	n_start = 1, 2	n_start = 2
this PR: checkpoint n_start overwritten	none (1-4 unchanged)	none (1-4 unchanged)
fresh-run p_all, master vs this PR	byte-identical, indices 0-6	byte-identical, indices 0-6

I also emulated the old and new save logic in float64 over 50k random dt sequences and t_save/t_stop combinations. The new logic never skipped or repeated a non-final index and never rewrote n_start. On fresh starts it saved at the same steps as the old logic. On restarts it differed only in the cases where the old logic rewrote n_start.

Regression test

New case Restart Roundtrip -> 1D -> cfl_adap_dt=T (25EB3D1F). For restart_check cases in CFL mode, run_restart runs the case straight, then restarts from each interior save index (1-4). After each restart it checks that p_all/*/<n_start>/ is byte-identical to what it was before.

The usual comparison of restarted output with the straight run is skipped for CFL cases. A CFL restart starts from a checkpoint stamped n_start*t_save rather than its true time, so its later states differ from the straight run.

t_save = 2^-8, so n_start*t_save is exact. On master this test fails here with Restart from n_start=1 rewrote its own restart checkpoint.

With the fix, the test is deterministic: no restart can write its own index. On master, whether the bug triggers depends on round-off, so a given compiler might not catch a regression. Checking four restarts makes a miss unlikely.

The golden file was generated with CCE 19 on Frontier.

Testing (Frontier login node, CCE CPU, --no-mpi)

• ./mfc.sh format, ./mfc.sh build --no-mpi -j 8
• ./mfc.sh test --no-mpi -j 8 -o <24 UUIDs>: every cfl_adap_dt / cfl_const_dt case and every restart_check case. 24 passed.
• ./mfc.sh test --no-mpi -j 8 -% 20: 142 passed, 0 failed.
• No existing goldens changed.

This PR was written and tested with Claude Code (AI) on OLCF Frontier (CCE CPU, no MPI).

---

Contribution Policy

We do not accept pull requests generated primarily by AI without genuine understanding or real-world usage context.

All contributions are expected to demonstrate:

• A clear understanding of the codebase
• Alignment with product direction
• Thoughtful reasoning behind changes
• Evidence of real-world usage or hands-on experience with the problem

If these expectations are not met, we would prefer to implement the changes ourselves rather than spend time reviewing low-effort submissions.

---

Acknowledgement

• I confirm this PR meets the above expectations and reflects my own understanding and real-world context.

---

PR template credit: junegunn

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[github-actions]

Lines of Code

File	Lines	Diff
src/simulation/p_main.fpp	76	+3

Directory	Lines	Diff
simulation	28428	+3
total	47409	+3

[codecov]

Codecov Report

❌ Patch coverage is 50.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 61.80%. Comparing base (27cae2c) to head (503a8d5).
⚠️ Report is 2 commits behind head on master.

Files with missing lines	Patch %	Lines
src/simulation/p_main.fpp	50.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1960      +/-   ##
==========================================
+ Coverage   61.79%   61.80%   +0.01%     
==========================================
  Files          86       86              
  Lines       22773    22775       +2     
  Branches     3353     3354       +1     
==========================================
+ Hits        14073    14077       +4     
+ Misses       6211     6208       -3     
- Partials     2489     2490       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.