Abort on a fluid cell with no valid time step instead of dropping it from the dt reduction

[sbryngelson]

Summary

s_compute_dt takes a min() over fluid cells. min() drops a NaN operand, so a cell whose state has gone NaN or unphysical (e.g. negative pressure, so a NaN sound speed) silently leaves the reduction: dt is set by the cells that are still valid, the bad state keeps spreading, and the run dies some steps later at the next save, or after an oversized step, far from where it started.

This flags any fluid cell whose inviscid dt candidate is not > 0 (NaN or non-positive) inside the same reduction (a max over its linear index) and aborts with the rank, local indices and coordinates of that cell.

.not. (x > 0) is used rather than ieee_is_nan, which MFC does not use in device code (only behind __INTEL_COMPILER).

Motivation

Found while running particle-resolved reacting-carbon cases with hot isothermal IBs (#1821). The NaN was first reported at a later save, on a rank with no particles; the first bad cells were next to particle surfaces.

Testing

• ./mfc.sh test --no-mpi -o Chemistry (15) and -o IBM (61): pass, unchanged (CCE 19, CPU). No golden changes: the check only fires on a state that would otherwise go on to fail.
• GPU (OpenMP offload, MI300A), 3D bed of 191 hot carbon spheres at 12 cells/d: the run now stops at step 4 with No valid time step: rank 0, local cell (j,k,l) = 78 75 45, x = 5.4167E-05 6.2917E-04 3.7917E-04, 2.4 cells from a particle surface.
• Cost (rocprofv3, 41M cells, 4 MI300A): the s_compute_dt kernel is 0.87 ms per call with the check, 0.16% of GPU time. No communication is added.

./mfc.sh lint passes except test_thermochem.py::test_precision_and_directives[dp-acc], which also fails without this change here: LLNL's gfortran 13.3.1 offloads OpenACC to nvptx, whose device libm lacks log10.

This PR was written and tested with Claude Code (AI) on LLNL Tuolumne (CCE 19 CPU; OpenMP offload on MI300A).

---

Acknowledgement

• I confirm this PR meets the above expectations and reflects my own understanding and real-world context.

[github-actions]

Lines of Code

File	Lines	Diff
src/simulation/m_time_steppers.fpp	906	+20

Directory	Lines	Diff
simulation	28445	+20
total	47426	+20

[Copilot]

🟢 Approval recommended

The reduction, index decoding, diagnostics, and CPU/GPU behavior are consistent with existing code patterns.

0 open findings

What changed in this PR

Adds immediate diagnostics when a fluid cell produces an invalid inviscid time-step candidate.

Changes:

• Detects NaN or non-positive inviscid time steps during GPU reduction.
• Reports the responsible rank, local indices, and coordinates before aborting.

File	Description
src/​simulation/​m_time_steppers.fpp	Adds invalid-cell reduction and location-aware abort handling.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[codecov]

Codecov Report

❌ Patch coverage is 0% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 61.75%. Comparing base (3db6b4c) to head (9f83377).
⚠️ Report is 6 commits behind head on master.

Files with missing lines	Patch %	Lines
src/simulation/m_time_steppers.fpp	0.00%	14 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1952      +/-   ##
==========================================
- Coverage   62.66%   61.75%   -0.91%     
==========================================
  Files          86       86              
  Lines       22435    22787     +352     
  Branches     3325     3357      +32     
==========================================
+ Hits        14058    14073      +15     
- Misses       6119     6225     +106     
- Partials     2258     2489     +231     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.