Repository navigation
Cylinder IBs: honor the axis set by length_y/length_z (fixes 3D start-up hang) - #1962
sbryngelson wants to merge 4 commits into
Conversation
…-up hang) Cylinder IBs (geometry 10) were assumed to lie along x in three places: - s_get_ib_bound used sqrt(radius**2 + length_x**2), so a y/z-aligned cylinder (length_x = dflt_real = -1e6) got a 1e6 bound. The automatic ib_neighborhood_radius then became ~1e9 hops and s_get_neighbor_bounds spun in MPI_Sendrecv forever. Use the half-length of the set axis. - The marker loops always tested about x, so y/z cylinders got no IB cells. - The levelset picked the axis by testing length /= 0, which always chose x because the unset default is -1e6, not 0. Select the axis with f_cylinder_axis in all three places and add a z-aligned cylinder regression test.
… (fixes GPU fault) On MI300A with CCE 19 OpenMP offload, the sharp 3D cylinder marker kernel in s_apply_ib_patches hit a GPU memory access fault at start-up. The device routine f_is_inside_ib_cylinder built a local array from the components of its patch_ib derived-type dummy; removing just that array constructor clears the fault, while passing the derived type alone does not trigger it. Follow the pattern of the other inside-tests in these kernels: build radius and length from patch_ib in the kernel and pass them in. f_cylinder_axis now takes the length array, which the levelset already has. CPU results are unchanged.
|
Pushed b338042: fixes a GPU memory access fault this PR introduced on MI300A (CCE 19, OpenMP offload). A sharp 3D cylinder IB faulted at start-up in the 3D marker kernel. Building a local array from the components of the derived-type dummy inside the device routine Verified on GPU: the sharp rod runs on 1 and 2 ranks, and its markers match the analytic cylinder (0 mismatches) and the CPU run bit-for-bit. CPU IBM (69) and Cylinder (8) tests pass. (Found and fixed with Claude Code.) |
b338042 to
408924e
Compare
Union the m_ib_patches public list: keep master's new lookup helpers (s_get_neighborhood_idx, s_update_ib_lookup, s_compact_ib_lookup, s_merge_ib_lookup) and its lookup arrays alongside this branch's f_cylinder_axis.
Lines of Code
|
|
Closing this for now to bring the number of open PRs from me down to four, so the ones that stay open get real review attention rather than being lost in a queue of 35. No work is lost and nothing needs re-doing. The branch I intend to reopen these in small batches as the open slots free up. Apologies for the notification noise. Opened and closed with Claude Code. |
1 participant
Summary
Cylinder immersed boundaries (
patch_ib%geometry = 10) are assumed to lie along x in three places, so a cylinder set along y or z (withlength_yorlength_z) either hangs the simulation at start-up or silently loses its body.s_get_ib_boundusessqrt(radius**2 + length_x**2). For a y/z cylinder,length_xis the unset default −1e6, so the bound is 10⁶ m. The automaticib_neighborhood_radius(1536 ib neighborhood radius automatic size checking #1736) turns that into ~10⁹ exchange rounds, ands_get_neighbor_boundsloops inMPI_Sendrecvindefinitely: all ranks were found there in stack traces of a hung GPU job, and the CPU MPI build hangs the same way. Nothing is printed, since Cray Fortran buffers stdout to files. Even for x-aligned cylinders the bound used the full length where half is meant.f_is_inside_cylinder(xyz_local(2), xyz_local(3), xyz_local(1), radius, length_x), so a y/z cylinder marks no cells (since 1454-projection-optimization #1549).length /= 0, which always selects x because the unset default is −1e6, not 0.Change
f_cylinder_axis(1/2/3, whichever length is set, default z) andf_is_inside_ib_cylinderinm_ib_patches; both marker loops and the levelset use them.s_get_ib_bound:sqrt(radius**2 + (0.5*max(length_x, length_y, length_z))**2).x-aligned cylinders are unchanged: no existing golden moved.
Verification
./mfc.sh test --only Cylinder: 8 passed;--only IBM: 69 passed (CPU).Not addressed here: a finite (non-periodic) rod develops a NaN near its end edges after ~20 steps on a coarse grid. That is present with x-aligned rods on master too, so it is a separate issue.
The bug was reproduced, located in a debugger and verified as described above.