Repository navigation
Conversation
laraPPr
added
accel:nvidia
2025.06-software.eessi.io
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-vsc-ugent for:arch=x86_64/amd/zen3,accel=nvidia/cc80 |
|
New job on instance
|
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-surf for:arch=x86_64/amd/zen4,accel=nvidia/cc90 |
|
New job on instance
|
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-surf for:arch=x86_64/amd/zen4,accel=nvidia/cc90 |
|
New job on instance
|
|
@casparvl why is cuda compute capabilities set like this? LAMMPS does not like it. |
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-vsc-ugent for:arch=x86_64/amd/zen3,accel=nvidia/cc80 |
|
New job on instance
|
Because that's the target for which we want CUDA code to be compiled :D If a particular package doesn't support the suffixes in the targets, we should make sure they get stripped. We could do this in an EESSI hook, but it would be better to do it upstream in EasyBuild. Both Note that the |
|
It is because it is not in this mapping, https://github.com/easybuilders/easybuild-easyblocks/blob/ad5538e0d532f06ecdc801794e390db49aa5c350/easybuild/easyblocks/l/lammps.py#L158-L177. From your explanation I see adding 9.0a as an option if building with |
|
We do not build lammps with We add the following mapping in the easyblock or overwrite cuda_cc in the hook. I prefer option 1. |
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-surf for:arch=x86_64/amd/zen4,accel=nvidia/cc90 |
|
New job on instance
|
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-jsc for:arch=aarch64/nvidia/grace,accel=nvidia/cc90 |
|
New job on instance
|
|
nevermind I have access |
|
Can't get to the build logs so need this before I can continue |
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-jsc for:arch=aarch64/nvidia/grace,accel=nvidia/cc90 |
|
New job on instance
|
|
Probably hitting the ARM Neon issues on aarch64 with Kokkos due to CUDA being lower than CUDA 13.1 (or 13.2, which officially fixes the issue)? |
|
New job on instance
|
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-vsc-ugent for:arch=x86_64/amd/zen3,accel=nvidia/cc80 |
|
New job on instance
|
|
A warning about the Zen 5 cc120 builds like #1461 (comment) These are actually not succeeding! This is coming from here: https://github.com/EESSI/software-layer-scripts/blob/2eb7da3098efdd57006783e4752e66862fce167d/eb_hooks.py#L759 With CUDA CC getting 12.0f from https://github.com/EESSI/software-layer-scripts/blob/6327b3a782f4a17ec54cc2f809624980e33a2f64/EESSI-extend-easybuild.eb#L113-L114 |
|
@Neves-P well... unless I'm overlooking something, I think this is completely fine and intentional. It's indeed putting a 'fake' module in place that warns you that you cannot load this LAMMPS module on CC120 systems, simply because the underlying CUDA version doesn't support these new GPUs. The alternative would have been to not put anything in place for CC120 systems at all, but that'd mean that the modules available on different systems for EESSI/2025.06 wouldn't be the same everywhere. And without a module that prints this error, we would have no way of explaining the user why that is :) |
|
It seems the reason this is working at UGent is this section in the site-config script of that bot: We don't have that at SURF. |
|
Hm, maybe that's not it. That seems to be set anyway by https://github.com/EESSI/software-layer-scripts/blob/6327b3a782f4a17ec54cc2f809624980e33a2f64/eessi_container.sh#L1043 so I guess the one in the UGent site config script isn't strictly needed anymore... |
|
Setting that didn't help. Tried some more basic debugging, on a GPU node. Since it seems related to nvidia-smi breaking or working, I'm using that as a surrogate. First, I start the container: Then, let's check if Then, I run So, it breaks as soon as I enter The main thing I'm wondering now is not: why does it not work on Snellius? I'm wondering: how would this work on any native system? It shouldn't. I'll have a look if I can login to the UGent system and see what happens if I try the above there. |
|
Failing ctests are now also at UGent and it also fails to run nvidia-smi check. Which then cause |
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-vsc-ugent for:arch=x86_64/amd/zen3,accel=nvidia/cc80 |
|
New job on instance
|
|
Ok, we figured it out it seems: the issue is not that the container clears environment, but This was never an issue before for other builds, because other builds would NOT rely on host binaries - they'd only use binaries from EESSI (which use the EESSI runtime linker, and thus find It is very specifically because our LAMMPS build does an EasyBuild check through Which in turn relies on I don't think there's any downside to propagating We've tested this by running: And then in the debug shell try: Which now succeeds (it failed before, if you don't set My suggestion is we do this from the |
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-vsc-ugent for:arch=x86_64/amd/zen3,accel=nvidia/cc80 |
|
New job on instance
|
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-vsc-ugent for:arch=x86_64/amd/zen3,accel=nvidia/cc80 |
Unable to download or merge changes between the source branch and the destination branch. |
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-vsc-ugent for:arch=x86_64/amd/zen3,accel=nvidia/cc80 |
|
New job on instance
|
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-surf for:arch=x86_64/intel/icelake,accel=nvidia/cc80 |
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
8 participants
$LD_PRELOADfor testing CUDA build without device present easybuilders/easybuild-easyblocks#4190$LD_PRELOADwhen crosscompiling NVIDIA GPU build on CPU-only system + disable Arm NEON to work around failing build on Arm when using CUDA < 13.2 easybuilders/easybuild-easyblocks#4222 (combo of last two PRs above)