Skip to content

Towards TSMP2-GPU on JUPITER - #147

Open
kvrigor wants to merge 26 commits into
masterfrom
env-s2026-nvhpc
Open

Towards TSMP2-GPU on JUPITER#147
kvrigor wants to merge 26 commits into
masterfrom
env-s2026-nvhpc

Conversation

@kvrigor

@kvrigor kvrigor commented Oct 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Working builds:

nvhpc=env/jsc.2026.nvhpc.openmpi
gcc=env/jsc.2026.gnu.openmpi
icon2026=/e/project1/training2647/TSMP2-X2/icon2026.04_coup-oas

# eCLM-ParFlow
./build_tsmp2.sh eclm --env $gcc
./build_tsmp2.sh eclm --env $nvhpc

# eCLM-ParFlow
./build_tsmp2.sh eclm parflowgpu --env $gcc
./build_tsmp2.sh eclm parflowgpu --env $nvhpc

# ICON-eCLM
./build_tsmp2.sh eclm icon --icon_src $icon2026 --env $nvhpc
./build_tsmp2.sh eclm icongpu --icon_src $icon2026 --env $nvhpc

# ICON-eCLM-Parflow
./build_tsmp2.sh parflowgpu eclm icongpu --icon_src $icon2026 --env $nvhpc

TODO:

@kvrigor kvrigor linked an issue Oct 14, 2026 that may be closed by this pull request
OASIS3-MCT fails to build under NVHPC+OpenMPI toolchain (Stages/2026) #142
Open
@kvrigor kvrigor linked an issue Oct 14, 2026 that may be closed by this pull request
Deprecate ./build_tsmp2.sh --<component-model> in favor of ./build_tsmp2.sh <component-model> #114
Open
kvrigor and others added 11 commits October 15, 2026 14:50
@kvrigor
eCLM-ICONGPU should build OASIS first.
2a567ec
@kvrigor
Merge branch 'master' into env-s2026-nvhpc
dee820a
@kvrigor
ParflowGPU NVHPC build hacks
e0773bf
@kvrigor
ICON build hacks
381a271
@kvrigor
Changed eCLM to 'dev-iconeclm-exchfix'
52445f9
@kvrigor
ICONGPU should also build eCLM in coupling mode
f5d182f
@kvrigor
eCLM: updated dev-iconeclm-exchfix
a3a0c08
@kvrigor
use loascplscheme_exchcoef only with ICON
d319165
@jjokella
BuildParFlow: Error message enforcing ParFlow on GPUs for NVHPC
8eccd1d
@kvrigor
Ensure OMPI_* vars are unset
ba9de00
@jjokella
Enable PDAF builds on JUPITER with NVHPC compilers (#150)
4d80f45 
## Pure PDAF

- **`BuildPDAF.cmake`**: Add NVHPC support

- **`BuildPDAFMODEL.cmake`**: Make `-lgfortran` conditional on GNU
compiler. Add GPU libraries for `ParFlowGPU` builds.

- **`BuildParFlow.cmake`**: Umpire is passed to ParFlow for
`ParFlowGPU+PDAF` builds

- **`models/parflow_pdaf`**: Update submodule `v3.14.1-pdaf` →
`v3.15.0-pdaf`, necessary update for GPU support

## OASIS (affects all coupled simulations)

- **`BuildOASIS3MCT.cmake`**: Add `make realclean` as configure step
@kvrigor kvrigor changed the title Stages/2026 support on JURECA-DC and JUPITER Towards TSMP2-GPU on JUPITER Oct 23, 2026
This was linked to issues Oct 23, 2026
ESiWACE3 Hackathon Planning #143
Closed
ParFlowGPU build fails on Stages/2026 NVHPC+OpenMPI #148
Open
@kvrigor kvrigor mentioned this pull request Oct 23, 2026
Closed
@kvrigor
Updated OASIS3-MCT to tsmp-patches-v0.2
b07a5a7 
Shortlog:

  554c09fd (tag: tsmp-patches-v0.2) declared character pointer arrays as contiguous
  7c4fc975 Fixed incorrect data type passed to MPI_GATHER
  14503d81 Replaced "include <mpif.h>" with "use mpi"
  9e43db07 Used 8 make threads to speed up build
  c850840d Set default build to S2026 and GNU+OpenMPI
  5253349d (tag: tsmp-patches-v0.1) rename `da_comm` to `COMM_model_oas

https://icg4geo.icg.kfa-juelich.de/ExternalReposPublic/oasis3-mct/-/commits/tsmp-patches-v0.2?ref_type=tags
@kvrigor
BuildICON: GPU build only works on NVHPC for now.
858c3d1
@kvrigor
BuildParflow: Removed unnecessary comment
4708287
@kvrigor
Added comment on why SUNDIALS module was disabled
72ce734
kvrigor added a commit that referenced this pull request Oct 26, 2026
@kvrigor
Updated eCLM to rc-0.1
22c31da 
eCLM shortlog:
  - 9c8f1ae22 (tag: rc-0.1) Added LICENSE header "University of Illinois/NCSA Open Source License" (#150)
  - b06eb7eba eCLM-ParFlow: Couple surface water and route all layer-1 water fluxes to ParFlow (#147)
  - 345124543 docs: Separate JSC HPC and generic HPC quick-start guides (#148)
  - e113460a9 cd back to static.resources after git lfs pull (#144)
  - f7daa6045 Fix directory path in quick start instructions (#143)
  - a8d1d89e1 eCLM-ParFlow: Couple urban landunits (#133)
  - f9c3fa96d eCLM-ParFlow: Normalize ParFlow source term over the gridcell (#132)
  - 55db76e4b eCLM-ParFlow: Calculate hydraulic conductivity based on ParFlow's water content (#129)
  - 66c98a5db eCLM-ParFlow: account for soil ice when receiving ParFlow water content (#126, #123) (#127)
  - c6268fe4e Workaround for cache-apt-pkgs-action issue (#137)
  - 102b7c5d2 Disable the balance-check reporting in eCLM-ParFlow simulations (#125)
  - 5be92812f (tag: beta-0.8) Fix excessively negative ET over sandy soil in eCLM-ParFlow coupling (#105) (#124)
kvrigor added a commit that referenced this pull request Oct 27, 2026
@kvrigor
Updated eCLM to rc-0.1 (#151)
ebb5731 
eCLM release candidate v0.1 (`rc-0.1`) mostly contains eCLM-Parflow
fixes by Stefan. Here's a short log of `rc-0.1` vs. `beta-0.8`:

```
9c8f1ae22 (tag: rc-0.1) Added LICENSE header "University of Illinois/NCSA Open Source License" (#150)
b06eb7eba eCLM-ParFlow: Couple surface water and route all layer-1 water fluxes to ParFlow (#147)
345124543 docs: Separate JSC HPC and generic HPC quick-start guides (#148)
e113460a9 cd back to static.resources after git lfs pull (#144)
f7daa6045 Fix directory path in quick start instructions (#143)
a8d1d89e1 eCLM-ParFlow: Couple urban landunits (#133)
f9c3fa96d eCLM-ParFlow: Normalize ParFlow source term over the gridcell (#132)
55db76e4b eCLM-ParFlow: Calculate hydraulic conductivity based on ParFlow's water content (#129)
66c98a5db eCLM-ParFlow: account for soil ice when receiving ParFlow water content (#126, #123) (#127)
c6268fe4e Workaround for cache-apt-pkgs-action issue (#137)
102b7c5d2 Disable the balance-check reporting in eCLM-ParFlow simulations (#125)
5be92812f (tag: beta-0.8) Fix excessively negative ET over sandy soil in eCLM-ParFlow coupling (#105) (#124)
```

Some extra minor changes in this PR too:
- CI: Removed 'ICON standalone' job
- CI: Updated OASIS download logic
- Deprecate env files for `Stages/2023` and `Stages/2024`
@kvrigor
Updated eCLM to rc-0.1
9045eeb 
Merged from `master` branch (PR #151)
@kvrigor
CI: Updated OASIS to tsmp-patches-v0.2
9e6786a 
Updates to OASIS mostly related to NVHPC fixes.
Expected tarball checksum is d5c2618563c4883bd309b196be63313e4f60dd0f58c3d926d3ed6bebad685039
@kvrigor
CI: OASIS version shouldn't be hardcoded
0403403
@kvrigor
Updated tsmp-patches-v0.2 to include gfortran compile fixes
2bb90da 
Details of the fix is at https://icg4geo.icg.kfa-juelich.de/ExternalReposPublic/oasis3-mct/-/commit/59e03411bbbe9e6c39612fb62402f189cc0c4a38
@kvrigor
kvrigor marked this pull request as ready for review October 27, 2026 12:15
@kvrigor

kvrigor commented Oct 27, 2026

Copy link
Copy Markdown
Member Author

Hello @AGonzalezNicolas , @s-poll , and @jjokella : this PR is ready to be merged and I would need help to review it. This is the same branch we used in the ESiWACE hackathon last week. While there are a lot of files changed in this PR, the changes can be summarized as so:

  • Introduced 2026 environment files. These files were tested on JURECA, JUWELS, and JUPITER.
  • Changes in build_tsmp2.sh and CMake scripts mostly relate to Stages/2026 NVHPC and ICONGPU fixes.
  • Updated OASIS3-MCT to tsmp-patches-v0.2. This update mostly contains fixes to make Stages/2026 NVHPC work.
  • PDAF compile fixes on NVHPC by Johannes.
  • As with the recent update in eCLM CI, TSMP2 CI is also now decoupled to IBG-3 Gitlab.

AGonzalezNicolas
AGonzalezNicolas reviewed Oct 27, 2026
View reviewed changes

@AGonzalezNicolas AGonzalezNicolas left a comment •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR drops the existing --icon alias (and likewise --eclm, --parflow, etc.).
With this PR, the build script will fail with --icon --eclm --parflow.

@kvrigor

kvrigor commented Oct 27, 2026

Copy link
Copy Markdown
Member Author

This PR drops the existing --icon alias (and likewise --eclm, --parflow, etc.). With this PR, the build script will fail with --icon --eclm --parflow.

Yes, the plan is to deprecate --<model> options as described in #114

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@AGonzalezNicolas AGonzalezNicolas AGonzalezNicolas approved these changes
@jjokella jjokella Awaiting requested review from jjokella
@s-poll s-poll Awaiting requested review from s-poll

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

3 participants

@kvrigor @AGonzalezNicolas @jjokella