Skip to content

Fix invalid integration comparisons and reference-generation failures - #8138

Open
cheerly-pku wants to merge 5 commits into
deepmodeling:developfrom
cheerly-pku:integration-reference-hardening-review
Open

cheerly-pku wants to merge 5 commits into
deepmodeling:developfrom
cheerly-pku:integration-reference-hardening-review

Conversation

@cheerly-pku

@cheerly-pku cheerly-pku commented Oct 13, 2026 •

Copy link
Copy Markdown

What's changed?

Integration comparisons could accept extra unmatched data or non-finite values. Reference generation could also record a failed comparison as an expected result, or return success after a collector failed. This PR rejects those comparisons and makes failed reference generation exit with a nonzero status and remove incomplete result.ref files.

Ordinary result.out files retain failed checks for the harness to report. Trailing blank lines are ignored; missing data, extra data and interior blank lines remain significant. The resolution against develop at c82d87d9e preserves the upstream Wannier gauge-aware validators and LR collectors, including failure propagation through those paths.

The stricter checks exposed stale electric-field references in 19_NO_SUPERSINE_TDDFT and 20_NO_MIXED_EFIELD_TDDFT, causing both CPU and CUDA CI to fail. Upstream 250241c5d intentionally changed the writer to retain zero fields outside the active window. These fixtures evolve steps 0 through 7 at td_dt=0.05 fs, with the field active through step 6. Commit ca7abf4ee appends the missing 0.35 0 sample to their 12 electric-field reference files. All existing samples, energy/vector-potential references, INPUT settings, and comparison tolerances are retained.

Linked Issue

Extracted from #8042 following review so changes affecting all integration tests can be reviewed independently of PPCG. This PR changes integration validation and the twelve incomplete reference files; it contains no solver implementation changes.

Unit Tests and/or Case Tests for my changes

Fresh local verification on 2026-10-26:

  • Full GNU 12.3 / OpenMPI 5.0.10 Release build at ca7abf4ee passed. The existing toolchain configuration enables MPI, OpenMP, LCAO, LibRI, Libxc, ELPA, MLALGO, JSON, CNPY and FLOAT_FFTW; CUDA, DFT-D4 and PEXSI are disabled locally.
  • All 28 CPU rt-TDDFT cases passed, using the exact ca7abf4ee executable, the complete tracked CASES_CPU.txt, 4 MPI ranks per case, one OpenMP/BLAS thread and four concurrent cases. Original acceptance tolerances were used. Case counts were read from test.sum; the legacy harness summary counts properties rather than cases.
  • 38 Python tests and 4 collector CTest tests passed, covering strict file comparison, reference cleanup, Wannier, LR, LibRPA and other validation helpers.
  • An independent evaluation of the analytic field formulas matched all 84 pre-existing field samples and all 96 samples in the repaired references/runtime output. A preliminary two-case run reproduced the original missing-row failure before the repair and passed after it.
  • PR-relative governance and whitespace checks passed with no findings.

Commands (the external validation directory keeps generated files out of the checkout):

validation_dir=/mnt/data/Cheerly/ppcg-pr-conflicts-20261026.qnsme2s7
cd "$validation_dir/pr8138"
cmake --build ../build8138 -j 8 --target abacus_max_para
PATH=/usr/bin:/bin:$PATH OMP_NUM_THREADS=1 python3 -m unittest discover -s tests/integrate/validation_tools -p 'test_*.py' -v
PATH=/usr/bin:/bin:$PATH OMP_NUM_THREADS=1 ctest --test-dir ../pr8138-cmake-build --output-on-failure -R '^(compare_file_test|catch_properties_test|lr_force_collector|lr_energy_collector)$'
python3 tools/03_code_analysis/agent_governance_check.py --base c82d87d9e8a09a8fda3692b94cdc934ae86526a9 --head HEAD --format text
git diff --check c82d87d9e8a09a8fda3692b94cdc934ae86526a9...HEAD

source /mnt/data/Cheerly/abacus-develop/toolchain/install/setup
export OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1
cd "$validation_dir/rt-tddft-8138-full/tests/05_rtTDDFT"
"$validation_dir/build8138/abacus_max_para" --version
bash ../integrate/Autotest.sh -a "$validation_dir/build8138/abacus_max_para" -n 4 -o 1 -j 4 -f CASES_CPU.txt

The collector CTest directory uses an external minimal CMake driver to register the repository's actual tests/integrate/CMakeLists.txt. Build commands, executable/input hashes and raw output are retained in the external validation directory.

Local runtime verification covers the complete CPU rt-TDDFT group. CUDA execution and the other integration groups are left to the fresh CI run triggered by this push. The previous CPU failure and CUDA failure both identify these same twelve missing field rows.

Governance Notes

  • INPUT behavior, core-module APIs, and numerical tolerances: unchanged.
  • Reference update rationale: restore the complete time series required by the documented upstream output change; all pre-existing reference bytes are retained.
  • No governance exceptions requested.

Reminder

@mohanchen mohanchen added Bugs Bugs that only solvable with sufficient knowledge of DFT Refactor Refactor ABACUS codes Tests/Examples Issues/PR related to unit tests and integrate tests labels Oct 13, 2026
@mohanchen

Copy link
Copy Markdown
Collaborator

some conflicts need to be solved. @cheerly-pku

Resolve validation-script conflicts with LibRPA producer changes: retain vxc.ref/vxc.txt naming and comparison failure propagation; preserve upstream MPI launcher flags, dedicated-validator mode and case-list filtering. Register script regressions when a Python interpreter is available without restoring a hard configure dependency.

Verified compare_file 7/7, expanded catch_properties shell regression, CTest 2/2 via the actual tests/integrate CMake subdirectory, upstream LibRPA parser 8/8, Python-disabled configuration, shell syntax and whitespace checks. No numerical references or tolerances changed; no PPCG source commits included. Full ABACUS build/runtime integration not rerun for this script-only resolution.
@cheerly-pku

cheerly-pku commented Oct 13, 2026 •

Copy link
Copy Markdown
Author

some conflicts need to be solved. @cheerly-pku

Thanks for the reminder! The conflicts have been resolved. @mohanchen

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static review of 6575824415cf3edf3a7c56e6d13be2a5b44799e5, with independent static verification. One issue is noted inline. Pinned governance and whitespace checks passed; builds and runtime tests were not run. CI diagnostics were unavailable through the selected account connector.

if test -z $1
then
$CATCH_SCRIPT result.out
bash -e "$CATCH_SCRIPT" result.out

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Propagate collector failure before checking partial results

Please check the collector exit status in both normal and debug modes before calling check_out. For 101_PW_W90, a missing OUT.autotest/diamond.amn now makes bash -e stop after the energy entries; if those match, check_out visits only that partial result.out, skips AMN/MMN/EIG validation, and can finish successfully. Return nonzero on collection failure, as the reference branch does, and add ordinary/debug regression coverage.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching this. Fixed in cd55ddc: both normal and debug modes now exit nonzero if the collector fails, before calling check_out. Added regression coverage for missing Wannier90 output after a valid partial result.out is written. The regression catches the old behavior, and the updated shell regression, comparison tests, syntax, whitespace, and governance checks all pass.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Statically verified at ca7abf4ee4a5a727209bad82395d3353f27a188f: the cd55ddcfb fix remains in both normal and debug paths, which exit before check_out when collection fails. The regression source exercises both modes after a valid result prefix. Builds and runtime tests were not run; CI diagnostics remain unavailable through the selected-account connector.

Upstream 250241c deliberately retains zero-valued samples after the analytic field window. Both supersine and mixed-field fixtures evolve steps 0 through 7 at 0.05 fs, while their fields end at step 6. Append the missing zero sample at 0.35 fs to the twelve affected references; retain all existing values and strict comparison tolerances.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bugs Bugs that only solvable with sufficient knowledge of DFT Refactor Refactor ABACUS codes Tests/Examples Issues/PR related to unit tests and integrate tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants