cogen: decorrelate per-ray KISS seeds across seeds, MPI ranks and batches (fixes #2807)

[willend]

Meant to eventually fix #2807.

Warning

Numerical behaviour change: every run using the default KISS RNG will now give different results for a given --seed than before. Statistics are unchanged (and better for MPI and small seeds), but outputs are not bit-identical to earlier releases. Builds with the Mersenne Twister RNG (RNG_ALG=1) are not affected.

Problem

In the generated raytrace_all and raytrace_all_funnel, each ray was seeded with srandom(_hash((pidx+1)*(seed+1))). Because the key is a product:

• ray p at seed 3 is identical to ray 2p+1 at seed 1. In general, runs at different seeds share rays.
• MPI ranks use mcseed + rank, so the ranks of one run overlap: with -np 8 --seed 1 only ~55% of rays are distinct.
• on MSVC, unsigned long is 32 bits, so the key wraps at 2^32. Seed 2147483647 alternates between just two rays.

Change 1: per-ray seed (code generator only)

srandom(_hash(_hash(seed) + cloop*gpu_innerloop + pidx));

• _hash (existing Mueller hash, randstate_t = uint64_t for KISS) is a 64-bit bijection, so distinct (seed, global ray index) pairs give distinct KISS seeds. The sum is computed in 64 bits on every platform, MSVC included.
• The per-batch seed += gpu_innerloop is removed. The global ray index cloop*gpu_innerloop + pidx replaces it. Without that, seed S batch 1 would equal seed S+gpu_innerloop batch 0.
• No runtime changes and no new helper. _hash is already #pragma acc routine seq.

Change 2: _uid unique across GPU batches and MPI ranks

Previously _uid = pidx (+ rank*ncount), so uids repeated in every GPU batch. The rank stride ncount could also be smaller than the loops*gpu_innerloop rays a rank actually traces when batching rounds ncount up. Now:

_uid = cloop*gpu_innerloop + pidx;
_uid += mpi_node_rank * loops*gpu_innerloop;   // USE_MPI

On CPU (loops=1, gpu_innerloop=ncount) the uids are unchanged. Checked: serial 1000 rays give uids 0..999 (normal loop and FUNNEL), mpirun -np 4 with 4000 rays gives unique uids 0..3999. The batched GPU path is not exercised locally.

Change 3: particle randstate sized by RANDSTATE_LEN

The particle struct had randstate_t randstate[7] hard-coded, and sort_absorb_last() (FUNNEL + SPLIT, mccode-r.c) copied randstate[0..6] by hand. Both now use RANDSTATE_LEN. Results don't change: KISS keeps 7 words (Test_RNG_rand01 md5 still 6c203d69…) and the MT struct shrinks to 1 word. This makes room for an RNG with another state length, e.g. the TinyMT64 GPU generator being prototyped separately. Checked: Unittest_SPLIT, legacy and FUNNEL, RNG_ALG 1 and 2, clean under ASan and UBSan.

Test references updated

Each failing low-statistics test was checked against a referee that does not use per-ray seeding at all: Mersenne Twister (RNG_ALG=1, one global stream), over many seeds.

Test	MT referee	new seeding	old seeding	Action
McXtrace Test_SAXS 8 (SAXSPDB, 1e4 rays)	5.37e-14 ± 0.16 (12 seeds)	5.66e-14 ± 0.20 (16)	6.41e-14 ± 0.28 (16)	reference 7.08e-14 → 5.54e-14
McStas templateSANS_Mantid 1 and 2 (1e6 rays, 2 MPI ranks as in CI)	9.51e7 ± 0.39 (16)	9.77e7 ± 0.32 (16)	10.42e7 ± 0.27 (16)	SPLIT 100 + reference 1.05e8 → 9.6e7
McStas FZJ_BenchmarkSfin2 (SPLIT 2000: the quoted 1% error is ~10× too small)	8136 ± 280 (8)	8832 ± 323 (8)	9464 ± 427 (8)	reference 7400 → 8.5e3 (CI 118%)
McStas Test_SANS 1	3.03e-9 ± 0.13 (8)	2.78e-9 ± 0.18 (8)	2.50e-9 ± 0.15 (8)	reference 2.529e-9 → 2.9e-9 (CI 113%)
McStas Test_Magnon_bcc_TAS 1 (~100 events)	5.03e5 (8, with SPLIT)	4.90e5 (8, with SPLIT)	5.14e5 ± 0.23 (8)	SPLIT 20 on the sample (~1900 events) + reference 4.6e5 → 4.97e5 (CI 102%); its %Scan still passes 21/21
McStas Test_TasReso (~13 events)	936 (8, with SPLIT)	982 (8, with SPLIT)	1135 ± 124 (8)	SPLIT 50 before the samples (~580 events) + reference 887 → 960 (CI 104%)

The new seeding agrees with the referee in every case. The old seeding was about 3σ high for SAXSPDB (in a single process), and about 2σ high for Mantid with MPI, so the old references inherited a bias. A Mantid run scattered ±15%, which is why CI with seed 1000 landed at 7.47e7. With SPLIT 100 on the sample the scatter is ~5% at 2.2 s per run (was 0.8 s), and seed 1000 gives 9.94e7 (2 ranks), 9.55e7 (4) and 9.41e7 (serial). Raising ncount was not an option, because mctest ignores -n in %Example for non-scan tests.

McStas Test_Dispersion_relation_TAS test 6 (a %Scan): the old 5×5 mesh (h, l = 0.6–1.4) put points on the edges of the dispersion ring. Point 22 has reference 0 and in CI caught a single stray event, which mctest can never accept against a 0 target. The mesh now covers h = 0.8–1.2, l = 1.1–1.3, inside the band: every point gets 280–2000 events and scatters 1–5% per run. The references are the mean of 8 seeds each with new seeding and MT, which agree at every point (worst 1.8σ). Seed 1000 (as in CI) and seeds 1 and 2 give 25/25 points within 94–106%.

Verification (macOS arm64, clang, CPU)

Standalone check using the runtime's verbatim _hash / kiss_srandom / kiss_random. A ray's fingerprint is its first 3 draws:

case	old	new
seeds {1,3}, 1e6 rays each, % distinct	75.0%	100%
seed 2147483647, 32-bit truncation, 1e6 rays	0.0002% (2 rays)	100%
seeds 1..8 (MPI -np 8), 125k rays/rank	54.7%	100%

Generated instrument (Progress_bar + Arm with an EXTEND that prints _uid, rand01(), rand01()):

• seeds 1 vs 3, 1e5 rays: old shares 50000 rays, new shares 0. Checked for both the legacy loop and -DFUNNEL.
• mpirun -np 8 -s 1: old 55.6% distinct, new 100%.
• same seed still reproducible run to run.

Not tested: MSVC, Linux gcc/clang, nvc/OpenACC GPU. CI covers the first two.

In progress: McStas and McXtrace test-suite runs are under way to measure how much results change across the instrument suites.

ADR: docs/GRAMMAR/ADR-records/ADR_20261018_PER_RAY_SEEDING.md. mccode-antlr needs the same change ported by hand: mccode-dev/mccode-antlr#392.

🤖 Generated with Claude Code

[willend]

McStas test-suite: https://github.com/mccode-dev/McCode/actions/runs/39104410513
(still running. Basictest already failed for templateSANS_Mantid - planning to correct value)

McXtrace test-suite: https://github.com/mccode-dev/McCode/actions/runs/39104430311
(finished, only Test_SAXS_8 is off - planning to correct value)

[willend]

@g5t, @mads-bertelsen, @farhi FYI patch incoming on KISS RNG. Will likely keep this open until after an evaluation tomorrow.