Skip to content

Simple mcplot and mcdisplay prototypes based on Scipp and Plopp - #2799

Open
willend wants to merge 12 commits into
mainfrom
plopp-prototypes
Open

willend wants to merge 12 commits into
mainfrom
plopp-prototypes

Conversation

@willend

@willend willend commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Free-form text area

Please describe what your PR is adding in terms of features or bugfixes:

Plopp-based prototypes: mcplot-plopp and mcdisplay-plopp

Two new Python frontends built on ESS plopp/scipp, alongside the existing matplotlib and pyqtgraph tools. Both reuse mccodelib for loading and tracing. By default they open a one-cell JupyterLab notebook with the interactive UI (ipympl/pythreejs widgets); -o FILE writes a static png/pdf/svg instead.

mcplot-plopp / mxplot-plopp

  • ASCII output is loaded via McCodeDataLoader and converted to scipp DataArrays:
    • 1D monitors get I_err² as variances and bin edges from xlimits
    • 2D monitors use dims ('y', 'x'), the same orientation as the matplotlib frontend
  • NeXus output: a mccode.h5/.nxs file, or a directory with mccode.h5 and no mccode.sim (the same rule the matplotlib/pyqtgraph mcplot use before handing off to nexpy).
    • 2D monitors are stored (nx, ny) in NeXus, the transpose of the ASCII block, and are transposed to the same ('y', 'x') layout.
    • The NeXus errors block gives 2D monitors variances.
    • h5py is imported only when reading NeXus.
  • Units are read from [...] in the axis labels (AA/Angs, meV, deg, us, cm, m, s). Unknown units stay in the label and the axis is dimensionless.
  • Scans: each monitor is stacked over the scan parameter (1D→2D, 2D→3D).
    • ASCII: numbered step folders, scan values from mccode.dat.
    • NeXus: one NXentry per step, stacked over the first parameter that varies across entry*/simulation/Param.
  • Event data: list-mode Monitor_nD pixel banks (neutron pixel min=… t, list all neutrons) in NeXus files are read via McStasToX.
    • McStasToX needs the source and sample components. The UI offers dropdowns (defaulting to components named like "source"/"sample"); the CLI takes --source/--sample.
    • Shows a pixel intensity map (3D scatter in the UI, z–x top view with -o) plus time-of-flight and wavelength histograms.
    • Wavelength per event uses the source → sample → pixel flight path, λ = 2π·AA2MS·t/L, with AA2MS from mcstas-r.h.
    • mcstastox is an optional dependency, imported only for event data.
  • UI: a monitor dropdown showing one figure at a time; scan monitors get plopp's slicer with a ▶ player.

mcdisplay-plopp / mxdisplay-plopp

  • Traces with --trace=2. Solids are drawn as 3D meshes with pp.mesh3d:
    • cylinder (solid or hollow), cone, disc, annulus, sphere
    • box (solid or hollow)
    • polyhedron (OFF and polygon geometry)
  • Lines, multilines and circles are drawn as thin tubes, with small spheres at corners.
  • Components are coloured per component; rays are thinner grey tubes.
  • UI: a params field, a ray count, a Trace button, and a thickness slider that redraws without re-running mcrun.
  • -o writes 2D z–x / z–y projections instead (solids as triangle outlines), since pythreejs can't render without a browser.
  • -y/--yes is forwarded to mcrun (use defaults for unset parameters). Without it, mcrun prompts as usual. The UI always passes it, because a widget can't answer prompts.
  • Shape conventions follow mcdisplay-webgl, the reference trace=2 renderer: shapes are centred on (x,y,z), a box's local y is turned onto n, and a cone's apex is at +h/2.

Dependencies

  • scipp and plopp are required. Missing either is caught by guarded imports, which exit with a message plus conda and pip install hints. The UI also needs ipympl and jupyterlab.
  • h5py (NeXus) and mcstastox (event data) are imported only when needed.

Build and packaging

  • CMake subprojects tools/Python/{mcplot,mcdisplay}/plopp, set up in parallel to the matplotlib tools. The bash wrappers only check that the tool script is executable.
  • Windows (MinGW):
    • .bat wrappers registered in InstallMCCODE.cmake
    • mkdist … nsis64 lines in build_windows_mcstas / build_windows_mcxtrace
    • both packages added to the MinGW metapackage .iss files
    • scipp and plopp are not added to meta-pkgs/windows/environment.yml; until users install plopp, the tools print the install message
  • No Debian packages: scipp/plopp aren't packaged by Debian.

Testing

  • Checked on macOS in a conda env with plopp/scipp 26.9.0.
  • mcplot-plopp:
    • real single-run, scan and McStas 2.7 (Odin, 117 monitors) ASCII output
    • NeXus from mcrun --format=NeXus: a non-square 30×10 PSD plus a 1D monitor gives DataArrays identical to the same-seed ASCII run (dims, edges, values); a 3-step lambda scan stacks correctly; BNL_H8 single and scan load all 9 monitors
    • events: a scratch instrument with two 32×16 pixel banks (one rotated 60°) and an Incoherent sample gives wavelengths 1.50–2.51 Å for a 2 ± 0.5 Å source, time of flight 1.1–1.9 ms over 3 m, and banks placed correctly (mcstastox 0.0.12)
  • mcdisplay-plopp: BNL_H8, mini, Polygon_ESS_butterfly_image, and a scratch instrument with Incoherent in every solid shape (bounding boxes match the parameters, including a hollow box rotated 30°).
  • All UIs, including the 3D views, run without errors in executed notebooks (nbconvert --execute).
  • Assert-based self-checks in test_mcplopp.py and test_mcdisplopp.py: units, orientation, a synthetic mccode.h5 (variances, scan stacking), tube and solid geometry.
  • CMake configure/build/install checked standalone for both McStas and McXtrace.

Not yet tested: Windows/NSIS builds, Linux, the McXtrace runtime, and the 3D views in a live browser session.

Known gaps / follow-ups

  • 2D monitors read from ASCII have no variances (mcplotloader skips the # Errors block); NeXus input has them.
  • NeXus scans over several parameters use only the first one that varies as the scan axis.
  • Event data is NeXus only. McStasToX doesn't read ASCII, and ASCII list files carry no rotations or pixel geometry. McStasToX also reads only entry1, the first step of a NeXus scan.
  • The notebook cell must be run by hand. Voila could make it a self-starting app, but it would be a new dependency.
  • The man-page .md.in docs aren't written yet.
  • annulus()'s 5th argument is named inner_radius in the runtime but drawn as a ring width by both webgl and matplotlib. I followed the frontends; the naming could be clarified.

Declaration of use of AI-tools

  • Please add a checkmark here if you used AI-tools during the work for this contribution
  • Furter, please describe how / where and for what the tools were used:

^ 🤖 Generated with Claude Code


Development OS / boundary conditions

Please describe what OS you developed and tested your additions on, and if any special dependencies are required:


PR Checklist for contributing to McStas/McXtrace

For a coherent and useful contribution to McStas/McXtrace, please fill in relevant parts of the checklist:

  • My contribution contains something else

    • Explanation is added in free form text above or below the checklist

willend and others added 4 commits October 13, 2026 08:50
New Python tools built on ESS plopp/scipp, alongside the matplotlib and
pyqtgraph frontends. Both reuse mccodelib for loading and tracing. By
default they open a one-cell JupyterLab notebook (ipympl widgets);
-o FILE writes a static png/pdf/svg instead.

mcplot-plopp (tools/Python/mcplot/plopp/mcplopp.py)
- Monitors via McCodeDataLoader, converted to scipp DataArrays:
  1D with I_err^2 variances and bin edges from xlimits; 2D as (y, x),
  consistent with the matplotlib frontend
- Axis units parsed from "[...]" labels (AA/Angs, meV, deg, us, cm, m, s);
  unknown units stay in the label and the axis is dimensionless
- Scans: each monitor stacked over the scan parameter (1D->2D, 2D->3D);
  UI offers a monitor dropdown plus a per-step slicer with player
- No variances for 2D monitors yet (mcplotloader skips "# Errors")

mcdisplay-plopp (tools/Python/mcdisplay/plopp/mcdisplopp.py)
- Traces via McDisplayReader (unset params take defaults); draws the
  z-x and z-y projections of components and rays
- UI: params text field, ray count and a Trace button that re-traces
- Wireframe primitives only (line/multiline/circle, as pyqtgraph);
  no 3D view, since plopp has no 3D line primitive

Build/packaging
- CMake subprojects in parallel to the matplotlib tools; installs
  {mc,mx}plot-plopp and {mc,mx}display-plopp. The bash wrappers check the
  plopp stack and suggest pip if it is missing
- Windows: plot-plopp/display-plopp .bat wrappers registered in
  InstallMCCODE.cmake; mkdist nsis64 lines in build_windows_mcstas and
  build_windows_mcxtrace; both packages added to the MinGW metapackage
  .iss files; scipp and plopp added to meta-pkgs/windows/environment.yml
- No Debian packaging: scipp/plopp are not packaged by Debian

Tested on macOS (mccode-dev env): static and UI paths against real
single-run, scan and McStas 2.7 output, and BNL_H8/mini traces. CMake
configure/build/install checked standalone for McStas and McXtrace.
Not yet tested: Windows/NSIS builds and Linux.
The JupyterLab UI now draws the instrument in 3D with plopp's mesh3d
(pythreejs) instead of the 2D projections:
- Lines, multilines and circles become thin 6-sided tubes, with a
  low-poly sphere (icosahedron) at each corner where the polyline turns
  by more than ~20 deg. Line ends, straight joints and arc points get
  none, which keeps the mesh light
- Components are coloured by index (tab20); rays are light grey tubes
  at half the radius
- Tube radius = thickness x instrument bounding-box diagonal (default
  2.5e-4, ~2 mm on BNL_H8). A thickness slider in the UI redraws the
  last trace without re-running mcrun
- Circles are drawn with 24 points instead of mccodelib's default 60
  (also used by the 2D view)

-o FILE still writes the 2D z-x/z-y projections, since pythreejs cannot
render without a browser.

Tests: closed icosahedron, ring radius about the segment axis, mesh
counts, and corner detection on an L-shaped line. The BNL_H8 UI
(initial draw and slider redraw) runs in an executed notebook without
errors; not yet viewed in a live browser session.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Trace in --trace=2 mode (was the McDisplayReader default, 1), so the
runtime's 3D draw calls reach plopp:
- cylinder (solid or hollow), cone, disc, annulus and sphere are swept
  around their axis n into triangle meshes (one _loft ring-stitching
  helper, 24 segments)
- box (solid or hollow) is rotated so local +y lies along n; a hollow
  box is open along that axis
- polyhedron JSON (OFF / polygon) is used as-is, with polygon faces
  fanned into triangles
- line, multiline, circle and new_circle stay as thin tubes; solids
  take their component's colour in the 3D view, and the 2D -o view
  draws them as triangle outlines

Conventions follow mcdisplay-webgl, the reference trace=2 renderer:
shapes are centred on (x,y,z), a cone's apex is at +h/2 along n, and
annulus's 5th argument is a ring width (as in webgl and matplotlib,
although the runtime names it inner_radius; no shipped component calls
annulus() directly). new_circle is drawn as a tube ring rather than
webgl's 1 cm band. DrawPolygon is not handled; at trace=2 the runtime
emits polygons as polyhedra.

-y/--yes is forwarded to mcrun so the instrument uses defaults for
unset parameters; this replaces auto-answering every stdin prompt.
Without -y, mcrun prompts as usual. The UI always passes -y, because
a widget cannot answer prompts.

Tests: box placement and rotation, cylinder/cone extents, closed
surfaces (vertices welded by position), polyhedron fan-out. Checked
against a scratch instrument with Incoherent in every shape (bounding
boxes match the parameters, including a hollow box rotated 30 deg)
and BNL_H8 / Polygon_ESS_butterfly_image in the 2D view. The 3D UI
runs without errors in an executed notebook; not yet viewed in a live
browser session.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@willend
willend marked this pull request as draft October 14, 2026 17:35
willend and others added 5 commits October 14, 2026 19:35
- mcplopp.py and mcdisplopp.py wrap the scipp/plopp imports in
  try/except ImportError and exit with a message saying they are
  required, with conda and pip install hints (plus ipympl and
  jupyterlab for the UI)
- The bash wrappers no longer test-import python modules; canrun()
  only checks that the tool script exists and is executable
- scipp and plopp removed again from meta-pkgs/windows/environment.yml;
  on Windows the tools now print the guard message until plopp is
  installed

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
load() now hands *.h5 / *.nxs files, and directories that have
mccode.h5 but no mccode.sim (the rule the matplotlib/pyqtgraph mcplot
use before spawning nexpy), to a new load_nexus():
- Monitors are the NXdata groups under
  entry*/instrument/components/<comp>/output/. type, title, labels and
  x(y)limits come from their attributes; edges come from the limits,
  since the axis datasets hold bin centres
- 2D data is stored (nx, ny) with axis=1 on x, the transpose of the
  ASCII block, so it is transposed to the ('y', 'x') layout used for
  ASCII. The NeXus errors block gives 2D monitors variances (the ASCII
  loader skips "# Errors")
- Scans: one NXentry per step. Monitors are stacked over the first
  parameter that varies across entry*/simulation/Param
- 0D and event-list monitors are skipped, as for ASCII
- h5py is imported only when reading NeXus; if it is missing the tool
  exits with a message

_curve()/_image() are shared by the ASCII and NeXus paths.

Validated against real mcrun --format=NeXus output: a non-square
30x10 PSD plus a 1D monitor gives DataArrays identical to the
same-seed ASCII run (dims, edges, values); a 3-step lambda scan stacks
correctly; BNL_H8 single and scan load all 9 monitors; the UI runs on
the NeXus scan in an executed notebook. test_mcplopp.py gains a
synthetic mccode.h5 check (orientation, variances, scan stacking).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
List-mode Monitor_nD banks with pixel ids ("neutron pixel min=... t,
list all neutrons") in a NeXus file are read through McStasToX
(mccode-dev/McStasToX), which needs the source and sample component
names:
- load_events(): McStasToX export_scipp() events, grouped by pixel
  with pixel positions; adds per-event wavelength from the source ->
  sample -> pixel flight path, lambda = 2 pi AA2MS t / L (AA2MS from
  mcstas-r.h)
- event_figures(): pixel intensity map (3D scatter in the UI, z-x top
  view for -o) plus time-of-flight and wavelength histograms
- UI: an "Event data" row appears when the file has pixel-id banks,
  with source/sample dropdowns (defaulting to components named like
  "source"/"sample") and a Load button
- CLI: --source/--sample add the event figures to the static output
- mcstastox is imported only for event data and fails with an install
  hint if missing
- The UI no longer fails on files that hold only event monitors (empty
  monitor dropdown)

NeXus only: McStasToX does not read ASCII, and ASCII list files carry
no rotations or pixel geometry. McStasToX reads entry1 only, i.e. the
first step of a NeXus scan.

Validated on a scratch instrument (Incoherent sample, two 32x16 pixel
banks, one rotated 60 deg): wavelengths 1.50-2.51 AA for a 2 +- 0.5 AA
source, TOF 1.1-1.9 ms over 3 m, banks placed correctly; the UI incl.
the 3D view runs in an executed notebook (mcstastox 0.0.12).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@willend
willend marked this pull request as ready for review October 18, 2026 11:33
@willend willend changed the title DRAFT: mcplot and mcdisplay prototypes based on Scipp and Plopp Simple mcplot and mcdisplay prototypes based on Scipp and Plopp Oct 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant