You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Add get_schedule() to the Python simulators module - #7500
The Python simulators give no access to the Schedule object the simulator itself uses. A Schedule built in Python is a separate object, so changing it, for example with shut_well(), has no effect on a simulator constructed from a file name. The only way to control wells from Python has been the constructor that takes Deck, EclipseState, Schedule and SummaryConfig objects. In a parallel run that constructor works only if None is passed for the EclipseState, which is not documented anywhere. OPM/opm-python-documentation#37 documents exactly that workaround for running in parallel with mpi4py.
This PR adds get_schedule(), which returns the simulator's own Schedule after step_init(). Well controls can then be changed with the filename constructor:
sim=BlackOilSimulator(filename="SPE1CASE1.DATA")
sim.setup_mpi(init=False, finalize=False)
sim.step_init()
sim.get_schedule().shut_well("PROD", 3) # on every MPI process
It also rewrites the docstring of the objects constructor, which did not say what the simulator does with each of its four arguments.
This branch is based on #7444 and contains its commit. Please review that one there; this PR adds the commits after it.
Add get_schedule() to the Python simulators (commit 1)
Main::schedulePtr(): a public getter for the std::shared_ptr<Schedule> that Main already holds and passes on to the vanguard. This is the object the simulation uses.
PyBaseSimulator::getSchedule(), bound as get_schedule() in all three simulator classes. It hands over the existing shared pointer, so Python and the simulator share one object. With the objects constructor it returns the very object that was passed in. Calling it before step_init() raises RuntimeError, as step() does.
Imports opm.io.schedule before returning. The Python type for Schedule is registered by opm-common's module. Without the import, pybind11 cannot convert the return value when the script has not imported opm.io itself, and every process fails with TypeError: Unregistered type : Opm::Schedule.
Docstring: in a parallel run every process holds its own copy of the Schedule, so a change must be made in the same way on every process.
New test python/test/test_get_schedule.py, registered as the ctest case python_get_schedule. It checks that get_schedule() raises before step_init(), that a well shut through it is shut in the simulation (FOPR is zero from that report step on), and that with the objects constructor it returns the object that was passed in.
Document that the Deck argument is used only for its file name (commit 2)
Adds a sentence to the Deck parameter of the objects constructor: the simulator reads the DATA file again itself, so changes made to the Deck object in Python do not reach the simulation. Commit 3 makes this more precise.
Say what each objects-constructor argument is used for (commit 3)
Deck: only its file name is used, and only when EclipseState is None. Otherwise the simulator takes the file name from the EclipseState.
EclipseState: None is recommended. When given, the simulator uses it instead of building its own, and takes the name of the DATA file to read from it. In a parallel run it must be None. (Commit 5 rewords this.)
Schedule and SummaryConfig: used as they are, so a schedule or a summary configuration built or changed in Python takes effect. From step_init() on, the given Schedule is the object get_schedule() returns.
Parallel runs: every change made from Python must be made in the same way on every process, both before and after step_init().
Points to the filename constructor together with get_schedule() as the simpler way to change well controls.
The signature now shows EclipseState: EclipseState | None.
Run the objects-constructor test from its own copy of the deck (commit 4)
Addresses the Copilot review. test_03_objects_constructor ran in the deck directory that test_schedule.py also uses. With an EclipseState given, the simulator clears old result files, and writes its .PRT and .DBG files, in the directory of the deck the EclipseState was built from, even when --output-dir names another one. Run concurrently, the two tests could delete each other's output. The test now copies the deck into a directory of its own.
Describe a given EclipseState as used, not as unable to carry changes (commit 5)
Addresses the Copilot review. A given EclipseState is used as it is, including its grid and properties, so the docstring no longer says that everything except the Schedule and SummaryConfig is read from the DATA file again. Instead of claiming that the state cannot carry changes made in Python, it now says that the state is used as given and should be built from the same DATA file.
Keep the get_schedule() docstring neutral about how to change it (commit 6)
Following the discussion of Changing the Fundamental Data Processing Model for the Schedule Object opm-common#5371: the get_schedule() docstring no longer names shut_well() and per-report-step changes as the way to use the returned Schedule. Those step-indexed operations belong to opm-common's Schedule bindings and are expected to change. It now only says that changes made to the returned object take effect in the simulation.
Verification
Wrote test_get_schedule.py first; all three tests failed with AttributeError before commit 1. All 8 python_* ctest cases pass after it, and after commit 6.
After commit 4, a run of python_get_schedule leaves no result or log files in the shared deck directory; the test's own copy gets them all.
On 4 MPI processes, SPE1CASE1, filename constructor, with get_schedule().shut_well("PROD", 3) called on every process: WOPR:PROD per report step is 20000, 20000, 20000, 0, 0, 0. Without the call it stays at 20000. The same result with a script that never imports opm.io before calling get_schedule(), which is what failed before the import was added.
A well shut at report step 0 through get_schedule() right after step_init() is shut from the first report step, on 1 and on 4 processes.
Checked each claim of the new docstring by running the simulator:
a Deck parsed from a copy named OTHER.DATA, together with an EclipseState from SPE1CASE1.DATA, makes the simulator read and write SPE1CASE1; with None for the state it reads and writes OTHER;
a Schedule built from the deck text with the oil rate target of PROD changed from 20000 to 10000 gives WOPR:PROD 10000 at every step, on 1 and on 4 processes;
a SummaryConfig built from the deck text with FOPT added makes FOPT appear in the summary file;
a keyword added with Deck.add() does not change the EclipseState built from that deck, because it lands at the end of the deck, in the SCHEDULE section.
Follow-ups
Two further PRs build on this one:
an early, clear error when an EclipseState object is passed in a parallel run, which today aborts while the grid is being distributed;
an error when the Schedule differs between the MPI processes, either before step_init() or before a report step. Today a change made on only some of the processes is silently ignored.
The two constructors in docstrings_simulators.json documented their
arguments as
__init__(deck_filename: str)
__init__(deck: Deck, state: EclipseState, schedule: Schedule,
summary_config: SummaryConfig)
but the bindings in python/simulators/Py*Simulator.cpp name them
filename, and Deck, EclipseState, Schedule, SummaryConfig. Passing the
arguments by keyword as documented raises TypeError; only positional
calls worked. OPM#7439 corrected the same kind of mismatch for four
methods, but did not cover the constructors.
Use the bound names in the signatures and in the :param: entries. The
documentation follows the bindings rather than the other way round,
since renaming the bound arguments would break existing keyword calls
such as BlackOilSimulator(filename=...).
Also document the optional args parameter that both constructors take.
It was not mentioned at all. It holds extra command-line options for
the simulator, which are added after the program and deck names and
applied by step_init().
The simulators had no way to give Python the Schedule object they use.
A Schedule built in Python is a different object, so changing it, for
example with shut_well(), has no effect on a simulator built from a
file name. The only way to control wells from Python was the objects
constructor, which in a parallel run works only when None is passed
for the EclipseState.
get_schedule() returns the simulator's own Schedule after step_init(),
as the shared pointer the simulator holds, so Python and the simulator
share one object. With the objects constructor it returns the object
that was passed in. It imports opm.io.schedule itself, since otherwise
pybind11 cannot convert the return value when the script has not
imported opm.io.
In a parallel run each rank holds its own copy of the Schedule, so a
change must be made on every rank; the docstring says so.
The objects constructor passes only the Deck's file name on to the
simulator, which then reads the DATA file again. Changes made to the
Deck object in Python therefore never reach the simulation. Say so in
the constructor docstring.
The docstring listed four objects without saying what the simulator
does with them, and the sentence about the Deck was too generous. In
fact:
- the Deck is used only for its file name, and only when EclipseState
is None; otherwise the file name comes from the EclipseState;
- the EclipseState only saves the simulator building its own, and it
cannot carry changes made in Python (its Python API is read-only); in
a parallel run it must be None;
- the Schedule and the SummaryConfig are used as they are, so changes
made to them in Python take effect.
Say so per argument, add the rule that in a parallel run every change
must be made on every process, before and after step_init(), and point
to the filename constructor plus get_schedule() as the simpler way to
change well controls.
Just a heads up: I appreciate the reasoning, but please know that once I get OPM/opm-common#5371 implemented, this isn't going to be very useful. Any operation that's currently based on using the Schedule object and a report step index will be superseded by direct ScheduleState access.
Any operation that's currently based on using the Schedule object and a report step index will be superseded by direct ScheduleState access.
@bska Thanks for the heads-up, that is useful to know before going further. I read through OPM/opm-common#5371, and I agree that the examples in this PR, shut_well(name, step) and friends, are exactly the report-step-indexed operations it will supersede. Those are opm-common's existing Python bindings, though, and as far as I can see they would need a new form after OPM/opm-common#5371 whether or not this PR goes in.
What this PR adds is narrower: a way for Python to reach the Schedule instance the simulator actually uses. At the moment a script can only change a Schedule it built itself and passed to the constructor, which in parallel only works with None for the EclipseState (see OPM/opm-python-documentation#37). After OPM/opm-common#5371 the Schedule object would still own the ScheduleDeck and the current ScheduleState, so I would expect Python to still need this access point, with different operations on it, much as PYACTION needs some way to change the schedule during a run.
So my question is whether you see get_schedule() itself as going away, or mainly the step-indexed operations on what it returns. If it is the latter, would you be OK with merging this as an interim step, with the docstring wording kept neutral about the step-indexed calls, and adapting the Python side together with OPM/opm-common#5371? If you would rather wait, I will keep it in draft. Is there a rough timeline for OPM/opm-common#5371?
test_03_objects_constructor ran in the deck directory that
test_schedule.py also uses. With an EclipseState given, the simulator
clears old result files, and writes its .PRT and .DBG files, in the
directory of the deck that the EclipseState was built from, even when
--output-dir names another directory. Run concurrently, either test
could therefore delete the other's output. Copy the deck into a
directory of its own and run the test there.
The objects-constructor docstring said that everything except the
Schedule and SummaryConfig is read from the DATA file again, and that
the EclipseState cannot carry changes made in Python. A given
EclipseState is in fact used as it is, including its grid and
properties. Say that, and that it should be built from the same DATA
file, instead of claiming what it cannot do.
So my question is whether you see get_schedule() itself as going away, or mainly the step-indexed operations on what it returns.
Once I have OPM/opm-common#5371, about the only thing you'll be able to meaningfully do to the Schedule object as a whole, at least from Python, is retrieve the "current" ScheduleState. I'm also hoping to limit the amount of Python-level operations defined as "construct keyword text and apply that to the current state", but that's a (much) bigger task.
I sadly don't have a (good) timeline for OPM/opm-common#5371. It probably won't be ready for testing for several weeks yet, end of October at the earliest. I'm working on it in the background in addition to reviewing other PRs and my release manager work for the upcoming 2026.10 release.
The docstring named shut_well() and per-report-step changes as the way
to use the returned Schedule. Those step-indexed operations belong to
opm-common's Schedule bindings and are expected to change. Only say
that changes made to the returned object take effect in the simulation.
the only thing you'll be able to meaningfully do to the Schedule object as a whole, at least from Python, is retrieve the "current" ScheduleState
@bska Thanks, that is clear. So after OPM/opm-common#5371, get_schedule() is still how a script reaches the simulator's own Schedule, and from there the current ScheduleState; what changes is what one can do with it. I have made the get_schedule() docstring neutral about how the returned object is changed (74abd03), so this PR no longer points at the step-indexed calls.
Since OPM/opm-common#5371 is still some weeks out, I would like to keep this as an interim step: until then it is the only documented way for a Python driver to change well controls in a parallel run. I will keep it in draft until #7444 is merged. Would you be OK with merging it then, or would you rather it wait for OPM/opm-common#5371, or until after the 2026.10 release? Either is fine with me.
Optional regression coverage: test_get_schedule.py imports opm.io.schedule at module load, so its tests would still pass if this import were removed. Add a fresh-interpreter test that imports only opm.simulators, initializes a filename-based simulator, and successfully obtains and uses get_schedule(). This would catch the reported unregistered-type failure.
🧠 Review effort: Balanced
Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.
So after OPM/opm-common#5371, get_schedule() is still how a script reaches the simulator's own Schedule, and from there the current ScheduleState; what changes is what one can do with it. I have made the get_schedule() docstring neutral about how the returned object is changed (74abd03), so this PR no longer points at the step-indexed calls.
That's a fair assessment, yes.
I would like to keep this as an interim step: until then it is the only documented way for a Python driver to change well controls in a parallel run. I will keep it in draft until #7444 is merged. Would you be OK with merging it then, or would you rather it wait for OPM/opm-common#5371, or until after the 2026.10 release?
I think it's fine to get this in somewhat early, after I've created the release branches for 2026.10, as the current behaviour will be what's exposed in 2026.10. The semantics will be different in 2027.04 however, so we'll have to adapt the descriptions to the new interfaces at that point.
This branch has not been deployed
No deployments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
manual:irrelevantThis PR is a minor fix and should not appear in the manual
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
hakonhagland
added
the
manual:irrelevant
Add
get_schedule()to the Python simulatorsThe Python simulators give no access to the
Scheduleobject the simulator itself uses. ASchedulebuilt in Python is a separate object, so changing it, for example withshut_well(), has no effect on a simulator constructed from a file name. The only way to control wells from Python has been the constructor that takesDeck,EclipseState,ScheduleandSummaryConfigobjects. In a parallel run that constructor works only ifNoneis passed for theEclipseState, which is not documented anywhere. OPM/opm-python-documentation#37 documents exactly that workaround for running in parallel with mpi4py.This PR adds
get_schedule(), which returns the simulator's ownScheduleafterstep_init(). Well controls can then be changed with the filename constructor:It also rewrites the docstring of the objects constructor, which did not say what the simulator does with each of its four arguments.
This branch is based on #7444 and contains its commit. Please review that one there; this PR adds the commits after it.
Add
get_schedule()to the Python simulators (commit 1)Main::schedulePtr(): a public getter for thestd::shared_ptr<Schedule>thatMainalready holds and passes on to the vanguard. This is the object the simulation uses.PyBaseSimulator::getSchedule(), bound asget_schedule()in all three simulator classes. It hands over the existing shared pointer, so Python and the simulator share one object. With the objects constructor it returns the very object that was passed in. Calling it beforestep_init()raisesRuntimeError, asstep()does.opm.io.schedulebefore returning. The Python type forScheduleis registered by opm-common's module. Without the import, pybind11 cannot convert the return value when the script has not importedopm.ioitself, and every process fails withTypeError: Unregistered type : Opm::Schedule.Schedule, so a change must be made in the same way on every process.python/test/test_get_schedule.py, registered as the ctest casepython_get_schedule. It checks thatget_schedule()raises beforestep_init(), that a well shut through it is shut in the simulation (FOPRis zero from that report step on), and that with the objects constructor it returns the object that was passed in.Document that the Deck argument is used only for its file name (commit 2)
Deckparameter of the objects constructor: the simulator reads the DATA file again itself, so changes made to theDeckobject in Python do not reach the simulation. Commit 3 makes this more precise.Say what each objects-constructor argument is used for (commit 3)
Deck: only its file name is used, and only whenEclipseStateisNone. Otherwise the simulator takes the file name from theEclipseState.EclipseState:Noneis recommended. When given, the simulator uses it instead of building its own, and takes the name of the DATA file to read from it. In a parallel run it must beNone. (Commit 5 rewords this.)ScheduleandSummaryConfig: used as they are, so a schedule or a summary configuration built or changed in Python takes effect. Fromstep_init()on, the givenScheduleis the objectget_schedule()returns.step_init().get_schedule()as the simpler way to change well controls.EclipseState: EclipseState | None.Run the objects-constructor test from its own copy of the deck (commit 4)
test_03_objects_constructorran in the deck directory thattest_schedule.pyalso uses. With anEclipseStategiven, the simulator clears old result files, and writes its.PRTand.DBGfiles, in the directory of the deck theEclipseStatewas built from, even when--output-dirnames another one. Run concurrently, the two tests could delete each other's output. The test now copies the deck into a directory of its own.Describe a given EclipseState as used, not as unable to carry changes (commit 5)
EclipseStateis used as it is, including its grid and properties, so the docstring no longer says that everything except theScheduleandSummaryConfigis read from the DATA file again. Instead of claiming that the state cannot carry changes made in Python, it now says that the state is used as given and should be built from the same DATA file.Keep the get_schedule() docstring neutral about how to change it (commit 6)
get_schedule()docstring no longer namesshut_well()and per-report-step changes as the way to use the returnedSchedule. Those step-indexed operations belong to opm-common'sSchedulebindings and are expected to change. It now only says that changes made to the returned object take effect in the simulation.Verification
test_get_schedule.pyfirst; all three tests failed withAttributeErrorbefore commit 1. All 8python_*ctest cases pass after it, and after commit 6.python_get_scheduleleaves no result or log files in the shared deck directory; the test's own copy gets them all.get_schedule().shut_well("PROD", 3)called on every process:WOPR:PRODper report step is 20000, 20000, 20000, 0, 0, 0. Without the call it stays at 20000. The same result with a script that never importsopm.iobefore callingget_schedule(), which is what failed before the import was added.get_schedule()right afterstep_init()is shut from the first report step, on 1 and on 4 processes.Deckparsed from a copy namedOTHER.DATA, together with anEclipseStatefromSPE1CASE1.DATA, makes the simulator read and writeSPE1CASE1; withNonefor the state it reads and writesOTHER;Schedulebuilt from the deck text with the oil rate target ofPRODchanged from 20000 to 10000 givesWOPR:PROD10000 at every step, on 1 and on 4 processes;SummaryConfigbuilt from the deck text withFOPTadded makesFOPTappear in the summary file;Deck.add()does not change theEclipseStatebuilt from that deck, because it lands at the end of the deck, in theSCHEDULEsection.Follow-ups
Two further PRs build on this one:
EclipseStateobject is passed in a parallel run, which today aborts while the grid is being distributed;Schedulediffers between the MPI processes, either beforestep_init()or before a report step. Today a change made on only some of the processes is silently ignored.