Remove log_prob's temporary output directory when the call returns

[atarutin]

Submission Checklist

• Run unit tests
• Declare copyright holder and open-source license: see below

Summary

Closes #867.

CmdStanModel.log_prob created its output directory with tempfile.mkdtemp(prefix=self.name, dir=_TMPDIR) and never removed it. _TMPDIR is only removed at interpreter exit, so a long-running process kept one directory per call.

The output directory is now a tempfile.TemporaryDirectory(prefix=self.name, dir=_TMPDIR) in the same with statement as the JSON inputs. Following review, it is still nested under _TMPDIR. pd.read_csv runs inside the with block, so the returned DataFrame is unchanged. The directory is removed when the call returns or raises.

Test: test_lp_removes_output_dir calls log_prob three times, once with valid parameters and once with parameters that make CmdStan fail (the RuntimeError path). It asserts that the entries of _TMPDIR are unchanged afterwards, and that the logged command wrote its output under _TMPDIR. Against develop without the fix, the test fails with three leftover bernoulli* directories in _TMPDIR.

CmdStanModel.diagnose has the same mkdtemp(..., dir=_TMPDIR) pattern. I left it alone to keep this PR to the issue. I can make the same change there, here or in a separate PR, if you want it.

What I ran. All runs were in a Linux container with Python 3.12.14 and CmdStan 2.40.0, on develop at edf9fe3:

• isort --check-only, black --check, flake8, pylint --rcfile=.pylintrc, and mypy on cmdstanpy test: all clean.
• pytest test/test_log_prob.py test/test_model.py: 34 passed.
• pytest test, excluding test_install_cmdstan.py and test_cxx_installation.py (they install toolchains) and the test_model.py and test_compilation.py modules (the first was run above): 407 passed, 3 skipped, 6 failed. None of the failures involve log_prob:
  • test_args_bad, test_bernoulli_bad, test_save_csv and test_validate_dir fail because I ran as root, where read-only directories are writable. They fail the same way on unmodified develop as root, and pass as a non-root user.
  • test_sample.py::test_multi_proc_1 and test_multi_proc_2 fail on unmodified develop too, when logistic.stan is already compiled: the expected "Chain [1] done processing" record is not captured. They passed in an earlier run in which the test compiled the model.

AI disclosure, per the Stan AI Contribution Policy: this change was prepared with Claude Code (Anthropic).

Copyright and Licensing

Please list the copyright holder for the work you are submitting (this will be you or your assignee, such as a university or company):

Aleksandr Tarutin

By submitting this pull request, the copyright holder is agreeing to license the submitted work under the following licenses:

• Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)

🤖 Generated with Claude Code

[WardBrian]

Thanks. One suggestion to make a comment more succinct.

Additionally, would you mind removing the Co-authored by metadata from the commit? Regardless of any tools used, you should be the official author of the change

[WardBrian]

Suggested change

# Not under _TMPDIR, which is only removed at interpreter exit;

[amas0]

One thought: is there any reason we can't have this temp dir be a subdir of _TMPDIR? It would still be removed on exit from the context manager, but nests it hierarchically rather than creating a separate dir.

[WardBrian]

I think that would be a good idea

[atarutin]

Done in b228372: the directory is now TemporaryDirectory(prefix=self.name, dir=_TMPDIR), the comment is shortened as suggested, and the test checks that _TMPDIR's entries are unchanged after three calls on the success and failure paths.

[atarutin]

Squashed to one commit with the Co-authored-by trailer removed; the tree is unchanged.

[atarutin]

Looks good.

[amas0]

Could you mark this (and your other PR) as ready to review if there are no more intended changes?

[atarutin]

Could you mark this (and your other PR) as ready to review if there are no more intended changes?

Done. Was waiting for CI checks to finish. :)