Skip to content

Use tempfile library for temporary directory handling - #14

Open
ashishsoni-ai wants to merge 1 commit into
RedHenLab:mainfrom
ashishsoni-ai:fix/use-tempfile-library
Open

ashishsoni-ai wants to merge 1 commit into
RedHenLab:mainfrom
ashishsoni-ai:fix/use-tempfile-library

Conversation

@ashishsoni-ai

Copy link
Copy Markdown

Resolves #2

Problem

Previously, rha.py and rha_quick_hack_x264_peter.py created temporary directories directly in the working directory using a homegrown format:

temp_dir = osp.abspath('anon_tmp_{}_{}'.format(osp.basename(args.inpath), osp.basename(args.facepath)))
os.makedirs(temp_dir, exist_ok=True)

and cleaned up manually via shutil.rmtree(temp_dir) in a finally block (or left uncleaned when commented out).

This caused several issues:

  1. Working directory pollution: Leftover anon_tmp_* directories would accumulate in the repository/working directory if the process crashed or was terminated.
  2. Cluster / Environment incompatibilities: Forced temporary intermediate video/audio files into the current working directory instead of respecting standard system temporary directories (such as $TMPDIR on HPC scratch disks).
  3. Collision / Race Conditions: Concurrent runs on identically named files in the same working directory shared and collided on the same directory.

Solution

  • Replaced homegrown directory creation with Python's standard library tempfile.TemporaryDirectory(prefix='anon_tmp_').
  • Leveraged context management to guarantee that temporary directories are automatically and cleanly removed under all exit paths (normal termination or exceptions).
  • Preserved existing console output compatibility (removing temporary files/folders, video saved at: ..., and TEMPDIR: ...).
  • Structured execution into main(args) behind if __name__ == '__main__': so functions can be cleanly imported and unit-tested without breaking CLI execution.
  • Added comprehensive unit tests in tests/test_rha_tempfile.py testing tempfile creation, cleanup on success, cleanup on exception, non-pollution of cwd, and isolation across concurrent executions.

Verification

Ran test suite with pytest tests/test_rha_tempfile.py -v:

tests/test_rha_tempfile.py::test_rha_tempfile_lifecycle_and_cleanup PASSED
tests/test_rha_tempfile.py::test_rha_tempfile_cleanup_on_exception PASSED
tests/test_rha_tempfile.py::test_rha_quick_hack_tempfile_cleanup PASSED
tests/test_no_cwd_pollution PASSED
tests/test_rha_video_and_audio_anonymization PASSED
tests/test_concurrent_tempfile_isolation PASSED
tests/test_cli_help PASSED
7 passed in 0.74s

Fixes RedHenLab#2.

- Replace homegrown temporary directory creation ('anon_tmp_{}_{}' with os.makedirs in cwd) with tempfile.TemporaryDirectory.
- Ensures unique, collision-free temp directories in standard system temp space (respecting TMPDIR).
- Provides automatic cleanup on exit and upon exceptions via context manager.
- Encapsulate execution into main(args) for unit testability without breaking CLI usage.
- Add unit test suite in tests/test_rha_tempfile.py covering lifecycle, exception cleanup, and concurrency.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

use tempfile library

1 participant