Skip to content

[feat] Standardize exit codes - #3745

Open
vkarak wants to merge 1 commit into
reframe-hpc:developfrom
vkarak:feat/exit-codes
Open

vkarak wants to merge 1 commit into
reframe-hpc:developfrom
vkarak:feat/exit-codes

Conversation

@vkarak

@vkarak vkarak commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

ReFrame now differentiates its exit codes as follows:

  • 0: success (as before)
  • 1: any error that is not one of the following (partially as before)
  • 2: command-line misuse (partially as before - only errors directly issued by the argument parser followed this convention; now all command-line usage errors exit with 2)
  • 3: sanity failure error: at least one test failed in sanity
  • 4: performance failure error: all failed tests failed in performance

Closes #2306.

Signed-off-by: Vasileios Karakasis <vkarakasis@nvidia.com>
@vkarak vkarak added this to the ReFrame 4.11 milestone Oct 23, 2026
@vkarak
vkarak requested review from ekouts and victorusu October 23, 2026 17:15
@vkarak vkarak self-assigned this Oct 23, 2026
@codecov

codecov Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 77.61194% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.91%. Comparing base (fa567a8) to head (064e795).

Files with missing lines Patch % Lines
reframe/frontend/cli.py 78.33% 13 Missing ⚠️
reframe/__init__.py 0.00% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #3745      +/-   ##
===========================================
+ Coverage    91.90%   91.91%   +0.01%     
===========================================
  Files           62       63       +1     
  Lines        13927    13943      +16     
===========================================
+ Hits         12799    12816      +17     
+ Misses        1128     1127       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

Differentiate ReFrame errors with exit codes

1 participant