Claim a Job (managed OH) - BED-9515

[StranDutton]

Summary

Ticket: BED-9515

A managed OpenHound collector can claim an available BHE job and validate its required credentials from AWS Secrets Manager before collection begins. Self-hosted scheduling remains unchanged when managed mode is disabled.

Flow

1. Find the first available job matching this collector and claim it. Use the job returned by the claim response as the authoritative job.
2. Retrieve the secret referenced by that claimed job.
3. Require a valid secret reference and a valid JSON credential object with usable values. (jusr validates the document's format, not authentication against the target)
4. Keep the validated job and credentials in memory for the managed runtime.

If retrieval or validation fails, report the claimed job as FAILED with the fixed message Credential validation failed and do not start a collection.

Reliability and credential protection

• Claim and failed-job reporting use bounded retries and request timeouts; checking for jobs has a request timeout.
• A claim conflict does not retrieve credentials or report a credential failure.
• If failed-job reporting remains unresolved, retry it before discovering another job. End-job 404 clears local state.
• Managed jobs never fall back to self-hosted collection credentials.
• Logs provide safe troubleshooting context without exposing credential values or raw provider errors.
• BHE authentication can refresh its separate API token after a 401 before retrying the request.

Current scope

Managed collection is not yet complete end to end. This PR adds the logic to claim a job and retrieve and validate its credentials, so the job details and credentials are ready for the collector to use. The scheduler currently stops before claiming work because the managed collection runtime is not connected yet. BED-9516 will supply managed start and execution.

Verification

Tests cover claim request shape, claim-to-credential-to-handoff ordering, BHE token refresh, validation failures, conflicts, retries, failure-report recovery, credential redaction, and unchanged unmanaged scheduling.

Summary by CodeRabbit

• New Features
  • Added managed collector-job scheduling, including job discovery, claiming, credential retrieval, validation, and handoff for processing.
  • AWS Secrets Manager credentials can now be read from UTF-8 binary secrets as well as string secrets.
• Bug Fixes
  • Improved handling of temporary request failures and invalid job responses, with retries for managed-job operations.
  • Sensitive provider details are excluded from managed-job errors and logs.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Essentials
• Run ID: e006687e-4e0e-4827-8f8c-a0a893ea47f8

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Essentials
• Run ID: 575b65da-c225-4758-b530-464ddf15f9a0

📥 Commits

Reviewing files that changed from the base of the PR and between 67d62ae and 2338f23.

📒 Files selected for processing (5)

• src/openhound/core/clients/aws_secrets_manager.py
• src/openhound/core/clients/bloodhound_enterprise.py
• src/openhound/scheduler/service.py
• tests/test_aws_secrets_manager.py
• tests/test_bhe_job_scheduling.py

Included review availability: This review used your included allowance. 4 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 5 reviews per hour.

---

Walkthrough

The change adds managed collector-job polling, claiming, credential validation, and runner handoff. It also adds bounded AWS client settings, binary secret parsing, and sanitized handling of managed-job request errors.

Changes

Managed Collector Scheduling

Layer / File(s)	Summary
AWS secret retrieval and parsing src/openhound/core/clients/aws_secrets_manager.py, tests/test_aws_secrets_manager.py	Internally created clients use configured timeouts and retries. Binary secrets are decoded as UTF-8. SDK logging is suppressed, and tests cover retrieval, parsing, and secret-safe errors and logs.
Managed-job API requests and retries src/openhound/core/clients/bloodhound_enterprise.py, tests/test_bhe_job_scheduling.py	Managed-job polling, claiming, and ending use retry limits and sanitized errors. Ending a job treats HTTP 404 as a no-op.
Managed job preparation and scheduler flow src/openhound/scheduler/service.py, tests/test_bhe_job_scheduling.py	Managed polling selects jobs for the configured collector, claims them, validates retrieved credentials, and hands prepared jobs to the runner. Failed credential reports remain pending for retry. Unmanaged polling remains separate.

Priority: ⬇️ Low

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Service
  participant BloodHoundEnterprise
  participant AWSSecretsManager
  participant ManagedJobRunner
  Service->>BloodHoundEnterprise: Poll queue and claim matching job
  BloodHoundEnterprise-->>Service: Return claimed job
  Service->>AWSSecretsManager: Retrieve secret for claimed job
  AWSSecretsManager-->>Service: Return decoded credentials
  Service->>ManagedJobRunner: Pass prepared job

Loading

Suggested reviewers: jplarose

Merge Risk: ⚪ Minimal · up to 2338f

Managed job claiming, credential validation and failure reporting are bounded and sanitized. Unmanaged scheduling is unchanged. No confirmed merge-blocking defect remains. It is still worth confirming how the server responds when ending an expired claim, because a persistent non-404 error would keep retrying the failure report.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 84 functions across 5 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: adding managed OpenHound job-claiming logic. The BED-9515 reference is also specific and relevant.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

_{A rabbit checks the job queue bright
Then claims the task and checks it right
A secret comes in bytes, then flows
The runner gets what validation knows
If reports must wait, they try once more
And quiet logs keep secrets off the floor}

_{Comment @coderabbitai help to get the list of available commands.}