Skip to content

fix: use the hub mirror when the CrowdSec hub CDN is unreachable - #1798

Merged
Wikid82 merged 16 commits into
developmentfrom
fix/crowdsec-hub-mirror-fallback
Oct 31, 2026
Merged

Wikid82 merged 16 commits into
developmentfrom
fix/crowdsec-hub-mirror-fallback

Conversation

@Wikid82

@Wikid82 Wikid82 commented Oct 31, 2026 •

Copy link
Copy Markdown
Owner

Summary

Addresses #1739 and #1738.

Makes CrowdSec's hub requests fall back to the GitHub mirror when the hub CDN is unreachable, and tells the presets list when the hub was not used.

  • Presets list and downloads: each hub address gets its own share of the time budget, so a hanging first address no longer uses up the whole pull budget. The presets list is bounded. If every address fails at transport level the answer is 503 hub_unreachable.
  • cscli (hub update, item installs, upgrades): a failed or timed-out attempt is retried once through a temporary copy of the cscli config that points at the mirror. The copy is mode 0600, always removed, and never appears in the file list, export or import. A configuration that already sets its own hub address is left alone (no retry). After the mirror answers, cscli goes to the mirror first for up to 10 minutes.
  • Container start: the entrypoint and the item-install script use the same retry through a shared helper (.docker/cscli-hub-mirror.sh), with bounded time for each step.
  • Presets page: GET /admin/crowdsec/presets gains hub_offline. When the hub list was not used the page shows one quiet note. Hub-only presets that are not downloaded are shown greyed out and are skipped by the auto-select.
  • The always-404 <mirror>/api/index.json address is no longer tried.

Known limits (documented): the 10-minute mirror-first window applies to cscli only, so during an outage each presets list or download request spends a few extra seconds on the hanging address first. On a very slow link, hub update and hub upgrade at start split their time between the main address and the mirror.

Testing

  • Backend coverage 92.5% (gate 87%), frontend lines 91.85% (gate 85%), patch coverage 98.7% (measured before the last few test-only and shell fixes; CI repeats them).
  • Shell test (scripts/cscli_hub_mirror_test.sh, 118 checks) passes under bash, dash and busybox sh; it runs in the new cscli-hub-mirror-tests job in quality-checks.yml.
  • Targeted Playwright: six CrowdSec specs on security-tests (107 passed), whitelist on firefox (12 passed).
  • Container gate with the hub CDN dropped or refused using iptables, in throwaway containers: hung CDN, refused CDN, startup with a hung CDN (three starts, mirror retry fires), both addresses down (bounded, hub_offline true, 503 hub_unreachable). Details in docs/reports/qa_report.md.
  • Not run in the container: the F6 case where a conflicting preset apply during a hub restore waits instead of answering busy; it matches existing behaviour and is tracked in CrowdSec: a preset apply during an in-flight hub restore waits instead of answering busy #1796.

Follow-ups filed

#1796 (apply during an in-flight restore waits), #1797 (export warns per symlink).

@safedep

safedep Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

SafeDep Report Summary

Green Malicious Packages Badge Green Vulnerable Packages Badge Green Risky License Badge

No dependency changes detected. Nothing to scan.

View complete scan results →

This report is generated by SafeDep GitHub App

@github-advanced-security

Copy link
Copy Markdown
Contributor

You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool.

What Enabling Code Scanning Means:

  • The 'Security' tab will display more code scanning analysis results (e.g., for the default branch).
  • Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results.
  • You will be able to see the analysis results for the pull request's branch on this overview once the scans have completed and the checks have passed.

For more information about GitHub Code Scanning, check out the documentation.

@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

✅ Supply Chain Verification Results

✅ PASSED

📦 SBOM Summary

  • Components: 1875

🔍 Vulnerability Scan

Severity Count
🔴 Critical 0
🟠 High 0
🟡 Medium 1
🟢 Low 0
Total 1

📎 Artifacts

  • SBOM (CycloneDX JSON) and Grype results available in workflow artifacts

Generated by Supply Chain Verification workflow • View Details

@codecov

codecov Bot commented Oct 31, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.66310% with 5 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
backend/internal/crowdsec/hub_mirror.go 98.42% 3 Missing ⚠️
backend/internal/crowdsec/cscli_run.go 98.43% 1 Missing ⚠️
frontend/src/pages/CrowdSecConfig.tsx 95.83% 0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

@Wikid82
Wikid82 merged commit 7989a84 into development Oct 31, 2026
57 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants