Add optional FlowView Query Service backend for parallel queries

[TheWitness]

Summary

Adds an optional FlowView Query Service — a small long-running Go service
(service/flowview-query) that runs FlowView's parallel (map-reduce) netflow
queries in-process — and makes it selectable from the FlowView settings page
as the Parallel Query Backend. The default stays the Legacy PHP Runner, so
behaviour is unchanged until an administrator opts in.

Why

Today parallel_database_query_run() forks flowview_runner.php per query, and
the parent runner forks another PHP process per shard, then both the caller
and parent poll the coordination tables with usleep. For small or cached
queries the useful work is a single reduce, but PHP interpreter startup +
process bookkeeping + table polling add fixed, often multi-second, latency.

What this does

• New setting flowview_parallel_backend (FlowView → Parallel Queries):
  Legacy PHP Runner (default) or FlowView Query Service, plus
  flowview_query_service_url / flowview_query_service_portfile for endpoint
  discovery.
• When the service backend is selected, parallel_database_query_run() replaces
  its exec_background() launch with a single POST /run {query_ids}. The
  service reads the same parallel_database_query / _shard rows, runs the
  identical cache-aware map-reduce across a goroutine worker pool capped by
  flowview_parallel_threads (routing map reads through MaxScale when
  flowview_use_maxscale is on), and writes results + status = complete
  back — so the existing result retrieval (and multi-request UNION) is unchanged.
• Automatic fallback: if the service can't be reached, PHP logs a warning and
  runs the legacy runner, so enabling it can never strand a query.
• Shared cache: the service uses the same
  parallel_database_query_shard_cache table, so a partition cached by either
  backend is reused by the other (a cold partition is read once).
• Scheduler goroutine performs the cache/partition maintenance PHP otherwise
  does inline (TTL expiry, aged-out partitions, deleted query/filter cache).

Service (service/flowview-query)

Pure Go + go-sql-driver/mysql. Files: main.go, types.go, config.go,
query.go, scheduler.go, server.go, demo.go, plus config.sample.json,
a systemd unit, and a README.md with the full analysis, compilation &
install steps (incl. cross-compile), the DB users/GRANTs for the Cacti control
DB and FlowView data DB, MaxScale notes, and the endpoint reference.

Endpoints: POST /run, GET /health, POST /maintenance,
POST /cache/invalidate. It binds localhost on an available port and writes
it to a port file for discovery.

Testing

• go vet + gofmt clean; go build succeeds.
• ./flowview-query -selftest runs the full map-reduce/worker-pool/partition-cache
  path in memory (no DB) and verifies the HTTP listener: cold run maps 4
  partitions in parallel; warm run serves all 4 from cache (~100× faster);
  GET /health → 200.
• php -l clean on setup.php and includes/functions.php.

Notes / scope

• The service compiles and is production-faithful, but I could not exercise the
  DB-backed path here (no MariaDB + flow data in this environment); the
  in-memory self-test exercises the concurrency/cache/reduce mechanics.
• Honors flowview_parallel_threads, flowview_parallel_runlimit,
  flowview_parallel_time_to_live, and flowview_use_maxscale by reading the
  live Cacti settings.

[Copilot]

🔵 Needs a closer look

The untested database-backed path has unresolved fallback, concurrency, security, and deployment issues.

10 open findings

• Reset failed query state before falling back to the legacy runner · New
• Reject non-loopback listen addresses · New
• Authenticate state-changing endpoints · New
• Allow configured remote database endpoints · New
• Re-select the read pool when MaxScale settings change · New
• Make self-tests exercise the production map-reduce path · New
• Use a shared concurrency limiter across all requests · New
• Normalize time values to SQL datetime strings · New
• Retain query metadata when intermediary table cleanup fails · New
• Regenerate the gettext translation template · New

What changed in this PR

Adds an optional Go-based backend for FlowView parallel queries while retaining the legacy PHP runner as the default.

Changes:

• Adds the query service, scheduler, HTTP API, configuration, and systemd deployment assets.
• Integrates backend selection, endpoint discovery, dispatch, and legacy fallback into PHP.
• Documents installation, database permissions, operation, and dependencies.

File	Description
setup.php	Adds query-backend settings.
includes/​functions.php	Dispatches queries to the service with fallback.
service/​flowview-query/​main.go	Wires service lifecycle and HTTP listener.
service/​flowview-query/​config.go	Loads database and runtime configuration.
service/​flowview-query/​types.go	Defines API and query data structures.
service/​flowview-query/​query.go	Implements map-reduce execution and caching.
service/​flowview-query/​scheduler.go	Performs query and cache maintenance.
service/​flowview-query/​server.go	Implements HTTP endpoints.
service/​flowview-query/​demo.go	Provides the in-memory self-test demonstration.
service/​flowview-query/​config.sample.json	Supplies example service configuration.
service/​flowview-query/​flowview-query.service	Adds the systemd unit.
service/​flowview-query/​README.md	Documents architecture and deployment.
service/​flowview-query/​go.mod	Defines the Go module and dependency.
service/​flowview-query/​go.sum	Records dependency checksums.
service/​flowview-query/​.gitignore	Excludes Go build outputs.
CHANGELOG.md	Records the new backend feature.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.