feat: monthly active users per platform, and Discord commands to feed and run jobs

README.md

@@ -10,11 +10,12 @@ share for translations. One package, `session_ops`, deployed to one self-hosted
 | --- | --- | --- |
 | `zendesk-digest` | Weekday Discord digest of the Zendesk tickets awaiting a reply, after closing positive app-store reviews | [zendesk-digest](docs/jobs/zendesk-digest.md) |
 | `zendesk-relay` | Drafts and sends Zendesk replies from `claude:` private notes | [zendesk-relay](docs/jobs/zendesk-relay.md) |
+| `session-ops-discord` | `/run` a job and `/mau-upload` an export from Discord | [session-ops-discord](docs/jobs/session-ops-discord.md) |
 | `github-prs-digest` | Weekday Discord digest of open pull requests from outside contributors | [github-prs-digest](docs/jobs/github-prs-digest.md) |
 | `crowdin-duplicates` | Crowdin string slots holding more than one translation, as they open and close | [crowdin-duplicates](docs/jobs/crowdin-duplicates.md) |
 | `crowdin-sync` | Weekday: Crowdin translations into iOS, Android and the localization module, and its submodule bumped in each client | [crowdin-sync](docs/jobs/crowdin-sync.md) |
 | `snode-list` | Weekday: the fallback service node list from the seed nodes into dynamic-assets, Desktop and iOS | [snode-list](docs/jobs/snode-list.md) |
-| `release-stats` | On demand: download counts of the latest releases | [release-stats](docs/jobs/release-stats.md) |
+| `mau` | Monthly: active users per platform and in total, from the store exports dropped in its inbox | [mau](docs/jobs/mau.md) |
 | `session-ops-silence` | Discord alerts for a job that failed, or stopped running | [session-ops-silence](docs/jobs/session-ops-silence.md) |
 | `token-expiry` | Discord alerts 14 days, 7 days and 24 hours before a token expires | [token-expiry](docs/jobs/token-expiry.md) |
 

deploy/README.md

@@ -7,8 +7,10 @@ does the work: accounts, venv, env files, units, timers, and migrating an older
 | Unit | What it is |
 | --- | --- |
 | `session-ops@<job>.timer` → `.service` | One per job; a generated drop-in sets its account, env files and schedule. |
+| `session-ops@<job>.path` → `.service` | For a job with a `watch`: starts it when a matching file lands in its state directory. |
 | `session-ops-queue.timer` → `.service` | Starts the jobs in `jobs.toml`'s `[queue]`, which then run one at a time in its order. |
 | `zendesk-relay.service` | Always on, `127.0.0.1:8080`: Zendesk's `claude:` note webhooks. |
+| `session-ops-discord.service` | Always on, `127.0.0.1:8081`: the `/run` and `/mau-upload` slash commands. See [Discord commands](#discord-commands). |
 | `session-ops-alert@.service` | Every unit's `OnFailure=` backstop; see [session-ops-silence](../docs/jobs/session-ops-silence.md). |
 
 `session-ops list` shows the jobs; `session-ops run <job> [--dry-run] [-- job arguments]`
@@ -52,6 +54,37 @@ From a host that ran the digests out of `/opt/zendesk`, `install.sh` copies thei
 files and state over. Remove `/opt/zendesk`, `/etc/zendesk` and `/var/lib/zendesk`
 (and the `github-prs` equivalents) once both digests have run from the new units.
 
+## Discord commands
+
+What `/run` and `/mau-upload` do, and who may run them:
+[session-ops-discord](../docs/jobs/session-ops-discord.md). To set them up:
+
+1. In the Developer Portal, create an application. Put its public key, its application
+   id, the server's id and who may run the commands in `/etc/session-ops/discord.env`,
+   then run `install.sh` again.
+2. Invite it with the `applications.commands` scope only:
+   `https://discord.com/oauth2/authorize?client_id=<app id>&scope=applications.commands`.
+3. Copy the `location = /discord/interactions` block from
+   [`nginx-webhooks.conf`](nginx-webhooks.conf) into the live
+   `/etc/nginx/sites-available/webhooks.session.codes`, which certbot owns, then
+   `nginx -t && systemctl reload nginx`. Until then the route is a 404 and Discord will
+   not save the URL.
+4. Set its Interactions Endpoint URL to `https://webhooks.session.codes/discord/interactions`.
+   Discord checks it with a signed PING before saving.
+5. Register the commands, with the bot token from the Bot page, typed in rather than kept:
+
+   ```bash
+   read -rs DISCORD_BOT_TOKEN && export DISCORD_BOT_TOKEN
+   set -a && . /etc/session-ops/discord.env && set +a
+   /opt/session-ops/.venv/bin/session-ops discord-register
+   ```
+
+6. The commands start hidden from everyone but server administrators. In Server
+   Settings → Integrations, allow them for the role or channel that runs jobs. Seeing
+   them is not running them: the relay's allowlist still decides.
+
+Register again after changing which jobs have `discord = true`.
+
 ## Secrets
 
 Each `/etc/session-ops/<name>.env` has a commented `<name>.env.example` beside it,
@@ -69,6 +102,8 @@ systemctl start session-ops@<job>.service && journalctl -fu session-ops@<job>
 systemctl start session-ops-alert@test.service          # posts to the alerts channel
 curl -sS -o /dev/null -w '%{http_code}\n' -X POST 127.0.0.1:8080/zendesk/notes \
   -H 'Content-Type: application/json' -d '{"ticket_id":"1"}'   # expect 401
+curl -sS -o /dev/null -w '%{http_code}\n' -X POST 127.0.0.1:8081/discord/interactions -d '{}'   # expect 401
+runuser -u opsbot -- systemctl --no-ask-password start session-ops@token-expiry.service   # expect Access denied
 ```
 
 ## Rehearsing on a spare host
@@ -85,7 +120,10 @@ To end it, stop the timers, then close the rehearsal pull requests:
 for link in /etc/systemd/system/timers.target.wants/session-ops@*.timer; do
   [ -L "$link" ] && systemctl disable --now "${link##*/}"
 done
-systemctl disable --now session-ops-queue.timer zendesk-relay.service
+for link in /etc/systemd/system/paths.target.wants/session-ops@*.path; do
+  [ -L "$link" ] && systemctl disable --now "${link##*/}"
+done
+systemctl disable --now session-ops-queue.timer zendesk-relay.service session-ops-discord.service
 for repo in session-android session-ios session-localization session-desktop-dynamic-assets \
     session-desktop session-app session-website session-appium session-playwright; do
   gh pr list -R "session-foundation/$repo" --state open --json number,headRefName \

deploy/env/alerts.env.example

@@ -1,3 +1,3 @@
-# /etc/session-ops/alerts.env: the OnFailure backstop, the silence checker and
-# release-stats. Empty disables the first two, and install.sh warns until it is set.
+# /etc/session-ops/alerts.env: the OnFailure backstop and the silence checker.
+# Empty disables both, and install.sh warns until it is set.
 ALERT_DISCORD_WEBHOOK_URL=

deploy/env/discord.env.example

@@ -0,0 +1,14 @@
+# /etc/session-ops/discord.env: the Discord app behind /run and /mau-upload.
+# A `#` starts a comment only as a line's first character. Restart session-ops-discord
+# after an edit: it reads this file at start.
+
+# General Information in the Developer Portal. Unset refuses every interaction.
+DISCORD_PUBLIC_KEY=
+# The one server the commands are registered on and answered from.
+DISCORD_GUILD_ID=
+# Comma-separated; either list grants. Both empty refuses everybody.
+ALLOWED_USER_IDS=
+ALLOWED_ROLE_IDS=
+# For `session-ops discord-register`. Its DISCORD_BOT_TOKEN is typed in when registering,
+# not kept here: the relay never needs it.
+DISCORD_APP_ID=

deploy/env/mau.env.example

@@ -0,0 +1,7 @@
+# /etc/session-ops/mau.env: the monthly active users post.
+# Where the figures, the reminders and the job's own failures are posted.
+MAU_DISCORD_WEBHOOK_URL=
+# An App Store Connect team API key with the Sales and Reports role, for Apple's opt-in
+# rate: its issuer and key IDs here, its .p8 at /etc/session-ops/asc-key.p8 (mode 600).
+ASC_ISSUER_ID=
+ASC_KEY_ID=

deploy/install.sh

@@ -37,7 +37,7 @@ account() {
     id -u "$1" >/dev/null 2>&1 ||
         useradd --system --no-create-home --home /nonexistent --shell /usr/sbin/nologin "$1"
 }
-for user in ghdigest crowdin publisher sessionops; do account "$user"; done
+for user in ghdigest crowdin publisher sessionops opsbot; do account "$user"; done
 # The Claude Code CLI keeps its binary, login and cache under this account's $HOME.
 if ! id -u zendesk >/dev/null 2>&1; then
     useradd --system --home /home/zendesk --shell /usr/sbin/nologin zendesk
@@ -68,15 +68,16 @@ move /etc/github-prs/env "$ETC/github-prs.env" 600 root
 move "$ETC/env" "$ETC/alerts.env" 600 root
 # systemd reads EnvironmentFile= as root before dropping privileges, so these need
 # no group: a job's account cannot read another job's secrets.
-for name in zendesk github-prs crowdin publish alerts; do
+for name in zendesk github-prs crowdin publish alerts mau discord; do
     [ -e "$ETC/$name.env" ] || install -m 600 /dev/null "$ETC/$name.env"
 done
 # What each file takes, commented; the env files stay empty until filled, since a job
 # is enabled once its env files have content.
 install -m 644 "$ROOT"/deploy/env/*.example "$ETC/"
-# The publishing units load this as a credential, and a missing file would stop them
-# starting; left empty, their runs exit naming the key.
+# The publishing units and mau load these as credentials, and a missing file would stop
+# them starting; left empty, their runs fail naming the key.
 [ -e "$ETC/github-app.pem" ] || install -m 600 /dev/null "$ETC/github-app.pem"
+[ -e "$ETC/asc-key.p8" ] || install -m 600 /dev/null "$ETC/asc-key.p8"
 
 install -d -m 755 "$STATE"
 install -d -o ghdigest -g ghdigest "$STATE/github-prs-digest"
@@ -92,6 +93,17 @@ if [ -e "$NEW_HOUSE" ] && grep -qx "ZENDESK_HOUSE_ANSWERS=$OLD_HOUSE" "$ETC/zend
     echo "pointed ZENDESK_HOUSE_ANSWERS in $ETC/zendesk.env at $NEW_HOUSE"
 fi
 
+# A watched job's inbox: root drops files in, and the job's account moves them out.
+# mau's also takes /mau-upload's, from the Discord relay's account.
+"$OPS" list --watched | while read -r job user dir; do
+    install -d -o "$user" -g "$user" -m 711 "$STATE/$job"
+    if [ "$job" = mau ]; then
+        install -d -o "$user" -g opsbot -m 770 "$dir"
+    else
+        install -d -o "$user" -g "$user" -m 700 "$dir"
+    fi
+done
+
 install -m 644 "$ROOT/deploy/session-ops.tmpfiles" /etc/tmpfiles.d/session-ops.conf
 systemd-tmpfiles --create session-ops.conf
 
@@ -104,14 +116,21 @@ rm -f "$UNITS/zendesk-alert@.service" "$UNITS/github-prs-alert@.service"
 systemctl disable --now crowdin-relay.service 2>/dev/null || true
 rm -f "$UNITS/crowdin-relay.service"
 
-install -m 644 "$ROOT"/deploy/*.service "$ROOT"/deploy/*.timer "$UNITS/"
+install -m 644 "$ROOT"/deploy/*.service "$ROOT"/deploy/*.timer "$ROOT"/deploy/*.path "$UNITS/"
 # Only the generated files go, so a drop-in added by hand survives.
 rm -f "$UNITS"/session-ops@*.service.d/job.conf "$UNITS"/session-ops@*.timer.d/schedule.conf \
-    "$UNITS"/session-ops-queue.timer.d/schedule.conf
+    "$UNITS"/session-ops@*.path.d/watch.conf "$UNITS"/session-ops-queue.timer.d/schedule.conf
 "$OPS" units --out "$UNITS" >/dev/null
+# polkit reloads its rules when this changes.
+if [ -d /etc/polkit-1/rules.d ]; then
+    "$OPS" polkit --out /etc/polkit-1/rules.d/50-session-ops-discord.rules >/dev/null
+else
+    echo "polkit is missing, so /run in Discord can start no job" >&2
+fi
 
 READY=$("$OPS" list --ready)
 QUEUED=$("$OPS" list --queued)
+WATCHED=$("$OPS" list --watched | cut -d' ' -f1)
 listed() { printf '%s\n' $2 | grep -qxF "$1"; }
 # What the queue's timer starts: its ready jobs, rebuilt from scratch each install.
 WANTS="$UNITS/session-ops-queue.service.wants"
@@ -142,6 +161,21 @@ for job in $READY; do
         echo "enabled session-ops@$job.timer"
     fi
 done
+for link in "$UNITS"/paths.target.wants/session-ops@*.path; do
+    [ -L "$link" ] || continue
+    job=${link##*/session-ops@}
+    job=${job%.path}
+    if ! listed "$job" "$READY" || ! listed "$job" "$WATCHED"; then
+        systemctl disable --now "session-ops@$job.path" >/dev/null
+        echo "disabled session-ops@$job.path (no longer a ready job with a watch)"
+    fi
+done
+for job in $WATCHED; do
+    if listed "$job" "$READY"; then
+        systemctl enable --now "session-ops@$job.path" >/dev/null
+        echo "enabled session-ops@$job.path"
+    fi
+done
 if [ -d "$WANTS" ]; then
     systemctl enable --now session-ops-queue.timer >/dev/null
     echo "enabled session-ops-queue.timer"
@@ -151,16 +185,21 @@ fi
 for job in $("$OPS" list --not-ready); do
     echo "not enabled: session-ops@$job (its env file is empty)"
 done
-if [ -s "$ETC/zendesk.env" ]; then
-    systemctl enable zendesk-relay.service >/dev/null
-    # A relay that hit its start limit refuses `start` until the limit is cleared.
-    systemctl reset-failed zendesk-relay.service 2>/dev/null || true
-    systemctl try-restart zendesk-relay.service
-    systemctl start zendesk-relay.service
-    echo "running zendesk-relay.service"
-else
-    echo "not enabled: zendesk-relay.service ($ETC/zendesk.env is empty)"
-fi
+# An always-on service, enabled once its env file has content.
+relay() {
+    if [ -s "$ETC/$2.env" ]; then
+        systemctl enable "$1.service" >/dev/null
+        # A relay that hit its start limit refuses `start` until the limit is cleared.
+        systemctl reset-failed "$1.service" 2>/dev/null || true
+        systemctl try-restart "$1.service"
+        systemctl start "$1.service"
+        echo "running $1.service"
+    else
+        echo "not enabled: $1.service ($ETC/$2.env is empty)"
+    fi
+}
+relay zendesk-relay zendesk
+relay session-ops-discord discord
 
 if [ ! -s "$ETC/alerts.env" ]; then
     cat >&2 <<EOF

deploy/nginx-webhooks.conf

@@ -1,4 +1,4 @@
-# TLS termination for the Zendesk note webhook endpoint.
+# TLS termination for the Zendesk note webhook and the Discord interactions endpoint.
 #
 # nginx rather than anything of its own, because the host already runs nginx with
 # certbot for other sites. A second web server would fight it for :80 and :443 and
@@ -48,9 +48,10 @@ server {
     listen [::]:80;
     server_name webhooks.session.codes;
 
-    # Webhook bodies are small; anything larger is not from Zendesk. This is a
-    # public endpoint, so the cap matters — nothing downstream limits a body, and the
-    # relay has to read the whole thing to verify its signature.
+    # Webhook bodies are small, a Discord upload being a link to the file; anything
+    # larger is from neither Zendesk nor Discord. This is a public endpoint, so the cap
+    # matters — nothing downstream limits a body, and each relay has to read the whole
+    # thing to verify its signature.
     client_max_body_size 256k;
 
     server_tokens off;
@@ -61,9 +62,8 @@ server {
     # matching uvicorn's --no-access-log in zendesk-relay.service.
     access_log off;
 
-    # Zendesk is the only thing that needs to reach this, and it only ever POSTs to
-    # one path. The path is vendor-namespaced so this host and certificate can carry
-    # other integrations later without a second of either.
+    # Zendesk and Discord each POST to one path, vendor-namespaced so this host and
+    # certificate carry both.
     #
     # A `claude:` private note on a ticket. Exact path, and nothing here may alter
     # the body: the HMAC signature covers the raw bytes, so any rewriting would turn
@@ -80,6 +80,18 @@ server {
         proxy_connect_timeout 5s;
     }
 
+    # Discord's slash commands. Same constraint as above: the Ed25519 signature covers
+    # the raw body. Discord waits three seconds for an answer.
+    location = /discord/interactions {
+        proxy_pass http://127.0.0.1:8081;
+        proxy_http_version 1.1;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+        proxy_read_timeout 10s;
+        proxy_connect_timeout 5s;
+    }
+
     location = /healthz {
         proxy_pass http://127.0.0.1:8080;
         proxy_set_header Host $host;

deploy/session-ops-discord.service

@@ -0,0 +1,40 @@
+[Unit]
+Description=Discord slash commands for session-ops jobs
+Documentation=https://github.com/session-foundation/session-shared-scripts
+After=network-online.target
+Wants=network-online.target
+OnFailure=session-ops-alert@%n.service
+StartLimitIntervalSec=5min
+StartLimitBurst=5
+
+[Service]
+Type=exec
+User=opsbot
+Group=opsbot
+EnvironmentFile=/etc/session-ops/discord.env
+# Loopback only: nginx terminates TLS, and the signature check should not be the only barrier.
+ExecStart=/opt/session-ops/.venv/bin/uvicorn session_ops.ops.discord_relay:app --host 127.0.0.1 --port 8081 \
+    --no-access-log --proxy-headers --forwarded-allow-ips 127.0.0.1
+Restart=always
+RestartSec=2
+
+NoNewPrivileges=yes
+PrivateTmp=yes
+PrivateDevices=yes
+ProtectSystem=strict
+ProtectHome=yes
+ProtectKernelTunables=yes
+ProtectKernelModules=yes
+ProtectControlGroups=yes
+# AF_UNIX for systemctl, which asks systemd and polkit over D-Bus.
+RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
+RestrictNamespaces=yes
+LockPersonality=yes
+MemoryDenyWriteExecute=yes
+SystemCallFilter=@system-service
+SystemCallErrorNumber=EPERM
+# `-`: a host without the inbox yet still answers /run.
+ReadWritePaths=-/var/lib/session-ops/mau/inbox
+
+[Install]
+WantedBy=multi-user.target

deploy/session-ops-queue.timer

@@ -5,7 +5,7 @@ Documentation=https://github.com/session-foundation/session-shared-scripts
 
 [Timer]
 Persistent=yes
-RandomizedDelaySec=2min
+# No random delay: /run in Discord refuses a job its timeout would carry past the next run.
 
 [Install]
 WantedBy=timers.target

deploy/session-ops@.path

@@ -0,0 +1,12 @@
+# Starts a job when a file matching its watch lands. Its drop-in, written by
+# `session-ops units`, sets PathExistsGlob= from jobs.toml's watch.
+[Unit]
+Description=Inbox watch for session-ops job %i
+Documentation=https://github.com/session-foundation/session-shared-scripts
+
+[Path]
+# Re-triggers for as long as a file matches, so the job must move each one out.
+Unit=session-ops@%i.service
+
+[Install]
+WantedBy=paths.target

docs/jobs/github-prs-digest.md

@@ -49,9 +49,8 @@ cache rather than something to back up.
 
 ### Late, not lost
 
-Two runs can be further apart than 72 hours: April's DST weekend is 73, the timer's
-`RandomizedDelaySec` adds up to two minutes, and a host that was down runs once when it
-comes back. So the state also keeps `covered_until`, the time the last run whose every
+Two runs can be further apart than 72 hours: April's DST weekend is 73, and a host that
+was down runs once when it comes back. So the state also keeps `covered_until`, the time the last run whose every
 message Discord accepted *started* its search, and each run reaches back to whichever is
 earlier, that or 72 hours ago. A run whose post fails partway, or that fails before
 posting, never moves it forward; a dry run writes nothing. The header then names the