Skip to content

feat: monthly active users per platform, and Discord commands to feed and run jobs - #71

Draft
Bilb wants to merge 15 commits into
mainfrom
feat/platform-stats
Draft

Bilb wants to merge 15 commits into
mainfrom
feat/platform-stats

Conversation

@Bilb

@Bilb Bilb commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

A monthly Discord post of active users per platform, and two Discord slash commands: /mau-upload to hand that job its store exports, and /run to start a job now. Includes #72.

mau: monthly active users

Sources

  • Android, Play and iOS: neither store has an API for active users. Someone exports Play's MAU report and App Store Connect's Active in Last 30 Days (both daily CSVs) and hands them to /mau-upload. The job takes each month-end value.
  • iOS counts only devices sharing analytics, so it is divided by the month's opt-in rate from the App Store Connect API's App Opt In report. The key is Sales and Reports, loaded as a systemd credential. The post names the report, the rate and its counts.
  • iOS does not use the API's App Sessions report: summing its rows counts a device once per app version, OS and territory it used in the month, 5% to 20% above App Analytics.
  • Android outside Play: one line, made of the latest stable release's GitHub APK downloads, plus an F-Droid estimate of 10% of every other figure, since F-Droid publishes no counts.
  • Desktop: the latest stable release's downloads per platform. Linux is .deb, .AppImage, .rpm, .freebsd and Flathub installs since the release day; macOS is .dmg and .zip; Windows is .exe. Desktop has no active-user count.
  • release-stats is removed: the post is now the one place Desktop downloads are counted.

Flow

  • Exports land in /var/lib/session-ops/mau/inbox/, from /mau-upload or rsync. A session-ops@mau.path unit starts the job on arrival.
  • Each export merges into a daily history.json. Changes to figures already in the history are kept until the month posts, which lists those for the two month ends it compares.
  • A month posts once both stores have a final figure for its last day. Apple's fractional "still counting" value is not final. The timer on the 10th at 11:00 Melbourne posts a reminder naming whatever is missing.
  • A file the job cannot read goes to rejected/. Any failure before the inbox is filed moves its exports to rejected/ as well, so the path unit never loops into its start limit.

watch in jobs.toml

  • A glob under the job's state directory. session-ops units writes the .path drop-in, and install.sh creates the inbox and enables the unit for ready jobs.
  • The job must move matching files out.

Discord commands

Commands

  • /run job:<job> starts session-ops@<job>.service. The choices come from the jobs jobs.toml marks discord = true: crowdin-sync and crowdin-duplicates for now. It is refused while that job or any queued job is running or waiting to, and when the queue timer's next run (from systemctl list-timers --output=json) is sooner than now plus the job's timeout, since systemd kills the job by then. That keeps the queue's no-overlap rule both ways. The check and the start run under one lock, so two /run a moment apart cannot both pass. The queue timer no longer has RandomizedDelaySec, so its next run is the scheduled time. The confirmation naming who started it is posted in the channel; refusals are ephemeral.
  • /mau-upload file:<csv> downloads the attachment from Discord's CDN and checks it with mau.parse_export, which takes either store's export and names its platform in the reply. Only then is it renamed into mau/inbox/ (written as a dotfile first, which the path unit's glob skips). A file mau would reject is refused in Discord and never lands. The monthly reminder points at it alone.

What the app sees
HTTP interactions only, applications.commands scope, no gateway, so Discord sends nothing but invocations of its own commands. Commands are registered with default_member_permissions: "0", hidden from everyone but server administrators until Server Settings → Integrations grants them to a role.

Gates

  • Ed25519 signature within a 5-minute window, 60 s of forward skew.
  • DISCORD_GUILD_ID match (DMs refused).
  • ALLOWED_USER_IDS / ALLOWED_ROLE_IDS; both empty refuses everybody.
  • The relay runs as opsbot. session-ops polkit generates a rule from jobs.toml allowing that account verb == "start" on exactly the discord = true units. The mau inbox (sessionops:opsbot 0770) is its only writable path.

Deploy

  • deploy/session-ops-discord.service on 127.0.0.1:8081, hardened like zendesk-relay, plus AF_UNIX for systemctl's D-Bus calls.
  • install.sh: the opsbot account, the group on mau's inbox (other watched inboxes stay 0700), the polkit rule, and starting the service once discord.env has content. The two relays now share one enable/restart helper.
  • nginx-webhooks.conf: a location = /discord/interactions block. certbot owns the live file, so it needs adding by hand.
  • Setup steps in deploy/README.md under "Discord commands". The bot token is only for session-ops discord-register and is typed in at registration, never stored in discord.env.

Not yet done

  • polkit is not installed on the host (no /etc/polkit-1). Until apt install polkitd (Debian 12 ships 122, which reads the JS rules.d rule), every /run gets "Access denied".
  • Nothing has run against a real Discord app or the host. After deploying:
    • An unsigned POST gets 401.
    • runuser -u opsbot -- systemctl --no-ask-password start session-ops@token-expiry.service is denied.
    • /run crowdin-duplicates starts the job and its post appears.
    • /mau-upload with a real export leads to a mau run.

Deploy

  • /etc/session-ops/mau.env: MAU_DISCORD_WEBHOOK_URL, ASC_ISSUER_ID, ASC_KEY_ID.
  • /etc/session-ops/asc-key.p8, mode 600. install.sh creates it empty, and an empty one fails the run naming it.
  • /etc/session-ops/discord.env and the nginx block for the commands (see above), then session-ops discord-register once.

Tests

uv run --locked python -m unittest discover -s tests -t . (860 OK, 1 skipped), ruff check . clean.

Bilb added 5 commits October 7, 2026 11:37
release-stats also writes desktop-platform-totals.csv: Linux (GitHub
.deb, .AppImage, .rpm, .freebsd plus Flathub installs since the release
day), macOS (.dmg, .zip) and Windows (.exe) for the latest stable
release.
The mau job merges each Play Console MAU export dropped into its inbox
into a daily history and posts last month once its last day is in,
with the change on the month before; from the 10th it posts a reminder
until then. A job may now watch a glob under its state directory: a
matching file starts it through a session-ops@<job>.path unit.
Desktop has no active-user count, so the post lists the latest stable
release's downloads per platform and adds them to Android's MAU for the
total. The timer moves to 11:00, an hour clear of the queue's posts.
The monthly post is now the one place Desktop downloads are counted,
so release-stats and its per-release CSVs go.
A Discord app on webhooks.session.codes, answering two guild slash commands:

- /run job:<job> starts session-ops@<job>.service for a job jobs.toml marks
  `discord = true` (crowdin-sync and crowdin-duplicates), refused while that job or
  any queued job is running or waiting to.
- /mau-upload file:<csv> downloads the attachment, checks it with mau's own parser,
  and moves it into mau's inbox, whose path unit runs the job.

The relay runs as opsbot behind nginx on 127.0.0.1:8081, with no gateway
connection. It refuses a request that Discord did not sign, that comes from another
server, or whose author is in neither allowlist. The polkit rule
`session-ops polkit` writes from jobs.toml lets opsbot start the discord jobs and
nothing else, and the mau inbox is the one path it can write.
Bilb added 7 commits October 8, 2026 20:51
/run refused while the queue was busy, but nothing stopped the queue's timer
starting while a /run job was still going: /run crowdin-duplicates at 09:55 then
ran beside the 10:00 crowdin-sync, two Crowdin clients against one rate limit.

/run now also refuses when the queue timer's next run is sooner than now plus
the job's timeout, since systemd kills the job by then. The next run comes from
`systemctl list-timers --output=json`: on systemd 252, the host's version,
`show --timestamp=unix -P NextElapseUSecRealtime` still prints local time.

The check and `systemctl start --no-block` now run under one lock, so a second
/run a moment later sees the first's job queued. A job's timeout is parsed as a
systemd time span when jobs.toml loads, so a bad one fails there.
…the URL

Discord checks the Interactions Endpoint URL with a request that, without the
route, hits the catch-all 404, and it then refuses to save the URL.
default_member_permissions "0" hides a command from every member without
Administrator; the owner and administrators still see it. The relay's allowlist
refuses them like anyone else.
install.sh made every watched job's inbox writable by opsbot, though the relay
writes only mau's. Other watched inboxes stay the job account's alone, at 0700.
A test ties the relay unit's ReadWritePaths= and install.sh to mau's watch.
The doc page named only the busy check. The start lock holds only within one
process, which is how session-ops-discord.service runs uvicorn.
/run refuses a job whose timeout would carry it past the queue's next run, read
from list-timers. With RandomizedDelaySec, re-arming the timer draws a new delay,
so the queue could start up to two minutes before the time /run read. On one
host the delay spread nothing.
The App Store Connect API's App Sessions report counts a device once per
app version, OS and territory it used in the month, so its sums run 5%
to 20% above App Analytics. iOS therefore comes from the same kind of
manual export as Android: the inbox tells the two apart by their
columns, and a month posts once both have its last day.
@Bilb Bilb changed the title feat: per-platform usage stats: Desktop downloads and Android MAU feat: monthly active users per platform: Android, iOS and Desktop Oct 9, 2026
Bilb added 3 commits October 9, 2026 12:52
iOS's opted-in devices are divided by the month's opt-in rate from the
App Store Connect API's App Opt In report, named with its counts in the
post. Android outside Play is one line: the latest release's GitHub APK
downloads, and an F-Droid estimate of 10% of every other figure, since
F-Droid publishes no counts.
/mau-upload is now the way exports reach the inbox: it takes either
store's export and names the platform, the reminder points at it alone,
and MAU_INBOX_HOST, which only fed the reminder's rsync line, goes.
- A failure before the inbox is filed moves its exports to rejected/,
  and an unreadable export is a rejection, so the path unit never
  restarts the job into its start limit.
- Apple's fractional value for a day it is still counting is kept, and
  a fractional month end counts as missing; the reminder says so.
- Revisions stay in history.json until the month posts, which lists
  those of the two month ends it compares and then forgets them.
- An empty asc-key.p8 fails naming the file.
@Bilb Bilb changed the title feat: monthly active users per platform: Android, iOS and Desktop feat: monthly active users per platform, and Discord commands to feed and run jobs Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant