muse/README.md

# muse

Request a single song, get it in your music library, correctly tagged.

```
you type "radiohead creep"
   ↓  Deezer track search
you click Get
   ↓  exact album, year and track number looked up
   ↓  slskd searches Soulseek, candidates scored
   ↓  best one downloaded
   ↓  tags written, hardlinked into the library
/music/Radiohead/Pablo Honey (1993)/02 - Creep.flac
```

## Why this exists

Lidarr thinks in MusicBrainz *releases*, so the smallest thing it can want is
an album — ask for one song and you end up monitoring a discography. Soulseek
trades individual files but has no concept of a request. Nothing bridged the
two, so getting one track meant either pulling whole albums through Lidarr or
hand-filing downloads out of slskd's UI.

muse is the missing piece: search → click → it appears in your library.

## The design point

**You pick the exact track before anything downloads.** That means muse holds
the artist, album, year and track number as ground truth *before the file
exists*, and tags deterministically.

Taggers like beets have to infer all of that afterwards from whatever a
stranger happened to name the file. muse never guesses.

## Why Deezer for search, not MusicBrainz

MusicBrainz was tried first and measured worse:

| query | MusicBrainz | Deezer |
|---|---|---|
| `radiohead creep` | 8 covers/mashups above the real recording | `Creep / Radiohead / Pablo Honey` first |
| `bohemian rhapsody` | Queen absent from 90 results | covers, but Queen appears once artist is named |

MusicBrainz is an excellent database and a poor search engine — its index
matches a word in a cover's *title* as readily as one in the artist field.
Client-side re-ranking can only reorder what came back, so it cannot rescue a
result set that lacks the right row.

Deezer's API needs no key and no OAuth. Its search payload omits track number
and release date, so those come from `/track/{id}` at request time — one extra
call per request, not per search result.

## The interface

Next.js 16 + React 19 + Tailwind 4 + Framer Motion, built as a **static
export** and served by the FastAPI process.

That last part is the design decision worth stating: `next start` alongside
uvicorn would mean a Node runtime in the image, a second process to supervise,
and a second thing that can be down while the other is up. muse has no use for
SSR — every view sits behind auth and renders from live JSON — so `next build`
emits plain files, Python serves them, and the deployment stays one container,
one process, one healthcheck.

What the UI does with the extra metadata:

- **Search returns songs with artwork, album, year, genre, duration** and an
  explicit flag. Year and genre aren't in Deezer's track search, so they're
  grafted on from `/album/{id}` — concurrently, deduplicated by album, and
  cached forever, because a release date is immutable.
- **Autocomplete** across songs, albums and artists as you type. Debounced, and
  aborted on each keystroke so a slow response to `rad` can't land after
  `radiohead` and repopulate a stale list.
- **30-second previews.** Deezer's payload includes a preview clip, so you can
  hear a track *before* committing a Soulseek search to it — the cheapest
  possible guard against importing the wrong take. It is not a library player;
  that's Navidrome's job, and its native clients do it better.
- **Album and artist pages.** Click through to a full track list with real
  track numbers, or an artist's top tracks and discography. `Get all` queues a
  whole album in one request.
- **The accent colour is sampled from the artwork.** The dominant colour of
  whatever is on screen drives the entire palette. Deezer's CDN sends
  `access-control-allow-origin: *`, which is what makes client-side canvas
  sampling possible with no image proxy and no server CPU.

### Album track numbers

`/api/album/{id}` calls Deezer twice: once for the header, once for
`/album/{id}/tracks`. The track list embedded in the first response has no
`track_position` — taking the list index instead would misnumber every
multi-disc release, filing disc 2 track 1 as `18 - Birthday.flac`. The
dedicated endpoint is the only source that carries `track_position` and
`disk_number`, and those numbers go straight into the filename and the tags.

### Developing

```bash
./dev.sh            # http://127.0.0.1:5041
```

Creates the venv on first run, keeps its state in `dev/` (gitignored), and
serves the prebuilt `web/out` exactly as the container does — same CSP, same
routes, same middleware. Add `--build` after editing the UI.

It sets **`MUSE_AUTH=off`**, so there is no OAuth app to register against a
localhost callback. With auth off every role check reports `admin`, which is
what keeps the admin panel reachable; the startup banner says so in capitals.

What works with nothing else running: search, autocomplete, 30-second previews,
album and artist pages, artwork-derived accent, the admin panel, and queueing.
Requests reach `searching` and then fail, because acquisition needs slskd —
point `SLSKD_URL`/`SLSKD_API_KEY` at a real instance to test that half.

Two things read differently on a Mac. **CPU and memory show `—`**: those come
from `/proc`, which is Linux-only, so they populate in the container and not
here (load average and disk use `getloadavg`/`statvfs` and work everywhere).
And the **`Users` count includes accounts you seed by hand** — with auth off
nobody can sign in, so add them through the panel's Invite button or the API.

For UI work, leave that running and start Next's dev server beside it:

```bash
cd web && npm install && npm run dev
```

`next dev` serves on :3000 with hot reload and rewrites `/api/*` and `/auth/*`
to the FastAPI process, so the browser sees one origin and cookies behave
exactly as they do in production. `npm run build` writes `web/out/`, which is
what ships. Note that :3000 serves the UI without the CSP the Python process
attaches — check anything CSP-sensitive on :5041 after a `--build`.

## Bulk import

Open **Import a list**, paste, hit **Queue all**. These all parse:

```
Radiohead - Karma Police
Juice WRLD – Lucid Dreams          (en dash)
3. Gorillaz - Feel Good Inc        (numbered list)
Black Hole Sun by Soundgarden
Come As You Are                    (bare title — Deezer copes)
https://deezer.com/album/12345     (expanded into its tracks)
https://deezer.com/playlist/6789   (same)
```

In bulk mode there's no picker — the top-ranked Deezer hit is taken
automatically. Every job shows what it *resolved to*, so a bad auto-pick is
visible in the list rather than quietly landing the wrong song. Lines that
match nothing are reported back as skipped instead of failing the batch.

Failed jobs get a **Retry** button. Most Soulseek failures are the peer going
offline, not a bad request, so retrying later usually just works.

### Everything goes through a bounded queue

`MUSE_WORKERS` (default 3) caps concurrent Soulseek searches. This matters:
each search is 10-40s of polling plus real load on slskd, so one task per
request — which an earlier version did — means a 50-track paste launches 50
concurrent searches and drowns the client. Requests return instantly and the
queue drains at a civil rate.

The queue is in memory, so anything still `queued` or `searching` is put back
on it at startup. A restart mid-import resumes instead of stranding jobs.

## How results are ranked

Two separate rankings, and they fail in different ways.

**Picking the track (Deezer).** Title relevance dominates — whatever query
words the artist name doesn't explain must appear in the title. That term
exists because without it a *completely different song by the right artist*
competes on equal footing: `Oasis - Live Forever` once resolved to
`Cloudburst`, because a small variant penalty was enough to tip between two
unrelated tracks. Artist match, popularity, and a penalty for derivative
versions (cover, karaoke, live, demo, remix…) come after.

Derivative terms match on **word boundaries**, not substrings — a bare `live`
is needed to catch `Come As You Are (Live)`, but substring matching would also
penalise `Deliver Me` and `Olive`. And a term only counts against a result when
you didn't type it, so `oasis live forever` and `nirvana unplugged` still work.

**Picking the file (Soulseek).** Format preference first, then whether the peer
can actually serve you (free upload slot, short queue), then filename
plausibility, then bitrate. A marginally better file behind 200 queued users
loses to a good-enough one available now.

## Review: when muse isn't sure

Picking a file and *trusting* it are different questions, so they use different
numbers. The rank above decides which candidate to fetch and is right to weigh
peer speed and queue length. None of that says anything about whether the file
is the correct recording — a fast peer must not be able to raise the confidence
of a wrong match. So a second score, **confidence**, is computed from the
filename alone:

| | weight |
|---|---|
| title present in the filename | 0.75 |
| artist present in the filename, folder, or partially | 0.25 |
| album name somewhere in the path | +0.05 |
| a variant tag you didn't ask for (live, remix, cover, karaoke…) | ×0.8, and capped below auto |

Then:

- **≥ 0.95** — filed automatically. Nothing to do.
- **0.70 – 0.95** — held in **Review** with the reasons shown.
- **< 0.70** — also held, but it will almost certainly be rejected.

Confidence gates the **import, not the download**. The transfer completes
either way and the file waits in the downloads directory: approving is then
instant and rejecting is a delete. Holding the *transfer* instead would mean
the peer has usually vanished by the time anyone looks.

The two cases this exists for are worth naming. A file called `02 Creep.flac`
in an unsorted folder scores 0.75 — the title is exactly right and the artist
is simply absent, which is the commonest honest case there is, so it must land
in review rather than be binned. And `Creep (Live at Glastonbury)` scores 0.80
with *both* artist and title matching perfectly: the variant cap is what stops
a live recording auto-filing over the studio version you asked for. Ask for
`Creep (Acoustic)` explicitly and it auto-imports at 0.97, because then the
variant is the request.

Tune with `MUSE_AUTO_CONFIDENCE` and `MUSE_REVIEW_CONFIDENCE`.
`tests/test_matching.py` asserts the bands, not the exact scores.

## Discover

Recommendations and a personal recap, both built from **muse's own request
history** plus Deezer's related-artists graph — never from listening data.
Navidrome owns play counts, favourites and history; a second source of truth
for those would end up disagreeing with the one you actually listen through.

So "because you added" seeds off the artists you have imported most, and the
recap counts requests, genres and release decades. What you *played* is
Navidrome's business.

## Email

Off unless `MUSE_SMTP_HOST` and `MUSE_SMTP_FROM` are set. muse writes to you
when a request lands in your library, and when a download finished but needs
your call in Review.

```
MUSE_SMTP_HOST=smtp.fastmail.com
MUSE_SMTP_PORT=587
MUSE_SMTP_USER=you@example.com
MUSE_SMTP_PASS=an-app-password
MUSE_SMTP_FROM=muse@example.com
MUSE_SMTP_TLS=starttls          # or ssl, or none
```

Each user sets their own address under **Settings**, or turns mail off
entirely. Google accounts need no configuration — the identity *is* an email
address. Sending runs on a worker thread and every failure is logged and
swallowed: a notification is never worth failing an import over.

## Authentication

OAuth against GitHub and/or Google, built so this can face the internet.

**It fails closed.** Auth is on unless `MUSE_AUTH=off` is set explicitly, and
if it is on with incomplete config the app *refuses to start*. The failure mode
worth engineering against isn't a broken login page — it's a misconfigured
deploy quietly serving an unauthenticated download client to the internet.

### Who may log in

Two rules, checked in order:

1. **`MUSE_ALLOWED_USERS`** — the bootstrap allowlist. Always admitted, on any
   enabled provider. This is how the first admin gets in when the database is
   empty, and the recovery path if someone locks themselves out.
2. **A `users` row that isn't disabled, on the provider it was invited for** —
   how everyone invited later gets in, with no env change and no redeploy.

Anyone matching neither is rejected and the attempt is logged with their
username. The check runs on **every request**, not just at login: an
authorization check that only runs at sign-in isn't revocation, it's a delay.

The first account to ever sign in becomes **admin**. No bootstrap ceremony, no
default password.

#### Invites are bound to a provider

Invite `friend@gmail.com` for Google and Google is the only way in for that
identity — the same address arriving over a different provider is refused. So
"the owner signs in with GitHub, everyone else with Google" is an enforced
boundary rather than a convention, and there is never a second, weaker door
onto an existing account.

For the same reason a successful login no longer refreshes the stored
`provider`: it is part of the identity an admin issued, so letting a sign-in
rewrite it would let the login quietly rewrite the rule that permitted it.

A Google invite must be the full email address, because a Google identity *is*
one — inviting a bare username would authorise something Google can never
return, and the person would just see "not authorised" with no explanation.

### Setting it up

1. <https://github.com/settings/developers> → **New OAuth App**
   - Homepage URL: `https://muse.yourdomain.com`
   - Authorization callback URL: `https://muse.yourdomain.com/auth/callback`
2. Generate a client secret.
3. Set `MUSE_GITHUB_CLIENT_ID`, `MUSE_GITHUB_CLIENT_SECRET`,
   `MUSE_ALLOWED_USERS` (your GitHub login), and `MUSE_BASE_URL`.

To add Google, set `MUSE_AUTH=github,google` plus the two Google variables.
**The callback URL is the same** — `/auth/callback` for every provider, with
the provider carried in the state cookie rather than the URL — so adding
Google needs no change to an already-registered GitHub app. With more than one
provider enabled, `/auth/login` shows a chooser.

Google identities are the **verified** email address; an unverified one is
refused outright, since anyone can put someone else's address on an account
they control and the allowlist is keyed on exactly that string.

| variable | default | meaning |
|---|---|---|
| `MUSE_AUTH` | `github` | comma list of `github`, `google`, or `off` |
| `MUSE_GITHUB_CLIENT_ID` / `_SECRET` | — | GitHub OAuth app |
| `MUSE_GOOGLE_CLIENT_ID` / `_SECRET` | — | Google OAuth client |
| `MUSE_ALLOWED_USERS` | — | bootstrap allowlist; required only while the users table is empty |
| `MUSE_BASE_URL` | — | external URL, e.g. `https://muse.yourdomain.com` |
| `MUSE_COOKIE_SECURE` | `true` | set `false` only for plain-http LAN testing |
| `MUSE_SESSION_DAYS` | `7` | session lifetime |

## Multiple users

Roles are **admin**, **user** and **guest**. Admins get an extra tab.

**The row is the authorisation** — add someone's Google address in the admin
panel and they can sign in. Remove them and they can't.

**The email is a notification, not a credential.** If SMTP is configured they
also get a message telling them they have access and which account to use. That
link carries **no token**, deliberately: the row already names the exact
identity permitted, and Google will not hand muse an unverified address — so
possession of the email is not what lets anyone in. Which means a forwarded or
leaked invite grants nothing, nothing expires, and **resend is always safe**.
The admin panel offers resend on anyone who hasn't signed in yet.

The alternative — a signed, expiring token — buys you the ability to withdraw
an invitation before it's used. Deleting the row does that already, so the
token would be machinery guarding something already guarded.

### Quotas

Two per-user limits, both `0` = unlimited:

- **Requests per month** — calendar month, resets on the 1st. A rolling 30-day
  window was rejected on purpose: "12 left until the 1st" is something a person
  can plan around, where a rolling window drips capacity back at unpredictable
  times.
- **Storage** — bytes of imported audio attributable to that user, measured
  from the file actually written rather than the size the Soulseek peer
  advertised.

Batches are checked against the **whole batch**: queueing a 30-track album with
5 requests left is refused up front, not accepted and then abandoned halfway.
Admins are exempt — somebody has to be able to fix the library when a quota is
what's in the way.

### What the admin panel shows

Host CPU, memory and disk (read straight from `/proc` — no psutil, no compiled
dependency, and inside a container `/proc` reports the *host*, which is what
you want for a homelab box), library size, slskd connection state, a 30-day
request chart, per-user and per-artist breakdowns, and the user table with
inline role and quota editing.

### Guard rails

- **The last active admin cannot be demoted, disabled or deleted.** An install
  with no admin can't be administered and nothing in the UI can undo it.
- **Disabling revokes sessions immediately** rather than at expiry.
- **Deleting a user keeps their jobs.** The history is a record of what's in
  the library and why; removing a person shouldn't rewrite it.
- **Requests are private by default.** Users see their own; admins see all
  (`?mine=true` to scope down). Retrying someone else's job returns **404**,
  not 403 — a distinct error would confirm the id exists.

### What it does

- **Server-side sessions.** The cookie carries a 256-bit random token and
  nothing else; the database stores only its SHA-256. No signed payload to get
  wrong, a stolen database yields no usable cookies, and logout genuinely
  revokes. Cookie is `HttpOnly`, `Secure`, `SameSite=Lax`.
- **Allowlist re-checked every request**, so removing someone locks them out
  immediately rather than whenever their session expires.
- **OAuth `state`** is generated per login and compared with
  `secrets.compare_digest`, so a callback you didn't initiate is rejected.
- **Strict CSP, with hashed scripts.** `script-src 'self'` plus a SHA-256 for
  each inline script Next emits. Those hashes are computed *at startup* by
  reading the built page, so a rebuild updates them with no maintenance and no
  per-request HTML rewriting — the usual nonce dance is unnecessary precisely
  because a static export never varies between requests.
  `style-src-attr 'unsafe-inline'` is the one concession: Framer Motion writes
  its pre-hydration `initial` state into `style=""`, and without it every
  animated element flashes in at full opacity. That directive permits style
  *attributes* only — injected `<style>` blocks and remote stylesheets are
  still blocked, and CSS-based exfiltration goes through `url()`, which
  `img-src` already pins to two hosts.
- **`img-src`/`media-src` name Deezer's CDN explicitly** rather than allowing
  `https:` wholesale. Under a blanket `https:`, injected markup can beacon to
  any host by requesting an image from it — a working exfiltration channel even
  with `connect-src` locked to `'self'`.
- Plus `frame-ancestors 'none'`, `base-uri 'none'`, `object-src 'none'`,
  `nosniff`, `X-Frame-Options: DENY`, `no-referrer`, HSTS.
- **The JS bundle is behind the session too.** There is no local login page to
  style — an unauthenticated visitor is redirected straight to GitHub and never
  receives the HTML — so `/_next/*` requires a cookie like everything else, and
  doesn't hand out a map of the app to whoever finds the URL.
- **No API docs** — `/docs`, `/redoc` and `/openapi.json` are disabled, so the
  login page doesn't hand out a map of every endpoint.
- **Denied logins are logged** with the GitHub username — on an
  internet-facing box that's your signal someone found the page.
- `/healthz` is the only unauthenticated route and returns a bare `{"ok":true}`
   — no versions, no config, no counts.

Behind a reverse proxy, `MUSE_BASE_URL` is read from config rather than from
`X-Forwarded-*` headers on purpose: those are attacker-controllable and this
one is a redirect target.

## Running it

```yaml
services:
  muse:
    image: ghcr.io/geneticglitch1/muse:latest
    container_name: muse
    user: 1000:1000
    environment:
      - SLSKD_URL=http://slskd-host:5030
      - SLSKD_API_KEY=<your slskd api key>
      - MUSE_DOWNLOAD_DIR=/data/torrents/soulseek/complete
      - MUSE_LIBRARY_DIR=/data/media/music
      - MUSE_DB=/config/muse.db
    volumes:
      - ./config:/config
      # ONE mount covering both the download dir and the library. See below.
      - /data:/data
    ports:
      - 5040:5040
```

### Mount both paths under a single volume

Imports are `os.link()`, never a move, so the download keeps being shared by
slskd while the library gets a second name for the same bytes.

`link()` between two different **mounts** fails with `EXDEV` even when both sit
on the same filesystem. Split `/data` into a downloads mount and a library
mount and every import silently degrades to a full copy, storing everything
twice. Verify after your first import:

```bash
stat -c "%h" "/data/media/music/<Artist>/<Album> (<Year>)/<NN> - <Title>.flac"
```

`2` is correct. `1` means it copied.

### Configuration

| variable | default | meaning |
|---|---|---|
| `SLSKD_URL` | `http://192.168.1.87:5030` | slskd base URL |
| `SLSKD_API_KEY` | — | slskd API key (**required**) |
| `MUSE_DOWNLOAD_DIR` | `/data/torrents/soulseek/complete` | where slskd puts finished files |
| `MUSE_LIBRARY_DIR` | `/data/media/music` | where the library lives |
| `MUSE_DB` | `/config/muse.db` | job history |
| `MUSE_FORMATS` | `flac,mp3` | ordered preference; anything else is rejected |
| `MUSE_MIN_MATCH` | `0.45` | filename-to-title similarity floor |
| `MUSE_AUTO_CONFIDENCE` | `0.95` | at or above this, import without asking |
| `MUSE_REVIEW_CONFIDENCE` | `0.70` | below this, held but probably wrong |
| `MUSE_WORKERS` | `3` | concurrent Soulseek searches |
| `MUSE_SMTP_HOST` | — | set to enable email; see **Email** above |
| `MUSE_SMTP_PORT` | `587` | |
| `MUSE_SMTP_USER` / `_PASS` | — | omit for an unauthenticated relay |
| `MUSE_SMTP_FROM` | `MUSE_SMTP_USER` | envelope sender |
| `MUSE_SMTP_TLS` | `starttls` | `starttls`, `ssl`, or anything else for none |
| `MUSE_LOG_LEVEL` | `INFO` | |

`MUSE_FORMATS` is a preference list, best first — rejecting everything not on
it is what keeps `.cue` and `.m4a` junk out.

`MUSE_MIN_MATCH` guards against importing the wrong song. Soulseek filenames
are chaotic (`08-radiohead-creep_(accoustic)-ent.mp3`), so the comparison is
against a normalised form and also accepts plain containment. Raise it if you
get bad matches; lowering it is how you quietly corrupt a library.

### How candidates are scored

In priority order: format preference, then whether the peer can actually serve
you (free upload slot, short queue), then filename plausibility, then bitrate.
A marginally better file behind 200 queued users loses to a good-enough one you
can have now.

## Pulling on a server

The GHCR package inherits this repo's visibility, so it is **private**. The
machine pulling it needs to authenticate once:

1. Create a classic PAT at <https://github.com/settings/tokens> with only
   **`read:packages`**.
2. On the server:

```bash
echo "<token>" | docker login ghcr.io -u <github-username> --password-stdin
```

That writes `~/.docker/config.json` and persists across reboots. Classic PATs
can be set to no expiry; if you set one, the pull starts failing on that date
with `denied`.

To make the image public instead — no login needed anywhere — go to the
package page → Package settings → Change visibility.

## API

| endpoint | purpose |
|---|---|
| `GET /` | web UI |
| `GET /api/search?q=` | track search, enriched with year and genre |
| `GET /api/suggest?q=` | autocomplete — songs, albums, artists |
| `GET /api/album/{id}` | album header plus its full track list |
| `GET /api/artist/{id}` | artist, top tracks, discography |
| `POST /api/request` | queue a track |
| `POST /api/request/batch` | queue many known tracks (a whole album) |
| `POST /api/import` | queue many from pasted text |
| `POST /api/retry/{id}` | requeue a failed job |
| `GET /api/jobs` | request history and state (own, or all for admins) |
| `GET /api/review` | downloads held back for a decision |
| `POST /api/review/{id}` | `approve` files it, `reject` deletes it |
| `GET /api/discover` | related artists, seeded from what you've imported |
| `GET /api/recap` | your top artists, genres and decades |
| `GET /api/me` | who you are, your role and quota usage |
| `PATCH /api/me` | your own email address and notification toggle |
| `GET /api/health` | slskd reachability, mount writability |
| `GET /api/admin/users` | list users with usage |
| `POST /api/admin/users` | invite, and email them if SMTP is set up |
| `POST /api/admin/users/{u}/resend` | send the invitation again |
| `PATCH /api/admin/users/{u}` | role, quotas, disable |
| `DELETE /api/admin/users/{u}` | remove (jobs retained) |
| `GET /api/admin/system` | CPU, memory, disk, slskd, library totals |
| `GET /api/admin/activity` | daily volume, per-user and per-artist breakdowns |

Everything except `/healthz` requires a session; `/api/admin/*` requires the
admin role.

## Releases

Every push to `main` publishes `:latest` and `:sha-<short>`. Tagging `vX.Y.Z`
also publishes `X.Y.Z` and `X.Y`. Pin a tag in production if you'd rather not
track `latest`.