Add bulk import, and a bounded queue to make it safe

Requesting songs one at a time does not scale past a handful, so this adds a
paste-a-list import. It accepts "Artist - Title", en dashes, "Title by
Artist", numbered lists, bare titles, and Deezer album/playlist URLs, which
get expanded into their tracks.

The queue is the load-bearing part. Every request previously spawned its own
asyncio task, so a 50-track paste would have launched 50 concurrent Soulseek
searches -- each 10-40s of polling and real load on slskd -- and drowned the
client. Work now goes through an asyncio.Queue drained by MUSE_WORKERS
consumers, default 3. Requests return immediately and the batch drains at a
civil rate. The queue is in memory, so jobs left 'queued' or 'searching' are
re-queued at startup rather than stranded by a restart.

Ranking needed two fixes found by testing against the real API:

Import was calling deezer_search(limit=1), which fetches two rows, so the
ranker was choosing between a demo and a rehearsal take while the studio
version -- six times the popularity -- sat unseen further down. It now pulls
a real candidate pool.

More seriously, the Deezer ranker had no title-relevance term at all: only
artist match, popularity and penalties. A different song by the right artist
therefore competed on equal footing, and "Oasis - Live Forever" resolved to
"Cloudburst" as soon as a variant penalty tipped the balance. Title overlap
now dominates, and the variant penalty is small enough only to break ties
between versions of the same song.

Derivative terms match on word boundaries rather than substrings, so a bare
"live" catches "Come As You Are (Live)" without penalising "Deliver Me" or
"Olive", and a term is ignored when the user typed it -- "oasis live forever"
still works.

Verified against live slskd: six tracks pasted, six resolved to the correct
song, five imported with correct album/year/track and link count 2, in about
two minutes. The sixth returned an empty Soulseek search and is retryable
from the UI.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
aryansinghandClaude Opus 5 committed 2026-08-04 03:23:17 -05:00
1 parent 269f13524d
commit 10ccc282d4
3 files changed
+373 -45

No files matched your search

+58
View File
@@ -50,6 +50,61 @@ Deezer's API needs no key and no OAuth. Its search payload omits track number
and release date, so those come from `/track/{id}` at request time — one extra
call per request, not per search result.
## Bulk import
Open **Import a list**, paste, hit **Queue all**. These all parse:
```
Radiohead - Karma Police
Juice WRLD – Lucid Dreams (en dash)
3. Gorillaz - Feel Good Inc (numbered list)
Black Hole Sun by Soundgarden
Come As You Are (bare title — Deezer copes)
https://deezer.com/album/12345 (expanded into its tracks)
https://deezer.com/playlist/6789 (same)
```
In bulk mode there's no picker — the top-ranked Deezer hit is taken
automatically. Every job shows what it *resolved to*, so a bad auto-pick is
visible in the list rather than quietly landing the wrong song. Lines that
match nothing are reported back as skipped instead of failing the batch.
Failed jobs get a **Retry** button. Most Soulseek failures are the peer going
offline, not a bad request, so retrying later usually just works.
### Everything goes through a bounded queue
`MUSE_WORKERS` (default 3) caps concurrent Soulseek searches. This matters:
each search is 10-40s of polling plus real load on slskd, so one task per
request — which an earlier version did — means a 50-track paste launches 50
concurrent searches and drowns the client. Requests return instantly and the
queue drains at a civil rate.
The queue is in memory, so anything still `queued` or `searching` is put back
on it at startup. A restart mid-import resumes instead of stranding jobs.
## How results are ranked
Two separate rankings, and they fail in different ways.
**Picking the track (Deezer).** Title relevance dominates — whatever query
words the artist name doesn't explain must appear in the title. That term
exists because without it a *completely different song by the right artist*
competes on equal footing: `Oasis - Live Forever` once resolved to
`Cloudburst`, because a small variant penalty was enough to tip between two
unrelated tracks. Artist match, popularity, and a penalty for derivative
versions (cover, karaoke, live, demo, remix…) come after.
Derivative terms match on **word boundaries**, not substrings — a bare `live`
is needed to catch `Come As You Are (Live)`, but substring matching would also
penalise `Deliver Me` and `Olive`. And a term only counts against a result when
you didn't type it, so `oasis live forever` and `nirvana unplugged` still work.
**Picking the file (Soulseek).** Format preference first, then whether the peer
can actually serve you (free upload slot, short queue), then filename
plausibility, then bitrate. A marginally better file behind 200 queued users
loses to a good-enough one available now.
## Running it
```yaml
@@ -99,6 +154,7 @@ stat -c "%h" "/data/media/music/<Artist>/<Album> (<Year>)/<NN> - <Title>.flac"
| `MUSE_DB` | `/config/muse.db` | job history |
| `MUSE_FORMATS` | `flac,mp3` | ordered preference; anything else is rejected |
| `MUSE_MIN_MATCH` | `0.45` | filename-to-title similarity floor |
| `MUSE_WORKERS` | `3` | concurrent Soulseek searches |
| `MUSE_LOG_LEVEL` | `INFO` | |
`MUSE_FORMATS` is a preference list, best first — rejecting everything not on
@@ -143,6 +199,8 @@ package page → Package settings → Change visibility.
| `GET /` | web UI |
| `GET /api/search?q=` | Deezer track search |
| `POST /api/request` | queue a track |
| `POST /api/import` | queue many from pasted text |
| `POST /api/retry/{id}` | requeue a failed job |
| `GET /api/jobs` | request history and state |
| `GET /api/health` | slskd reachability, mount writability |