Stats: standard users had a tab whose only content was "Failed to load
statistics". /api/stats now scopes them to their own jobs and searches,
admins still see every account, peons still get nothing. Reset, mismatches
and the shared library size stay admin-only; a `scope` field says which.
Year on album folders (#101): album_folder_year, off by default, names new
folders "Album (2004)" like Lidarr. Album downloads and auto-routed singles
reuse an existing folder with or without a year. Tags take the album name
from .albuminfo, not the folder, which also stops a reused Lidarr folder's
"(2004)" leaking into the ALBUM tag, including via "Add to album...".
Built with a human and a grumpy-AI combo, the latter now fluent in Lidarr.
The Watched tab's new M3U Files pill lists every playlist file on disk and
opens an editor: drag to reorder, remove, prune dead entries, add tracks from
the library index, create, and bin. Untouched entries go back byte for byte,
saves are atomic, and a file a download touched mid-edit refuses the save
rather than losing the track. Watched-playlist M3Us are rebuilt on every
refresh, so they open read-only with Save as Copy.
The same library picker gives missing watched tracks a From Library button,
recording the chosen file as the track's resolved path so it joins the M3U.
Playlists, Upgrades, Audit and M3U Files are now pills, each loading only
when opened.
Built by a human with opinions and an AI that was told about em dashes.
- Search replacement on any album-browser track: the bad copy goes to
Trash, the replacement keeps the folder, pressing, tags, and disc/track
numbering, and the album M3U refreshes. Busy albums, stale selections,
and ambiguous files get refused rather than guessed at.
- The Artists tab explains what following an artist actually includes:
release-level credits, shared and featured credits, and the MusicBrainz
release types that are skipped.
- Settings shows the configured bind address and port, /api/config
exposes them, LISTEN_PORT rejects nonsense at startup, and the README
covers VPN-container networking for Compose and Unraid.
Built with a human, a grumpy AI, and Codex, which did the heavy lifting
on this one while the grumpy AI held the clipboard.
Click it and the Queue opens filtered to failed downloads. The filter is
a new status parameter on /api/jobs, so old failures turn up too rather
than hiding beyond the newest 250. Your own account's scope and any Queue
search survive the trip, and Show everything drops only the filter.
Stats counts every account; where that disagrees with your Queue, the
status line says so instead of quietly showing fewer.
Built with a human and a grumpy-AI combo, plus Codex rearranging the
furniture in the next room.
- Tabs left open across a redeploy now offer a Reload banner: the health
poll carries the static build token, and the cache-bust token finally
watches style.css and quips.js too
- style.css spacing migrated onto the scale (half-steps added for 2, 6, 10
and 20px; the stragglers snapped to the nearest step)
- Header wraps from 520px, so Sign out stops falling off the edge
- Rabbit hole breadcrumb sticks to the top, as an exit should
- app.py boot work moved into the lifespan, so importing it for unit tests
no longer starts every scheduler and a network-wide health sweep
- SeleniumBase 4.49 to 4.54, checked against real Amazon albums
- Live tests put their M3Us in a hidden .musicgrabber-tests folder, the
ListenBrainz test points at a playlist that does not rotate, and
run_tests.sh stops installing curl_cffi for a source that left
- Last em dashes evicted from the UI and the changelog
Built with a human and a grumpy-AI combo, one tidy drawer at a time.
India's biggest streaming service, asked directly through the api.php its
own web player uses. Most tracks are 320 kbps AAC (saved as .m4a), some
160, and results declare which, so the quality filter and upgrade scanner
know what they're getting. Stream links arrive DES-encrypted with a key
that has been public for years, hence pycryptodome in the image.
Found via Hayasaka, which reaches the same catalogue through someone's
proxy; we skip the middleman so there is nothing to vanish. Catch:
JioSaavn filters search by IP, so outside India the Western catalogue has
holes. The Indian catalogue, which the other sources are thin on, is the
point.
Wired in wherever its siblings live: registry, download routing (.m4a
staging), health probe, URL allowlist, quality tiers, provenance, settings
toggle (on by default), and the frontend badge, stats, quips and preview.
Assembled by one human and one AI who now know far too much about DES.
Adds an update banner above the search box (#99): the server asks GitLab's
public tags API for the newest vX.Y.Z at most twice a day, the browser only
ever asks MusicGrabber, and dismissing it lasts until the next release. Comes
with a changelog link and a Buy me a coffee button, because hope springs
eternal. On by default; Check for Updates in Settings or
UPDATE_CHECK_ENABLED=false and GitLab never hears from you.
Phones get the version number back once the header wraps, and the README
now points Nix folk at rrvsh's community flake (#98).
Refs #99, #98
Built by a human with opinions and an AI with fewer of them.
- MP3Phoenix retired again: source, toggle, preview snippets, curl_cffi.
DB migration v20 removes its setting and the Monochrome hifi-api URL.
- Monochrome search tops up a thin Deezer page from the tracks API.
- /api/download refuses unknown sources rather than misrouting them to
YouTube and failing on a "truncated YouTube ID".
- Deno 2.9.5 -> 2.9.7.
- Live tests draw downloads from a shuffled pool of short clips and
searches from ten songs (tests/live_media.py), instead of fetching
"Me at the zoo" a dozen times a run. Fast tests wait for their download
before deleting it, so nothing is left in the library.
- The MusicBrainz routing tests never routed anything: 186 of 195 Thriller
jobs were dupe hits. They now use Blitzkrieg Bop, and a dupe hit fails
loudly. Doing so exposed playlist album routing not happening; logged.
Built by a human and an AI who now know more about Keyboard Cat's
discography than either of them wanted to.
Monochrome moved from monochrome.tf to monochrome.st and switched its
Turnstile-guarded unified playback off, so every Monochrome download died
on a 503 before Chrome had its coat on.
The site now plays from a plain tracks API: search by artist and title,
keep the exact ISRC match, fetch the FLAC. No token, no Turnstile, no
browser. That is now the primary download route; qbdlx, the Deezer ISRC
rescue and the Tidal stream follow as before. Its address is read from
Monochrome's own client (following a retired domain's meta refresh) so
the next move should not need a release. Lone 502s get two quick retries,
as they do in Monochrome's own player.
The browser route, its broker, the CENC decryption and its settings
toggle are retired, with a v18 migration to tidy the orphaned setting.
Parked at tag archive/monochrome-browser-leg in case it ever returns.
Catch: the tracks API is 16/44.1 only, so no more 24-bit from Monochrome.
Built with a human and a grumpy-AI combo, both muttering about domains.
- Watched-artist search turns a MusicBrainz outage into a 503 with a useful
message, and the page shows it instead of claiming no such artist exists.
- Apple library playlists try every bundle token like public ones do, and
blame an expired media-user-token when Apple refuses the lot.
- PYTHONUNBUFFERED=1, so docker logs arrive on time and in order.
- The e2e test switches the Navidrome dupe check on for Phase B itself
rather than assuming the tester's taste.
Built with a human and a grumpy-AI combo.
Cookies (#95, round two). Saving them wrote to your settings row and then
synced the *shared* file, which is a different file entirely. Empty shared
setting and the sync deleted cookies.txt for the whole house; non-empty and
you carried on downloading as whoever was in the shared jar. For a
personalised playlist that is the original #95 bug in a better suit:
authenticated, plausible, not yours. The save now syncs what it just wrote,
clear_expired_cookies() clears the scope it read from rather than binning
the shared ones and leaving the dead ones in place, and a named user asking
for a personalised playlist gets their own cookies or an honest 401. Shared
jar still rescues ordinary downloads, where the worst case is somebody
else's bitrate. Old installs rebuild the missing per-user file on first use,
because "re-export everything you already saved" is not an upgrade path.
Jars are 0600 now too. Reported, diagnosed and patched by michael.j.strong,
who found it while migrating a fork back to upstream. Good citizen.
Weekly playlists (#97). A ListenBrainz "Created for You" playlist renames
itself every Monday, so a track surviving into the new week still lives in
last week's folder, which the rebuild treated as "not mine" and wrote in as
an absolute path. Fine here, useless anywhere else, which is a poor showing
for a file whose entire job is being handed to something else.
The feature #97 actually asked for turned out to be Mirror sync mode, doing
it quietly since forever. Now in the README, where somebody might find it.
Five tests, each made to fail against the old code first, on the grounds
that a test passing either way is decoration.
Built with a human and a grumpy-AI combo, as usual.
"Find the song in the queue tab" is sound advice right up until the queue is
a 900-track playlist import, at which point it becomes "scroll through ninety
pages and good luck to you". So: a search box.
Server-side, deliberately. A client-side filter over the newest 250 jobs the
tab already caches would have been about thirty lines and would have failed
at precisely the case that prompted it; on Karl's real 2,794-job queue a
search for "creep" turns up 212 jobs the Queue tab previously had no way of
reaching at all.
- GET /api/jobs gains a q param: LIKE across artist, title, album and
playlist name, with the user's own wildcards escaped so somebody searching
for "50_Cent" gets 50 Cent rather than every 50-anything-Cent going
- The response now also carries queue_totals, unfiltered status counts for
the whole queue. Without it, searching down to one card convinced the live
summary and the queue-size guard that the other 800 jobs had emigrated.
Those two were already quietly under-counting anything past the newest 250
- Debounced input, Escape and an x to clear, a match count that owns up when
it has stopped counting at 250, and an empty state that says "no matches"
rather than the cheerfully wrong "no downloads yet"
- 13 tests covering the reach past the window, the four searchable fields,
literal underscores and percent signs, user scoping and the totals
Also fixed while in there: the queue header controls had been running off the
side of a phone for some time, which the search box made worse before it made
better. Wrapping is scoped to the Queue tab rather than the shared header
rule; the wider audit is logged in the backlog.
Built with a human and a grumpy-AI combo.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Each bulk import track now gets its own row as it works through the
list. The moment a track has a real download job behind it, that row
becomes the exact same job card the Queue tab renders, retry, report,
edit tags, the lot, rather than a thinner duplicate of that UI. A track
still being searched shows a plain placeholder row until it graduates
to a real job.
Made possible by two small, reused-not-duplicated pieces:
- renderJobs() now takes a target container instead of always writing
into the Queue tab, so Bulk Import can call the exact same renderer.
- GET /api/jobs gained an ?import_id= filter (joining through
bulk_import_tracks), so Bulk Import can fetch just its own jobs, in
input order, without the usual recent-jobs limit truncating a big
playlist partway through.
Also fixes the other genuine gap from #91: a page reload used to lose
track of a still-running import entirely (the download carried on
fine, the UI just forgot it existed). currentBulkImportId now has a
localStorage backup and reattaches automatically on load.
The bulk-import status endpoint's recent_tracks (10 tracks, priority
sorted) became tracks (every track, input order, job_id included) to
support this; nothing else consumed the old field.
Built with a human and a grumpy-AI combo.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The Artists tab now has two halves behind a pill control: Followed Artists,
unchanged, and a new place to go looking. Seeds come from what you actually
listen to, Navidrome play counts first, then what fills your library, then what
you have downloaded; first source that answers wins outright, since the three
live on different scales and blending them would invent a ranking none of them
gave us. Compilations are excluded throughout, because "Various Artists" is 128
albums of Karl's library and says nothing whatsoever about taste.
Each row says why it is there, every card has a thirty-second preview, and Grab
it hands "{artist} - {title}" to the ordinary search and download pipeline, so a
discovery arrives properly tagged and routed like anything else. Clicking an
artist's name opens their own lookalikes as a new row, with a breadcrumb out.
Built by two agents against a contract fixed up front, then reviewed, which is
where the interesting bits were:
- The contract hand-off leaked, exactly where it was flagged it might. The
frontend treated a missing deezer_id as "no preview, no download", but those
are the ListenBrainz-sourced suggestions, the artists Deezer has never heard
of, so the most obscure finds were the ones rendered unusable. The backend
resolves by name perfectly well without an id. Three places, all fixed; the
play handler already degraded gracefully so pre-emptively disabling the button
was worse than simply trying.
- The route picked seeds twice per request, once for the rows and once for the
signal, so Navidrome and the database were queried twice for an answer already
in hand. discovery_rows now returns both. Honestly flagged by the agent; it
was my contract that forced it.
- Deezer's related ordering is neither similarity nor popularity. It opened the
Shaboozey row with "Gareth", 971 fans, whose top track is a cover of somebody
else's song off an album called "Covers (From Across The Pond)", ahead of
Jelly Roll's 227,785. A row is a shelf you scroll, so the leftmost card decides
whether anyone trusts the rest of it. Sorted by reach now; the obscure ones are
still all there, further along, which is the right way round for browsing.
That one change turned the rows from awkward to genuinely good: Netsky,
Noisia, Camo & Krooked for Pendulum.
Verified in an actual browser rather than by reading the diff, since this is UI
work: pills switch, five rows render, previews play and resolve their titles,
the rabbit hole appends and breadcrumbs, the manual box works, no horizontal
scroll at 400px, no console errors. Kept at docs/ui-test-discover.py.
39 discovery tests. Mutation-tested, seventeen mutations, no survivors,
including four aimed squarely at the agent's own work. Fast suite 1,253.
Built with a human, a grumpy-AI combo and two junior assistants who did the
typing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Karl, on being shown Therapy? suggestions of Nirvana and Faith No More: "I have
tons of that in my library already - this is about artist discovery". Quite.
Measured before fixing it: across eight real seeds, 61% of what we suggested was
already in the library. Nirvana and Nine Inch Nails both 80%, Taylor Swift 90%.
A recommendation you already own is not a recommendation, so suggestions are now
checked against the library index and the ones you have are dropped. Therapy?
now answers Life of Agony, Primus, L7 and Urban Dance Squad.
library_index gains known_artists(root), which expands each stored credit
through artist_credit_lookup_variants before keying it. The tag on disk is a
credit, not an identity: a library holding "Pendulum feat. Jasmine Yee" does own
Pendulum, and without the expansion we would have offered them Pendulum as a
discovery. Worth 307 extra keys on a real library, caught Duke Dumont and The
Qemists hiding inside collaborations, and costs a tenth of a second an hour.
None and the empty set are kept firmly apart there. "I could not look" must
never become "they own nothing", or discovery would suggest somebody their
entire record collection. When the index cannot answer, nothing is filtered
rather than everything, because a few familiar names beat a blank page.
Also, artist_taster: a track to introduce each artist by, with Deezer's
thirty-second preview, which needs no key and is what makes hovering to listen
possible. Deliberately not always the top track, per Karl: the most famous song
is the one they are most likely to already have, so it offers the best-ranked
one they have not got. Radiohead leads with "Let Down" rather than "Creep".
Owning all ten, as with Nirvana, returns the hit flagged as owned rather than
dredging up an obscurity nobody chose; verified that he really does own all ten.
Related artists now carry their Deezer id through, so following one costs one
round trip rather than two, and resolution is cached, including the misses. The
whole shape of this feature is clicking from artist to artist and it was
re-resolving the same names every hop: 362ms cold, 0.01ms warm.
Pandora, asked about as another source, is partner-only by invitation and is not
accepting new partnership requests. TheAudioDB is keyless and works but has no
similarity data at all, only genre, style and mood, which may be useful later as
a diversity axis. Both recorded in the research doc.
30 tests. Mutation-tested: twelve mutations, no survivors. One earlier candidate
was dropped as an equivalent mutant rather than chased, since an empty owned-set
and None filter identically.
Backward compatible: the existing Explore bar reads .artist and still does.
Built with a human and a grumpy-AI combo.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Explore button asked ListenBrainz and, on Karl's real library, ListenBrainz
had nothing at all to say about five of his eight favourite artists. Not poor
suggestions: none, silently. Pendulum, Bring Me the Horizon, Shaboozey, Therapy?
and Sonny Fodera all came back empty. That is the whole of "recommendations are
lame" explained, and it is a coverage problem rather than a taste one.
Deezer answers all eight, wants no account, no key and no sign-up, and offered
the highest proportion of artists not already in the library when the three
candidate sources were measured against each other. It leads now; ListenBrainz
stays for the occasional artist Deezer has never heard of, and is skipped
entirely when Deezer has already answered, since MusicBrainz asks for a
one-second gap between requests and there is no reason to spend it.
The subtler half is resolving which artist you meant. Deezer's search returns
the first match rather than the best, so "Pendulum" is an act with eighteen fans
and no related artists at all, ahead of the real one with three hundred and
seventy thousand; "Nirvana" is a sixties band with two hundred and twenty-two
fans ahead of the one with ten million. Now the ten best candidates are scored
on the name first, with popularity only breaking ties between genuine matches.
It also declines to guess. A compilation credit for "Jay Burnett" matched to
"Marcos Brunet" on popularity alone, and recommended an entire row of
Spanish-language worship music off the back of it. A name nothing matches now
returns nothing.
Name matching reuses utils.artist_credit_match_score rather than growing a third
normaliser; the 1.0 floor was picked by running it against thirteen real cases,
including the ones above, and is the only threshold that gets all thirteen right.
In its own file at Karl's request, and he is right: app.py is 6,684 lines and
downloads.py 5,966. The route is four lines now.
Coverage 3/8 to 8/8, about 0.25s cold, 0.01ms cached, verified live through the
authenticated API. 17 tests, offline and deterministic. Mutation-tested with
docs/mutate-discovery.py: eight mutations, no survivors, though one survived the
first run because the test covering it used a candidate the name floor had
already thrown out, so the tie-break never actually ran.
Groundwork for the Find New Artists tab. Full research in
docs/music-discovery-research.md.
Built with a human and a grumpy-AI combo, the grumpy half still chuckling about
the worship music.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Library index Phase 5. The hooks were written and tested back in 3a and have
been sitting there unwired ever since, on the reasoning that a daily
reconciliation is good enough. It mostly is: every reader validates what it
gets back, so staleness costs speed and not correctness.
Mostly. The exception is the Queue's retag, because find_by_tags treats a
completed index's miss as authoritative by design, so a row still carrying the
pre-retag artist and title matches a later download of that name and gets it
refused as a duplicate of a file that no longer carries it. Introduced by 3c
earlier in this same unreleased version, so nobody has been bitten in the wild.
A retag therefore removes the row and reads it again rather than relocating it;
the bytes changed, not just the name, and that includes an album-only edit which
renames nothing at all.
Five other sites now report in too: download completion, trash, restore, the
upgrade swap and the quarantine undo.
The arrival hook is deliberately not at the move. Every download path moves the
file into the library before validating it, and an integrity or bitrate failure
then deletes it again, so hooking _promote_from_staging or _move_completed_file
would mint rows for files we thought better of seconds later. _update_job with a
completed status and a final_path is the one point where the file is definitively
staying, and it is one site rather than six.
Deletions are not hooked, and working out why settled where to stop. A row
pointing at a file that has gone cannot produce a wrong answer, because every
candidate goes through authoritative_audio_file and a missing path fails it,
leaving "no duplicate", which is correct. Arrivals are the asymmetric case: an
unindexed new file makes the index say "not here" authoritatively when it simply
has not looked yet.
Also here, two things found while reviewing rather than while writing:
- The hooks now decline anything that is not audio, or that sits in a bin the
walker skips. Watched playlist jobs record an .m3u as their final path, and
without this the index would hold rows no scan would ever yield again. Note for
the archives: our own trash and quarantine live under /data, not inside
music_dir, contrary to four comments I confidently wrote earlier today.
- 3e dropped the per-file resolve() from the admin rebuild and justified it by
claiming final_path is stored unresolved. It is not; _local_final_path resolves
before storing. On a library reached through a symlink that would have minted a
second history row for every file already in the queue, which is the exact
failure the comment claimed to prevent. The measurement that backed it up was
true and useless: Karl's library has no symlinks, so resolve was a no-op there.
Now the root is resolved once and each relative path rebased onto it, which is
equivalent because no scan descends a symlinked directory. Keeps 3e's saving.
An arrival costs 67ms on a real file over SMB, and 0.9ms when the index already
describes it, which is what the fingerprint short-circuit is for: an "already
exists" outcome still records a completed job, so a bulk import of 500 tracks of
which 400 were owned would otherwise have reopened all 400 to rewrite identical
rows. Confirmed against the live 20,470-file index that a rebuilt row keeps its
SOURCE tag, since indexing our own downloads as somebody else's would have been a
considerably worse bug than the one this fixes.
18 new tests, each checked for teeth by neutering the fix it covers and watching
it fail. Fast suite 1,196 passed; slow suite 1,226 passed. perform_upgrade's swap
hook is wired but untested, because nothing in tests/ can reach that function
without a staged download, fpcalc and content hashes; logged in the backlog.
Built with a human and a grumpy-AI combo, the grumpy half having spent a good
while proving its own previous comment wrong.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Steps 3d and 3e. Every consumer of the library index now reads it.
3d, the upgrade scan. It can only ever act on files MusicGrabber
downloaded itself, which on a real library is a small minority, and it
opened all the others on every pass to re-confirm they were still not
ours. The index recorded that answer last time it read them, so a row
matching this exact size and mtime_ns now answers instead. Nanoseconds
rather than whole seconds, so the gate is stricter than the scan's own
cache. Measured on Karl's library over SMB: a steady-state pass over
1,673 files went 33.6s to 2.2s, with 1,287 file reads replaced by
lookups. One reader held open for the whole walk, because a connection
per file would have eaten a good slice of the winnings.
3e, the admin rebuild. Reads the index instead of walking three roots,
and loses its per-file resolve() on the way. That resolve was at best
thousands of syscalls to change nothing, since final_path is stored
unresolved, and at worst a way to mint duplicate history for a library
reached through a symlink. Index and walk were compared directly on the
real library and agreed on all 1,813 paths; a test now pins that down,
because the same library giving two different histories depending on
whether a reconciliation has run yet is nobody's idea of a good time.
paths_under() distinguishes "no audio in there" from "I could not
look", which matters more than usual when the answer becomes rows in
the jobs table. Both callers keep their old scan as the fallback, so a
missing, cold, locked or half-migrated index costs only speed.
Fast suite 1,178 passed, slow suite 1,207. The new tests were each
neutered in turn to confirm they actually fail without the fix.
Built by a human and a grumpy AI, the latter having finally found
something it could measure instead of merely assert.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Phase 3a and 3b of the library index plan. The index itself, plus exactly one
consumer: the storage tile, chosen because being wrong about a number on the
Stats tab is the cheapest imaginable way to find out something is off.
library_index.py holds one row per audio file beneath each music root: size,
mtime, inode, codec, bitrate, duration, tags, and whether we downloaded it. It
reconciles against the disk once a day, opening only files that are genuinely
new or changed. A file that merely moved is recognised by its size and mtime and
keeps everything already known about it, so an organiser's overnight tidy-up no
longer means re-reading the lot.
It lives in its own database file. No ATTACH, no shared lock domain with jobs
and history, so index work cannot block a download, and recovery from a corrupt
index is "delete the file". A version bump discards the contents and rebuilds
from disk rather than migrating, because a cache has nothing worth migrating and
the disk always knows.
Keyed on the root, not the user: three users pointed at one library is one set
of rows, not three identical ones. Isolation still comes from
authoritative_audio_file checking containment on the way out, which is where it
always actually came from.
Files we did not download get a row too, saying so. On this library that is
19,874 rows out of 20,369, and it is most of the point: without them we open
every one of those files on every scan, forever, to rediscover that they are not
ours.
The delete guard is the one part that can hurt. A walk returning under half the
previous count refuses to prune at all, logs why, and leaves the index intact
for the next pass. A mount having a bad day must never be published as "the user
deleted their library".
Measured on the real library, 20,369 files and 486.8 GB:
cold build 1,333s and 1,344s across two runs, one-off, background
warm pass 24-49s, zero probes, zero writes
storage totals 4-12ms, byte-identical to an independent walk
/api/stats 7ms end to end
Two things fixed by re-reading my own code rather than by measuring it: writing
last_seen_scan for all 20,369 files every pass when the prune works from the set
of paths the walk just produced, and the missing schema-version guard. An earlier
19.9s warm figure quoted in the plan was a page-cache artefact and is corrected
there.
Verified in the wild as well as in tests: the index survived a full container
rebuild carrying its rows, the empty-index fallback returns correct figures from
the walk, and the schema guard genuinely discarded and rebuilt when bumped to v2.
Hooks are written and tested but deliberately not wired into the download paths
yet. Duplicate detection and the upgrade scan still walk the disk, untouched.
1126 fast, 1153 slow, 30 new unit tests.
Built with a human and an AI that has now read this library end to end more
times than anyone involved would consider healthy.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Phase 1: iter_library_audio_files now runs on os.scandir and yields a
LibraryFile(path, stat) rather than a bare Path. The directory entry already
knows whether something is a file, a directory or a symlink, so those three
questions are free, and one entry.stat() covers size, mtime and inode for
every consumer that used to go back and ask again. The per-file resolve() is
gone: symlinked directories are never descended into and symlinked files are
never yielded, so containment holds by construction rather than by a realpath
per file.
Measured on a real library, 1,675 files: 16,475 metadata operations down to
2,547, or 9.84 per file down to 1.52. All four scans that share the walker
benefit, the storage tile most visibly, since it was doing a full crawl plus a
stat per file every five minutes to produce two integers.
Phase 2: scan_user_library queues its row writes and flushes 500 at a time
instead of committing after every single file. One pass over the same library
went from 384 commits to 1. The subtlety is that a batched INSERT has no
rowid until it flushes and the end-of-pass prune deletes every row it did not
see, so capture_id collects lastrowid at flush time; there is a test standing
guard over that, because the alternative is inserting someone's entire library
and then deleting it again.
Combined with the mtime gate from the previous commit, a scan of an unchanged
library is 159.5s -> 34.2s and reads not one byte of audio.
Verified: 1096 fast, 1124 slow, one unrelated transient MusicBrainz 500 that
passes on re-run. Storage totals cross-checked against an independent walk and
agree to the byte.
Built with a human and an AI that has now taken the NAS's side twice in one day.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A retry always opens a new Queue row rather than reviving the failed one, which
is correct, but the failed card's own history panel was pulling the whole
target's saga regardless of whose job it was. Result: a card stuck on "Failed"
could show a later cycle's "completed" line under its own red status, which is
about as reassuring as a doctor telling you you're fine while holding your x-ray
upside down. Each card now only sees its own cycle and earlier, with a "Later
succeeded on retry" pointer straight to the job that actually landed it.
Also: the scoring-rationale Copy button silently did nothing if you hadn't
already clicked "Why this result?" first (it only ever read from a cache that
click filled in). It now fetches for itself and says something either way.
Verified live against the real container: seeded a target with a failed cycle
2 and successful cycles 3/4, confirmed the API scopes history correctly, and
walked the UI with Playwright (scoped card, working jump link, highlight,
auto-expand, cold Copy actually copying). No automated test coverage for the
scoping logic itself, since app.py runs real startup side effects on import
and this codebase deliberately doesn't unit-test route-adjacent helpers for
that reason; live verification stands in for it here.
Built with a human and a grumpy-AI combo.
fetch_artist_albums used to paginate every raw pressing MusicBrainz has
ever catalogued for an artist (100/page, forced 1s sleep between pages)
just to build a deduplicated album list. Browsing release-groups instead
gets the deduplicated list directly, so most artists load in one request
instead of several seconds of enforced waiting. The catch: a release-group
has no release ID of its own, so the specific pressing to download is now
resolved lazily, once, only for the album a user actually opens.
Also fixed a bug this would otherwise have introduced: Watched Artists'
auto-add-albums background job assumed every album always carried a
resolved release ID (true before this change, no longer true after), and
would have silently gone inert. Both its seeding and queueing paths now
resolve on demand too, same as the interactive browser, with new tests
covering exactly that.
Found by a human who noticed the album list was slow and asked "can we
lazy load this", and a very thorough AI who went looking for why and
kept pulling on the thread. Built with a human and a grumpy-AI combo.
Filename matching gets a metadata fallback (a cached whole-library Artist/Title
tag index) so MusicBee and beets can rename files without MusicGrabber ordering
a second copy. Watched playlists and watched artists each get their own
missing-file policy instead of inheriting the global setting blindly. Restoring
a rejected download from Trash now repairs its Queue row instead of leaving it
stuck failed. Append-playlist unresolved counts now include retained history.
Built with a human and a grumpy-AI combo.
- Queue history can now be brought forward by outcome while downloading and queued work stays pinned at the top.\n- Manual search downloads routed into watched playlists now link to their watched track as soon as they are queued, so Retry and Replace work immediately.
- Amazon Music album URLs finally resolve. Playwright never got past
twenty empty <music-image-row> skeletons on an album page, so
amazonalbum.py drives a SeleniumBase undetected browser instead and
just reads the document title Amazon kindly sets: "Play <Album> by
<Artist> on Amazon Music Unlimited". That's all the album pipeline
ever needed; MusicBrainz supplies the real tracklist.
- Amazon playlist scraper: strips a redundant featured-artist suffix
properly, and stops mistaking a cold container's empty first scrape
for an emptied playlist by reloading once before giving up.
- YouTube Music album links MusicBrainz can't confidently place now
fall back to importing as a playlist, same as they did before album
matching existed, instead of just dead-ending.
- delete_job_file's M3U rebuild now carries custom_subdir and user_id
through properly. "Add to album..." singles pick up a year from
.albuminfo instead of going tagless.
Built with a human and a suspiciously enthusiastic AI.
Issues #82 and #83:
- "Create M3U" off now actually means off. _append_to_physical_m3u checked
only that a playlist had been named, never the setting, so it held up
beautifully right until something downloaded. The file it should never
have created also drifted out of the upstream running order, because
appends land in download order while the rebuild it stood in for sorts
by position. The guard keys on playlist name and user rather than job id,
because the manual Search route never links its job to the watched row.
- Replace survives a download job that has been cleared. "Clear completed"
bins jobs rows and leaves watched_playlist_tracks.job_id pointing at the
gap, so the endpoint 404'd over a file that was sitting right there being
perfectly readable. It now falls back to the watched track's own
resolved_path, the M3U rebuild lookup has a fallback for the same cold
trail, and the toast stops blaming the trash for a missing database row.
Also carries the rest of the uncommitted v4.0.4 work, all of it already
written up under v4.0.4 (DEV): MP3Phoenix previews, the redownload-missing
-files setting, the desktop preview button, Amazon playlist ordering,
album release years, and the yt-dlp boot update that got YouTube previews
playing again.
Fast suite 1034 passed, slow suite 1064 passed, both against a local
container built from this tree.
Built with a human and a grumpy-AI combo.
Track Upgrades got a proper talking-to this round:
- Upgrading a track no longer stamps "data" over your Artist tag and a temp
folder name over your Album. The scratch staging dir was being read as an
Artist/Album path, which is certainly one way to rename U2, just not a
good one. Spotted by serguma88.
- New Quarantine panel: restore a swapped-out file (whatever replaced it
goes to quarantine in turn, so the undo is itself undoable) or bin it for
good, instead of rummaging through a hidden folder by hand.
- An upgrade that lands better but still below your target now stays on the
list and says so, rather than vanishing and sheepishly reappearing later
looking like it never happened.
- "Upgrade All" reports progress as it goes and refreshes the list itself,
instead of a toast politely asking you to keep hitting reload.
- Swap-in now pokes Jellyfin and Plex too, not just Navidrome.
Album URLs from YouTube Music (the OLAK5uy_ ones), Beatport /release/,
Tidal and Monochrome now reach the proper Albums pipeline instead of being
flattened into a track list or refused outright. All four verified against
the real services. Amazon resisted arrest: its album page is an empty app
shell and the existing Playwright scraper finds precisely nothing there, so
it stays on the backlog rather than shipping half-built.
Also in the box: Plex library scans after downloads, bulk imports that
manage two tracks at once, Soulseek checking the library before bothering a
peer for bytes it already has, emptying your own Trash without needing an
admin (it is your bin), and a test fixture that had been reading the wrong
config flag and quietly skipping every multi-user path it was meant to test.
Built with a human and a grumpy-AI combo.
perform_upgrade() used to insert a normal job row so it could reuse the
ordinary download pipeline, then delete it the instant the file landed,
"to keep the Queue clean". Handy for a rejected candidate, less so for a
successful swap: the one attempt you actually wanted a record of left no
trace anywhere, Queue included.
A completed upgrade now stays in the Queue like any other finished
download, pointed at wherever the swapped-in file actually ended up, and
wearing a small UPGRADE badge so it isn't mistaken for a fresh
acquisition. Rejected and errored attempts are still cleaned up as
before, since those were never real acquisitions. Re-download on one of
these rows is now turned away with a clear reason instead of silently
writing into a staging folder that was rmtree'd minutes earlier.
Built with a human and a mildly offended-on-your-behalf AI.
A backlog of finished work lands together on dev. The security hardening
pass (SSRF-proof URL-backed sources, path-traversal-proof custom folders,
per-account trash bins, Watched Artists checking the right library, Soulseek
no longer deleting a same-named stranger's file, Force Accept checking
ownership first), the Monochrome browser-fallback playback route with its
own Settings test button and working previews, and assorted Queue/album/
library fixes are all previously-written and already documented in
CHANGELOG.md, just not committed until now.
New on top of that: long-form YouTube splitting. Paste a single YouTube
video into Bulk Import's Fetch box and, if it runs over 12 minutes,
MusicGrabber checks it for YouTube chapters, then a cue sheet in the video's
own description, then one hunted out of the comments, before falling back to
manual entry, always landing on an editable start/end/title/artist table the
user confirms before anything downloads or gets cut. Confirming downloads
the audio once, cuts each segment with ffmpeg, runs the same integrity check
every other download gets, and tags directly from the table with no
automatic MusicBrainz guessing against noisy chapter titles.
Fast suite: 931 passed, 2 skipped.
Built with a human and a grumpy-AI combo who spent a worrying amount of that
time listening to a Red Jerry chillout mix for testing purposes, obviously.
Turns out warming the browser only got the preview as far as a confident 200
and a URL nothing could play. Monochrome's unified playback serves CENC
encrypted FLAC-in-MP4 at every quality tier it offers, so the audio element
was being handed a locked box and a cheerful smile. Downloads have always
had ffmpeg and the key for this; previews now do the same on a smaller
scale, decoding the first thirty seconds to MP3 and serving it back through
a uuid4 token that resolves to whoever asked. About two seconds hover to
sound, capped, aged out, and tidied up after.
The browser session is also warmed once at startup, but only when Monochrome
is actually an enabled source, because heating a browser for a source nobody
searches is an elaborate way to warm a container.
While in the neighbourhood: the search box's clear cross was pinned a fixed
distance from the right edge, measured for the word "Search". Say
"Searching..." instead and the button grew wide enough to sit right on top
of it. It now keeps one width whatever it is saying.
Fast suite: 825 passed, 2 skipped.
Built with a human and a grumpy AI who was quietly certain it had already
fixed this one, twice.
The tag editor's "Guess Again" button searched MusicBrainz by text alone
and sorted by raw relevance, with none of the alt-take protection ordinary
downloads already have. Real casualty: The Beatles' "Let It Be" guessed a
rehearsal take off the "Artifacts II" bootleg box set, complete with 1994
and track 11 of 23. It now shares the same canonicity-scored recording
picker an ordinary download uses, and tries fingerprinting the actual file
with AcoustID first when it's available, so a re-guess is grounded in what
the audio actually is rather than whatever a text search matched.
Also in this batch:
- Bulk Import's album browser note no longer touches the search box above
it, and stopped quietly rendering a duplicate of itself
- The Queue's live summary line now picks from a pool of quips per state
instead of reciting the same two forever
Built with a human and a grumpy-AI combo.
Use Selenium only for Cloudflare clearance, then hand search and downloads to curl_cffi with bounded refresh and fallback handling. Add container child reaping, source controls, tests, and Rise of the Phoenix documentation.\n\nBuilt with a human and a grumpy-AI combo.
Five things, all of which started as "hang on, why is it doing that?".
Soulseek result cards now show the artist, album, year, size, bitrate, cover
art and a match percentage, nearly all of which was already in the search
response and being quietly binned. Downloads can optionally move rather than
copy, verifying the arrival by size and SHA-256 before deleting anything,
because two copies of every FLAC adds up.
The path parser stopped guessing. A live "paramore - ignorance" search used to
credit its top five results to Paramore, Musique, FLAC Archive, P, and a
filename; one in five is not a hit rate, it's a coin toss with extra steps. It
now uses the fact that the search already told us who we wanted.
Album lists stopped hiding EPs behind the word "album", which had reduced Knife
Party's entire discography to one record and made clicking "Haunted House"
report that MusicBrainz had never heard of it. MusicBrainz had. We never asked.
The singles start date is now something you choose rather than something that
happens to you: switching Singles on asks for a date and saves nothing until
you say so, and the card lets you move it afterwards.
And the container finally carries a JavaScript runtime, because yt-dlp has been
telling us it wanted one and Thriller had been failing to download in protest.
Built with a human and a grumpy-AI combo, one "why is that folder called
Musique" at a time.
"Convert to FLAC" never meant FLAC. It meant "convert to whatever
AUDIO_FORMAT says", which through most of v3 quietly meant Opus. That is
how genuine-FLAC Monochrome downloads arrived as 265 kbps Opus while the
job row sat there insisting convert_to_flac = 1, next to an audio_quality
of "OPUS (from FLAC)". Both were telling the truth; only one of them was
being honest about it.
The flag is now convert_audio in the request models, the browser, every
API response, every internal function, and the jobs, bulk_imports,
watched_playlists and watched_artists columns. The stored setting becomes
default_convert_audio. Everything is renamed in place on first boot with
its values intact.
The column rename deliberately runs BEFORE the numbered migrations rather
than as a v13 block after them: the v1 migration rebuilds two of those
tables by listing their columns by name, and it still runs on a brand-new
database, so it has to find the new name already in place. The legacy jobs
ALTER is guarded for the same reason, otherwise it would bolt an empty
convert_audio onto an existing database and leave everyone's real settings
sulking in the old column.
Old spellings still work: convert_to_flac is accepted on every request and
still emitted in every response, and DEFAULT_CONVERT_TO_FLAC is still read
from the environment. Nobody's compose file changes behaviour on upgrade.
The rename is one-way, mind, so an upgraded database will not enjoy being
reintroduced to a v3 image.
Also in this batch, the Soulseek work already in flight: downloads now wait
in per-job staging while their embedded ARTIST tag is read, so a properly
tagged FLAC stops being filed under "Music (FLAC)", "complete", "Musique",
or the letter P.
Fast suite 659 pass, slow suite 685 pass. The migration was verified against
a real 80MB database, with per-flag row counts matching the pre-rename
backup exactly.
Built with a human and a grumpy-AI combo.
Second report from Syrius-consulting, and a sight to behold: a Paramore track
tagged as being by "2jqll9htuy62asp1wu", then filed in the library under that
name. The artist was in the file's own path the whole time:
soulseek://2jqll9htuy62asp1wu/@@kvkwm\Music\Paramore\Brand New Eyes\2. Ignorance.flac
slskd works that out correctly and keeps the artist and the peer in separate
fields. The API then threw the artist away, hardcoding it to None before handing
results to the browser, which fell back to "channel". For every other source
that is the uploader and a fair proxy for the artist; for Soulseek it is
whichever stranger happens to be sharing the file.
Longstanding rather than new, present since v3.0.3 at least. It was simply
unreachable while Soulseek search itself was broken, so fixing that uncovered
this. The download side now also refuses an artist that is merely the peer's
username, so retrying an older job cannot resurrect it.
While in there, the two places that shape search results for the browser were
merged into one. They were duplicated despite a docstring promising they stayed
identical, which is precisely how a fix gets applied to one and not the other.
Also raises the interactive search slot wait from 5s to 15s, this time measured
rather than guessed: a YouTube search takes ~2.5s on an idle box but 5 to 7s
under load, so 5s was almost exactly one search long and amounted to a coin
toss. Requests were giving up and retrying instead of simply queueing. The slow
suite has gone 4, 4, 2, 1, 0 failures across five runs while this was chased,
and is fully green for the first time.
Built with a human and a grumpy-AI combo, the AI having discovered that fixing
one bug is the most reliable known method of exposing the next one.
Three things that all turned out to be the same thing wearing different hats:
something was busy, we took that as a "no", and acted on it permanently.
Albums are now matched as a release against Deezer before any track is fetched,
so a track can only come from the record you actually asked for. RAYE has two
albums four years apart that both end on a track called "Fin."; searching per
track picked the wrong one, filed it as track 15, and reported success. Both
albums now resolve 17/17 and 15/15, with every ISRC agreeing with MusicBrainz
independently.
Searches queue for a busy provider instead of bouncing off it. Every source has
one admission slot, but it was grabbed non-blocking, which suited an impatient
user retyping a query and nobody else: a bulk import got an instant refusal
indistinguishable from "nobody has this", and wrote the track off for good. That
cost thirteen RAYE tracks that were sitting on Monochrome the whole time. Six
simultaneous searches for one source used to leave five empty-handed; they now
all return results, politely, one after another. Busy is reported as its own
state, so nothing is ever failed for a search that did not happen, and the
single-source endpoint answers with 503 and a Retry-After rather than a 500.
Bulk imports get a heartbeat and a maintenance sweep, because an import whose
worker died sat there claiming to be processing until the end of time. The test
box had 36 of them, the oldest insisting it had been busy since January. Stalled
ones are now picked up where they left off, restarts are handled at boot, and it
knows when to stop: three revivals, nothing older than a week, and nothing at all
for a watched playlist you already deleted (which was 32 of the 36).
Also: download cards stop crediting the source of the take that went in the bin,
artist search stops burying artists who spell their name in capitals, and the
"Duration" on a job card admits it was always how long the job took.
Built with a human and a grumpy-AI combo, the AI having spent the day being
told "busy" and taking it far too personally.
Monochrome's lossless downloads came back from the dead, and a database bug
wandered in and got fixed while we waited for the artist-centred restructure
to get going properly.
Added:
- Optional SeleniumBase browser fallback for Monochrome. Monochrome hid its
unified playback behind a Cloudflare Turnstile exchange, so the HTTP-only
legs were politely locked out. A short-lived headed Chrome session under
Xvfb now completes the same check a human gets and resolves one authorised
audio resource; the audio itself still downloads through the normal httpx
path. CENC AES-CTR FLAC-in-MP4 resources are decrypted and remuxed by
ffmpeg with the key held in memory only. Never launched for previews, one
session at a time, switchable off in Settings or via
MONOCHROME_BROWSER_FALLBACK_ENABLED. Image now carries Google Chrome on
amd64 and Debian Chromium on ARM, so it has grown a bit.
Fixed:
- Watched playlist counts now describe the playlist rather than its entire
life story. Departures are recorded in both sync modes, so "158 downloaded
of 150" is no longer a number anybody has to defend.
- "database is locked" no longer ambushes bulk imports. A half-read SELECT
left a stale WAL snapshot on a pooled connection, and the next write failed
instantly with SQLITE_BUSY_SNAPSHOT, which busy_timeout was never going to
help with. Connections are now cleaned on return, and lock errors get a
retry with backoff (DB_BUSY_TIMEOUT_MS to tune).
- Recording a bulk import failure can no longer become the failure itself and
take the rest of the import with it.
- The M3U toggle's tooltip now admits it stops updating the file rather than
deleting one already on disk.
Fast suite: 540 passed, 2 skipped.
Built with a human and a grumpy-AI combo, one of whom needed a lie down after
the WAL snapshot thing.
The headline features: opt-in ReplayGain 2.0 tagging that never re-encodes a
sample, a full-library Audio Provenance Audit that refuses to promote a file
out of "unknown" on vibes alone, MusicBrainz album URLs going down the proper
album pipeline, and a minimum-quality filter that is honest about what it
cannot know.
The fixes worth shouting about:
- Startup went from 30-odd seconds to under one. The connection pool treated
an empty pool as a busy one and waited out a 15 second timeout on the first
query of every boot, before uvicorn had even opened the port. A boot panel
now covers the remaining gap so an empty page stops looking like a broken one.
- Two downloads of the same track no longer sabotage each other. Every job
wrote its working files into the destination folder under the same name, so
concurrent jobs deleted each other's .part and .webp files mid-flight. This
had been filed as a flaky thumbnail CDN for months. It was never the CDN.
- Album selection stopped drawing raffle tickets: 3.7 of 31 test tracks landed
on the right album before, 26 after.
- Watched schedules learned to read a clock, so "every 30 minutes" means every
30 minutes rather than whenever tomorrow turns up.
- SMTP setup now tells you what actually went wrong, with a test button.
Built with a human and a grumpy-AI combo, the AI having been made to prove
every claim in here with a stopwatch and a live container.
Added:
- ReplayGain 2.0 track and album tags on new downloads, opt-in and
non-destructive; nothing is re-encoded, the player does the turning down
- Audio Provenance Audit under Watched: a read-only sweep of the whole
library that keeps stored format, recorded provenance and effective
quality as three separate ideas rather than one hopeful guess
- MusicBrainz release and release-group URLs now go down the album pipeline
- Minimum-quality filter on search results, honest about which sources
actually declare a quality before you download (spoiler: not YouTube)
Fixed:
- Monochrome downloads went from 37 seconds to 0.2. The shared qbdlx pool
holds 28 tokens of which 3 work, and they sit at the bottom, so we now
remember the one that delivered and try it first. A v9 migration retires
two default Qobuz proxies that were beyond saving: one answered with a
Cloudflare 522 after burning a 20-second timeout, the other has no DNS
record left at all
- Monochrome downloads landing as 30-second previews: sample-only tokens
are now skipped rather than treated as a finished lossless file
- Singles finally get their track number, search and fingerprint paths both
- Library scans stop counting everything you ever deleted, because NAS bins
live inside the share and we were walking straight into them
- Downloads land on the album they actually came from: 3.7 out of 31 on the
old raffle-ticket behaviour, 26 out of 31 now
- Romanised non-English titles find their metadata via aliases
Renamed "Results" to "Tracks", which is what it was always doing anyway.
452 passed, 2 skipped. Built with a human and a grumpy-AI combo, the latter
having spent an hour blaming Monochrome before noticing the connection pool
was the one dawdling.
Add adaptive source balancing, persistent health, targeted retries, automated search caching, source journeys, bounded provider calls, and the shared lossless Monochrome preview resolver. Built with a human and a grumpy-AI combo.
g33kphr33k