PATCH /location/{location} returned 404 for names like "Shelf 1/2",
even URL-encoded, because the path is decoded before routing and the
default converter stops at a slash. Use the path converter.
Backup folders on network shares often refuse SQLite's file locks (CIFS
without nobrl, NFS without lockd). SQLite reports a refused lock as
SQLITE_BUSY, so every backup to such a folder retried until the 60 second
give-up and blamed a locked database, while the live database was not
locked at all (#1246).
The target is a scratch .pending file that nothing else opens, so open it
with nolock=1. The give-up message now names the live database, since that
is the only file a backup can still wait on.
can_write_to_data_dir() touched and then deleted data/test.txt, which
silently wiped any file of the user's that happened to have that name.
It now uses tempfile.NamedTemporaryFile in the data dir, which picks an
unused name and removes only the file it created.
The metrics refresh only ever set label sets, never removed them, so a
spool that was archived or deleted (or a deleted filament, or a renamed
filament's old info labels) kept being exported with its last values
until the server restarted.
Each refresh now drops the label sets the database no longer has. The
current values are set before the stale ones are removed rather than
clearing the gauges first, because /metrics is served from a worker
thread and a scrape landing mid-refresh would otherwise see live series
briefly disappear.
sqlite3's Connection.backup() retries forever while the source is busy,
calling progress with 0 of 0 pages each time. The backup ran synchronously
on the event loop, so a lock that did not clear froze every request and
spammed "Copied 0 of 0 pages." into the log (#1191).
Run backups in a worker thread, give up after the source has been locked
for 60 seconds, and serialize backups now that they can overlap. Per-page
progress drops to debug.
measure() treated a spool_weight of 0 the same as a missing one and fell
back to the filament's tare, and since #1164 the vendor's. A refill wound
on no spool has a real tare of 0, so its scale reading was short by the
filament's spool weight. Only a missing tare now falls back.
The filament and manufacturer lists now sort by every field of the
filament or manufacturer, their custom fields included, plus spool and
filament counts, and offer the same filters as the spool view. A filter
on the spools themselves leaves out the filaments with none, as it does
there.
/spool/group takes the matching sort keys: filament.<column>,
filament.vendor.<column>, filament.extra.<key>,
filament.vendor.extra.<key> and group.filament_count. Keys it can't
resolve are still ignored.
Rows show the id as #id on the left like spool rows, and the Library
tab menu loses its descriptive sublines.
The Library gets two power-user sub-views, /?view=filaments and
/?view=manufacturers, reached from a menu on the Library tab, the search
panel and the manufacturer inspector. Rows are lists rather than tables so
translated labels and long names never widen them. The spool view is
unchanged.
The group endpoint gains what the lists need, all additive: a
group.vendor_name sort, include_empty for group_by=vendor, filament_count
on vendor groups, an opt-in preview of a group's first spools or
filaments, and X-Total-Spools / X-Total-Remaining-Weight headers.
The server checks the latest GitHub release in the background at startup
and every 12h, logs a warning when a newer version exists, and exposes
latest_version/update_available on /info. The client_v2 footer shows a
link to the new release. Disable with SPOOLMAN_UPDATE_CHECK=FALSE.
String filters split their value on every comma before checking for quotes, so an exact
match like location="Top shelf, Rack 3" became two fuzzy terms that matched nothing.
The dashboard and library filters always send quoted values, so a location with a comma
showed as empty. Split with a quote-aware helper shared by the built-in string filters
and extra-field filters. Unquoted values split exactly as before.
Fixes#1169
measure() walked the tare from the spool to its filament and then fell back to
zero, so a spool whose tare only exists on its vendor was recorded as if it had
no core at all. A filament copies its vendor's empty spool weight in create(),
which makes that inheritance a snapshot: a tare given to a vendor later never
reaches the filaments that already exist, and the value is then known one level
up while measure() treats it as absent.
With a vendor tare of 250 g, a 1000 g filament and a 704 g reading, the spool
came back with 704 g remaining and 296 g used instead of 454 g and 546 g.
The vendor is now selected alongside the filament in the query that already
runs, and its tare is the level below the filament's, in the same order
create() resolves the value in. The join is an outer join on purpose:
filament.vendor_id is nullable, and a filament without a vendor still has to
return its row, or measuring it fails with "Filament not found for spool".
Fixes#1117
The filament pickers only ever showed the first 8 SpoolmanDB matches, so entries further down a broad search could not be reached. They now load 20 at a time and fetch the next page when the result list is scrolled to the bottom.
GET /external/filament/search takes an optional offset and returns the total match count in x-total-count, like the other list endpoints. Requests without offset behave as before. The lowercased search text is built once per catalog load instead of on every search, since paging means one scan per page.
Fixes#1136
A tag can now be linked to a filament type, not just to one spool of it. The
tag table already carried filament_id and target_type for exactly this, so
there is no migration.
API, all additive:
POST /filament/{id}/tag link; 409 if anything else holds the UID
DELETE /filament/{id}/tag/{uid} unlink
GET /filament?tag=<uid> exact lookup on the normalized UID
Filament.tags also inside every spool's embedded filament
TagScan.matched_filament_id always present, like matched_spool_id
TagConflictMessage.filament_id names a filament holder on a 409
One UID still identifies one thing across both kinds: a spool's tag cannot be
linked to a filament or the reverse, and neither ?tag= lookup answers with the
other kind. The scan debounce key now includes the kind, since spool 7 and
filament 7 are different answers.
SpoolTag and SpoolTagParameters are renamed Tag and TagLinkParameters while
they are unreleased, so the schema does not describe filament tags as spool
tags forever.
client_v2: the filament inspector gets the Tags section, the link dialog names
and moves a tag held by either kind, and scanning a filament's tag opens that
filament when auto-navigate is on.
Tag having two delete-orphan parents was checked on SQLAlchemy 2.0.51: new tags
flush, unlinking deletes the row, and deleting either parent cascades.
Proven on sqlite, postgres, mariadb and cockroachdb: 558 integration tests
each. Unit tests, svelte-check and the Playwright tag specs pass.
The Add tag dialog waits for a fresh tap. That is fine when the tag is a
sticker in your hand, and awkward when the reader is a scale: the spool is
sitting on the pad to be weighed, and filling in the field means lifting it
and putting it back. A reader also cannot help here, because it has no reason
to report a tag that has not moved.
The registry already tracks each reader by id, name and last_seen, all in
memory. Keeping the last UID beside them is enough for a client to offer what
is on the reader right now, with no back channel to the device, no protocol
change, and nothing to update on the agent side - every existing ESPHome
reader gets this for free.
One UID, not a history: this answers "what is on the reader", not "what has
been". It lives and dies with the registry entry, so it is gone on restart
like the rest of it, and no scan is written anywhere.
The relay debounced on (uid, reader_id) and time alone, so a scan whose match
had changed looked like a duplicate of one that resolved to nothing. An agent
that taps an unlinked tag, links it in its own UI, then re-reports the scan had
its correction swallowed for the whole 3 s window, leaving every subscriber that
trusts the broadcast still showing an unknown tag.
The key now includes matched_spool_id, so a repeat is only a repeat while the
scan still resolves to the same thing. Repeats of a still-unlinked tag keep
collapsing on (uid, reader, None), which is the reason the debounce exists.
should_broadcast takes the match as a required argument rather than an optional
one so a future call site cannot quietly drop it again.
Fixes#1115
d43552b renamed the ORM class from SpoolTag to Tag and missed this annotation.
It is evaluated when the class body runs, so importing spoolman.api.v1.models
raised AttributeError -- the server could not start and every test in the tree
failed to collect.
Master split filter application in two: the filters a filament can answer on
its own (_apply_filament_filters) now serve both the spool query and the new
include_empty query, which selects FROM the filaments so that a filament with
no spools still produces a group.
The tag filter stays where it was, in _apply_spool_filters, and the comment
there now says why it cannot move: a tag is linked to a spool, so a filament
on its own can no more answer "which tag is on you" than it can answer "which
shelf are you on". Nothing routes a tag filter into the filament-driven query
-- it never calls this builder, and the group endpoint takes no `tag` -- so
there is no silently-dropped filter to guard against.
This reverts 9aea271.
The column bought one thing, that phase 2 would need no migration, and
cost something worse: it commits the schema to a shape nobody has run a
decoder against. Width, and above all the choice not to make it unique,
were read off a design document rather than off working code, and
goeland86's codec in #880 computes an effective_instance_uuid that may
not mean the same thing. Getting that wrong leaves us altering a
populated column on four databases, which is the exact operation the
"settle the shape before release" rule exists to avoid.
That rule earns its keep for target_type, target_value and filament_id,
where retrofitting a discriminator or a nullability change is painful.
Appending one nullable column later is not painful, so phase 2 can add
it when there is something to write into it.
A sqlite3.Connection used as a context manager is a transaction manager: it
commits or rolls back on exit, it does not close the connection. Both handles
stayed open past the `with` block in Database.backup().
backup_and_rotate() then unlinks and moves the .pending file it just wrote.
POSIX allows unlinking a file with open handles, so this was invisible on
Linux, but on Windows it raises WinError 32 and breaks both the scheduled
midnight backup and the manual backup endpoint. It also leaked a pair of
connections per backup on every platform.
Wrap both connections in contextlib.closing instead. Connection.backup()
commits on the destination internally, so nothing is lost by dropping the
transaction-managing context manager.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
OpenPrintTag derives an instance_uuid from the brand uuid plus the tag's
hardware UID, and phase 2 needs it stored so that rescanning a tag updates
the spool it already created instead of making a second one.
Added now, into the migration that has shipped to nobody, because the
table is the expensive half to change once it is populated on four
databases. Nothing writes the column yet, which is the same position
filament_id is already in.
Indexed for that lookup and deliberately not unique. uid already enforces
that one physical tag identifies one thing, and copying a tag's payload
onto a second sticker can legitimately put one instance identity on two
rows pointing at the same spool, which a unique constraint would reject.
The first cut of include_empty grew a second 177-line function next to
find_groups that repeated its filters, ordering, pagination and hydration, so
every future change to grouping had to be made twice or silently diverge.
Only the FROM ever differed. Writing the aggregates so they hold for a group
with no spools in it -- COUNT over a column instead of COUNT(*), one explicit
zero for the one SUM -- removes the need for a second set, and with it the
COALESCE that CockroachDB rejected and the CAST that worked around it. What is
left is one query with a branch over how its rows are selected, and one shared
tail.
Spool-level filters are now refused alongside include_empty rather than quietly
producing a contradiction (every filament as empty, or no empty groups at all).
That also leaves the archived rule as the only spool-level condition the
filament-driven query has to carry, in its JOIN.
Group ordering now breaks ties on the grouped column. Every ordering can tie,
and empty groups tie on all four aggregates, so paging could otherwise repeat or
drop a group between pages.
Grouping by filament aggregates over spools, so a filament you own no spools of
could never produce a group and simply vanished from the library. A filtered
view then looked fully stocked when it was the opposite, and the thing you
needed to re-order was the one thing not on screen (#1092).
GET /api/v1/spool/group takes include_empty, which inverts the query: spools are
aggregated per filament in a subquery that the filament list is LEFT JOINed
against, so filaments with no matching spools survive with their aggregates
folded to zero. Which side of that join a filter goes is the whole subtlety, and
the docstring spells it out - spool-scoped conditions have to stay in the
subquery or they delete the very rows the flag exists to keep.
The client asks for it while grouping by filament and every active filter is one
a filament can answer on its own. "Which of these are on Shelf A" has no answer
for a filament that is nowhere, and including them there would bury the shelf
under the whole catalogue. Such a group renders as an out-of-stock row: no
disclosure, no fetch, and a marker in place of the zeroes it would otherwise
report.
The group header also picks up the truncation tooltip added in the previous
commit. Its title sits under a link stretched across the whole row, so the
pointer never lands on the text itself and the tooltip has to hang off the row.
A tag maps a UID to "the thing to bring up", and only some of those things are
spools -- or rows at all. Locations are the case that settles the shape: a
location in Spoolman is a String(64) on Spool, not a table, so a tag meaning
"show me Shelf A" can never be a foreign key.
Getting this wrong is asymmetrically expensive. Renaming the table, dropping
spool_id's NOT NULL and adding a discriminator all mean altering a populated
table on four databases; adding an endpoint or a target kind later is additive
and free. The migration has shipped to nobody -- it is not on master and
v0.26.1 predates it -- so the schema is settled now and the unshipped revision
is rewritten in place rather than chased with a second one.
spool_tag becomes tag:
uid unchanged, still globally unique -- one physical tag, one thing
target_type which kind of target is in force, rather than leaving readers
to infer it from whichever column is non-null
spool_id nullable now; targets that ARE rows keep real foreign keys, so
filament_id the database still cascades deletes and refuses orphans
target_value targets that are not rows; a new kind of this sort costs no
migration at all, only a new target_type string
Exactly-one-target is enforced in database/tag.py rather than by a CHECK: no
migration in this tree uses one, and MySQL below 8.0.16 parses and silently
ignores them, so a constraint here would be real on three databases out of
four. Single write path beats a constraint that is only sometimes a constraint
-- the same argument, in the same module, as normalize_uid.
filament_id gets a column and a foreign key but no ORM relationship: nothing
writes filament tags, and a selectin collection on Filament would add a query
to every filament listing for rows that cannot exist. ON DELETE CASCADE holds
integrity until the feature that needs the relationship arrives.
No API surface changes. POST /spool/{id}/tag, ?tag=, the spool response's tags
and matched_spool_id are all exactly as they were; the reads are just explicit
now about wanting a spool rather than assuming the tag they found is one.
TagConflictError's spool_id becomes optional for the same reason, which the
client already handled -- it only offers "move it here" when there is an id.
Proven on sqlite, postgres, mariadb and cockroachdb: 526 integration tests each.
The field is typed str with default=None, which is a default the type does not
allow. Nothing relied on it: the backup endpoint returns 500 when there is no
path and only constructs the model with one. Dropping the default makes the
documented response schema match what is actually sent.
These descriptions ship in the public OpenAPI docs, where an integrator wants
the rule rather than the reasoning. Says the UID is case insensitive and that
separators are optional, and shows both spellings as examples, instead of
explaining why normalization exists.
register() kept datetime.utcnow() while should_broadcast() and readers() moved
to datetime.now(tz=utc), so the naive last_seen it stored was compared against
an aware cutoff during pruning: GET /tag/reader raised "can't compare
offset-naive and offset-aware datetimes" as soon as any reader had scanned.
The existing unit tests could not catch it because they inject `now` into every
call. Added one that exercises each method's own clock instead.
Also adds design/nfc-phase-1-frontend-handoff.md: the API shapes the client
will consume, the behaviours it has to get right, two stale paths in the phase
1 doc's frontend plan, and the open UI decisions.
Implements the backend half of design/nfc-phase-1-tag-identity.md. Format
agnostic: it keys only on a tag's hardware UID, so it works with blank NTAG
stickers, Prusa NFC-V tags, and the Bambu/Creality/Qidi MIFARE tags whose
contents Spoolman can never decrypt.
Answers the question the whole external ecosystem has been working around:
which spool is this physical tag? Every project so far invented its own way to
smuggle a Spoolman id into the tag contents, which fails the moment the tag is
a vendor tag you cannot write to.
Data model: new spool_tag table, additive only, with a unique index on uid.
Many tags per spool, since copying a vendor tag onto a blank sticker leaves one
spool with two UIDs. UIDs are normalized to uppercase hex with separators
stripped, in the database layer rather than a Pydantic validator, so the unique
constraint holds for callers that never arrive over HTTP.
API, all additive:
GET /spool?tag=<uid> exact match on the normalized UID
POST /spool/{id}/tag 201, 409 with the conflicting spool id
DELETE /spool/{id}/tag/{uid} 204
POST /tag/scan broadcast and return the match
GET /tag/reader readers seen recently
WS /tag/scan[/{reader_id}] the relay
Link and unlink emit the existing spool event, so clients need no new wiring.
Spool responses gain a tags array.
The relay decouples where the reader is from where the user is looking: an
agent POSTs a scan, a paired browser reacts. Scans are ephemeral, never
persisted, and go through a SECOND WebsocketManager instance rather than the
shared one. The shared tree propagates to subscribers at every level, and the
root /api/v1/ pool means "any change", so putting scans there would inject a
novel resource into every existing consumer's stream. An integration test pins
that they never reach the entity websockets.
Repeated identical (uid, reader_id) scans are debounced before the broadcast
only, since readers re-read a tag that is sitting still. The response always
carries the match, so a de-duplicated scan never looks to a device like a
failed lookup.
Migration creates the table and its indexes in one revision, following the
extra-fields tables rather than the doc's suggested split. The DDL/DML split
cockroachdb needs does not apply when there is no data migration, and the 4-DB
suite confirms it.
Verified: 506 integration tests on sqlite, postgres, mariadb and cockroachdb;
263 unit tests; ruff clean.
Frontend, docs, and the #783 action links are not included.
SubscriptionTree.send let a per-socket failure escape the broadcast loop, so
every subscriber after the failing one silently received nothing. A socket can
start closing between the CONNECTED state check and the send, and starlette
then raises "connection is closing"; on the shared tree that means one browser
tab closing could stop Moonraker hearing about a spool.
Each subscriber is now isolated: a write failure drops that socket and the
broadcast continues. Iterate a snapshot too, since dropping mutates the set.
Dropping also had to stop going through remove(path, ...). By the time send()
is walking a node, path is the part of the route still to be traversed, not the
pool the socket subscribed with, so remove() descended into a child and looked
in the wrong set: the dead socket stayed put to fail again on the next event.
Observed in the integration suite as a subscriber missing an event it was
subscribed to, alongside "Failed to send websocket message" on an unrelated
filament broadcast.
The previous commit claimed the built-in timestamps and the datetime extra
fields share a grammar, then wrote that grammar out twice: once in
add_where_clause_datetime_opt for the typed columns, once inline in
add_where_clause_extra_field for the JSON-stored ones. Two copies of a
grammar drift, which would quietly falsify the thing the change was for.
The `<start>|<end>` split and its "at least one end" rule now live in
split_datetime_range_filter, which both call.
Only the parsing is shared, because only the parsing can be: an extra
field is JSON text in a joined key/value table, reached through a subquery,
while a built-in is a typed column on the row. The comparison that follows
the split therefore stays separate, and the two are not equivalent. The
built-in path parses the bound and compares real datetimes; the extra-field
path compares decoded ISO strings, so an offset or a missing '.000' changes
the answer there and not here. That asymmetry is now written down at both
sites, and tracked in #1057.
The two commits before this one invented `<field>_after`, `<field>_before`
and `<field>_unset` for the built-in timestamps. That was a third way to
ask a question this API already answers: extra-field filters have taken
`<start>|<end>` for datetimes and `<min>:<max>` for numbers since v0.26.0,
with an empty value meaning "unset", the same as it means for `location`
or `lot_nr`.
Two grammars for one question is bad on its own, but the split was worse
than cosmetic. Someone who learned `extra.opened_on=2025-01-01T00:00:00Z|`
and tried `last_used=2025-01-01T00:00:00Z|` got every spool back, because
an unknown query param is silently ignored.
So the eight parameters become three, named after their columns and
taking the shipped grammar:
/spool?last_used=2026-08-01T00:00:00Z| since
/spool?last_used=|2026-01-01T00:00:00Z until
/spool?last_used=<start>|<end> between
/spool?last_used= never used
Commas OR several of those, and quotes are accepted and ignored, so every
filter on the endpoint is spelled the same way. `_unset` disappears: the
empty value already means unset everywhere else here. No new convention
enters the API, and the next filterable column costs one parameter rather
than three.
Two consequences worth naming. Malformed input is now a 400 with a message
instead of FastAPI's automatic 422, matching the other unparseable filters
on this endpoint. And "has a timestamp" (the old `_unset=false`) has no
spelling any more, because the shipped grammar has none; nothing offered
it in the UI.
The client is unchanged apart from how it builds the request. The chip
value in the app URL keeps its own grammar, since it also carries relative
offsets and "never", neither of which the API takes.
A date range cannot express "no date at all": a NULL fails every
comparison, so no bound, however far back it reaches, selects the spools
that have never been used. That state needed a filter of its own.
Backend: /spool and /spool/group take first_used_unset / last_used_unset.
True matches only the spools missing that timestamp, false only those
that have one. Registered gets no such param — the server sets it at
creation and a spool never lacks one.
Client: the chip value `never` sits alongside the range grammar, and
Last Used / First Used offer it under their presets. It is labelled
"Never", deliberately not "unused": the list already calls a spool unused
when no weight has come off it, which is a different claim — a spool can
be unused and still carry a last_used date, or have used weight and no
date at all.
The library could filter by every discrete property a spool has but by
none of its timestamps, so "what have I used this week" and "what has
been sitting untouched for a year" were questions the UI could not ask.
Backend: /spool and /spool/group take first_used / last_used /
registered `_after` and `_before` bounds, each optional, inclusive, and
resolved against the UTC-naive form the columns store (an offset-aware
value is converted; a naive one is read as UTC). A spool with no
timestamp matches neither end.
Client: the Filter menu gains the three timestamps, each opening presets
(last 24 hours ... older than 1 year) plus a custom range whose empty
end means open. A range lives in the URL as a chip value, `<from>..<to>`,
where an end is a calendar day or an offset back from now. Relative ends
stay relative all the way into the URL and are resolved at request time,
so a bookmarked "last 24 hours" keeps meaning the last 24 hours instead
of drifting into "since the day I made it".
Fixes#1019. A dashboard card grouped by a custom spool field counted more
spools than it listed: the group aggregate summed them, the card's own query
returned a subset.
Grouping and filtering disagreed about what "no value" means. group_by keys on
the decoded value, so a spool holding the JSON empty string a cleared text box
writes became its own group, separate from the null one; the client renders
both as "unassigned" and adds the counts. The card then fetches its spools with
`extra.<key>=`, whose empty branch matched only NULL and JSON null - never the
empty string. Built-in string columns have matched both NULL and '' since
forever (add_where_clause_str_opt), which is why grouping by location was fine.
- The empty extra-field filter matches a stored empty string too.
- Value-keyed group axes (extra fields, location, material) fold a blank into
NULL, so "unassigned" is one group rather than two the client can only draw
as one.
- Emptying a text extra field clears it instead of storing "", so no new spool
lands in the state above.
- The dashboard clears an extra field with a real JSON null. It sent the string
'null', which the API rejects as not-a-string, so dragging a spool onto the
unassigned card 400'd on every extra-field board.
A spool can hold its own full weight, empty-spool (tare) weight and price,
falling back to its filament's where it has none. The API has had all three
since #339, and the v1 client exposed them, but client_v2 never mapped
spool_weight at all and had no field for it or for the full weight.
The result was silent wrong arithmetic (#1013): the API gives every spool it
creates a copy of the filament's tare weight, so correcting the *filament*
later looked like it had worked — the spool panel showed the new figure — while
"adjust by measured weight" went on subtracting the old one. A reporter's empty
261 g spool came back with 129 g still on it, exactly the difference.
Both ends of a disagreement are now marked. The spool's field says what it
shadows, the filament's row says what applies instead and is struck through,
and the same treatment covers the manufacturer's empty-spool weight, which
seeds a filament and is then left behind. Only where the values actually
differ: merely having one of its own is already visible in the field, and
marking every spool would teach everyone to stop reading the marks.
Also fixes a 500 from PUT /spool/{id}/measure when nothing in the chain has a
tare weight — reachable in one click now that the field can be cleared.
Dev M