Commit Graph
95 Commits
Author SHA1 Message Date
Donkie 6ade40517f backend: notify spools when their filament changes
Spool responses carry fields derived from the filament (remaining_length,
plus the initial_weight/price fall-backs). Editing a filament emitted only a
filament WS event, so subscribers to that filament's spools kept stale derived
values until each spool was next touched — visible in client_v2 as a length
inconsistent with the displayed density, but affecting every consumer.

filament.update now re-broadcasts the filament's spools (archived included)
as spool "updated" events after commit. Adds a WS integration test asserting
a density patch reaches /api/v1/spool with the new density and changed length.

Claude-Session: https://claude.ai/code/session_014Cwk5Yyze9yPvkYkUtMAP4
2026-07-21 10:58:09 +02:00
Donkie 593b653841 Match every whitespace-separated term in search
The search treated the whole query as one substring match, so "bambu
petg-cf" had to appear verbatim in a single field and returned nothing,
even though "bambu" and "petg-cf" each matched on their own. Vendor-name
matching also ran as a separate query, so a term could never be satisfied
by the vendor while another was satisfied by the filament.

Split the query on whitespace and require every term to match some
searchable field, letting different terms match different fields:

- Filaments search vendor.name in the same pass as their own fields via an
  outerjoin, replacing the separate vendor query. This is what makes
  "bambu petg-cf" work.
- Extra-field matching moves from a follow-up query to a correlated EXISTS
  inside each term's OR-clause, so extras participate in the AND too.
- Ranking generalizes to a per-field-kind base rank (native 0, vendor 2,
  extra 4) with prefix hits beating substrings. A row ranks by its worst
  term, so rows matching everything strongly sort first; the reported
  match_field is the single best hit.

Single-term queries behave as before. Standard SQL only (ilike, correlated
EXISTS), so this holds on all four supported databases.

Claude-Session: https://claude.ai/code/session_014Cwk5Yyze9yPvkYkUtMAP4
2026-07-19 09:31:02 +02:00
Donkie 9b16121ed2 Minor search backend issue fix 2026-07-18 15:55:06 +02:00
Donkie d7486d5d51 Improved filtering/sorting. Allows filtering/sorting by extra fields. 2026-07-18 15:54:58 +02:00
Donkie 7ea89dc347 Add cross-entity search to v2 client
Implements the search feature for client_v2: a new backend search endpoint
plus a results-panel UI, replacing the old query box that only filtered the
library list by filament name.

Backend (additive, API v1 backward compatibility preserved):
- New GET /api/v1/search returning categorized spool/filament/vendor results,
  each annotated with the field that matched, plus an is_color_query flag.
- Case-insensitive text search over native text fields and text/choice extra
  fields; exact spool-id match for numeric queries; color-similarity search
  (CSS color name or hex) via find_by_color, ordered closest first.
- spoolman/colors.py resolves the CSS4 color names / hex codes to hex.
- Integration tests (run across SQLite/PostgreSQL/MySQL/CockroachDB in CI).

Frontend (client_v2):
- SearchBox overlay with Spools/Filaments/Manufacturers sections, per-result
  match badges, keyboard navigation, a color-similarity threshold slider
  (persisted to localStorage), and an English-only color-name hint.
- Clicking a result opens the matching inspector on the Library page.
- Removes the old q -> filament.name inline list filtering.

Claude-Session: https://claude.ai/code/session_01Ne7RsfHejKZX97uAq3aVMn
2026-07-13 19:33:52 +02:00
Donkie c82e3911e1 WIP: add v2 client (Svelte) with spool grouping backend
Introduces client_v2, a new SvelteKit-based client, alongside the
backend it depends on: a /spool/group endpoint that returns
per-group aggregates (count, in-use count, remaining weight,
last-used) grouped by filament, vendor, material or location, so the
client can paginate whole groups instead of fetching every spool.

Not yet complete/wired up for release.
2026-07-12 19:21:34 +02:00
470d8f3bcc Add filtering and sorting for custom fields (#773)
* Add filtering and sorting for custom fields

* Fix tests and make code postgres compatible

* Fix bugs and expand test coverage for custom field filter/sort

* trigger CI

* fix: address extra field filter review feedback

* fix: clean up custom field table state handling

* docs: update extra field table view description

* fix: narrow boolean custom field filter syntax

* Expand test coverage for all custom field types/entities and fix range filter/sort

- Add comprehensive tests covering all 9 field types (text, integer, float,
  boolean, single-choice, multi-choice, datetime, integer_range, float_range)
  for filter and sort on spool, filament, and vendor entities
- Add invalid-filter 400 tests for float, integer_range, and float_range
- Fix integer_range/float_range filter to use LIKE pattern matching against
  Python's deterministic json.dumps output instead of fragile .contains()
- Add integer_range/float_range sort support via a @compiles helper
  (_JsonArrayFirstElement) that emits CAST(col AS JSON)->>0 on PostgreSQL
  and JSON_EXTRACT(col, '$[0]') on SQLite/MariaDB
- Add logger and __all__ to extra_fields.py (aligns with PR #893)
- All tests verified passing on postgres, sqlite, and mariadb

* Add filter UI for all custom field types and fix range filters

- Add text input filter dropdown for text fields (substring search)
- Add range input filter dropdowns (min/max) for integer, float, integer_range, and float_range fields
- Add datetime picker filter dropdown for datetime fields
- Fix boolean "No" filter to use empty value so it correctly matches unset/false fields
- Fix integer_range/float_range filter to use numeric comparisons instead of LIKE-based exact matching (stored_min >= filter_min, stored_max <= filter_max)
- Add range filter support for integer/float fields (min:max format)
- Add _JsonArraySecondElement cross-database helper for extracting second JSON array element

* Range filter for integer/float fields and fix range_field filter semantics

Backend:
- Add _JsonArraySecondElement cross-database helper (mirrors _JsonArrayFirstElement)
- integer_range/float_range filter now uses numeric comparisons:
  stored_min >= filter_min and stored_max <= filter_max (was LIKE exact match)
- integer/float filter now supports min:max range format in addition to exact match:
  stored_value >= min and/or stored_value <= max

Frontend:
- integer and float fields now use a range filter dropdown (min/max inputs)
  instead of a single exact-value input

Tests:
- Fix integer_range/float_range spool and vendor tests broken by new >= semantics
  (filter values updated to discriminate between test entries)
- Add range filter tests (min only, max only, both) for integer and float fields

* Replace 12 copy-pasted numeric field tests with 4 parametrized tests

Introduce a _create_entity helper and @pytest.mark.parametrize("entity_type",
["spool", "filament", "vendor"]) to run each numeric field type test against
all three entity types from a single test function.

- Removes 12 individual tests (integer/float/integer_range/float_range × 3 entities)
- Adds 4 parametrized tests covering the same ground plus the previously missing
  min-only / max-only range filter cases for filament and vendor
- Uses try/finally for cleanup so entities are always deleted even on assertion failure

* Add datetime range filter and restore integer/float range filter support

- Backend: add datetime range filter using '|' separator (ISO dates contain ':')
- Backend: restore integer/float range filter support (lost during attribution rewrite)
- Frontend: replace single DateTimePicker with From/To range pickers for datetime fields

* Use local timezone in filter picker; add range filter tests

The DateTimeRangeFilterDropdown was initializing pickers with dayjs.utc(),
causing them to display and accept UTC times directly. The entry form shows
local time and converts to UTC, so the filter was inconsistent: entering
the same displayed time would produce a filter value 2h offset from what
was stored (in UTC+2), making exact boundaries fail unexpectedly.

Fix: initialize picker values with dayjs() (local mode) so the filter picker
behaves the same as the entry form — shows local times and converts to UTC.

Tests: replace 3 separate entity-specific datetime tests with one parametrized
test covering spool/filament/vendor, and add range filter cases (start|, |end,
start|end).

* Replace all entity-specific tests with parametrized tests

Replace 16 separate spool/filament/vendor tests for text, boolean,
single-choice, multi-choice, and empty-filter with 5 parametrized tests
that run against all three entity types. Each test is now defined once
and executed 3 times, eliminating copy-paste and ensuring consistent
coverage across all entity types.

Parametrize invalid-filter-value 400 tests across all entity types

The non-numeric-value error uses "Invalid integer/float range filter value"
while the missing-colon error uses "Invalid range filter value". Both contain
"range filter value", so use that as the assertion substring.

* Align custom filter dropdowns with Ant Design choice filter style

All three custom filter dropdowns (text, number range, datetime range) now
match the built-in choice filter footer: a border-top separator, Reset as a
link button on the left (disabled when no filter is active), and OK as a
primary button on the right.

* fix(fields): correct custom-field filter edge cases across databases

- Float exact-match filters now compare numerically (cast to Float) instead of
  by JSON string, so float fields stored as integer JSON (e.g. "2") or
  non-canonical decimals (e.g. "2.50") match an equivalent filter.
- integer_range/float_range filter and sort no longer return HTTP 500 on
  CockroachDB: the PostgreSQL '->>' JSON extraction is now also registered for
  the cockroachdb dialect, which previously fell back to the unsupported
  JSON_EXTRACT function.
- Text exact-match and single-choice equality use ensure_ascii=False so
  non-ASCII values match how the frontend's JSON.stringify stores them.

Co-authored-by: Dieter Blomme <dieterblomme@gmail.com>

* test(fields): expand custom-field filter/sort coverage

Add tests covering filter composition (multiple extra fields, extra + built-in,
filter one field / sort another), pagination total-count, unknown-field
handling, text case-insensitivity/substring/non-ASCII, numeric multi-value OR,
boolean empty semantics and token rejection, multi-choice substring collisions,
empty filters on numeric fields, null-bounded ranges, and additional
invalid-value 400 paths.

Verified against sqlite, postgres, mariadb and cockroachdb.

Co-authored-by: Dieter Blomme <dieterblomme@gmail.com>

* style(client): format column.tsx with prettier

The custom-field filter dropdowns' Reset buttons were written as one-line
onClick handlers that prettier wraps across multiple lines. Apply prettier
so the CI style check passes.

Co-authored-by: Dieter Blomme <dieterblomme@gmail.com>

* fix(fields): decode JSON scalars in-DB for text/choice/datetime custom-field filters

Match text, single-choice and datetime custom-field filters against the
database-decoded JSON scalar (new _JsonScalarText cross-dialect helper:
#>> '{}' on postgres/cockroachdb, JSON_UNQUOTE(JSON_EXTRACT) on mysql,
json_extract on sqlite) instead of reconstructing the exact JSON
serialization the client wrote. This decouples matching from
json.dumps/JSON.stringify encoding quirks (non-ASCII escaping, surrounding
quotes). Also escape LIKE wildcards so '%' and '_' in text and multi-choice
queries match literally rather than acting as wildcards.

Verified against sqlite, postgres, mariadb and cockroachdb.

Co-authored-by: Dieter Blomme <dieterblomme@gmail.com>

* test(fields): cover LIKE-wildcard escaping in custom-field filters

Add parametrized tests (spool/filament/vendor) asserting that '%' and '_'
are matched literally in text and multi-choice filters, that a literal '/'
(the internal LIKE escape char) matches literally, and that single-choice
equality handles a '%' value exactly.

Co-authored-by: Dieter Blomme <dieterblomme@gmail.com>

---------

Co-authored-by: Donkie <daniel.cf.hultgren@gmail.com>
Co-authored-by: akira69 <akira69@gmail.com>
2026-07-08 10:14:27 +02:00
Donkie da54ead06f Bump deps to clear Dependabot alerts (path-to-regexp, pytest)
- path-to-regexp: force >=8.4.0 (8.4.2) under @ant-design/pro-layout via a
  scoped npm override, fixing the ReDoS advisories (GHSA sequential optional
  groups + multiple wildcards). Express keeps its own 0.1.x.
- pytest 8.3 -> 9.1.1 and pytest-asyncio 0.23/0.25 -> 1.4.0 in
  tests_integration/requirements.txt and the dev group; regenerated uv.lock.

Verified: client build + typecheck pass, npm audit clean, and the full
integration suite (223 tests) passes on postgres, sqlite, mariadb, and
cockroachdb under podman.

Claude-Session: https://claude.ai/code/session_01BQjJiwEbu6bHbeDjPPDm3V
2026-07-06 20:39:46 +02:00
Donkie e3c5f3fc1d Make the integration test suite runnable under Podman
Running tests_integration against rootless Podman surfaced three issues; none
affect the Docker/CI path:

- run.py: add a SPOOLMAN_CONTAINER_ENGINE env var (defaults to "docker") so the
  build/compose commands can be driven by podman instead.
- Dockerfile: drop `--chown=app:app` from the builder-stage COPYs. The "app"
  user only exists in the runner stage; Docker/BuildKit tolerated the dangling
  reference but Podman rejects it. Final ownership is unchanged — it is set when
  the files are copied into the runner stage.
- docker-compose-postgres.yml: add a pg_isready healthcheck and gate spoolman on
  `condition: service_healthy`, matching the mariadb/cockroachdb compose files.
  Without it spoolman raced postgres' initdb and died on connection-refused
  before running migrations.

Verified: full 4-DB suite (sqlite/postgres/mariadb/cockroachdb) green via
`SPOOLMAN_CONTAINER_ENGINE=podman uv run poe itest` — 223 tests each.
2026-07-06 12:16:17 +02:00
Donkie 0f5775c2d4 Enforced eslint and prettier, added lefthook config 2026-01-07 23:02:32 +01:00
Donkie 70ea7ad414 Update python dependencies and ruff/black versions 2025-03-16 21:34:02 +01:00
Donkie 902e98b059 Increased itest connection timeout 2024-08-07 15:27:22 +02:00
Donkie 3454002c09 Fixed conftest 2024-08-07 15:19:35 +02:00
Donkie 515722bf4f Attempt to make itest spoolman up check more robust 2024-08-07 15:12:42 +02:00
Donkie efa93615ec Updated dependencies in integration tests 2024-08-07 14:45:43 +02:00
Donkie cc7dceb2a3 Fixed backend not supporting editing multi color dir 2024-06-08 22:50:01 +02:00
Donkie 8c164ac55f Added backend support for multi-color filaments 2024-05-28 18:49:24 +02:00
Donkie b00075ab87 Added backend support to perform exact matches 2024-05-25 20:56:22 +02:00
Donkie c851950701 Tests fully pass again 2024-05-23 20:13:51 +02:00
Donkie 535fa40ad2 Updated most python dependencies
Primarily FastAPI and Pydantic to v2. Also ruff to latest.

Updated some code to support Pydantic v2
2024-05-23 19:42:27 +02:00
Donkie c4693d3f34 Implemented used_length spool sort 2024-05-15 20:18:24 +02:00
Donkie e4d2ecf0dc Filament/vendor find by ext id + integration tests 2024-05-12 21:58:01 +02:00
Matt Gerega fdec032d19 Updated handling of higher measurements 2024-04-15 16:41:14 -04:00
Matt Gerega c6f9abeced Changed initial_weight to be net weight, not gross weight 2024-04-10 11:00:46 -04:00
Matt Gerega 23cfb42ba8 Updated based on PR requests 2024-04-08 17:38:18 -04:00
Matt Gerega 951592ab4c Add more integration tests for spool weights 2024-03-28 09:38:24 -04:00
Matt Gerega a6ae808389 Added additional integration tests for measure and add 2024-03-28 09:29:39 -04:00
Matt Gerega 19c414da4b Add integration tests for new measure endpoint 2024-03-27 15:44:54 -04:00
Hereward Cooper c0714e053d change test var name 2024-01-30 14:09:32 -08:00
hcooper 1273b20224 Support dynamic ports 2024-01-23 11:16:57 -08:00
Donkie f8cc9f7eda Updated integration tests to not require exact match
This matches how we want clients to behave, they should ignore any extra fields that are sent in the response.
2024-01-23 18:58:58 +01:00
Donkie 7972f4d401 Update test_add.py 2024-01-23 18:58:58 +01:00
Donkie 1caae75d81 Update conftest.py 2024-01-23 18:58:57 +01:00
Donkie 4adee1327b Ruff fix 2024-01-23 18:58:57 +01:00
Donkie a868e1889f Added integration tests for using extra fields 2024-01-23 18:58:57 +01:00
Donkie 36a346bf64 Added restrictions on changing existing fields 2024-01-23 18:58:57 +01:00
Donkie 86e126f775 Added some basic field CRUD tests 2024-01-23 18:58:57 +01:00
Donkie 77ac9d6a5c Added custom extra fields to backend
No integration tests yet, TBD
2024-01-23 18:58:57 +01:00
hcooper ccfec28bcd Switch from "docker-compose" to "docker compose"
"docker compose" is the modern usage.
2024-01-23 00:09:32 -08:00
Donkie 44da991d47 Added a new backend settings system
This can be used for saving runtime-configurable settings in the database.
2024-01-16 19:17:06 +01:00
pdsccode 28a70d5190 Merged spool price testing into existing test methods 2024-01-07 14:24:14 +01:00
pdsccode 05dfb611cf Revert "Fixed test error that asserts 404 on create"
This reverts commit f13c61b9e9.
2024-01-07 14:17:59 +01:00
pdsccode f13c61b9e9 Fixed test error that asserts 404 on create 2024-01-07 13:44:04 +01:00
pdsccode 44b20fb360 Fixed code formatting 2024-01-07 13:25:05 +01:00
pdsccode a3be982632 Fixing test update spool issue 2024-01-06 23:04:53 +01:00
pdsccode 577ed1e680 Changed to show filament price if no spool price set and added tests 2024-01-06 21:42:09 +01:00
Donkie cf8d7472f5 Added backend filament search by similar color 2023-12-19 21:37:07 +01:00
Donkie 9595aec675 Fixed registered field having wrong timezone
This happened with at least Postgres, which took it's server's timezone. It's complicated. Let's just set it from our side instead where we have some control.
2023-10-02 21:36:50 +02:00
Donkie 205574fa2c Fixed timezone being returned incorrectly
Happened if you've set TZ to non-UTC.

Apparently astimezone and replace doesn't do the same thing even if tzinfo is None.
2023-10-02 21:36:50 +02:00
Donkie 597dca7c28 Fixed combining empty and non-empty filter
Previously it would then act as a wildcard filter
2023-09-20 21:02:49 +02:00