From b86caa7332cb8570e54828b5da25fc5259f8da9c Mon Sep 17 00:00:00 2001 From: Yuhong Sun Date: Wed, 21 Oct 2026 21:40:21 +0000 Subject: [PATCH] feat(indexing): add new embedding models and upgrade-only model selection (#15351) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Description Add 11 selectable embedding models and an upgrade-only model picker. Fresh self-hosted installs use Granite. Existing deployments keep their stored model until an admin starts a re-index. On startup, the API server waits until the model server can embed with the self-hosted model in use. If that model is not in its cache, the model server downloads it first. | Type | Model | Output dimensions | | --- | --- | --- | | Self-hosted | `ibm-granite/granite-embedding-97m-multilingual-r2` (default, in the image) | 384 | | Self-hosted | `voyageai/voyage-4-nano` | 2048 | | Self-hosted | `nvidia/Nemotron-3-Embed-1B-BF16` | 2048 | | Cohere | `embed-v5.0-pro`, `embed-v5.0-fast` | 2048 | | Google | `gemini-embedding-2` | 3072 | | OpenAI | `text-embedding-3-large`, `text-embedding-3-small` | 3072 / 1536 | | Voyage | `voyage-4-large`, `voyage-4`, `voyage-4-lite` (added on main) | 1024 | The 16 older models are now **legacy**: nomic, E5, GTE-small, Cohere v3/v4, Voyage v2 and older Gemini/Vertex models. Existing indexes on them keep working. The API refuses a legacy model as a new target, but a re-index with the current model is still allowed. ## Size of this PR After review, I removed the parts that the feature does not need. The behavior stays the same. The diff against `main` went from **80 files, +9,855 / โˆ’504** to **69 files, +4,740 / โˆ’467**. | Area | Files | Added | Removed | | --- | --- | --- | --- | | Backend source | 16 | 1,004 | 96 | | Backend unit tests and test managers | 16 | 1,568 | 32 | | Backend manual suites (opt-in, not in CI) | 7 | 617 | 0 | | Web source | 6 | 617 | 255 | | Web tests | 5 | 630 | 1 | | Web messages (9 locales) | 9 | 225 | 72 | | Deployment, CI and feature map | 10 | 79 | 11 | The feature map (`.agents/feature-map/`) now describes the new selection rules. Removed: - **The branch's own cloud probe.** All selectable cloud models now use main's probe (`probe_embedding_dimension`), which Bifrost already used. It now also sends the stored Vertex configuration, the Azure API version and the deployment name. Before this change, the branch's probe ignored the Vertex configuration, so a Google provider that uses Workload Identity could not switch to `gemini-embedding-2`. - **The Vertex fan-out.** Main already sends `gemini-embedding-2` texts one at a time. The Vertex code is now the same as on main. - **The `expected_dim` path** in the model-server loader, and the startup warning for custom models with registry names. These protected only one rare case; see **Custom models with a registry name** in [Known limits](#known-limits). - **The UI fallback** that tested a new key with a default model when the key failed with the current model. The credential test now embeds with the staged or current model of that provider. Only a provider with neither uses its first selectable model. - **Picker cards for current and staged bring-your-own models.** Main does not show them either. The current model shows in the summary at the top of the page. - **Shared gates and fixtures for the manual suites.** Each suite has one gate env var. - **The screenshots in `docs/assets`.** The images below come from an earlier commit of this PR. - **Tests for removed code, and repeated test cases.** The unit tests went from 3,842 to 2,667 lines. ### CPU speed and memory cards All nine locales show these planning estimates: | Model | Card speed | Recommended CPU RAM | | --- | --- | --- | | Granite 97M | Up to **200 docs / min / CPU core** | **4 GB** per model server | | Voyage 4 Nano | Up to **50 docs / min / CPU core** | **4 GB** per model server | | Nemotron 3 Embed 1B BF16 | Up to **5 docs / min / CPU core** | **8 GB** per model server | Nemotron says **GPU recommended**. It runs on CPU too, but slowly. The cards tell the admin that speed changes with the hardware and the document size. The RAM value is for one loaded model. A re-index and the rest of Onyx need more memory. These values are planning estimates, not guarantees. The Nemotron per-core value is an extrapolation from an earlier multi-thread run. Document counts depend on chunking, and throughput does not always scale linearly with cores. The RAM values are recommendations, not measured minimum or peak values. ## Implementation - **Registry.** `shared_configs/embedding_models.py` defines the model names, dimensions, prefixes, normalization, status and pinned Hugging Face revisions. A parity test compares it with the web registry. - **Upgrade-only guard.** `_check_embedding_model_target` runs in `set-new-search-settings`: - The current model can always re-index, legacy models too. The dimension must stay the same. In Onyx Cloud, `reduced_dimension` must also stay the same, because tenants share the index of a model. - A selectable model must use the registry settings (dimension, normalization, prefixes). - Other OpenAI, Cohere, Google and Voyage models, and legacy self-hosted models, get `INVALID_INPUT`. The error lists the selectable models. - Custom Hugging Face models, LiteLLM, Azure and Bifrost stay open. - A self-hosted model gets `INVALID_INPUT` for a new `reduced_dimension`, because the model server ignores it. A re-index keeps a stored value. - **Cloud probe.** Before a re-index to a selectable cloud model, the API embeds a test text with the stored credentials and checks the dimension. A bad key, region or model ID fails before indexing starts. The probe runs before the PRESENT row lock, with no open transaction, as main's Bifrost probe does. - **Self-hosted loads.** New models load at a pinned revision with `trust_remote_code=False`, a dimension check and a finite-vector check. CPU-only hosts load FP32. CUDA and MPS keep the checkpoint dtype. Legacy and custom models keep main's `SentenceTransformer` call, so their vectors do not change. - **Voyage 4 Nano.** An Onyx loader adds bidirectional attention and the learned 1024-to-2048 projection. The generic loader with remote code disabled does not apply them. - **Model server.** All models load from the cache first, then from Hugging Face. Loads run off the event loop on two load workers, so at most two sets of weights (PRESENT and FUTURE) load at the same time. Registry models get 32 tokens of headroom for the prefix. The RoPE pre-warm now runs again only for a longer context. On main, it ran for every change of context length. - **Image.** The model-server image bundles pinned Granite. It no longer bundles nomic: nomic downloads on first use. The startup cache merge keeps the models that a volume already has. - **Picker.** Self-hosted and cloud groups, a Legacy label on the current legacy model, and an **Advanced** section for custom Hugging Face models, LiteLLM, Azure and Bifrost. A staged custom model keeps the custom provider, also when its name is the same as a registry model. - **Error detail.** `EmbeddingModel` now keeps the model server's error detail. Before, every HTTP failure showed "HTTP error occurred - response is None". ### Startup wait for self-hosted models On startup, the API server waits until the inference model server can embed with the current self-hosted model. The model server loads the model on the first request, and downloads it first if it is not cached. Example: a deployment that upgrades while it uses nomic, which the new image does not bundle. - The API server logs the wait, each failed attempt with its cause, and the time when the model is ready. The model server logs a download. - `LOCAL_EMBEDDING_MODEL_STARTUP_TIMEOUT` sets the wait (default 1800 seconds). When the wait runs out, startup fails. The error names the model server, the model, the last error and the fix. `0` disables the wait. - The check request runs in a daemon thread. A request that still waits at the deadline (for example, during a slow download) therefore does not keep the failed process alive. Before this fix, the process stayed alive until the model server read timeout (600 s by default). - There is no wait in Onyx Lite (`DISABLE_VECTOR_DB`), for cloud models, in multi-tenant deployments, in integration tests, or with `DISABLE_MODEL_SERVER`. - The wait uses the existing embed endpoint, so it also works with an older model server during a rolling upgrade. - This replaces the startup call to `warm_up_bi_encoder`. `SKIP_WARM_UP` (default `true`) skipped that call, so it did not run by default. - Only the inference model server is checked. The indexing model server still loads a model on first use. In Compose, the API server does not know the host of the indexing model server. - `env.template` and Helm `values.yaml` document the setting. The Helm chart is now 0.9.19. Live test at commit `0eb5881da9`, with an existing nomic deployment on a local stack. The size reduction did not change the wait: | Case | Result | | --- | --- | | nomic not cached, model server online | The model server downloaded nomic (523 MB). Startup continued after 19 s. | | nomic cached | Ready in 3 s, with no download. | | Model server starts after the API server | A warning every 10 s, then ready at 33 s. | | nomic not cached, no internet access | Each warning showed the cause ("not in the model server's Hugging Face cache and could not be downloaded"). Startup failed at the timeout, and the process exited. | | Onyx Lite, no model server | No wait. | | Timeout `0` | No wait. The model loads on first use. | ## Upgrade notes - **Offline deployments that use nomic.** The new image does not bundle nomic. A deployment with no internet access and no kept model cache cannot download it. The API server then fails at startup with a clear error, instead of starting with a search that cannot embed queries. Before you upgrade such a deployment, add the current model to the model server's Hugging Face cache, or keep the cache on a volume. Helm has no cache volume by default. This PR has no schema migration, so a return to the previous image works while the current model stays the same. - **Helm.** The chart has no Hugging Face cache volume for the model servers, so a model that the image does not bundle (nomic, Voyage 4 Nano, Nemotron) downloads again on every pod start. `values.yaml` documents an opt-in volume with the existing `inferenceCapability.volumes` and `volumeMounts` values. A default volume is not added: a ReadWriteOnce claim hangs rolling updates across nodes, breaks `replicaCount` above 1, and needs a StorageClass. The bundled Granite works with or without a volume. - **API probes.** `/health` answers only after the startup wait. A liveness probe needs a startup probe that allows the migrations plus `LOCAL_EMBEDDING_MODEL_STARTUP_TIMEOUT`. The api probe example in `values.yaml` shows one. - **Multi-tenant template snapshots.** Migration `dbaa756c2ccf` seeds a search settings row with `DOCUMENT_ENCODER_MODEL`. When that variable is empty, the seed is now Granite, not nomic. If a deploy runs `alembic/check_tenant_snapshot.py`, it reports a difference in `search_settings` until the template snapshots are built again (`--snapshot-template`). To keep the seed the same as on main, set `DOCUMENT_ENCODER_MODEL=nomic-ai/nomic-embed-text-v1`. - **No return to a legacy model.** After a switch, the old index is cleaned up and the upgrade-only rule blocks a return to a legacy model. Keep a compatible backup before you switch. ## Audit: retries and failures | Layer | Behavior | Compared with main | | --- | --- | --- | | Local encode | Catches `RuntimeError`, waits 0.1 seconds, then tries again: **up to four encode attempts**. It also retries errors that do not go away, such as OOM. | Unchanged. | | API to model server | Passage requests retry request failures three times with five-second waits. HTTP 429 allows ten attempts with ten-second waits. Connect/read timeouts are 30/600 seconds. Query requests do not get these retries. | Unchanged. | | Document indexing | A failed batch falls back to one document at a time after a two-second delay. Failures are recorded, and the other documents continue. | Unchanged. | | Cloud embedding | Authentication errors are separate. Errors mapped to `RuntimeError` get exponential backoff with jitter: eight attempts in indexing-only mode, else two. | Unchanged. `gemini-embedding-2` sends one text per request, in sequence, as on main. | | Model loading | Cache first, then online. A partial cache falls back online. A failed load is not cached, so a later request can try again. Requests for the same model and context length share one load job. | Cache-first lookup is new for all models. Registry models add pinned revisions and checks. | Model downloads run off the event loop. When a waiting request is cancelled, the shared load continues. A load job has no deadline, so it can continue after an HTTP timeout. One gap from main remains: the response JSON is parsed outside the retry wrapper. See [local retries](https://github.com/onyx-dot-app/onyx/blob/4784da70697fb59261f43088383d85671544ec98/backend/model_server/encoders.py#L157), [request retries](https://github.com/onyx-dot-app/onyx/blob/4784da70697fb59261f43088383d85671544ec98/backend/onyx/natural_language_processing/search_nlp_models.py#L1322), and [cloud retries](https://github.com/onyx-dot-app/onyx/blob/4784da70697fb59261f43088383d85671544ec98/backend/onyx/natural_language_processing/search_nlp_models.py#L942). ## Audit: migration and cancellation There is **no schema migration, and no automatic change to existing embedding settings**. Granite is the fresh-install default. Cloud tenant provisioning still uses Cohere v3. Existing stored prefixes and dimensions stay in use. 1. The API checks the target with the upgrade-only guard. 2. A new cloud target embeds a test text with the stored credentials before the migration. The probe checks the dimension. It runs before the PRESENT row lock. 3. The API locks PRESENT and refuses overlapping migrations. It also refuses an index name that is live or still waiting for cleanup. FUTURE, the first work and the cleanup consent commit together. 4. The existing port flow reads stored documents from the current index, embeds them again and writes the new index. Connectors do not fetch again. All-connectors mode includes paused connectors. Active-only mode leaves them out and needs consent for the data that it leaves out. User files are included. 5. Normal promotion waits for successful required ports, no open writers or sync work, and a sample check in the new index. A failed or paused required port blocks promotion. The sample check does not check the full corpus. 6. Port batches retry up to five times. The cursor moves after a successful batch. The first watchdog retry is immediate. Repeated failures at the same cursor use exponential backoff from 30 seconds, up to one hour. Five failures in sequence at that cursor pause the unit. An operator can resume it. 7. A cancel of FUTURE keeps PRESENT active, stops port work at a batch boundary, and removes the partial index after writers stop. **Instant switch promotes first**, so search is not complete during backfill, and this cancel path is not available. See [migration endpoint](https://github.com/onyx-dot-app/onyx/blob/4784da70697fb59261f43088383d85671544ec98/backend/onyx/server/manage/search_settings.py#L140), [upgrade-only guard](https://github.com/onyx-dot-app/onyx/blob/4784da70697fb59261f43088383d85671544ec98/backend/onyx/server/manage/search_settings.py#L456), [swap gates](https://github.com/onyx-dot-app/onyx/blob/4784da70697fb59261f43088383d85671544ec98/backend/onyx/db/swap_index.py#L367), and [cancellation](https://github.com/onyx-dot-app/onyx/blob/4784da70697fb59261f43088383d85671544ec98/backend/onyx/server/manage/search_settings.py#L668). ## Known limits - **Model memory.** Two load workers limit how many models load at the same time. Loaded models are not evicted (as on main), so the old and new models stay in memory during a switch. Helm's default memory limit for the indexing model server is **6 GiB**, below the **8 GB** that the Nemotron card recommends. See [load executor](https://github.com/onyx-dot-app/onyx/blob/4784da70697fb59261f43088383d85671544ec98/backend/model_server/encoders.py#L35) and [Helm limits](https://github.com/onyx-dot-app/onyx/blob/4784da70697fb59261f43088383d85671544ec98/deployment/helm/charts/onyx/values.yaml#L309). - **Legacy models are not selectable.** This is by design. The current legacy model stays visible and can re-index. After a switch, the standard flow cannot select it again. - **Bring-your-own options are under Advanced.** Unknown model names for the built-in OpenAI, Cohere, Google and Voyage providers get `INVALID_INPUT`. LiteLLM, Azure, Bifrost and custom models stay open. - **The cloud probe reads the provider settings once.** If an admin changes the key or region while a re-index starts, the probe can use the old values. A key can also change at any time during a re-index, so no preflight check can guarantee the key that the workers use. Indexing failures then show in the re-index status, and the admin can cancel. - **Custom models with a registry name.** Before this PR, an admin could add a registry model name, such as `voyageai/voyage-4-nano`, as a custom model. The model server now loads that name as the registry model. For `voyage-4-nano`, the vectors then have 2048 dimensions, not 1024, so a 1024-dimension index fails with an error. Re-index to the registry model to fix it. For the other registry models, a generic load gives the same dimension. - **Shared `max_seq_length`.** The model server changes `max_seq_length` on the shared model for each request. Two requests with different context lengths can overlap. This is the same on main. This PR does not change it. - **Voyage 4 Nano depends on the Transformers and SentenceTransformers layout.** Run the real-weight suite when those dependencies change. - **No reduced dimensions for Cohere v5 and the self-hosted models.** Cloud provisioning stays on Cohere v3. These are follow-ups. - **The cancel dialog has two buttons labeled Cancel.** One closes the dialog, the other cancels the re-index. This is the same on main. - The branch is rebased onto `main` at `ad9c0030e9`. This PR does not claim better retrieval quality. ## Validation for this update - **Backend: 15,343 unit tests passed** before the review fixes. I left out the EE license tests, because they write a key to a live Redis. `ty` and pre-commit passed. After the review fixes and the merge of `main`, the PR's backend unit tests passed again (199 tests). Pre-commit passed on all 69 changed files. - **Startup exit check:** a fake model server that never answers, with a 3 s wait and a 20 s read timeout. The wait raised at 3.3 s. The process exited at 5 s with the fix, and at 22 s without it. - **Web: type check, lint and 3,236 Jest tests (307 suites) passed.** - **Self-hosted real-weight suite: 6 of 6 passed.** Golden scores and the bidirectional-attention check for Granite, Voyage 4 Nano and Nemotron. - **Cloud live suite: `gemini-embedding-2` passed.** - **End to end on a local stack with branch code: 11 of 11 passed.** The API refused all 6 legacy self-hosted targets and accepted a same-model re-index. Then the stack switched to Granite, Voyage 4 Nano, Nemotron and `gemini-embedding-2`, in sequence. After each switch, a pure vector search found each of 5 seeded facts first. - **Not run for this update: Cohere, OpenAI and Voyage cloud tests.** A TLS proxy on the test network blocked `api.cohere.com` and `api.openai.com`. The probe then refused the switch with `400`, as designed. These tests passed on an earlier commit of this PR, and the probe that they use is main's probe. I had no Voyage key. - **CI:** at `9321cb51db`, every check passed except the Dependency Audit: Playwright, integration, acceptance, database, external-dependency unit, Jest and quality checks. The audit fails on open Dependabot alerts for Pillow and certifi in other manifests (`loadtest/uv.lock` and the Craft sandbox requirements). Other open PRs fail the same way at the moment. - **Not run for this update:** speed benchmarks.
Earlier validation in this PR, not run again for this update - 28 real-weight tests and 24 cloud tests passed. - End-to-end switches to all eight selectable models of that time passed. The API refused all 16 legacy targets. - Legacy-upgrade equivalence, Docker cache keeping and multi-tenant provisioning checks passed. - 11,521 backend unit tests passed. 10 EE license failures also occurred on main.
## Voyage retrieval-quality check โ€” October 14, 2026 The branch's Voyage loader passed a local EnterpriseRAG-Bench loading check. **Vector-only mean document Recall@10 was 91.85%**, above the proposed 50% threshold. This supports a correct load. It does not prove numerical equality with the upstream implementation. The corpus had **5,000 documents**: all **722 labeled supporting documents** and **4,278 random distractors**, sampled with seed `20261013`. All **470 questions with document labels** were scored. The 30 questions with no labels were left out. No question or answer text went into the corpus. | Retrieval mode | Mean Recall@1 | Mean Recall@5 | Mean Recall@10 | Mean Recall@20 | | --- | --- | --- | --- | --- | | Onyx content-vector-only retrieval | 66.13% | 87.37% | **91.85%** | 94.16% | | Onyx hybrid search API | 75.85% | 93.32% | **96.37%** | 97.70% | Recall is the fraction of labeled documents found for each question, averaged over the questions. Rankings use the first hit of each document. Vector-only hit rate@10 (at least one relevant document) was 94.26%. That is a different metric from recall. | Question category | Questions | Vector-only Recall@10 | Hybrid Recall@10 | | --- | --- | --- | --- | | Basic | 175 | 97.71% | 99.43% | | Semantic | 125 | 84.80% | 92.80% | | Intra-document reasoning | 40 | 97.50% | 100.00% | | Project related | 40 | 86.43% | 95.35% | | Constrained | 30 | 98.33% | 98.33% | | Conflicting information | 20 | 90.00% | 97.50% | | Completeness | 20 | 68.03% | 79.03% | | Miscellaneous | 20 | 100.00% | 100.00% | The check used CPU FP32, 2048-dimensional normalized vectors, and the registry's query and document prefixes. MPS and CUDA were off. There was no query expansion, reranking, or LLM document selection. All 5,000 document IDs matched the saved manifest, with 15,612 indexed chunks. All query vectors were finite and had unit length. The last indexing attempt had no document errors. Two earlier attempts stopped after stale-heartbeat timeouts and continued from saved progress. **Search path:** on this OpenSearch branch, `hybrid_alpha=1` still runs hybrid retrieval, so the API result includes keyword scores. The vector-only result calls `OpenSearchDocumentIndex.semantic_retrieval` directly, with content vectors only. Keyword matches therefore cannot hide a broken embedding loader in that result. **Scope:** this is an easier 5,000-document subset that contains all gold documents, not the full 511,958-document benchmark. Do not compare these values with full-corpus leaderboard scores. No loader or ranking code changed to get these results. Revisions: Onyx `913bfa8d7a32c9cda340df92f42d0f706fd424c2`; EnterpriseRAG-Bench `d36685e273713975ee20299bbf1ab64165575b3c`; Voyage `67fabc9bef010dabc5f6024aa1b1b6b93410426f`. Runtime: PyTorch 2.9.1, Transformers 5.14.1, SentenceTransformers 5.4.1. ## UI screenshots From the local development stack at commit `98bfc70080`, before the size reduction. These show the real flow, not mockups.
Self-hosted cards and current model ![Self-hosted model cards with CPU throughput and RAM guidance](https://raw.githubusercontent.com/onyx-dot-app/onyx/98bfc70080a541f9346517d062f1c9edbdd51ffb/docs/assets/embedding-models/01-model-cards.jpg)
Cloud model catalog ![Cloud model catalog](https://raw.githubusercontent.com/onyx-dot-app/onyx/98bfc70080a541f9346517d062f1c9edbdd51ffb/docs/assets/embedding-models/02-cloud-models.jpg)
Staged Granite change and migration strategy ![Staged model change](https://raw.githubusercontent.com/onyx-dot-app/onyx/98bfc70080a541f9346517d062f1c9edbdd51ffb/docs/assets/embedding-models/03-staged-change.jpg)
Re-index in progress ![Re-index progress and cancellation action](https://raw.githubusercontent.com/onyx-dot-app/onyx/98bfc70080a541f9346517d062f1c9edbdd51ffb/docs/assets/embedding-models/04-reindex-progress.jpg)
Cancel confirmation ![Cancel re-index confirmation](https://raw.githubusercontent.com/onyx-dot-app/onyx/98bfc70080a541f9346517d062f1c9edbdd51ffb/docs/assets/embedding-models/05-cancel-confirmation.jpg)
Advanced custom model setup ![Custom embedding model setup](https://raw.githubusercontent.com/onyx-dot-app/onyx/98bfc70080a541f9346517d062f1c9edbdd51ffb/docs/assets/embedding-models/06-custom-model-dialog.jpg)
## Additional Options - [ ] [Optional] Please cherry-pick this PR to the latest release version. - [ ] [Optional] Override Linear Check ๐Ÿค– Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 --- .agents/feature-map/PATHS.md | 1 + .../feature-map/components/document-index.md | 29 +- .../components/file-store-and-user-files.md | 5 +- .github/workflows/pr-python-tests.yml | 1 + backend/Dockerfile | 6 + backend/Dockerfile.model_server | 11 +- .../versions/dbaa756c2ccf_embedding_models.py | 2 +- .../model_server/embedding_model_loader.py | 205 ++++++ backend/model_server/encoders.py | 166 +++-- backend/model_server/voyage_nano.py | 56 ++ backend/onyx/configs/app_configs.py | 7 + backend/onyx/configs/model_configs.py | 48 +- backend/onyx/db/search_settings.py | 9 +- .../search_nlp_models.py | 15 +- .../onyx/natural_language_processing/utils.py | 17 +- backend/onyx/server/manage/search_settings.py | 208 +++++- backend/onyx/setup.py | 95 ++- backend/shared_configs/configs.py | 3 +- backend/shared_configs/embedding_models.py | 244 ++++++++ backend/shared_configs/utils.py | 6 + ...ault_model_server_embeddings_are_finite.py | 31 +- .../tests/daily/embedding/test_embeddings.py | 41 +- .../db/test_index_reclaim.py | 2 +- .../common_utils/managers/document_search.py | 3 + .../common_utils/managers/reindex_port.py | 7 +- backend/tests/manual/__init__.py | 0 .../tests/manual/embedding_models/README.md | 58 ++ .../tests/manual/embedding_models/__init__.py | 0 .../manual/embedding_models/provider_env.py | 28 + .../test_cloud_embedding_models_live.py | 98 +++ .../test_embedding_model_switch_e2e.py | 301 +++++++++ .../test_self_hosted_embedding_models.py | 132 ++++ .../test_dockerfile_bundled_model.py | 57 ++ .../tests/unit/model_server/test_embedding.py | 169 ++++- .../test_embedding_model_loader.py | 304 +++++++++ .../test_embedding_model_registry_configs.py | 151 +++++ .../test_search_nlp_models.py | 17 +- .../test_utils_hf_revision.py | 54 ++ .../test_embedding_model_upgrade_guard.py | 516 +++++++++++++++ .../test_reindex_gateway_embedding_guard.py | 1 + .../test_setup_local_embedding_model_wait.py | 112 ++++ .../shared_configs/test_embedding_models.py | 50 ++ ...est_web_embedding_model_registry_parity.py | 89 +++ .../embedded/docker_compose/README.md | 17 + .../embedded/docker_compose/env.template | 4 + deployment/docker_compose/README.md | 17 + deployment/docker_compose/env.template | 4 + deployment/helm/charts/onyx/Chart.yaml | 8 +- deployment/helm/charts/onyx/values.yaml | 33 +- web/src/i18n/messages/ar.json | 33 +- web/src/i18n/messages/de.json | 33 +- web/src/i18n/messages/en.json | 33 +- web/src/i18n/messages/es.json | 33 +- web/src/i18n/messages/fr.json | 33 +- web/src/i18n/messages/ja.json | 33 +- web/src/i18n/messages/ko.json | 33 +- web/src/i18n/messages/pt.json | 33 +- web/src/i18n/messages/zh.json | 33 +- web/src/lib/searchSettings/constants.ts | 156 ++++- web/src/lib/searchSettings/index.test.ts | 151 +++++ web/src/lib/searchSettings/index.ts | 85 ++- web/src/lib/searchSettings/svc.test.ts | 3 + web/src/lib/searchSettings/svc.ts | 12 +- web/src/lib/searchSettings/types.ts | 28 +- .../admin/IndexSettingsPage/index.test.tsx | 451 +++++++++++++ .../views/admin/IndexSettingsPage/index.tsx | 591 +++++++++++------- .../views/admin/IndexSettingsPage/modals.tsx | 22 +- .../admin/index-settings/IndexSettingsPage.ts | 24 +- .../index-settings/index_settings.spec.ts | 2 + 69 files changed, 4779 insertions(+), 481 deletions(-) create mode 100644 backend/model_server/embedding_model_loader.py create mode 100644 backend/model_server/voyage_nano.py create mode 100644 backend/shared_configs/embedding_models.py create mode 100644 backend/tests/manual/__init__.py create mode 100644 backend/tests/manual/embedding_models/README.md create mode 100644 backend/tests/manual/embedding_models/__init__.py create mode 100644 backend/tests/manual/embedding_models/provider_env.py create mode 100644 backend/tests/manual/embedding_models/test_cloud_embedding_models_live.py create mode 100644 backend/tests/manual/embedding_models/test_embedding_model_switch_e2e.py create mode 100644 backend/tests/manual/embedding_models/test_self_hosted_embedding_models.py create mode 100644 backend/tests/unit/model_server/test_dockerfile_bundled_model.py create mode 100644 backend/tests/unit/model_server/test_embedding_model_loader.py create mode 100644 backend/tests/unit/onyx/configs/test_embedding_model_registry_configs.py create mode 100644 backend/tests/unit/onyx/natural_language_processing/test_utils_hf_revision.py create mode 100644 backend/tests/unit/onyx/server/manage/test_embedding_model_upgrade_guard.py create mode 100644 backend/tests/unit/onyx/test_setup_local_embedding_model_wait.py create mode 100644 backend/tests/unit/shared_configs/test_embedding_models.py create mode 100644 backend/tests/unit/shared_configs/test_web_embedding_model_registry_parity.py create mode 100644 web/src/lib/searchSettings/index.test.ts create mode 100644 web/src/views/admin/IndexSettingsPage/index.test.tsx diff --git a/.agents/feature-map/PATHS.md b/.agents/feature-map/PATHS.md index 573e43ce82..e8f45f1461 100644 --- a/.agents/feature-map/PATHS.md +++ b/.agents/feature-map/PATHS.md @@ -66,6 +66,7 @@ here. Read the component that documents the setting in its env table. | `backend/onyx/natural_language_processing/` | document-index, internal-search | | `backend/onyx/db/search_settings.py`, `db/swap_index.py` | document-index | | `backend/onyx/server/manage/search_settings.py`, `server/manage/embedding/` | document-index | +| `backend/shared_configs/embedding_models.py`, `backend/model_server/embedding_model_loader.py`, `backend/model_server/voyage_nano.py` | document-index | | `backend/onyx/server/features/search/`, `server/query_and_chat/query_backend.py`, `backend/ee/onyx/server/query_and_chat/search_backend.py`, `backend/ee/onyx/search/` | internal-search | | `backend/ee/onyx/server/query_and_chat/token_limit.py` | rate-and-usage-limits | | `backend/onyx/access/`, `backend/ee/onyx/access/`, `backend/onyx/db/document_access.py`, `db/permissions.py`, `db/scoped_permissions.py` | access-control | diff --git a/.agents/feature-map/components/document-index.md b/.agents/feature-map/components/document-index.md index 29723aaf06..6c43c85209 100644 --- a/.agents/feature-map/components/document-index.md +++ b/.agents/feature-map/components/document-index.md @@ -14,7 +14,9 @@ `port_copy.py`, `string_filtering.py`, `cc_pair_ids_backfill.py`), `backend/onyx/db/search_settings.py`, `swap_index.py`, `backend/onyx/natural_language_processing/search_nlp_models.py`, -`backend/onyx/server/manage/search_settings.py`, `backend/onyx/server/manage/embedding/` +`backend/onyx/server/manage/search_settings.py`, `backend/onyx/server/manage/embedding/`, +`backend/shared_configs/embedding_models.py`, `backend/model_server/embedding_model_loader.py`, +`backend/model_server/voyage_nano.py` **Does not own:** the retrieval orchestration that decides what to search and how to merge results across sources ([[internal-search]]), or the chunking/embedding pipeline @@ -40,9 +42,13 @@ in the banner, but cannot trigger a popup. See `backend/onyx/document_index/open for thresholds, timeouts, and recovery behavior. An admin experiences it directly on the embedding-model page. They pick a new -embedding model (self-hosted, Cohere, OpenAI, Azure, Bedrock, Vertex, LiteLLM, Bifrost, -and more), optionally run a sample embedding test (a fixed test string, not a document), and start a re-index. Bifrost -always runs the test, because its model ID is free text. From that point, +embedding model and start a re-index. The picker lists only the selectable models of +the registry (self-hosted Granite, Voyage 4 Nano and Nemotron; Cohere, OpenAI, Google +and Voyage cloud models). Custom Hugging Face models, LiteLLM, Azure and Bifrost are in +a collapsed Advanced section. A legacy model (for example nomic, E5, Cohere v3) shows +only while it is the current model. It can re-index, but it cannot be chosen as a new +target. Before the re-index starts, the API embeds a fixed test string with the stored +provider settings for Bifrost and for every selectable cloud model. From that point, Onyx builds a second, parallel index in the background using the new model while the old index keeps serving live search. A progress view shows re-index status and errors. When indexing catches up (the exact criterion depends on the switchover type chosen), @@ -64,7 +70,7 @@ dropped the migration tables, and #15336 removed Vespa and the retrieval toggle. |---|---|---|---| | GET | `/manage/admin/opensearch-health` | `read_resource_health` | Cached resource pressure; requires `FULL_ADMIN_PANEL_ACCESS`. | | POST | `/manage/admin/opensearch-health/popup` | `claim_resource_popup` | Claims a tenant-scoped, per-admin 24-hour popup allowance in Redis. | -| POST | `/search-settings/set-new-search-settings` | `set_new_search_settings` | Creates a FUTURE `SearchSettings` row, starts a re-index. Requires `FULL_ADMIN_PANEL_ACCESS`. For Bifrost it first embeds a test string and rejects a failed call or a dimension mismatch. | +| POST | `/search-settings/set-new-search-settings` | `set_new_search_settings` | Creates a FUTURE `SearchSettings` row, starts a re-index. Requires `FULL_ADMIN_PANEL_ACCESS`. For Bifrost and every selectable cloud model, it first embeds a test string with the stored provider settings (`_validate_cloud_embedding_model`) and rejects a failed call or a dimension mismatch. `_check_embedding_model_target` then refuses a legacy model or an unknown OpenAI, Cohere, Google or Voyage model as a new target with `INVALID_INPUT`. A re-index with the PRESENT model is allowed. | | POST | `/search-settings/cancel-new-embedding` | `cancel_new_embedding` | Cancels the in-flight FUTURE re-index. | | DELETE | `/search-settings/delete-search-settings` | | | | GET | `/search-settings/get-current-search-settings` | | The PRESENT row. | @@ -100,6 +106,7 @@ Per the frontend rule in `CLAUDE.md`, always call these through the web server | `OPENSEARCH_EXPLAIN_ENABLED` | false | Adds scoring breakdowns; documented as roughly 1000x slower for hybrid search in practice. | | `PIT_KEEP_ALIVE` | `"5m"` | Point-in-time lease used by the reindex port's consistent scan. | | `VERIFY_CREATE_OPENSEARCH_INDEX_ON_INIT_MT` | true | Whether multi-tenant cloud checks/creates the index on every `OpenSearchDocumentIndex` construction. | +| `LOCAL_EMBEDDING_MODEL_STARTUP_TIMEOUT` | 1800 | Seconds that API server startup (`setup.py:wait_for_local_embedding_model`) waits until the inference model server embeds with the PRESENT self-hosted model. The model server downloads the model first if it is not cached. Startup fails when the wait runs out. `0` disables the wait. There is no wait for cloud models, with `MULTI_TENANT`, `INTEGRATION_TESTS_MODE` or `DISABLE_MODEL_SERVER`. | --- @@ -377,6 +384,17 @@ query-time embedding, pulling `model_name`, `normalize`, `query_prefix`, `reduced_dimension` straight off the `SearchSettings` row. Separate `server_host` values are used depending on whether the call is from indexing or inference. +The registry in `backend/shared_configs/embedding_models.py` lists each known model as +SELECTABLE or LEGACY, with its dimension, prefixes, normalization and, for self-hosted +models, a pinned Hugging Face revision. `web/src/lib/searchSettings/constants.ts` must +match it (`backend/tests/unit/shared_configs/test_web_embedding_model_registry_parity.py`). On the model server, +`model_server/embedding_model_loader.py` loads a self-hosted registry model at its pinned +revision, in float32 on CPU-only hosts, and checks the output dimension. +`voyageai/voyage-4-nano` also gets bidirectional attention and its 1024-to-2048 +projection (`model_server/voyage_nano.py`). Every other name keeps the plain +`SentenceTransformer` call. All models load from the cache first, then from Hugging +Face. The model-server image bundles only the default model, Granite. + `RerankingModel` (`search_nlp_models.py`) is a fully implemented class (local cross-encoder plus direct API paths for Cohere/Bedrock) with **no active call site**. Its only instantiation in the codebase is inside @@ -460,6 +478,7 @@ comment reads `# No longer used`. See ยง9. | changes hybrid weights or subqueries | run a retrieval-quality eval before shipping; the weights-sum-to-1.0 assert catches arithmetic mistakes but not quality regressions | | adds a filter | `_get_search_filters` and every one of its private helper functions that builds one clause; the hybrid, keyword, semantic, and random query builders all call the same function, so a filter added there applies everywhere automatically, but a filter added ad hoc to just one query builder will not | | changes `SearchSettings` | `create_search_settings`, `update_search_settings`, the reindex request models in `server/manage/search_settings.py`, and `EmbeddingModel.from_db_model` if the field feeds the embedder | +| adds an embedding model or changes its registry entry | the web registry in `web/src/lib/searchSettings/constants.ts` (parity test), the card copy in every locale, `_check_embedding_model_target`, and for a new default the bake in `backend/Dockerfile.model_server`. Do not change the dimension or prefixes of a model that indexes already use. | | touches the swap state machine (`IndexModelStatus` or `IndexReclaimStatus`) | `swap_index.py:_perform_index_swap`, every `advance_to_*` helper's prior-state guard, and the reclaim beat task; a broken guard can double-delete or skip the soak window | | changes `get_default_document_index` or `OpenSearchIndexPair` fan-out rules | every caller, since most pass `None` as the secondary; a change to which calls reach the secondary affects the port backfill and swap-time deletes | diff --git a/.agents/feature-map/components/file-store-and-user-files.md b/.agents/feature-map/components/file-store-and-user-files.md index 0f145fc2f3..a2c76108be 100644 --- a/.agents/feature-map/components/file-store-and-user-files.md +++ b/.agents/feature-map/components/file-store-and-user-files.md @@ -312,8 +312,9 @@ Computed **at upload time**, before any chat turn runs: `natural_language_processing/utils.py:get_tokenizer(model_name, provider_type)` from the workspace's default LLM model (`db/llm.py:fetch_default_model(..., LLMModelFlowType.CHAT)`), falling back to -`_get_default_tokenizer()` (`HuggingFaceTokenizer(DOCUMENT_ENCODER_MODEL)`) -when the provider type is unrecognized. Text/document files are counted with +`_get_default_tokenizer()` (`HuggingFaceTokenizer(DEFAULT_TOKENIZER_MODEL)`: +the `DOCUMENT_ENCODER_MODEL` env var, else nomic, whose tokenizer the API image +bakes) when the provider type is unrecognized. Text/document files are counted with `natural_language_processing/utils.py:count_tokens` against the extracted text. Images are **not** tokenized against real content; their count is a heuristic, `projects_file_utils.py:estimate_image_tokens_for_upload`, which diff --git a/.github/workflows/pr-python-tests.yml b/.github/workflows/pr-python-tests.yml index 411e2ea536..eb6013996a 100644 --- a/.github/workflows/pr-python-tests.yml +++ b/.github/workflows/pr-python-tests.yml @@ -36,6 +36,7 @@ jobs: filters: | python: - 'backend/**' + - 'web/src/lib/searchSettings/constants.ts' - '.agents/hooks/**' - 'pyproject.toml' - '.github/workflows/pr-python-tests.yml' diff --git a/backend/Dockerfile b/backend/Dockerfile index ff3440e7ff..5ebcfd4dbc 100644 --- a/backend/Dockerfile +++ b/backend/Dockerfile @@ -144,6 +144,12 @@ RUN ln -s /usr/local/bin/supervisord /usr/bin/supervisord && \ RUN python -c "from tokenizers import Tokenizer; \ Tokenizer.from_pretrained('nomic-ai/nomic-embed-text-v1')" +# Tokenizer of the default self-hosted embedding model (nomic above stays the +# default tokenizer). The API server loads it at hf_revision from +# shared_configs/embedding_models.py, so the revision must match. +RUN python -c "from tokenizers import Tokenizer; \ +Tokenizer.from_pretrained('ibm-granite/granite-embedding-97m-multilingual-r2', revision='835ad14087e140460703cf0fae09f97d469d65c2')" + # Pre-downloading tiktoken for setups with limited egress RUN python -c "import tiktoken; \ tiktoken.get_encoding('cl100k_base')" diff --git a/backend/Dockerfile.model_server b/backend/Dockerfile.model_server index 7fcd07e0f5..f46f61ca31 100644 --- a/backend/Dockerfile.model_server +++ b/backend/Dockerfile.model_server @@ -55,10 +55,14 @@ RUN uv pip install --python /app/.venv/bin/python --no-cache-dir --no-deps --req -r /tmp/requirements.txt && \ rm -rf ~/.cache/uv /tmp/*.txt -# Cache the embedding model's weights, tokenizer, and metadata in the image. +# Cache the default embedding model at its pinned revision, so fresh and air-gapped +# installs embed with no download. The name and revision must match +# shared_configs/embedding_models.py (a unit test checks this). Other self-hosted +# models, nomic included, download on first use. FROM builder AS embedding-models RUN python -c "from sentence_transformers import SentenceTransformer; \ -SentenceTransformer(model_name_or_path='nomic-ai/nomic-embed-text-v1', trust_remote_code=False);" +SentenceTransformer(model_name_or_path='ibm-granite/granite-embedding-97m-multilingual-r2', \ +revision='835ad14087e140460703cf0fae09f97d469d65c2', trust_remote_code=False);" # Runtime stage. We run as the `onyx` user (UID 1001, carried over from the previous # image; see USER below) and chown everything the runtime writes to it. The DHI runtime @@ -97,7 +101,8 @@ COPY --from=builder --chown=1001:1001 /var/log/onyx /var/log/onyx # In case the user has volumes mounted to /app/.cache/huggingface that they've downloaded while # running Onyx, move the current contents of the cache folder to a temporary location to ensure -# it's preserved in order to combine with the user's cache contents +# it's preserved in order to combine with the user's cache contents. The model server merges it +# in at startup without overwriting files, so a volume keeps its models (e.g. nomic). COPY --chown=1001:1001 --from=embedding-models /app/.cache/huggingface /app/.cache/temp_huggingface # Utils used by model server diff --git a/backend/alembic/versions/dbaa756c2ccf_embedding_models.py b/backend/alembic/versions/dbaa756c2ccf_embedding_models.py index 91eef429a1..43d601b86c 100644 --- a/backend/alembic/versions/dbaa756c2ccf_embedding_models.py +++ b/backend/alembic/versions/dbaa756c2ccf_embedding_models.py @@ -22,7 +22,7 @@ from onyx.db.enums import EmbeddingPrecision from onyx.db.models import IndexModelStatus from onyx.db.search_settings import user_has_overridden_embedding_model from onyx.indexing.models import IndexingSetting -from onyx.natural_language_processing.search_nlp_models import clean_model_name +from shared_configs.utils import clean_model_name # revision identifiers, used by Alembic. revision = "dbaa756c2ccf" diff --git a/backend/model_server/embedding_model_loader.py b/backend/model_server/embedding_model_loader.py new file mode 100644 index 0000000000..544b4a62f3 --- /dev/null +++ b/backend/model_server/embedding_model_loader.py @@ -0,0 +1,205 @@ +"""Loads self-hosted embedding models for the model server. + +Registry models (the exact names that ``get_local_model_spec`` matches) load at +a pinned revision, in float32 on CPU-only hosts, with a per-model fix-up and an +output dimension check. Every other name (legacy and custom models) keeps the +``SentenceTransformer`` call from before the registry, so existing vectors do +not change; only the file lookup is local first. Settings come only from the +server-side registry, never from a request. All functions here block. +""" + +import os +from collections.abc import Callable +from typing import TYPE_CHECKING, Any, TypeVar + +import numpy as np +import torch + +from model_server.voyage_nano import ( + VOYAGE_NANO_PROJECTION_FILE, + add_bidirectional_projection, + read_projection_weight, +) +from onyx.utils.logger import setup_logger +from shared_configs.embedding_models import ( + EmbeddingModelSpec, + LocalModelLoader, + get_local_model_spec, +) + +if TYPE_CHECKING: + from sentence_transformers import SentenceTransformer + +logger = setup_logger() + +_T = TypeVar("_T") + +# What a local-only load raises when an interrupted download left a partial +# cache (sentence-transformers 5.4.1, huggingface-hub 1.26). The online retry +# downloads the missing files; a real load error comes back from that retry. +_PARTIAL_CACHE_ERRORS: tuple[type[Exception], ...] = (OSError, ValueError, TypeError) + +# sentence-transformers reads this env var as its cache folder when it is set. +_SENTENCE_TRANSFORMERS_HOME_ENV = "SENTENCE_TRANSFORMERS_HOME" +_DIMENSION_PROBE_TEXT = "probe" + + +class EmbeddingModelLoadError(RuntimeError): + """A self-hosted embedding model loaded, but is not usable.""" + + +def _cache_dir() -> str | None: + return os.environ.get(_SENTENCE_TRANSFORMERS_HOME_ENV) or None + + +def _modules_json_is_cached(model_name: str, revision: str | None) -> bool: + """True if the cache has modules.json at this revision, or a marker that the + repo has none. Without it, a local-only load silently builds default modules + (mean pooling, no Normalize).""" + from huggingface_hub import try_to_load_from_cache + + try: + cached = try_to_load_from_cache( + model_name, "modules.json", cache_dir=_cache_dir(), revision=revision + ) + except ValueError: + # Not a Hub repo id (e.g. a local directory). sentence-transformers + # reads a local directory directly, with no download. + return False + return cached is not None + + +def _load_local_first( + model_name: str, + revision: str | None, + load: Callable[[bool], _T], +) -> _T: + """Run ``load(local_files_only)`` from the local cache first, then online, so + a cached model (baked or downloaded before) loads with no network access.""" + cached = _modules_json_is_cached(model_name, revision) + if cached: + try: + return load(True) + except _PARTIAL_CACHE_ERRORS as e: + logger.warning( + "Could not load %s from the local cache (%s: %s). Loading it from Hugging Face.", + model_name, + type(e).__name__, + e, + ) + else: + logger.notice( + "%s is not in the model server's Hugging Face cache. Downloading it.", + model_name, + ) + try: + return load(False) + except Exception as e: + if cached: + raise + # huggingface_hub reports an unreachable Hub with unclear errors (e.g. + # "the client has been closed"). Say what is missing and what to do. + raise EmbeddingModelLoadError( + f"{model_name} is not in the model server's Hugging Face cache and " + f"could not be downloaded ({type(e).__name__}: {e}). If this " + "deployment has no internet access, add the model to the cache " + f"({_cache_dir() or 'the default HF_HOME cache'}) first." + ) from e + + +def _registry_model_kwargs() -> dict[str, Any]: + """The new checkpoints are bf16, which is 8-10x slower than float32 on CPU, + so CPU-only hosts load float32. CUDA and MPS keep the checkpoint dtype.""" + if torch.cuda.is_available() or torch.backends.mps.is_available(): + return {} + return {"dtype": torch.float32} + + +def _check_embedding_dim( + model: "SentenceTransformer", spec: EmbeddingModelSpec +) -> None: + """Fail unless a probe embedding has the registry dimension and is finite. The + configured dimension is not enough: a wrong voyage-4-nano load reports 2048 + but returns 1024-dim vectors.""" + probe = model.encode([_DIMENSION_PROBE_TEXT], show_progress_bar=False) + dim = probe.shape[-1] + if dim != spec.model_dim: + raise EmbeddingModelLoadError( + f"{spec.model_name} returned {dim}-dim vectors, but the registry " + f"expects {spec.model_dim}. The model is not used." + ) + if not np.isfinite(probe).all(): + raise EmbeddingModelLoadError( + f"{spec.model_name} returned non-finite values for a probe text. " + "The model is not used." + ) + + +def _load_registry_model(spec: EmbeddingModelSpec) -> "SentenceTransformer": + from sentence_transformers import SentenceTransformer + + revision = spec.hf_revision + model_kwargs = _registry_model_kwargs() + + def load(local_files_only: bool) -> "SentenceTransformer": + return SentenceTransformer( + model_name_or_path=spec.model_name, + revision=revision, + trust_remote_code=False, + local_files_only=local_files_only, + model_kwargs=model_kwargs, + ) + + model = _load_local_first(spec.model_name, revision, load) + + if spec.loader == LocalModelLoader.VOYAGE_BIDIRECTIONAL_PROJECTION: + from huggingface_hub import hf_hub_download + + # The base load cached this file, and a commit-sha revision reads a + # cached file with no network call. + weights_path = hf_hub_download( + spec.model_name, + VOYAGE_NANO_PROJECTION_FILE, + revision=revision, + cache_dir=_cache_dir(), + ) + model = add_bidirectional_projection( + model, read_projection_weight(weights_path) + ) + logger.info( + "%s: transformers reports 'linear.weight' as UNEXPECTED. This is " + "expected: Onyx adds it back as the projection layer.", + spec.model_name, + ) + + _check_embedding_dim(model, spec) + logger.notice( + "Loaded %s at revision %s (device=%s, dtype=%s)", + spec.model_name, + revision, + model.device, + model.dtype, + ) + return model + + +def _load_legacy_model(model_name: str) -> "SentenceTransformer": + from sentence_transformers import SentenceTransformer + + def load(local_files_only: bool) -> "SentenceTransformer": + # The call from before the registry. Only local_files_only changes. + return SentenceTransformer( + model_name_or_path=model_name, + local_files_only=local_files_only, + trust_remote_code=False, + ) + + return _load_local_first(model_name, None, load) + + +def load_embedding_model(model_name: str) -> "SentenceTransformer": + """Load a self-hosted embedding model. Blocking; raises on failure.""" + spec = get_local_model_spec(model_name) + if spec is not None: + return _load_registry_model(spec) + return _load_legacy_model(model_name) diff --git a/backend/model_server/encoders.py b/backend/model_server/encoders.py index 97421cab7b..ee306c5fc1 100644 --- a/backend/model_server/encoders.py +++ b/backend/model_server/encoders.py @@ -1,12 +1,16 @@ import asyncio +import threading import time +from concurrent.futures import Future, ThreadPoolExecutor +from functools import partial from typing import TYPE_CHECKING, Any from fastapi import APIRouter, HTTPException, Request +from model_server.embedding_model_loader import load_embedding_model from model_server.utils import simple_log_function_time from onyx.utils.logger import setup_logger -from shared_configs.configs import DEFAULT_DOCUMENT_ENCODER_MODEL +from shared_configs.embedding_models import get_local_model_spec from shared_configs.enums import EmbedTextType from shared_configs.model_server_models import Embedding, EmbedRequest, EmbedResponse @@ -19,6 +23,66 @@ router = APIRouter(prefix="/encoder") _GLOBAL_MODELS_DICT: dict[str, "SentenceTransformer"] = {} +# The longest sequence that each model's RoPE caches are pre-warmed to. +_ROPE_PREWARMED_TO: dict[str, int] = {} + +# One lock per model, so a model loads once even if requests race. +_MODEL_LOCKS: dict[str, threading.Lock] = {} +_MODEL_LOCKS_GUARD = threading.Lock() + +# Loads (downloads included) run here, off the event loop. Requests for the same +# model and context length share one job, so requests that wait on a download +# hold no threads. +# Two workers: at most PRESENT + FUTURE weights load at once, which caps memory. +_MODEL_LOAD_EXECUTOR = ThreadPoolExecutor( + max_workers=2, thread_name_prefix="embedding-model-load" +) +_PENDING_MODEL_JOBS: dict[tuple[str, int], Future["SentenceTransformer"]] = {} +_PENDING_MODEL_JOBS_GUARD = threading.Lock() + +# Registry models get room for the query/passage prefix, which the chunker does +# not count, so the prefix does not cut off the end of a full chunk. +REGISTRY_MODEL_PREFIX_HEADROOM_TOKENS = 32 + + +def _model_lock(model_name: str) -> threading.Lock: + with _MODEL_LOCKS_GUARD: + lock = _MODEL_LOCKS.get(model_name) + if lock is None: + lock = threading.Lock() + _MODEL_LOCKS[model_name] = lock + return lock + + +def _target_max_seq_length(model_name: str, max_context_length: int) -> int: + if get_local_model_spec(model_name) is not None: + return max_context_length + REGISTRY_MODEL_PREFIX_HEADROOM_TOKENS + return max_context_length + + +def _prewarm_rope( + model_name: str, st_model: "SentenceTransformer", target_len: int +) -> None: + """ + Build RoPE cos/sin caches once on the final device/dtype so later forwards only read. + Works by calling the underlying HF model directly with dummy IDs/attention. + """ + try: + # ensure > max seq after tokenization + # Ideally we would use the saved tokenizer, but whatever it's ok + # we'll make an assumption about tokenization here + long_text = "x " * (target_len * 2) + _ = st_model.encode( + [long_text], + batch_size=1, + convert_to_tensor=True, + show_progress_bar=False, + normalize_embeddings=False, + ) + _ROPE_PREWARMED_TO[model_name] = target_len + logger.info("RoPE pre-warm successful") + except Exception as e: + logger.warning("RoPE pre-warm skipped/failed: %s", e) def get_embedding_model( @@ -26,53 +90,65 @@ def get_embedding_model( max_context_length: int, ) -> "SentenceTransformer": """ - Loads or returns a cached SentenceTransformer, sets max_seq_length, pins device, - pre-warms rotary caches once, and wraps encode() with a lock to avoid cache races. + Loads or returns a cached SentenceTransformer, sets max_seq_length and + pre-warms rotary caches. + + Blocking: a first load can download gigabytes. Async code must use + `_get_embedding_model_off_loop`. """ - from sentence_transformers import SentenceTransformer + max_seq_length = _target_max_seq_length(model_name, max_context_length) - def _prewarm_rope(st_model: "SentenceTransformer", target_len: int) -> None: - """ - Build RoPE cos/sin caches once on the final device/dtype so later forwards only read. - Works by calling the underlying HF model directly with dummy IDs/attention. - """ - try: - # ensure > max seq after tokenization - # Ideally we would use the saved tokenizer, but whatever it's ok - # we'll make an assumption about tokenization here - long_text = "x " * (target_len * 2) - _ = st_model.encode( - [long_text], - batch_size=1, - convert_to_tensor=True, - show_progress_bar=False, - normalize_embeddings=False, + with _model_lock(model_name): + model = _GLOBAL_MODELS_DICT.get(model_name) + if model is None: + logger.notice("Loading %s", model_name) + # Raises on failure, so a failed or rejected model is never cached. + model = load_embedding_model(model_name) + model.max_seq_length = max_seq_length + _prewarm_rope(model_name, model, max_seq_length) + _GLOBAL_MODELS_DICT[model_name] = model + elif max_seq_length != model.max_seq_length: + model.max_seq_length = max_seq_length + # Caches built for a longer sequence also serve a shorter one. + if max_seq_length > _ROPE_PREWARMED_TO.get(model_name, 0): + _prewarm_rope(model_name, model, max_seq_length) + + return model + + +def _forget_model_job( + job_key: tuple[str, int], job: Future["SentenceTransformer"] +) -> None: + with _PENDING_MODEL_JOBS_GUARD: + if _PENDING_MODEL_JOBS.get(job_key) is job: + del _PENDING_MODEL_JOBS[job_key] + + +async def _get_embedding_model_off_loop( + model_name: str, max_context_length: int +) -> "SentenceTransformer": + """`get_embedding_model` without blocking the event loop.""" + model = _GLOBAL_MODELS_DICT.get(model_name) + if model is not None and model.max_seq_length == _target_max_seq_length( + model_name, max_context_length + ): + return model + + job_key = (model_name, max_context_length) + with _PENDING_MODEL_JOBS_GUARD: + job = _PENDING_MODEL_JOBS.get(job_key) + is_new_job = job is None + if job is None: + job = _MODEL_LOAD_EXECUTOR.submit( + get_embedding_model, model_name, max_context_length ) - logger.info("RoPE pre-warm successful") - except Exception as e: - logger.warning("RoPE pre-warm skipped/failed: %s", e) + _PENDING_MODEL_JOBS[job_key] = job + if is_new_job: + # Outside the guard: the callback runs at once if the job is done. + job.add_done_callback(partial(_forget_model_job, job_key)) - global _GLOBAL_MODELS_DICT - - if model_name not in _GLOBAL_MODELS_DICT: - logger.notice("Loading %s", model_name) - model = SentenceTransformer( - model_name_or_path=model_name, - local_files_only=model_name == DEFAULT_DOCUMENT_ENCODER_MODEL, - trust_remote_code=False, - ) - model.max_seq_length = max_context_length - _prewarm_rope(model, max_context_length) - _GLOBAL_MODELS_DICT[model_name] = model - else: - model = _GLOBAL_MODELS_DICT[model_name] - if max_context_length != model.max_seq_length: - model.max_seq_length = max_context_length - prev = getattr(model, "_rope_prewarmed_to", 0) # ods: ignore[getattr] - if max_context_length > int(prev or 0): - _prewarm_rope(model, max_context_length) - - return _GLOBAL_MODELS_DICT[model_name] + # A cancelled request must not cancel the job that other requests share. + return await asyncio.shield(asyncio.wrap_future(job)) ENCODING_RETRIES = 3 @@ -132,7 +208,7 @@ async def embed_text( prefixed_texts = [f"{prefix}{text}" for text in texts] if prefix else texts - local_model = get_embedding_model( + local_model = await _get_embedding_model_off_loop( model_name=model_name, max_context_length=max_context_length ) # Run CPU-bound embedding in a thread pool diff --git a/backend/model_server/voyage_nano.py b/backend/model_server/voyage_nano.py new file mode 100644 index 0000000000..08b178e645 --- /dev/null +++ b/backend/model_server/voyage_nano.py @@ -0,0 +1,56 @@ +"""Onyx-owned load fix-up for ``voyageai/voyage-4-nano``, with no remote code. + +A plain SentenceTransformer load of this checkpoint gives wrong vectors and no +error: transformers loads the native, causal ``Qwen3Model`` and drops the +bias-free ``linear.weight`` projection (1024 -> 2048). The checkpoint's remote +code fails on transformers 5.14.1, and Onyx never runs remote code anyway. The +fix makes attention bidirectional and adds the projection as a bias-free +``Dense`` layer after mean pooling: pooling is linear, so this equals the remote +code's per-token projection. Checked on real weights against an independent +re-implementation of the remote model: 1 - cosine < 2e-7. +""" + +from typing import TYPE_CHECKING, cast + +import torch + +if TYPE_CHECKING: + from sentence_transformers import SentenceTransformer + +# The checkpoint file that holds the projection, and the tensor name in it. +VOYAGE_NANO_PROJECTION_FILE = "model.safetensors" +_PROJECTION_WEIGHT_KEY = "linear.weight" + + +def read_projection_weight(safetensors_path: str) -> torch.Tensor: + """Read the bias-free projection weight, shape [out_dim, hidden_dim].""" + from safetensors import safe_open + + with safe_open(safetensors_path, framework="pt") as checkpoint: + return checkpoint.get_tensor(_PROJECTION_WEIGHT_KEY) + + +def add_bidirectional_projection( + model: "SentenceTransformer", projection_weight: torch.Tensor +) -> "SentenceTransformer": + """Return voyage-4-nano with bidirectional attention and the 2048-dim projection. + + ``model`` is the plain native load (Transformer -> Pooling -> Normalize). + """ + from sentence_transformers import SentenceTransformer + from sentence_transformers.base.modules import Transformer + from sentence_transformers.sentence_transformer.modules import Dense + + transformer, pooling, normalize = model[0], model[1], model[2] + # The Qwen3 forward reads this on every call. It must be set after the load: + # Qwen3Config ignores an is_causal config kwarg. + cast(Transformer, transformer).auto_model.config.is_causal = False + + projection = Dense( + in_features=projection_weight.shape[1], + out_features=projection_weight.shape[0], + bias=False, + activation_function=None, # identity + init_weight=projection_weight, + ) + return SentenceTransformer(modules=[transformer, pooling, projection, normalize]) diff --git a/backend/onyx/configs/app_configs.py b/backend/onyx/configs/app_configs.py index 6407bdbad8..d1da63ebcb 100644 --- a/backend/onyx/configs/app_configs.py +++ b/backend/onyx/configs/app_configs.py @@ -577,6 +577,13 @@ DOCUMENT_INDEX_NUM_ATTEMPTS_ON_STARTUP = int( os.environ.get("NUM_RETRIES_ON_STARTUP") or 10 ) +# Seconds the API server waits on startup for the model server to load the +# self-hosted embedding model, which includes a download if it is not cached. +# Startup fails when the wait runs out. 0 disables the wait. +LOCAL_EMBEDDING_MODEL_STARTUP_TIMEOUT = int( + os.environ.get("LOCAL_EMBEDDING_MODEL_STARTUP_TIMEOUT") or 1800 +) + # Number of documents in a batch during indexing (further batching done by chunks before passing to bi-encoder) INDEX_BATCH_SIZE = int(os.environ.get("INDEX_BATCH_SIZE") or 16) diff --git a/backend/onyx/configs/model_configs.py b/backend/onyx/configs/model_configs.py index 53b256c0c7..ae988c1ff4 100644 --- a/backend/onyx/configs/model_configs.py +++ b/backend/onyx/configs/model_configs.py @@ -1,7 +1,9 @@ import json import os +from collections.abc import Mapping from shared_configs.configs import DEFAULT_DOCUMENT_ENCODER_MODEL +from shared_configs.embedding_models import find_embedding_model_spec ##### # Embedding/Reranking Model Configs @@ -17,11 +19,38 @@ from shared_configs.configs import DEFAULT_DOCUMENT_ENCODER_MODEL DOCUMENT_ENCODER_MODEL = ( os.environ.get("DOCUMENT_ENCODER_MODEL") or DEFAULT_DOCUMENT_ENCODER_MODEL ) -DOC_EMBEDDING_DIM = int(os.environ.get("DOC_EMBEDDING_DIM") or 768) +# The settings below only seed the first search settings row of a fresh DB +# (alembic dbaa756c2ccf). Defaults come from the registry spec of +# DOCUMENT_ENCODER_MODEL (the old literals for other models). Their env vars +# count only with DOCUMENT_ENCODER_MODEL set, so an env file written for nomic +# cannot give the default model nomic's dimension or prefixes. +_MODEL_ENV: Mapping[str, str] = ( + os.environ if os.environ.get("DOCUMENT_ENCODER_MODEL") else {} +) +_DOCUMENT_ENCODER_SPEC = find_embedding_model_spec(None, DOCUMENT_ENCODER_MODEL) + +DOC_EMBEDDING_DIM = int( + _MODEL_ENV.get("DOC_EMBEDDING_DIM") + or (_DOCUMENT_ENCODER_SPEC.model_dim if _DOCUMENT_ENCODER_SPEC else 768) +) +_NORMALIZE_EMBEDDINGS_DEFAULT = ( + _DOCUMENT_ENCODER_SPEC.normalize if _DOCUMENT_ENCODER_SPEC else True +) NORMALIZE_EMBEDDINGS = ( - os.environ.get("NORMALIZE_EMBEDDINGS") or "true" + _MODEL_ENV.get("NORMALIZE_EMBEDDINGS") + or ("true" if _NORMALIZE_EMBEDDINGS_DEFAULT else "false") ).lower() == "true" +# The default self-hosted model before granite. +PREVIOUS_DEFAULT_DOCUMENT_ENCODER_MODEL = "nomic-ai/nomic-embed-text-v1" + +# Tokenizer for chat token counting and for the fallback when a model's own +# tokenizer can't load: env DOCUMENT_ENCODER_MODEL, else nomic (baked into the +# API image), so token counts do not depend on the default embedding model. +DEFAULT_TOKENIZER_MODEL = ( + os.environ.get("DOCUMENT_ENCODER_MODEL") or PREVIOUS_DEFAULT_DOCUMENT_ENCODER_MODEL +) + # Old default model settings, which are needed for an automatic easy upgrade OLD_DEFAULT_DOCUMENT_ENCODER_MODEL = "thenlper/gte-small" OLD_DEFAULT_MODEL_DOC_EMBEDDING_DIM = 384 @@ -32,8 +61,19 @@ OLD_DEFAULT_MODEL_NORMALIZE_EMBEDDINGS = False SIM_SCORE_RANGE_LOW = float(os.environ.get("SIM_SCORE_RANGE_LOW") or 0.0) SIM_SCORE_RANGE_HIGH = float(os.environ.get("SIM_SCORE_RANGE_HIGH") or 1.0) # Certain models like e5, BGE, etc use a prefix for asymmetric retrievals (query generally shorter than docs) -ASYM_QUERY_PREFIX = os.environ.get("ASYM_QUERY_PREFIX", "search_query: ") -ASYM_PASSAGE_PREFIX = os.environ.get("ASYM_PASSAGE_PREFIX", "search_document: ") +# An env var set to "" stays "" (no fallback to the default). +ASYM_QUERY_PREFIX = _MODEL_ENV.get( + "ASYM_QUERY_PREFIX", + _DOCUMENT_ENCODER_SPEC.query_prefix if _DOCUMENT_ENCODER_SPEC else "search_query: ", +) +ASYM_PASSAGE_PREFIX = _MODEL_ENV.get( + "ASYM_PASSAGE_PREFIX", + ( + _DOCUMENT_ENCODER_SPEC.passage_prefix + if _DOCUMENT_ENCODER_SPEC + else "search_document: " + ), +) # Purely an optimization, memory limitation consideration # User's set embedding batch size overrides the default encoding batch sizes diff --git a/backend/onyx/db/search_settings.py b/backend/onyx/db/search_settings.py index fe1cd80295..3ad654a66d 100644 --- a/backend/onyx/db/search_settings.py +++ b/backend/onyx/db/search_settings.py @@ -5,6 +5,7 @@ from sqlalchemy.orm import Session from onyx.configs.model_configs import ( DEFAULT_DOCUMENT_ENCODER_MODEL, DOCUMENT_ENCODER_MODEL, + PREVIOUS_DEFAULT_DOCUMENT_ENCODER_MODEL, ) from onyx.context.search.models import SavedSearchSettings from onyx.db.llm import fetch_embedding_provider @@ -283,7 +284,13 @@ def update_search_settings_status( def user_has_overridden_embedding_model() -> bool: - return DOCUMENT_ENCODER_MODEL != DEFAULT_DOCUMENT_ENCODER_MODEL + """True if env DOCUMENT_ENCODER_MODEL names a non-default model, for the alembic + seed (dbaa756c2ccf). nomic, the previous default, counts as a default so that + DOCUMENT_ENCODER_MODEL=nomic seeds the same rows as an install before granite.""" + return DOCUMENT_ENCODER_MODEL not in ( + DEFAULT_DOCUMENT_ENCODER_MODEL, + PREVIOUS_DEFAULT_DOCUMENT_ENCODER_MODEL, + ) # Old-index reclamation (post-reindex deletion of the now-PAST index). diff --git a/backend/onyx/natural_language_processing/search_nlp_models.py b/backend/onyx/natural_language_processing/search_nlp_models.py index b04e4bec83..2b66598f21 100644 --- a/backend/onyx/natural_language_processing/search_nlp_models.py +++ b/backend/onyx/natural_language_processing/search_nlp_models.py @@ -333,10 +333,6 @@ WARM_UP_STRINGS = [ ] -def clean_model_name(model_str: str) -> str: - return model_str.replace("/", "_").replace("-", "_").replace(".", "_").lower() - - def build_model_server_url( model_server_host: str, model_server_port: int, @@ -496,7 +492,7 @@ def _extract_cohere_embeddings(response_embeddings: Any) -> list[Embedding]: """Normalize Cohere's embed response across SDK versions. v3 models return ``response.embeddings`` as a flat ``list[list[float]]``. - v4 models return an ``EmbedByTypeResponseEmbeddings`` object that carries + v4 and v5 models return an ``EmbedByTypeResponseEmbeddings`` object that carries the embeddings under ``float_`` (and possibly other per-type buckets when ``embedding_types`` is set). We always read the float bucket here since we never request alternate types. @@ -1328,19 +1324,18 @@ class EmbeddingModel: reraise=True, )(final_make_request_func) - response: Response | None = None - try: response = final_make_request_func() return EmbedResponse(**response.json()) except requests.HTTPError as e: - if not response: + # raise_for_status() raised, so the failed response is only on the error. + if e.response is None: raise HTTPError("HTTP error occurred - response is None.") from e try: - error_detail = response.json().get("detail", str(e)) + error_detail = e.response.json().get("detail", str(e)) except Exception: - error_detail = response.text + error_detail = e.response.text raise HTTPError(f"HTTP error occurred: {error_detail}") from e except requests.RequestException as e: raise HTTPError(f"Request failed: {str(e)}") from e diff --git a/backend/onyx/natural_language_processing/utils.py b/backend/onyx/natural_language_processing/utils.py index 190094b2fb..4cb4db4bfd 100644 --- a/backend/onyx/natural_language_processing/utils.py +++ b/backend/onyx/natural_language_processing/utils.py @@ -4,10 +4,11 @@ from copy import copy from tokenizers import Encoding, Tokenizer -from onyx.configs.model_configs import DOCUMENT_ENCODER_MODEL +from onyx.configs.model_configs import DEFAULT_TOKENIZER_MODEL from onyx.context.search.models import InferenceChunk from onyx.utils.logger import setup_logger from shared_configs.configs import DOC_EMBEDDING_CONTEXT_SIZE +from shared_configs.embedding_models import get_local_model_spec from shared_configs.enums import EmbeddingProvider TRIM_SEP_PAT = "\n... {n} tokens removed...\n" @@ -69,7 +70,15 @@ class TiktokenTokenizer(BaseTokenizer): class HuggingFaceTokenizer(BaseTokenizer): def __init__(self, model_name: str): - self.encoder: Tokenizer = Tokenizer.from_pretrained(model_name) + # Registry models load at their pinned revision, the same one the model + # server and the image bake use. Any other name keeps the plain call. + local_model_spec = get_local_model_spec(model_name) + revision = local_model_spec.hf_revision if local_model_spec else None + self.encoder: Tokenizer = ( + Tokenizer.from_pretrained(model_name) + if revision is None + else Tokenizer.from_pretrained(model_name, revision=revision) + ) def _safer_encode(self, string: str) -> Encoding: """ @@ -113,7 +122,7 @@ def _check_tokenizer_cache( if not tokenizer: logger.info( "Falling back to default embedding model tokenizer: %s", - DOCUMENT_ENCODER_MODEL, + DEFAULT_TOKENIZER_MODEL, ) tokenizer = _get_default_tokenizer() @@ -168,7 +177,7 @@ def _get_default_tokenizer() -> BaseTokenizer: """Lazy-load the default tokenizer to avoid loading it at module import time.""" global _DEFAULT_TOKENIZER if _DEFAULT_TOKENIZER is None: - _DEFAULT_TOKENIZER = HuggingFaceTokenizer(DOCUMENT_ENCODER_MODEL) + _DEFAULT_TOKENIZER = HuggingFaceTokenizer(DEFAULT_TOKENIZER_MODEL) return _DEFAULT_TOKENIZER diff --git a/backend/onyx/server/manage/search_settings.py b/backend/onyx/server/manage/search_settings.py index 6fb2bbf753..b0615b84fc 100644 --- a/backend/onyx/server/manage/search_settings.py +++ b/backend/onyx/server/manage/search_settings.py @@ -97,7 +97,6 @@ from onyx.file_processing.unstructured import ( ) from onyx.indexing.port_reembed import ReembedStrategy, select_reembed_strategy from onyx.natural_language_processing.embedding_auth import build_embedding_auth -from onyx.natural_language_processing.search_nlp_models import clean_model_name from onyx.server.manage.embedding.models import SearchSettingsDeleteRequest from onyx.server.manage.embedding.probe import probe_embedding_dimension from onyx.server.manage.models import ( @@ -123,7 +122,15 @@ from shared_configs.configs import ( PRESERVED_SEARCH_FIELDS, ) from shared_configs.contextvars import get_current_tenant_id +from shared_configs.embedding_models import ( + EmbeddingModelSpec, + EmbeddingModelStatus, + embedding_model_match_key, + find_embedding_model_spec, + selectable_embedding_model_specs, +) from shared_configs.enums import EmbeddingProvider +from shared_configs.utils import clean_model_name router = APIRouter(prefix="/search-settings") logger = setup_logger() @@ -140,15 +147,32 @@ def set_new_search_settings( Only one re-index runs at a time. This raises CONFLICT instead of superseding an existing one: either a secondary FUTURE is already in flight, or an INSTANT switchover is still backfilling the live index. Cancel the running re-index first. + + Embedding models only move forward: legacy models are refused as a new target, + except for a re-index with the PRESENT model (see _check_embedding_model_target). """ if search_settings_new.index_name: logger.warning("Index name was specified by request, this is not suggested") # Disallow contextual RAG for cloud deployments. if MULTI_TENANT and search_settings_new.enable_contextual_rag: - raise HTTPException( - status_code=status.HTTP_400_BAD_REQUEST, - detail="Contextual RAG disabled in Onyx Cloud", + raise OnyxError( + OnyxErrorCode.INVALID_INPUT, "Contextual RAG disabled in Onyx Cloud" + ) + + spec = find_embedding_model_spec( + search_settings_new.provider_type, search_settings_new.model_name + ) + selectable_spec = ( + spec + if spec is not None and spec.status == EmbeddingModelStatus.SELECTABLE + else None + ) + # A selectable model gets its canonical name: the probe, the model server and + # the tokenizer match it exactly. + if selectable_spec is not None: + search_settings_new = search_settings_new.model_copy( + update={"model_name": selectable_spec.model_name} ) # Validate cloud provider exists or create new LiteLLM provider. @@ -163,8 +187,14 @@ def set_new_search_settings( f"No embedding provider exists for cloud embedding type {search_settings_new.provider_type}", ) - if search_settings_new.provider_type == EmbeddingProvider.BIFROST: - _validate_gateway_embedding_model(search_settings_new, cloud_provider) + # Before the PRESENT row lock below: the probe is a network call. Nothing + # is written yet, so end the read transaction instead of idling in it. + if ( + search_settings_new.provider_type == EmbeddingProvider.BIFROST + or selectable_spec is not None + ): + db_session.commit() + _validate_cloud_embedding_model(search_settings_new, cloud_provider) validate_contextual_rag_model( model_configuration_id=search_settings_new.contextual_rag_model_configuration_id, @@ -200,6 +230,10 @@ def set_new_search_settings( "A re-index is already in progress. Cancel it before starting a new one.", ) + # Upgrade-only: a new target must be the PRESENT model or a selectable model. + # Runs before anything below writes or commits. + _check_embedding_model_target(search_settings_new, search_settings) + if search_settings_new.index_name is None: search_values = search_settings_new.model_dump() search_values["index_name"] = _compute_index_name( @@ -306,14 +340,13 @@ def set_new_search_settings( return IdReturn(id=new_search_settings.id) -def _validate_gateway_embedding_model( +def _validate_cloud_embedding_model( search_settings_new: SearchSettingsCreationRequest, cloud_provider: CloudEmbeddingProviderModel, ) -> None: - """A gateway accepts any model ID, so prove the model embeds at the set dimension. - - Rejects a chat model or a wrong dimension before indexing starts. - """ + """Prove the model embeds at the set dimension with the stored credentials, so + a bad key, region or model ID (a gateway accepts any) fails before indexing + starts.""" api_key: str | None = ( cloud_provider.api_key.get_value(apply_mask=False) if cloud_provider.api_key is not None @@ -324,7 +357,11 @@ def _validate_gateway_embedding_model( api_key=api_key, api_url=cloud_provider.api_url, model_name=search_settings_new.model_name, - auth=build_embedding_auth(cloud_provider.provider_type, api_key), + auth=build_embedding_auth( + cloud_provider.provider_type, api_key, cloud_provider.vertex_config + ), + api_version=cloud_provider.api_version, + deployment_name=cloud_provider.deployment_name, reduced_dimension=search_settings_new.reduced_dimension, ) if dimension != search_settings_new.final_embedding_dim: @@ -336,6 +373,153 @@ def _validate_gateway_embedding_model( ) +# Providers whose model list Onyx owns. Only SELECTABLE registry models are valid +# new targets for them. LiteLLM, Azure and Bifrost take free-form names, so they +# stay open. +_REGISTRY_ONLY_CLOUD_PROVIDERS = frozenset( + { + EmbeddingProvider.OPENAI, + EmbeddingProvider.COHERE, + EmbeddingProvider.GOOGLE, + EmbeddingProvider.VOYAGE, + } +) + + +def _embedding_provider_label(provider_type: EmbeddingProvider | None) -> str: + return provider_type.value if provider_type is not None else "self-hosted" + + +def _selectable_embedding_models_hint() -> str: + names_by_provider: dict[str, list[str]] = {} + for spec in selectable_embedding_model_specs(): + names_by_provider.setdefault( + _embedding_provider_label(spec.provider_type), [] + ).append(spec.model_name) + return "; ".join( + f"{provider_label}: {', '.join(names)}" + for provider_label, names in names_by_provider.items() + ) + + +def _not_selectable_error( + request: SearchSettingsCreationRequest, spec: EmbeddingModelSpec | None +) -> OnyxError: + model = ( + f"'{request.model_name}' ({_embedding_provider_label(request.provider_type)})" + ) + if spec is not None: + reason = ( + f"{model} is a legacy embedding model. Existing indexes on it keep " + "working, but it can't be chosen for a new index." + ) + else: + reason = f"{model} is not an embedding model that Onyx supports." + return OnyxError( + OnyxErrorCode.INVALID_INPUT, + f"{reason} Choose one of these models: {_selectable_embedding_models_hint()}. " + "A re-index with the current model is still allowed.", + ) + + +def _validate_selectable_embedding_request( + request: SearchSettingsCreationRequest, spec: EmbeddingModelSpec +) -> None: + """A selectable model must use the registry's settings: the index and the + embedding code depend on them.""" + problems: list[str] = [] + if request.model_dim != spec.model_dim: + problems.append(f"model_dim must be {spec.model_dim}, not {request.model_dim}") + if request.normalize != spec.normalize: + problems.append(f"normalize must be {spec.normalize}") + if (request.query_prefix or "") != spec.query_prefix: + problems.append(f"query_prefix must be {spec.query_prefix!r}") + if (request.passage_prefix or "") != spec.passage_prefix: + problems.append(f"passage_prefix must be {spec.passage_prefix!r}") + if request.reduced_dimension is not None: + if MULTI_TENANT: + problems.append("reduced_dimension is not supported in Onyx Cloud") + elif not spec.supports_reduced_dimension: + problems.append("this model does not support reduced_dimension") + elif not 0 < request.reduced_dimension < spec.model_dim: + problems.append( + f"reduced_dimension must be between 1 and {spec.model_dim - 1}" + ) + if problems: + raise OnyxError( + OnyxErrorCode.INVALID_INPUT, + f"Invalid settings for embedding model '{spec.model_name}': " + f"{'; '.join(problems)}.", + ) + + +def _check_embedding_model_target( + request: SearchSettingsCreationRequest, present: SearchSettings +) -> None: + """Upgrade-only rules for a new embedding target. Allowed: the PRESENT model + (legacy included), a selectable model with the registry settings, and any + model of a provider whose list Onyx does not own (custom HF models, LiteLLM, + Azure, Bifrost). Other OpenAI, Cohere, Google and Voyage models and legacy + self-hosted models raise OnyxError(INVALID_INPUT).""" + spec = find_embedding_model_spec(request.provider_type, request.model_name) + selectable_spec = ( + spec + if spec is not None and spec.status == EmbeddingModelStatus.SELECTABLE + else None + ) + is_present_model = embedding_model_match_key( + request.provider_type, request.model_name + ) == embedding_model_match_key(present.provider_type, present.model_name) + + # The model server ignores reduced_dimension, so a self-hosted index with it + # gets vectors of the wrong size. A re-index keeps a stored value. + if ( + request.provider_type is None + and request.reduced_dimension is not None + and not ( + is_present_model and request.reduced_dimension == present.reduced_dimension + ) + ): + raise OnyxError( + OnyxErrorCode.INVALID_INPUT, + "Self-hosted embedding models do not support reduced_dimension: the " + "model server returns vectors of the full dimension.", + ) + + # A re-index with the PRESENT model (quantization, contextual RAG, switchover). + # Stored rows often differ from the registry (e.g. empty prefixes), so only + # the dimension is checked. Tenants share the index of a model, so + # MULTI_TENANT also keeps reduced_dimension. + if is_present_model: + allowed_dims = {present.model_dim} + if selectable_spec is not None: + allowed_dims.add(selectable_spec.model_dim) + if request.model_dim not in allowed_dims: + raise OnyxError( + OnyxErrorCode.INVALID_INPUT, + f"A re-index with the current model must keep its dimension: " + f"model_dim must be {' or '.join(str(d) for d in sorted(allowed_dims))}, " + f"not {request.model_dim}.", + ) + if MULTI_TENANT and request.reduced_dimension != present.reduced_dimension: + raise OnyxError( + OnyxErrorCode.INVALID_INPUT, + "reduced_dimension can't be changed in Onyx Cloud: " + "all tenants on a model share its index.", + ) + return + + if selectable_spec is not None: + _validate_selectable_embedding_request(request, selectable_spec) + return + + # Any spec left here is LEGACY. + if request.provider_type in _REGISTRY_ONLY_CLOUD_PROVIDERS or ( + request.provider_type is None and spec is not None + ): + raise _not_selectable_error(request, spec) + + def _validate_vector_quantization_supported( vector_quantization: VectorQuantization, ) -> None: diff --git a/backend/onyx/setup.py b/backend/onyx/setup.py index 0b1f4911f1..4dfb68109f 100644 --- a/backend/onyx/setup.py +++ b/backend/onyx/setup.py @@ -7,6 +7,7 @@ from onyx.configs.app_configs import ( DISABLE_VECTOR_DB, DOCUMENT_INDEX_NUM_ATTEMPTS_ON_STARTUP, INTEGRATION_TESTS_MODE, + LOCAL_EMBEDDING_MODEL_STARTUP_TIMEOUT, ) from onyx.configs.constants import KV_REINDEX_KEY from onyx.configs.model_configs import GEN_AI_API_KEY, GEN_AI_MODEL_VERSION @@ -30,6 +31,7 @@ from onyx.db.llm import ( update_default_model, upsert_llm_provider, ) +from onyx.db.models import SearchSettings from onyx.db.search_settings import ( get_active_search_settings, get_current_search_settings, @@ -49,10 +51,7 @@ from onyx.key_value_store.factory import get_kv_store from onyx.key_value_store.interface import KvKeyNotFoundError from onyx.llm.constants import LlmProviderNames from onyx.llm.well_known_providers.llm_provider_options import get_openai_model_names -from onyx.natural_language_processing.search_nlp_models import ( - EmbeddingModel, - warm_up_bi_encoder, -) +from onyx.natural_language_processing.search_nlp_models import EmbeddingModel from onyx.server.manage.llm.models import ( LLMProviderUpsertRequest, ModelConfigurationUpsertRequest, @@ -64,14 +63,20 @@ from onyx.server.settings.store import ( ) from onyx.utils.gpu_utils import gpu_status_request from onyx.utils.logger import setup_logger +from onyx.utils.threadpool_concurrency import start_thread_future from shared_configs.configs import ( + DISABLE_MODEL_SERVER, MODEL_SERVER_HOST, MODEL_SERVER_PORT, MULTI_TENANT, ) +from shared_configs.enums import EmbedTextType logger = setup_logger() +_STARTUP_EMBED_TEXT = "Onyx startup check" +_STARTUP_RETRY_SECONDS = 10 + def setup_onyx( db_session: Session, @@ -145,21 +150,83 @@ def setup_onyx( logger.notice( "Model Server: http://%s:%s", MODEL_SERVER_HOST, MODEL_SERVER_PORT ) - if search_settings.provider_type is None: - # In integration tests, do not block API startup on warm-up - warm_up_bi_encoder( - embedding_model=EmbeddingModel.from_db_model( - search_settings=search_settings, - server_host=MODEL_SERVER_HOST, - server_port=MODEL_SERVER_PORT, - ), - non_blocking=INTEGRATION_TESTS_MODE, - ) + # Multi-tenant tenant creation and integration tests must not block on it. + if ( + search_settings.provider_type is None + and LOCAL_EMBEDDING_MODEL_STARTUP_TIMEOUT > 0 + and not (MULTI_TENANT or INTEGRATION_TESTS_MODE or DISABLE_MODEL_SERVER) + ): + wait_for_local_embedding_model(search_settings) # update multipass indexing setting based on GPU availability update_default_multipass_indexing(db_session) +def wait_for_local_embedding_model(search_settings: SearchSettings) -> None: + """Block until the inference model server embeds with the self-hosted model in + use. Its first request loads the model, with a download if it is not cached + (e.g. nomic, which the image does not bundle). Raise when the wait runs out, + so the API server does not start with search that cannot embed queries.""" + model_name = search_settings.model_name + embedding_model = EmbeddingModel.from_db_model( + search_settings=search_settings, + server_host=MODEL_SERVER_HOST, + server_port=MODEL_SERVER_PORT, + ) + logger.notice( + "Waiting up to %ss for the model server to load embedding model %s. " + "It downloads the model first if the model is not in its cache.", + LOCAL_EMBEDDING_MODEL_STARTUP_TIMEOUT, + model_name, + ) + start = time.monotonic() + deadline = start + LOCAL_EMBEDDING_MODEL_STARTUP_TIMEOUT + last_error: Exception | None = None + while (remaining := deadline - time.monotonic()) > 0: + try: + # A daemon thread: a request that still waits at the deadline does + # not keep the failed process from exiting. + start_thread_future( + lambda: embedding_model.encode( + texts=[_STARTUP_EMBED_TEXT], text_type=EmbedTextType.QUERY + ), + name="embedding-model-startup-check", + ).result(timeout=remaining) + except TimeoutError: + # The deadline passed during a request, e.g. a slow download. + last_error = None + break + except Exception as e: + last_error = e + logger.warning( + "Embedding model %s is not ready on the model server after %.0fs: %s", + model_name, + time.monotonic() - start, + e, + ) + time.sleep(min(_STARTUP_RETRY_SECONDS, max(deadline - time.monotonic(), 0))) + continue + logger.notice( + "Embedding model %s is ready on the model server (%.0fs).", + model_name, + time.monotonic() - start, + ) + return + + reason = ( + f"Last error: {last_error}" + if last_error + else "The last request was still waiting for the model server." + ) + raise RuntimeError( + f"The model server at {MODEL_SERVER_HOST}:{MODEL_SERVER_PORT} did not load " + f"embedding model {model_name} within {LOCAL_EMBEDDING_MODEL_STARTUP_TIMEOUT}s. " + f"{reason} If the model server cannot download the model (for example, it " + "has no internet access), add the model to its Hugging Face cache. " + "LOCAL_EMBEDDING_MODEL_STARTUP_TIMEOUT sets the wait; 0 disables it." + ) from last_error + + def mark_reindex_flag(db_session: Session) -> None: kv_store = get_kv_store() try: diff --git a/backend/shared_configs/configs.py b/backend/shared_configs/configs.py index d709be0720..f77b44c087 100644 --- a/backend/shared_configs/configs.py +++ b/backend/shared_configs/configs.py @@ -35,7 +35,8 @@ INDEXING_MODEL_SERVER_PORT = int( CONNECTOR_CLASSIFIER_MODEL_REPO = "Danswer/filter-extraction-model" CONNECTOR_CLASSIFIER_MODEL_TAG = "1.0.0" INTENT_MODEL_VERSION = "onyx-dot-app/hybrid-intent-token-classifier" -DEFAULT_DOCUMENT_ENCODER_MODEL = "nomic-ai/nomic-embed-text-v1" +# Fresh-install default self-hosted model, baked into the model-server image. +DEFAULT_DOCUMENT_ENCODER_MODEL = "ibm-granite/granite-embedding-97m-multilingual-r2" # INTENT_MODEL_TAG = "v1.0.3" INTENT_MODEL_TAG: str | None = None # Bi-Encoder, other details diff --git a/backend/shared_configs/embedding_models.py b/backend/shared_configs/embedding_models.py new file mode 100644 index 0000000000..c70c94b94d --- /dev/null +++ b/backend/shared_configs/embedding_models.py @@ -0,0 +1,244 @@ +"""Registry of known embedding models, shared by the API server (upgrade-only +guard, tokenizer revision, config defaults) and the model server (loader +settings). Keep it pure (pydantic, the stdlib, ``shared_configs``): the +model-server image has no ``onyx`` package and the API image has no torch. + +Only add entries: a deployment can hold any entry as PRESENT, FUTURE or PAST. +``model_name`` is the exact ``SearchSettings.model_name``: the bare name for +cloud providers, the HF repo id for self-hosted models. +""" + +from enum import Enum + +from pydantic import BaseModel + +from shared_configs.configs import DEFAULT_DOCUMENT_ENCODER_MODEL +from shared_configs.enums import EmbeddingProvider +from shared_configs.utils import clean_model_name + + +class EmbeddingModelStatus(str, Enum): + """Whether a model can be chosen as a new embedding target.""" + + # Offered to admins as a new target. + SELECTABLE = "selectable" + # Kept only so existing deployments keep working. Not a valid new target, + # except for a same-model re-index of the PRESENT model. + LEGACY = "legacy" + + +class LocalModelLoader(str, Enum): + """How the model server loads a SELECTABLE self-hosted model.""" + + # Plain SentenceTransformer load. + STANDARD = "standard" + # voyage-4-nano: native Qwen3 with bidirectional attention, plus the + # bias-free 1024 -> 2048 projection from the checkpoint. No remote code. + VOYAGE_BIDIRECTIONAL_PROJECTION = "voyage_bidirectional_projection" + + +class EmbeddingModelSpec(BaseModel, frozen=True): + """One known embedding model. LEGACY values only seed env defaults + (``onyx.configs.model_configs``). Stored rows often differ (e.g. empty + prefixes), so never compare a request or a row against them.""" + + # None means self-hosted (the model server runs it). + provider_type: EmbeddingProvider | None + model_name: str + model_dim: int + normalize: bool + query_prefix: str + passage_prefix: str + status: EmbeddingModelStatus + # Pinned HF commit sha. Set only for SELECTABLE self-hosted models. + hf_revision: str | None = None + loader: LocalModelLoader = LocalModelLoader.STANDARD + # True if the embedding path honors `reduced_dimension` (Matryoshka). + supports_reduced_dimension: bool = False + + +_SELECTABLE = EmbeddingModelStatus.SELECTABLE +_LEGACY = EmbeddingModelStatus.LEGACY + +_E5_QUERY_PREFIX = "query: " +_E5_PASSAGE_PREFIX = "passage: " + + +def _cloud_spec( + provider_type: EmbeddingProvider, + model_name: str, + model_dim: int, + status: EmbeddingModelStatus, + supports_reduced_dimension: bool = False, +) -> EmbeddingModelSpec: + """Cloud models: no Onyx-side normalize and no prefixes.""" + return EmbeddingModelSpec( + provider_type=provider_type, + model_name=model_name, + model_dim=model_dim, + normalize=False, + query_prefix="", + passage_prefix="", + status=status, + supports_reduced_dimension=supports_reduced_dimension, + ) + + +def _legacy_self_hosted_spec( + model_name: str, + model_dim: int, + normalize: bool, + query_prefix: str, + passage_prefix: str, +) -> EmbeddingModelSpec: + return EmbeddingModelSpec( + provider_type=None, + model_name=model_name, + model_dim=model_dim, + normalize=normalize, + query_prefix=query_prefix, + passage_prefix=passage_prefix, + status=_LEGACY, + ) + + +EMBEDDING_MODEL_SPECS: tuple[EmbeddingModelSpec, ...] = ( + # ---- SELECTABLE: self-hosted ---- + EmbeddingModelSpec( + provider_type=None, + model_name=DEFAULT_DOCUMENT_ENCODER_MODEL, + model_dim=384, + normalize=True, + query_prefix="", + passage_prefix="", + status=_SELECTABLE, + hf_revision="835ad14087e140460703cf0fae09f97d469d65c2", + ), + EmbeddingModelSpec( + provider_type=None, + model_name="voyageai/voyage-4-nano", + model_dim=2048, + normalize=True, + query_prefix="Represent the query for retrieving supporting documents: ", + passage_prefix="Represent the document for retrieval: ", + status=_SELECTABLE, + hf_revision="67fabc9bef010dabc5f6024aa1b1b6b93410426f", + loader=LocalModelLoader.VOYAGE_BIDIRECTIONAL_PROJECTION, + ), + EmbeddingModelSpec( + provider_type=None, + model_name="nvidia/Nemotron-3-Embed-1B-BF16", + model_dim=2048, + normalize=True, + query_prefix="query: ", + passage_prefix="passage: ", + status=_SELECTABLE, + hf_revision="c0c9fea93ea424587517f2c59e20db9f1d6bf615", + ), + # ---- SELECTABLE: cloud ---- + _cloud_spec(EmbeddingProvider.COHERE, "embed-v5.0-pro", 2048, _SELECTABLE), + _cloud_spec(EmbeddingProvider.COHERE, "embed-v5.0-fast", 2048, _SELECTABLE), + _cloud_spec( + EmbeddingProvider.GOOGLE, + "gemini-embedding-2", + 3072, + _SELECTABLE, + supports_reduced_dimension=True, + ), + _cloud_spec( + EmbeddingProvider.OPENAI, + "text-embedding-3-large", + 3072, + _SELECTABLE, + supports_reduced_dimension=True, + ), + _cloud_spec( + EmbeddingProvider.OPENAI, + "text-embedding-3-small", + 1536, + _SELECTABLE, + supports_reduced_dimension=True, + ), + _cloud_spec(EmbeddingProvider.VOYAGE, "voyage-4-large", 1024, _SELECTABLE), + _cloud_spec(EmbeddingProvider.VOYAGE, "voyage-4", 1024, _SELECTABLE), + _cloud_spec(EmbeddingProvider.VOYAGE, "voyage-4-lite", 1024, _SELECTABLE), + # ---- LEGACY: self-hosted ---- + _legacy_self_hosted_spec( + "nomic-ai/nomic-embed-text-v1", 768, True, "search_query: ", "search_document: " + ), + _legacy_self_hosted_spec( + "intfloat/e5-base-v2", 768, True, _E5_QUERY_PREFIX, _E5_PASSAGE_PREFIX + ), + _legacy_self_hosted_spec( + "intfloat/e5-small-v2", 384, True, _E5_QUERY_PREFIX, _E5_PASSAGE_PREFIX + ), + _legacy_self_hosted_spec( + "intfloat/multilingual-e5-base", 768, True, _E5_QUERY_PREFIX, _E5_PASSAGE_PREFIX + ), + _legacy_self_hosted_spec( + "intfloat/multilingual-e5-small", + 384, + True, + _E5_QUERY_PREFIX, + _E5_PASSAGE_PREFIX, + ), + _legacy_self_hosted_spec("thenlper/gte-small", 384, False, "", ""), + # ---- LEGACY: cloud ---- + _cloud_spec(EmbeddingProvider.COHERE, "embed-english-v3.0", 1024, _LEGACY), + _cloud_spec(EmbeddingProvider.COHERE, "embed-english-light-v3.0", 384, _LEGACY), + _cloud_spec(EmbeddingProvider.COHERE, "embed-v4.0", 1536, _LEGACY), + _cloud_spec(EmbeddingProvider.GOOGLE, "gemini-embedding-001", 3072, _LEGACY), + _cloud_spec(EmbeddingProvider.GOOGLE, "text-embedding-005", 768, _LEGACY), + _cloud_spec(EmbeddingProvider.GOOGLE, "gemini-embedding-2-preview", 3072, _LEGACY), + _cloud_spec(EmbeddingProvider.GOOGLE, "text-embedding-004", 768, _LEGACY), + _cloud_spec(EmbeddingProvider.GOOGLE, "textembedding-gecko@003", 768, _LEGACY), + _cloud_spec(EmbeddingProvider.VOYAGE, "voyage-large-2-instruct", 1024, _LEGACY), + _cloud_spec(EmbeddingProvider.VOYAGE, "voyage-light-2-instruct", 1024, _LEGACY), +) + + +def embedding_model_match_key( + provider_type: EmbeddingProvider | None, model_name: str +) -> tuple[EmbeddingProvider | None, str]: + """The fold that names indexes, so case, punctuation and whitespace variants + of a model name match.""" + return provider_type, clean_model_name(model_name.strip()) + + +_SPECS_BY_MATCH_KEY = { + embedding_model_match_key(spec.provider_type, spec.model_name): spec + for spec in EMBEDDING_MODEL_SPECS +} +if len(_SPECS_BY_MATCH_KEY) != len(EMBEDDING_MODEL_SPECS): + raise ValueError("Two embedding model specs share a match key") + + +def find_embedding_model_spec( + provider_type: EmbeddingProvider | None, model_name: str +) -> EmbeddingModelSpec | None: + """The SELECTABLE or LEGACY spec for ``embedding_model_match_key``, or None.""" + return _SPECS_BY_MATCH_KEY.get(embedding_model_match_key(provider_type, model_name)) + + +def get_local_model_spec(model_name: str) -> EmbeddingModelSpec | None: + """The spec the model server loads ``model_name`` with: only the exact name of + a pinned SELECTABLE self-hosted entry. Any other name (legacy, custom, a case + variant) returns None and keeps the legacy load.""" + for spec in EMBEDDING_MODEL_SPECS: + if ( + spec.provider_type is None + and spec.status == EmbeddingModelStatus.SELECTABLE + and spec.hf_revision is not None + and spec.model_name == model_name + ): + return spec + return None + + +def selectable_embedding_model_specs() -> tuple[EmbeddingModelSpec, ...]: + """Return the SELECTABLE entries, in registry order.""" + return tuple( + spec + for spec in EMBEDDING_MODEL_SPECS + if spec.status == EmbeddingModelStatus.SELECTABLE + ) diff --git a/backend/shared_configs/utils.py b/backend/shared_configs/utils.py index 4537aa94ae..9ec98c987d 100644 --- a/backend/shared_configs/utils.py +++ b/backend/shared_configs/utils.py @@ -8,3 +8,9 @@ def batch_list( batch_size: int, ) -> list[list[T]]: return [lst[i : i + batch_size] for i in range(0, len(lst), batch_size)] + + +def clean_model_name(model_str: str) -> str: + """Fold a model name into the form used in index names + (`danswer_chunk_`), so the output must never change.""" + return model_str.replace("/", "_").replace("-", "_").replace(".", "_").lower() diff --git a/backend/tests/airgap/test_default_model_server_embeddings_are_finite.py b/backend/tests/airgap/test_default_model_server_embeddings_are_finite.py index 751f552d49..fc0521fbd3 100644 --- a/backend/tests/airgap/test_default_model_server_embeddings_are_finite.py +++ b/backend/tests/airgap/test_default_model_server_embeddings_are_finite.py @@ -2,16 +2,13 @@ import json import math import urllib.request -from onyx.configs.model_configs import ( - ASYM_QUERY_PREFIX, - DEFAULT_DOCUMENT_ENCODER_MODEL, - DOC_EMBEDDING_DIM, -) from shared_configs.configs import ( + DEFAULT_DOCUMENT_ENCODER_MODEL, DOC_EMBEDDING_CONTEXT_SIZE, MODEL_SERVER_HOST, MODEL_SERVER_PORT, ) +from shared_configs.embedding_models import EmbeddingModelSpec, get_local_model_spec from shared_configs.enums import EmbedTextType from shared_configs.model_server_models import EmbedRequest, EmbedResponse @@ -22,16 +19,18 @@ _HTTP_POST_METHOD = "POST" _JSON_HEADERS = {"Content-Type": "application/json"} _SAMPLE_TEXT = "hello world" _REQUEST_TIMEOUT_SECONDS = 120 +# float32 on CPU gives norm 1.0. bf16 on a GPU normalizes in bf16: up to ~0.4% off. +_UNIT_NORM_TOLERANCE = 1e-2 -def _embed_sample_text() -> EmbedResponse: +def _embed_sample_text(spec: EmbeddingModelSpec) -> EmbedResponse: embed_request = EmbedRequest( texts=[_SAMPLE_TEXT], - model_name=DEFAULT_DOCUMENT_ENCODER_MODEL, + model_name=spec.model_name, max_context_length=DOC_EMBEDDING_CONTEXT_SIZE, - normalize_embeddings=True, + normalize_embeddings=spec.normalize, text_type=EmbedTextType.QUERY, - manual_query_prefix=ASYM_QUERY_PREFIX, + manual_query_prefix=spec.query_prefix, ) request = urllib.request.Request( _EMBEDDING_ENDPOINT, @@ -44,11 +43,19 @@ def _embed_sample_text() -> EmbedResponse: def test_default_model_server_embeddings_are_finite() -> None: - response = _embed_sample_text() + """The default model of fresh installs, which the image bundles, embeds with + no network access.""" + spec = get_local_model_spec(DEFAULT_DOCUMENT_ENCODER_MODEL) + assert spec is not None + response = _embed_sample_text(spec) assert len(response.embeddings) == 1 - assert len(response.embeddings[0]) == DOC_EMBEDDING_DIM - assert all(math.isfinite(value) for value in response.embeddings[0]) + embedding = response.embeddings[0] + assert len(embedding) == spec.model_dim + assert all(math.isfinite(value) for value in embedding) + if spec.normalize: + norm = math.sqrt(sum(value * value for value in embedding)) + assert abs(norm - 1.0) < _UNIT_NORM_TOLERANCE if __name__ == "__main__": diff --git a/backend/tests/daily/embedding/test_embeddings.py b/backend/tests/daily/embedding/test_embeddings.py index a9f13e4a9e..8c168fe74e 100644 --- a/backend/tests/daily/embedding/test_embeddings.py +++ b/backend/tests/daily/embedding/test_embeddings.py @@ -7,6 +7,8 @@ from tenacity import ( ) from onyx.natural_language_processing.search_nlp_models import EmbeddingModel +from shared_configs.configs import DEFAULT_DOCUMENT_ENCODER_MODEL +from shared_configs.embedding_models import EmbeddingModelSpec, get_local_model_spec from shared_configs.enums import EmbedTextType from shared_configs.model_server_models import EmbeddingProvider from tests.utils.secret_names import TestSecret @@ -69,8 +71,41 @@ def test_cohere_embedding(cohere_embedding_model: EmbeddingModel) -> None: _run_embeddings(TOO_LONG_SAMPLE, cohere_embedding_model, 384) +def _default_model_spec() -> EmbeddingModelSpec: + # The default self-hosted model, baked into the model-server image. + spec = get_local_model_spec(DEFAULT_DOCUMENT_ENCODER_MODEL) + assert spec is not None + return spec + + +@pytest.fixture +def local_default_embedding_model() -> EmbeddingModel: + spec = _default_model_spec() + return EmbeddingModel( + server_host="localhost", + server_port=9000, + model_name=spec.model_name, + normalize=spec.normalize, + query_prefix=spec.query_prefix, + passage_prefix=spec.passage_prefix, + api_key=None, + provider_type=None, + api_url=None, + ) + + +def test_local_default_embedding( + local_default_embedding_model: EmbeddingModel, +) -> None: + model_dim = _default_model_spec().model_dim + _run_embeddings(VALID_SAMPLE, local_default_embedding_model, model_dim) + _run_embeddings(TOO_LONG_SAMPLE, local_default_embedding_model, model_dim) + + @pytest.fixture def local_nomic_embedding_model() -> EmbeddingModel: + # Legacy model. It is no longer baked into the model-server image, so the + # first request downloads it from Hugging Face. return EmbeddingModel( server_host="localhost", server_port=9000, @@ -91,9 +126,9 @@ def test_local_nomic_embedding(local_nomic_embedding_model: EmbeddingModel) -> N @pytest.fixture def local_e5_small_embedding_model() -> EmbeddingModel: - # Unlike nomic (pre-baked into the image), e5-small is not in the image, so the - # first request forces the model server to download it from HuggingFace at - # runtime -- exercising the outbound-TLS path that the custom-CA bundle governs. + # e5-small is not in the image, so the first request forces the model server + # to download it from HuggingFace at runtime -- exercising the outbound-TLS + # path that the custom-CA bundle governs. return EmbeddingModel( server_host="localhost", server_port=9000, diff --git a/backend/tests/external_dependency_unit/db/test_index_reclaim.py b/backend/tests/external_dependency_unit/db/test_index_reclaim.py index 2e52a60f1b..7b4b6d68c5 100644 --- a/backend/tests/external_dependency_unit/db/test_index_reclaim.py +++ b/backend/tests/external_dependency_unit/db/test_index_reclaim.py @@ -58,8 +58,8 @@ from onyx.db.search_settings import ( ) from onyx.error_handling.error_codes import OnyxErrorCode from onyx.error_handling.exceptions import OnyxError -from onyx.natural_language_processing.search_nlp_models import clean_model_name from shared_configs.configs import ALT_INDEX_SUFFIX +from shared_configs.utils import clean_model_name from tests.external_dependency_unit.conftest import create_test_user, delete_test_user from tests.external_dependency_unit.indexing_helpers import ( cleanup_cc_pair, diff --git a/backend/tests/integration/common_utils/managers/document_search.py b/backend/tests/integration/common_utils/managers/document_search.py index efc9ac289d..55eca18ba1 100644 --- a/backend/tests/integration/common_utils/managers/document_search.py +++ b/backend/tests/integration/common_utils/managers/document_search.py @@ -12,6 +12,7 @@ class DocumentSearchManager: def search_documents( query: str, user_performing_action: DATestUser, + hybrid_alpha: float | None = None, ) -> list[str]: """ Search for documents using the EE search API. @@ -19,6 +20,7 @@ class DocumentSearchManager: Args: query: The search query string user_performing_action: The user performing the search (for auth) + hybrid_alpha: 1.0 for a pure vector search. None keeps the default. Returns: A list of document content strings (blurbs) from the search results @@ -27,6 +29,7 @@ class DocumentSearchManager: search_query=query, filters=None, stream=False, + hybrid_alpha=hybrid_alpha, ) result = client.post( url=f"{API_SERVER_URL}/search/send-search-message", diff --git a/backend/tests/integration/common_utils/managers/reindex_port.py b/backend/tests/integration/common_utils/managers/reindex_port.py index 70ad185eac..ffd8dca4de 100644 --- a/backend/tests/integration/common_utils/managers/reindex_port.py +++ b/backend/tests/integration/common_utils/managers/reindex_port.py @@ -74,6 +74,7 @@ class ReindexPortManager: vector_quantization: str | None = None, passage_prefix: str | None = None, index_name: str | None = None, + settings_overrides: dict[str, Any] | None = None, ) -> httpx.Response: """Reindex to the current embedding model (a new ALT index). With no vector setting changed, the port copies the stored vectors; a changed @@ -91,7 +92,10 @@ class ReindexPortManager: Leaving `enable_contextual_rag` / `contextual_rag_model_configuration_id` unset preserves whatever the tenant already has, so a reindex that isn't testing - contextual RAG can't silently disable it. Pass either explicitly to change it.""" + contextual RAG can't silently disable it. Pass either explicitly to change it. + + `settings_overrides` replaces payload fields, for example the model fields of + another target model.""" if acknowledged_wont_port_cc_pair_ids is None: acknowledged_wont_port_cc_pair_ids = ( ReindexPortManager.wont_port_cc_pair_ids( @@ -129,6 +133,7 @@ class ReindexPortManager: "contextual_rag_model_configuration_id": contextual_rag_model_configuration_id, "acknowledged_wont_port_cc_pair_ids": acknowledged_wont_port_cc_pair_ids, } + payload.update(settings_overrides or {}) return client.post( f"{SEARCH_SETTINGS_URL}/set-new-search-settings", json=payload, diff --git a/backend/tests/manual/__init__.py b/backend/tests/manual/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/backend/tests/manual/embedding_models/README.md b/backend/tests/manual/embedding_models/README.md new file mode 100644 index 0000000000..1e7f955e8f --- /dev/null +++ b/backend/tests/manual/embedding_models/README.md @@ -0,0 +1,58 @@ +# Manual embedding model tests + +Opt-in suites for the selectable embedding models. CI never collects +`backend/tests/manual`, and each suite skips unless its gate env var is `true`. +The suites read keys from plain env vars only. Do not put a gate in +`backend/.test.env`: pytest loads that file on every run. + +| Suite | Gate env var | Needs | +|---|---|---| +| `test_self_hosted_embedding_models.py` | `ONYX_RUN_SELF_HOSTED_EMBEDDING_TESTS` | Internet for the first download (about 3.2 GB). About 8 GB of free RAM for Nemotron. No keys, no services. | +| `test_cloud_embedding_models_live.py` | `ONYX_RUN_CLOUD_EMBEDDING_TESTS` | Provider keys. No services. Costs less than 0.10 USD. | +| `test_embedding_model_switch_e2e.py` | `ONYX_EMBEDDING_E2E_ALLOW_MUTATION` | A disposable Onyx stack with EE search. **Changes the stack.** | + +Key env vars. A cloud model without its key skips: `COHERE_API_KEY`, +`OPENAI_API_KEY`, `VOYAGE_API_KEY`, `VERTEX_CREDENTIALS` (the path to a Google +service-account JSON file). Keep them, and the e2e admin login +(`ONYX_E2E_ADMIN_EMAIL`, `ONYX_E2E_ADMIN_PASSWORD`), in the gitignored +`.vscode/.env.embedding-tests`, not on the command line. + +## Commands + +Run each command from the repository root. Add `-k ` to run one model. + +```bash +ONYX_RUN_SELF_HOSTED_EMBEDDING_TESTS=true \ + uv run pytest -v -rs backend/tests/manual/embedding_models/test_self_hosted_embedding_models.py + +ONYX_RUN_CLOUD_EMBEDDING_TESTS=true uv run --env-file .vscode/.env.embedding-tests \ + pytest -v -rs backend/tests/manual/embedding_models/test_cloud_embedding_models_live.py + +ONYX_EMBEDDING_E2E_ALLOW_MUTATION=true API_SERVER_PORT=18080 \ +ONYX_EMBEDDING_E2E_MODELS=ibm-granite/granite-embedding-97m-multilingual-r2,embed-v5.0-fast \ + uv run --env-file .vscode/.env.embedding-tests \ + pytest -v -s -rs backend/tests/manual/embedding_models/test_embedding_model_switch_e2e.py +``` + +## End to end + +The e2e suite re-embeds every document on the stack once for each target. It +saves the provider keys on the stack. The last target stays live, and the +upgrade-only rule then blocks a return to a legacy model. Use a disposable +stack, or take a Postgres and OpenSearch snapshot first. + +- `API_SERVER_HOST`, `API_SERVER_PORT`: the target api_server. The suite refuses + port 8080 (the normal dev stack) unless `ONYX_EMBEDDING_E2E_ALLOW_PORT_8080=true`. +- `ONYX_E2E_ADMIN_EMAIL`, `ONYX_E2E_ADMIN_PASSWORD`: an existing admin. The suite + never creates users or resets the database. +- `ONYX_EMBEDDING_E2E_MODELS`: optional. Comma-separated exact model names, run + in this order. The default is every selectable model. +- `ONYX_EMBEDDING_E2E_TIMEOUT_SECONDS`: optional wait for each swap. The default + is 3600. + +Start the stack with `ENABLE_PAID_ENTERPRISE_EDITION_FEATURES=true`: the search +check uses the EE search API. The suite never consents to connector deletion, +so the API refuses to re-index a stack with INVALID connectors. Fix or delete +them first. To check the same-model re-index of a legacy model, start with a +stack that uses a legacy model, for example nomic. Do not use `-n`: the tests +run in file order. diff --git a/backend/tests/manual/embedding_models/__init__.py b/backend/tests/manual/embedding_models/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/backend/tests/manual/embedding_models/provider_env.py b/backend/tests/manual/embedding_models/provider_env.py new file mode 100644 index 0000000000..0921fbcc12 --- /dev/null +++ b/backend/tests/manual/embedding_models/provider_env.py @@ -0,0 +1,28 @@ +"""Cloud embedding keys for the manual suites, from plain env vars only.""" + +import os +from pathlib import Path + +import pytest + +from shared_configs.enums import EmbeddingProvider + +KEY_ENV_BY_PROVIDER: dict[EmbeddingProvider, str] = { + EmbeddingProvider.COHERE: "COHERE_API_KEY", + EmbeddingProvider.OPENAI: "OPENAI_API_KEY", + EmbeddingProvider.VOYAGE: "VOYAGE_API_KEY", + # The path to a Google service-account JSON file. + EmbeddingProvider.GOOGLE: "VERTEX_CREDENTIALS", +} + + +def provider_api_key(provider: EmbeddingProvider) -> str: + """The api_key that Onyx stores for ``provider``. Skips the test if unset.""" + env_name = KEY_ENV_BY_PROVIDER[provider] + value = os.environ.get(env_name, "").strip() + if not value: + pytest.skip(f"{env_name} is not set.") + if provider == EmbeddingProvider.GOOGLE: + # Onyx stores the service-account JSON itself as the Google api_key. + return Path(value).expanduser().read_text() + return value diff --git a/backend/tests/manual/embedding_models/test_cloud_embedding_models_live.py b/backend/tests/manual/embedding_models/test_cloud_embedding_models_live.py new file mode 100644 index 0000000000..c4273dd409 --- /dev/null +++ b/backend/tests/manual/embedding_models/test_cloud_embedding_models_live.py @@ -0,0 +1,98 @@ +"""Live checks for the SELECTABLE cloud embedding models. + +Opt in with ONYX_RUN_CLOUD_EMBEDDING_TESTS=true. A model whose key env var is +unset (see provider_env.py) skips. A full run costs less than 0.10 USD. Calls +go through EmbeddingModel.encode, the path that indexing and queries use, so no +Onyx service has to run. +""" + +import os + +import numpy as np +import pytest + +from shared_configs.embedding_models import ( + EmbeddingModelSpec, + selectable_embedding_model_specs, +) +from shared_configs.enums import EmbedTextType +from tests.manual.embedding_models.provider_env import provider_api_key + +pytestmark = pytest.mark.skipif( + os.environ.get("ONYX_RUN_CLOUD_EMBEDDING_TESTS") != "true", + reason="Set ONYX_RUN_CLOUD_EMBEDDING_TESTS=true to run.", +) + +CLOUD_SPECS = [ + spec + for spec in selectable_embedding_model_specs() + if spec.provider_type is not None +] + +TOPICAL_PASSAGES: dict[str, str] = { + "pto": "Full-time employees accrue 1.5 days of paid time off per month, up to " + "30 days per year. Unused vacation days roll over until the end of March.", + "expenses": "Expense reports and receipts for reimbursement must be submitted " + "in the finance portal within 30 days of the purchase date, otherwise they " + "will be rejected.", + "vpn": "To reach internal services from home, install the corporate VPN " + "client, sign in with SSO, and approve the push notification on your phone.", + "k8s": "Our Kubernetes clusters use the cluster autoscaler; worker nodes are " + "added when pending pods cannot be scheduled because of CPU or memory " + "pressure.", + "retirement": "The company matches 401(k) retirement contributions dollar for " + "dollar up to 4% of salary, vesting immediately.", + "parking": "Visitors can park in lot B; badge holders may use the underground " + "garage after 6pm.", + "oncall": "The on-call engineer must acknowledge PagerDuty alerts within 5 " + "minutes and open an incident channel for any SEV1.", + "laptop": "Replacement laptops are issued every three years; request one " + "through the IT helpdesk with your asset tag.", +} +QUERY_ANSWERS: dict[str, str] = { + "How many vacation days do I get each year?": "pto", + "deadline to hand in receipts to get my money back": "expenses", + "how do I access internal tools while working remotely": "vpn", + "how do we add more machines when pods can't be scheduled": "k8s", + "does my employer match my retirement savings": "retirement", + # German: "How do I connect to the company VPN from home?" + "Wie verbinde ich mich von zu Hause mit dem Firmen-VPN?": "vpn", +} +# Filler first, so the topical passages come after the first provider request +# (Cohere takes 96 texts, Voyage 128, Gemini 2 one). If Onyx joins the provider +# responses in the wrong order, the ranking check fails. +PASSAGES = [ + f"Cafeteria note #{i}: the kitchen serves soup, salad and dessert number {i % 7}." + for i in range(122) +] + list(TOPICAL_PASSAGES.values()) + + +@pytest.mark.parametrize("spec", CLOUD_SPECS, ids=lambda spec: spec.model_name) +def test_embeds_in_order_with_registry_dim(spec: EmbeddingModelSpec) -> None: + from onyx.natural_language_processing.search_nlp_models import EmbeddingModel + + provider = spec.provider_type + assert provider is not None + model = EmbeddingModel( + # Not used: cloud models do not call the model server. + server_host="localhost", + server_port=9000, + model_name=spec.model_name, + normalize=spec.normalize, + query_prefix=None, + passage_prefix=None, + api_key=provider_api_key(provider), + api_url=None, + provider_type=provider, + ) + passages = np.asarray(model.encode(PASSAGES, EmbedTextType.PASSAGE)) + queries = np.asarray(model.encode(list(QUERY_ANSWERS), EmbedTextType.QUERY)) + for vectors, count in ((passages, len(PASSAGES)), (queries, len(QUERY_ANSWERS))): + assert vectors.shape == (count, spec.model_dim) + assert np.isfinite(vectors).all() + assert (np.linalg.norm(vectors, axis=1) > 0).all() + + passages /= np.linalg.norm(passages, axis=1, keepdims=True) + queries /= np.linalg.norm(queries, axis=1, keepdims=True) + best = [PASSAGES[i] for i in (queries @ passages.T).argmax(axis=1)] + assert best == [TOPICAL_PASSAGES[key] for key in QUERY_ANSWERS.values()] diff --git a/backend/tests/manual/embedding_models/test_embedding_model_switch_e2e.py b/backend/tests/manual/embedding_models/test_embedding_model_switch_e2e.py new file mode 100644 index 0000000000..e429abe449 --- /dev/null +++ b/backend/tests/manual/embedding_models/test_embedding_model_switch_e2e.py @@ -0,0 +1,301 @@ +"""End-to-end embedding model switches against a live, disposable Onyx stack. + +WARNING: this suite changes the target stack. Read README.md before you run it. +Opt in with ONYX_EMBEDDING_E2E_ALLOW_MUTATION=true. The tests run in file +order, so do not use -n: +1. The API refuses each LEGACY self-hosted model as a new target. +2. The API accepts a re-index to the live model. The suite cancels it. +3. For each target model: re-index to it, wait for the swap, and check that a + pure vector search finds each seeded fact first. +""" + +import os +import time +from collections.abc import Generator +from datetime import datetime, timezone +from typing import Any + +import httpx +import pytest + +from onyx.configs.constants import DocumentSource +from onyx.error_handling.error_codes import OnyxErrorCode +from onyx.server.documents.models import FileUploadResponse +from shared_configs.embedding_models import ( + EMBEDDING_MODEL_SPECS, + EmbeddingModelSpec, + EmbeddingModelStatus, + find_embedding_model_spec, + selectable_embedding_model_specs, +) +from shared_configs.enums import EmbeddingProvider +from tests.integration.common_utils import http_client +from tests.integration.common_utils.constants import ( + API_SERVER_PORT, + API_SERVER_URL, + GENERAL_HEADERS, +) +from tests.integration.common_utils.http_client import client +from tests.integration.common_utils.managers.cc_pair import CCPairManager +from tests.integration.common_utils.managers.document_search import ( + DocumentSearchManager, +) +from tests.integration.common_utils.managers.file import FileManager +from tests.integration.common_utils.managers.reindex_port import ReindexPortManager +from tests.integration.common_utils.managers.user import UserManager +from tests.integration.common_utils.test_models import DATestUser +from tests.manual.embedding_models.provider_env import provider_api_key + +pytestmark = pytest.mark.skipif( + os.environ.get("ONYX_EMBEDDING_E2E_ALLOW_MUTATION") != "true", + reason="Changes the target stack. Set ONYX_EMBEDDING_E2E_ALLOW_MUTATION=true.", +) + +SELECTABLE_BY_NAME = { + spec.model_name: spec for spec in selectable_embedding_model_specs() +} +# Comma-separated exact model names, run in this order. The last one stays live. +_MODEL_NAMES = os.environ.get("ONYX_EMBEDDING_E2E_MODELS", "").split(",") +TARGETS = [ + SELECTABLE_BY_NAME[name.strip()] for name in _MODEL_NAMES if name.strip() +] or list(SELECTABLE_BY_NAME.values()) +LEGACY_SELF_HOSTED = [ + spec + for spec in EMBEDDING_MODEL_SPECS + if spec.status == EmbeddingModelStatus.LEGACY and spec.provider_type is None +] +SWAP_TIMEOUT_SECONDS = float( + os.environ.get("ONYX_EMBEDDING_E2E_TIMEOUT_SECONDS", "3600") +) +# A cloud target is probed before the API accepts it, and the first query after +# a swap can load the new model on the model server. +HTTP_TIMEOUT = httpx.Timeout(900.0, connect=10.0) + +# (content, query). Each query is a paraphrase with little word overlap, so only +# the vectors can match it. The search still mixes in keyword scores, so a query +# must share no words with the other facts either. +FACTS: list[tuple[str, str]] = [ + ( + "Employees of the Halvorsen Robotics lab must wear anti-static wrist " + "straps whenever they handle the prototype circuit boards.", + "What protective gear does Halvorsen Robotics require when staff touch " + "unfinished electronics?", + ), + ( + "The Pinecrest community garden waters its tomato beds with rainwater " + "collected in three large barrels behind the tool shed.", + "How are the vegetable plots at Pinecrest irrigated?", + ), + ( + "At Ostrander Middle School, students who forget their lunch can get a " + "free sandwich from the front office before noon.", + "What happens at Ostrander when a pupil arrives without the midday meal?", + ), + ( + "The Vellmar ferry stops running across the bay whenever wind speeds " + "exceed forty knots.", + "When is the Vellmar boat service suspended because of the weather?", + ), + ( + "Kestrel Analytics reimburses staff for up to two hundred dollars of " + "home office furniture each calendar year.", + "How much can Kestrel Analytics workers claim back each year for desks " + "and chairs they buy for remote work?", + ), +] + + +@pytest.fixture(scope="module", autouse=True) +def _live_api_server() -> Generator[None, None, None]: + """Point the shared Manager client at the target stack.""" + # The Managers fall back to port 8080, the port of the normal dev stack. + if ( + API_SERVER_PORT == "8080" + and os.environ.get("ONYX_EMBEDDING_E2E_ALLOW_PORT_8080") != "true" + ): + pytest.fail( + "Set API_SERVER_PORT to the api_server of a disposable stack. If the " + "stack on port 8080 is disposable, set " + "ONYX_EMBEDDING_E2E_ALLOW_PORT_8080=true." + ) + with httpx.Client( + transport=http_client.RetryingTransport(), timeout=HTTP_TIMEOUT + ) as live_client: + http_client.set_test_client(live_client) + yield + + +@pytest.fixture(scope="module") +def admin() -> DATestUser: + """An existing admin. The suite never creates users or resets the database.""" + return UserManager.login_as_user( + DATestUser( + id="", + email=os.environ["ONYX_E2E_ADMIN_EMAIL"], + password=os.environ["ONYX_E2E_ADMIN_PASSWORD"], + headers=dict(GENERAL_HEADERS), + # login_as_user sets the real values from /me. + is_admin=False, + is_active=True, + ) + ) + + +@pytest.fixture(scope="module") +def seeded_facts(admin: DATestUser) -> Generator[None, None, None]: + """Index FACTS once through a File connector. Each switch re-embeds them. + + A File connector needs no license tier, unlike ingestion API keys. A clean-up + failure fails the teardown, so leftovers show in the report. + """ + response = FileManager.upload_connector_files( + [(f"fact_{i}.txt", content.encode()) for i, (content, _) in enumerate(FACTS)], + admin, + content_type="text/plain", + ) + response.raise_for_status() + uploaded = FileUploadResponse(**response.json()) + started = datetime.now(timezone.utc) + cc_pair = CCPairManager.create_from_scratch( + user_performing_action=admin, + source=DocumentSource.FILE, + connector_specific_config={ + "file_locations": uploaded.file_paths, + "file_names": uploaded.file_names, + "zip_metadata_file_id": None, + }, + ) + try: + CCPairManager.wait_for_indexing_completion( + cc_pair=cc_pair, after=started, user_performing_action=admin + ) + # Fails before the first switch if the stack has no EE search API. + DocumentSearchManager.search_documents(FACTS[0][1], admin, hybrid_alpha=1.0) + yield + finally: + # The deletion task removes the seeded documents. + CCPairManager.delete(cc_pair, user_performing_action=admin) + + +def _target_settings(spec: EmbeddingModelSpec) -> dict[str, Any]: + """The search settings fields that select ``spec``.""" + return { + "model_name": spec.model_name, + "model_dim": spec.model_dim, + "normalize": spec.normalize, + "query_prefix": spec.query_prefix, + "passage_prefix": spec.passage_prefix, + "provider_type": spec.provider_type.value if spec.provider_type else None, + "reduced_dimension": None, + } + + +def _connect_provider(provider: EmbeddingProvider, admin: DATestUser) -> None: + """Save the key from the provider's env var on the stack. Skips if unset.""" + response = client.put( + f"{API_SERVER_URL}/admin/embedding/embedding-provider", + json={ + "provider_type": provider.value, + "api_key": provider_api_key(provider), + "api_key_changed": True, + }, + headers=admin.headers, + ) + assert response.is_success, response.text + + +def _wait_for_swap(original_index_name: str, admin: DATestUser) -> dict[str, Any]: + """Poll until the new index is live. + + A FAILED port unit retries on its own. A PAUSED unit waits for an operator, + so it fails at once. + """ + deadline = time.monotonic() + SWAP_TIMEOUT_SECONDS + while time.monotonic() < deadline: + current = ReindexPortManager.get_current_settings(admin) + if current["index_name"] != original_index_name: + return current + progress = ReindexPortManager.get_progress(admin) + assert not progress.paused, ( + f"Port paused: {ReindexPortManager.get_errors(admin)}" + ) + print(f"Waiting for the swap: {progress}") + time.sleep(30) + pytest.fail( + f"No swap within {SWAP_TIMEOUT_SECONDS:.0f}s. Errors: " + f"{ReindexPortManager.get_errors(admin)}" + ) + + +def _missed_queries(admin: DATestUser) -> list[str]: + """The fact queries whose top pure vector search hit is not their fact.""" + return [ + query + for content, query in FACTS + if DocumentSearchManager.search_documents(query, admin, hybrid_alpha=1.0)[:1] + != [content] + ] + + +@pytest.mark.parametrize("legacy", LEGACY_SELF_HOSTED, ids=lambda spec: spec.model_name) +def test_legacy_target_is_rejected( + legacy: EmbeddingModelSpec, admin: DATestUser +) -> None: + current = ReindexPortManager.get_current_settings(admin) + if ( + current.get("provider_type") is None + and find_embedding_model_spec(None, current["model_name"]) == legacy + ): + pytest.skip("This legacy model is live. A same-model re-index is allowed.") + response = ReindexPortManager.start_reindex_response( + admin, + acknowledged_wont_port_cc_pair_ids=[], + settings_overrides=_target_settings(legacy), + ) + if response.is_success: + # Do not leave a re-index to a legacy model running. + ReindexPortManager.cancel_reindex(admin) + assert response.json().get("error_code") == OnyxErrorCode.INVALID_INPUT.code, ( + response.text + ) + + +def test_same_model_reindex_is_accepted(admin: DATestUser) -> None: + # An empty consent list: the API refuses the re-index if it would delete + # connectors (INVALID status), so the suite never deletes them. + ReindexPortManager.wait_for_reindex_accepted( + admin, acknowledged_wont_port_cc_pair_ids=[] + ) + ReindexPortManager.cancel_reindex(admin) + + +@pytest.mark.usefixtures("seeded_facts") +@pytest.mark.parametrize("spec", TARGETS, ids=lambda spec: spec.model_name) +def test_switch_to_model(spec: EmbeddingModelSpec, admin: DATestUser) -> None: + provider = spec.provider_type + if provider is not None: + _connect_provider(provider, admin) + original_index_name = ReindexPortManager.get_current_settings(admin)["index_name"] + ReindexPortManager.wait_for_reindex_accepted( + admin, + acknowledged_wont_port_cc_pair_ids=[], + settings_overrides=_target_settings(spec), + ) + try: + current = _wait_for_swap(original_index_name, admin) + except BaseException: + # Do not leave a failed re-index running on the stack. + ReindexPortManager.cancel_reindex(admin) + raise + assert (current["model_name"], current["model_dim"]) == ( + spec.model_name, + spec.model_dim, + ) + + # The new index and the model server can take a short time to settle. + deadline = time.monotonic() + 60 + missed = _missed_queries(admin) + while missed and time.monotonic() < deadline: + time.sleep(10) + missed = _missed_queries(admin) + assert not missed, f"Pure vector search with {spec.model_name} missed: {missed}" diff --git a/backend/tests/manual/embedding_models/test_self_hosted_embedding_models.py b/backend/tests/manual/embedding_models/test_self_hosted_embedding_models.py new file mode 100644 index 0000000000..9d1809b0e2 --- /dev/null +++ b/backend/tests/manual/embedding_models/test_self_hosted_embedding_models.py @@ -0,0 +1,132 @@ +"""Real-weights checks for the SELECTABLE self-hosted embedding models. + +Opt in with ONYX_RUN_SELF_HOSTED_EMBEDDING_TESTS=true. The first run downloads +about 3.2 GB of weights. Each request goes through the model server's +process_embed_request in this process. CUDA and MPS are hidden, so the models +run in fp32 on CPU, like the golden scores. Add -k voyage to run one model. +""" + +import asyncio +import os +from collections.abc import Generator + +import numpy as np +import numpy.typing as npt +import pytest + +from shared_configs.configs import DOC_EMBEDDING_CONTEXT_SIZE +from shared_configs.embedding_models import ( + EmbeddingModelSpec, + selectable_embedding_model_specs, +) +from shared_configs.enums import EmbedTextType + +pytestmark = pytest.mark.skipif( + os.environ.get("ONYX_RUN_SELF_HOSTED_EMBEDDING_TESTS") != "true", + reason="Set ONYX_RUN_SELF_HOSTED_EMBEDDING_TESTS=true to run.", +) + +SELF_HOSTED_SPECS = [ + spec for spec in selectable_embedding_model_specs() if spec.provider_type is None +] + +RED_PLANET_QUERY = "Which planet is known as the Red Planet?" +RED_PLANET_DOCS = [ + "Venus is often called Earth's twin because of its similar size and proximity.", + "Mars, known for its reddish appearance, is often referred to as the Red Planet.", + "Jupiter, the largest planet in our solar system, has a prominent red spot.", + "Saturn, famous for its rings, is sometimes mistaken for the Red Planet.", +] +# fp32 cosine scores of the query against each doc, with the registry prefixes, +# from the verified reference loader. The Onyx load matches them to 4 decimals +# on transformers 5.14.1 and sentence-transformers 5.6.0. +GOLDEN_SCORES: dict[str, list[float]] = { + "ibm-granite/granite-embedding-97m-multilingual-r2": [ + 0.8453, + 0.9388, + 0.9119, + 0.8907, + ], + "voyageai/voyage-4-nano": [0.4044, 0.6517, 0.5437, 0.5342], + "nvidia/Nemotron-3-Embed-1B-BF16": [0.2481, 0.6449, 0.4405, 0.4168], +} +# Embedded in the same batch as the docs, so each doc gets about 50 pad tokens. +# If bidirectional attention ignores the padding mask, the scores move. +PADDING_TEXT = ( + "The rover team planned a long traverse across the crater floor. Engineers " + "checked the wheels, the power budget and the communication windows before " + "they sent the drive commands. Scientists picked three rock targets for the " + "arm instruments and asked for images of the layered outcrops near the rim." +) + + +@pytest.fixture(scope="module", autouse=True) +def _cpu_only() -> Generator[None, None, None]: + """Hide CUDA and MPS, so the model server takes its CPU (fp32) path.""" + import torch + + with pytest.MonkeyPatch.context() as monkeypatch: + monkeypatch.setattr(torch.cuda, "is_available", lambda: False) + monkeypatch.setattr(torch.backends.mps, "is_available", lambda: False) + yield + + +def _embed( + spec: EmbeddingModelSpec, texts: list[str], text_type: EmbedTextType +) -> npt.NDArray[np.float64]: + from model_server.encoders import process_embed_request + from shared_configs.model_server_models import EmbedRequest + + request = EmbedRequest( + texts=texts, + model_name=spec.model_name, + max_context_length=DOC_EMBEDDING_CONTEXT_SIZE, + normalize_embeddings=True, + text_type=text_type, + manual_query_prefix=spec.query_prefix, + manual_passage_prefix=spec.passage_prefix, + ) + response = asyncio.run(process_embed_request(request)) + return np.asarray(response.embeddings, dtype=np.float64) + + +def _short_name(spec: EmbeddingModelSpec) -> str: + return spec.model_name.split("/")[-1] + + +@pytest.mark.parametrize("spec", SELF_HOSTED_SPECS, ids=_short_name) +def test_vectors_match_golden_scores(spec: EmbeddingModelSpec) -> None: + query = _embed(spec, [RED_PLANET_QUERY], EmbedTextType.QUERY) + docs = _embed(spec, [*RED_PLANET_DOCS, PADDING_TEXT], EmbedTextType.PASSAGE) + for vectors in (query, docs): + assert vectors.shape[1] == spec.model_dim + assert np.isfinite(vectors).all() + assert np.allclose(np.linalg.norm(vectors, axis=1), 1.0, atol=1e-3) + scores = docs[: len(RED_PLANET_DOCS)] @ query[0] + assert np.allclose(scores, GOLDEN_SCORES[spec.model_name], atol=0.01), ( + f"scores {np.round(scores, 4).tolist()}" + ) + + +@pytest.mark.parametrize("spec", SELF_HOSTED_SPECS, ids=_short_name) +def test_attention_is_bidirectional(spec: EmbeddingModelSpec) -> None: + """A change to the last token must change the output of the first token. + + A causal model gives exactly 0. voyage-4-nano, the weakest, gave 0.0485. + """ + from model_server.encoders import get_embedding_model + + model = get_embedding_model(spec.model_name, DOC_EMBEDDING_CONTEXT_SIZE) + first_a, first_b = ( + model.encode( + [f"{spec.passage_prefix}The quick brown fox jumps over the lazy {word}"], + output_value="token_embeddings", + convert_to_numpy=False, + )[0] + for word in ("dog", "cat") + ) + # Each is a (tokens, hidden) matrix. One input has no padding, so row 0 is + # the first real token. Only row 0 tells causal from bidirectional. + assert first_a.dim() == 2 and first_b.dim() == 2 + change = (first_a[0] - first_b[0]).abs().max().item() + assert change > 1e-3, f"Max change {change:.3g}: the attention is causal." diff --git a/backend/tests/unit/model_server/test_dockerfile_bundled_model.py b/backend/tests/unit/model_server/test_dockerfile_bundled_model.py new file mode 100644 index 0000000000..f9dd6d0935 --- /dev/null +++ b/backend/tests/unit/model_server/test_dockerfile_bundled_model.py @@ -0,0 +1,57 @@ +"""The images bake the default model at its pinned revision: the model-server image +its weights, the API image its tokenizer. The API image also bakes the nomic +tokenizer for chat token counting.""" + +import re +from pathlib import Path + +from shared_configs.configs import DEFAULT_DOCUMENT_ENCODER_MODEL +from shared_configs.embedding_models import get_local_model_spec + +_BACKEND_DIR = Path(__file__).resolve().parents[3] +_NOMIC = "nomic-ai/nomic-embed-text-v1" +_BAKED_NAME_PATTERN = re.compile(r"model_name_or_path='(?P[^']+)'") +_BAKE_PATTERN = re.compile( + r"SentenceTransformer\(model_name_or_path='(?P[^']+)',\s*\\?\s*" + r"revision='(?P[^']+)',\s*trust_remote_code=False\)" +) +_TOKENIZER_BAKE_PATTERN = re.compile( + r"Tokenizer\.from_pretrained\('(?P[^']+)'" + r"(?:,\s*revision='(?P[^']+)')?\)" +) + + +def _default_model_revision() -> str: + spec = get_local_model_spec(DEFAULT_DOCUMENT_ENCODER_MODEL) + assert spec is not None and spec.hf_revision is not None + return spec.hf_revision + + +def test_model_server_image_bakes_only_the_default_model() -> None: + text = (_BACKEND_DIR / "Dockerfile.model_server").read_text() + + baked = { + match.group("name"): match.group("revision") + for match in _BAKE_PATTERN.finditer(text) + } + assert baked == {DEFAULT_DOCUMENT_ENCODER_MODEL: _default_model_revision()} + # No other model (e.g. nomic-ai/nomic-embed-text-v1) is baked. + all_baked_names = {m.group("name") for m in _BAKED_NAME_PATTERN.finditer(text)} + assert all_baked_names == set(baked) + + +def test_api_image_bakes_the_tokenizers_it_loads() -> None: + text = (_BACKEND_DIR / "Dockerfile").read_text() + + baked = { + match.group("name"): match.group("revision") + for match in _TOKENIZER_BAKE_PATTERN.finditer(text) + } + # The default tokenizer (chat token counting, the fallback) must work offline. + # It loads without a revision, so the bake has none. + assert _NOMIC in baked and baked[_NOMIC] is None + # The API server loads registry tokenizers at their pinned revision. + assert baked.get(DEFAULT_DOCUMENT_ENCODER_MODEL) == _default_model_revision() + for model_name, revision in baked.items(): + spec = get_local_model_spec(model_name) + assert spec is None or revision == spec.hf_revision, model_name diff --git a/backend/tests/unit/model_server/test_embedding.py b/backend/tests/unit/model_server/test_embedding.py index c1ac676b63..ced391177a 100644 --- a/backend/tests/unit/model_server/test_embedding.py +++ b/backend/tests/unit/model_server/test_embedding.py @@ -1,5 +1,8 @@ import asyncio +import threading import time +from collections.abc import Callable, Iterator +from concurrent.futures import ThreadPoolExecutor from typing import Any, List from unittest.mock import MagicMock, patch @@ -12,33 +15,163 @@ from shared_configs.enums import EmbedTextType from shared_configs.model_server_models import EmbedRequest _CUSTOM_MODEL_NAME = "custom/embedding-model" +_NOMIC_MODEL_NAME = "nomic-ai/nomic-embed-text-v1" +_LOAD_EMBEDDING_MODEL = "model_server.encoders.load_embedding_model" + + +@pytest.fixture(autouse=True) +def _empty_model_cache() -> Iterator[None]: + with ( + patch.object(encoders, "_GLOBAL_MODELS_DICT", {}), + patch.object(encoders, "_ROPE_PREWARMED_TO", {}), + patch.object(encoders, "_PENDING_MODEL_JOBS", {}), + ): + yield @pytest.mark.parametrize( - ("model_name", "expected_local_files_only"), + ("model_name", "headroom"), [ - (DEFAULT_DOCUMENT_ENCODER_MODEL, True), - (_CUSTOM_MODEL_NAME, False), + ( + DEFAULT_DOCUMENT_ENCODER_MODEL, + encoders.REGISTRY_MODEL_PREFIX_HEADROOM_TOKENS, + ), + (_NOMIC_MODEL_NAME, 0), ], + ids=["registry-model", "legacy-model"], ) -def test_only_bundled_embedding_model_uses_local_files( - model_name: str, - expected_local_files_only: bool, +def test_only_registry_models_get_prefix_headroom( + model_name: str, headroom: int ) -> None: - model = MagicMock() - with ( - patch("sentence_transformers.SentenceTransformer", return_value=model) as load, - patch.object(encoders, "_GLOBAL_MODELS_DICT", {}), - ): - encoders.get_embedding_model(model_name, max_context_length=512) + """The chunker does not count a registry model's query/passage prefix, so the + prefix must not cut off the end of a full chunk.""" + with patch(_LOAD_EMBEDDING_MODEL, return_value=MagicMock()): + for max_context_length in (512, 2048): + model = encoders.get_embedding_model( + model_name, max_context_length=max_context_length + ) + assert model.max_seq_length == max_context_length + headroom - load.assert_called_once_with( - model_name_or_path=model_name, - local_files_only=expected_local_files_only, - trust_remote_code=False, + +def test_rope_prewarm_runs_again_only_for_a_longer_context() -> None: + model = MagicMock() + with patch(_LOAD_EMBEDDING_MODEL, return_value=model): + for max_context_length in (512, 2048, 512, 2048): + encoders.get_embedding_model( + _NOMIC_MODEL_NAME, max_context_length=max_context_length + ) + # The load pre-warms to 512 and the first 2048 to 2048. The rest reuse them. + assert model.encode.call_count == 2 + + +# ---- Loading off the event loop ---- + + +def _slow_loader( + delay_seconds: float, + calls: list[str], + started: threading.Event | None = None, +) -> Callable[[str], MagicMock]: + def load(model_name: str) -> MagicMock: + calls.append(model_name) + if started is not None: + started.set() + time.sleep(delay_seconds) + model = MagicMock() + model.encode.return_value = [[0.1, 0.2]] + return model + + return load + + +async def _embed(model_name: str, max_context_length: int = 512) -> list[list[float]]: + return await embed_text( + texts=["hello"], + model_name=model_name, + max_context_length=max_context_length, + normalize_embeddings=True, + prefix=None, ) +@pytest.mark.asyncio +async def test_model_load_does_not_block_event_loop() -> None: + started = threading.Event() + with patch(_LOAD_EMBEDDING_MODEL, side_effect=_slow_loader(0.3, [], started)): + embed_task = asyncio.create_task(_embed(_CUSTOM_MODEL_NAME)) + + ticks = 0 + while not embed_task.done(): + await asyncio.sleep(0.02) + ticks += 1 + + assert started.is_set() + assert await embed_task == [[0.1, 0.2]] + # The loop kept running while the load slept. + assert ticks >= 5 + + +@pytest.mark.asyncio +async def test_concurrent_requests_share_one_load() -> None: + """Requests for a model and context length share one load job, so waiting + requests hold no threads. A job for another context length waits on the + model lock and does not load the model again.""" + calls: list[str] = [] + executor = encoders._MODEL_LOAD_EXECUTOR + with ( + patch(_LOAD_EMBEDDING_MODEL, side_effect=_slow_loader(0.3, calls)), + patch.object(executor, "submit", wraps=executor.submit) as submit, + ): + results = await asyncio.gather( + *[_embed(_CUSTOM_MODEL_NAME) for _ in range(8)], + _embed(_CUSTOM_MODEL_NAME, max_context_length=1024), + ) + + assert results == [[[0.1, 0.2]]] * 9 + assert calls == [_CUSTOM_MODEL_NAME] + assert submit.call_count == 2 + + +@pytest.mark.asyncio +async def test_cancelled_request_does_not_cancel_shared_load() -> None: + calls: list[str] = [] + with ( + ThreadPoolExecutor(max_workers=1) as busy_executor, + patch.object(encoders, "_MODEL_LOAD_EXECUTOR", busy_executor), + patch(_LOAD_EMBEDDING_MODEL, side_effect=_slow_loader(0.1, calls)), + ): + # The shared job waits in the queue. A cancel there would succeed. + busy_executor.submit(time.sleep, 0.2) + cancelled = asyncio.create_task(_embed(_CUSTOM_MODEL_NAME)) + kept = asyncio.create_task(_embed(_CUSTOM_MODEL_NAME)) + await asyncio.sleep(0.05) + cancelled.cancel() + + assert await kept == [[0.1, 0.2]] + with pytest.raises(asyncio.CancelledError): + await cancelled + + assert calls == [_CUSTOM_MODEL_NAME] + + +@pytest.mark.asyncio +async def test_failed_load_is_retried_by_next_request() -> None: + model = MagicMock() + model.encode.return_value = [[0.1, 0.2]] + with patch( + _LOAD_EMBEDDING_MODEL, side_effect=[OSError("network down"), model] + ) as load: + with pytest.raises(OSError, match="network down"): + await _embed(_CUSTOM_MODEL_NAME) + assert await _embed(_CUSTOM_MODEL_NAME) == [[0.1, 0.2]] + + assert load.call_count == 2 + assert encoders._PENDING_MODEL_JOBS == {} + + +# ---- embed_text / process_embed_request ---- + + @pytest.mark.asyncio async def test_embed_text_no_model_name() -> None: # Test that the function raises an error when no model name is provided @@ -99,6 +232,10 @@ async def test_concurrent_embeddings() -> None: reduced_dimension=None, ) + # process_embed_request imports litellm on first use, which takes seconds. + # Import it before the clock starts so the test only measures the embeds. + import litellm.exceptions # noqa: F401 + with patch("model_server.encoders.get_embedding_model") as mock_get_model: mock_model = MagicMock() mock_model.encode = mock_encode diff --git a/backend/tests/unit/model_server/test_embedding_model_loader.py b/backend/tests/unit/model_server/test_embedding_model_loader.py new file mode 100644 index 0000000000..65725e0642 --- /dev/null +++ b/backend/tests/unit/model_server/test_embedding_model_loader.py @@ -0,0 +1,304 @@ +from pathlib import Path +from typing import Any +from unittest.mock import MagicMock, patch + +import numpy as np +import pytest +import torch +from safetensors.torch import save_file +from sentence_transformers.base.modules import Transformer +from sentence_transformers.sentence_transformer.modules import Dense + +from model_server import embedding_model_loader as loader +from model_server.embedding_model_loader import ( + EmbeddingModelLoadError, + load_embedding_model, +) +from model_server.voyage_nano import VOYAGE_NANO_PROJECTION_FILE +from shared_configs.configs import DEFAULT_DOCUMENT_ENCODER_MODEL +from shared_configs.embedding_models import EmbeddingModelSpec, get_local_model_spec + +_ST_CLASS = "sentence_transformers.SentenceTransformer" +_MODULES_JSON_IS_CACHED = "model_server.embedding_model_loader._modules_json_is_cached" +_HF_HUB_DOWNLOAD = "huggingface_hub.hf_hub_download" +_SHA_A = "a" * 40 +_SHA_B = "b" * 40 + + +def _registry_spec(model_name: str) -> EmbeddingModelSpec: + spec = get_local_model_spec(model_name) + assert spec is not None + return spec + + +_GRANITE = _registry_spec(DEFAULT_DOCUMENT_ENCODER_MODEL) +_VOYAGE = _registry_spec("voyageai/voyage-4-nano") + + +def _fake_model(dim: int) -> MagicMock: + model = MagicMock() + model.encode.return_value = np.zeros((1, dim), dtype=np.float32) + return model + + +# ---- Registry models ---- + + +@pytest.mark.parametrize( + ("cuda", "mps", "model_kwargs"), + [ + (False, False, {"dtype": torch.float32}), + (True, False, {}), + (False, True, {}), + ], + ids=["cpu", "cuda", "mps"], +) +def test_registry_model_load_call( + cuda: bool, + mps: bool, + model_kwargs: dict[str, Any], + monkeypatch: pytest.MonkeyPatch, +) -> None: + """The pinned revision, no remote code, the local cache first. CPU-only hosts + load the bf16 checkpoints as float32.""" + monkeypatch.setattr(torch.cuda, "is_available", lambda: cuda) + monkeypatch.setattr(torch.backends.mps, "is_available", lambda: mps) + model = _fake_model(_GRANITE.model_dim) + with ( + patch(_MODULES_JSON_IS_CACHED, return_value=True) as is_cached, + patch(_ST_CLASS, return_value=model) as load, + ): + assert load_embedding_model(_GRANITE.model_name) is model + + is_cached.assert_called_once_with(_GRANITE.model_name, _GRANITE.hf_revision) + load.assert_called_once_with( + model_name_or_path=_GRANITE.model_name, + revision=_GRANITE.hf_revision, + trust_remote_code=False, + local_files_only=True, + model_kwargs=model_kwargs, + ) + + +@pytest.mark.parametrize( + ("output", "error"), + [ + (np.zeros((1, _GRANITE.model_dim // 2)), f"expects {_GRANITE.model_dim}"), + (np.full((1, _GRANITE.model_dim), np.nan), "non-finite"), + ], + ids=["wrong-dim", "non-finite"], +) +def test_registry_model_with_unusable_output_is_rejected( + output: np.ndarray, error: str +) -> None: + model = MagicMock() + model.encode.return_value = output + with ( + patch(_MODULES_JSON_IS_CACHED, return_value=True), + patch(_ST_CLASS, return_value=model), + pytest.raises(EmbeddingModelLoadError, match=error), + ): + load_embedding_model(_GRANITE.model_name) + + +@pytest.mark.parametrize( + ("model_name", "cached"), + [ + ("nomic-ai/nomic-embed-text-v1", True), + ("my-org/custom-model", False), + # Only the exact registry name gets the registry load. + ("IBM-Granite/granite-embedding-97m-multilingual-r2", False), + ], +) +def test_other_models_keep_the_load_call_from_before_the_registry( + model_name: str, cached: bool +) -> None: + """Another call would change the vectors of existing indexes.""" + with ( + patch(_MODULES_JSON_IS_CACHED, return_value=cached), + patch(_ST_CLASS, return_value=MagicMock()) as load, + ): + load_embedding_model(model_name) + + load.assert_called_once_with( + model_name_or_path=model_name, + local_files_only=cached, + trust_remote_code=False, + ) + + +def test_voyage_loader_adds_bidirectional_projection(tmp_path: Path) -> None: + weight = torch.randn(6, 4) + weights_path = tmp_path / VOYAGE_NANO_PROJECTION_FILE + save_file({"linear.weight": weight, "other.weight": torch.zeros(1)}, weights_path) + transformer = MagicMock(spec=Transformer) + pooling = MagicMock() + normalize = MagicMock() + base = MagicMock() + base.__getitem__.side_effect = [transformer, pooling, normalize].__getitem__ + final = _fake_model(_VOYAGE.model_dim) + + with ( + patch(_MODULES_JSON_IS_CACHED, return_value=True), + patch(_ST_CLASS, side_effect=[base, final]) as st_class, + patch(_HF_HUB_DOWNLOAD, return_value=str(weights_path)) as download, + ): + assert load_embedding_model(_VOYAGE.model_name) is final + + base_load, rebuild = st_class.call_args_list + assert base_load.kwargs["revision"] == _VOYAGE.hf_revision + download.assert_called_once_with( + _VOYAGE.model_name, + VOYAGE_NANO_PROJECTION_FILE, + revision=_VOYAGE.hf_revision, + cache_dir=None, + ) + + assert transformer.auto_model.config.is_causal is False + modules = rebuild.kwargs["modules"] + assert modules[0] is transformer + assert modules[1] is pooling + assert modules[3] is normalize + projection = modules[2] + assert isinstance(projection, Dense) + assert (projection.in_features, projection.out_features) == (4, 6) + assert projection.linear.bias is None + assert isinstance(projection.activation_function, torch.nn.Identity) + assert torch.equal(projection.linear.weight.detach(), weight) + # The dimension check ran on the final model. + final.encode.assert_called_once() + + +# ---- Local-first resolution ---- + + +@pytest.mark.parametrize( + "error", + [ + OSError("couldn't find them in the cached files"), + ValueError("Unrecognized processing class"), + TypeError("missing 1 required positional argument"), + ], +) +def test_load_local_first_retries_online_on_partial_cache(error: Exception) -> None: + calls: list[bool] = [] + + def load(local_files_only: bool) -> str: + calls.append(local_files_only) + if local_files_only: + raise error + return "online" + + with patch(_MODULES_JSON_IS_CACHED, return_value=True): + assert loader._load_local_first("org/model", None, load) == "online" + assert calls == [True, False] + + +def test_load_local_first_raises_real_errors() -> None: + calls: list[bool] = [] + + def load(local_files_only: bool) -> str: + calls.append(local_files_only) + raise RuntimeError("Error(s) in loading state_dict") + + with ( + patch(_MODULES_JSON_IS_CACHED, return_value=True), + pytest.raises(RuntimeError, match="state_dict"), + ): + loader._load_local_first("org/model", None, load) + assert calls == [True] + + +@pytest.mark.parametrize( + ("cached", "expected_error"), + [ + # The online retry reads the same cached files, so its error is the real one. + (True, RuntimeError), + # Air-gapped hosts get a clear error, not huggingface_hub's retry noise. + (False, EmbeddingModelLoadError), + ], +) +def test_load_local_first_online_failure( + cached: bool, expected_error: type[Exception] +) -> None: + def load(local_files_only: bool) -> str: + if local_files_only: + raise OSError("partial cache") + raise RuntimeError("Cannot send a request, as the client has been closed.") + + with ( + patch(_MODULES_JSON_IS_CACHED, return_value=cached), + pytest.raises(RuntimeError) as exc_info, + ): + loader._load_local_first("org/model", None, load) + + # EmbeddingModelLoadError is a RuntimeError, so compare the exact type. + assert type(exc_info.value) is expected_error + if not cached: + assert "org/model is not in the model server's Hugging Face cache" in str( + exc_info.value + ) + assert isinstance(exc_info.value.__cause__, RuntimeError) + + +def _make_hub_cache( + root: Path, + repo_id: str, + commit: str, + files: list[str], + ref_main: bool, + no_exist: list[str] | None = None, +) -> None: + repo_dir = root / f"models--{repo_id.replace('/', '--')}" + snapshot = repo_dir / "snapshots" / commit + snapshot.mkdir(parents=True) + for name in files: + (snapshot / name).write_text("{}") + if ref_main: + (repo_dir / "refs").mkdir() + (repo_dir / "refs" / "main").write_text(commit) + for name in no_exist or []: + marker = repo_dir / ".no_exist" / commit / name + marker.parent.mkdir(parents=True, exist_ok=True) + marker.touch() + + +@pytest.fixture +def hub_cache(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path: + import huggingface_hub.constants + + cache = tmp_path / "hub" + cache.mkdir() + monkeypatch.delenv("SENTENCE_TRANSFORMERS_HOME", raising=False) + monkeypatch.setattr(huggingface_hub.constants, "HF_HUB_CACHE", str(cache)) + return cache + + +def test_modules_json_is_cached(hub_cache: Path) -> None: + _make_hub_cache(hub_cache, "org/st-model", _SHA_A, ["modules.json"], ref_main=True) + # A snapshot at a pinned revision with no refs/main, like the image bake. + _make_hub_cache(hub_cache, "org/pinned", _SHA_B, ["modules.json"], ref_main=False) + # config.json and weights cached (e.g. by transformers) but no modules.json. + _make_hub_cache( + hub_cache, + "org/partial", + _SHA_A, + ["config.json", "model.safetensors"], + ref_main=True, + ) + # A plain transformers model: the cache records that modules.json is absent. + _make_hub_cache( + hub_cache, + "org/plain-hf", + _SHA_A, + ["config.json"], + ref_main=True, + no_exist=["modules.json"], + ) + + assert loader._modules_json_is_cached("org/st-model", None) is True + assert loader._modules_json_is_cached("org/pinned", _SHA_B) is True + assert loader._modules_json_is_cached("org/partial", None) is False + assert loader._modules_json_is_cached("org/plain-hf", None) is True + # A local directory is not a repo id. sentence-transformers reads it directly. + assert loader._modules_json_is_cached("/models/my-model", None) is False diff --git a/backend/tests/unit/onyx/configs/test_embedding_model_registry_configs.py b/backend/tests/unit/onyx/configs/test_embedding_model_registry_configs.py new file mode 100644 index 0000000000..a1cfdd234c --- /dev/null +++ b/backend/tests/unit/onyx/configs/test_embedding_model_registry_configs.py @@ -0,0 +1,151 @@ +"""The first search settings rows of a fresh DB (alembic dbaa756c2ccf) and the +default tokenizer, which onyx.configs.model_configs derives from the env and the +embedding registry.""" + +import json +import os +import subprocess +import sys +from pathlib import Path +from typing import Any + +import pytest + +_MODEL_ENV_VARS = ( + "DOCUMENT_ENCODER_MODEL", + "DOC_EMBEDDING_DIM", + "NORMALIZE_EMBEDDINGS", + "ASYM_QUERY_PREFIX", + "ASYM_PASSAGE_PREFIX", +) + +_BACKEND_DIR = Path(__file__).resolve().parents[4] + + +def _run_python(code: str, env: dict[str, str]) -> subprocess.CompletedProcess[str]: + """Run code in a new interpreter with exactly the given model env vars. + + model_configs reads the env at import. A reload in this process would rebind + its globals for every later test, so each case gets its own process.""" + child_env = {k: v for k, v in os.environ.items() if k not in _MODEL_ENV_VARS} + child_env.update(env) + child_env["PYTHONPATH"] = str(_BACKEND_DIR) + result = subprocess.run( + [sys.executable, "-c", code], + cwd=_BACKEND_DIR, + env=child_env, + capture_output=True, + text=True, + timeout=120, + ) + assert result.returncode == 0, result.stderr + return result + + +def _last_json_line(result: subprocess.CompletedProcess[str]) -> Any: + return json.loads(result.stdout.strip().splitlines()[-1]) + + +# Loads alembic dbaa756c2ccf from its file and prints the default tokenizer and +# the rows that the migration seeds. +_SEED_PROBE = """ +import importlib.util +import json + +spec = importlib.util.spec_from_file_location( + "seed", "alembic/versions/dbaa756c2ccf_embedding_models.py" +) +migration = importlib.util.module_from_spec(spec) +spec.loader.exec_module(migration) +from onyx.configs.model_configs import DEFAULT_TOKENIZER_MODEL + +rows = [["PRESENT", migration._get_old_default_embedding_model()]] +if not migration.user_has_overridden_embedding_model(): + rows.append(["FUTURE", migration._get_new_default_embedding_model()]) +print(json.dumps([DEFAULT_TOKENIZER_MODEL, [ + [status, s.model_name, s.model_dim, s.normalize, s.query_prefix, + s.passage_prefix, s.index_name] + for status, s in rows +]])) +""" + +NOMIC = "nomic-ai/nomic-embed-text-v1" +CUSTOM = "my-org/custom-model" +_GTE_PLACEHOLDER = [ + "PRESENT", + "thenlper/gte-small", + 384, + False, + "", + "", + "danswer_chunk", +] +_GRANITE_FUTURE = [ + "FUTURE", + "ibm-granite/granite-embedding-97m-multilingual-r2", + 384, + True, + "", + "", + "danswer_chunk_ibm_granite_granite_embedding_97m_multilingual_r2", +] + + +@pytest.mark.parametrize( + "env,tokenizer,rows", + [ + # Fresh default: the gte-small placeholder plus a granite FUTURE row, from + # granite's registry spec. The tokenizer stays nomic. + ({}, NOMIC, [_GTE_PLACEHOLDER, _GRANITE_FUTURE]), + # nomic via env (the previous default) seeds exactly what main seeded. + ( + {"DOCUMENT_ENCODER_MODEL": NOMIC}, + NOMIC, + [ + _GTE_PLACEHOLDER, + [ + "FUTURE", + NOMIC, + 768, + True, + "search_query: ", + "search_document: ", + "danswer_chunk_nomic_ai_nomic_embed_text_v1", + ], + ], + ), + # Any other model via env is an override: PRESENT on danswer_chunk, with + # the model env vars over the old literal defaults. + ( + {"DOCUMENT_ENCODER_MODEL": CUSTOM, "DOC_EMBEDDING_DIM": "1024"}, + CUSTOM, + [ + [ + "PRESENT", + CUSTOM, + 1024, + True, + "search_query: ", + "search_document: ", + "danswer_chunk", + ] + ], + ), + # An env file written for nomic, without DOCUMENT_ENCODER_MODEL: a working + # granite row, not granite with nomic's 768 dims and prefixes. + ( + { + "DOC_EMBEDDING_DIM": "768", + "ASYM_QUERY_PREFIX": "search_query: ", + "ASYM_PASSAGE_PREFIX": "search_document: ", + }, + NOMIC, + [_GTE_PLACEHOLDER, _GRANITE_FUTURE], + ), + ], + ids=["no-env", "nomic-env", "custom-env", "nomic-era-env-without-model"], +) +def test_alembic_dbaa756c2ccf_seed( + env: dict[str, str], tokenizer: str, rows: list[list[object]] +) -> None: + assert _last_json_line(_run_python(_SEED_PROBE, env)) == [tokenizer, rows] diff --git a/backend/tests/unit/onyx/natural_language_processing/test_search_nlp_models.py b/backend/tests/unit/onyx/natural_language_processing/test_search_nlp_models.py index a8a5052515..db390e71c4 100644 --- a/backend/tests/unit/onyx/natural_language_processing/test_search_nlp_models.py +++ b/backend/tests/unit/onyx/natural_language_processing/test_search_nlp_models.py @@ -3,7 +3,9 @@ from threading import Lock from typing import Any from unittest.mock import AsyncMock, MagicMock, patch +import httpx import pytest +import requests from google.auth.credentials import Credentials from httpx import AsyncClient from litellm.exceptions import RateLimitError @@ -17,11 +19,11 @@ from onyx.natural_language_processing.search_nlp_models import ( EmbeddingModel, EmbeddingRequestPolicy, EmbeddingRetryBackoff, - clean_model_name, ) from onyx.natural_language_processing.vertex_auth import VertexEmbeddingConfig from shared_configs.enums import EmbeddingProvider, EmbedTextType from shared_configs.model_server_models import EmbedRequest, EmbedResponse +from shared_configs.utils import clean_model_name @pytest.fixture @@ -824,6 +826,19 @@ def test_query_embeddings_never_wait_for_the_rate_limit() -> None: load_limiter.assert_not_called() +def test_model_server_error_detail_reaches_the_caller() -> None: + model = _make_local_embedding_model() + failed = MagicMock(status_code=500) + failed.raise_for_status.side_effect = requests.HTTPError("500", response=failed) + failed.json.return_value = {"detail": "model is not in the cache"} + + with ( + patch(f"{_SEARCH_NLP_MODULE}.requests.post", return_value=failed), + pytest.raises(httpx.HTTPError, match="model is not in the cache"), + ): + model.encode(texts=["text_0"], text_type=EmbedTextType.QUERY) + + @pytest.mark.asyncio @pytest.mark.parametrize( ("block_on_provider_rate_limit", "expected_retries"), diff --git a/backend/tests/unit/onyx/natural_language_processing/test_utils_hf_revision.py b/backend/tests/unit/onyx/natural_language_processing/test_utils_hf_revision.py new file mode 100644 index 0000000000..8fd24497fb --- /dev/null +++ b/backend/tests/unit/onyx/natural_language_processing/test_utils_hf_revision.py @@ -0,0 +1,54 @@ +"""API-side HF tokenizers: registry models load at their pinned revision, other +names keep the plain call, and the default tokenizer stays the pre-granite model.""" + +from unittest.mock import patch + +import pytest + +from onyx.natural_language_processing import utils +from shared_configs.configs import DEFAULT_DOCUMENT_ENCODER_MODEL +from shared_configs.embedding_models import get_local_model_spec + +_HF_TOKENIZER_CLASS = "onyx.natural_language_processing.utils.Tokenizer" +_NOMIC = "nomic-ai/nomic-embed-text-v1" + + +def _pinned_revision(model_name: str) -> str | None: + spec = get_local_model_spec(model_name) + assert spec is not None + return spec.hf_revision + + +@pytest.mark.parametrize( + ("model_name", "kwargs"), + [ + ( + DEFAULT_DOCUMENT_ENCODER_MODEL, + {"revision": _pinned_revision(DEFAULT_DOCUMENT_ENCODER_MODEL)}, + ), + (_NOMIC, {}), + # Only the exact registry name is pinned; a variant keeps the plain call. + (DEFAULT_DOCUMENT_ENCODER_MODEL.upper(), {}), + ], + ids=["registry-model", "legacy-model", "registry-name-variant"], +) +def test_tokenizer_load_call(model_name: str, kwargs: dict[str, str | None]) -> None: + with patch(_HF_TOKENIZER_CLASS) as mock_class: + utils.HuggingFaceTokenizer(model_name) + + mock_class.from_pretrained.assert_called_once_with(model_name, **kwargs) + + +def test_no_model_name_uses_the_default_tokenizer_model( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """Not DOCUMENT_ENCODER_MODEL, which defaults to granite. The env semantics of + DEFAULT_TOKENIZER_MODEL are pinned in test_embedding_model_registry_configs.py.""" + monkeypatch.setattr(utils, "_DEFAULT_TOKENIZER", None) + monkeypatch.setattr(utils, "_TOKENIZER_CACHE", {}) + monkeypatch.setattr(utils, "DEFAULT_TOKENIZER_MODEL", _NOMIC) + with patch(_HF_TOKENIZER_CLASS) as mock_class: + result = utils.get_tokenizer(model_name=None, provider_type=None) + + mock_class.from_pretrained.assert_called_once_with(_NOMIC) + assert isinstance(result, utils.HuggingFaceTokenizer) diff --git a/backend/tests/unit/onyx/server/manage/test_embedding_model_upgrade_guard.py b/backend/tests/unit/onyx/server/manage/test_embedding_model_upgrade_guard.py new file mode 100644 index 0000000000..d5fbd1d5f3 --- /dev/null +++ b/backend/tests/unit/onyx/server/manage/test_embedding_model_upgrade_guard.py @@ -0,0 +1,516 @@ +"""Upgrade-only guard in set_new_search_settings: a new embedding target must be the +PRESENT model (same-model re-index) or a selectable registry model. Legacy registry +models are refused; custom self-hosted models, LiteLLM and Azure stay open. Bifrost +and selectable cloud targets are probed before the PRESENT row lock.""" + +from collections.abc import Iterator +from dataclasses import dataclass +from typing import Any +from unittest.mock import MagicMock, patch + +import pytest + +from onyx.context.search.models import ( + SavedSearchSettings, + SearchSettingsCreationRequest, +) +from onyx.error_handling.error_codes import OnyxErrorCode +from onyx.error_handling.exceptions import OnyxError +from onyx.natural_language_processing.embedding_auth import VertexEmbeddingAuth +from onyx.server.manage.search_settings import ( + _check_embedding_model_target, + _validate_cloud_embedding_model, + set_new_search_settings, +) +from shared_configs.configs import DEFAULT_DOCUMENT_ENCODER_MODEL +from shared_configs.embedding_models import ( + EmbeddingModelSpec, + find_embedding_model_spec, + selectable_embedding_model_specs, +) +from shared_configs.enums import EmbeddingProvider +from shared_configs.utils import clean_model_name + +_MODULE = "onyx.server.manage.search_settings" + +NOMIC = "nomic-ai/nomic-embed-text-v1" +E5 = "intfloat/e5-base-v2" +GRANITE = DEFAULT_DOCUMENT_ENCODER_MODEL +VOYAGE_NANO = "voyageai/voyage-4-nano" +NEMOTRON = "nvidia/Nemotron-3-Embed-1B-BF16" +COHERE = EmbeddingProvider.COHERE +GOOGLE = EmbeddingProvider.GOOGLE +OPENAI = EmbeddingProvider.OPENAI + + +def _request( + model_name: str, + *, + provider_type: EmbeddingProvider | None = None, + model_dim: int = 768, + normalize: bool = True, + query_prefix: str | None = "", + passage_prefix: str | None = "", + reduced_dimension: int | None = None, +) -> SearchSettingsCreationRequest: + return SearchSettingsCreationRequest( + model_name=model_name, + model_dim=model_dim, + normalize=normalize, + query_prefix=query_prefix, + passage_prefix=passage_prefix, + provider_type=provider_type, + index_name=None, + multipass_indexing=False, + reduced_dimension=reduced_dimension, + enable_contextual_rag=False, + contextual_rag_model_configuration_id=None, + ) + + +def _spec_request( + provider_type: EmbeddingProvider | None, + model_name: str, + *, + sent_name: str | None = None, + **overrides: Any, +) -> SearchSettingsCreationRequest: + """A request with the registry's settings for the model, as the UI sends it. + sent_name replaces the model name in the request (e.g. a case variant).""" + spec = find_embedding_model_spec(provider_type, model_name) + assert spec is not None + values: dict[str, Any] = { + "provider_type": spec.provider_type, + "model_dim": spec.model_dim, + "normalize": spec.normalize, + "query_prefix": spec.query_prefix, + "passage_prefix": spec.passage_prefix, + } + values.update(overrides) + return _request(sent_name or model_name, **values) + + +def _present( + model_name: str, + provider_type: EmbeddingProvider | None = None, + *, + model_dim: int | None = None, + reduced_dimension: int | None = None, +) -> MagicMock: + """PRESENT row. model_dim defaults to the registry dim, else 768.""" + if model_dim is None: + spec = find_embedding_model_spec(provider_type, model_name) + model_dim = spec.model_dim if spec is not None else 768 + present = MagicMock() + present.id = 2 + present.use_port_flow = False + present.port_backfill_source_id = None + present.model_name = model_name + present.provider_type = provider_type + present.model_dim = model_dim + present.reduced_dimension = reduced_dimension + present.index_name = f"danswer_chunk_{clean_model_name(model_name)}" + return present + + +# ---- The rules ---- + +_ALLOWED = [ + pytest.param( + _present(NOMIC), _request(NOMIC), False, id="same-legacy-model-empty-prefixes" + ), + pytest.param( + _present(E5), _request("IntFloat/E5_Base.V2"), False, id="same-model-variant" + ), + pytest.param( + _present(VOYAGE_NANO, model_dim=1024), + _request(VOYAGE_NANO, model_dim=1024), + False, + id="same-model-keeps-a-custom-dim", + ), + pytest.param( + _present(VOYAGE_NANO, model_dim=1024), + _spec_request(None, VOYAGE_NANO), + False, + id="same-model-moves-to-the-registry-dim", + ), + pytest.param( + _present("embed-english-v3.0", COHERE), + _request("embed-english-v3.0", provider_type=COHERE, model_dim=1024), + False, + id="same-legacy-cloud-model", + ), + pytest.param( + _present(NOMIC, reduced_dimension=256), + _request(NOMIC, reduced_dimension=256), + False, + id="same-self-hosted-model-keeps-a-stored-reduced-dimension", + ), + pytest.param( + _present("text-embedding-3-large", OPENAI, reduced_dimension=1024), + _spec_request(OPENAI, "text-embedding-3-large", reduced_dimension=1024), + True, + id="same-model-keeps-reduced-dimension-in-multi-tenant", + ), + pytest.param( + _present(NOMIC), + _spec_request(None, GRANITE, query_prefix=None, passage_prefix=None), + False, + id="selectable-with-null-prefixes", + ), + pytest.param( + _present(NOMIC), + _spec_request(OPENAI, "text-embedding-3-large", reduced_dimension=1024), + False, + id="selectable-with-reduced-dimension", + ), + pytest.param( + _present(GRANITE), + _request("my-org/custom-embedder", model_dim=1024), + False, + id="custom-self-hosted-model", + ), + pytest.param( + _present(GRANITE), + _request( + "embed-english-v3.0", provider_type=EmbeddingProvider.LITELLM, model_dim=123 + ), + False, + id="litellm-with-a-legacy-looking-name", + ), + pytest.param( + _present(GRANITE), + _request( + "text-embedding-3-large", + provider_type=EmbeddingProvider.AZURE, + model_dim=123, + ), + False, + id="azure", + ), +] + +_REJECTED = [ + pytest.param( + _present(GRANITE), + _request(GRANITE, model_dim=768), + False, + "must keep its dimension", + id="same-model-with-another-dim", + ), + pytest.param( + _present("text-embedding-3-large", OPENAI), + _spec_request(OPENAI, "text-embedding-3-large", reduced_dimension=1024), + True, + "share its index", + id="same-model-changes-reduced-dimension-in-multi-tenant", + ), + pytest.param( + _present(GRANITE), + _spec_request(None, GRANITE, reduced_dimension=256), + False, + "do not support reduced_dimension", + id="same-self-hosted-model-adds-reduced-dimension", + ), + pytest.param( + _present(GRANITE), + _request("my-org/custom-embedder", model_dim=1024, reduced_dimension=256), + False, + "do not support reduced_dimension", + id="custom-self-hosted-model-with-reduced-dimension", + ), + pytest.param( + _present("embed-english-v3.0"), + _request("embed-english-v3.0", provider_type=COHERE, model_dim=1024), + False, + "legacy embedding model", + id="same-name-but-another-provider", + ), + pytest.param( + _present(NOMIC), + _spec_request(None, GRANITE, model_dim=768), + False, + "model_dim must be 384", + id="selectable-with-another-dim", + ), + pytest.param( + _present(NOMIC), + _spec_request(None, GRANITE, normalize=False), + False, + "normalize must be True", + id="selectable-with-another-normalize", + ), + pytest.param( + _present(NOMIC), + _spec_request(None, NEMOTRON, query_prefix="query:"), + False, + "query_prefix must be 'query: '", + id="selectable-prefix-without-its-trailing-space", + ), + pytest.param( + _present(NOMIC), + _spec_request(OPENAI, "text-embedding-3-large", reduced_dimension=0), + False, + "between 1 and 3071", + id="reduced-dimension-zero", + ), + pytest.param( + _present(NOMIC), + _spec_request(OPENAI, "text-embedding-3-large", reduced_dimension=3072), + False, + "between 1 and 3071", + id="reduced-dimension-full-size", + ), + pytest.param( + _present(NOMIC), + _spec_request(COHERE, "embed-v5.0-pro", reduced_dimension=256), + False, + "does not support reduced_dimension", + id="reduced-dimension-without-support", + ), + pytest.param( + _present("embed-english-v3.0", COHERE), + _spec_request(OPENAI, "text-embedding-3-large", reduced_dimension=1024), + True, + "not supported in Onyx Cloud", + id="reduced-dimension-in-multi-tenant", + ), + pytest.param( + _present(GRANITE), + _spec_request(None, NOMIC), + False, + "legacy embedding model", + id="legacy-self-hosted-model", + ), + pytest.param( + _present(GRANITE), + _request(" nomic_ai/Nomic.Embed.Text.V1 "), + False, + "legacy embedding model", + id="legacy-self-hosted-name-variant", + ), + pytest.param( + _present(GRANITE), + _spec_request(GOOGLE, "text-embedding-005"), + False, + "legacy embedding model", + id="legacy-cloud-model", + ), + pytest.param( + _present(GRANITE), + _request("text-embedding-ada-002", provider_type=OPENAI), + False, + "not an embedding model that Onyx supports", + id="unknown-model-of-a-registry-provider", + ), +] + + +@pytest.mark.parametrize(("present", "target", "multi_tenant"), _ALLOWED) +def test_allowed_target( + present: MagicMock, target: SearchSettingsCreationRequest, multi_tenant: bool +) -> None: + with patch(f"{_MODULE}.MULTI_TENANT", multi_tenant): + _check_embedding_model_target(target, present) + + +@pytest.mark.parametrize(("present", "target", "multi_tenant", "problem"), _REJECTED) +def test_rejected_target( + present: MagicMock, + target: SearchSettingsCreationRequest, + multi_tenant: bool, + problem: str, +) -> None: + with ( + patch(f"{_MODULE}.MULTI_TENANT", multi_tenant), + pytest.raises(OnyxError) as exc, + ): + _check_embedding_model_target(target, present) + + assert exc.value.error_code == OnyxErrorCode.INVALID_INPUT + assert problem in exc.value.detail + + +@pytest.mark.parametrize( + "spec", selectable_embedding_model_specs(), ids=lambda spec: spec.model_name +) +def test_every_selectable_model_is_allowed_with_registry_settings( + spec: EmbeddingModelSpec, +) -> None: + _check_embedding_model_target( + _spec_request(spec.provider_type, spec.model_name), _present(NOMIC) + ) + + +def test_rejection_names_the_selectable_models() -> None: + with pytest.raises(OnyxError) as exc: + _check_embedding_model_target(_spec_request(None, NOMIC), _present(GRANITE)) + + for spec in selectable_embedding_model_specs(): + assert spec.model_name in exc.value.detail + + +# ---- Wiring in set_new_search_settings ---- + + +class _GuardPassed(Exception): + """Patched into create_search_settings to prove the guards let the request through.""" + + +@dataclass +class _EndpointMocks: + provider: MagicMock + current: MagicMock + create: MagicMock + probe: MagicMock + + def created_settings(self) -> SavedSearchSettings: + self.create.assert_called_once() + return self.create.call_args.kwargs["search_settings"] + + +@pytest.fixture +def mocks() -> Iterator[_EndpointMocks]: + with ( + patch(f"{_MODULE}.validate_contextual_rag_model"), + patch(f"{_MODULE}._validate_vector_quantization_supported"), + patch(f"{_MODULE}.get_embedding_provider_from_provider_type") as provider, + patch(f"{_MODULE}.get_current_search_settings") as current, + patch(f"{_MODULE}.get_secondary_search_settings", return_value=None), + patch(f"{_MODULE}.port_backfill_has_pending_work", return_value=False), + patch(f"{_MODULE}._guard_index_name_reuse"), + patch(f"{_MODULE}.compute_wont_port_cc_pair_ids", return_value=[]), + patch(f"{_MODULE}.set_reclaim_intent_on_current__no_commit"), + patch(f"{_MODULE}.create_search_settings", side_effect=_GuardPassed) as create, + patch(f"{_MODULE}._validate_cloud_embedding_model") as probe, + ): + yield _EndpointMocks( + provider=provider, current=current, create=create, probe=probe + ) + + +def _submit( + target: SearchSettingsCreationRequest, db_session: MagicMock | None = None +) -> None: + set_new_search_settings(target, _=MagicMock(), db_session=db_session or MagicMock()) + + +def test_selectable_cloud_target_is_probed_before_the_present_lock( + mocks: _EndpointMocks, +) -> None: + events: list[str] = [] + + def _locked_read(*_args: object, **_kwargs: object) -> MagicMock: + events.append("locked_read") + return _present(GRANITE) + + mocks.current.side_effect = _locked_read + mocks.probe.side_effect = lambda *_args: events.append("probe") + db_session = MagicMock() + db_session.commit.side_effect = lambda: events.append("commit") + + with pytest.raises(_GuardPassed): + _submit( + _spec_request(COHERE, "embed-v5.0-pro", sent_name="Embed-V5.0-Pro"), + db_session, + ) + + # The read transaction ends before the network call; nothing is locked yet. + assert events == ["commit", "probe", "locked_read"] + probed_request, probed_provider = mocks.probe.call_args.args + assert probed_request.model_name == "embed-v5.0-pro" + assert probed_provider is mocks.provider.return_value + assert mocks.created_settings().model_name == "embed-v5.0-pro" + + +@pytest.mark.parametrize( + ("present", "target", "stored_name"), + [ + pytest.param( + _present(NOMIC), + _spec_request( + None, + GRANITE, + sent_name=" IBM-Granite/Granite-Embedding-97M-Multilingual-R2 ", + ), + GRANITE, + id="new-target", + ), + pytest.param( + _present(VOYAGE_NANO), + _spec_request(None, VOYAGE_NANO, sent_name="VoyageAI/Voyage-4-Nano"), + VOYAGE_NANO, + id="same-model-reindex", + ), + ], +) +def test_selectable_target_is_stored_with_its_canonical_name( + mocks: _EndpointMocks, + present: MagicMock, + target: SearchSettingsCreationRequest, + stored_name: str, +) -> None: + """The model server and the tokenizer match registry names exactly. A variant + would load without the pinned revision and the model's fix-ups.""" + mocks.current.return_value = present + + with pytest.raises(_GuardPassed): + _submit(target) + + created = mocks.created_settings() + assert created.model_name == stored_name + assert created.index_name is not None + assert created.index_name.startswith( + f"danswer_chunk_{clean_model_name(stored_name)}" + ) + mocks.probe.assert_not_called() + + +def test_rejected_target_creates_nothing(mocks: _EndpointMocks) -> None: + mocks.current.return_value = _present(GRANITE) + + with pytest.raises(OnyxError) as exc: + _submit(_spec_request(COHERE, "embed-english-v3.0")) + + assert exc.value.error_code == OnyxErrorCode.INVALID_INPUT + # Only Bifrost and selectable cloud targets are probed. + mocks.probe.assert_not_called() + mocks.create.assert_not_called() + + +def test_cloud_probe_failure_blocks_the_reindex(mocks: _EndpointMocks) -> None: + mocks.probe.side_effect = OnyxError(OnyxErrorCode.VALIDATION_ERROR, "no access") + + with pytest.raises(OnyxError) as exc: + _submit(_spec_request(GOOGLE, "gemini-embedding-2")) + + assert exc.value.detail == "no access" + mocks.create.assert_not_called() + mocks.current.assert_not_called() + + +def test_cloud_probe_uses_the_stored_vertex_config() -> None: + """A Workload Identity provider has no service account JSON, so the probe + needs the stored vertex_config to authenticate.""" + provider = MagicMock() + provider.provider_type = EmbeddingProvider.GOOGLE + provider.api_key = None + provider.api_url = None + provider.api_version = None + provider.deployment_name = None + provider.vertex_config = { + "auth_method": "workload_identity", + "project_id": "target-project", + "location": "us", + } + + with patch(f"{_MODULE}.probe_embedding_dimension", return_value=3072) as probe: + _validate_cloud_embedding_model( + _spec_request(GOOGLE, "gemini-embedding-2"), provider + ) + + auth = probe.call_args.kwargs["auth"] + assert isinstance(auth, VertexEmbeddingAuth) + assert auth.config.auth_method == "workload_identity" + assert auth.config.project_id == "target-project" + assert probe.call_args.kwargs["model_name"] == "gemini-embedding-2" diff --git a/backend/tests/unit/onyx/server/manage/test_reindex_gateway_embedding_guard.py b/backend/tests/unit/onyx/server/manage/test_reindex_gateway_embedding_guard.py index 8853b29b2e..c00695c0b0 100644 --- a/backend/tests/unit/onyx/server/manage/test_reindex_gateway_embedding_guard.py +++ b/backend/tests/unit/onyx/server/manage/test_reindex_gateway_embedding_guard.py @@ -45,6 +45,7 @@ def _provider( provider.provider_type = provider_type provider.api_url = "https://bifrost.example" provider.api_key.get_value.return_value = "sk-bf-stored" + provider.vertex_config = None return provider diff --git a/backend/tests/unit/onyx/test_setup_local_embedding_model_wait.py b/backend/tests/unit/onyx/test_setup_local_embedding_model_wait.py new file mode 100644 index 0000000000..ce272f6022 --- /dev/null +++ b/backend/tests/unit/onyx/test_setup_local_embedding_model_wait.py @@ -0,0 +1,112 @@ +"""API server startup waits for the model server to load the self-hosted +embedding model, and fails with a clear error when the wait runs out.""" + +import threading +from collections.abc import Iterator +from unittest.mock import MagicMock, patch + +import pytest + +from onyx.setup import wait_for_local_embedding_model + +_NOMIC = "nomic-ai/nomic-embed-text-v1" +_NOT_CACHED_ERROR = ( + "HTTP error occurred: Error during embedding process: nomic-ai/nomic-embed-text-v1 " + "is not in the model server's Hugging Face cache and could not be downloaded" +) + + +class _FakeClock: + """Stands in for the `time` module in onyx.setup: sleep advances monotonic.""" + + def __init__(self) -> None: + self.now = 0.0 + self.sleeps: list[float] = [] + + def monotonic(self) -> float: + return self.now + + def sleep(self, seconds: float) -> None: + self.sleeps.append(seconds) + self.now += seconds + + +@pytest.fixture +def clock() -> Iterator[_FakeClock]: + fake = _FakeClock() + with patch("onyx.setup.time", fake): + yield fake + + +@pytest.fixture +def encode() -> Iterator[MagicMock]: + with patch("onyx.setup.EmbeddingModel") as embedding_model_cls: + yield embedding_model_cls.from_db_model.return_value.encode + + +def _search_settings() -> MagicMock: + settings = MagicMock() + settings.model_name = _NOMIC + return settings + + +def test_retries_until_the_model_server_is_ready( + clock: _FakeClock, encode: MagicMock +) -> None: + encode.side_effect = [ + Exception("Request failed: Connection refused"), + Exception(_NOT_CACHED_ERROR), + [[0.1, 0.2]], + ] + + wait_for_local_embedding_model(_search_settings()) + + assert encode.call_count == 3 + assert clock.sleeps == [10, 10] + + +def test_fails_with_the_last_error_when_the_wait_runs_out( + clock: _FakeClock, encode: MagicMock +) -> None: + encode.side_effect = Exception(_NOT_CACHED_ERROR) + + with ( + patch("onyx.setup.LOCAL_EMBEDDING_MODEL_STARTUP_TIMEOUT", 30), + pytest.raises(RuntimeError) as exc_info, + ): + wait_for_local_embedding_model(_search_settings()) + + # The operator sees the last error and the setting that controls the wait. + assert _NOT_CACHED_ERROR in str(exc_info.value) + assert "LOCAL_EMBEDDING_MODEL_STARTUP_TIMEOUT" in str(exc_info.value) + assert str(exc_info.value.__cause__) == _NOT_CACHED_ERROR + assert encode.call_count == 3 + assert clock.now == 30 + + +@pytest.mark.usefixtures("clock") +def test_fails_when_a_request_is_still_waiting_at_the_deadline( + encode: MagicMock, +) -> None: + release = threading.Event() + started = threading.Event() + request_threads: list[threading.Thread] = [] + + def wait_for_release(**_: object) -> None: + request_threads.append(threading.current_thread()) + started.set() + release.wait() + + encode.side_effect = wait_for_release + try: + with ( + patch("onyx.setup.LOCAL_EMBEDDING_MODEL_STARTUP_TIMEOUT", 0.1), + pytest.raises(RuntimeError, match="still waiting for the model server"), + ): + wait_for_local_embedding_model(_search_settings()) + # The waiting request must not keep the failed process from exiting. + # On a slow runner, the request thread can start after the deadline. + assert started.wait(timeout=5) + assert [thread.daemon for thread in request_threads] == [True] + finally: + release.set() diff --git a/backend/tests/unit/shared_configs/test_embedding_models.py b/backend/tests/unit/shared_configs/test_embedding_models.py new file mode 100644 index 0000000000..db1c151bc0 --- /dev/null +++ b/backend/tests/unit/shared_configs/test_embedding_models.py @@ -0,0 +1,50 @@ +"""Integrity of the shared embedding-model registry.""" + +import re +import subprocess +import sys +from pathlib import Path + +from shared_configs.embedding_models import ( + EMBEDDING_MODEL_SPECS, + EmbeddingModelStatus, + LocalModelLoader, +) + +_BACKEND_DIR = Path(__file__).resolve().parents[3] + + +def test_registry_invariants() -> None: + for spec in EMBEDDING_MODEL_SPECS: + # get_local_model_spec only matches pinned entries, so a selectable + # self-hosted model without a revision would get the legacy load. + needs_revision = spec.provider_type is None and ( + spec.status == EmbeddingModelStatus.SELECTABLE + ) + assert (spec.hf_revision is not None) == needs_revision, spec.model_name + if spec.hf_revision is not None: + assert re.fullmatch(r"[0-9a-f]{40}", spec.hf_revision), spec.model_name + if spec.provider_type is not None: + # The cloud path refuses prefixes, and loader settings are local-only. + assert spec.query_prefix == spec.passage_prefix == "", spec.model_name + assert spec.normalize is False, spec.model_name + assert spec.loader == LocalModelLoader.STANDARD, spec.model_name + + +def test_registry_imports_no_onyx_or_ml_packages() -> None: + """The model-server image has no onyx package and the API image has no torch.""" + code = ( + "import sys\n" + "import shared_configs.embedding_models\n" + "banned = {'onyx', 'ee', 'model_server', 'torch', 'transformers'," + " 'sentence_transformers'}\n" + "print(sorted(m for m in sys.modules if m.split('.')[0] in banned))\n" + ) + result = subprocess.run( + [sys.executable, "-c", code], + cwd=_BACKEND_DIR, + capture_output=True, + text=True, + check=True, + ) + assert result.stdout.strip() == "[]" diff --git a/backend/tests/unit/shared_configs/test_web_embedding_model_registry_parity.py b/backend/tests/unit/shared_configs/test_web_embedding_model_registry_parity.py new file mode 100644 index 0000000000..0e54ee5af6 --- /dev/null +++ b/backend/tests/unit/shared_configs/test_web_embedding_model_registry_parity.py @@ -0,0 +1,89 @@ +"""The web model picker mirrors the backend embedding model registry. + +The admin page sends a model's name, dimension, normalize flag and prefixes +as-is to set-new-search-settings, and the upgrade-only guard rejects a +selectable model whose values differ from the backend registry. So the two +copies must agree, and a model the backend treats as legacy must be marked +legacy in the web registry (the picker hides it).""" + +import re +from pathlib import Path + +from shared_configs.embedding_models import ( + EMBEDDING_MODEL_SPECS, + EmbeddingModelStatus, +) +from shared_configs.enums import EmbeddingProvider + +_WEB_REGISTRY = ( + Path(__file__).resolve().parents[4] + / "web" + / "src" + / "lib" + / "searchSettings" + / "constants.ts" +) +_GROUP = re.compile(r"providerName:\s*EmbeddingProviderName\.(\w+)") +# Model objects hold no nested braces. +_MODEL_BLOCK = re.compile(r"\{([^{}]*\bmodelName:[^{}]*)\}") + +# (provider_type, model_name) -> (dim, normalize, query_prefix, passage_prefix, +# legacy). Legacy values are informational only, so a legacy entry keeps only +# its legacy flag. +_ModelKey = tuple[EmbeddingProvider | None, str] +_ModelValues = tuple[int | None, bool | None, str | None, str | None, bool] + + +def _field(block: str, name: str, value_pattern: str) -> str: + match = re.search(rf"\b{name}:\s*{value_pattern}", block) + assert match, f"No literal `{name}` in a model of {_WEB_REGISTRY.name}:\n{block}" + return match.group(1) + + +def _web_models() -> dict[_ModelKey, _ModelValues]: + source = _WEB_REGISTRY.read_text() + source = source[: source.index("export const CUSTOM_PROVIDER")] + self_hosted_start = source.index("export const SELF_HOSTED_PROVIDERS") + groups = list(_GROUP.finditer(source)) + group_ends = [group.start() for group in groups[1:]] + [len(source)] + models: dict[_ModelKey, _ModelValues] = {} + for group, end in zip(groups, group_ends, strict=True): + # Self-hosted groups have provider_type None on the backend. + provider_type = ( + None + if group.start() > self_hosted_start + else EmbeddingProvider(group.group(1).lower()) + ) + for block in _MODEL_BLOCK.findall(source[group.end() : end]): + key = (provider_type, _field(block, "modelName", r'"([^"]*)"')) + assert key not in models, f"Duplicate model {key} in the web registry" + if re.search(r"\blegacy:\s*true\b", block): + models[key] = (None, None, None, None, True) + continue + models[key] = ( + int(_field(block, "modelDim", r"(\d+)")), + _field(block, "normalize", r"(true|false)") == "true", + _field(block, "queryPrefix", r'"([^"]*)"'), + _field(block, "passagePrefix", r'"([^"]*)"'), + False, + ) + return models + + +def test_web_registry_matches_the_backend_registry() -> None: + expected: dict[_ModelKey, _ModelValues] = { + (spec.provider_type, spec.model_name): ( + (None, None, None, None, True) + if spec.status == EmbeddingModelStatus.LEGACY + else ( + spec.model_dim, + spec.normalize, + spec.query_prefix, + spec.passage_prefix, + False, + ) + ) + for spec in EMBEDDING_MODEL_SPECS + } + + assert _web_models() == expected diff --git a/cli/internal/deploy/deployfiles/embedded/docker_compose/README.md b/cli/internal/deploy/deployfiles/embedded/docker_compose/README.md index e498349464..c62fc4cb5f 100644 --- a/cli/internal/deploy/deployfiles/embedded/docker_compose/README.md +++ b/cli/internal/deploy/deployfiles/embedded/docker_compose/README.md @@ -135,3 +135,20 @@ Every image publishes a `-dev` twin for each of its tags (e.g. `latest-dev`, `v1 its `-dev` twin adds interactive debugging tools (vim, nano, curl, ps, psql) that the default image leaves out to stay minimal. The web-server, model-server, and sandbox `-dev` tags are identical to their plain counterparts and exist so that one version string covers every image. + +### Self-hosted embedding models and air-gapped deployments +The model-server image contains one embedding model: `ibm-granite/granite-embedding-97m-multilingual-r2`, +the default for new installs. It works with no network access. + +The model servers download all other self-hosted embedding models from Hugging Face the first time +they use them. This includes `nomic-ai/nomic-embed-text-v1`, which older images contained. The +downloads go to the `model_cache_huggingface` and `indexing_huggingface_model_cache` volumes. +When a model is in these volumes, the model servers load it from there with no network access. An +upgrade keeps the models that are already in the volumes. + +For an air-gapped deployment: +- Keep these volumes when you upgrade. Do not delete them. +- To use a self-hosted model that is not in the volumes, give the model servers network access to + Hugging Face once, or copy the model into the Hugging Face cache in both volumes before you select it. + Onyx loads its recommended self-hosted models at the revision that + `backend/shared_configs/embedding_models.py` pins, so copy that revision. diff --git a/cli/internal/deploy/deployfiles/embedded/docker_compose/env.template b/cli/internal/deploy/deployfiles/embedded/docker_compose/env.template index 4c0cdf763c..58512c2fc8 100644 --- a/cli/internal/deploy/deployfiles/embedded/docker_compose/env.template +++ b/cli/internal/deploy/deployfiles/embedded/docker_compose/env.template @@ -398,6 +398,10 @@ LOG_ONYX_MODEL_INTERACTIONS=False # ASYM_PASSAGE_PREFIX= # DISABLE_RERANK_FOR_STREAMING= # MODEL_SERVER_PORT= +# Seconds the API server waits on startup for the model server to load a self-hosted +# embedding model, including a download if it is not cached (default 1800). Startup +# fails when the wait runs out. 0 disables the wait. +# LOCAL_EMBEDDING_MODEL_STARTUP_TIMEOUT= # INDEX_BATCH_SIZE= # MIN_THREADS_ML_MODELS= diff --git a/deployment/docker_compose/README.md b/deployment/docker_compose/README.md index e498349464..c62fc4cb5f 100644 --- a/deployment/docker_compose/README.md +++ b/deployment/docker_compose/README.md @@ -135,3 +135,20 @@ Every image publishes a `-dev` twin for each of its tags (e.g. `latest-dev`, `v1 its `-dev` twin adds interactive debugging tools (vim, nano, curl, ps, psql) that the default image leaves out to stay minimal. The web-server, model-server, and sandbox `-dev` tags are identical to their plain counterparts and exist so that one version string covers every image. + +### Self-hosted embedding models and air-gapped deployments +The model-server image contains one embedding model: `ibm-granite/granite-embedding-97m-multilingual-r2`, +the default for new installs. It works with no network access. + +The model servers download all other self-hosted embedding models from Hugging Face the first time +they use them. This includes `nomic-ai/nomic-embed-text-v1`, which older images contained. The +downloads go to the `model_cache_huggingface` and `indexing_huggingface_model_cache` volumes. +When a model is in these volumes, the model servers load it from there with no network access. An +upgrade keeps the models that are already in the volumes. + +For an air-gapped deployment: +- Keep these volumes when you upgrade. Do not delete them. +- To use a self-hosted model that is not in the volumes, give the model servers network access to + Hugging Face once, or copy the model into the Hugging Face cache in both volumes before you select it. + Onyx loads its recommended self-hosted models at the revision that + `backend/shared_configs/embedding_models.py` pins, so copy that revision. diff --git a/deployment/docker_compose/env.template b/deployment/docker_compose/env.template index 4c0cdf763c..58512c2fc8 100644 --- a/deployment/docker_compose/env.template +++ b/deployment/docker_compose/env.template @@ -398,6 +398,10 @@ LOG_ONYX_MODEL_INTERACTIONS=False # ASYM_PASSAGE_PREFIX= # DISABLE_RERANK_FOR_STREAMING= # MODEL_SERVER_PORT= +# Seconds the API server waits on startup for the model server to load a self-hosted +# embedding model, including a download if it is not cached (default 1800). Startup +# fails when the wait runs out. 0 disables the wait. +# LOCAL_EMBEDDING_MODEL_STARTUP_TIMEOUT= # INDEX_BATCH_SIZE= # MIN_THREADS_ML_MODELS= diff --git a/deployment/helm/charts/onyx/Chart.yaml b/deployment/helm/charts/onyx/Chart.yaml index c6649b06bf..4976b1f8bf 100644 --- a/deployment/helm/charts/onyx/Chart.yaml +++ b/deployment/helm/charts/onyx/Chart.yaml @@ -5,7 +5,7 @@ home: https://www.onyx.app/ sources: - "https://github.com/onyx-dot-app/onyx" type: application -version: 0.9.18 +version: 0.9.19 appVersion: latest annotations: category: Productivity @@ -21,9 +21,9 @@ annotations: image: docker.io/onyxdotapp/onyx-backend:latest artifacthub.io/changes: | - kind: added - description: Add the TABULAR_RECHUNK_ENABLED setting. It runs a one-time - migration that re-chunks already indexed tabular documents with the current - chunker, without re-crawling the source. + description: Add the LOCAL_EMBEDDING_MODEL_STARTUP_TIMEOUT setting. The API + server waits on startup until the model server loads the self-hosted + embedding model, which includes a download if it is not cached. dependencies: # Helm uses the first condition path that exists: `postgresqlOperator.enabled` # is unset by default, so this falls through to `postgresql.enabled`. Set it diff --git a/deployment/helm/charts/onyx/values.yaml b/deployment/helm/charts/onyx/values.yaml index deeb07bfbf..f12d4ca92b 100644 --- a/deployment/helm/charts/onyx/values.yaml +++ b/deployment/helm/charts/onyx/values.yaml @@ -201,6 +201,19 @@ autoscaling: inferenceCapability: extraEnv: [] + # Optional Hugging Face cache volume. Without one, a model that the image + # does not bundle (only the default model is bundled; nomic is not) downloads + # again on every pod start. At startup, the model server copies the bundled + # model into the volume. The chart does not create the claim. With a + # ReadWriteOnce claim, keep replicaCount at 1 and set strategy.type: Recreate. + # Give indexCapability its own claim. + # volumes: + # - name: hf-cache + # persistentVolumeClaim: + # claimName: onyx-inference-model-hf-cache + # volumeMounts: + # - name: hf-cache + # mountPath: /app/.cache/huggingface volumes: [] volumeMounts: [] service: @@ -278,6 +291,7 @@ inferenceCapability: indexCapability: extraEnv: [] + # Optional Hugging Face cache volume: see inferenceCapability.volumes. volumes: [] volumeMounts: [] service: @@ -624,10 +638,21 @@ api: # every replica at once, and a lone not-ready pod leaves the Service empty. # Do NOT put /health/ready on the liveness probe. # - # Liveness and startup probes stay opt-in. Size a startupProbe for the longest - # migration you expect, or it kills the pod mid-migration. + # Liveness and startup probes stay opt-in. /health answers only after startup: + # the migrations, then the wait for the model server to load a self-hosted + # embedding model, with a download if it is not cached (up to + # LOCAL_EMBEDDING_MODEL_STARTUP_TIMEOUT, 1800 s by default). A livenessProbe + # with no startupProbe therefore restarts the pod during a download. Size the + # startupProbe (periodSeconds x failureThreshold) for the longest migration + # you expect plus that timeout. # # Example: + # startupProbe: + # httpGet: + # path: /health + # port: api-server-port + # periodSeconds: 10 + # failureThreshold: 200 # 2000 s # readinessProbe: # httpGet: # path: /health/ready @@ -1755,6 +1780,10 @@ configMap: ASYM_PASSAGE_PREFIX: "" DISABLE_RERANK_FOR_STREAMING: "" MODEL_SERVER_PORT: "" + # Seconds the API server waits on startup for the model server to load a + # self-hosted embedding model, including a download if it is not cached; + # "" = 1800. Startup fails when the wait runs out. "0" disables the wait. + LOCAL_EMBEDDING_MODEL_STARTUP_TIMEOUT: "" MIN_THREADS_ML_MODELS: "" # Indexing Configs NUM_INDEXING_WORKERS: "" diff --git a/web/src/i18n/messages/ar.json b/web/src/i18n/messages/ar.json index 59912298e2..143f3b1a65 100644 --- a/web/src/i18n/messages/ar.json +++ b/web/src/i18n/messages/ar.json @@ -6863,6 +6863,9 @@ "noModelsTooltip": "ุงู„ุงุณุชุฑุฌุงุน ุงู„ุณูŠุงู‚ูŠ ู…ุนุทู‘ู„ ู„ุนุฏู… ูˆุฌูˆุฏ ู†ู…ุงุฐุฌ ู…ู‡ูŠุฃุฉ. ู‚ู… ุจุฅุนุฏุงุฏ [ู†ู…ูˆุฐุฌ ู„ุบุฉ]({link}) ุฃูˆู„ู‹ุง.", "title": "ุงู„ุงุณุชุฑุฌุงุน ุงู„ุณูŠุงู‚ูŠ" }, + "currentModel": { + "legacyHint": "ู‡ุฐุง ู†ู…ูˆุฐุฌ ู‚ุฏูŠู…. ูŠุณุชู…ุฑ ููŠ ุงู„ุนู…ู„ุŒ ู„ูƒู† ู„ุง ูŠู…ูƒู†ูƒ ุงุฎุชูŠุงุฑู‡ ู…ุฑุฉ ุฃุฎุฑู‰ ู„ูู‡ุฑุณ ุฌุฏูŠุฏ. ู„ู„ุชุฑู‚ูŠุฉุŒ ุงู†ู‚ุฑ ุนู„ู‰ ยซุนุฑุถ ุฌู…ูŠุน ุงู„ู†ู…ุงุฐุฌยป ูˆุงุฎุชุฑ ู†ู…ูˆุฐุฌู‹ุง ุฃุญุฏุซ." + }, "embeddingModel": { "cloudManaged": { "title": "ู†ู…ูˆุฐุฌ ุงู„ุชุถู…ูŠู† ูˆุฅุนุฏุงุฏุงุชู‡ ูŠุฏูŠุฑู‡ู…ุง Onyx Cloud." @@ -7001,9 +7004,12 @@ "currentButton": { "label": "ุงู„ู†ู…ูˆุฐุฌ ุงู„ุญุงู„ูŠ" }, - "deprecated": { - "connectTooltip": "ู†ู…ูˆุฐุฌ ุงู„ุชุถู…ูŠู† ู‡ุฐุง ู…ู‡ู…ู„ ูˆู„ุง ูŠู…ูƒู† ุงู„ุงุชุตุงู„ ุจู‡.", - "selectTooltip": "ู†ู…ูˆุฐุฌ ุงู„ุชุถู…ูŠู† ู‡ุฐุง ู…ู‡ู…ู„ ูˆู„ุง ูŠู…ูƒู† ุงุฎุชูŠุงุฑู‡." + "gpuRecommended": { + "tag": "ูŠูู†ุตุญ ุจุงุณุชุฎุฏุงู… GPU" + }, + "legacy": { + "tag": "ู‚ุฏูŠู…", + "tooltip": "ู„ุง ูŠู…ูƒู†ูƒ ุงุฎุชูŠุงุฑ ู‡ุฐุง ุงู„ู†ู…ูˆุฐุฌ ู„ูู‡ุฑุณ ุฌุฏูŠุฏ. ูŠุณุชู…ุฑ ููŠ ุงู„ุนู…ู„ ุญุชู‰ ุชุบูŠู‘ุฑ ุงู„ู†ู…ูˆุฐุฌ." }, "selectButton": { "label": "ุงุฎุชูŠุงุฑ ุงู„ู†ู…ูˆุฐุฌ" @@ -7013,9 +7019,15 @@ } }, "modelPicker": { + "cpuEstimateHint": "ุณุฑุนุงุช CPU ุชู‚ุฏูŠุฑูŠุฉ ูˆุชุฎุชู„ู ุญุณุจ ุงู„ุนุชุงุฏ ูˆุญุฌู… ุงู„ู…ุณุชู†ุฏ. ุชูˆุตูŠุงุช RAM ุชุฎุต ู†ู…ูˆุฐุฌู‹ุง ูˆุงุญุฏู‹ุง ู…ุญู…ู‘ู„ู‹ุงุŒ ูˆู„ูŠุณุช ู…ุชุทู„ุจุงุช ู†ุธุงู… Onyx ุจุงู„ูƒุงู…ู„. ูˆูู‘ุฑ ุฐุงูƒุฑุฉ ุฅุถุงููŠุฉ ุฃุซู†ุงุก ุฅุนุงุฏุฉ ุงู„ูู‡ุฑุณุฉ.", "addCustomModel": { "label": "ุฅุถุงูุฉ ู†ู…ูˆุฐุฌ ู…ุฎุตุต" }, + "bringYourOwn": { + "cloudDescription": "ุงุฑุจุท ูˆูƒูŠู„ LiteLLM ุฃูˆ ู†ุดุฑ Azure OpenAI ุฃูˆ ุจูˆุงุจุฉ Bifrost. ุชุญุฏุฏ ุฃู†ุช ุงุณู… ุงู„ู†ู…ูˆุฐุฌ. ูˆููŠ LiteLLM ูˆAzure ุชุญุฏุฏ ุฃูŠุถู‹ุง ุงู„ุฃุจุนุงุฏ.", + "selfHostedDescription": "ุดุบู‘ู„ ู†ู…ูˆุฐุฌ ุชุถู…ูŠู† ู…ู† Hugging Face ุบูŠุฑ ู…ูˆุฌูˆุฏ ููŠ ุงู„ู‚ุงุฆู…ุฉ ุฃุนู„ุงู‡. ุชุญุฏุฏ ุฃู†ุช ุงุณู… ุงู„ู†ู…ูˆุฐุฌ ูˆุฃุจุนุงุฏู‡ ูˆุจุงุฏุฆุงุชู‡.", + "title": "ู…ุชู‚ุฏู…: ุงุณุชุฎุฏู… ู†ู…ูˆุฐุฌูƒ ุฃูˆ ุงู„ูˆูƒูŠู„ ุงู„ุฎุงุต ุจูƒ" + }, "cloudTab": { "label": "ุณุญุงุจูŠ" }, @@ -7214,22 +7226,27 @@ "title": "{count, plural, one {ุณูŠูุญุฐู ู‡ุฐุง ุงู„ู…ูˆุตู„} other {ุณุชูุญุฐู ู‡ุฐู‡ ุงู„ู…ูˆุตู„ุงุช (#)}}" }, "modelDescriptions": { + "cohereEmbedV5Pro": "ุฃูุถู„ ู†ู…ูˆุฐุฌ ุชุถู…ูŠู† ู…ู† Cohere. ุฃูุถู„ ุชูˆุงุฒู† ุจูŠู† ุงู„ุฌูˆุฏุฉ ูˆุงู„ุชูƒู„ูุฉ. ู…ุฎุฑุฌุงุช ุจุฃุจุนุงุฏ 2048.", + "cohereEmbedV5Fast": "ุฅุตุฏุงุฑ ุฃู‚ู„ ุชูƒู„ูุฉ ู…ู† Embed 5 ู…ู† Cohere. ูˆู„ุง ูŠุฒุงู„ ุฎูŠุงุฑู‹ุง ู…ู…ุชุงุฒู‹ุง. ู…ุฎุฑุฌุงุช ุจุฃุจุนุงุฏ 2048.", "cohereEmbedEnglishV3": "ู†ู…ูˆุฐุฌ ุงู„ุชุถู…ูŠู† ุงู„ุฅู†ุฌู„ูŠุฒูŠ ู…ู† Cohere. ุฃุฏุงุก ุฌูŠุฏ ู„ู„ู…ู‡ุงู… ุจุงู„ู„ุบุฉ ุงู„ุฅู†ุฌู„ูŠุฒูŠุฉ.", "cohereEmbedEnglishLightV3": "ู†ู…ูˆุฐุฌ ุงู„ุชุถู…ูŠู† ุงู„ุฅู†ุฌู„ูŠุฒูŠ ุงู„ุฎููŠู ู…ู† Cohere. ุฃุณุฑุน ูˆุฃูƒุซุฑ ูƒูุงุกุฉ ู„ู„ู…ู‡ุงู… ุงู„ุฃุจุณุท.", - "cohereEmbedV4": "ุฃุญุฏุซ ู†ู…ูˆุฐุฌ ุชุถู…ูŠู† ู…ุชุนุฏุฏ ุงู„ู„ุบุงุช ู…ู† Cohere ุจู…ุฎุฑุฌุงุช ุงูุชุฑุงุถูŠุฉ ุจุฃุจุนุงุฏ 1536 ู„ุฌูˆุฏุฉ ุงุณุชุฑุฌุงุน ุฃุนู„ู‰.", - "openaiTextEmbedding3Large": "ู†ู…ูˆุฐุฌ ุงู„ุชุถู…ูŠู† ุงู„ูƒุจูŠุฑ ู…ู† OpenAI. ุฃูุถู„ ุฃุฏุงุก ู„ูƒู†ู‡ ุฃุนู„ู‰ ุชูƒู„ูุฉ.", - "openaiTextEmbedding3Small": "ู†ู…ูˆุฐุฌ ุงู„ุชุถู…ูŠู† ุงู„ุฃุญุฏุซ ูˆุงู„ุฃูƒุซุฑ ูƒูุงุกุฉ ู…ู† OpenAI. ุชูˆุงุฒู† ุฌูŠุฏ ุจูŠู† ุงู„ุฃุฏุงุก ูˆุงู„ุชูƒู„ูุฉ.", + "cohereEmbedV4": "ู†ู…ูˆุฐุฌ ุงู„ุชุถู…ูŠู† ู…ุชุนุฏุฏ ุงู„ู„ุบุงุช ู…ู† CohereุŒ ุจู…ุฎุฑุฌุงุช ุจุฃุจุนุงุฏ 1536.", + "openaiTextEmbedding3Large": "ู†ู…ูˆุฐุฌ ุงู„ุชุถู…ูŠู† ุงู„ูƒุจูŠุฑ ู…ู† OpenAI. ุฎูŠุงุฑ ุฌูŠุฏ ุฅุฐุง ูƒู†ุช ุชุณุชุฎุฏู… OpenAI ุจุงู„ูุนู„. ู…ุฎุฑุฌุงุช ุจุฃุจุนุงุฏ 3072.", + "openaiTextEmbedding3Small": "ู†ู…ูˆุฐุฌ ุงู„ุชุถู…ูŠู† ุงู„ุตุบูŠุฑ ู…ู† OpenAI. ุงู„ุฎูŠุงุฑ ุงู„ุณุญุงุจูŠ ุงู„ุฃูƒุซุฑ ุงู‚ุชุตุงุฏู‹ุง ุจูุงุฑู‚ ูƒุจูŠุฑ. ู…ุฎุฑุฌุงุช ุจุฃุจุนุงุฏ 1536.", "googleGeminiEmbedding001": "ู†ู…ูˆุฐุฌ ุชุถู…ูŠู† Gemini ู…ู† Google. ู‚ูˆูŠ ูˆูุนู‘ุงู„.", "googleTextEmbedding005": "ู†ู…ูˆุฐุฌ ุชุถู…ูŠู† ุฃุตุบุฑ ูˆุฃุฎู ู…ู† Google.", - "googleGeminiEmbedding2": "ุฃุญุฏุซ ู†ู…ูˆุฐุฌ ุชุถู…ูŠู† ู…ุชุนุฏุฏ ุงู„ูˆุณุงุฆุท ู…ู† Google (ู…ุชุงุญ ู„ู„ุนู…ูˆู…). ุฃุนู„ู‰ ุฌูˆุฏุฉ ุงุณุชุฑุฌุงุน ุจู…ุฎุฑุฌุงุช ุจุฃุจุนุงุฏ 3072.", + "googleGeminiEmbedding2": "ุฃูุถู„ ู†ู…ูˆุฐุฌ ุชุถู…ูŠู† ู…ู† Google. ู‚ูˆูŠ ุฌุฏู‹ุงุŒ ู„ูƒู†ู‡ ู…ูƒู„ู ูˆุฃุจุทุฃ. ู…ุฎุฑุฌุงุชู‡ ุจุฃุจุนุงุฏ 3072 ุชุจู†ูŠ ูู‡ุฑุณู‹ุง ูƒุจูŠุฑู‹ุง. ุฎูŠุงุฑ ู…ุชูŠู† ู„ู„ูุฑู‚ ุงู„ุชูŠ ุชุณุชุฎุฏู… Google ุจุงู„ูุนู„.", "googleGeminiEmbedding2Preview": "ุงู„ุฅุตุฏุงุฑ ุงู„ุชุฌุฑูŠุจูŠ ู…ู† Gemini Embedding 2. ุญู„ู‘ ู…ุญู„ู‡ gemini-embedding-2 ููŠ ุนู…ู„ูŠุงุช ุงู„ุฅุนุฏุงุฏ ุงู„ุฌุฏูŠุฏุฉ.", "voyageLarge2Instruct": "ู†ู…ูˆุฐุฌ ุงู„ุชุถู…ูŠู† ุงู„ูƒุจูŠุฑ ู…ู† Voyage. ุฃุฏุงุก ุนุงู„ู ู…ุน ุถุจุท ุฏู‚ูŠู‚ ุจุงู„ุชุนู„ูŠู…ุงุช.", "voyage4Large": "ุฃูƒุจุฑ ู†ู…ูˆุฐุฌ ุชุถู…ูŠู† ููŠ ุณู„ุณู„ุฉ Voyage 4. ุฃูุถู„ ุฌูˆุฏุฉ ุงุณุชุฑุฌุงุน ู„ุฏู‰ VoyageุŒ ูˆูŠุฏุนู… ู„ุบุงุช ู…ุชุนุฏุฏุฉ.", "voyage4": "ู†ู…ูˆุฐุฌ ุงู„ุชุถู…ูŠู† ู…ุชุนุฏุฏ ุงู„ุฃุบุฑุงุถ ู…ู† ุณู„ุณู„ุฉ Voyage 4. ุฌูˆุฏุฉ ุงุณุชุฑุฌุงุน ุนุงู„ูŠุฉ ุจุชูƒู„ูุฉ ู…ุนุชุฏู„ุฉ.", "voyage4Lite": "ู†ู…ูˆุฐุฌ ุงู„ุชุถู…ูŠู† ุงู„ุฎููŠู ู…ู† ุณู„ุณู„ุฉ Voyage 4. ุฃู‚ู„ ุชูƒู„ูุฉ ูˆุฃุนู„ู‰ ุญุฏูˆุฏ ู„ู„ู…ุนุฏู„.", "voyageLight2Instruct": "ู†ู…ูˆุฐุฌ ุงู„ุชุถู…ูŠู† ุงู„ุฎููŠู ู…ู† Voyage. ุชูˆุงุฒู† ุฌูŠุฏ ุจูŠู† ุงู„ุฃุฏุงุก ูˆุงู„ูƒูุงุกุฉ.", + "graniteEmbedding97mMultilingualR2": "ู†ู…ูˆุฐุฌ IBM ุงู„ุตุบูŠุฑ ู…ุชุนุฏุฏ ุงู„ู„ุบุงุช. ู…ุถู…ู‘ู† ู…ุน {appName}. โ€97 ู…ู„ูŠูˆู† ู…ุนู„ู…ุฉ ูˆู…ุฎุฑุฌุงุช ุฐุงุช 384 ุจูุนุฏู‹ุง. ุญุชู‰ 200 ู…ุณุชู†ุฏ / ุฏู‚ูŠู‚ุฉ / ู†ูˆุงุฉ CPU. ุฐุงูƒุฑุฉ RAM ุงู„ู…ูˆุตู‰ ุจู‡ุง ู„ู„ุชุดุบูŠู„ ุนู„ู‰ CPU: โ€4 ุฌูŠุฌุงุจุงูŠุช ู„ูƒู„ ุฎุงุฏู… ู†ู…ุงุฐุฌ.", + "voyage4Nano": "ู†ู…ูˆุฐุฌ Voyage AI ุฐูˆ ุงู„ุฃูˆุฒุงู† ุงู„ู…ูุชูˆุญุฉ. 346 ู…ู„ูŠูˆู† ู…ุนู„ู…ุฉ ูˆู…ุฎุฑุฌุงุช ุฐุงุช 2048 ุจูุนุฏู‹ุง. ุญุชู‰ 50 ู…ุณุชู†ุฏู‹ุง / ุฏู‚ูŠู‚ุฉ / ู†ูˆุงุฉ CPU. ุฐุงูƒุฑุฉ RAM ุงู„ู…ูˆุตู‰ ุจู‡ุง ู„ู„ุชุดุบูŠู„ ุนู„ู‰ CPU: โ€4 ุฌูŠุฌุงุจุงูŠุช ู„ูƒู„ ุฎุงุฏู… ู†ู…ุงุฐุฌ.", + "nemotron3Embed1b": "ู†ู…ูˆุฐุฌ NVIDIA ู…ุชุนุฏุฏ ุงู„ู„ุบุงุช. 1.1 ู…ู„ูŠุงุฑ ู…ุนู„ู…ุฉ ูˆู…ุฎุฑุฌุงุช ุฐุงุช 2048 ุจูุนุฏู‹ุง. ุญุชู‰ 5 ู…ุณุชู†ุฏุงุช / ุฏู‚ูŠู‚ุฉ / ู†ูˆุงุฉ CPU. ุฐุงูƒุฑุฉ RAM ุงู„ู…ูˆุตู‰ ุจู‡ุง ู„ู„ุชุดุบูŠู„ ุนู„ู‰ CPU: โ€8 ุฌูŠุฌุงุจุงูŠุช ู„ูƒู„ ุฎุงุฏู… ู†ู…ุงุฐุฌ. ูŠูู†ุตุญ ุจุงุณุชุฎุฏุงู… GPU.", "nomicEmbedTextV1": "ู†ู…ูˆุฐุฌ ุงู„ุชุถู…ูŠู† ู…ู† Nomic ุงู„ู…ุชุฎุตุต ููŠ ุงู„ุงุณุชุฑุฌุงุน ูˆุงู„ุชุดุงุจู‡ ูˆุงู„ุชุฌู…ูŠุน ูˆุงู„ุชุตู†ูŠู.", - "e5BaseV2": "ู†ู…ูˆุฐุฌ ุฃุตุบุฑ ูˆุฃุณุฑุน ู…ู† ุงู„ู†ู…ูˆุฐุฌ ุงู„ุงูุชุฑุงุถูŠ. ุฃุณุฑุน ุจู†ุญูˆ ู…ุฑุชูŠู† ู…ู† ุงู„ู†ู…ูˆุฐุฌ ุงู„ุงูุชุฑุงุถูŠ ุนู„ู‰ ุญุณุงุจ ุฌูˆุฏุฉ ุจุญุซ ุฃู‚ู„.", + "e5BaseV2": "ู†ู…ูˆุฐุฌ ุชุถู…ูŠู† ู„ู„ุบุฉ ุงู„ุฅู†ุฌู„ูŠุฒูŠุฉ ู…ู† ุนุงุฆู„ุฉ E5 ู…ู† Microsoft. ูŠูˆุงุฒู† ุจูŠู† ุงู„ุณุฑุนุฉ ูˆุฌูˆุฏุฉ ุงู„ุจุญุซ.", "e5SmallV2": "ุฃุตุบุฑ ูˆุฃุณุฑุน ุฅุตุฏุงุฑ ู…ู† ุณู„ุณู„ุฉ ู†ู…ุงุฐุฌ E5. ุฅุฐุง ูƒู†ุช ุชุดุบู‘ู„ {appName} ุนู„ู‰ ู†ุธุงู… ู…ุญุฏูˆุฏ ุงู„ู…ูˆุงุฑุฏ ูู‚ุฏ ูŠูƒูˆู† ู‡ุฐุง ุฎูŠุงุฑู‹ุง ุฌูŠุฏู‹ุง.", "multilingualE5Base": "ู„ู„ู…ุฌู…ูˆุนุงุช ุงู„ู†ุตูŠุฉ ุจู„ุบุงุช ุบูŠุฑ ุงู„ุฅู†ุฌู„ูŠุฒูŠุฉุŒ ู‡ุฐุง ู‡ูˆ ุงู„ุฎูŠุงุฑ ุงู„ู…ู†ุงุณุจ.", "multilingualE5Small": "ู„ู„ู…ุฌู…ูˆุนุงุช ุงู„ู†ุตูŠุฉ ุจู„ุบุงุช ุบูŠุฑ ุงู„ุฅู†ุฌู„ูŠุฒูŠุฉ ู…ุน ู†ุธุงู… ู…ุญุฏูˆุฏ ุงู„ู…ูˆุงุฑุฏุŒ ู‡ุฐุง ู‡ูˆ ุงู„ุฎูŠุงุฑ ุงู„ู…ู†ุงุณุจ." diff --git a/web/src/i18n/messages/de.json b/web/src/i18n/messages/de.json index ab32d0869d..a0b8248efe 100644 --- a/web/src/i18n/messages/de.json +++ b/web/src/i18n/messages/de.json @@ -6863,6 +6863,9 @@ "noModelsTooltip": "Contextual Retrieval ist deaktiviert, weil keine Modelle konfiguriert sind. Richten Sie zuerst ein [Sprachmodell]({link}) ein.", "title": "Contextual Retrieval" }, + "currentModel": { + "legacyHint": "Dies ist ein veraltetes Modell. Es funktioniert weiter, aber Sie kรถnnen es nicht erneut fรผr einen neuen Index auswรคhlen. Klicken Sie zum Upgrade auf โ€žAlle Modelle anzeigenโ€œ und wรคhlen Sie ein neueres Modell." + }, "embeddingModel": { "cloudManaged": { "title": "Das Embedding-Modell und die Einstellungen werden von Onyx Cloud verwaltet." @@ -7001,9 +7004,12 @@ "currentButton": { "label": "Aktuelles Modell" }, - "deprecated": { - "connectTooltip": "Dieses Embedding-Modell ist veraltet und kann nicht verbunden werden.", - "selectTooltip": "Dieses Embedding-Modell ist veraltet und kann nicht ausgewรคhlt werden." + "gpuRecommended": { + "tag": "GPU empfohlen" + }, + "legacy": { + "tag": "Veraltet", + "tooltip": "Sie kรถnnen dieses Modell nicht fรผr einen neuen Index auswรคhlen. Es funktioniert weiter, bis Sie das Modell wechseln." }, "selectButton": { "label": "Modell auswรคhlen" @@ -7013,9 +7019,15 @@ } }, "modelPicker": { + "cpuEstimateHint": "CPU-Geschwindigkeiten sind Schรคtzungen und hรคngen von Hardware und DokumentgrรถรŸe ab. RAM-Empfehlungen gelten fรผr ein geladenes Modell, nicht fรผr den gesamten Onyx-Stack. Planen Sie beim Neuindizieren mehr Speicher ein.", "addCustomModel": { "label": "Benutzerdefiniertes Modell hinzufรผgen" }, + "bringYourOwn": { + "cloudDescription": "Verbinden Sie einen LiteLLM-Proxy, ein Azure-OpenAI-Deployment oder ein Bifrost-Gateway. Den Modellnamen geben Sie selbst an. Fรผr LiteLLM und Azure geben Sie auch die Dimension an.", + "selfHostedDescription": "Betreiben Sie ein Hugging-Face-Embedding-Modell, das nicht in der Liste oben steht. Modellname, Dimension und Prรคfixe geben Sie selbst an.", + "title": "Erweitert: eigenes Modell oder eigenen Proxy verwenden" + }, "cloudTab": { "label": "Cloud-basiert" }, @@ -7214,22 +7226,27 @@ "restoreHint": "Um einen Konnektor zu behalten, brechen Sie hier ab, setzen Sie ihn fort oder korrigieren Sie ihn und starten Sie die Neuindizierung erneut โ€“ dann wird er รผbernommen. Andernfalls fรผgen Sie ihn nach Abschluss der Neuindizierung erneut hinzu." }, "modelDescriptions": { + "cohereEmbedV5Pro": "Das Top-Embedding-Modell von Cohere. Das beste Verhรคltnis von Qualitรคt und Kosten. 2048 Dimensionen.", + "cohereEmbedV5Fast": "Gรผnstigere Variante von Embed 5 von Cohere. Trotzdem eine ausgezeichnete Wahl. 2048 Dimensionen.", "cohereEmbedEnglishV3": "Englisches Embedding-Modell von Cohere. Gute Leistung bei englischsprachigen Aufgaben.", "cohereEmbedEnglishLightV3": "Leichtgewichtiges englisches Embedding-Modell von Cohere. Schneller und effizienter fรผr einfachere Aufgaben.", - "cohereEmbedV4": "Neuestes mehrsprachiges Embedding-Modell von Cohere mit standardmรครŸig 1536 Dimensionen fรผr bessere Retrieval-Qualitรคt.", - "openaiTextEmbedding3Large": "GroรŸes Embedding-Modell von OpenAI. Beste Leistung, aber teurer.", - "openaiTextEmbedding3Small": "Neueres, effizienteres Embedding-Modell von OpenAI. Gutes Verhรคltnis von Leistung und Kosten.", + "cohereEmbedV4": "Mehrsprachiges Embedding-Modell von Cohere mit 1536 Dimensionen.", + "openaiTextEmbedding3Large": "GroรŸes Embedding-Modell von OpenAI. Eine gute Wahl, wenn Sie OpenAI bereits nutzen. 3072 Dimensionen.", + "openaiTextEmbedding3Small": "Kleines Embedding-Modell von OpenAI. Mit Abstand die gรผnstigste Cloud-basierte Option. 1536 Dimensionen.", "googleGeminiEmbedding001": "Gemini-Embedding-Modell von Google. Leistungsstark und effizient.", "googleTextEmbedding005": "Kleineres, leichteres Embedding-Modell von Google.", - "googleGeminiEmbedding2": "Neuestes multimodales Embedding-Modell von Google (GA). Hรถchste Retrieval-Qualitรคt mit 3072 Dimensionen.", + "googleGeminiEmbedding2": "Das Top-Embedding-Modell von Google. Sehr leistungsstark, aber teuer und langsamer. Mit 3072 Dimensionen entsteht ein groรŸer Index. Eine solide Wahl fรผr Teams, die Google bereits nutzen.", "googleGeminiEmbedding2Preview": "Vorschauversion von Gemini Embedding 2. Fรผr neue Einrichtungen durch gemini-embedding-2 ersetzt.", "voyageLarge2Instruct": "GroรŸes Embedding-Modell von Voyage. Hohe Leistung durch Instruction-Fine-Tuning.", "voyage4Large": "GrรถรŸtes Embedding-Modell der Voyage-4-Serie. Beste Retrieval-Qualitรคt von Voyage, mehrsprachig.", "voyage4": "Universelles Embedding-Modell der Voyage-4-Serie. Hohe Retrieval-Qualitรคt zu moderaten Kosten.", "voyage4Lite": "Leichtgewichtiges Embedding-Modell der Voyage-4-Serie. Niedrigste Kosten und hรถchste Rate-Limits.", "voyageLight2Instruct": "Leichtgewichtiges Embedding-Modell von Voyage. Gutes Verhรคltnis von Leistung und Effizienz.", + "graniteEmbedding97mMultilingualR2": "Das kleine mehrsprachige Modell von IBM. In {appName} enthalten. 97 Mio. Parameter, 384 Dimensionen. Bis zu 200 Dokumente / Min. / CPU-Kern. Empfohlener CPU-RAM: 4 GB pro Modellserver.", + "voyage4Nano": "Das Modell mit offenen Gewichten von Voyage AI. 346 Mio. Parameter, 2048 Dimensionen. Bis zu 50 Dokumente / Min. / CPU-Kern. Empfohlener CPU-RAM: 4 GB pro Modellserver.", + "nemotron3Embed1b": "Das mehrsprachige Modell von NVIDIA. 1,1 Mrd. Parameter, 2048 Dimensionen. Bis zu 5 Dokumente / Min. / CPU-Kern. Empfohlener CPU-RAM: 8 GB pro Modellserver. GPU empfohlen.", "nomicEmbedTextV1": "Embedding-Modell von Nomic, spezialisiert auf Retrieval, ร„hnlichkeit, Clustering und Klassifikation.", - "e5BaseV2": "Ein kleineres und schnelleres Modell als der Standard. Etwa doppelt so schnell wie das Standardmodell, bei geringerer Suchqualitรคt.", + "e5BaseV2": "Englisches Embedding-Modell aus der E5-Familie von Microsoft. Ausgewogen zwischen Geschwindigkeit und Suchqualitรคt.", "e5SmallV2": "Die kleinste und schnellste Variante der E5-Modellreihe. Eine gute Wahl, wenn {appName} auf einem System mit begrenzten Ressourcen lรคuft.", "multilingualE5Base": "Fรผr Korpora in anderen Sprachen als Englisch ist dies die richtige Wahl.", "multilingualE5Small": "Fรผr Korpora in anderen Sprachen als Englisch auf einem System mit begrenzten Ressourcen ist dies die richtige Wahl." diff --git a/web/src/i18n/messages/en.json b/web/src/i18n/messages/en.json index 8cd4d190da..d14dbe4492 100644 --- a/web/src/i18n/messages/en.json +++ b/web/src/i18n/messages/en.json @@ -6863,6 +6863,9 @@ "noModelsTooltip": "Contextual Retrieval is disabled because you have no models configured. Set up a [Language Model]({link}) first.", "title": "Contextual Retrieval" }, + "currentModel": { + "legacyHint": "This is a legacy model. It keeps working, but you cannot select it again for a new index. To upgrade, click View All Models and select a newer model." + }, "embeddingModel": { "cloudManaged": { "title": "Embedding model and settings are managed by Onyx Cloud." @@ -7001,9 +7004,12 @@ "currentButton": { "label": "Current Model" }, - "deprecated": { - "connectTooltip": "This embedding model is deprecated and cannot be connected to.", - "selectTooltip": "This embedding model is deprecated and cannot be selected." + "gpuRecommended": { + "tag": "GPU recommended" + }, + "legacy": { + "tag": "Legacy", + "tooltip": "You cannot select this model for a new index. It keeps working until you change the model." }, "selectButton": { "label": "Select Model" @@ -7013,9 +7019,15 @@ } }, "modelPicker": { + "cpuEstimateHint": "CPU speeds are estimates and vary with hardware and document size. RAM recommendations cover one loaded model, not the full Onyx stack. Allow more memory during re-indexing.", "addCustomModel": { "label": "Add Custom Model" }, + "bringYourOwn": { + "cloudDescription": "Connect a LiteLLM proxy, an Azure OpenAI deployment or a Bifrost gateway. You supply the model name. For LiteLLM and Azure, you also supply the dimension.", + "selfHostedDescription": "Run a Hugging Face embedding model that is not in the list above. You supply the model name, dimension and prefixes.", + "title": "Advanced: bring your own model or proxy" + }, "cloudTab": { "label": "Cloud-based" }, @@ -7214,22 +7226,27 @@ "restoreHint": "To keep a connector, cancel here, resume or fix it, then re-index โ€” it will be carried over. Otherwise, add it back once the re-index finishes." }, "modelDescriptions": { + "cohereEmbedV5Pro": "Cohere's top embedding model. The best balance of quality and cost. 2048-dim output.", + "cohereEmbedV5Fast": "A lower-cost version of Cohere's Embed 5. Still an excellent option. 2048-dim output.", "cohereEmbedEnglishV3": "Cohere's English embedding model. Good performance for English-language tasks.", "cohereEmbedEnglishLightV3": "Cohere's lightweight English embedding model. Faster and more efficient for simpler tasks.", - "cohereEmbedV4": "Cohere's latest multilingual embedding model with the default 1536-dim output for stronger retrieval quality.", - "openaiTextEmbedding3Large": "OpenAI's large embedding model. Best performance, but more expensive.", - "openaiTextEmbedding3Small": "OpenAI's newer, more efficient embedding model. Good balance of performance and cost.", + "cohereEmbedV4": "Cohere's multilingual embedding model, with a 1536-dim output.", + "openaiTextEmbedding3Large": "OpenAI's large embedding model. A good choice if you already use OpenAI. 3072-dim output.", + "openaiTextEmbedding3Small": "OpenAI's small embedding model. By far the most economical cloud-based option. 1536-dim output.", "googleGeminiEmbedding001": "Google's Gemini embedding model. Powerful and efficient.", "googleTextEmbedding005": "Smaller, lighter-weight embedding model from Google.", - "googleGeminiEmbedding2": "Google's latest multimodal embedding model (GA). Highest-quality retrieval with a 3072-dim output.", + "googleGeminiEmbedding2": "Google's top embedding model. Very powerful, but expensive and slower. Its 3072-dim output builds a large index. A solid choice for teams that already use Google.", "googleGeminiEmbedding2Preview": "Preview version of Gemini Embedding 2. Superseded by gemini-embedding-2 for new setups.", "voyageLarge2Instruct": "Voyage's large embedding model. High performance with instruction fine-tuning.", "voyage4Large": "Voyage's largest 4-series embedding model. Best Voyage retrieval quality, multilingual.", "voyage4": "Voyage's general-purpose 4-series embedding model. High retrieval quality at moderate cost.", "voyage4Lite": "Voyage's lightweight 4-series embedding model. Lowest cost and highest rate limits.", "voyageLight2Instruct": "Voyage's lightweight embedding model. Good balance of performance and efficiency.", + "graniteEmbedding97mMultilingualR2": "IBM's small multilingual model. Ships with {appName}. 97M parameters, 384-dim output. Up to 200 docs / min / CPU core. Recommended CPU RAM: 4 GB per model server.", + "voyage4Nano": "Voyage AI's open-weights model. 346M parameters, 2048-dim output. Up to 50 docs / min / CPU core. Recommended CPU RAM: 4 GB per model server.", + "nemotron3Embed1b": "NVIDIA's multilingual model. 1.1B parameters, 2048-dim output. Up to 5 docs / min / CPU core. Recommended CPU RAM: 8 GB per model server. GPU recommended.", "nomicEmbedTextV1": "Nomic's embedding model specialized for retrieval, similarity, clustering and classification.", - "e5BaseV2": "A smaller and faster model than the default. It is around 2x faster than the default model at the cost of lower search quality.", + "e5BaseV2": "English embedding model from Microsoft's E5 family. A balance of speed and search quality.", "e5SmallV2": "The smallest and fastest version of the E5 line of models. If you're running {appName} on a resource constrained system, then this may be a good choice.", "multilingualE5Base": "For corpora in other languages besides English, this is the one to choose.", "multilingualE5Small": "For corpora in other languages besides English, as well as being on a resource constrained system, this is the one to choose." diff --git a/web/src/i18n/messages/es.json b/web/src/i18n/messages/es.json index 44234dee97..dae062df8f 100644 --- a/web/src/i18n/messages/es.json +++ b/web/src/i18n/messages/es.json @@ -6863,6 +6863,9 @@ "noModelsTooltip": "La recuperaciรณn contextual estรก desactivada porque no tienes modelos configurados. Configura primero un [modelo de lenguaje]({link}).", "title": "Recuperaciรณn contextual" }, + "currentModel": { + "legacyHint": "Este es un modelo heredado. Sigue funcionando, pero no puedes volver a seleccionarlo para un รญndice nuevo. Para actualizar, haz clic en ยซVer todos los modelosยป y selecciona un modelo mรกs reciente." + }, "embeddingModel": { "cloudManaged": { "title": "Onyx Cloud gestiona el modelo de embeddings y su configuraciรณn." @@ -7001,9 +7004,12 @@ "currentButton": { "label": "Modelo actual" }, - "deprecated": { - "connectTooltip": "Este modelo de embeddings estรก obsoleto y no se puede conectar.", - "selectTooltip": "Este modelo de embeddings estรก obsoleto y no se puede seleccionar." + "gpuRecommended": { + "tag": "GPU recomendada" + }, + "legacy": { + "tag": "Heredado", + "tooltip": "No puedes seleccionar este modelo para un รญndice nuevo. Seguirรก funcionando hasta que cambies de modelo." }, "selectButton": { "label": "Seleccionar modelo" @@ -7013,9 +7019,15 @@ } }, "modelPicker": { + "cpuEstimateHint": "Las velocidades de CPU son estimaciones y varรญan segรบn el hardware y el tamaรฑo del documento. La RAM recomendada cubre un modelo cargado, no toda la infraestructura de Onyx. Reserva mรกs memoria durante la reindexaciรณn.", "addCustomModel": { "label": "Aรฑadir modelo personalizado" }, + "bringYourOwn": { + "cloudDescription": "Conecta un proxy de LiteLLM, una implementaciรณn de Azure OpenAI o una puerta de enlace de Bifrost. Tรบ indicas el nombre del modelo. Para LiteLLM y Azure, tambiรฉn indicas la dimensiรณn.", + "selfHostedDescription": "Ejecuta un modelo de embeddings de Hugging Face que no estรฉ en la lista anterior. Tรบ indicas el nombre, la dimensiรณn y los prefijos del modelo.", + "title": "Avanzado: usa tu propio modelo o proxy" + }, "cloudTab": { "label": "En la nube" }, @@ -7214,22 +7226,27 @@ "restoreHint": "Para conservar un conector, cancela aquรญ, reanรบdalo o corrรญgelo y vuelve a reindexar: asรญ se trasladarรก. Si no, aรฑรกdelo de nuevo cuando termine la reindexaciรณn." }, "modelDescriptions": { + "cohereEmbedV5Pro": "El modelo de embeddings principal de Cohere. El mejor equilibrio entre calidad y coste. Salida de 2048 dimensiones.", + "cohereEmbedV5Fast": "Versiรณn de menor coste de Embed 5 de Cohere. Sigue siendo una opciรณn excelente. Salida de 2048 dimensiones.", "cohereEmbedEnglishV3": "Modelo de embeddings en inglรฉs de Cohere. Buen rendimiento en tareas en inglรฉs.", "cohereEmbedEnglishLightV3": "Modelo ligero de embeddings en inglรฉs de Cohere. Mรกs rรกpido y eficiente para tareas sencillas.", - "cohereEmbedV4": "El modelo de embeddings multilingรผe mรกs reciente de Cohere, con salida de 1536 dimensiones por defecto para una mejor calidad de recuperaciรณn.", - "openaiTextEmbedding3Large": "Modelo de embeddings grande de OpenAI. El mejor rendimiento, pero mรกs caro.", - "openaiTextEmbedding3Small": "Modelo de embeddings mรกs nuevo y eficiente de OpenAI. Buen equilibrio entre rendimiento y coste.", + "cohereEmbedV4": "El modelo de embeddings multilingรผe de Cohere, con salida de 1536 dimensiones.", + "openaiTextEmbedding3Large": "Modelo de embeddings grande de OpenAI. Una buena opciรณn si ya usas OpenAI. Salida de 3072 dimensiones.", + "openaiTextEmbedding3Small": "Modelo de embeddings pequeรฑo de OpenAI. Con diferencia, la opciรณn en la nube mรกs econรณmica. Salida de 1536 dimensiones.", "googleGeminiEmbedding001": "Modelo de embeddings Gemini de Google. Potente y eficiente.", "googleTextEmbedding005": "Modelo de embeddings mรกs pequeรฑo y ligero de Google.", - "googleGeminiEmbedding2": "El modelo de embeddings multimodal mรกs reciente de Google (GA). Recuperaciรณn de mรกxima calidad con salida de 3072 dimensiones.", + "googleGeminiEmbedding2": "El modelo de embeddings principal de Google. Muy potente, pero caro y mรกs lento. Su salida de 3072 dimensiones genera un รญndice grande. Una opciรณn sรณlida para equipos que ya usan Google.", "googleGeminiEmbedding2Preview": "Versiรณn preliminar de Gemini Embedding 2. Sustituida por gemini-embedding-2 en nuevas configuraciones.", "voyageLarge2Instruct": "Modelo de embeddings grande de Voyage. Alto rendimiento con ajuste fino por instrucciones.", "voyage4Large": "El modelo de embeddings mรกs grande de la serie 4 de Voyage. La mejor calidad de recuperaciรณn de Voyage, multilingรผe.", "voyage4": "Modelo de embeddings de uso general de la serie 4 de Voyage. Alta calidad de recuperaciรณn a un coste moderado.", "voyage4Lite": "Modelo ligero de embeddings de la serie 4 de Voyage. El menor coste y los lรญmites de uso mรกs altos.", "voyageLight2Instruct": "Modelo ligero de embeddings de Voyage. Buen equilibrio entre rendimiento y eficiencia.", + "graniteEmbedding97mMultilingualR2": "El modelo multilingรผe pequeรฑo de IBM. Incluido en {appName}. 97 M de parรกmetros, 384 dimensiones. Hasta 200 documentos / min / nรบcleo de CPU. RAM recomendada para CPU: 4 GB por servidor de modelos.", + "voyage4Nano": "El modelo de pesos abiertos de Voyage AI. 346 M de parรกmetros, 2048 dimensiones. Hasta 50 documentos / min / nรบcleo de CPU. RAM recomendada para CPU: 4 GB por servidor de modelos.", + "nemotron3Embed1b": "El modelo multilingรผe de NVIDIA. 1100 M de parรกmetros, 2048 dimensiones. Hasta 5 documentos / min / nรบcleo de CPU. RAM recomendada para CPU: 8 GB por servidor de modelos. GPU recomendada.", "nomicEmbedTextV1": "Modelo de embeddings de Nomic especializado en recuperaciรณn, similitud, agrupaciรณn y clasificaciรณn.", - "e5BaseV2": "Un modelo mรกs pequeรฑo y rรกpido que el predeterminado. Es unas 2 veces mรกs rรกpido que el modelo predeterminado a costa de una menor calidad de bรบsqueda.", + "e5BaseV2": "Modelo de embeddings en inglรฉs de la familia E5 de Microsoft. Equilibra velocidad y calidad de bรบsqueda.", "e5SmallV2": "La versiรณn mรกs pequeรฑa y rรกpida de la lรญnea de modelos E5. Si ejecutas {appName} en un sistema con recursos limitados, puede ser una buena opciรณn.", "multilingualE5Base": "Para corpus en idiomas distintos del inglรฉs, esta es la opciรณn indicada.", "multilingualE5Small": "Para corpus en idiomas distintos del inglรฉs en un sistema con recursos limitados, esta es la opciรณn indicada." diff --git a/web/src/i18n/messages/fr.json b/web/src/i18n/messages/fr.json index 7e540e2ccd..423e576bfb 100644 --- a/web/src/i18n/messages/fr.json +++ b/web/src/i18n/messages/fr.json @@ -6863,6 +6863,9 @@ "noModelsTooltip": "La rรฉcupรฉration contextuelle est dรฉsactivรฉe car aucun modรจle n'est configurรฉ. Configurez d'abord un [modรจle de langage]({link}).", "title": "Rรฉcupรฉration contextuelle" }, + "currentModel": { + "legacyHint": "Ce modรจle est un ancien modรจle. Il continue de fonctionner, mais vous ne pouvez plus le sรฉlectionner pour un nouvel index. Pour passer ร  un modรจle plus rรฉcent, cliquez sur ยซ Voir tous les modรจles ยป et sรฉlectionnez-en un." + }, "embeddingModel": { "cloudManaged": { "title": "Le modรจle d'embedding et ses paramรจtres sont gรฉrรฉs par Onyx Cloud." @@ -7001,9 +7004,12 @@ "currentButton": { "label": "Modรจle actuel" }, - "deprecated": { - "connectTooltip": "Ce modรจle d'embedding est obsolรจte et ne peut pas รชtre connectรฉ.", - "selectTooltip": "Ce modรจle d'embedding est obsolรจte et ne peut pas รชtre sรฉlectionnรฉ." + "gpuRecommended": { + "tag": "GPU recommandรฉ" + }, + "legacy": { + "tag": "Ancien", + "tooltip": "Vous ne pouvez pas sรฉlectionner ce modรจle pour un nouvel index. Il continue de fonctionner jusqu'ร  ce que vous changiez de modรจle." }, "selectButton": { "label": "Sรฉlectionner le modรจle" @@ -7013,9 +7019,15 @@ } }, "modelPicker": { + "cpuEstimateHint": "Les vitesses CPU sont des estimations qui varient selon le matรฉriel et la taille des documents. La RAM recommandรฉe couvre un modรจle chargรฉ, pas toute lโ€™infrastructure Onyx. Prรฉvoyez plus de mรฉmoire pendant la rรฉindexation.", "addCustomModel": { "label": "Ajouter un modรจle personnalisรฉ" }, + "bringYourOwn": { + "cloudDescription": "Connectez un proxy LiteLLM, un dรฉploiement Azure OpenAI ou une passerelle Bifrost. Vous indiquez vous-mรชme le nom du modรจle. Pour LiteLLM et Azure, vous indiquez aussi la dimension.", + "selfHostedDescription": "Exรฉcutez un modรจle d'embedding Hugging Face absent de la liste ci-dessus. Vous indiquez vous-mรชme le nom, la dimension et les prรฉfixes du modรจle.", + "title": "Avancรฉ : utiliser votre propre modรจle ou proxy" + }, "cloudTab": { "label": "Dans le cloud" }, @@ -7214,22 +7226,27 @@ "restoreHint": "Pour conserver un connecteur, annulez ici, reprenez-le ou corrigez-le, puis relancez la rรฉindexation : il sera alors repris. Sinon, rajoutez-le une fois la rรฉindexation terminรฉe." }, "modelDescriptions": { + "cohereEmbedV5Pro": "Le modรจle d'embedding phare de Cohere. Le meilleur รฉquilibre entre qualitรฉ et coรปt. Sortie de 2048 dimensions.", + "cohereEmbedV5Fast": "Version moins coรปteuse d'Embed 5 de Cohere. Reste une excellente option. Sortie de 2048 dimensions.", "cohereEmbedEnglishV3": "Modรจle d'embedding anglais de Cohere. Bonnes performances pour les tรขches en anglais.", "cohereEmbedEnglishLightV3": "Modรจle d'embedding anglais lรฉger de Cohere. Plus rapide et plus efficace pour les tรขches simples.", - "cohereEmbedV4": "Dernier modรจle d'embedding multilingue de Cohere, avec une sortie par dรฉfaut de 1536 dimensions pour une meilleure qualitรฉ de recherche.", - "openaiTextEmbedding3Large": "Grand modรจle d'embedding d'OpenAI. Meilleures performances, mais plus coรปteux.", - "openaiTextEmbedding3Small": "Modรจle d'embedding plus rรฉcent et plus efficace d'OpenAI. Bon รฉquilibre entre performances et coรปt.", + "cohereEmbedV4": "Modรจle d'embedding multilingue de Cohere, avec une sortie de 1536 dimensions.", + "openaiTextEmbedding3Large": "Grand modรจle d'embedding d'OpenAI. Un bon choix si vous utilisez dรฉjร  OpenAI. Sortie de 3072 dimensions.", + "openaiTextEmbedding3Small": "Petit modรจle d'embedding d'OpenAI. De loin l'option dans le cloud la plus รฉconomique. Sortie de 1536 dimensions.", "googleGeminiEmbedding001": "Modรจle d'embedding Gemini de Google. Puissant et efficace.", "googleTextEmbedding005": "Modรจle d'embedding plus petit et plus lรฉger de Google.", - "googleGeminiEmbedding2": "Dernier modรจle d'embedding multimodal de Google (GA). Recherche de la plus haute qualitรฉ avec une sortie de 3072 dimensions.", + "googleGeminiEmbedding2": "Le modรจle d'embedding phare de Google. Trรจs puissant, mais coรปteux et plus lent. Sa sortie de 3072 dimensions produit un index volumineux. Un choix solide pour les รฉquipes qui utilisent dรฉjร  Google.", "googleGeminiEmbedding2Preview": "Version prรฉliminaire de Gemini Embedding 2. Remplacรฉe par gemini-embedding-2 pour les nouvelles configurations.", "voyageLarge2Instruct": "Grand modรจle d'embedding de Voyage. Hautes performances grรขce au fine-tuning par instructions.", "voyage4Large": "Le plus grand modรจle d'embedding de la sรฉrie 4 de Voyage. Meilleure qualitรฉ de recherche de Voyage, multilingue.", "voyage4": "Modรจle d'embedding polyvalent de la sรฉrie 4 de Voyage. Haute qualitรฉ de recherche ร  coรปt modรฉrรฉ.", "voyage4Lite": "Modรจle d'embedding lรฉger de la sรฉrie 4 de Voyage. Coรปt le plus bas et limites de dรฉbit les plus รฉlevรฉes.", "voyageLight2Instruct": "Modรจle d'embedding lรฉger de Voyage. Bon รฉquilibre entre performances et efficacitรฉ.", + "graniteEmbedding97mMultilingualR2": "Le petit modรจle multilingue dโ€™IBM. Inclus dans {appName}. 97 M de paramรจtres, sortie de 384 dimensions. Jusquโ€™ร  200 documents / min / cล“ur CPU. RAM CPU recommandรฉe : 4 Go par serveur de modรจles.", + "voyage4Nano": "Le modรจle ร  poids ouverts de Voyage AI. 346 M de paramรจtres, sortie de 2048 dimensions. Jusquโ€™ร  50 documents / min / cล“ur CPU. RAM CPU recommandรฉe : 4 Go par serveur de modรจles.", + "nemotron3Embed1b": "Le modรจle multilingue de NVIDIA. 1,1 milliard de paramรจtres, sortie de 2048 dimensions. Jusquโ€™ร  5 documents / min / cล“ur CPU. RAM CPU recommandรฉe : 8 Go par serveur de modรจles. GPU recommandรฉ.", "nomicEmbedTextV1": "Modรจle d'embedding de Nomic spรฉcialisรฉ dans la recherche, la similaritรฉ, le clustering et la classification.", - "e5BaseV2": "Un modรจle plus petit et plus rapide que celui par dรฉfaut. Environ 2 fois plus rapide que le modรจle par dรฉfaut, au prix d'une qualitรฉ de recherche moindre.", + "e5BaseV2": "Modรจle d'embedding anglais de la famille E5 de Microsoft. Un รฉquilibre entre vitesse et qualitรฉ de recherche.", "e5SmallV2": "La version la plus petite et la plus rapide de la gamme E5. Un bon choix si {appName} tourne sur un systรจme aux ressources limitรฉes.", "multilingualE5Base": "Pour les corpus dans d'autres langues que l'anglais, c'est le modรจle ร  choisir.", "multilingualE5Small": "Pour les corpus dans d'autres langues que l'anglais sur un systรจme aux ressources limitรฉes, c'est le modรจle ร  choisir." diff --git a/web/src/i18n/messages/ja.json b/web/src/i18n/messages/ja.json index 4c34149064..bee91fa537 100644 --- a/web/src/i18n/messages/ja.json +++ b/web/src/i18n/messages/ja.json @@ -6863,6 +6863,9 @@ "noModelsTooltip": "ใƒขใƒ‡ใƒซใŒ่จญๅฎšใ•ใ‚Œใฆใ„ใชใ„ใŸใ‚ใ€ใ‚ณใƒณใƒ†ใ‚ญใ‚นใƒˆๆคœ็ดขใฏ็„กๅŠนใงใ™ใ€‚ๅ…ˆใซ [่จ€่ชžใƒขใƒ‡ใƒซ]({link}) ใ‚’่จญๅฎšใ—ใฆใใ ใ•ใ„ใ€‚", "title": "ใ‚ณใƒณใƒ†ใ‚ญใ‚นใƒˆๆคœ็ดข" }, + "currentModel": { + "legacyHint": "ใ“ใ‚Œใฏใƒฌใ‚ฌใ‚ทใƒผใƒขใƒ‡ใƒซใงใ™ใ€‚ๅผ•ใ็ถšใๅ‹•ไฝœใ—ใพใ™ใŒใ€ๆ–ฐใ—ใ„ใ‚คใƒณใƒ‡ใƒƒใ‚ฏใ‚นใซๅ†ๅบฆ้ธๆŠžใ™ใ‚‹ใ“ใจใฏใงใใพใ›ใ‚“ใ€‚ใ‚ขใƒƒใƒ—ใ‚ฐใƒฌใƒผใƒ‰ใ™ใ‚‹ใซใฏใ€ใ€Œใ™ในใฆใฎใƒขใƒ‡ใƒซใ‚’่กจ็คบใ€ใ‚’ใ‚ฏใƒชใƒƒใ‚ฏใ—ใฆๆ–ฐใ—ใ„ใƒขใƒ‡ใƒซใ‚’้ธๆŠžใ—ใฆใใ ใ•ใ„ใ€‚" + }, "embeddingModel": { "cloudManaged": { "title": "embedding ใƒขใƒ‡ใƒซใจ้–ข้€ฃ่จญๅฎšใฏ Onyx Cloud ใŒ็ฎก็†ใ—ใพใ™ใ€‚" @@ -7001,9 +7004,12 @@ "currentButton": { "label": "็พๅœจใฎใƒขใƒ‡ใƒซ" }, - "deprecated": { - "connectTooltip": "ใ“ใฎ embedding ใƒขใƒ‡ใƒซใฏ้žๆŽจๅฅจใฎใŸใ‚ๆŽฅ็ถšใงใใพใ›ใ‚“ใ€‚", - "selectTooltip": "ใ“ใฎ embedding ใƒขใƒ‡ใƒซใฏ้žๆŽจๅฅจใฎใŸใ‚้ธๆŠžใงใใพใ›ใ‚“ใ€‚" + "gpuRecommended": { + "tag": "GPUๆŽจๅฅจ" + }, + "legacy": { + "tag": "ใƒฌใ‚ฌใ‚ทใƒผ", + "tooltip": "ใ“ใฎใƒขใƒ‡ใƒซใฏๆ–ฐใ—ใ„ใ‚คใƒณใƒ‡ใƒƒใ‚ฏใ‚นใซใฏ้ธๆŠžใงใใพใ›ใ‚“ใ€‚ใƒขใƒ‡ใƒซใ‚’ๅค‰ๆ›ดใ™ใ‚‹ใพใงใฏๅผ•ใ็ถšใๅ‹•ไฝœใ—ใพใ™ใ€‚" }, "selectButton": { "label": "ใƒขใƒ‡ใƒซใ‚’้ธๆŠž" @@ -7013,9 +7019,15 @@ } }, "modelPicker": { + "cpuEstimateHint": "CPUใฎ้€Ÿๅบฆใฏ็›ฎๅฎ‰ใงใ€ใƒใƒผใƒ‰ใ‚ฆใ‚งใ‚ขใจๆ–‡ๆ›ธใ‚ตใ‚คใ‚บใซใ‚ˆใฃใฆ็•ฐใชใ‚Šใพใ™ใ€‚ๆŽจๅฅจRAMใฏ่ชญใฟ่พผใ‚“ใ ใƒขใƒ‡ใƒซ1ใคๅˆ†ใงใ‚ใ‚Šใ€Onyxๅ…จไฝ“ใฎๅฟ…่ฆ้‡ใงใฏใ‚ใ‚Šใพใ›ใ‚“ใ€‚ๅ†ใ‚คใƒณใƒ‡ใƒƒใ‚ฏใ‚นๆ™‚ใฏ่ฟฝๅŠ ใฎใƒกใƒขใƒชใ‚’็ขบไฟใ—ใฆใใ ใ•ใ„ใ€‚", "addCustomModel": { "label": "ใ‚ซใ‚นใ‚ฟใƒ ใƒขใƒ‡ใƒซใ‚’่ฟฝๅŠ " }, + "bringYourOwn": { + "cloudDescription": "LiteLLM ใƒ—ใƒญใ‚ญใ‚ทใ€Azure OpenAI ใƒ‡ใƒ—ใƒญใ‚คใƒกใƒณใƒˆใ€ใพใŸใฏ Bifrost ใ‚ฒใƒผใƒˆใ‚ฆใ‚งใ‚คใ‚’ๆŽฅ็ถšใ—ใพใ™ใ€‚ใƒขใƒ‡ใƒซๅใฏใ”่‡ช่บซใงๆŒ‡ๅฎšใ—ใพใ™ใ€‚LiteLLM ใจ Azure ใงใฏๆฌกๅ…ƒใ‚‚ๆŒ‡ๅฎšใ—ใพใ™ใ€‚", + "selfHostedDescription": "ไธŠใฎไธ€่ฆงใซใชใ„ Hugging Face ใฎๅŸ‹ใ‚่พผใฟใƒขใƒ‡ใƒซใ‚’ๅฎŸ่กŒใ—ใพใ™ใ€‚ใƒขใƒ‡ใƒซๅใ€ๆฌกๅ…ƒใ€ใƒ—ใƒฌใƒ•ใ‚ฃใƒƒใ‚ฏใ‚นใฏใ”่‡ช่บซใงๆŒ‡ๅฎšใ—ใพใ™ใ€‚", + "title": "่ฉณ็ดฐ่จญๅฎš: ็‹ฌ่‡ชใฎใƒขใƒ‡ใƒซใพใŸใฏใƒ—ใƒญใ‚ญใ‚ทใ‚’ไฝฟ็”จ" + }, "cloudTab": { "label": "ใ‚ฏใƒฉใ‚ฆใƒ‰ๅž‹" }, @@ -7214,22 +7226,27 @@ "restoreHint": "ใ‚ณใƒใ‚ฏใ‚ฟใ‚’ๆฎ‹ใ—ใŸใ„ๅ ดๅˆใฏใ€ใ“ใ“ใงใ‚ญใƒฃใƒณใ‚ปใƒซใ—ใ€ๅ†้–‹ใพใŸใฏไฟฎๆญฃใ—ใฆใ‹ใ‚‰ๅ†ใ‚คใƒณใƒ‡ใƒƒใ‚ฏใ‚นใ‚’ๅฎŸ่กŒใ—ใฆใใ ใ•ใ„ใ€‚ใใ†ใ™ใ‚Œใฐๆ–ฐใ—ใ„ใ‚คใƒณใƒ‡ใƒƒใ‚ฏใ‚นใซๅผ•ใ็ถ™ใŒใ‚Œใพใ™ใ€‚ใใ†ใ—ใชใ„ๅ ดๅˆใฏใ€ๅ†ใ‚คใƒณใƒ‡ใƒƒใ‚ฏใ‚นใฎๅฎŒไบ†ๅพŒใซ่ฟฝๅŠ ใ—็›ดใ—ใฆใใ ใ•ใ„ใ€‚" }, "modelDescriptions": { + "cohereEmbedV5Pro": "Cohere ใฎไธปๅŠ›ๅŸ‹ใ‚่พผใฟใƒขใƒ‡ใƒซใ€‚ๅ“่ณชใจใ‚ณใ‚นใƒˆใฎใƒใƒฉใƒณใ‚นใŒๆœ€ใ‚‚ๅ„ชใ‚Œใฆใ„ใพใ™ใ€‚2048 ๆฌกๅ…ƒๅ‡บๅŠ›ใ€‚", + "cohereEmbedV5Fast": "Cohere ใฎ Embed 5 ใฎไฝŽใ‚ณใ‚นใƒˆ็‰ˆใ€‚ใใ‚Œใงใ‚‚้žๅธธใซๅ„ชใ‚ŒใŸ้ธๆŠž่‚ขใงใ™ใ€‚2048 ๆฌกๅ…ƒๅ‡บๅŠ›ใ€‚", "cohereEmbedEnglishV3": "Cohere ใฎ่‹ฑ่ชžๅ‘ใ‘ๅŸ‹ใ‚่พผใฟใƒขใƒ‡ใƒซใ€‚่‹ฑ่ชžใ‚ฟใ‚นใ‚ฏใง่‰ฏๅฅฝใชๆ€ง่ƒฝใ€‚", "cohereEmbedEnglishLightV3": "Cohere ใฎ่ปฝ้‡ใช่‹ฑ่ชžๅ‘ใ‘ๅŸ‹ใ‚่พผใฟใƒขใƒ‡ใƒซใ€‚ๅ˜็ด”ใชใ‚ฟใ‚นใ‚ฏใงใฏใ‚ˆใ‚Š้ซ˜้€ŸใงๅŠน็Ž‡็š„ใ€‚", - "cohereEmbedV4": "Cohere ใฎๆœ€ๆ–ฐใฎๅคš่จ€่ชžๅŸ‹ใ‚่พผใฟใƒขใƒ‡ใƒซใ€‚ใƒ‡ใƒ•ใ‚ฉใƒซใƒˆใฎ 1536 ๆฌกๅ…ƒๅ‡บๅŠ›ใงๆคœ็ดขๅ“่ณชใ‚’ๅ‘ไธŠใ€‚", - "openaiTextEmbedding3Large": "OpenAI ใฎๅคง่ฆๆจกๅŸ‹ใ‚่พผใฟใƒขใƒ‡ใƒซใ€‚ๆœ€้ซ˜ใฎๆ€ง่ƒฝใ ใŒ้ซ˜ใ‚ณใ‚นใƒˆใ€‚", - "openaiTextEmbedding3Small": "OpenAI ใฎๆ–ฐใ—ใๅŠน็Ž‡็š„ใชๅŸ‹ใ‚่พผใฟใƒขใƒ‡ใƒซใ€‚ๆ€ง่ƒฝใจใ‚ณใ‚นใƒˆใฎใƒใƒฉใƒณใ‚นใŒ่‰ฏใ„ใ€‚", + "cohereEmbedV4": "Cohere ใฎๅคš่จ€่ชžๅŸ‹ใ‚่พผใฟใƒขใƒ‡ใƒซใ€‚1536 ๆฌกๅ…ƒๅ‡บๅŠ›ใ€‚", + "openaiTextEmbedding3Large": "OpenAI ใฎๅคง่ฆๆจกๅŸ‹ใ‚่พผใฟใƒขใƒ‡ใƒซใ€‚ใ™ใงใซ OpenAI ใ‚’ๅˆฉ็”จใ—ใฆใ„ใ‚‹ๅ ดๅˆใซ้ฉใ—ใฆใ„ใพใ™ใ€‚3072 ๆฌกๅ…ƒๅ‡บๅŠ›ใ€‚", + "openaiTextEmbedding3Small": "OpenAI ใฎๅฐๅž‹ๅŸ‹ใ‚่พผใฟใƒขใƒ‡ใƒซใ€‚ใ‚ฏใƒฉใ‚ฆใƒ‰ๅž‹ใฎ้ธๆŠž่‚ขใฎไธญใง็พคใ‚’ๆŠœใ„ใฆ็ตŒๆธˆ็š„ใงใ™ใ€‚1536 ๆฌกๅ…ƒๅ‡บๅŠ›ใ€‚", "googleGeminiEmbedding001": "Google ใฎ Gemini ๅŸ‹ใ‚่พผใฟใƒขใƒ‡ใƒซใ€‚ๅผทๅŠ›ใงๅŠน็Ž‡็š„ใ€‚", "googleTextEmbedding005": "Google ใฎใ‚ˆใ‚Šๅฐๅž‹ใง่ปฝ้‡ใชๅŸ‹ใ‚่พผใฟใƒขใƒ‡ใƒซใ€‚", - "googleGeminiEmbedding2": "Google ใฎๆœ€ๆ–ฐใฎใƒžใƒซใƒใƒขใƒผใƒ€ใƒซๅŸ‹ใ‚่พผใฟใƒขใƒ‡ใƒซ๏ผˆGA๏ผ‰ใ€‚3072 ๆฌกๅ…ƒๅ‡บๅŠ›ใงๆœ€้ซ˜ๅ“่ณชใฎๆคœ็ดขใ€‚", + "googleGeminiEmbedding2": "Google ใฎไธปๅŠ›ๅŸ‹ใ‚่พผใฟใƒขใƒ‡ใƒซใ€‚้žๅธธใซ้ซ˜ๆ€ง่ƒฝใงใ™ใŒใ€ใ‚ณใ‚นใƒˆใŒ้ซ˜ใไฝŽ้€Ÿใงใ™ใ€‚3072 ๆฌกๅ…ƒๅ‡บๅŠ›ใฎใŸใ‚ใ‚คใƒณใƒ‡ใƒƒใ‚ฏใ‚นใŒๅคงใใใชใ‚Šใพใ™ใ€‚ใ™ใงใซ Google ใ‚’ๅˆฉ็”จใ—ใฆใ„ใ‚‹ใƒใƒผใƒ ใซใฏๅ …ๅฎŸใช้ธๆŠž่‚ขใงใ™ใ€‚", "googleGeminiEmbedding2Preview": "Gemini Embedding 2 ใฎใƒ—ใƒฌใƒ“ใƒฅใƒผ็‰ˆใ€‚ๆ–ฐ่ฆใ‚ปใƒƒใƒˆใ‚ขใƒƒใƒ—ใงใฏ gemini-embedding-2 ใซ็ฝฎใๆ›ใˆใ‚‰ใ‚Œใพใ—ใŸใ€‚", "voyageLarge2Instruct": "Voyage ใฎๅคง่ฆๆจกๅŸ‹ใ‚่พผใฟใƒขใƒ‡ใƒซใ€‚ๆŒ‡็คบใซใ‚ˆใ‚‹ใƒ•ใ‚กใ‚คใƒณใƒใƒฅใƒผใƒ‹ใƒณใ‚ฐใง้ซ˜ๆ€ง่ƒฝใ€‚", "voyage4Large": "Voyage 4 ใ‚ทใƒชใƒผใ‚บๆœ€ๅคงใฎๅŸ‹ใ‚่พผใฟใƒขใƒ‡ใƒซใ€‚Voyage ใงๆœ€้ซ˜ใฎๆคœ็ดขๅ“่ณชใ€ๅคš่จ€่ชžๅฏพๅฟœใ€‚", "voyage4": "Voyage 4 ใ‚ทใƒชใƒผใ‚บใฎๆฑŽ็”จๅŸ‹ใ‚่พผใฟใƒขใƒ‡ใƒซใ€‚ๆ‰‹้ ƒใชใ‚ณใ‚นใƒˆใง้ซ˜ใ„ๆคœ็ดขๅ“่ณชใ€‚", "voyage4Lite": "Voyage 4 ใ‚ทใƒชใƒผใ‚บใฎ่ปฝ้‡ๅŸ‹ใ‚่พผใฟใƒขใƒ‡ใƒซใ€‚ๆœ€ใ‚‚ไฝŽใ‚ณใ‚นใƒˆใงใ€ใƒฌใƒผใƒˆๅˆถ้™ใŒๆœ€ใ‚‚้ซ˜ใ„ใ€‚", "voyageLight2Instruct": "Voyage ใฎ่ปฝ้‡ใชๅŸ‹ใ‚่พผใฟใƒขใƒ‡ใƒซใ€‚ๆ€ง่ƒฝใจๅŠน็Ž‡ใฎใƒใƒฉใƒณใ‚นใŒ่‰ฏใ„ใ€‚", + "graniteEmbedding97mMultilingualR2": "IBMใฎๅฐๅž‹ๅคš่จ€่ชžใƒขใƒ‡ใƒซใ€‚{appName}ใซๅŒๆขฑใ€‚9700ไธ‡ใƒ‘ใƒฉใƒกใƒผใ‚ฟใ€384ๆฌกๅ…ƒๅ‡บๅŠ›ใ€‚CPUใ‚ณใ‚ขใ‚ใŸใ‚Šๆœ€ๅคง200ๆ–‡ๆ›ธ/ๅˆ†ใ€‚CPUๅฎŸ่กŒๆ™‚ใฎๆŽจๅฅจRAM๏ผšใƒขใƒ‡ใƒซใ‚ตใƒผใƒใƒผใ‚ใŸใ‚Š4 GBใ€‚", + "voyage4Nano": "Voyage AIใฎใ‚ชใƒผใƒ—ใƒณใ‚ฆใ‚งใ‚คใƒˆใƒขใƒ‡ใƒซใ€‚3ๅ„„4600ไธ‡ใƒ‘ใƒฉใƒกใƒผใ‚ฟใ€2048ๆฌกๅ…ƒๅ‡บๅŠ›ใ€‚CPUใ‚ณใ‚ขใ‚ใŸใ‚Šๆœ€ๅคง50ๆ–‡ๆ›ธ/ๅˆ†ใ€‚CPUๅฎŸ่กŒๆ™‚ใฎๆŽจๅฅจRAM๏ผšใƒขใƒ‡ใƒซใ‚ตใƒผใƒใƒผใ‚ใŸใ‚Š4 GBใ€‚", + "nemotron3Embed1b": "NVIDIAใฎๅคš่จ€่ชžใƒขใƒ‡ใƒซใ€‚11ๅ„„ใƒ‘ใƒฉใƒกใƒผใ‚ฟใ€2048ๆฌกๅ…ƒๅ‡บๅŠ›ใ€‚CPUใ‚ณใ‚ขใ‚ใŸใ‚Šๆœ€ๅคง5ๆ–‡ๆ›ธ/ๅˆ†ใ€‚CPUๅฎŸ่กŒๆ™‚ใฎๆŽจๅฅจRAM๏ผšใƒขใƒ‡ใƒซใ‚ตใƒผใƒใƒผใ‚ใŸใ‚Š8 GBใ€‚GPUๆŽจๅฅจใ€‚", "nomicEmbedTextV1": "ๆคœ็ดขใ€้กžไผผๅบฆใ€ใ‚ฏใƒฉใ‚นใ‚ฟใƒชใƒณใ‚ฐใ€ๅˆ†้กžใซ็‰นๅŒ–ใ—ใŸ Nomic ใฎๅŸ‹ใ‚่พผใฟใƒขใƒ‡ใƒซใ€‚", - "e5BaseV2": "ใƒ‡ใƒ•ใ‚ฉใƒซใƒˆใ‚ˆใ‚Šๅฐๅž‹ใง้ซ˜้€Ÿใชใƒขใƒ‡ใƒซใ€‚ๆคœ็ดขๅ“่ณชใฏไธ‹ใŒใ‚‹ใŒใ€ใƒ‡ใƒ•ใ‚ฉใƒซใƒˆใƒขใƒ‡ใƒซใฎ็ด„ 2 ๅ€้ซ˜้€Ÿใ€‚", + "e5BaseV2": "Microsoft ใฎ E5 ใƒ•ใ‚กใƒŸใƒชใƒผใฎ่‹ฑ่ชžๅŸ‹ใ‚่พผใฟใƒขใƒ‡ใƒซใ€‚้€Ÿๅบฆใจๆคœ็ดขๅ“่ณชใฎใƒใƒฉใƒณใ‚นใŒๅ–ใ‚Œใฆใ„ใพใ™ใ€‚", "e5SmallV2": "E5 ็ณปใƒขใƒ‡ใƒซใฎไธญใงๆœ€ๅฐใ‹ใคๆœ€้€Ÿใฎใƒใƒผใ‚ธใƒงใƒณใ€‚ใƒชใ‚ฝใƒผใ‚นใŒ้™ใ‚‰ใ‚ŒใŸ็’ฐๅขƒใง {appName} ใ‚’ๅฎŸ่กŒใ—ใฆใ„ใ‚‹ๅ ดๅˆใซ้ฉใ—ใฆใ„ใพใ™ใ€‚", "multilingualE5Base": "่‹ฑ่ชžไปฅๅค–ใฎ่จ€่ชžใฎใ‚ณใƒผใƒ‘ใ‚นใซใฏใ€ใ“ใ‚Œใ‚’้ธๆŠžใ—ใฆใใ ใ•ใ„ใ€‚", "multilingualE5Small": "่‹ฑ่ชžไปฅๅค–ใฎ่จ€่ชžใฎใ‚ณใƒผใƒ‘ใ‚นใงใ€ใ‹ใคใƒชใ‚ฝใƒผใ‚นใŒ้™ใ‚‰ใ‚ŒใŸ็’ฐๅขƒใฎๅ ดๅˆใฏใ€ใ“ใ‚Œใ‚’้ธๆŠžใ—ใฆใใ ใ•ใ„ใ€‚" diff --git a/web/src/i18n/messages/ko.json b/web/src/i18n/messages/ko.json index 506e31d535..4e0f265c14 100644 --- a/web/src/i18n/messages/ko.json +++ b/web/src/i18n/messages/ko.json @@ -6863,6 +6863,9 @@ "noModelsTooltip": "์„ค์ •๋œ ๋ชจ๋ธ์ด ์—†์–ด ์ปจํ…์ŠคํŠธ ๊ฒ€์ƒ‰์„ ์‚ฌ์šฉํ•  ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค. ๋จผ์ € [์–ธ์–ด ๋ชจ๋ธ]({link})์„ ์„ค์ •ํ•˜์„ธ์š”.", "title": "์ปจํ…์ŠคํŠธ ๊ฒ€์ƒ‰" }, + "currentModel": { + "legacyHint": "๋ ˆ๊ฑฐ์‹œ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. ๊ณ„์† ์ž‘๋™ํ•˜์ง€๋งŒ ์ƒˆ ์ธ๋ฑ์Šค์— ๋‹ค์‹œ ์„ ํƒํ•  ์ˆ˜๋Š” ์—†์Šต๋‹ˆ๋‹ค. ์—…๊ทธ๋ ˆ์ด๋“œํ•˜๋ ค๋ฉด โ€œ๋ชจ๋“  ๋ชจ๋ธ ๋ณด๊ธฐโ€๋ฅผ ํด๋ฆญํ•˜๊ณ  ์ตœ์‹  ๋ชจ๋ธ์„ ์„ ํƒํ•˜์„ธ์š”." + }, "embeddingModel": { "cloudManaged": { "title": "Embedding ๋ชจ๋ธ๊ณผ ์„ค์ •์€ Onyx Cloud์—์„œ ๊ด€๋ฆฌํ•ฉ๋‹ˆ๋‹ค." @@ -7001,9 +7004,12 @@ "currentButton": { "label": "ํ˜„์žฌ ๋ชจ๋ธ" }, - "deprecated": { - "connectTooltip": "์ด embedding ๋ชจ๋ธ์€ ์ง€์›์ด ์ค‘๋‹จ๋˜์–ด ์—ฐ๊ฒฐํ•  ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค.", - "selectTooltip": "์ด embedding ๋ชจ๋ธ์€ ์ง€์›์ด ์ค‘๋‹จ๋˜์–ด ์„ ํƒํ•  ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค." + "gpuRecommended": { + "tag": "GPU ๊ถŒ์žฅ" + }, + "legacy": { + "tag": "๋ ˆ๊ฑฐ์‹œ", + "tooltip": "์ด ๋ชจ๋ธ์€ ์ƒˆ ์ธ๋ฑ์Šค์— ์„ ํƒํ•  ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค. ๋ชจ๋ธ์„ ๋ณ€๊ฒฝํ•˜๊ธฐ ์ „๊นŒ์ง€๋Š” ๊ณ„์† ์ž‘๋™ํ•ฉ๋‹ˆ๋‹ค." }, "selectButton": { "label": "๋ชจ๋ธ ์„ ํƒ" @@ -7013,9 +7019,15 @@ } }, "modelPicker": { + "cpuEstimateHint": "CPU ์†๋„๋Š” ์ถ”์ •์น˜์ด๋ฉฐ ํ•˜๋“œ์›จ์–ด์™€ ๋ฌธ์„œ ํฌ๊ธฐ์— ๋”ฐ๋ผ ๋‹ฌ๋ผ์ง‘๋‹ˆ๋‹ค. ๊ถŒ์žฅ RAM์€ ๋กœ๋“œ๋œ ๋ชจ๋ธ ํ•˜๋‚˜์— ๋Œ€ํ•œ ๊ฐ’์ด๋ฉฐ ์ „์ฒด Onyx ๊ตฌ์„ฑ์˜ ์š”๊ตฌ๋Ÿ‰์ด ์•„๋‹™๋‹ˆ๋‹ค. ์žฌ์ธ๋ฑ์‹ฑ ์‹œ ๋ฉ”๋ชจ๋ฆฌ๋ฅผ ์ถ”๊ฐ€๋กœ ํ™•๋ณดํ•˜์„ธ์š”.", "addCustomModel": { "label": "์‚ฌ์šฉ์ž ์ง€์ • ๋ชจ๋ธ ์ถ”๊ฐ€" }, + "bringYourOwn": { + "cloudDescription": "LiteLLM ํ”„๋ก์‹œ, Azure OpenAI ๋ฐฐํฌ ๋˜๋Š” Bifrost ๊ฒŒ์ดํŠธ์›จ์ด๋ฅผ ์—ฐ๊ฒฐํ•ฉ๋‹ˆ๋‹ค. ๋ชจ๋ธ ์ด๋ฆ„์€ ์ง์ ‘ ์ž…๋ ฅํ•ฉ๋‹ˆ๋‹ค. LiteLLM๊ณผ Azure๋Š” ์ฐจ์›๋„ ์ž…๋ ฅํ•ฉ๋‹ˆ๋‹ค.", + "selfHostedDescription": "์œ„ ๋ชฉ๋ก์— ์—†๋Š” Hugging Face ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ์„ ์‹คํ–‰ํ•ฉ๋‹ˆ๋‹ค. ๋ชจ๋ธ ์ด๋ฆ„, ์ฐจ์›, ์ ‘๋‘์‚ฌ๋Š” ์ง์ ‘ ์ž…๋ ฅํ•ฉ๋‹ˆ๋‹ค.", + "title": "๊ณ ๊ธ‰: ์ž์ฒด ๋ชจ๋ธ ๋˜๋Š” ํ”„๋ก์‹œ ์‚ฌ์šฉ" + }, "cloudTab": { "label": "ํด๋ผ์šฐ๋“œ ๊ธฐ๋ฐ˜" }, @@ -7214,22 +7226,27 @@ "restoreHint": "์ปค๋„ฅํ„ฐ๋ฅผ ์œ ์ง€ํ•˜๋ ค๋ฉด ์—ฌ๊ธฐ์„œ ์ทจ์†Œํ•˜๊ณ  ์žฌ๊ฐœํ•˜๊ฑฐ๋‚˜ ๋ฌธ์ œ๋ฅผ ํ•ด๊ฒฐํ•œ ๋’ค ๋‹ค์‹œ ์žฌ์ธ๋ฑ์‹ฑํ•˜์„ธ์š”. ๊ทธ๋Ÿฌ๋ฉด ์ƒˆ ์ธ๋ฑ์Šค๋กœ ์ด์ „๋ฉ๋‹ˆ๋‹ค. ๊ทธ๋ ‡์ง€ ์•Š์œผ๋ฉด ์žฌ์ธ๋ฑ์‹ฑ์ด ๋๋‚œ ๋’ค ๋‹ค์‹œ ์ถ”๊ฐ€ํ•˜์„ธ์š”." }, "modelDescriptions": { + "cohereEmbedV5Pro": "Cohere์˜ ๋Œ€ํ‘œ ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. ํ’ˆ์งˆ๊ณผ ๋น„์šฉ์˜ ๊ท ํ˜•์ด ๊ฐ€์žฅ ์ข‹์Šต๋‹ˆ๋‹ค. 2048์ฐจ์› ์ถœ๋ ฅ.", + "cohereEmbedV5Fast": "Cohere Embed 5์˜ ์ €๋น„์šฉ ๋ฒ„์ „์ž…๋‹ˆ๋‹ค. ๊ทธ๋ž˜๋„ ๋งค์šฐ ํ›Œ๋ฅญํ•œ ์„ ํƒ์ž…๋‹ˆ๋‹ค. 2048์ฐจ์› ์ถœ๋ ฅ.", "cohereEmbedEnglishV3": "Cohere์˜ ์˜์–ด ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. ์˜์–ด ์ž‘์—…์— ์ข‹์€ ์„ฑ๋Šฅ์„ ๋ณด์ž…๋‹ˆ๋‹ค.", "cohereEmbedEnglishLightV3": "Cohere์˜ ๊ฒฝ๋Ÿ‰ ์˜์–ด ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. ๋‹จ์ˆœํ•œ ์ž‘์—…์— ๋” ๋น ๋ฅด๊ณ  ํšจ์œจ์ ์ž…๋‹ˆ๋‹ค.", - "cohereEmbedV4": "Cohere์˜ ์ตœ์‹  ๋‹ค๊ตญ์–ด ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ๋กœ, ๊ธฐ๋ณธ 1536์ฐจ์› ์ถœ๋ ฅ์œผ๋กœ ๋” ๋†’์€ ๊ฒ€์ƒ‰ ํ’ˆ์งˆ์„ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค.", - "openaiTextEmbedding3Large": "OpenAI์˜ ๋Œ€ํ˜• ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. ์„ฑ๋Šฅ์€ ๊ฐ€์žฅ ์ข‹์ง€๋งŒ ๋น„์šฉ์ด ๋” ๋†’์Šต๋‹ˆ๋‹ค.", - "openaiTextEmbedding3Small": "OpenAI์˜ ๋” ์ƒˆ๋กญ๊ณ  ํšจ์œจ์ ์ธ ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. ์„ฑ๋Šฅ๊ณผ ๋น„์šฉ์˜ ๊ท ํ˜•์ด ์ข‹์Šต๋‹ˆ๋‹ค.", + "cohereEmbedV4": "Cohere์˜ ๋‹ค๊ตญ์–ด ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ๋กœ, 1536์ฐจ์› ์ถœ๋ ฅ์„ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค.", + "openaiTextEmbedding3Large": "OpenAI์˜ ๋Œ€ํ˜• ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. ์ด๋ฏธ OpenAI๋ฅผ ์‚ฌ์šฉ ์ค‘์ด๋ผ๋ฉด ์ข‹์€ ์„ ํƒ์ž…๋‹ˆ๋‹ค. 3072์ฐจ์› ์ถœ๋ ฅ.", + "openaiTextEmbedding3Small": "OpenAI์˜ ์†Œํ˜• ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. ํด๋ผ์šฐ๋“œ ๊ธฐ๋ฐ˜ ์˜ต์…˜ ์ค‘ ๋‹จ์—ฐ ๊ฐ€์žฅ ๊ฒฝ์ œ์ ์ž…๋‹ˆ๋‹ค. 1536์ฐจ์› ์ถœ๋ ฅ.", "googleGeminiEmbedding001": "Google์˜ Gemini ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. ๊ฐ•๋ ฅํ•˜๊ณ  ํšจ์œจ์ ์ž…๋‹ˆ๋‹ค.", "googleTextEmbedding005": "Google์˜ ๋” ์ž‘๊ณ  ๊ฐ€๋ฒผ์šด ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค.", - "googleGeminiEmbedding2": "Google์˜ ์ตœ์‹  ๋ฉ€ํ‹ฐ๋ชจ๋‹ฌ ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ(GA)์ž…๋‹ˆ๋‹ค. 3072์ฐจ์› ์ถœ๋ ฅ์œผ๋กœ ์ตœ๊ณ  ํ’ˆ์งˆ์˜ ๊ฒ€์ƒ‰์„ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค.", + "googleGeminiEmbedding2": "Google์˜ ๋Œ€ํ‘œ ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. ๋งค์šฐ ๊ฐ•๋ ฅํ•˜์ง€๋งŒ ๋น„์šฉ์ด ๋†’๊ณ  ๋” ๋Š๋ฆฝ๋‹ˆ๋‹ค. 3072์ฐจ์› ์ถœ๋ ฅ์œผ๋กœ ์ธ๋ฑ์Šค๊ฐ€ ์ปค์ง‘๋‹ˆ๋‹ค. ์ด๋ฏธ Google์„ ์‚ฌ์šฉํ•˜๋Š” ํŒ€์—๊ฒŒ ํƒ„ํƒ„ํ•œ ์„ ํƒ์ž…๋‹ˆ๋‹ค.", "googleGeminiEmbedding2Preview": "Gemini Embedding 2์˜ ํ”„๋ฆฌ๋ทฐ ๋ฒ„์ „์ž…๋‹ˆ๋‹ค. ์ƒˆ ์„ค์ •์—์„œ๋Š” gemini-embedding-2๋กœ ๋Œ€์ฒด๋˜์—ˆ์Šต๋‹ˆ๋‹ค.", "voyageLarge2Instruct": "Voyage์˜ ๋Œ€ํ˜• ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. ์ง€์‹œ ๋ฏธ์„ธ ์กฐ์ •์œผ๋กœ ๋†’์€ ์„ฑ๋Šฅ์„ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค.", "voyage4Large": "Voyage 4 ์‹œ๋ฆฌ์ฆˆ์˜ ๊ฐ€์žฅ ํฐ ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. Voyage ์ตœ๊ณ ์˜ ๊ฒ€์ƒ‰ ํ’ˆ์งˆ๊ณผ ๋‹ค๊ตญ์–ด๋ฅผ ์ง€์›ํ•ฉ๋‹ˆ๋‹ค.", "voyage4": "Voyage 4 ์‹œ๋ฆฌ์ฆˆ์˜ ๋ฒ”์šฉ ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. ์ ๋‹นํ•œ ๋น„์šฉ์œผ๋กœ ๋†’์€ ๊ฒ€์ƒ‰ ํ’ˆ์งˆ์„ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค.", "voyage4Lite": "Voyage 4 ์‹œ๋ฆฌ์ฆˆ์˜ ๊ฒฝ๋Ÿ‰ ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. ๋น„์šฉ์ด ๊ฐ€์žฅ ๋‚ฎ๊ณ  ์†๋„ ์ œํ•œ์ด ๊ฐ€์žฅ ๋†’์Šต๋‹ˆ๋‹ค.", "voyageLight2Instruct": "Voyage์˜ ๊ฒฝ๋Ÿ‰ ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. ์„ฑ๋Šฅ๊ณผ ํšจ์œจ์˜ ๊ท ํ˜•์ด ์ข‹์Šต๋‹ˆ๋‹ค.", + "graniteEmbedding97mMultilingualR2": "IBM์˜ ์†Œํ˜• ๋‹ค๊ตญ์–ด ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. {appName}์— ํฌํ•จ๋ฉ๋‹ˆ๋‹ค. 9700๋งŒ ๊ฐœ ํŒŒ๋ผ๋ฏธํ„ฐ, 384์ฐจ์› ์ถœ๋ ฅ. CPU ์ฝ”์–ด๋‹น ๋ถ„๋‹น ์ตœ๋Œ€ 200๊ฐœ ๋ฌธ์„œ. CPU ๊ถŒ์žฅ RAM: ๋ชจ๋ธ ์„œ๋ฒ„๋‹น 4 GB.", + "voyage4Nano": "Voyage AI์˜ ์˜คํ”ˆ ์›จ์ดํŠธ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. 3์–ต 4600๋งŒ ๊ฐœ ํŒŒ๋ผ๋ฏธํ„ฐ, 2048์ฐจ์› ์ถœ๋ ฅ. CPU ์ฝ”์–ด๋‹น ๋ถ„๋‹น ์ตœ๋Œ€ 50๊ฐœ ๋ฌธ์„œ. CPU ๊ถŒ์žฅ RAM: ๋ชจ๋ธ ์„œ๋ฒ„๋‹น 4 GB.", + "nemotron3Embed1b": "NVIDIA์˜ ๋‹ค๊ตญ์–ด ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. 11์–ต ๊ฐœ ํŒŒ๋ผ๋ฏธํ„ฐ, 2048์ฐจ์› ์ถœ๋ ฅ. CPU ์ฝ”์–ด๋‹น ๋ถ„๋‹น ์ตœ๋Œ€ 5๊ฐœ ๋ฌธ์„œ. CPU ๊ถŒ์žฅ RAM: ๋ชจ๋ธ ์„œ๋ฒ„๋‹น 8 GB. GPU๋ฅผ ๊ถŒ์žฅํ•ฉ๋‹ˆ๋‹ค.", "nomicEmbedTextV1": "๊ฒ€์ƒ‰, ์œ ์‚ฌ๋„, ํด๋Ÿฌ์Šคํ„ฐ๋ง, ๋ถ„๋ฅ˜์— ํŠนํ™”๋œ Nomic์˜ ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค.", - "e5BaseV2": "๊ธฐ๋ณธ ๋ชจ๋ธ๋ณด๋‹ค ๋” ์ž‘๊ณ  ๋น ๋ฅธ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. ๊ฒ€์ƒ‰ ํ’ˆ์งˆ์€ ๋‚ฎ์•„์ง€์ง€๋งŒ ๊ธฐ๋ณธ ๋ชจ๋ธ๋ณด๋‹ค ์•ฝ 2๋ฐฐ ๋น ๋ฆ…๋‹ˆ๋‹ค.", + "e5BaseV2": "Microsoft E5 ๊ณ„์—ด์˜ ์˜์–ด ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. ์†๋„์™€ ๊ฒ€์ƒ‰ ํ’ˆ์งˆ์˜ ๊ท ํ˜•์„ ๊ฐ–์ถ”์—ˆ์Šต๋‹ˆ๋‹ค.", "e5SmallV2": "E5 ๋ชจ๋ธ ๊ณ„์—ด ์ค‘ ๊ฐ€์žฅ ์ž‘๊ณ  ๋น ๋ฅธ ๋ฒ„์ „์ž…๋‹ˆ๋‹ค. ๋ฆฌ์†Œ์Šค๊ฐ€ ์ œํ•œ๋œ ์‹œ์Šคํ…œ์—์„œ {appName}๋ฅผ ์‹คํ–‰ํ•œ๋‹ค๋ฉด ์ข‹์€ ์„ ํƒ์ž…๋‹ˆ๋‹ค.", "multilingualE5Base": "์˜์–ด ์ด์™ธ์˜ ์–ธ์–ด๋กœ ๋œ ์ฝ”ํผ์Šค์—๋Š” ์ด ๋ชจ๋ธ์„ ์„ ํƒํ•˜์„ธ์š”.", "multilingualE5Small": "์˜์–ด ์ด์™ธ์˜ ์–ธ์–ด๋กœ ๋œ ์ฝ”ํผ์Šค์ด๋ฉด์„œ ๋ฆฌ์†Œ์Šค๊ฐ€ ์ œํ•œ๋œ ์‹œ์Šคํ…œ์ด๋ผ๋ฉด ์ด ๋ชจ๋ธ์„ ์„ ํƒํ•˜์„ธ์š”." diff --git a/web/src/i18n/messages/pt.json b/web/src/i18n/messages/pt.json index e1518af868..dc78dbeb93 100644 --- a/web/src/i18n/messages/pt.json +++ b/web/src/i18n/messages/pt.json @@ -6863,6 +6863,9 @@ "noModelsTooltip": "A recuperaรงรฃo contextual estรก desativada porque vocรช nรฃo tem modelos configurados. Configure primeiro um [modelo de linguagem]({link}).", "title": "Recuperaรงรฃo contextual" }, + "currentModel": { + "legacyHint": "Este รฉ um modelo legado. Ele continua funcionando, mas vocรช nรฃo pode selecionรก-lo de novo para um novo รญndice. Para atualizar, clique em โ€œVer todos os modelosโ€ e selecione um modelo mais recente." + }, "embeddingModel": { "cloudManaged": { "title": "O modelo de embedding e as configuraรงรตes sรฃo gerenciados pelo Onyx Cloud." @@ -7001,9 +7004,12 @@ "currentButton": { "label": "Modelo atual" }, - "deprecated": { - "connectTooltip": "Este modelo de embedding estรก descontinuado e nรฃo รฉ possรญvel se conectar a ele.", - "selectTooltip": "Este modelo de embedding estรก descontinuado e nรฃo pode ser selecionado." + "gpuRecommended": { + "tag": "GPU recomendada" + }, + "legacy": { + "tag": "Legado", + "tooltip": "Vocรช nรฃo pode selecionar este modelo para um novo รญndice. Ele continua funcionando atรฉ vocรช trocar de modelo." }, "selectButton": { "label": "Selecionar modelo" @@ -7013,9 +7019,15 @@ } }, "modelPicker": { + "cpuEstimateHint": "As velocidades de CPU sรฃo estimativas e variam conforme o hardware e o tamanho dos documentos. A RAM recomendada cobre um modelo carregado, nรฃo toda a infraestrutura do Onyx. Reserve mais memรณria durante a reindexaรงรฃo.", "addCustomModel": { "label": "Adicionar modelo personalizado" }, + "bringYourOwn": { + "cloudDescription": "Conecte um proxy LiteLLM, uma implantaรงรฃo do Azure OpenAI ou um gateway Bifrost. Vocรช informa o nome do modelo. Para LiteLLM e Azure, vocรช tambรฉm informa a dimensรฃo.", + "selfHostedDescription": "Execute um modelo de embeddings do Hugging Face que nรฃo estรก na lista acima. Vocรช informa o nome, a dimensรฃo e os prefixos do modelo.", + "title": "Avanรงado: use seu prรณprio modelo ou proxy" + }, "cloudTab": { "label": "Na nuvem" }, @@ -7214,22 +7226,27 @@ "restoreHint": "Para manter um conector, cancele aqui, retome-o ou corrija-o e reindexe novamente โ€” assim ele serรก levado adiante. Caso contrรกrio, adicione-o de volta quando a reindexaรงรฃo terminar." }, "modelDescriptions": { + "cohereEmbedV5Pro": "O principal modelo de embeddings da Cohere. O melhor equilรญbrio entre qualidade e custo. Saรญda de 2048 dimensรตes.", + "cohereEmbedV5Fast": "Versรฃo de menor custo do Embed 5 da Cohere. Ainda assim, uma excelente opรงรฃo. Saรญda de 2048 dimensรตes.", "cohereEmbedEnglishV3": "Modelo de embeddings em inglรชs da Cohere. Bom desempenho em tarefas em inglรชs.", "cohereEmbedEnglishLightV3": "Modelo leve de embeddings em inglรชs da Cohere. Mais rรกpido e eficiente para tarefas simples.", - "cohereEmbedV4": "O modelo de embeddings multilรญngue mais recente da Cohere, com saรญda padrรฃo de 1536 dimensรตes para melhor qualidade de recuperaรงรฃo.", - "openaiTextEmbedding3Large": "Modelo de embeddings grande da OpenAI. Melhor desempenho, porรฉm mais caro.", - "openaiTextEmbedding3Small": "Modelo de embeddings mais novo e eficiente da OpenAI. Bom equilรญbrio entre desempenho e custo.", + "cohereEmbedV4": "O modelo de embeddings multilรญngue da Cohere, com saรญda de 1536 dimensรตes.", + "openaiTextEmbedding3Large": "Modelo de embeddings grande da OpenAI. Uma boa escolha se vocรช jรก usa a OpenAI. Saรญda de 3072 dimensรตes.", + "openaiTextEmbedding3Small": "Modelo de embeddings pequeno da OpenAI. De longe a opรงรฃo na nuvem mais econรดmica. Saรญda de 1536 dimensรตes.", "googleGeminiEmbedding001": "Modelo de embeddings Gemini do Google. Poderoso e eficiente.", "googleTextEmbedding005": "Modelo de embeddings menor e mais leve do Google.", - "googleGeminiEmbedding2": "O modelo de embeddings multimodal mais recente do Google (GA). Recuperaรงรฃo da mais alta qualidade com saรญda de 3072 dimensรตes.", + "googleGeminiEmbedding2": "O principal modelo de embeddings do Google. Muito poderoso, mas caro e mais lento. Sua saรญda de 3072 dimensรตes gera um รญndice grande. Uma escolha sรณlida para equipes que jรก usam o Google.", "googleGeminiEmbedding2Preview": "Versรฃo prรฉvia do Gemini Embedding 2. Substituรญda pelo gemini-embedding-2 em novas configuraรงรตes.", "voyageLarge2Instruct": "Modelo de embeddings grande da Voyage. Alto desempenho com ajuste fino por instruรงรตes.", "voyage4Large": "O maior modelo de embeddings da sรฉrie 4 da Voyage. A melhor qualidade de recuperaรงรฃo da Voyage, multilรญngue.", "voyage4": "Modelo de embeddings de uso geral da sรฉrie 4 da Voyage. Alta qualidade de recuperaรงรฃo a um custo moderado.", "voyage4Lite": "Modelo leve de embeddings da sรฉrie 4 da Voyage. Menor custo e maiores limites de taxa.", "voyageLight2Instruct": "Modelo leve de embeddings da Voyage. Bom equilรญbrio entre desempenho e eficiรชncia.", + "graniteEmbedding97mMultilingualR2": "O pequeno modelo multilรญngue da IBM. Incluรญdo no {appName}. 97 milhรตes de parรขmetros, saรญda de 384 dimensรตes. Atรฉ 200 documentos / min / nรบcleo de CPU. RAM recomendada para CPU: 4 GB por servidor de modelos.", + "voyage4Nano": "O modelo de pesos abertos da Voyage AI. 346 milhรตes de parรขmetros, saรญda de 2048 dimensรตes. Atรฉ 50 documentos / min / nรบcleo de CPU. RAM recomendada para CPU: 4 GB por servidor de modelos.", + "nemotron3Embed1b": "O modelo multilรญngue da NVIDIA. 1,1 bilhรฃo de parรขmetros, saรญda de 2048 dimensรตes. Atรฉ 5 documentos / min / nรบcleo de CPU. RAM recomendada para CPU: 8 GB por servidor de modelos. GPU recomendada.", "nomicEmbedTextV1": "Modelo de embeddings da Nomic especializado em recuperaรงรฃo, similaridade, agrupamento e classificaรงรฃo.", - "e5BaseV2": "Um modelo menor e mais rรกpido que o padrรฃo. ร‰ cerca de 2x mais rรกpido que o modelo padrรฃo, com menor qualidade de busca.", + "e5BaseV2": "Modelo de embeddings em inglรชs da famรญlia E5 da Microsoft. Equilibra velocidade e qualidade de busca.", "e5SmallV2": "A versรฃo menor e mais rรกpida da linha de modelos E5. Se vocรช executa o {appName} em um sistema com recursos limitados, pode ser uma boa escolha.", "multilingualE5Base": "Para corpora em outros idiomas alรฉm do inglรชs, esta รฉ a escolha certa.", "multilingualE5Small": "Para corpora em outros idiomas alรฉm do inglรชs em um sistema com recursos limitados, esta รฉ a escolha certa." diff --git a/web/src/i18n/messages/zh.json b/web/src/i18n/messages/zh.json index f2e211d392..1d961f8a50 100644 --- a/web/src/i18n/messages/zh.json +++ b/web/src/i18n/messages/zh.json @@ -6863,6 +6863,9 @@ "noModelsTooltip": "ไธŠไธ‹ๆ–‡ๆฃ€็ดขๅทฒ็ฆ็”จ๏ผŒๅ› ไธบๆ‚จๅฐšๆœช้…็ฝฎไปปไฝ•ๆจกๅž‹ใ€‚่ฏทๅ…ˆ่ฎพ็ฝฎ[่ฏญ่จ€ๆจกๅž‹]({link})ใ€‚", "title": "ไธŠไธ‹ๆ–‡ๆฃ€็ดข" }, + "currentModel": { + "legacyHint": "่ฟ™ๆ˜ฏๆ—ง็‰ˆๆจกๅž‹ใ€‚ๅฎƒไผš็ปง็ปญๅทฅไฝœ๏ผŒไฝ†ไธ่ƒฝๅ†ๆฌก้€‰ๆ‹ฉ็”จไบŽๆ–ฐ็ดขๅผ•ใ€‚ๅฆ‚้œ€ๅ‡็บง๏ผŒ่ฏท็‚นๅ‡ปโ€œๆŸฅ็œ‹ๆ‰€ๆœ‰ๆจกๅž‹โ€ๅนถ้€‰ๆ‹ฉ่พƒๆ–ฐ็š„ๆจกๅž‹ใ€‚" + }, "embeddingModel": { "cloudManaged": { "title": "Embedding ๆจกๅž‹ๅ’Œ็›ธๅ…ณ่ฎพ็ฝฎ็”ฑ Onyx Cloud ็ฎก็†ใ€‚" @@ -7001,9 +7004,12 @@ "currentButton": { "label": "ๅฝ“ๅ‰ๆจกๅž‹" }, - "deprecated": { - "connectTooltip": "ๆญค embedding ๆจกๅž‹ๅทฒๅผƒ็”จ๏ผŒๆ— ๆณ•่ฟžๆŽฅใ€‚", - "selectTooltip": "ๆญค embedding ๆจกๅž‹ๅทฒๅผƒ็”จ๏ผŒๆ— ๆณ•้€‰ๆ‹ฉใ€‚" + "gpuRecommended": { + "tag": "ๅปบ่ฎฎไฝฟ็”จ GPU" + }, + "legacy": { + "tag": "ๆ—ง็‰ˆ", + "tooltip": "ๆญคๆจกๅž‹ไธ่ƒฝ็”จไบŽๆ–ฐ็ดขๅผ•ใ€‚ๅœจๆ‚จๆ›ดๆขๆจกๅž‹ไน‹ๅ‰๏ผŒๅฎƒไผš็ปง็ปญๅทฅไฝœใ€‚" }, "selectButton": { "label": "้€‰ๆ‹ฉๆจกๅž‹" @@ -7013,9 +7019,15 @@ } }, "modelPicker": { + "cpuEstimateHint": "CPU ้€Ÿๅบฆไธบไผฐ่ฎกๅ€ผ๏ผŒไผšๅ› ็กฌไปถๅ’Œๆ–‡ๆกฃๅคงๅฐ่€Œๅผ‚ใ€‚RAM ๅปบ่ฎฎ้€‚็”จไบŽไธ€ไธชๅทฒๅŠ ่ฝฝๆจกๅž‹๏ผŒๅนถ้žๆ•ดไธช Onyx ็ณป็ปŸ็š„ๅ†…ๅญ˜่ฆๆฑ‚ใ€‚้‡ๆ–ฐ็ดขๅผ•ๆ—ถ่ฏท้ข„็•™ๆ›ดๅคšๅ†…ๅญ˜ใ€‚", "addCustomModel": { "label": "ๆทปๅŠ ่‡ชๅฎšไน‰ๆจกๅž‹" }, + "bringYourOwn": { + "cloudDescription": "่ฟžๆŽฅ LiteLLM ไปฃ็†ใ€Azure OpenAI ้ƒจ็ฝฒๆˆ– Bifrost ็ฝ‘ๅ…ณใ€‚ๆจกๅž‹ๅ็งฐ็”ฑๆ‚จ่‡ช่กŒๅกซๅ†™ใ€‚ๅฏนไบŽ LiteLLM ๅ’Œ Azure๏ผŒ่ฟ˜้œ€ๅกซๅ†™็ปดๅบฆใ€‚", + "selfHostedDescription": "่ฟ่กŒไธŠๆ–นๅˆ—่กจไธญๆฒกๆœ‰็š„ Hugging Face ๅตŒๅ…ฅๆจกๅž‹ใ€‚ๆจกๅž‹ๅ็งฐใ€็ปดๅบฆๅ’Œๅ‰็ผ€็”ฑๆ‚จ่‡ช่กŒๅกซๅ†™ใ€‚", + "title": "้ซ˜็บง๏ผšไฝฟ็”จ่‡ชๅทฑ็š„ๆจกๅž‹ๆˆ–ไปฃ็†" + }, "cloudTab": { "label": "ไบ‘็ซฏ" }, @@ -7214,22 +7226,27 @@ "restoreHint": "ๅฆ‚้œ€ไฟ็•™ๆŸไธช่ฟžๆŽฅๅ™จ๏ผŒ่ฏทๅœจๆญคๅ–ๆถˆ๏ผŒๅ…ˆๆขๅคๆˆ–ไฟฎๅคๅฎƒ๏ผŒ็„ถๅŽ้‡ๆ–ฐ็ดขๅผ•๏ผŒๅฎƒๅฐฑไผš่ขซ่ฟ็งป่ฟ‡ๅŽปใ€‚ๅฆๅˆ™่ฏทๅœจ้‡ๆ–ฐ็ดขๅผ•ๅฎŒๆˆๅŽ้‡ๆ–ฐๆทปๅŠ ใ€‚" }, "modelDescriptions": { + "cohereEmbedV5Pro": "Cohere ็š„ๆ——่ˆฐๅตŒๅ…ฅๆจกๅž‹ใ€‚่ดจ้‡ไธŽๆˆๆœฌ็š„ๆœ€ไฝณๅนณ่กกใ€‚2048 ็ปด่พ“ๅ‡บใ€‚", + "cohereEmbedV5Fast": "Cohere Embed 5 ็š„ไฝŽๆˆๆœฌ็‰ˆๆœฌใ€‚ไพ็„ถๆ˜ฏ้žๅธธๅ‡บ่‰ฒ็š„้€‰ๆ‹ฉใ€‚2048 ็ปด่พ“ๅ‡บใ€‚", "cohereEmbedEnglishV3": "Cohere ็š„่‹ฑ่ฏญๅตŒๅ…ฅๆจกๅž‹ใ€‚ๅœจ่‹ฑ่ฏญไปปๅŠกไธŠ่กจ็Žฐ่‰ฏๅฅฝใ€‚", "cohereEmbedEnglishLightV3": "Cohere ็š„่ฝป้‡็บง่‹ฑ่ฏญๅตŒๅ…ฅๆจกๅž‹ใ€‚ๅค„็†็ฎ€ๅ•ไปปๅŠกๆ›ดๅฟซใ€ๆ›ด้ซ˜ๆ•ˆใ€‚", - "cohereEmbedV4": "Cohere ๆœ€ๆ–ฐ็š„ๅคš่ฏญ่จ€ๅตŒๅ…ฅๆจกๅž‹๏ผŒ้ป˜่ฎค 1536 ็ปด่พ“ๅ‡บ๏ผŒๆฃ€็ดข่ดจ้‡ๆ›ด้ซ˜ใ€‚", - "openaiTextEmbedding3Large": "OpenAI ็š„ๅคงๅž‹ๅตŒๅ…ฅๆจกๅž‹ใ€‚ๆ€ง่ƒฝๆœ€ไฝณ๏ผŒไฝ†ๆˆๆœฌๆ›ด้ซ˜ใ€‚", - "openaiTextEmbedding3Small": "OpenAI ๆ›ดๆ–ฐใ€ๆ›ด้ซ˜ๆ•ˆ็š„ๅตŒๅ…ฅๆจกๅž‹ใ€‚ๆ€ง่ƒฝไธŽๆˆๆœฌๅนณ่กก่‰ฏๅฅฝใ€‚", + "cohereEmbedV4": "Cohere ็š„ๅคš่ฏญ่จ€ๅตŒๅ…ฅๆจกๅž‹๏ผŒ1536 ็ปด่พ“ๅ‡บใ€‚", + "openaiTextEmbedding3Large": "OpenAI ็š„ๅคงๅž‹ๅตŒๅ…ฅๆจกๅž‹ใ€‚้€‚ๅˆๅทฒๅœจไฝฟ็”จ OpenAI ็š„ๅ›ข้˜Ÿใ€‚3072 ็ปด่พ“ๅ‡บใ€‚", + "openaiTextEmbedding3Small": "OpenAI ็š„ๅฐๅž‹ๅตŒๅ…ฅๆจกๅž‹ใ€‚ๅœจไบ‘็ซฏ้€‰้กนไธญๆœ€็ปๆตŽ๏ผŒไผ˜ๅŠฟๆ˜Žๆ˜พใ€‚1536 ็ปด่พ“ๅ‡บใ€‚", "googleGeminiEmbedding001": "Google ็š„ Gemini ๅตŒๅ…ฅๆจกๅž‹ใ€‚ๅผบๅคงไธ”้ซ˜ๆ•ˆใ€‚", "googleTextEmbedding005": "Google ๆ›ดๅฐใ€ๆ›ด่ฝป้‡็š„ๅตŒๅ…ฅๆจกๅž‹ใ€‚", - "googleGeminiEmbedding2": "Google ๆœ€ๆ–ฐ็š„ๅคšๆจกๆ€ๅตŒๅ…ฅๆจกๅž‹๏ผˆGA๏ผ‰ใ€‚3072 ็ปด่พ“ๅ‡บ๏ผŒๆฃ€็ดข่ดจ้‡ๆœ€้ซ˜ใ€‚", + "googleGeminiEmbedding2": "Google ็š„ๆ——่ˆฐๅตŒๅ…ฅๆจกๅž‹ใ€‚ๅŠŸ่ƒฝ้žๅธธๅผบๅคง๏ผŒไฝ†ๆˆๆœฌ้ซ˜ใ€้€Ÿๅบฆ่พƒๆ…ขใ€‚3072 ็ปด่พ“ๅ‡บไผš็”Ÿๆˆ่พƒๅคง็š„็ดขๅผ•ใ€‚ๅฏนๅทฒๅœจไฝฟ็”จ Google ็š„ๅ›ข้˜Ÿๆฅ่ฏดๆ˜ฏ็จณๅฆฅ็š„้€‰ๆ‹ฉใ€‚", "googleGeminiEmbedding2Preview": "Gemini Embedding 2 ็š„้ข„่งˆ็‰ˆใ€‚ๆ–ฐ้ƒจ็ฝฒๅทฒ็”ฑ gemini-embedding-2 ๅ–ไปฃใ€‚", "voyageLarge2Instruct": "Voyage ็š„ๅคงๅž‹ๅตŒๅ…ฅๆจกๅž‹ใ€‚็ป่ฟ‡ๆŒ‡ไปคๅพฎ่ฐƒ๏ผŒๆ€ง่ƒฝๅ‡บ่‰ฒใ€‚", "voyage4Large": "Voyage 4 ็ณปๅˆ—ไธญๆœ€ๅคง็š„ๅตŒๅ…ฅๆจกๅž‹ใ€‚Voyage ๆœ€ไฝณๆฃ€็ดข่ดจ้‡๏ผŒๆ”ฏๆŒๅคš่ฏญ่จ€ใ€‚", "voyage4": "Voyage 4 ็ณปๅˆ—้€š็”จๅตŒๅ…ฅๆจกๅž‹ใ€‚ไปฅ้€‚ไธญๆˆๆœฌๆไพ›้ซ˜ๆฃ€็ดข่ดจ้‡ใ€‚", "voyage4Lite": "Voyage 4 ็ณปๅˆ—่ฝป้‡็บงๅตŒๅ…ฅๆจกๅž‹ใ€‚ๆˆๆœฌๆœ€ไฝŽ๏ผŒ้€Ÿ็Ž‡้™ๅˆถๆœ€้ซ˜ใ€‚", "voyageLight2Instruct": "Voyage ็š„่ฝป้‡็บงๅตŒๅ…ฅๆจกๅž‹ใ€‚ๆ€ง่ƒฝไธŽๆ•ˆ็Ž‡ๅนณ่กก่‰ฏๅฅฝใ€‚", + "graniteEmbedding97mMultilingualR2": "IBM ็š„ๅฐๅž‹ๅคš่ฏญ่จ€ๆจกๅž‹ใ€‚้š {appName} ๆไพ›ใ€‚9700 ไธ‡ๅ‚ๆ•ฐ๏ผŒ384 ็ปด่พ“ๅ‡บใ€‚ๆฏไธช CPU ๆ ธๅฟƒๆฏๅˆ†้’Ÿๆœ€ๅคšๅค„็† 200 ็ฏ‡ๆ–‡ๆกฃใ€‚CPU ๆŽจ็†ๅปบ่ฎฎ RAM๏ผšๆฏไธชๆจกๅž‹ๆœๅŠกๅ™จ 4 GBใ€‚", + "voyage4Nano": "Voyage AI ็š„ๅผ€ๆ”พๆƒ้‡ๆจกๅž‹ใ€‚3.46 ไบฟๅ‚ๆ•ฐ๏ผŒ2048 ็ปด่พ“ๅ‡บใ€‚ๆฏไธช CPU ๆ ธๅฟƒๆฏๅˆ†้’Ÿๆœ€ๅคšๅค„็† 50 ็ฏ‡ๆ–‡ๆกฃใ€‚CPU ๆŽจ็†ๅปบ่ฎฎ RAM๏ผšๆฏไธชๆจกๅž‹ๆœๅŠกๅ™จ 4 GBใ€‚", + "nemotron3Embed1b": "NVIDIA ็š„ๅคš่ฏญ่จ€ๆจกๅž‹ใ€‚11 ไบฟๅ‚ๆ•ฐ๏ผŒ2048 ็ปด่พ“ๅ‡บใ€‚ๆฏไธช CPU ๆ ธๅฟƒๆฏๅˆ†้’Ÿๆœ€ๅคšๅค„็† 5 ็ฏ‡ๆ–‡ๆกฃใ€‚CPU ๆŽจ็†ๅปบ่ฎฎ RAM๏ผšๆฏไธชๆจกๅž‹ๆœๅŠกๅ™จ 8 GBใ€‚ๅปบ่ฎฎไฝฟ็”จ GPUใ€‚", "nomicEmbedTextV1": "Nomic ็š„ๅตŒๅ…ฅๆจกๅž‹๏ผŒไธ“ๆณจไบŽๆฃ€็ดขใ€็›ธไผผๅบฆใ€่š็ฑปๅ’Œๅˆ†็ฑปใ€‚", - "e5BaseV2": "ๆฏ”้ป˜่ฎคๆจกๅž‹ๆ›ดๅฐใ€ๆ›ดๅฟซ็š„ๆจกๅž‹ใ€‚้€Ÿๅบฆ็บฆไธบ้ป˜่ฎคๆจกๅž‹็š„ 2 ๅ€๏ผŒไฝ†ๆœ็ดข่ดจ้‡่พƒไฝŽใ€‚", + "e5BaseV2": "Microsoft E5 ็ณปๅˆ—็š„่‹ฑๆ–‡ๅตŒๅ…ฅๆจกๅž‹ใ€‚ๅœจ้€ŸๅบฆไธŽๆœ็ดข่ดจ้‡ไน‹้—ดๅ–ๅพ—ๅนณ่กกใ€‚", "e5SmallV2": "E5 ็ณปๅˆ—ๆจกๅž‹ไธญๆœ€ๅฐใ€ๆœ€ๅฟซ็š„็‰ˆๆœฌใ€‚ๅฆ‚ๆžœๆ‚จๅœจ่ต„ๆบๅ—้™็š„็ณป็ปŸไธŠ่ฟ่กŒ {appName}๏ผŒ่ฟ™ๆ˜ฏไธ€ไธชไธ้”™็š„้€‰ๆ‹ฉใ€‚", "multilingualE5Base": "ๅฆ‚ๆžœ่ฏญๆ–™ๅบ“ๅŒ…ๅซ่‹ฑ่ฏญไปฅๅค–็š„่ฏญ่จ€๏ผŒ่ฏท้€‰ๆ‹ฉๆญคๆจกๅž‹ใ€‚", "multilingualE5Small": "ๅฆ‚ๆžœ่ฏญๆ–™ๅบ“ๅŒ…ๅซ่‹ฑ่ฏญไปฅๅค–็š„่ฏญ่จ€๏ผŒไธ”็ณป็ปŸ่ต„ๆบๅ—้™๏ผŒ่ฏท้€‰ๆ‹ฉๆญคๆจกๅž‹ใ€‚" diff --git a/web/src/lib/searchSettings/constants.ts b/web/src/lib/searchSettings/constants.ts index 549e591b02..73be24d0a4 100644 --- a/web/src/lib/searchSettings/constants.ts +++ b/web/src/lib/searchSettings/constants.ts @@ -1,12 +1,14 @@ -import { SvgHardDrive } from "@opal/icons"; +import { SvgHardDrive, SvgServer } from "@opal/icons"; import { SvgAzure, SvgBifrost, SvgCohere, SvgGoogle, + SvgIBM, SvgLiteLLM, SvgMicrosoft, SvgNomic, + SvgNVIDIA, SvgOpenAI, SvgVoyage, } from "@opal/logos"; @@ -18,6 +20,11 @@ import { DOCS_ADMINS_PATH } from "@/lib/constants"; // โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ• // Embedding +// +// Mirrors backend/shared_configs/embedding_models.py, and +// test_web_embedding_model_registry_parity.py pins the two together. Never +// remove or rename an entry: mark it `legacy`. List selectable models first in +// each group. // โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ• export const CLOUD_BASED_PROVIDERS: EmbeddingProvider[] = [ @@ -29,6 +36,22 @@ export const CLOUD_BASED_PROVIDERS: EmbeddingProvider[] = [ apiLink: "https://dashboard.cohere.ai/api-keys", costslink: "https://cohere.com/pricing", embeddingModels: [ + { + modelName: "embed-v5.0-pro", + modelDim: 2048, + normalize: false, + queryPrefix: "", + passagePrefix: "", + descriptionKey: "modelDescriptions.cohereEmbedV5Pro", + }, + { + modelName: "embed-v5.0-fast", + modelDim: 2048, + normalize: false, + queryPrefix: "", + passagePrefix: "", + descriptionKey: "modelDescriptions.cohereEmbedV5Fast", + }, { modelName: "embed-english-v3.0", modelDim: 1024, @@ -36,6 +59,7 @@ export const CLOUD_BASED_PROVIDERS: EmbeddingProvider[] = [ queryPrefix: "", passagePrefix: "", descriptionKey: "modelDescriptions.cohereEmbedEnglishV3", + legacy: true, }, { modelName: "embed-english-light-v3.0", @@ -44,6 +68,7 @@ export const CLOUD_BASED_PROVIDERS: EmbeddingProvider[] = [ queryPrefix: "", passagePrefix: "", descriptionKey: "modelDescriptions.cohereEmbedEnglishLightV3", + legacy: true, }, { modelName: "embed-v4.0", @@ -52,6 +77,7 @@ export const CLOUD_BASED_PROVIDERS: EmbeddingProvider[] = [ queryPrefix: "", passagePrefix: "", descriptionKey: "modelDescriptions.cohereEmbedV4", + legacy: true, }, ], }, @@ -89,22 +115,6 @@ export const CLOUD_BASED_PROVIDERS: EmbeddingProvider[] = [ apiLink: "https://console.cloud.google.com/apis/credentials", costslink: "https://cloud.google.com/vertex-ai/pricing", embeddingModels: [ - { - modelName: "gemini-embedding-001", - modelDim: 3072, - normalize: false, - queryPrefix: "", - passagePrefix: "", - descriptionKey: "modelDescriptions.googleGeminiEmbedding001", - }, - { - modelName: "text-embedding-005", - modelDim: 768, - normalize: false, - queryPrefix: "", - passagePrefix: "", - descriptionKey: "modelDescriptions.googleTextEmbedding005", - }, { modelName: "gemini-embedding-2", modelDim: 3072, @@ -113,6 +123,24 @@ export const CLOUD_BASED_PROVIDERS: EmbeddingProvider[] = [ passagePrefix: "", descriptionKey: "modelDescriptions.googleGeminiEmbedding2", }, + { + modelName: "gemini-embedding-001", + modelDim: 3072, + normalize: false, + queryPrefix: "", + passagePrefix: "", + descriptionKey: "modelDescriptions.googleGeminiEmbedding001", + legacy: true, + }, + { + modelName: "text-embedding-005", + modelDim: 768, + normalize: false, + queryPrefix: "", + passagePrefix: "", + descriptionKey: "modelDescriptions.googleTextEmbedding005", + legacy: true, + }, { modelName: "gemini-embedding-2-preview", modelDim: 3072, @@ -120,6 +148,23 @@ export const CLOUD_BASED_PROVIDERS: EmbeddingProvider[] = [ queryPrefix: "", passagePrefix: "", descriptionKey: "modelDescriptions.googleGeminiEmbedding2Preview", + legacy: true, + }, + { + modelName: "text-embedding-004", + modelDim: 768, + normalize: false, + queryPrefix: "", + passagePrefix: "", + legacy: true, + }, + { + modelName: "textembedding-gecko@003", + modelDim: 768, + normalize: false, + queryPrefix: "", + passagePrefix: "", + legacy: true, }, ], }, @@ -162,7 +207,7 @@ export const CLOUD_BASED_PROVIDERS: EmbeddingProvider[] = [ queryPrefix: "", passagePrefix: "", descriptionKey: "modelDescriptions.voyageLarge2Instruct", - deprecated: true, + legacy: true, }, { modelName: "voyage-light-2-instruct", @@ -171,10 +216,12 @@ export const CLOUD_BASED_PROVIDERS: EmbeddingProvider[] = [ queryPrefix: "", passagePrefix: "", descriptionKey: "modelDescriptions.voyageLight2Instruct", - deprecated: true, + legacy: true, }, ], }, + // Bring-your-own providers: no registry models. The admin enters the model + // spec in the connect modal. See `isBringYourOwnProvider`. { providerName: EmbeddingProviderName.LITELLM, displayName: "LiteLLM", @@ -203,6 +250,56 @@ export const CLOUD_BASED_PROVIDERS: EmbeddingProvider[] = [ ]; export const SELF_HOSTED_PROVIDERS: EmbeddingProvider[] = [ + { + providerName: EmbeddingProviderName.IBM, + displayName: "IBM", + icon: SvgIBM, + docsLink: "https://huggingface.co/ibm-granite", + embeddingModels: [ + { + modelName: "ibm-granite/granite-embedding-97m-multilingual-r2", + modelDim: 384, + normalize: true, + queryPrefix: "", + passagePrefix: "", + descriptionKey: "modelDescriptions.graniteEmbedding97mMultilingualR2", + }, + ], + }, + { + providerName: EmbeddingProviderName.VOYAGE_SELF_HOSTED, + displayName: "Voyage AI", + icon: SvgVoyage, + docsLink: "https://huggingface.co/voyageai", + embeddingModels: [ + { + modelName: "voyageai/voyage-4-nano", + modelDim: 2048, + normalize: true, + queryPrefix: + "Represent the query for retrieving supporting documents: ", + passagePrefix: "Represent the document for retrieval: ", + descriptionKey: "modelDescriptions.voyage4Nano", + }, + ], + }, + { + providerName: EmbeddingProviderName.NVIDIA, + displayName: "NVIDIA", + icon: SvgNVIDIA, + docsLink: "https://huggingface.co/nvidia", + embeddingModels: [ + { + modelName: "nvidia/Nemotron-3-Embed-1B-BF16", + modelDim: 2048, + normalize: true, + queryPrefix: "query: ", + passagePrefix: "passage: ", + descriptionKey: "modelDescriptions.nemotron3Embed1b", + gpuRecommended: true, + }, + ], + }, { providerName: EmbeddingProviderName.NOMIC, displayName: "Nomic", @@ -216,6 +313,7 @@ export const SELF_HOSTED_PROVIDERS: EmbeddingProvider[] = [ queryPrefix: "search_query: ", passagePrefix: "search_document: ", descriptionKey: "modelDescriptions.nomicEmbedTextV1", + legacy: true, }, ], }, @@ -232,6 +330,7 @@ export const SELF_HOSTED_PROVIDERS: EmbeddingProvider[] = [ queryPrefix: "query: ", passagePrefix: "passage: ", descriptionKey: "modelDescriptions.e5BaseV2", + legacy: true, }, { modelName: "intfloat/e5-small-v2", @@ -240,6 +339,7 @@ export const SELF_HOSTED_PROVIDERS: EmbeddingProvider[] = [ queryPrefix: "query: ", passagePrefix: "passage: ", descriptionKey: "modelDescriptions.e5SmallV2", + legacy: true, }, { modelName: "intfloat/multilingual-e5-base", @@ -248,6 +348,7 @@ export const SELF_HOSTED_PROVIDERS: EmbeddingProvider[] = [ queryPrefix: "query: ", passagePrefix: "passage: ", descriptionKey: "modelDescriptions.multilingualE5Base", + legacy: true, }, { modelName: "intfloat/multilingual-e5-small", @@ -256,6 +357,23 @@ export const SELF_HOSTED_PROVIDERS: EmbeddingProvider[] = [ queryPrefix: "query: ", passagePrefix: "passage: ", descriptionKey: "modelDescriptions.multilingualE5Small", + legacy: true, + }, + ], + }, + { + providerName: EmbeddingProviderName.GTE, + displayName: "GTE", + icon: SvgServer, + docsLink: "https://huggingface.co/thenlper", + embeddingModels: [ + { + modelName: "thenlper/gte-small", + modelDim: 384, + normalize: false, + queryPrefix: "", + passagePrefix: "", + legacy: true, }, ], }, diff --git a/web/src/lib/searchSettings/index.test.ts b/web/src/lib/searchSettings/index.test.ts new file mode 100644 index 0000000000..813e204491 --- /dev/null +++ b/web/src/lib/searchSettings/index.test.ts @@ -0,0 +1,151 @@ +import { + CLOUD_BASED_PROVIDERS, + SELF_HOSTED_PROVIDERS, +} from "@/lib/searchSettings/constants"; +import { + defaultTestModelName, + findProviderModel, + findRegistryModel, + isBringYourOwnProvider, + isCloudBased, + visibleEmbeddingProviders, +} from "@/lib/searchSettings"; +import { + EmbeddingProvider, + EmbeddingProviderName, +} from "@/lib/searchSettings/types"; + +const ALL_PROVIDERS = [...CLOUD_BASED_PROVIDERS, ...SELF_HOSTED_PROVIDERS]; + +function modelNames(providers: EmbeddingProvider[]): string[] { + return providers.flatMap((p) => p.embeddingModels.map((m) => m.modelName)); +} + +function groupModels( + providers: EmbeddingProvider[], + providerName: EmbeddingProviderName +): string[] | undefined { + return providers + .find((p) => p.providerName === providerName) + ?.embeddingModels.map((m) => m.modelName); +} + +// backend/tests/unit/shared_configs/test_web_embedding_model_registry_parity.py +// pins the model specs and legacy flags. These tests cover what it cannot. +describe("the embedding model registry", () => { + it("uses each model name once, so a name lookup is unambiguous", () => { + const names = modelNames(ALL_PROVIDERS); + + expect(new Set(names).size).toBe(names.length); + }); + + it("never routes a self-hosted group to a cloud API", () => { + for (const provider of SELF_HOSTED_PROVIDERS) { + expect(isCloudBased(provider.providerName)).toBe(false); + } + }); + + it("has a selectable model to test the credentials of each cloud provider", () => { + for (const provider of CLOUD_BASED_PROVIDERS) { + if (isBringYourOwnProvider(provider.providerName)) continue; + const modelName = defaultTestModelName(provider.providerName); + + expect( + findProviderModel(provider.providerName, modelName)?.legacy + ).toBeFalsy(); + } + }); +}); + +describe("visibleEmbeddingProviders", () => { + const GRANITE = { + providerName: EmbeddingProviderName.IBM, + modelName: "ibm-granite/granite-embedding-97m-multilingual-r2", + }; + // No cloud group of the registry has only legacy models, so build one. + const LEGACY_ONLY_VOYAGE = CLOUD_BASED_PROVIDERS.map((p) => + p.providerName === EmbeddingProviderName.VOYAGE + ? { ...p, embeddingModels: p.embeddingModels.filter((m) => m.legacy) } + : p + ); + + it("hides legacy models that are not current and drops the groups it empties", () => { + const visible = visibleEmbeddingProviders(ALL_PROVIDERS, null); + + for (const name of modelNames(visible)) { + expect(findRegistryModel(name)?.legacy).toBeFalsy(); + } + expect( + visibleEmbeddingProviders(SELF_HOSTED_PROVIDERS, null).map( + (p) => p.providerName + ) + ).toEqual([ + EmbeddingProviderName.IBM, + EmbeddingProviderName.VOYAGE_SELF_HOSTED, + EmbeddingProviderName.NVIDIA, + ]); + }); + + it("keeps the bring-your-own groups, which have no registry models", () => { + expect( + visibleEmbeddingProviders(LEGACY_ONLY_VOYAGE, GRANITE).map( + (p) => p.providerName + ) + ).toEqual([ + EmbeddingProviderName.COHERE, + EmbeddingProviderName.OPENAI, + EmbeddingProviderName.GOOGLE, + EmbeddingProviderName.LITELLM, + EmbeddingProviderName.AZURE, + EmbeddingProviderName.BIFROST, + ]); + }); + + it("shows a current legacy model in its own group, after the selectable ones", () => { + const selfHosted = visibleEmbeddingProviders(SELF_HOSTED_PROVIDERS, { + providerName: EmbeddingProviderName.MICROSOFT, + modelName: "intfloat/e5-base-v2", + }); + + expect(groupModels(selfHosted, EmbeddingProviderName.MICROSOFT)).toEqual([ + "intfloat/e5-base-v2", + ]); + expect( + groupModels(selfHosted, EmbeddingProviderName.NOMIC) + ).toBeUndefined(); + + const cloud = visibleEmbeddingProviders(CLOUD_BASED_PROVIDERS, { + providerName: EmbeddingProviderName.COHERE, + modelName: "embed-english-v3.0", + }); + + expect(groupModels(cloud, EmbeddingProviderName.COHERE)).toEqual([ + "embed-v5.0-pro", + "embed-v5.0-fast", + "embed-english-v3.0", + ]); + }); + + it("keeps an emptied group with saved credentials, so they can be disconnected", () => { + const visible = visibleEmbeddingProviders( + LEGACY_ONLY_VOYAGE, + GRANITE, + new Set([EmbeddingProviderName.VOYAGE]) + ); + + expect(groupModels(visible, EmbeddingProviderName.VOYAGE)).toEqual([]); + }); + + it("matches the current model on provider and name", () => { + // An Azure deployment labelled like a legacy Cohere model. + const visible = visibleEmbeddingProviders(CLOUD_BASED_PROVIDERS, { + providerName: EmbeddingProviderName.AZURE, + modelName: "embed-english-v3.0", + }); + + expect(modelNames(visible)).not.toContain("embed-english-v3.0"); + expect( + findProviderModel(EmbeddingProviderName.AZURE, "text-embedding-3-large") + ).toBeNull(); + }); +}); diff --git a/web/src/lib/searchSettings/index.ts b/web/src/lib/searchSettings/index.ts index 5a57377784..07c9ad8029 100644 --- a/web/src/lib/searchSettings/index.ts +++ b/web/src/lib/searchSettings/index.ts @@ -5,6 +5,7 @@ import { } from "@/lib/searchSettings/constants"; import { EmbeddingModel, + EmbeddingModelRef, EmbeddingProvider, EmbeddingProviderName, IndexSettingsTranslator, @@ -35,12 +36,27 @@ export function findProvider( * provider โ€” i.e. credentials are managed via API keys, the row should * have an editable creds modal, and the backend should route through a * cloud SDK rather than the local model server. Returns `false` for - * self-hosted buckets (`NOMIC`, `MICROSOFT`) and `CUSTOM`. + * self-hosted buckets (e.g. `IBM`, `VOYAGE_SELF_HOSTED`, `NOMIC`) and `CUSTOM`. */ export function isCloudBased(providerName: EmbeddingProviderName): boolean { return CLOUD_BASED_PROVIDERS.some((p) => p.providerName === providerName); } +/** + * `true` for the cloud providers that have no registry models (LiteLLM, + * Azure, Bifrost). The admin enters the model spec in the connect modal. The + * picker shows them in the "bring your own" section, not in the main list. + */ +export function isBringYourOwnProvider( + providerName: EmbeddingProviderName +): boolean { + return ( + providerName === EmbeddingProviderName.LITELLM || + providerName === EmbeddingProviderName.AZURE || + providerName === EmbeddingProviderName.BIFROST + ); +} + /** * Find an {@link EmbeddingModel} spec by `modelName` across both registries. * Returns `null` for models that aren't pre-registered (e.g. custom @@ -59,6 +75,65 @@ export function findRegistryModel(modelName: string): EmbeddingModel | null { return null; } +/** + * Find the registry model of one provider group. Unlike `findRegistryModel`, + * a LiteLLM or Azure deployment whose label equals a registry name (e.g. + * `text-embedding-3-large`) does not match the OpenAI entry. + */ +export function findProviderModel( + providerName: EmbeddingProviderName, + modelName: string +): EmbeddingModel | null { + return ( + findProvider(providerName).embeddingModels.find( + (m) => m.modelName === modelName + ) ?? null + ); +} + +/** + * The provider groups and models the picker lists and searches. A legacy model + * shows only while it is the current model, matched on provider and name. A + * group that this empties drops out, unless `keepProviders` has it (saved + * credentials: its header holds the Edit and Disconnect buttons). + */ +export function visibleEmbeddingProviders( + providers: EmbeddingProvider[], + current: EmbeddingModelRef | null, + keepProviders: { + has(providerName: EmbeddingProviderName): boolean; + } = new Set() +): EmbeddingProvider[] { + return providers.flatMap((provider) => { + const models = provider.embeddingModels.filter( + (m) => + !m.legacy || + (current?.providerName === provider.providerName && + current.modelName === m.modelName) + ); + const emptied = models.length === 0 && provider.embeddingModels.length > 0; + return emptied && !keepProviders.has(provider.providerName) + ? [] + : [{ ...provider, embeddingModels: models }]; + }); +} + +/** + * The model a credential test embeds with when no model is being connected or + * edited: the provider's first selectable registry model. + */ +export function defaultTestModelName( + providerName: EmbeddingProviderName +): string { + const model = findProvider(providerName).embeddingModels.find( + (m) => !m.legacy + ); + if (!model) { + throw new Error(`No selectable ${providerName} model to test with`); + } + return model.modelName; +} + /** Translated registry description, undefined without a key. */ export function embeddingModelDescription( model: EmbeddingModel | null | undefined, @@ -98,8 +173,8 @@ export function embeddingModelDescription( * * 2. Which logical bucket does this model belong to for UI purposes โ€” * icon, modal selection, displayName? (Currently UNREPRESENTED in the - * schema for self-hosted models: Nomic vs Microsoft vs custom must - * be inferred from `model_name` by walking a frontend registry.) + * schema for self-hosted models: IBM vs Nomic vs Microsoft vs custom + * must be inferred from `model_name` by walking a frontend registry.) * * Because (2) is unrepresented in the schema, the frontend has historically * resorted to one of two hacks: @@ -139,7 +214,7 @@ export function embeddingModelDescription( * backend already filled it in), this resolver SIMULATES the proper schema * by deriving the canonical `EmbeddingProviderName`, including the * synthetic self-hosted bucket values that the backend cannot currently - * persist (`NOMIC`, `MICROSOFT`, `CUSTOM`). + * persist (e.g. `IBM`, `NVIDIA`, `NOMIC`, `CUSTOM`). * * Resolution rules, in order: * @@ -156,7 +231,7 @@ export function embeddingModelDescription( * is yet available. * * 3. Otherwise, walk `SELF_HOSTED_PROVIDERS` for a `modelName` match and - * return that bucket's `providerName` (e.g. `NOMIC`, `MICROSOFT`). + * return that bucket's `providerName` (e.g. `IBM`, `MICROSOFT`). * * 4. Otherwise โ€” the model isn't in any registry โ€” fall through to * `EmbeddingProviderName.CUSTOM`. This is the "user added a custom diff --git a/web/src/lib/searchSettings/svc.test.ts b/web/src/lib/searchSettings/svc.test.ts index 869eea68dd..f1eca30e92 100644 --- a/web/src/lib/searchSettings/svc.test.ts +++ b/web/src/lib/searchSettings/svc.test.ts @@ -73,6 +73,7 @@ describe("Google embedding authentication", () => { providerType: "google", apiKey: null, apiUrl: "", + modelName: "gemini-embedding-2", apiVersion: null, deploymentName: null, vertexConfig, @@ -88,6 +89,7 @@ describe("Google embedding authentication", () => { expect(fetchSpy.mock.calls[1]?.[1]?.method).toBe("PUT"); const testBody = JSON.parse(String(fetchSpy.mock.calls[0]?.[1]?.body)); const saveBody = JSON.parse(String(fetchSpy.mock.calls[1]?.[1]?.body)); + expect(testBody.model_name).toBe("gemini-embedding-2"); expect(testBody.vertex_config).toEqual(vertexConfig); expect(testBody.api_key).toBeNull(); expect(saveBody.vertex_config).toEqual(vertexConfig); @@ -106,6 +108,7 @@ describe("Google embedding authentication", () => { providerType: "google", apiKey: null, apiUrl: "", + modelName: "gemini-embedding-2", apiVersion: null, deploymentName: null, vertexConfig: { diff --git a/web/src/lib/searchSettings/svc.ts b/web/src/lib/searchSettings/svc.ts index 05bdd7580b..edfb542372 100644 --- a/web/src/lib/searchSettings/svc.ts +++ b/web/src/lib/searchSettings/svc.ts @@ -38,9 +38,6 @@ export async function testEmbedding({ deploymentName, vertexConfig, }: TestEmbeddingArgs) { - const testModelName = - provider_type === "openai" ? "text-embedding-3-small" : modelName; - return await fetch("/api/admin/embedding/test-embedding", { method: "POST", headers: { "Content-Type": "application/json" }, @@ -48,7 +45,7 @@ export async function testEmbedding({ provider_type: provider_type, api_key: apiKey, api_url: apiUrl, - model_name: testModelName, + model_name: modelName, api_version: apiVersion, deployment_name: deploymentName, vertex_config: vertexConfig, @@ -67,13 +64,14 @@ export async function testEmbedding({ * * Returns the vector length of the test embedding, or `null` when no test ran. * `alwaysTest` runs the test even when the stored key is kept, for providers - * whose model is free text (Bifrost). + * whose model is free text (Bifrost). The test embeds with `modelName`, which + * must not be empty. */ export async function connectEmbeddingProvider({ providerType, apiKey, apiUrl, - modelName = "", + modelName, apiVersion, deploymentName, vertexConfig, @@ -82,7 +80,7 @@ export async function connectEmbeddingProvider({ providerType: string; apiKey: string | null; apiUrl: string; - modelName?: string; + modelName: string; apiVersion: string | null; deploymentName: string | null; vertexConfig?: VertexEmbeddingConfig | null; diff --git a/web/src/lib/searchSettings/types.ts b/web/src/lib/searchSettings/types.ts index 91c4b2038c..a1f43f82cd 100644 --- a/web/src/lib/searchSettings/types.ts +++ b/web/src/lib/searchSettings/types.ts @@ -22,9 +22,16 @@ export enum EmbeddingProviderName { AZURE = "azure", BIFROST = "bifrost", - // Self-hosted + // Self-hosted display groups. Frontend-only: the backend stores + // provider_type=null for every self-hosted model, so these values are never + // sent. Do not reuse a cloud value (e.g. VOYAGE) for a self-hosted group: + // `isCloudBased` would then route the model to that cloud API. + IBM = "ibm", + VOYAGE_SELF_HOSTED = "voyage_self_hosted", + NVIDIA = "nvidia", NOMIC = "nomic", MICROSOFT = "microsoft", + GTE = "gte", // Custom self-hosted (frontend-only sentinel; backend stores provider_type=null) CUSTOM = "custom", @@ -73,19 +80,28 @@ export interface EmbeddingModel { passagePrefix?: string | null; // Absent for custom models, which have no registry description. descriptionKey?: IndexSettingsMessageKey; - /** - * When true, existing usage is allowed, but selecting it as a new embedding - * model is blocked. + * Kept only so existing deployments still display and re-index correctly. + * Not offered as a new target: the picker shows a legacy model only while it + * is the current model, and the backend rejects it as a new target. */ - deprecated?: boolean; + legacy?: boolean; + /** Indexing on CPU is too slow for practical use. */ + gpuRecommended?: boolean; } +/** The fields sent to the backend. Registry display flags are left out. */ export type EmbeddingModelSpec = Omit< EmbeddingModel, - "descriptionKey" | "deprecated" + "descriptionKey" | "legacy" | "gpuRecommended" >; +/** The active embedding model, identified by its display group and name. */ +export interface EmbeddingModelRef { + modelName: string; + providerName: EmbeddingProviderName; +} + /** * Always write all three fields together. A name without its spec and provider forces * the submit path to guess them back, which misroutes LiteLLM and Azure models. diff --git a/web/src/views/admin/IndexSettingsPage/index.test.tsx b/web/src/views/admin/IndexSettingsPage/index.test.tsx new file mode 100644 index 0000000000..38abdc2f28 --- /dev/null +++ b/web/src/views/admin/IndexSettingsPage/index.test.tsx @@ -0,0 +1,451 @@ +/** + * @jest-environment jsdom + */ + +import { + render, + screen, + setupUser, + waitFor, + within, +} from "@tests/setup/test-utils"; +import { toast } from "@opal/layouts"; +import IndexSettingsPage from "@/views/admin/IndexSettingsPage"; +import type { useConnectorIndexingStatusWithPagination } from "@/lib/hooks"; +import type { LlmDefaults } from "@/lib/languageModels/types"; +import { + ApplicationStatus, + QueryHistoryType, + type AppSettings, +} from "@/lib/settings/types"; +import { + EmbeddingProviderName, + type ConfiguredEmbeddingProvider, + type EmbeddingModelResponse, + type ReindexProgress, + type SavedSearchSettings, + VectorQuantization, +} from "@/lib/searchSettings/types"; + +jest.mock("next/navigation", () => ({ + useRouter: () => ({ + push: jest.fn(), + replace: jest.fn(), + refresh: jest.fn(), + }), + usePathname: () => "/admin/configuration/search", + useSearchParams: () => new URLSearchParams(), +})); + +jest.mock("@/lib/settings/hooks", () => ({ + useSettings: () => mockAppSettings, +})); + +jest.mock("@/lib/adminNavLabels", () => ({ + useAdminRouteTitle: () => (route: { title: string }) => route.title, +})); + +// No requireActual: the real module has an import cycle through the +// language-model modals. +jest.mock("@/lib/hooks", () => ({ + useConnectorIndexingStatusWithPagination: () => mockIndexingStatuses, +})); + +jest.mock("@/lib/languageModels/hooks", () => ({ + ...jest.requireActual("@/lib/languageModels/hooks"), + useLlmDefaults: () => mockLlmDefaults, +})); + +jest.mock("@/lib/searchSettings/hooks", () => ({ + ...jest.requireActual("@/lib/searchSettings/hooks"), + useCurrentEmbeddingModel: () => ({ + data: mockState.currentModel, + isLoading: false, + }), + useCurrentSearchSettings: () => ({ + data: mockState.searchSettings, + isLoading: false, + }), + useSecondarySearchSettings: () => ({ data: null }), + useReindexProgress: () => ({ data: mockReindexProgress }), + useConfiguredEmbeddingProviders: () => ({ + data: mockState.configuredProviders, + }), +})); + +// Stable objects: the page memoizes on these, as it does on SWR data. +const mockAppSettings: AppSettings = { + default_language: "en", + anonymous_user_enabled: false, + invite_only_enabled: false, + notifications: [], + needs_reindexing: false, + gpu_enabled: false, + application_status: ApplicationStatus.ACTIVE, + auto_scroll: true, + temperature_override_enabled: true, + query_history_type: QueryHistoryType.NORMAL, + image_extraction_and_analysis_enabled: false, + image_analysis_max_size_mb: 20, + hide_provider_grouping: false, + enterprise: null, + appName: "Onyx", + logoUrl: null, + vectorDbEnabled: false, + isLoading: false, + error: undefined, +}; +const mockIndexingStatuses: Pick< + ReturnType, + "data" | "isLoading" | "isValidating" | "error" +> = { + data: [], + isLoading: false, + isValidating: false, + error: undefined, +}; +const mockLlmDefaults: LlmDefaults = { + llmProviders: [], + hasAnyLlm: false, + hasAnyVisionLlm: false, + defaultLlm: null, + defaultVision: null, + isLoading: false, +}; +const mockReindexProgress: ReindexProgress = { + total: 0, + waiting: 0, + in_progress: 0, + completed: 0, + failed: 0, + paused: 0, + reembeds: true, +}; + +interface MockState { + currentModel: EmbeddingModelResponse | null; + searchSettings: SavedSearchSettings | null; + configuredProviders: ConfiguredEmbeddingProvider[]; +} +const mockState: MockState = { + currentModel: null, + searchSettings: null, + configuredProviders: [], +}; + +function setCurrentModel( + modelName: string, + providerType: EmbeddingProviderName | null, + modelDim: number +) { + const current: EmbeddingModelResponse = { + model_name: modelName, + model_dim: modelDim, + normalize: providerType === null, + query_prefix: "", + passage_prefix: "", + provider_type: providerType, + api_key: null, + api_url: null, + index_name: "danswer_chunk", + }; + mockState.currentModel = current; + mockState.searchSettings = { + ...current, + index_name: current.index_name, + multipass_indexing: false, + enable_contextual_rag: false, + contextual_rag_model_configuration_id: null, + multilingual_expansion: [], + disable_rerank_for_streaming: false, + api_url: null, + num_rerank: 0, + reduced_dimension: null, + vector_quantization: VectorQuantization.NONE, + rerank_model_name: null, + rerank_provider_type: null, + rerank_api_key: null, + rerank_api_url: null, + use_port_flow: true, + }; +} + +function configureProviders(...providerTypes: EmbeddingProviderName[]) { + mockState.configuredProviders = providerTypes.map((provider_type) => ({ + provider_type, + api_key: "sk-****", + api_url: null, + api_version: null, + deployment_name: null, + })); +} + +function modelCard(providerName: EmbeddingProviderName, modelName: string) { + return screen.getByTestId( + `embedding-model-card:${providerName}:${modelName}` + ); +} + +function queryModelCard( + providerName: EmbeddingProviderName, + modelName: string +) { + return screen.queryByTestId( + `embedding-model-card:${providerName}:${modelName}` + ); +} + +const BRING_YOUR_OWN = "Advanced: bring your own model or proxy"; +const GRANITE = "ibm-granite/granite-embedding-97m-multilingual-r2"; + +async function openModelPicker(user: ReturnType) { + await user.click(screen.getByRole("button", { name: "View All Models" })); +} + +interface SentRequest { + url: string; + method: string; + body: Record; +} + +describe("IndexSettingsPage embedding model picker", () => { + let requests: SentRequest[]; + // A test embedding with this model fails, as with a retired model. + let failingTestModel: string | null; + + beforeEach(() => { + mockState.currentModel = null; + mockState.searchSettings = null; + mockState.configuredProviders = []; + requests = []; + failingTestModel = null; + jest + .spyOn(global, "fetch") + .mockImplementation( + async (input: RequestInfo | URL, init?: RequestInit) => { + const body = init?.body ? JSON.parse(String(init.body)) : {}; + requests.push({ + url: String(input), + method: init?.method ?? "GET", + body, + }); + if ( + String(input).endsWith("/test-embedding") && + body.model_name === failingTestModel + ) { + return new Response(JSON.stringify({ detail: "model retired" }), { + status: 400, + }); + } + return new Response("{}", { status: 200 }); + } + ); + }); + + afterEach(() => jest.restoreAllMocks()); + + function testedModels(): unknown[] { + return requests + .filter((r) => r.url.endsWith("/test-embedding")) + .map((r) => r.body.model_name); + } + + it("shows a legacy current model as current, with a Legacy tag and an upgrade hint", async () => { + setCurrentModel("nomic-ai/nomic-embed-text-v1", null, 768); + const user = setupUser(); + render(); + + const summary = await screen.findByTestId("current-embedding-model"); + expect(within(summary).getByText("Legacy")).toBeInTheDocument(); + expect( + within(summary).getByText(/This is a legacy model/) + ).toBeInTheDocument(); + + await openModelPicker(user); + + const card = modelCard( + EmbeddingProviderName.NOMIC, + "nomic-ai/nomic-embed-text-v1" + ); + expect( + within(card).getByRole("button", { name: "Current Model" }) + ).toBeInTheDocument(); + expect(within(card).getByText("Legacy")).toBeInTheDocument(); + // Other legacy models stay hidden; the new ones are selectable. + expect( + queryModelCard(EmbeddingProviderName.MICROSOFT, "intfloat/e5-base-v2") + ).toBeNull(); + expect( + within(modelCard(EmbeddingProviderName.IBM, GRANITE)).getByRole( + "button", + { name: "Select Model" } + ) + ).toBeInTheDocument(); + expect( + within( + modelCard( + EmbeddingProviderName.NVIDIA, + "nvidia/Nemotron-3-Embed-1B-BF16" + ) + ).getByText("GPU recommended") + ).toBeInTheDocument(); + }); + + it("does not find a hidden legacy model by search", async () => { + setCurrentModel(GRANITE, null, 384); + const user = setupUser(); + render(); + await openModelPicker(user); + + await user.type(screen.getByPlaceholderText("Search models..."), "e5"); + + expect( + await screen.findByText("No self-hosted models found") + ).toBeInTheDocument(); + }); + + it("keeps the bring-your-own options in the collapsed Advanced section", async () => { + setCurrentModel( + "text-embedding-3-small", + EmbeddingProviderName.OPENAI, + 1536 + ); + configureProviders(EmbeddingProviderName.OPENAI); + const user = setupUser(); + render(); + await openModelPicker(user); + + const toggle = screen.getByRole("button", { name: BRING_YOUR_OWN }); + expect(toggle).toHaveAttribute("aria-expanded", "false"); + expect( + screen.queryByText("Add configs for your Azure embedding providers.") + ).toBeNull(); + + await user.click(toggle); + + expect( + screen.getByText("Add configs for your LiteLLM embedding providers.") + ).toBeInTheDocument(); + expect( + screen.getByText("Add configs for your Azure embedding providers.") + ).toBeInTheDocument(); + await user.click(screen.getByRole("tab", { name: "Self-hosted" })); + expect( + screen.getByRole("button", { name: "Add Custom Model" }) + ).toBeInTheDocument(); + }); + + it("keeps a current Azure deployment apart from the OpenAI model of the same name", async () => { + setCurrentModel( + "text-embedding-3-large", + EmbeddingProviderName.AZURE, + 1536 + ); + configureProviders( + EmbeddingProviderName.AZURE, + EmbeddingProviderName.OPENAI + ); + const user = setupUser(); + render(); + + const summary = await screen.findByTestId("current-embedding-model"); + expect( + within(summary).queryByText(/OpenAI's large embedding model/) + ).toBeNull(); + + await openModelPicker(user); + const openAiCard = modelCard( + EmbeddingProviderName.OPENAI, + "text-embedding-3-large" + ); + await user.click( + within(openAiCard).getByRole("button", { name: "Select Model" }) + ); + + expect( + within(openAiCard).getByRole("button", { name: "Selected" }) + ).toBeInTheDocument(); + }); + + it("stages a custom model with a registry name as a custom model", async () => { + setCurrentModel(GRANITE, null, 384); + const user = setupUser(); + render(); + await openModelPicker(user); + await user.click(screen.getByRole("button", { name: BRING_YOUR_OWN })); + await user.click(screen.getByRole("button", { name: "Add Custom Model" })); + + const dialog = await screen.findByRole("dialog", { + name: /set up custom model/i, + }); + await user.type( + within(dialog).getByLabelText(/model name/i), + "voyageai/voyage-4-nano" + ); + await user.type(within(dialog).getByLabelText(/model dimension/i), "1024"); + await user.click(within(dialog).getByRole("button", { name: "Connect" })); + + expect( + await screen.findByRole("button", { name: "Apply & Re-index" }) + ).toBeInTheDocument(); + // The registry card of the same name is not the staged model. + expect( + within( + modelCard( + EmbeddingProviderName.VOYAGE_SELF_HOSTED, + "voyageai/voyage-4-nano" + ) + ).getByRole("button", { name: "Select Model" }) + ).toBeInTheDocument(); + }); + + it("tests the credentials against the model being connected", async () => { + setCurrentModel( + "text-embedding-3-small", + EmbeddingProviderName.OPENAI, + 1536 + ); + configureProviders(EmbeddingProviderName.OPENAI); + const user = setupUser(); + render(); + await openModelPicker(user); + + await user.click( + within( + modelCard(EmbeddingProviderName.COHERE, "embed-v5.0-pro") + ).getByRole("button", { name: "Connect" }) + ); + const dialog = await screen.findByRole("dialog", { + name: /set up cohere/i, + }); + await user.type(within(dialog).getByLabelText(/api key/i), "co-key"); + await user.click(within(dialog).getByRole("button", { name: "Connect" })); + + await waitFor(() => expect(testedModels()).toEqual(["embed-v5.0-pro"])); + }); + + it("does not save new credentials that fail the test with the current model", async () => { + setCurrentModel("embed-english-v3.0", EmbeddingProviderName.COHERE, 1024); + configureProviders(EmbeddingProviderName.COHERE); + failingTestModel = "embed-english-v3.0"; + const toastError = jest.spyOn(toast, "error"); + const user = setupUser(); + render(); + await openModelPicker(user); + + await user.click(screen.getByRole("button", { name: "Edit credentials" })); + const dialog = await screen.findByRole("dialog", { + name: /manage cohere/i, + }); + const apiKeyInput = within(dialog).getByLabelText(/api key/i); + await user.clear(apiKeyInput); + await user.type(apiKeyInput, "co-new-key"); + await user.click(within(dialog).getByRole("button", { name: "Update" })); + + await waitFor(() => + expect(toastError).toHaveBeenCalledWith("model retired") + ); + expect(testedModels()).toEqual(["embed-english-v3.0"]); + expect(requests.some((r) => r.method === "PUT")).toBe(false); + }); +}); diff --git a/web/src/views/admin/IndexSettingsPage/index.tsx b/web/src/views/admin/IndexSettingsPage/index.tsx index b3366e205a..a846cadce9 100644 --- a/web/src/views/admin/IndexSettingsPage/index.tsx +++ b/web/src/views/admin/IndexSettingsPage/index.tsx @@ -40,6 +40,7 @@ import { SvgClock, SvgCloud, SvgEmpty, + SvgExpand, SvgExternalLink, SvgFold, SvgPlusCircle, @@ -62,6 +63,7 @@ import { VectorQuantization, type ConfiguredEmbeddingProvider, type EmbeddingModel, + type EmbeddingModelRef, type EmbeddingModelRequest, type EmbeddingModelSelection, type EmbeddingModelState, @@ -77,9 +79,11 @@ import { import { embeddingModelDescription, findProvider, - findRegistryModel, + findProviderModel, + isBringYourOwnProvider, isCloudBased, resolveProviderName, + visibleEmbeddingProviders, } from "@/lib/searchSettings"; import { isSameModelSelection, @@ -107,7 +111,6 @@ import { useSecondarySearchSettings, } from "@/lib/searchSettings/hooks"; import { useLlmDefaults } from "@/lib/languageModels/hooks"; -import useFilter from "@/hooks/useFilter"; import { SimpleModelSelector } from "@/lib/languageModels/components"; import { filterModelConfigurations, @@ -225,8 +228,10 @@ function EmbeddingProviderInfo({ providerName }: EmbeddingProviderInfoProps) { interface ProviderGroupProps { provider: EmbeddingProvider; - currentModelName?: string; - selectedModelName?: string; + /** The active embedding model, in this group or another one. */ + current: EmbeddingModelRef | null; + /** The staged model, in this group or another one. */ + staged: EmbeddingModelRef | null; isCloud?: boolean; existingCredentials?: ConfiguredEmbeddingProvider; /** @@ -248,8 +253,8 @@ interface ProviderGroupProps { function ProviderGroup({ provider, - currentModelName, - selectedModelName, + current, + staged, isCloud = false, existingCredentials, existingModel, @@ -258,6 +263,7 @@ function ProviderGroup({ }: ProviderGroupProps) { const t = useTranslations("admin.indexSettings"); const models = provider.embeddingModels; + const isBringYourOwn = isBringYourOwnProvider(provider.providerName); const isConfigured = isCloud ? !!existingCredentials : true; const disconnectModal = useCreateModal(); const connectModal = useCreateModal(); @@ -265,9 +271,14 @@ function ProviderGroup({ const providerCreationModal = useCreateModal(); const [pendingConnectModel, setPendingConnectModel] = useState(null); - const providerGroupContainsCurrentModelName = models.some( - (m) => m.modelName === currentModelName - ); + // The current model uses this provider's credentials, whether or not it is + // a registry model, so they cannot be disconnected. + const isCurrentProvider = current?.providerName === provider.providerName; + const currentModelName = isCurrentProvider ? current?.modelName : undefined; + const selectedModelName = + staged?.providerName === provider.providerName + ? staged.modelName + : undefined; const handleDisconnect = useCallback(async () => { if (!isCloud) return; @@ -309,7 +320,6 @@ function ProviderGroup({ const handleModelSelect = useCallback( (model: EmbeddingModel) => { - if (model.deprecated) return; const state = getModelState(model); if (state === "selected" || state === "current") { @@ -317,6 +327,10 @@ function ProviderGroup({ return; } + // A legacy model is listed only while it is current; it is never a new + // target. + if (model.legacy) return; + if (state === "unconnected" && isCloud) { setPendingConnectModel(model); connectModal.toggle(true); @@ -335,6 +349,23 @@ function ProviderGroup({ ] ); + function renderModelCard(model: EmbeddingModel) { + const state = getModelState(model); + // The current card loses its highlight to a staged model in any group. + const isPrioritized = + state === "selected" || (state === "current" && staged === null); + return ( + handleModelSelect(model)} + /> + ); + } + return ( <> {isCloud && ( @@ -433,9 +464,9 @@ function ProviderGroup({ icon={SvgUnplug} prominence="tertiary" size="sm" - disabled={providerGroupContainsCurrentModelName} + disabled={isCurrentProvider} tooltip={ - providerGroupContainsCurrentModelName + isCurrentProvider ? t("providerGroup.disconnectButton.disabledTooltip") : undefined } @@ -456,7 +487,8 @@ function ProviderGroup({ - {models.length === 0 ? ( + {models.map(renderModelCard)} + {isBringYourOwn && ( - ) : ( - models.map((model) => { - const state = getModelState(model); - const isPrioritized = - state === "selected" || - (state === "current" && !selectedModelName); - return ( - handleModelSelect(model)} - /> - ); - }) )} @@ -523,7 +538,6 @@ function EmbeddingModelCard({ }: EmbeddingModelCardProps) { const t = useTranslations("admin.indexSettings"); const { appName } = useSettings(); - const isDeprecated = model.deprecated; const topRightButton = (() => { switch (modelState) { case "unconnected": @@ -532,26 +546,13 @@ function EmbeddingModelCard({ prominence="tertiary" rightIcon={SvgArrowExchange} onClick={onSelect} - disabled={isDeprecated} - tooltip={ - isDeprecated - ? t("modelCard.deprecated.connectTooltip") - : undefined - } > {t("modelCard.connectButton.label")} ); case "connected": return ( - ); @@ -580,19 +581,13 @@ function EmbeddingModelCard({ } })(); - const isClickable = - !isDeprecated && - (modelState === "unconnected" || - modelState === "connected" || - modelState === "current" || - modelState === "selected"); - return ( @@ -600,6 +595,17 @@ function EmbeddingModelCard({ icon={provider.icon} title={model.modelName} description={embeddingModelDescription(model, t, appName)} + tag={ + model.legacy + ? { + title: t("modelCard.legacy.tag"), + color: "amber", + tooltip: t("modelCard.legacy.tooltip"), + } + : model.gpuRecommended + ? { title: t("modelCard.gpuRecommended.tag"), color: "gray" } + : undefined + } sizePreset="main-ui" variant="section" /> @@ -613,6 +619,113 @@ function EmbeddingModelCard({ ); } +interface BringYourOwnSectionProps { + open: boolean; + onOpenChange: (open: boolean) => void; + description: string | RichStr; + children: React.ReactNode; +} + +/** + * Low-prominence, collapsible home for the bring-your-own options (LiteLLM, + * Azure, Bifrost, custom self-hosted models), below the registry models. + */ +function BringYourOwnSection({ + open, + onOpenChange, + description, + children, +}: BringYourOwnSectionProps) { + const t = useTranslations("admin.indexSettings"); + return ( + + + + + {open && ( + <> + + + {description} + + + {children} + + )} + + ); +} + +interface CustomModelGroupProps { + onAddCustomModel: () => void; +} + +/** Self-hosted models entered by hand ("Add Custom Model"). */ +function CustomModelGroup({ onAddCustomModel }: CustomModelGroupProps) { + const t = useTranslations("admin.indexSettings"); + return ( + + + + + + + + + {t("modelPicker.addCustomModel.label")} + + } + center + /> + + + ); +} + interface IndexSettingsFormValues extends EmbeddingModelSelection { enable_contextual_rag: boolean; contextual_rag_model_configuration_id: number | null; @@ -783,35 +896,8 @@ export default function IndexSettingsPage() { null ); - const allModels = useMemo( - () => [...CLOUD_BASED_PROVIDERS, ...SELF_HOSTED_PROVIDERS], - [] - ); - - const { - query, - setQuery, - filtered: filteredProviders, - } = useFilter( - allModels, - (embeddingProvider) => - `${embeddingProvider.displayName} ${embeddingProvider.embeddingModels - .map((embeddingModel) => embeddingModel.modelName) - .join(" ")}` - ); - - const { filteredCloudProviders, filteredSelfHostedProviders } = - useMemo(() => { - const matched = new Set(filteredProviders); - return { - filteredCloudProviders: CLOUD_BASED_PROVIDERS.filter((p) => - matched.has(p) - ), - filteredSelfHostedProviders: SELF_HOSTED_PROVIDERS.filter((p) => - matched.has(p) - ), - }; - }, [filteredProviders]); + // Whether the admin opened the bring-your-own section. + const [bringYourOwnOpen, setBringYourOwnOpen] = useState(false); const { data: secondarySearchSettings } = useSecondarySearchSettings(); // INSTANT switchover swaps immediately โ€” no secondary settings โ€” and backfills on the @@ -855,27 +941,85 @@ export default function IndexSettingsPage() { }; }, [currentEmbeddingModel]); - const currentProviderName = currentEmbeddingModel - ? resolveProviderName( - currentEmbeddingModel.model_name, - currentEmbeddingModel.provider_type - ) - : null; + const currentModelName = currentEmbeddingModel?.model_name; + const currentProviderType = currentEmbeddingModel?.provider_type ?? null; + const currentModelRef: EmbeddingModelRef | null = useMemo( + () => + currentModelName === undefined + ? null + : { + modelName: currentModelName, + providerName: resolveProviderName( + currentModelName, + currentProviderType + ), + }, + [currentModelName, currentProviderType] + ); + const currentProviderName = currentModelRef?.providerName ?? null; const currentProvider = currentProviderName ? findProvider(currentProviderName) : null; const isCurrentCloudBased = currentProviderName ? isCloudBased(currentProviderName) : false; + // Matched on provider AND name: an Azure deployment labelled like an OpenAI + // model must not borrow that model's registry entry. + const currentRegistryModel = currentModelRef + ? findProviderModel(currentModelRef.providerName, currentModelRef.modelName) + : null; - const { data: searchSettings, isLoading: isLoadingSearchSettings } = - useCurrentSearchSettings({ pollIntervalMs: isReindexing ? 5000 : 0 }); const { data: configuredProvidersList } = useConfiguredEmbeddingProviders(); const configuredProviders = useMemo( () => new Map((configuredProvidersList ?? []).map((p) => [p.provider_type, p])), [configuredProvidersList] ); + const visibleCloudProviders = useMemo( + () => + visibleEmbeddingProviders( + CLOUD_BASED_PROVIDERS, + currentModelRef, + configuredProviders + ), + [currentModelRef, configuredProviders] + ); + const visibleSelfHostedProviders = useMemo( + () => visibleEmbeddingProviders(SELF_HOSTED_PROVIDERS, currentModelRef), + [currentModelRef] + ); + const [query, setQuery] = useState(""); + const isSearching = query.trim() !== ""; + // The search runs over the visible groups, so it cannot find a hidden + // legacy model. + const { + filteredCloudProviders, + filteredBringYourOwnProviders, + filteredSelfHostedProviders, + customModelMatched, + } = useMemo(() => { + const needle = query.trim().toLowerCase(); + const matches = (provider: EmbeddingProvider) => + `${provider.displayName} ${provider.embeddingModels + .map((model) => model.modelName) + .join(" ")}` + .toLowerCase() + .includes(needle); + const cloud = visibleCloudProviders.filter(matches); + return { + filteredCloudProviders: cloud.filter( + (p) => !isBringYourOwnProvider(p.providerName) + ), + filteredBringYourOwnProviders: cloud.filter((p) => + isBringYourOwnProvider(p.providerName) + ), + filteredSelfHostedProviders: visibleSelfHostedProviders.filter(matches), + customModelMatched: matches(CUSTOM_PROVIDER), + }; + }, [query, visibleCloudProviders, visibleSelfHostedProviders]); + + const { data: searchSettings, isLoading: isLoadingSearchSettings } = + useCurrentSearchSettings({ pollIntervalMs: isReindexing ? 5000 : 0 }); const cancelReindexModal = useCreateModal(); const forwardOnlyModal = useCreateModal(); const customModelModal = useCreateModal(); @@ -944,9 +1088,12 @@ export default function IndexSettingsPage() { () => savedModelSelection( currentEmbeddingModelSpec, - currentEmbeddingModel?.provider_type ?? null + // Custom models are staged as CUSTOM, so the saved one must match. + currentProviderName === EmbeddingProviderName.CUSTOM + ? EmbeddingProviderName.CUSTOM + : (currentEmbeddingModel?.provider_type ?? null) ), - [currentEmbeddingModelSpec, currentEmbeddingModel] + [currentEmbeddingModelSpec, currentEmbeddingModel, currentProviderName] ); const initialFormValues: IndexSettingsFormValues = useMemo( @@ -1271,10 +1418,56 @@ export default function IndexSettingsPage() { void setFieldValue("model_spec", selection.model_spec); void setFieldValue("model_provider", selection.model_provider); }; + // Compares the provider and the spec too, like classifyChanges: + // an OpenAI model with the name of the current Azure deployment + // is a staged change. const isModelStaged = - values.model_name !== initialFormValues.model_name && - !!values.model_name; - const stagedModelName = isModelStaged ? values.model_name : null; + !!values.model_name && + !isSameModelSelection(values, initialFormValues); + // Registry picks carry no provider; their name resolves it. + // Custom models carry CUSTOM, even with a registry name. + const stagedModelRef: EmbeddingModelRef | null = isModelStaged + ? { + modelName: values.model_name, + providerName: + values.model_provider ?? + resolveProviderName(values.model_name, null), + } + : null; + // A search that matches a bring-your-own option shows it. + const bringYourOwnExpanded = bringYourOwnOpen || isSearching; + const renderCloudProviderGroup = ( + provider: EmbeddingProvider + ) => ( + + applySelection({ + model_name: name, + model_spec: customModel + ? { ...customModel, modelName: name } + : null, + model_provider: customModel + ? provider.providerName + : null, + }) + } + onDeselectModel={() => applySelection(savedSelection)} + /> + ); // Block apply when Contextual Retrieval is on but no LLM is set. const contextualRagModelMissing = values.enable_contextual_rag && @@ -1511,7 +1704,7 @@ export default function IndexSettingsPage() { ...customModel, modelName: customModel.modelName, }, - model_provider: null, + model_provider: EmbeddingProviderName.CUSTOM, }); } customModelModal.toggle(false); @@ -1765,55 +1958,29 @@ export default function IndexSettingsPage() { expandedContent={ <> - {filteredCloudProviders.length > 0 ? ( + {filteredCloudProviders.length > 0 || + filteredBringYourOwnProviders.length > + 0 ? ( {filteredCloudProviders.map( - (provider) => ( - - applySelection({ - model_name: name, - model_spec: customModel - ? { - ...customModel, - modelName: name, - } - : null, - model_provider: customModel - ? provider.providerName - : null, - }) - } - onDeselectModel={() => - applySelection(savedSelection) - } - /> - ) + renderCloudProviderGroup + )} + {filteredBringYourOwnProviders.length > + 0 && ( + + {filteredBringYourOwnProviders.map( + renderCloudProviderGroup + )} + )} ) : ( @@ -1830,23 +1997,25 @@ export default function IndexSettingsPage() { - {filteredSelfHostedProviders.length > - 0 ? ( + {filteredSelfHostedProviders.length > 0 || + customModelMatched ? ( + + {t("modelPicker.cpuEstimateHint")} + {filteredSelfHostedProviders.map( (shProvider) => ( applySelection({ model_name: name, @@ -1861,63 +2030,21 @@ export default function IndexSettingsPage() { ) )} - -
- - -
- -
- -
- - - customModelModal.toggle(true) - } + {customModelMatched && ( + - - customModelModal.toggle( - true - ) - } - > - {t( - "modelPicker.addCustomModel.label" - )} - + + customModelModal.toggle(true) } - center /> - - + + )} ) : ( ) : ( -
+
@@ -2013,29 +2151,38 @@ export default function IndexSettingsPage() { /> )}
+ {currentRegistryModel?.legacy && ( + + + {t("currentModel.legacyHint")} + + + )}