Commit Graph
94 Commits
Author SHA1 Message Date
4a8c42b789 feat: per-user usage + cost tracking — accounting (2/4) [modifying #11887] (#12551)
Co-authored-by: Berat Erkan Elcelik <beraterkanelcelik@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 21:39:37 +00:00
Jamison Lahman 46af76e98a chore(fmt): remove force-single-line (#12996) 2026-07-20 17:15:00 +00:00
Justin Tahara f81a261862 fix(celery): model-server request timeouts + wire the orphaned hierarchy queue (#12964) 2026-07-13 22:14:03 +00:00
SubashMohanandClaude Opus 4.8 cba40dab12 feat(reindex): reindex-port flow — copy & re-embed live chunks into the new index (#11977)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 15:46:53 +00:00
Justin Tahara aef147889e feat(tracing): DB-backed tracing provider config with live env fallback (PR 1/3) (#12529) 2026-07-01 21:32:23 +00:00
7dc62c3483 feat(web-search): add Tavily as a web-search and content provider (#12350)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Danelegend <43459662+Danelegend@users.noreply.github.com>
Co-authored-by: Dane Urban <danelegend13@gmail.com>
2026-06-24 20:02:10 +00:00
roshanandClaude Opus 4.8 4299c92af2 fix(craft): set readOnlyRootFilesystem on the sandbox egress proxy (#12218)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 01:34:03 +00:00
Justin Tahara b901cd7677 fix(CORS): Never pair wildcard CORS origins with allow_credentials (ENG-4126) (#11948) 2026-06-11 19:56:35 +00:00
Justin Tahara 6743a52eff feat(logging): opt-in structured JSON logs via LOG_FORMAT=json (#11490) 2026-05-28 19:31:35 +00:00
Jamison Lahman da1c09e762 chore(security): enable ruff flake8-bandit (S) rules (#11153) 2026-05-18 18:48:18 +00:00
Jamison LahmanandClaude Opus 4.7 f73c5cdd64 chore(devtools): migrate from black to ruff format (#11080)
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 04:41:31 +00:00
Jamison Lahman 1dc20f3511 chore(devtools): replace autoflake and reorder-python-imports with ruff (#10489) 2026-04-22 20:59:15 -07:00
Justin Tahara 2b04b2ec80 chore(sentry): make trace sample rates env-configurable, default celery to 0 (#10374) 2026-04-20 18:40:13 +00:00
SubashMohan 146b5449d2 feat: configurable file upload size and token limits via admin settings (#9232) 2026-03-27 04:23:16 +00:00
Jamison Lahman 7b927e79c2 chore(format): format files where ruff and black agree (#9339) 2026-03-13 20:18:49 +00:00
Wenxi 787cf90d96 chore: set trial api usage to 0 and show ui (#8664) 2026-02-23 01:41:23 +00:00
Justin Tahara 5b54687077 feat(web): Initial Framework for Brave Web Search 1/3 (#8594) 2026-02-20 01:38:19 +00:00
Nikolas Garza f59792b4ac feat(metrics): add SQLAlchemy connection pool Prometheus metrics (#8543) 2026-02-19 02:32:42 +00:00
Jamison Lahman d8fd6d398e chore(ruff): enable flake8s unused arg rules (#8206) 2026-02-05 23:25:07 +00:00
Evan Lohn ffd30ae72a chore: bump default usage limits (#8188) 2026-02-05 19:43:46 +00:00
Jessica Singh 9ccbb6a04b feat(web search): exa crawler (#7326) 2026-01-13 01:42:16 +00:00
Evan Lohn 6256b2854d chore: bump indexing usage (#7307) 2026-01-09 17:46:27 +00:00
Evan Lohn c23046f7c0 chore: bump limits on cloud LLM usage (#7195) 2026-01-04 21:38:13 +00:00
Evan Lohn d5f66ac146 feat: cloud usage limits (#7192) 2026-01-04 06:51:12 +00:00
Yuhong Sun 1dfb62bb69 chore: Remove unused resources from model server (#7094) 2025-12-29 16:18:37 -08:00
Justin Tahara e507378244 fix(vertex-ai): Bump Default Batch Size (#6982) 2025-12-22 17:21:55 -08:00
trial-danswerandWeves 7181cc41af feat: adding support for SearXNG as an option for web search. It operates a… (#6653)
Co-authored-by: Weves <chrisweaver101@gmail.com>
2025-12-17 12:27:19 -08:00
きわみざむらいandJamison Lahman 81cb0f2518 fix: Add proper DISABLE_MODEL_SERVER environment variable support (#6468)
Co-authored-by: Jamison Lahman <jamison@lahman.dev>
2025-12-02 21:11:09 +00:00
Jessica Singh a368556282 feat(web search providers): adding support and changing env var approach (#6273) 2025-11-19 02:49:54 +00:00
SubashMohan e8da5722df feat(upload): increase token limit to 100k and configurable skip (#6187) 2025-11-12 09:53:37 -08:00
Richard Guanandgreptile-apps[bot] 2c2f6e7c23 feat(framework): simple agent to feature flags (#5692)
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-10-14 20:27:07 -07:00
8c52444bda fix(infra): lazy load and don't warm up model server models (#5527)
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-10-07 09:07:40 -07:00
Evan Lohn da02962a67 fix: thread safe approach to docprocessing logging (#5185)
* thread safe approach to docprocessing logging

* unify approaches

* reset
2025-08-12 02:25:47 +00:00
Chris Weaver 67c488ff1f Improve support for non-default postgres schemas (#5046) 2025-07-17 23:51:39 -07:00
Chris Weaverandgreptile-apps[bot] dc542fd7fa Enable default quantization (#4815)
* Adjust migration

* update default in form

* Add cloud indices for bfloat16

* Update backend/shared_configs/configs.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update vespa schema gen script

* Move embedding configs

* Remove unused imports

* remove import from shared configs

* Remove unused model

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-06-05 14:02:08 -07:00
Chris Weaver 7763e2fa23 Fix non-default schema in KV store (#4655)
* Fix non-default schema in KV store

* Fix custom schema
2025-05-04 22:19:35 +00:00
pablonyxandWeves df67ca18d8 My docs cleanup (#4519)
* update

* improved my docs

* nit

* nit

* k

* push changes

* update

* looking good

* k

* fix preprocessing

* try a fix

* k

* update

* nit

* k

* quick nits

* Cleanup / fixes

* Fixes

* Fix build

* fix

* fix quality checks

---------

Co-authored-by: Weves <chrisweaver101@gmail.com>
2025-04-25 05:20:33 +00:00
Chris Weaver a5edc8aa0f Fix default log level (#4501)
* Fix default log level

* fix
2025-04-14 16:40:11 +00:00
24184024bb Bugfix/dependency updates (#4482)
* bump fastapi and starlette

* bumping llama index and nltk and associated deps

* bump to fix python-multipart

* bump aiohttp

* update package lock for examples/widget

* bump black

* sentencesplitter has changed namespaces

* fix reorder import check, fix missing passlib

* update package-lock.json

* black formatter updated

* reformatted again

* change to black compatible reorder

* change to black compatible reorder-python-imports fork

* fix pytest dependency

* black format again

* we don't need cdk.txt. update packages to be consistent across all packages

---------

Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app>
Co-authored-by: Richard Kuo <rkuo@rkuo.com>
2025-04-10 08:23:02 +00:00
rkuo-danswerandRichard Kuo 81d04db08f Feature/request id middleware 2 (#4427)
* stubbing out request id

* passthru or create request id's in api and model server

* add onyx request id

* get request id logging into uvicorn

* no logs

* change prefixes

* fix comment

* docker image needs specific shared files

---------

Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app>
2025-04-02 22:30:03 +00:00
evan-danswer 56f8ab927b Contextual Retrieval (#4029)
* contextual rag implementation

* WIP

* indexing test fix

* workaround for chunking errors, WIP on fixing massive memory cost

* mypy and test fixes

* reformatting

* fixed rebase
2025-03-30 18:49:09 +00:00
joachim-danswer 463340b8a1 Reduce ranking scores for short chunks without actual information (#4098)
* remove title for slack

* initial working code

* simplification

* improvements

* name change to information_content_model

* avoid boost_score > 1.0

* nit

* EL comments and improvements

Improvements:
  - proper import of information content model from cache or HF
  - warm up for information content model

Other:
  - EL PR review comments

* nit

* requirements version update

* fixed docker file

* new home for model_server configs

* default off

* small updates

* YS comments - pt 1

* renaming to chunk_boost & chunk table def

* saving and deleting chunk stats in new table

* saving and updating chunk stats

* improved dict score update

* create columns for individual boost factors

* RK comments

* Update migration

* manual import reordering
2025-03-13 17:35:45 +00:00
870b59a1cc Bugfix/vertex crash (#4181)
* Update text embedding model to version 005 and enhance embedding retrieval process

* re

* Fix formatting issues

* Add support for Bedrock reranking provider and AWS credentials handling

* fix: improve AWS key format validation and error messages

* Fix vertex embedding model crash

* feat: add environment template for local development setup

* Add display name for Claude 3.7 Sonnet model

* Add display names for Gemini 2.0 models and update Claude 3.7 Sonnet entry

* Fix ruff errors by ensuring lines are within 130 characters

* revert to currently default onyx browser settings

* add / fix boto requirements

---------

Co-authored-by: ferdinand loesch <f.loesch@sportradar.com>
Co-authored-by: Ferdinand Loesch <ferdinandloesch@me.com>
Co-authored-by: Richard Kuo (Danswer) <rkuo@onyx.app>
2025-03-05 01:59:46 +00:00
Chris Weaver f25e1e80f6 Add option to not re-index (#4157)
* Add option to not re-index

* Add quantizaton / dimensionality override support

* Fix build / ut
2025-03-03 10:54:11 -08:00
pablonyx a98dcbc7de Update tenant logic (#4122)
* k

* k

* k

* quick nit

* nit
2025-02-26 03:53:46 +00:00
pablonyx 47fd4fa233 Strict Tenant ID Enforcement (#3871)
* strict tenant id enforcement

* k

* k

* nit

* merge

* nit

* k
2025-02-19 00:52:56 +00:00
Chris Weaver a5d2f0d9ac Fix airtable connector w/ mt cloud + move telem logic to match new st… (#3868)
* Fix airtable connector w/ mt cloud + move telem logic to match new standard

* Address Greptile comment

* Small fixes/improvements

* Revert back monitoring frequency

* Small monitoring fix
2025-01-31 16:29:04 -08:00
Richard Kuo 174e5968f8 we didn't want to rename the alt index suffix, reverting 2024-12-31 11:28:11 -08:00
pablodanswer 21ec5ed795 welcome to onyx 2024-12-13 09:56:10 -08:00
Weves 18e63889b7 Change default log level back to info 2024-12-06 10:07:14 -08:00