* move api-based embeddings/reranking calls to api server out of model server, added/modified unit tests
* ran pre-commit
* fix mypy errors
* mypy and precommit
* move utils to right place and add requirements
* precommit check
* removed extra constants, changed error msg
* Update backend/onyx/utils/search_nlp_models_utils.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* greptile
* addressed comments
* added code enforcement to throw error
---------
Co-authored-by: Jessica Singh <jessicasingh@Mac.attlocal.net>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* feat: add non-root user to backend and model-server image
* feat: update values to support security context for index, inference, and celery_shared
* feat: add security context support for index and inference
* feat: add celery_shared security context support to celery worker templates
* fix: cache management strategy
* fix: update deployment files for volume mount
* fix: address comments
* fix: bump helm chart version for new security context template changes
* fix: bump helm chart version for new security context template changes
* feat: move useradd earlier in build for reduced image size
---------
Co-authored-by: Phil Critchfield <phil.critchfield@liatrio.com>
* bump fastapi and starlette
* bumping llama index and nltk and associated deps
* bump to fix python-multipart
* bump aiohttp
* update package lock for examples/widget
* bump black
* sentencesplitter has changed namespaces
* fix reorder import check, fix missing passlib
* update package-lock.json
* black formatter updated
* reformatted again
* change to black compatible reorder
* change to black compatible reorder-python-imports fork
* fix pytest dependency
* black format again
* we don't need cdk.txt. update packages to be consistent across all packages
---------
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app>
Co-authored-by: Richard Kuo <rkuo@rkuo.com>
* pass through various id's and log them in the model server for better tracking
* fix test
---------
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app>
* add prometheus metrics endpoints via helper package
* model server specific requirements
* mark as public endpoint
---------
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app>
* stubbing out request id
* passthru or create request id's in api and model server
* add onyx request id
* get request id logging into uvicorn
* no logs
* change prefixes
* fix comment
* docker image needs specific shared files
---------
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app>
* remove title for slack
* initial working code
* simplification
* improvements
* name change to information_content_model
* avoid boost_score > 1.0
* nit
* EL comments and improvements
Improvements:
- proper import of information content model from cache or HF
- warm up for information content model
Other:
- EL PR review comments
* nit
* requirements version update
* fixed docker file
* new home for model_server configs
* default off
* small updates
* YS comments - pt 1
* renaming to chunk_boost & chunk table def
* saving and deleting chunk stats in new table
* saving and updating chunk stats
* improved dict score update
* create columns for individual boost factors
* RK comments
* Update migration
* manual import reordering
* Update text embedding model to version 005 and enhance embedding retrieval process
* re
* Fix formatting issues
* Add support for Bedrock reranking provider and AWS credentials handling
* fix: improve AWS key format validation and error messages
* Fix vertex embedding model crash
* feat: add environment template for local development setup
* Add display name for Claude 3.7 Sonnet model
* Add display names for Gemini 2.0 models and update Claude 3.7 Sonnet entry
* Fix ruff errors by ensuring lines are within 130 characters
* revert to currently default onyx browser settings
* add / fix boto requirements
---------
Co-authored-by: ferdinand loesch <f.loesch@sportradar.com>
Co-authored-by: Ferdinand Loesch <ferdinandloesch@me.com>
Co-authored-by: Richard Kuo (Danswer) <rkuo@onyx.app>
* improve model server logging
* improve exception logging with provider/model names
* get everything into one log line
---------
Co-authored-by: Richard Kuo <rkuo@rkuo.com>
