* feat: add non-root user to backend and model-server image
* feat: update values to support security context for index, inference, and celery_shared
* feat: add security context support for index and inference
* feat: add celery_shared security context support to celery worker templates
* fix: cache management strategy
* fix: update deployment files for volume mount
* fix: address comments
* fix: bump helm chart version for new security context template changes
* fix: bump helm chart version for new security context template changes
* feat: move useradd earlier in build for reduced image size
---------
Co-authored-by: Phil Critchfield <phil.critchfield@liatrio.com>
* WIP
* renamed and moved tasks (WIP)
* minio migration
* bug fixes and finally add document batch storage
* WIP: can suceed but status is error
* WIP
* import fixes
* working v1 of decoupled
* catastrophe handling
* refactor
* remove unused db session in prep for new approach
* renaming and docstrings (untested)
* renames
* WIP with no more indexing fences
* robustness improvements
* clean up rebase
* migration and salesforce rate limits
* minor tweaks
* test fix
* connector pausing behavior
* correct checkpoint resumption logic
* cleanups in docfetching
* add heartbeat file
* update template jsonc
* deployment fixes
* fix vespa httpx pool
* error handling
* cosmetic fixes
* dumb
* logging improvements and non checkpointed connector fixes
* didnt save
* misc fixes
* fix import
* fix deletion of old files
* add in attempt prefix
* fix attempt prefix
* tiny log improvement
* minor changes
* fixed resumption behavior
* passing int tests
* fix unit test
* fixed unit tests
* trying timeout bump to see if int tests pass
* trying timeout bump to see if int tests pass
* fix autodiscovery
* helm chart fixes
* helm and logging
Changed the restart policy to unless-stopped to ensure containers
automatically restart after failures or reboots but allow manual stop
without immediate restart.
This is preferable over always because it prevents containers from
restarting automatically after a manual stop, enabling controlled
shutdowns and maintenance without unintended restarts.
* k
* functional iam auth
* k
* k
* improve typing
* add deployment options
* cleanup
* quick clean up
* minor cleanup
* additional clarity for db session operations
* nit
* k
* k
* update configs
* docker compose spacing
* fresh indexing feature branch
* cherry pick test
* Revert "cherry pick test"
This reverts commit 2a62422068.
* set multitenant so that vespa fields match when indexing
* cleanup pass
* mypy
* pass through env var to control celery indexing concurrency
* comments on task kickoff and some logging improvements
* use get_session_with_tenant
* comment out all of update.py
* rename to RedisConnectorIndexingFenceData
* first check num_indexing_workers
* refactor RedisConnectorIndexingFenceData
* comment out on_worker_process_init
* fix where num_indexing_workers falls back
* remove extra brace
* set broker_connection_retry_on_startup to silence deprecation warning (we're OK with retrying on startup)
* env var for CELERY_BROKER_POOL_LIMIT
* add redis retry on timeout and health check interval
* set socket_keepalive = True
* remove shadow declaration of REDIS_HEALTH_CHECK_INTERVAL, add socket_keepalive_options where possible
* fix mypy complaint
* pass through vars in docker compose
* remove extra '='
* wrap in a try
* Allow config of background concurrency
* Add comment
* Fix light worker
* use backslashes to continue lines in supervisord with bash
---------
Co-authored-by: Richard Kuo (Danswer) <rkuo@danswer.ai>
* allow setting of CORS origin
* simplify
* add environment variable + rename
* slightly more efficient
* simplify so mypy doens't complain
* temp
* go back to my preferred formatting
* first cut at redis
* fix startup dependencies on redis
* kombu cleanup - fail silently
* mypy
* add redis_host environment override
* update REDIS_HOST env var in docker-compose.dev.yml
* update the rest of the docker files
* update contributing guide
* renaming cache to cache_volume
* add redis password to various deployments
* try setting up pr testing for helm
* fix indent
* hopefully this release version actually exists
* fix command line option to --chart-dirs
* fetch-depth 0
* edit values.yaml
* try setting ct working directory
* bypass testing only on change for now
* move files and lint them
* update helm testing
* some issues suggest using --config works
* add vespa repo
* add postgresql repo
* increase timeout
* try amd64 runner
* fix redis password reference
* add comment to helm chart testing workflow
* rename helm testing workflow to disable it
---------
Co-authored-by: Richard Kuo <rkuo@rkuo.com>
Sam Waddell