starred/onyx
1
0 Fork 0
mirror of https://github.com/onyx-dot-app/onyx.git synced 2026-10-22 03:14:06 -05:00
Code Issues Packages Projects Releases 10 Wiki Activity
9,412 Commits 2,071 Branches 2,537 Tags
f9239e7338586bcfe65aeebdf8a2d2e17837cd45
Commit Graph
220 Commits
Author SHA1 Message Date
Jamison LahmanandClaude Fable 5 e52d68d0ab fix(build): remove dev tools from the default backend image, publish a -dev variant (#13441) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-27 18:05:41 +00:00
Jamison LahmanandClaude Opus 4.8 26868070e6 fix(security): harden default MinIO/S3 object-storage credentials (#12003) 
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-12 19:27:18 +00:00
acaprau 3529e29e98 feat(ods): Per-worktree docker compose isolation with dynamic port allocation (#11337) 2026-05-28 01:54:22 +00:00
acaprau fa8fc678f9 feat(opensearch): Remove Vespa as an option entirely (#11207) 2026-05-20 02:03:45 +00:00
Jamison Lahmanandgreptile-apps[bot] a1ec88269f chore(docker): configurable api_server resource limits (#9424) 
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
 2026-03-17 14:52:21 -07:00
Jamison Lahmanandgreptile-apps[bot] 665640fac8 chore(opensearch): unset container ulimits in dev (#9277) 
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
 2026-03-11 21:58:43 +00:00
acaprau 15da1e0a88 feat(opensearch): Add OpenSearch in docker compose (#8611) 2026-02-21 02:57:31 +00:00
acaprauandgreptile-apps[bot] 7d1b9e4356 chore(opensearch): Migration 2- Introduce external dependency tests (#8045) 
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
 2026-02-02 17:35:49 +00:00
Jamison Lahman 02e17871cc chore(devtools): recommend starting dev dockers with --wait (#7365) 2026-01-12 19:13:00 +00:00
Jamison Lahman 7db7d4c965 chore(docker): publish inference_model_server port 9000 in dev (#7166) 2026-01-02 10:04:45 -08:00
Yuhong Sun 06ce7484b3 chore: docker compose no MCP server (#7100) 2025-12-29 16:40:15 -08:00
Wenxi bc35354ced feat(MCP): basic Onyx MCP server with search tools and indexed sources resource (#6309) 2025-11-25 02:30:55 +00:00
Chris Weaver 33fee46d71 feat: code interpreter (python) (#6136) 2025-11-22 07:19:25 +00:00
Chris WeaverClaudejustin-tahara
 		d186c5e82e feat(docker): Add DEV_MODE flag for exposing service ports (#5499) 
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: justin-tahara <justintahara@gmail.com>
 2025-09-25 15:08:20 -07:00
Yuhong Sun 2e0bc8caf0 feat: Easy Install (#5461) 2025-09-24 15:31:45 -07:00
Wenxiandgreptile-apps[bot] d248d2f4e9 refactor: update seeded docs (#5364) 
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
 2025-09-08 18:06:29 -07:00
Chris Weaver 427945e757 fix: model server build (#5362) 2025-09-08 14:00:33 -07:00
Evan Lohn cc48a0c38e fix: jira cloud api v3 (#5337) 2025-09-04 21:50:35 -07:00
Wenxi 255c2af1d6 feat: reorganize connectors pages (#5186) 
* Add popular connectors sections and cleanup connectors page

* Add other connectors env var

* other connectors env var to vscode env template

* update playwright tests

* sort by popuarlity

* recategorize and sort by popularity
 2025-08-29 16:59:00 -07:00
Sam Waddell c4076d16b6 fix: update all log paths to reflect change related to non-root user (#5244) 2025-08-25 14:11:18 -07:00
Sam WaddellandPhil Critchfield 5867ab1d7d feat: add non-root user to backend and model-server images (#5134) 
* feat: add non-root user to backend and model-server image

* feat: update values to support security context for index, inference, and celery_shared

* feat: add security context support for index and inference

* feat: add celery_shared security context support to celery worker templates

* fix: cache management strategy

* fix: update deployment files for volume mount

* fix: address comments

* fix: bump helm chart version for new security context template changes

* fix: bump helm chart version for new security context template changes

* feat: move useradd earlier in build for reduced image size

---------

Co-authored-by: Phil Critchfield <phil.critchfield@liatrio.com>
 2025-08-20 13:49:50 -07:00
Devin 508b456b40 fix: explicit api_server dependency on minio in docker compose files (#5066) 2025-07-23 13:37:42 -07:00
Evan Lohn bd06147d26 feat: connector indexing decoupling (#4893) 
* WIP

* renamed and moved tasks (WIP)

* minio migration

* bug fixes and finally add document batch storage

* WIP: can suceed but status is error

* WIP

* import fixes

* working v1 of decoupled

* catastrophe handling

* refactor

* remove unused db session in prep for new approach

* renaming and docstrings (untested)

* renames

* WIP with no more indexing fences

* robustness improvements

* clean up rebase

* migration and salesforce rate limits

* minor tweaks

* test fix

* connector pausing behavior

* correct checkpoint resumption logic

* cleanups in docfetching

* add heartbeat file

* update template jsonc

* deployment fixes

* fix vespa httpx pool

* error handling

* cosmetic fixes

* dumb

* logging improvements and non checkpointed connector fixes

* didnt save

* misc fixes

* fix import

* fix deletion of old files

* add in attempt prefix

* fix attempt prefix

* tiny log improvement

* minor changes

* fixed resumption behavior

* passing int tests

* fix unit test

* fixed unit tests

* trying timeout bump to see if int tests pass

* trying timeout bump to see if int tests pass

* fix autodiscovery

* helm chart fixes

* helm and logging
 2025-07-22 03:33:25 +00:00
WenxiandWenxi Onyx 0e79b96091 Feature/revised internet search (#4994) 
* remove unused pruning config

* add env vars

* internet search date time toggle

* revised internet search supporting multiple providers

* env var

* simplify retries and fix mypy issues

* greptile nits

* more mypy

* please mypy

* mypy final straw

* cursor vs. mypy

* simplify fields from provider results

* type-safe prompt, enum nit, provider enums, indexingdoc processing change

---------

Co-authored-by: Wenxi Onyx <wenxi-onyx@Wenxis-MacBook-Pro.local>
 2025-07-14 10:24:03 -07:00
Arun Philip e74a0398dc Update Docker Compose restart policy to unless-stopped 
Changed the restart policy to unless-stopped to ensure containers
automatically restart after failures or reboots but allow manual stop
without immediate restart.

This is preferable over always because it prevents containers from
restarting automatically after a manual stop, enabling controlled
shutdowns and maintenance without unintended restarts.
 2025-06-24 13:27:50 -07:00
joachim-danswer 3272050975 docker dev and prod template (#4936) 
* docker dev and prod template

* more dev files
 2025-06-23 21:43:42 +00:00
Chris Weaver 6f8176092e S3 like file store (#4897) 
* Move to an S3-like file store

* Add non-mocked test

* Add S3 tests

* Improve migration / add auto-running tests

* Refactor

* Fix mypy

* Small fixes

* Improve migration to handle downgrades

* fix file store tests

* Fix file store tests again

* Fix file store tests again

* Fix mypy

* Fix default values

* Add MinIO to other compose files

* Working helm w/ minio

* Fix test

* Address greptile comments

* Harden migration

* Fix README

* Fix it

* Address more greptile comments

* Fix it

* Rebase

* Handle multi-tenant case

* Fix mypy

* Fix test

* fix test

* Improve migration

* Fix test
 2025-06-20 14:22:05 -07:00
joachim-danswerandRei Meguro c5adbe4180 Knowledge Graph v1 (#4626) 
* db setup

* transfer 1 - incomplete

* more adjustments

* relationship table + query update

* temp view creation

* restructuring

* nits

* updates

* separate read_only engine

* extraction revamp

* focus on metadata relatonships 1

* dev

* migration downgrade fix

* rebase migration change

* a3+

* progress

* base

* new extraction

* progress

* fixed KG extraction

* nits

* updates

* simplifications & cleanup

* fixes

* updates

* more feature flag checks

* fixes

* extraction process fix

* read-only user creation as part of setup

* fix for missing entity attributes

* kg read-only user creation as part of migration

* typo

* EL initial comments

* initial Account/SF Connector chnges

* SF Connector update

 - include account information

* base w/ salesforce

* evan updates + quite a bit more

* kg-filtered search

* EL changes pt 2

* migrations and env vars

* quick migration fix

* migration update

* post_rebase fixes

* mypy fixes

* test fixes

* test fix

* test fix

* read_only pool + misc

* nf

* env vars

* test improvements

* salesforce fix

* test update

* small changes

* small adjustments

* SF Connector fix & kg_stage removal for one table

* mypy fix

* small fixes

* EL + RK (pt 1) comments

* nit

* setting updated

* Salesforce test update

* EL comments

* read-only user replacement & cleanup

* SQL View fix

* converting entity type-name separators

* sql view group ownership

* view fix

* SQL tweak

* dealing with docs that were skipped by indexing

* increased error handling

* more error handling

* Output formatting fix

* kg-incremental-reindexing

* 0-doc found improvement

* celery

* migration correction

* timeout adjustments

* nit

* Updated migration

* Entity Normalization for KG Dev 1 (#4746)

* feat: trigrams column

* fix: reranking and db

* feat: v1

* fix: convert to orm

* feat: parallel

* fix: default to id_name

* fix: renamed semantic_id and semantic_id_trigrams

* fix: scalar subquery

* fix: tuning + redundancy

* fix: threshold

* fix: typo

* fix: shorten names

* wip

* fix: reverted

* feat: config

* feat: works but it was dumb

* feat: clustering works

* fix: mypy

* normalization <-> language awareness for SQL generation

* small type fixes

---------

Co-authored-by: joachim-danswer <joachim@danswer.ai>

* mypy

* typo and dead code

* kg_time_fencing

* feat: remove temp views on migration downgrade

* remove functions and triggers for now

* rebase adjustments

* EL code review results

* quick fix + trigger/funcs for single tenant

* fix: typo, mypy, dead code

* fix: autoflake

* small updatesd

* nit

* fix: typo

* early + faster view creation

* Extension creation in MT migration

* nit changes to default ETs

* Incremental Clustering and KG Refactor V1 (#4784)

Optimized/restructured incremental clustering. New pipeline actually that moves vespa updates to clustering.
Also, celery configuration has been updated.
---------

Co-authored-by: joachim-danswer <joachim@danswer.ai>

* prompt tweak & ET extraction reset

* more general hierarchical structure

* feat: better vespa reset logic

* prompt optimization and entity replacemants

* small prompt changes

* KG Refactor V2 (#4814)

Clustering & Extraction improvements & various nits 

Co-authored-by: joachim-danswer <joachim@danswer.ai>

* add connector-level coverage days

* fix: nit

* initial  EL responses

* refactor: helper functions for formatting

* fix: more helper fns & comments

* fix: comment code that's been implemented elsewhere

* fix: tenant_id missing arg

* fix: removed debugging stuff

* fix: moved kg_interactions db query to helper fn

* fix: tenant_id

* fix: tenant_id & removed outdated helper fn

* fix always set entity class

* fix: typo

* fix alembic heads

* fix: celery logging

* fix: migrations fix

* fix: multi tenant permissions

* fix: temp connector fix

* fix: downgrade

* Fix upgrade migration

* fix: tenant for normalization

* added additional acl

* stray EL comments

* fix: connector test

* fix mypy

* fix: temporary connector test fix

* fix: jira connector test

* nit

* small nits

* fix: black

* fix: mypy

* fix: mypy

---------

Co-authored-by: Rei Meguro <36625832+Orbital-Web@users.noreply.github.com>
 2025-06-07 23:14:20 +00:00
rkuo-danswerandRichard Kuo 51be9000bb Feature/vespa bump (#4819) 
* bump cloudformation

* update kubernetes

* bump helm chart

* bump docker compose

* update chart.lock

* ai accident!

* bump vespa helm chart for fix

* increase timeout

---------

Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app>
 2025-06-04 00:03:01 +00:00
Evan Lohn 4c71a5f5ff drive perm sync logs + misc deployment improvements (#4788) 
* some logs

* give postgress more memory

* give postgress more memory

* give postgress more memory

* revert

* give postgress more memory

* bump external access limit

* vespa timeout

* deployment consistency

* bump vespa version

* skip upgrade check

* retry permission by ids

* logs

* fix temp docx file issue

* fix drive file deduping

* RK comments

* mypy

* aggregate logs
 2025-06-01 23:36:57 +00:00
rkuo-danswerandRichard Kuo 2282f6a42e fix restart (#4726) 
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app>
 2025-05-17 17:02:27 +00:00
Chris Weaver 70df685709 Non default schema fix (#4667) 
* Use correct postgres schema

* Remove raw Session() use

* Refactor + add test

* Fix comment
 2025-05-06 20:35:59 -07:00
Weves 72ce2f75cc Add env var to docker compose file 2025-04-13 23:14:06 -07:00
Chris Weaver e3aab8e85e Improve index attempt display (#4511) 2025-04-13 15:57:47 -07:00
Weves 614d0f8d72 Add more options to dev compose file 2025-04-07 10:03:34 -07:00
Chris Weaver 7648627503 Save all logs + add log persistence to most Onyx-owned containers (#4368) 
* Save all logs + add log persistence to most Onyx-owned containers

* Separate volumes for each container

* Small fixes
 2025-03-26 22:25:39 -07:00
Chris Weaver 0d3c72acbf Add basic memory logging (#4234) 
* Add basic memory logging

* Small tweaks

* Switch to monotonic
 2025-03-08 03:49:47 +00:00
Chris Weaver f731beca1f Add ONYX_QUERY_HISTORY_TYPE to the dev compose files (#4196) 2025-03-05 17:34:55 +00:00
pablonyx 5a131f4547 Fix integration tests (#4059) 2025-02-21 15:56:11 -08:00
Chris Weaver f1fc8ac19b Connector checkpointing (#3876) 
* wip checkpointing/continue on failure

more stuff for checkpointing

Basic implementation

FE stuff

More checkpointing/failure handling

rebase

rebase

initial scaffolding for IT

IT to test checkpointing

Cleanup

cleanup

Fix it

Rebase

Add todo

Fix actions IT

Test more

Pagination + fixes + cleanup

Fix IT networking

fix it

* rebase

* Address misc comments

* Address comments

* Remove unused router

* rebase

* Fix mypy

* Fixes

* fix it

* Fix tests

* Add drop index

* Add retries

* reset lock timeout

* Try hard drop of schema

* Add timeout/retries to downgrade

* rebase

* test

* test

* test

* Close all connections

* test closing idle only

* Fix it

* fix

* try using null pool

* Test

* fix

* rebase

* log

* Fix

* apply null pool

* Fix other test

* Fix quality checks

* Test not using the fixture

* Fix ordering

* fix test

* Change pooling behavior
 2025-02-16 02:34:39 +00:00
Yuhong Sun ab8081c36b k 2025-02-15 13:42:43 -08:00
Weves af2061c4db Add Linear OAuth env variables to dev compose 2025-01-04 16:02:04 -08:00
Chris Weaver d64464ca7c Add support for OAuth connectors that require user input (#3571) 
* Add support for OAuth connectors that require user input

* Cleanup

* Fix linear

* Small re-naming

* Remove console.log
 2024-12-31 18:03:33 -08:00
pablonyx e2700b2bbd Remove left over yaml errors (#3527) 
* remove left over vim command

* additional misconfigurations

* ensure all regions updated
 2024-12-29 01:45:07 +00:00
pablonyx aa4bfa2a78 Forgot password feature (#3437) 
* forgot password feature

* improved config

* nit

* nit
 2024-12-20 04:53:37 +00:00
Chris Weaver 23ecf654ed Add support for custom LLM error messages (#3501) 
* Add support for custom LLM error messages

* Fix mypy
 2024-12-17 22:58:17 -08:00
pablonyx 8db6d49fe5 IAM Auth for RDS (#3479) 
* k

* functional iam auth

* k

* k

* improve typing

* add deployment options

* cleanup

* quick clean up

* minor cleanup

* additional clarity for db session operations

* nit

* k

* k

* update configs

* docker compose spacing
 2024-12-17 22:02:37 +00:00
pablodanswer a6ab8a8da4 organize fe tests 2024-12-13 12:06:26 -08:00
pablodanswer 21ec5ed795 welcome to onyx 2024-12-13 09:56:10 -08:00
Weves 056b671cd4 Small tweaks to get Egynte to work on our cloud 2024-12-10 17:43:46 -08:00