mirror of
https://github.com/onyx-dot-app/onyx.git
synced 2026-10-23 11:53:08 -05:00
4b72f53be27103d68c19c2c193608388d5cfd083
11031
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 roshan
|
4b72f53be2 | refactor(web): use Section for the Plugins row chevron slot | ||
|
roshan
|
49a07f8cdc | fix(web): open the Plugins submenu over the tools menu on phones | ||
|
roshan
|
05c989948f | fix(web): align plugin rows and the Plugins entry in the tools menu | ||
|
roshan
|
d02900dff9 | fix(web): stack the Chat Preferences connectors header on narrow screens | ||
|
roshan
|
5c8028d94a | fix(web): list every connector source in Chat Preferences and explain the section | ||
|
roshan
|
29b59437a1 | fix(chat): keep the error icon on blocked custom tool steps | ||
|
roshan
|
f13fe0b3f5 | test(chat): cover blocked and failed custom tool statuses | ||
|
roshan
|
549697ca19 | fix(chat): keep the blocked tool status visible on the error surface | ||
|
roshan
|
035639897f | fix(chat): show blocked and redirected custom tool calls as failed | ||
|
roshan
|
85451dc407 | fix(web): clear the stale OpenAPI schema error when the definition changes | ||
|
roshan
|
ea5d22d7b6 | fix(web): show the OpenAPI schema validation error once | ||
|
roshan
|
be63963a3d | fix(web): use plugin wording in the OpenAPI auth tooltip and Spanish help | ||
|
roshan
|
d1f93aedbf | fix(web): call OpenAPI plugins plugins, not actions, in the Plugins flow | ||
|
Raunak Bhagat
|
5159bd8287 |
feat(web): rework the cost-overrides panel and modal (#16324)
## Description
Reworks the Cost Overrides panel on the admin Language Models page.
- **Modal:** adding and editing happen in a "New / Edit Cost Override"
modal instead of an inline form. It runs on Formik with a Yup schema,
like the language-model provider modals: Save / Add stay disabled until
the form is valid and changed, and Enter submits.
- **Fields:** `InputHorizontal` rows with short titles and descriptions.
The three rates use `InputCurrency` (2 decimals). The edit path shows
the override's model in the model selector, disabled.
- **Cards:** each override is a `SelectCard` like the provider cards:
model icon, model name, provider and update date, the rates on a line
below, and trash / settings buttons beside the title. Delete now asks
for confirmation.
- **Search focus:** `InputSingleSelect` and `SimpleModelSelector` take a
`container`, passed to the `Dropdown`. The modal passes its content
element so the model search field keeps focus inside the dialog's focus
trap.
- Copy updated in all nine locales; unused placeholder keys removed.
UI only: it works with the current cost-override API. The data-model
change (overrides referencing `model_configuration`) comes in a separate
PR.
## Changes
### No overrides
<img width="867" height="203" alt="image"
src="https://github.com/user-attachments/assets/4ad5ee5a-27b9-49f1-b3e3-b7d0eb1140db"
/>
### >1 overrides
<img width="876" height="226" alt="image"
src="https://github.com/user-attachments/assets/0031f8cb-8f55-4d92-a809-c9fdf3f6e960"
/>
### Creating new override
<img width="655" height="420" alt="image"
src="https://github.com/user-attachments/assets/c700c36b-e099-4d98-b71e-855ff10f56e6"
/>
### Editing existing override
<img width="660" height="429" alt="image"
src="https://github.com/user-attachments/assets/121a09fc-5653-4435-894e-1c23cb002819"
/>
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Reworks the Cost Overrides panel on the admin Language Models page:
adding and editing now happen in a New/Edit Cost Override modal, and
each override renders as a card matching the provider cards above it. UI
only — the overrides API is unchanged.
**New Features**
- Add/edit moved from an inline form to a Formik + Yup modal; Add/Save
stays disabled until the form is valid and changed, and Enter submits.
Fields use `InputHorizontal` with short titles and unit descriptions.
- Rates use `InputCurrency` with unlimited precision, so sub-cent
negotiated rates ($0.075, $0.01875) save and display exactly; each card
lists In, Out, and (when set) Cache as separate items on a line under
the description. The edit path shows the override's model in a disabled
selector, or a read-only field when no configured model matches.
- Delete now asks for confirmation, disables itself while the request
runs, and shows success/error toasts; copy is updated in all nine
locales, unused placeholder keys were removed, and the API error path
now reuses `parseErrorDetail`.
- Tests that drove a mocked model picker were dropped: they didn't
exercise the real modal.
**Bug Fixes**
- `InputSingleSelect` and `SimpleModelSelector` accept a `container`
passed to the `Dropdown`, so the model search field keeps focus inside
the modal's focus trap.
<sup>Written for commit
|
||
|
Wenxi
|
fa9decff34 |
fix(devcontainer): use the main Ubuntu archive for ARM64 (#16363)
## Description Fix ARM64 devcontainer builds that fail to install Playwright browser dependencies because the ports archive lacks required packages. Ubuntu 26.04 moved ARM64 packages to the main archive ([Ubuntu announcement](https://discourse.ubuntu.com/t/ubuntu-on-arm-summer-26-update/84872)). The pinned base image still uses ports. Update its ARM64 repository URL before installing packages. Other architectures keep their existing sources. The main archive contains matching `xvfb` and `xserver-common` packages. This change uses normal Ubuntu package downloads and signature verification. ## How Has This Been Tested? - Verified ARM64 package index hashes against published InRelease metadata. - Downloaded matching `xvfb` and `xserver-common` packages and verified their hashes. - Pre-commit checks passed. - [Fresh ARM64 devcontainer CI build passed](https://github.com/onyx-dot-app/onyx/actions/runs/39534888216/job/119009199000) in 6 minutes. Playwright dependencies installed in 15.7 seconds, including matching `xvfb` and `xserver-common` packages. ## Additional Options - [ ] [Optional] Please cherry-pick this PR to the latest release version. |
||
|
Raunak Bhagat
|
610809c564 |
feat(opal): type and paste into InputDatePicker like InputCurrency (#16364)
## Description
`InputDatePicker` now types and pastes like `InputCurrency`.
- Typing: a separator key (`/`, `-` or `.`) in the month or day splits
the segment at the caret. Digits before it stay and get padded (`1`
becomes `01`). Digits after it move to the front of the next segment,
and focus moves there. So `1/2/2026` types as it reads. Typing past a
full segment is blocked. Other non-digits do nothing.
- Pasting reads the text as a date in any segment:
- a whole date fills every segment: `10/19/2026`, `10-19-2026`,
`10.19.2026` or ISO `2026-10-19`;
- digits alone go into the focused segment at the caret, clipped to fit;
- anything else (`10/19`, `Oct 19 2026`, `1/2/26`) is ignored.
- Arabic-Indic digits and invisible direction marks are handled as in
`InputCurrency`.
- A pasted date commits like a typed one, on blur or Enter.
`toAsciiDigits`, `cleanPastedText` and `usePendingCaret` move from
`InputCurrency` into `inputs/segmented.tsx`, so both fields share one
implementation. `InputTime` also reads Arabic-Indic digits now.
## Changes
TODO
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
`InputDatePicker` now types and pastes like `InputCurrency`, so a date
can be entered as it reads.
- Typing a separator (`/`, `-`, `.`) in the month or day splits the
segment at the caret: digits before it stay and get padded (`1` → `01`),
digits after it move to the front of the next segment, and focus moves
there, so `1/2/2026` fills all three segments. A separator in an empty
segment does nothing, keeping `10/19/2026` typing as it reads. Other
non-digits do nothing, and typing past a full segment is blocked.
- Pasting reads the text as a date in any segment: a whole date
(`10/19/2026`, `10-19-2026`, `10.19.2026`, or ISO `2026-10-19`) fills
every segment, digits alone go into the focused segment at the caret
clipped to fit, and anything else (`10/19`, `Oct 19 2026`, `1/2/26`) is
ignored. A pasted date commits like a typed one, on blur or Enter.
- Arabic-Indic digits and invisible direction marks are handled as in
`InputCurrency`.
The digit, paste-cleanup, and caret helpers move from `InputCurrency`
into `inputs/segmented.tsx` so both fields share one implementation; the
caret helper is now typed on the segments shape. `InputTime` also reads
Arabic-Indic digits now.
<sup>Written for commit
|
||
|
Raunak Bhagat
|
d8a6fe50b6 |
feat(opal): add @opal/InputCurrency (#16323)
## Description
Adds `InputCurrency` to Opal: a segmented amount field that reads `$
[whole] . [fraction]` on the shared segmented-input chrome.
- `currency` (ISO 4217, default `USD`): the host locale sets the symbol,
its side (`3,00 €`) and the decimal separator, through a new
`OpalStrings.inputCurrencyParts`. `getCurrencyParts(locale, currency)`
is exported for the app's `OpalStringsBridge`.
- `precision` (default: the currency's usual decimals): the fraction
accepts at most this many digits; typing more is blocked. `0` hides the
fraction.
- The whole segment always resolves: leading zeros trim as soon as focus
leaves it (`0000` → `0`).
- Keyboard: `.` / `,` jump to the fraction, pasting `12.50` fills both
segments, arrows cross segment edges. Inside a form, Enter submits from
either segment; outside a form, Enter steps to the fraction.
- `onChange` fires on every edit (every draft is a valid amount); blur
and Enter tidy the display (`3.5` → `3.50`).
- New `inputs/numbers/` group: `InputCurrency` lives there and
`InputNumber` moves in beside it.
- New Opal strings for the field and segment labels, in all nine
locales.
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Adds `@opal/InputCurrency`, a segmented currency amount field that reads
`$ [whole] . [fraction]` on the shared segmented-input chrome, with the
symbol, its side, and the decimal separator driven by the host locale.
**New Features**
- Typing `.` or `,` jumps to the fraction and splits the amount at the
caret: an empty whole part fills `0` first (so `.5` reads `0.5`), digits
after the caret move to the front of the fraction clipped to the
precision, dropping any padded zeros (`1|234` on `1234.00` reads `1 .
234`), and with no fraction segment the separator does nothing. Other
non-digits, grouping keys included, do nothing. Pasting reads the text
as an amount: it drops whitespace, direction marks, grouping, and the
symbol or code, fills both segments (`12.50`) or inserts into the
focused segment, and ignores anything that is not an amount.
- `precision` caps the fraction digits (`0` hides the fraction,
`"unlimited"` and larger numbers clamp to 15, the most digits a JS
number holds exactly) and `currency` is typed as an ISO 4217
`CurrencyCode`; either change rebuilds the segments. Display rounds half
up from the shortest decimal form (0.075 shows 0.075, or 0.08 at
precision 2); leading zeros trim as soon as the whole segment loses
focus.
- `onChange` fires on every edit; blur and Enter tidy the display; Enter
submits a surrounding form from either segment.
- `getCurrencyParts(locale, currency)` is exported for
`OpalStringsBridge`, with new strings in all nine locales.
- `InputNumber` moves into a new `inputs/numbers/` group next to it.
<sup>Written for commit
|
||
|
claude[bot]andClaude
|
52edd0986e |
fix(deps): bump certifi to 2026.10.19 in sandbox image requirements (#16361)
<!-- claude-slack-attribution -->
_Requested by **Nikolas Garza** · [Slack
thread](https://onyx-company.slack.com/archives/C0C5SMAM4TT/p1792538621965719?thread_ts=1792538621.965719&cid=C0C5SMAM4TT)_
## Description
Before: `initial-requirements.txt` pins `certifi==2026.6.17`, which has
Dependabot alert GHSA-6q48-6c7w-9gw2.
After: `initial-requirements.txt` pins `certifi==2026.10.19`, the fixed
version. It matches `uv.lock`.
This changes one line in the sandbox image requirements.
How: The CVE bot cannot edit this uv-pip-compile file, so #16337 missed
it. This fixes the remaining certifi pin flagged by GHSA-6q48-6c7w-9gw2.
Only the certifi line changes. Other pins stay the same.
Linear: none
## How Has This Been Tested?
Checked that certifi 2026.10.19 exists on PyPI and that `uv.lock` uses
it. The file has no hashes, and no other pin caps certifi.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01E2K27DFf4J3iAGXYpe4xtR)_
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Fixes the Dependabot alert GHSA-6q48-6c7w-9gw2 by bumping `certifi` from
2026.6.17 to 2026.10.19 in the sandbox image `initial-requirements.txt`.
The new pin matches `uv.lock` and only the `certifi` line changes.
<sup>Written for commit
|
||
|
claude[bot]andClaude
|
4f69237d85 |
ci(cve-alerts): let the fix bot edit compiled requirements files (#16362)
<!-- claude-slack-attribution -->
_Requested by **Nikolas Garza** · [Slack
thread](https://onyx-company.slack.com/archives/C0C5SMAM4TT/p1792538621965719?thread_ts=1792538621.965719&cid=C0C5SMAM4TT)_
Before: The CVE Alerts fix bot could edit only `package.json` and
`pyproject.toml` (plus regenerated lockfiles). It could not fix a pin in
`initial-requirements.txt`, so the audit kept failing.
After: The bot can also edit compiled requirements files
(`*requirements*.txt`, today only `initial-requirements.txt`). It still
rejects source code, workflows, and all other files.
This lets the bot fix a pin in any manifest file that the audit can fail
on and that the workflow can safely validate.
How: PR #16337 missed a certifi pin in `initial-requirements.txt`, and
#16361 fixed it by hand. The change adds `**/*requirements*.txt` to the
editable file list and to the patch allowlist in `cve-alerts.yml`.
`cve-check-manifests.ts` now checks `.txt` edits: only the package's
`==` line may change, and only to the fixed version. `cve-fix.md` tells
the bot to edit that line and keep the header and `# via` comments.
`backend/requirements/*.txt` stay workflow-regenerated and off-limits to
the bot. Go and Rust alerts stay out of scope, because the bot has no
regeneration step for them.
Linear: none
---
_Generated by [Claude
Code](https://claude.ai/code/session_01E2K27DFf4J3iAGXYpe4xtR)_
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Lets the CVE Alerts fix bot edit compiled requirements files so it can
fix a pin in `initial-requirements.txt` instead of leaving the audit
failing.
- Requirements files are validated line by line: only the package's
plain `==` pin may change to the fixed version, byte-identical
elsewhere.
- `backend/requirements/*.txt` stay workflow-regenerated and off-limits;
source code, workflows, and other files are still rejected.
<sup>Written for commit
|
||
|
Evan LohnandClaude Opus 5.5
|
ae9b863612 |
feat(jira): re-fetch failed documents by id (Resolver) (#16102)
## Description
`JiraConnector` now implements `Resolver.reindex`. Targeted reindex can
now re-fetch failed Jira documents by id.
**Document types covered**
- Jira issues. The issue is the only document type the connector writes.
Comments are part of the issue text, and the connector does not index
attachments. The document id is the issue URL (`<base>/browse/<KEY>`).
**Document types not covered**
- None.
**How it works**
- `reindex` parses the issue key from each document URL. The URL must
start with the connector's own `<base>/browse/`, and the key must be a
valid Jira key. Other ids yield a `ConnectorFailure` with the input id.
- It fetches the issues with one JQL search per batch of 50 keys: `key
in (...)`, limited to the configured project or JQL (new `scope_jql`
helper, which `build_jql_query` now uses too). The search uses the same
paths as the crawl (Cloud: enhanced search + bulk fetch; Server/DC: v2
search).
- Server/DC rejects a JQL query that names a missing key. When a batch
fails, the connector fetches each key alone, so one bad key does not
fail the rest. A 401 or 403 still fails the whole run, as in the crawl.
- Each input id that does not give a document yields a
`ConnectorFailure` with the reason: fetch error, conversion error,
skipped by label or size, or not found in scope (deleted, moved, or out
of scope).
- Entity failures (`failed_document is None`) are skipped, as in
Confluence.
- The per-issue conversion (hierarchy nodes, then the document) moves
out of `_load_from_checkpoint` into `_issue_to_items`. The crawl and
`reindex` both use it. `reindex` yields the project node and the Epic
nodes before the documents.
**Permissions**
- With `include_permissions=True`, `reindex` sets `external_access` with
`_get_project_permissions(project_key, add_prefix=True)`. This is the
same call that `load_from_checkpoint_with_perm_sync` uses, and it caches
each project.
## How Has This Been Tested?
- New daily test
`backend/tests/daily/connectors/jira/test_jira_resolver.py`, against the
live CI site (project AS):
`uv run --env-file .vscode/.env pytest -xv
backend/tests/daily/connectors/jira/test_jira_resolver.py` -> 9 passed.
It covers: reindex of one issue matches the crawl output and hierarchy
nodes; Story and Epic together (classic and scoped token); missing key;
out-of-scope issue (JQL scope); unparseable ids; empty input;
entity-only input; `include_permissions=True` sets `external_access`.
- New unit test
`backend/tests/unit/onyx/connectors/jira/test_jira_reindex.py` for the
Server/DC per-key fallback and the 401 path. All Jira unit tests after
the rebase onto main: 141 passed. The daily results below are from
before the rebase.
- All Jira daily tests: 29 passed.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
- [x] [Optional] Override Linear Check
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Adds `Resolver.reindex` to the Jira connector so targeted reindexes can
re-fetch failed issues by document id.
- Parses each document URL's issue key, fetches issues in batches via
`key in (...)` JQL scoped to the configured project or JQL, and yields
the project and Epic hierarchy nodes before each document.
- On Server/DC, a batch that fails on a missing key falls back to
per-key fetches; 401, 403, and 429 errors stop the run as in the crawl.
- Moves per-issue conversion into a shared helper used by both crawl and
reindex; unknown, unparseable, or out-of-scope ids yield a
`ConnectorFailure`.
<sup>Written for commit
|
||
|
Evan LohnandClaude Opus 5.5
|
d327db1ee7 |
feat(hubspot): re-fetch failed documents by id (Resolver) (#16104)
## Description
`HubSpotConnector` now implements `Resolver`. A targeted reindex
re-reads the failed HubSpot records by id. It does not run a crawl or
use a checkpoint.
**How it works**
- `reindex` parses each `hubspot_<noun>_<id>` document id
(`parse_hubspot_document_id`). It groups the ids by object type.
- It reads each group with the CRM batch-read API, 100 ids for each
call. All calls go through the gateway's rate limiter.
- It builds each document with `_build_document`. I moved this helper
out of the crawl loop, so the crawl and the reindex use the same code.
The yielded `Document.id` is the input id exactly.
- HubSpot has no hierarchy nodes, so reindex yields none. The crawl also
yields none.
**Covered document types:** tickets, companies, deals and contacts.
These are all the types the connector writes. Notes are not documents.
They are sections in their parent record, so a reindex of the parent
reads them again.
**Not covered:** none.
**Failures**
- An id that does not parse, or names an unknown type, gives a
`ConnectorFailure` for that id.
- An id whose type is not in the connector's `object_types` gives a
`ConnectorFailure` (out of scope).
- An id that HubSpot does not return (deleted, archived, unreadable, or
merged into another record) gives a `ConnectorFailure`.
- Entity failures are skipped, as in Confluence. The HubSpot crawl never
yields them.
- A 401, a missing credential, or an exhausted rate limit ends the run.
The crawl also raises on these errors. Other errors fail only the
targets of that batch.
**Permissions:** with `include_permissions=True` (perm-synced cc-pair),
reindex sets `external_access` with `HubSpotPermissionReader`. The doc
sync uses the same reader. It asks the permitted-users API for viewers,
20 records for each call. If the viewer read fails for a record, reindex
yields a failure for it. It does not index the record without access.
**Filter parity with the crawl.** I compared each crawl filter with the
by-id read:
- `object_types` config: applied. An id of another type is a failure and
is not read.
- Archived records: the crawl's listing and search return unarchived
records only. The batch read now sends `archived=False` explicitly.
Reindex also drops a record that HubSpot marks `archived` (new
`archived` field on `HubSpotRecord`).
- Merged records: HubSpot can return a merged record under its new id.
Reindex yields only the ids it was asked for. The crawl indexes the new
id on its own.
- Code-side skips: the crawl has none.
- The time window is not an access filter, so reindex does not apply it.
I also updated the HubSpot row in
`.agents/feature-map/components/permission-sync.md`. It now says that a
targeted reindex also sets access.
## How Has This Been Tested?
- New daily test
`backend/tests/daily/connectors/hubspot/test_hubspot_resolver.py` runs
against the CI HubSpot portal (`HUBSPOT_ACCESS_TOKEN`). It covers: one
record, records of each type, a bogus id that is well-formed, ids that
do not parse, an out-of-scope type, empty input, entity-only input, and
`include_permissions=True` with `external_access` set. 8 passed.
- The test portal has no tickets. The "each type" test covers companies,
deals and contacts, plus any type the portal has records of.
- New unit test
`backend/tests/unit/onyx/connectors/hubspot/test_hubspot_reindex.py`: an
archived record, a merged record, an unconfigured type, a 401 that ends
the run, and a 500 that fails only its batch.
- Existing tests: `backend/tests/daily/connectors/hubspot/` 28 passed;
HubSpot unit tests 61 passed.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
- [x] [Optional] Override Linear Check
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
`HubSpotConnector` now implements `Resolver`, so a targeted reindex
re-reads failed records by id through the batch-read API instead of
running a crawl.
- Parses each `hubspot_<noun>_<id>` id, groups ids by object type, and
batch-reads 100 records at a time through the gateway's rate limiter.
The batch read sends `archived=False` explicitly, and reindex drops
records HubSpot marks archived.
- Unparseable ids, out-of-scope types, and records HubSpot doesn't
return (deleted, archived, or merged) yield a `ConnectorFailure`. A 401,
missing credential, or exhausted rate limit ends the run; other errors
fail only their batch.
- The portal id is read once per run, and a target whose associations or
notes cannot be read fails rather than indexing partial content — the
crawl still skips those reads.
- With `include_permissions=True`, reindex sets `external_access`
through `HubSpotPermissionReader`; a viewer-read failure prevents
indexing that record.
- Added daily and unit tests covering reindex behavior, failures, and
permission sync; entity failures are skipped, as in Confluence.
<sup>Written for commit
|
||
|
Evan LohnandClaude Opus 5.5
|
9384d95024 |
feat(gmail): re-fetch failed documents by id (Resolver) (#16106)
## Description
`GmailConnector` now implements `Resolver.reindex`. Targeted reindex can
re-fetch failed Gmail threads by document id.
**Document types**
- Covered: email threads. Gmail emits only this type. The document id is
the Gmail thread id, the same id the crawl yields.
- Not covered: none.
**How reindex finds the mailbox**
A Gmail thread id is unique only inside one mailbox. The id does not
name the mailbox. The crawl reads every Workspace user's mailbox through
service-account impersonation (or one mailbox with OAuth).
1. Crawl failures now link to their mailbox:
`https://mail.google.com/mail/u/<email>/#all/<thread id>`. Reindex reads
the mailbox from this link and fetches there first. This changes only
the `ConnectorFailure` link, not the document's section links.
2. Ids without that link (admin-requested targets, older failures) are
looked up in every mailbox the crawl reads, the primary admin's first.
Reindex stops when all ids resolve.
3. Each mailbox lookup uses batched `threads.get` calls (50 per batch).
403/404 means "not in this mailbox". Rate limits and 5xx errors are
retried, and become per-thread failures if they persist. They never
count as "not found".
4. A mailbox that cannot be impersonated (suspended user) is skipped. If
the primary admin's mailbox fails, the error is raised, because the
credential is bad. Missing scopes raise `PermissionError`, as in the
crawl.
**Failures**
- An id that is not a hex thread id yields a `ConnectorFailure` with no
API call.
- An id found in no mailbox yields a `ConnectorFailure`.
- Entity failures are skipped, as in Confluence.
**Filter parity with the crawl**
The crawl applies these filters. Reindex applies each one:
- Mailbox scope: the crawl reads only the users from
`_get_all_user_emails`. Reindex searches only those mailboxes. A mailbox
link that names another user is ignored.
- `threads.list` without `includeSpamTrash`: the crawl never lists a
thread whose messages are all in Spam or Trash. Reindex reads `labelIds`
and yields a `ConnectorFailure` for such a thread. It then removes the
labels, so the document text matches the crawl's.
- Mailbox not provisioned (400 `failedPrecondition`): the crawl skips
the mailbox. Reindex treats it as "not in this mailbox".
- Thread with no messages: the crawl yields nothing. Reindex yields a
`ConnectorFailure`.
- The time-range query is not an access filter, so reindex does not
apply it.
- The connector has no other config (only `batch_size`).
**Permissions**
The crawl always sets `external_access` to the mailbox owner's email
(`thread_to_document`). Reindex uses the same function, so
`include_permissions` changes nothing. Perm sync
(`ee/.../gmail/doc_sync.py`) gives the same single-owner access.
## How Has This Been Tested?
Unit: `uv run pytest -v backend/tests/unit/onyx/connectors/gmail/` gives
10 passed. Daily: `uv run --env-file .vscode/.env pytest -v
backend/tests/daily/connectors/gmail/` gave 10 passed before the rebase
onto main (8 resolver, 2 existing crawl tests).
- New `backend/tests/daily/connectors/gmail/test_gmail_resolver.py`
(service account, real test workspace): one thread from a non-admin
mailbox, all four threads across four mailboxes, mailbox link hint,
bogus but well-formed id, unparseable id, empty input, entity-only
input, and `include_permissions=True` with `external_access` checked per
owner.
- New unit tests in
`backend/tests/unit/onyx/connectors/gmail/test_connector.py`: a thread
with all messages in Spam/Trash yields a failure; a listed thread yields
a document without label text; OAuth path with a mocked Gmail client:
the Admin SDK returns 403, so reindex reads only the admin mailbox, and
a thread from another mailbox yields a failure.
- OAuth is not tested live. CI keeps no Gmail OAuth credential, because
a token that needs refreshing can make CI fail from time to time. The
unit test above covers the OAuth path.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
- [x] [Optional] Override Linear Check
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Implements `Resolver.reindex` on `GmailConnector` so documents that
failed during a crawl can be re-fetched by Gmail thread id.
- Gmail thread ids don't name their mailbox: reindex reads the mailbox
from the crawl failure link when available, otherwise searches every
mailbox the crawl reads with batched `threads.get` calls (50 per batch),
admin first, stopping when all ids resolve.
- Rate limits and 5xx errors are retried at the request and whole-batch
level; persistent ones become per-thread failures and never count as
"not found".
- Reindex applies the crawl's filters: threads in Spam/Trash, empty
threads, and threads in no crawled mailbox yield `ConnectorFailure`s;
invalid ids fail without an API call.
<sup>Written for commit
|
||
|
Evan LohnandClaude Opus 5.5
|
e9510d68ba |
feat(github): re-fetch failed documents by id (Resolver) (#16107)
## Description
`GithubConnector` now subclasses `Resolver`. `reindex` re-fetches each
document that a `ConnectorFailure` names, without a checkpoint.
**Document types covered (all types the crawl writes):**
- Pull requests: `https://<host>/<owner>/<repo>/pull/<n>`. One `GET
/pulls/{n}` call.
- Issues: `.../issues/<n>`. One `GET /issues/{n}` call.
- Files: `.../blob/<branch>/<path>`. One contents call. The fetch uses
the file helper that the crawl uses, now split out as
`_file_to_document`.
No document type is left out. The connector writes no hierarchy nodes,
so `reindex` yields none.
**How it works:**
- The connector groups targets by repository and reads each repository
once.
- Each read retries on rate limits with
`sleep_after_rate_limit_exception`, as the crawl does.
- A yielded `Document.id` always equals the input id. If GitHub now
serves the item at a different URL (for example, after a rename), the
connector yields a failure.
- An input id that it cannot parse, cannot find, or that is out of scope
gives a `ConnectorFailure` keyed on the input id.
- Entity failures are skipped, as in Confluence.
- One bad target does not stop the others. A 401, an SSO-required error,
a missing credential, or a rate limit that outlasts the retries stops
the whole run.
**Permissions:** when `include_permissions=True`, the connector sets
`external_access` once for each repository. It uses
`get_external_access_permission(repo)`, the helper the checkpoint
perm-sync path uses. Public repositories are public. Private
repositories use the collaborators group. Internal repositories use the
organization group.
**Filter parity with the crawl.** `reindex` never yields a document that
the crawl would skip. It applies each crawl filter to the by-id fetch,
or yields a `ConnectorFailure`:
- Owner and the configured repository list.
- Without a repository list, a user owner's repositories come from `GET
/users/{login}/repos`, which lists public repositories only. So a
private repository of a user is out of scope.
- `include_prs`, `include_issues`, and `include_files`.
- `state_filter`, for pull requests and issues.
- The issues listing returns pull requests, and the crawl skips them. An
`/issues/<n>` id that is a pull request fails.
- For files: the indexed branch (configured or default),
`is_indexable_path` (path denylist, extensions, names), and the size
limit. The size check uses the fetched content length. Directories and
submodules have no content, so they fail.
- The time window is not an access filter, so `reindex` does not apply
it.
**Path safety (file targets and all ids).** A stored id must not reach
another repository.
- Owner and repository names must match `[A-Za-z0-9._-]+` and must not
be `.` or `..`.
- An id with control characters fails before parsing. The parser does
not use `urlparse`: the crawl writes file paths unescaped, so `#` and
`?` stay part of a file path. Traversal checks run on the full path,
including any part after `#` or `?`.
- Each branch and path segment must not be empty, `.` or `..`, and must
not contain `\` or control characters. The connector checks the raw path
and the percent-decoded path. So `x/../../<other-repo>`, `%2e%2e`,
`docs//a.md` and `docs\..\a.md` all fail with no content fetch.
- The denylist and file-type rules also run on the decoded path. So
`node%5Fmodules/...` fails.
- A `%` alone is not refused. Git paths can contain `%`, and the crawl
writes the tree path unescaped
(`f"{repo.html_url}/blob/{branch}/{path}"`). The gateway escapes `%` to
`%25`, so a literal `%` reads only that literal file. Decoding before
the fetch would read a different file than the crawl indexed, so the
connector checks the decoded form but fetches the raw path.
- After the fetch, the reindex path checks that GitHub reports the same
`path` and an `html_url` in the resolved repository. Otherwise the
target fails. The contents call now exposes `path` and `html_url` for
this check. The crawl does not use this check, so its behavior does not
change. Side effect: a symlink file whose target has another path fails
on reindex.
## How Has This Been Tested?
- New daily test
`backend/tests/daily/connectors/github/test_github_resolver.py`. It runs
against `onyx-dot-app/documentation` with `ACCESS_TOKEN_GITHUB`. It gets
real ids from the slim crawl. Then it reindexes one PR and two of each
type, and checks `external_access` with EE on. It also checks unknown
and out-of-scope ids, unparseable ids, empty input, and entity-only
input. Result: 7 passed. The CI job already runs this directory, so CI
needs no new secret.
- New unit test
`backend/tests/unit/onyx/connectors/github/test_github_reindex.py`
(mocked gateway). It covers the excluded cases: state filter, a PR read
as an issue, disabled doc types, other owners and repositories, a
private user repository, and file filters (denylist, extension, size,
branch). It also covers rate-limit retry, one missing target among
others, and a 401 that stops the run. Path-safety cases: `..` traversal,
`.`, `%2e%2e`, escaped `node%5Fmodules`, empty segments, backslash,
newline, and unsafe owner and repository names. Each gives a
`ConnectorFailure` with no fetch. Other cases: a literal `%` in a file
name still works, and content that GitHub reports at another path or in
another repository fails. All GitHub unit tests pass: 265 passed.
- The existing `test_github_basic.py` daily test passes.
- Limit: the test repository is public. Private-repository permissions
are covered by the shared perm-sync helper only, not by a live test.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
- [x] [Optional] Override Linear Check
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
- **New Features**
- Added targeted reindexing for GitHub pull requests, issues, and
repository files.
- Added validation for document URLs, repository scope, branches, paths,
and indexable content.
- Added retries for rate-limited requests and per-document failure
reporting so other targets can continue.
- **Bug Fixes**
- Improved GitHub file metadata handling and document retrieval.
- Improved handling of permissions, renamed files, and repository access
errors.
- **Tests**
- Added comprehensive coverage for GitHub reindexing, filtering,
permissions, retries, and failure scenarios.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Evan LohnandClaude Opus 5.5
|
dccf305598 |
feat(canvas): re-fetch failed documents by id (Resolver) (#16100)
## Description `CanvasConnector` now subclasses `Resolver`. `reindex` fetches each failed document again by its id, without a checkpoint. **Covered doc types.** Canvas emits three doc types. `reindex` covers all three: | Doc id | Fetch | |---|---| | `canvas-page-<course>-<page_id>` | `GET courses/<course>/pages/<page_id>` | | `canvas-assignment-<course>-<id>` | `GET courses/<course>/assignments/<id>` with the crawl's `overrides` and `assignment_visibility` includes | | `canvas-announcement-<course>-<id>` | `GET courses/<course>/discussion_topics/<id>?include[]=sections` | No doc type is left out. **Scope and failures.** - The connector lists the token's courses once, as the crawl does. A course that is not in that list gives a failure for the target. - A target that the crawl listing would not return gives a failure. One predicate, `_listing_exclusion_reason`, applies the crawl's filters to the single-item payload: - all types: `published` must be true, and `workflow_state` (when present) must not be `unpublished`, `deleted` or `post_delayed`; - announcements (`active_only=true` parity): `is_announcement` must be true, `delayed_post_at` must not be in the future, and `posted_at` must be set. This stops a draft or scheduled announcement from getting course-wide access before its release. Time-window checks are not applied. - An id that does not match the crawl's id format gives a failure. - Entity failures (whole course or stage) are skipped, as Confluence does. - An error on one target gives a failure for that target only. This includes 404, invalid JSON, and a 401/403 for an item the user may not read. Canvas answers 401 or 403 for such items, so on a 401/403 the connector calls `users/self`. Only a 401 there (a rejected token) stops the run with `CredentialExpiredError`. A host/scheme error (`CanvasUnsafeUrlError`) also stops the run. **Permissions.** With `include_permissions=True`, each doc goes through the crawl's `_maybe_attach_permissions`. The course permission context is cached per course, so a batch fetches it once for each course. **Refactor.** A new `_item_to_document` helper converts an item and attaches permissions. The crawl (`_process_items`) and `reindex` both use it. The crawl has no hierarchy nodes, so `reindex` yields none. ## How Has This Been Tested? New daily test `backend/tests/daily/connectors/canvas/test_canvas_resolver.py`. It runs against the live Canvas test instance with the teacher token: - Reindex one page. Reindex every crawled doc of all three types, and compare the sections, metadata and `doc_updated_at` with the crawl. - Well-formed but unknown ids for each type, plus an out-of-scope course, give failures. - A bad target does not stop a good target in the same batch. - Ids that cannot be parsed give failures. Empty input and entity-only input give no output. - `include_permissions=True`: `external_access` is set, is not empty, and is equal to the access from `retrieve_all_slim_docs_perm_sync`. ``` uv run --env-file .vscode/.env pytest -v backend/tests/daily/connectors/canvas/test_canvas_resolver.py 9 passed ``` Unit tests in `test_canvas_connector.py` (`TestReindexListingParity`) use mocked payloads to check that unpublished, deleted, post-delayed, future `delayed_post_at` and non-announcement items give a failure and not a document. `TestReindexErrorScope` covers item 401/403 with a valid token, a rejected token, invalid JSON, and an off-host redirect. The Canvas unit tests pass (130 passed). A live daily test reindexes every course doc with the student token. Docs the student may not read fail one by one, and the batch goes on. The full Canvas daily directory passes: `uv run --env-file .vscode/.env pytest -q backend/tests/daily/connectors/canvas` gives 16 passed. This includes `test_canvas_perm_sync_daily.py` (7 passed, with `test_canvas_perm_sync_validation`). ## Additional Options - [ ] [Optional] Please cherry-pick this PR to the latest release version. - [x] [Optional] Override Linear Check 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **New Features** * Added Canvas reindexing for pages, assignments, and announcements. * Individual Canvas items can be refreshed and validated without interrupting other valid results. * Reindexing preserves document metadata and access permissions. * **Bug Fixes** * Excludes unpublished, deleted, delayed, missing, or otherwise unlisted Canvas items. * Reports invalid or unrecognized document references as connector failures. * Distinguishes individual item access failures from credential-wide authentication failures. * Safely rejects malformed, insecure, or cross-host Canvas URLs. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
onyx-cherry-pick[bot]
|
572c400cc3 |
chore(dev): upgrade onyx-devtools to 0.13.14 (#16360)
Automated upgrade of `onyx-devtools` to `0.13.14`, published by [this
release
run](https://github.com/onyx-dot-app/onyx/actions/runs/39525145296).
Updates the `ods` dependency group in `pyproject.toml`, then regenerates
`uv.lock` and `backend/requirements/dev.txt` to match.
Generated by the Release Devtools workflow.
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Upgrades `onyx-devtools` from 0.13.11 to 0.13.14 in the `ods` dependency
group.
Pins the new version in `pyproject.toml` and regenerates `uv.lock` and
`backend/requirements/dev.txt` to match.
<sup>Written for commit
|
||
|
Evan LohnandClaude Opus 5.5
|
f6557e2021 |
feat(connectors): connectors without named checks get the design's generic row (#16206)
## Description
Backend PR for the connector checks redesign (plan item A7), stacked on
#16205. A source without named checks shows the design's generic row.
**The generic row**
- The fallback check is "Test connection". A pass gives "Connected".
- A failure of the legacy `validate_connector_settings` probe gets a
line and a description for its kind:
| Kind | Line |
|---|---|
| Rejected sign-in (401) | Sign-in rejected |
| Denied permission (403) | No permission for the connector |
| Not found (404) | Not found at the source |
| Other settings error | Settings not accepted |
| Rate limit (429) | Rate limited by the source |
| Connection error or timeout | Cannot reach the source |
| Unknown error | Unexpected response from the source |
- A typed error keeps its class. An untyped error (for example a raw
`requests.HTTPError`) gets its class from its HTTP status, as the named
checks map them. The description names the status, e.g. "(HTTP 401)".
- The probe's own text stays the `message`, so Error Details show it.
- An error that gives its own row text keeps it.
- The check is like a sign-in: on a form run, an unreachable source
fails it (`fails_when_unreachable`).
- The blocking-validation recorder writes the same row, so stored
reports and draft runs agree. It keeps the original error class in
`error_type`, for support.
- The new module `capability_checks/generic_row.py` holds the row text.
The check and the recorder both use it.
**Connectors that run nothing**
- 23 connectors keep the no-op `validate_connector_settings` of
`BaseConnector` (for example GitLab, Gmail, Asana). The fallback check
passed for them without a request to the source.
- These sources now get no INDEXING check
(`factory.py:source_has_settings_probe`). The plan and the runs hold no
check, and the recorder writes no row for a pass. A failure (for example
at construction) is still recorded.
- The frontend already hides the card when no check applies. Phase C can
show "No checks for this connector yet" from the same empty list. No new
API field is necessary.
**Not in this PR**
- The EE perm-sync fallback ("Permission sync validation") keeps its
name and text. The design has no generic row for it.
- The generic row does not set `validates_binding`. It needs the full
config, so a create form could not unlock on it.
## How Has This Been Tested?
- New unit tests (`test_generic_check_row.py`): the pass line, each
failure kind (typed and from an HTTP status), the status in the
description and in Error Details, an error with its own row, the
form-run unreachable policy, and the recorder's rows (with no row for a
pass without a probe).
- Updated `test_registry_fallbacks.py` (name, pass line, no check
without a probe) and the ratchet test (an INDEXING check exists only
with named checks or a settings probe).
- Updated the integration test `test_capability_check_run.py`: a
mock-connector run now holds no check. I did not run integration tests.
- Connector, server and EE unit tests: 7878 passed, 1 failed. The
failure
(`test_browser_lifecycle.py::test_download_is_not_retried_and_does_not_relaunch_browser`)
also fails on the stack below. Pre-commit is clean.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
- [x] [Optional] Override Linear Check
---
_Generated by [Claude
Code](https://claude.ai/code/session_01KFMazQbuG8ui3jxBxm47sC)_
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Sources without named checks now show the design's generic "Test
connection" row instead of a pass-through fallback. Sources whose
connector keeps the no-op `validate_connector_settings` probe get no
INDEXING check.
- The fallback maps each failure kind of the legacy settings probe to a
line and description; a typed error keeps its class, an untyped one gets
its class from its HTTP status (401, 403, 404, 429), and the probe's own
text stays the message for Error Details.
- The blocking-validation recorder writes the same row, so draft runs
and stored reports agree, and a later pass on a source without a
settings probe replaces an earlier stale failure.
- Fallback checks keep the original stack trace in Error Details for
every error kind, and the EE permission-sync fallback keeps its existing
name and text, since the generic row does not apply to it.
<sup>Written for commit
|
||
|
Evan LohnandClaude Opus 5.5
|
d1627305f0 |
feat(connectors): config models check the design's field formats (Atlassian, GitHub) (#16205)
## Description
Backend PR for the connector checks redesign, stacked on #16204. The
Confluence, Jira and GitHub config models check the design's field
formats. This is the first half of the format-rule work. Google Drive,
Microsoft, Slack and Notion follow in a later PR.
**Rules** (field validators in each `config.py`)
| Connector | Field | Rejects | Message (the design's line) |
|---|---|---|---|
| Confluence | `page_id` | a value that is not a number | Enter the
number from the page URL, e.g. 131368. |
| Confluence | `cql_query` | `lastModified` outside a quoted value |
Unsupported filter: lastModified. Each sync adds its own. |
| Confluence | `cql_query` | `ORDER BY` outside a quoted value |
Unsupported clause: ORDER BY. Each sync orders the results itself. |
| Jira | `jql_query` | `ORDER BY` outside a quoted value | Unsupported
clause: ORDER BY. Jira does not allow it inside the sync filter. |
| GitHub | `repositories` | a name with an owner (`onyx-dot-app/onyx`) |
Enter the name without the owner, e.g. onyx. |
- **Where the errors go.** `/plan` and draft runs already return field
errors as `form_errors`, keyed by the field name. Create and edit
validate the full config with the same rules. The checks that read the
field wait in the form, and fail as "Invalid connector settings:
{field}: {message}" in other runs.
- **Clean messages.** A new helper, `field_format_error` in
`connector_config.py`, raises a `PydanticCustomError`. So the message
has no "Value error, " prefix. Its message must be a literal string, so
it cannot hold the input value.
- **One source.** The CQL and JQL rules share one module,
`onyx/connectors/atlassian_query.py`. The checks no longer test these
formats: a check cannot run while its field is invalid, so that code
could not run.
**Not in this PR** (the design's other field rules for these
connectors):
- Jira "Email (account)" and GitHub "Enterprise Server URL": these are
credential fields, not config fields.
- GitHub "Select at least one: pull requests, issues or documents": this
rule reads three fields. A config model must not have model validators
(the form validates field by field), so the "Check selected content"
check keeps it.
**Stored connectors.** A stored config that breaks a rule still indexes:
`build_connector_kwargs` uses the stored config as-is, and logs a
warning. The detail page's breakdown uses the top-level nodes for it.
Its stored check runs fail the config-reading checks as invalid
settings. The indexing hold reads only a pair's first attempt, so a pair
that indexed before is not held. An edit that changes the config must
fix the field.
## How Has This Been Tested?
- New unit test (`test_draft_plan_format_rules.py`): `/plan` gives each
rule's message on its field; create and edit validation reject the same
values; valid values (quoted words, blank values) give no error; a
stored config that breaks a rule still builds the connector.
- Existing Confluence, Jira and GitHub check tests updated: an invalid
field makes the check wait on the form, and fail as an invalid setting
in other runs, without a request.
- The daily live tests lose the ORDER BY and owner-in-name cases: these
no longer reach the source. I did not run the live tests.
- Connector and server unit tests: 6689 passed, 1 failed. The failure
(`test_browser_lifecycle.py::test_download_is_not_retried_and_does_not_relaunch_browser`)
also fails on the stack tip without this PR. `ty` and pre-commit are
clean.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
- [x] [Optional] Override Linear Check
---
_Generated by [Claude
Code](https://claude.ai/code/session_01KFMazQbuG8ui3jxBxm47sC)_
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
The Confluence, Jira, and GitHub config models now validate field
formats at config time, so invalid values are rejected before a
capability check runs or a request reaches the source.
- Validators reject non-numeric Confluence `page_id`, `lastModified` and
`ORDER BY` outside quoted values in Confluence CQL, `ORDER BY` outside
quoted values in Jira JQL, and GitHub repository names that include an
owner.
- Violations surface as `form_errors` keyed by field in `/plan` and
create/edit, with the design's line as the message; checks wait on the
form and fail as invalid settings in other runs.
- The CQL and JQL shape rules move to a shared `atlassian_query.py`
module, and the checks no longer test these formats.
- Stored configs that break a rule still index; any check that reads the
field fails as invalid settings until the config is fixed on edit.
<sup>Written for commit
|
||
|
Evan LohnandClaude Opus 5.5
|
573234ed2e |
feat(connectors): Outlook checks follow the design's rows, copy and states (#16204)
## Description
Thirteenth backend PR for the connector checks redesign, stacked on
#16203. Outlook follows the design's rows, copy and states. It uses the
shared Microsoft sign-in and Graph error rows from the Microsoft helpers
PR.
**Outlook**
- **Rows:**
- The 7 checks are the design's 6 rows, with its names: "Sign in to
Microsoft", "List mailboxes", "Find selected mailboxes", "Find selected
groups", "Sample mail", "Sample calendars".
- The mailbox count (a discovery check) shares the "List mailboxes" row
(`row_id`). "List mailboxes" sets `validates_binding`, as the design's
form unlock says.
- Dependencies follow the design's run order: every row after "List
mailboxes" runs after it.
- **Copy:**
- Every failure gives the design's line and description, with the grant
the app needs ("Grant `Mail.Read` (Microsoft Graph, application) and
admin-consent it.").
- Passing checks give what they found: "Tenant user list is readable",
"Up to {n} mailboxes", "All {n} selected mailboxes open", "All {n}
selected groups found", "Read mail in {mailbox}", "Read an event in
{mailbox}".
- A mailbox the admin did not name stays unnamed: the lines say "a
sampled mailbox".
- **Partial finds:** "Find selected mailboxes" and "Find selected
groups".
- Some unreachable: Incomplete, for example "Mailbox not found:
ghost@contoso.com" or "2 of 3 groups not found".
- None found: Failed ("No selected mailbox found", "No selected group
found").
- The samples open the first selected mailbox that answers, and walk the
groups that resolve. Before, they failed on the first missing one.
- **Honest passes.** These checks passed while testing nothing:
- An empty mailbox: now Incomplete ("No mail found in {mailbox}").
- A calendar with no events: now Incomplete ("No events found to test").
- "Sample calendars" when calendars are off: now does not apply (no
row). Before, it passed with no call.
- "Find selected mailboxes" and "Find selected groups" with nothing
selected: now do not apply. Before, they passed with no call.
- A sample message without attachments still passes, with a caveat.
- **Other states.** Empty pages from Graph ("EmptyPages") are Incomplete
("Message listing did not finish", "Event listing did not finish",
"Mailbox lookup did not finish"), not unexpected errors. A group lookup
that never ends is "Group lookup did not end".
**Shared code.** `describe_unavailable_mailboxes` and
`describe_unavailable_groups` (`outlook/mailboxes.py`) return typed
problems, so the checks and the connector's validation read one list.
The connector's validation message is unchanged.
**Not in this PR:** the design's two-step "List mailboxes" row
("counting…", and the caveat "Could not count every mailbox"). That
needs the planned two-step framework PR (A4). Until then, the two checks
share one row through `row_id`.
## How Has This Been Tested?
- New unit tests (`test_outlook_check_row_text.py`): the names and the
shared row, the pass lines (named and sampled mailboxes), the attachment
caveat, the user denial and the stopped samples, the partial mailbox
find (and that the sample opens the found mailbox), none found, the
group rows, the empty mailbox, the body and event-body denials, empty
pages, no events, and the rate limit.
- Existing Outlook tests updated to the new states.
- Connector, server and EE unit tests (top of the Microsoft stack): 5599
passed, 1 failed (`web/test_browser_lifecycle.py`, a web connector test
that also fails without this change), 25 skipped. `ty` and pre-commit
are clean.
- Outlook has no live capability-check test.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
- [x] [Optional] Override Linear Check
---
_Generated by [Claude
Code](https://claude.ai/code/session_01KFMazQbuG8ui3jxBxm47sC)_
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Reworks Outlook's capability checks to match the design's rows, copy,
and states so the checks screen reads consistently with the other
Microsoft connectors.
- Maps the 7 checks onto the design's 6 rows: the mailbox count shares
the "List mailboxes" row via `row_id`, which now validates the
credential binding, and every later row runs after it.
- Every failure names the grant the app needs, and passing checks report
what they found, like "Up to {n} mailboxes" or "Read a sampled mailbox".
A capped count says "At least {n} mailboxes".
- "Find selected mailboxes" and "Find selected groups" are partial
finds: some unreachable is Incomplete, none found is Failed. The summary
names the missing item, cut to fit; the description gives the full
address and reason.
- Checks that previously passed without testing anything now report
honestly: empty mailboxes and calendars are Incomplete, and samples with
nothing selected or calendars off do not apply.
- Graph's empty pages map to Incomplete instead of unexpected errors.
- `MailboxProblem`, `GroupProblem`, and `MailboxSample` live in
`outlook/models.py`; `describe_unavailable_mailboxes` and
`describe_unavailable_groups` return typed problems so the checks and
connector validation read one list. The Outlook check tests share their
gateway and context builders from a helper module.
- The design's two-step "List mailboxes" counting row is deferred to the
planned two-step framework PR.
<sup>Written for commit
|
||
|
Evan LohnandClaude Opus 5.5
|
9e2a456432 |
feat(connectors): OneDrive checks follow the design's rows, copy and states (#16203)
## Description
Twelfth backend PR for the connector checks redesign, stacked on #16202.
OneDrive follows the design's rows, copy and states. It uses the shared
Microsoft sign-in and Graph error rows from the Microsoft helpers PR.
**OneDrive**
- **Rows:**
- The 9 checks are the design's 8 rows, with its names: "Sign in to
Microsoft", "List users", "Find selected users", "Find a OneDrive",
"Read OneDrive changes", "Read file permissions", "List groups", "Read
group members".
- The drive count (a discovery check) shares the "List users" row
(`row_id`). "List users" sets `validates_binding`, as the design's form
unlock says.
- Dependencies follow the design's run order. One change from the
design: "Find a OneDrive" also runs after "Find selected users". When no
selected user is found, there is no OneDrive to open.
- **Copy:**
- Every failure gives the design's line and description, with the grant
the app needs.
- Passing checks give what they found: "Tenant user list is readable",
"Up to {n} OneDrives" (the count), "All {n} selected users found",
"Opened the OneDrive of {user}", "Read changes in {user}'s OneDrive",
"Read members of {group}".
- **Partial find:** "Find selected users".
- Some missing: Incomplete, for example "User not found:
gone@example.com". The drive checks after it walk the found users.
- None found: Failed ("No selected user found").
- An entry that is not an email ("Not an email address: {entry}") and a
list of blanks ("Users holds only blank entries") fail with their rows.
A locked user (423) is Incomplete ("User is locked").
- **Honest passes.** These checks passed while testing nothing:
- "Read file permissions" on OneDrives with no file: now Incomplete ("No
files found to test").
- "Read group members" when every sampled group hides its members: now
Incomplete ("No readable groups found").
- **Other states.** A group that went away between the listing and the
member read is Incomplete ("Sample group disappeared"). A user count
that does not end is Incomplete ("User list did not end"), not an
unexpected error. A sample with no OneDrive is Incomplete ("No OneDrives
found to test").
**Not in this PR:** the design's two-step "List users" row ("counting…"
while the count runs, and Successful with the caveat "Could not count
every OneDrive" when it does not finish). That needs the planned
two-step framework PR (A4). Until then, the two checks share one row
through `row_id`.
## How Has This Been Tested?
- New unit tests (`test_onedrive_check_row_text.py`): the names and the
shared row, the pass lines, the user denial, the partial find (and that
the drive check after it runs), none found (and that it stops the drive
check), the entry rows, the sample states, the group stop, and the rate
limit.
- Existing OneDrive tests updated to the new states.
- Connector, server and EE unit tests (top of the Microsoft stack): 5599
passed, 1 failed (`web/test_browser_lifecycle.py`, a web connector test
that also fails without this change), 25 skipped. `ty` and pre-commit
are clean.
- OneDrive has no live capability-check test.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
- [x] [Optional] Override Linear Check
---
_Generated by [Claude
Code](https://claude.ai/code/session_01KFMazQbuG8ui3jxBxm47sC)_
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Redesigns the OneDrive capability checks to match the connector checks
redesign: the 9 checks now map to the design's 8 rows ("List users",
"Find selected users", "Find a OneDrive", …), with the drive count
sharing the "List users" row, which now validates the binding. Check
ordering follows the design, with one change: "Find a OneDrive" also
runs after "Find selected users" so no found user stops it.
Failure rows now name the grant the app needs, and passing rows report
what the check found. Checks that previously passed without testing
anything now report Incomplete: an empty OneDrive for the permission
read, and sample groups that all hide their members. "Find selected
users" is a partial find: missing users are Incomplete and the drive
checks walk the found ones, while none found fails the row. Invalid
entries and blank user lists get their own rows. A drive probe that ends
on its last sampled feed page now moves on to the next drive instead of
raising.
The design's two-step "List users" row (counting, then a caveat) is
deferred to the planned two-step framework PR; until then the two checks
share one row through `row_id`.
<sup>Written for commit
|
||
|
Evan LohnandClaude Opus 5.5
|
d400ba5637 |
feat(connectors): SharePoint checks follow the design's rows, copy and states (#16202)
## Description
Eleventh backend PR for the connector checks redesign, stacked on
#16201. SharePoint follows the design's rows, copy and states. It uses
the shared Microsoft sign-in and Graph error rows from #16201.
**SharePoint**
- **Rows:**
- The 15 checks are the design's 15 rows, with its names: "Check
selected content", "List sites", "Find selected sites", "Find selected
libraries", "Sample documents", "Sign in with certificate", "Read site
permissions", "Read nested groups" and so on.
- Dependencies follow the design's run order. Two changes from the
design:
- "Sample documents" also runs after "Find selected sites". "Find
selected libraries" does not apply when no site URL names a library, so
without this a missing site would not stop the sample.
- "Read group members" and "Read nested groups" also run after "Sign in
with certificate". Both start from a site's SharePoint REST permissions,
which a client secret cannot read.
- **Copy:**
- Every failure gives the design's line and description. Lines name the
site, library, folder or group, cut to fit 40 characters. Descriptions
name the grant, for example "Grant `Sites.FullControl.All` (SharePoint
API, application) to the app registration and admin-consent it."
- The URL rules give their own rows: "Not a site URL", "Site URL not
allowed", "URL names no site" and "Site is on another host".
- Passing checks give what they found: "Indexing documents, site pages",
"{n} sites visible", "{n} selected sites found", "Read a file in
{library}", "Read a page on {site}", "Certificate valid until
YYYY/MM/DD", "Read members of {group}", "Read groups inside {group}".
- **Partial finds:** "Find selected sites" and "Find selected
libraries".
- Some missing or refused: Incomplete, for example "Site not found: ops"
or "2 of 3 sites not found". The indexing checks after them run on the
found sites and libraries.
- None found: Failed ("No selected site found", "No selected library
found").
- More than 5: Successful with a caveat ("First 5 of 7 sites found").
- **Honest passes.** These checks passed while testing nothing:
- No file in the sampled libraries: Incomplete ("No files found to
test"). Library permissions read but no file: Incomplete.
- No library, site page, site group or Entra group in scope: Skipped.
- A sample file that went away during the permission read: Successful
with a caveat.
- A sample file or page that went away during the read: Incomplete.
- **Empty scope.** Every selected site excluded, or every listed site
excluded, fails the site check.
**Fixes from the design's review**
- **REST token.** A SharePoint REST 401 gives "No token for the
SharePoint host".
- **Certificate.** "Sign in with certificate" gives the certificate's
end date, read from the .pfx with no call (a new gateway operation,
`get_certificate_not_after`).
- **Typed URL errors.** `connector_utils.validate_site_url` raises three
subclasses of `ConnectorValidationError` (format, blocked, host), so the
check can give each its row. Indexing sees the same base class.
- **Library metadata.** A library without traversal metadata raises
`DriveMetadataError` (a `ValueError`), so it is not taken for an
ambiguous name.
**Not in this PR:**
- "Tenant host unknown": the gateway raises the root-site failure as a
plain `ConnectorValidationError`, with no row.
- The permission checks do not run after "Find selected sites" (the
design's order). They read the configured sites themselves, so a missing
site fails their own read.
## How Has This Been Tested?
- New unit tests (`test_sharepoint_check_row_text.py`): the 15 names,
the pass lines, the URL rule rows, the empty scope, the partial site
find (and that the sample after it reads only the found site), none
found, the 5-site caveat, the sample states, the grant lines, the REST
401, the client-secret stop and the rate-limit stop.
- Existing SharePoint tests updated to the new states.
- Connector, server and EE unit tests (top of the Microsoft stack): 5599
passed, 1 failed (`web/test_browser_lifecycle.py`, a web connector test
that also fails without this change), 25 skipped. `ty` and pre-commit
are clean.
- The live tests (`test_sharepoint_capability_checks_live.py`) are
updated: a missing library now fails "Find selected libraries". I did
not run them: the test tenant's secrets are in AWS Secrets Manager, and
this session has no AWS access.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
- [x] [Optional] Override Linear Check
---
_Generated by [Claude
Code](https://claude.ai/code/session_01KFMazQbuG8ui3jxBxm47sC)_
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
SharePoint's connector checks now follow the design's row names, copy,
and states, reusing the shared Microsoft sign-in and Graph error rows.
Inconclusive samples now pass perm-sync validation, and a locked site
gets its own row. Not included: "Tenant host unknown" still surfaces as
a plain validation error, and the permission checks read the configured
sites themselves rather than running after "Find selected sites".
**Behavior changes**
- The 15 checks match the design's names and run order, with two
deviations: "Sample documents" also runs after "Find selected sites",
and the Entra group checks also run after "Sign in with certificate"
since both read SharePoint REST permissions a client secret cannot.
- "Find selected sites" and "Find selected libraries" are partial finds:
some missing reports INCOMPLETE (later checks run on what was found),
none found fails, and more than 5 found passes with a caveat.
- Checks that tested nothing now say so: no file in scope is INCOMPLETE
(such a sample passes perm-sync validation), nothing in scope is
SKIPPED, a file that vanished passes with a caveat, and an empty scope
fails when "List sites" reads the whole listing (a page-cap-stopped
listing is inconclusive).
- Every failure names the site, library, folder, or group (trimmed to 40
characters) and the grant needed.
- URL rules each get their own row, a locked site gets a "Site is
locked" row, and passing checks report what they found.
**Underlying changes**
- New gateway operation `get_certificate_not_after` reads the
certificate's end date from the .pfx without calling Graph.
- `validate_site_url` now raises typed subclasses of
`ConnectorValidationError` (format, blocked, host) so each URL rule maps
to its own row.
- A library without traversal metadata raises `DriveMetadataError`, no
longer mistaken for an ambiguous name, and the site listing now reports
whether a page cap stopped it early.
<sup>Written for commit
|
||
|
Evan LohnandClaude Opus 5.5
|
5751b31c15 |
feat(connectors): Microsoft checks share their sign-in row and Graph error rows (#16201)
## Description
Tenth backend PR for the connector checks redesign, stacked on #16200.
It adds what the SharePoint, OneDrive and Outlook checks share. The
three connector PRs that follow use it for their own rows.
**Sign in to Microsoft (all three connectors)**
- **One row for a failed sign-in.** Every probe signs in first. Before,
a bad secret made every check fail with the same error. Now every check
after sign-in depends on it, so it shows "Not run: Sign in to Microsoft
failed".
- **Name and lines.** The sign-in check is "Sign in to Microsoft". Each
sign-in refusal gives the design's line and description: "A required
account field is empty", "Certificate file does not open", "Directory ID
not found", "Secret or certificate was rejected", "App registration not
found", "Microsoft did not issue a token", "Microsoft refused the
sign-in request", "Microsoft sign-in is unavailable" and "Unexpected
answer from Microsoft sign-in".
- **The fields to fix.** A description names the account's fields with
the connector's labels ("SharePoint Client ID", "OneDrive Certificate",
"Microsoft Outlook Client Secret"). It names the fields of the account's
sign-in method: the secret for a secret account, the certificate and its
password for a certificate account.
- **Expiry.** An expired client secret (AADSTS7000222) says so. A
certificate account reads the certificate's end date from the .pfx, with
no call to Microsoft. An expired certificate gives its end date. A
passing sign-in gives "Certificate valid until YYYY/MM/DD", and a caveat
when the certificate ends in 30 days or less.
- **Pass line.** "Signed in as {app name}", from the access token's
`app_displayname` claim. The claims are read without a signature check:
they only name the app. Without the claim, the line names the app id.
- **Form run policy.** On a form run, a sign-in that cannot reach
Microsoft fails (`fails_when_unreachable=True`). Other runs keep it
Incomplete.
- **Form unlock.** OneDrive and Outlook sign-in now set
`validates_binding`, as SharePoint does. Their authority host and Graph
API host are bound to the credential (`MicrosoftCloudBinding`), and
sign-in proves the credential works with them.
**Graph errors (all Microsoft callers)**
- **Rate limit.** A 429 raises `RateLimitedValidationError` ("Microsoft
is throttling requests"), which stops the run. Before, it was a generic
unexpected error.
- **Unreachable.** A request that gets no answer after retries raises
`UnreachableValidationError` ("Cannot reach Microsoft").
- **Rejected token.** A 401 gives "Microsoft rejected the app's token".
- **Unexpected answer.** A 5xx or another status gives "Unexpected
answer from Microsoft", with the name of the request.
- **Rows from the caller.** `raise_for_graph_error` takes the row of a
403 (`denied_row`) and of a 404 or 423 (`not_found_row`). The connector
PRs use them.
- **Error Details.** `MicrosoftGraphError` is a `SourceApiError`. It
carries the request method, the URL and the response body, so Error
Details show them (redacted as in #16160).
**Gateways.** The three `check_token` operations return one shared
`MicrosoftTokenInfo` (lifetime, app name, app id, certificate end date).
The three connector token models are removed.
**Deviations from the design**
- "Directory ID not found" and "Cannot reach Microsoft sign-in": the
design's descriptions end with "(HTTP 429)". A missing directory is not
a 429, and a 429 at sign-in is the shared rate-limit row. I left the
status out of both.
- "Secret or certificate was rejected" and "A required account field is
empty": the design's text names only the secret fields. A certificate
account gets the certificate fields.
- "Sign-in timed out": not done. The runner owns the timeout text for
every source, so this row keeps the runner's "Timed out" line.
## How Has This Been Tested?
- New unit tests (`test_microsoft_check_rows.py`): every sign-in refusal
line, the expired secret and certificate, the token request statuses
(429, no answer, 5xx, 4xx, a body that is not JSON), the Graph rate
limit, unreachable, 401 and caller rows, Error Details from a Graph
refusal, the certificate end date from a real .pfx, the app name from
the token, and the expiry caveat. For each of the three connectors: a
failed sign-in stops the other checks, an unreachable sign-in fails only
a form run, and a throttled sign-in stops the run.
- Existing SharePoint, OneDrive and Outlook tests updated to the shared
token model.
- Connector, server and EE unit tests: 5527 passed, 1 failed
(`web/test_browser_lifecycle.py`, a web connector test that also fails
without this change). `ty` and pre-commit are clean.
- The live SharePoint test (`test_sharepoint_capability_checks_live.py`)
checks the new sign-in line and the stopped checks. I did not run it:
the test tenant's secrets are in AWS Secrets Manager, and this session
has no AWS access.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
- [x] [Optional] Override Linear Check
---
_Generated by [Claude
Code](https://claude.ai/code/session_01KFMazQbuG8ui3jxBxm47sC)_
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Unifies the SharePoint, OneDrive, and Outlook capability checks around
one shared sign-in probe and shared Graph error rows. A failed sign-in
now shows a single "Sign in to Microsoft" row, and every check after it
reports "Not run" instead of failing with the same error.
**Sign-in row**
- Sign-in refusals map to the design's lines and descriptions, naming
the account's fields with the connector's labels.
- An expired client secret or certificate says so; a certificate account
reads the certificate's end date from the .pfx, with a caveat in the
last 30 days.
- A passing sign-in shows "Signed in as {app name}" from the token's
claims, or the app id when the claim is missing. A name too long for the
summary is cut there and given in full in the description.
- An unreachable sign-in fails on a form run and stays Incomplete
otherwise. OneDrive and Outlook sign-in now validate the credential
binding, as SharePoint already did.
**Graph errors and Error Details**
- A 429 stops the run as `RateLimitedValidationError`; no answer becomes
`UnreachableValidationError`; a 401 and unexpected answers have their
own rows.
- Callers pass the rows for a 403 and a 404 or 423.
- `MicrosoftGraphError` is now a `SourceApiError` carrying the request
method, URL, and response body, so Error Details show them.
- The three connector token models are replaced by one
`MicrosoftTokenInfo`.
<sup>Written for commit
|
||
|
claude[bot]andClaude
|
b5ba31c965 |
fix(audit): drop Dependabot alerts whose manifest is not in the checkout (#16358)
<!-- claude-slack-attribution -->
_Requested by **Nikolas Garza** · [Slack
thread](https://onyx-company.slack.com/archives/C0C5SMAM4TT/p1792538621965719?thread_ts=1792538621.965719&cid=C0C5SMAM4TT)_
## Description
Before: The CVE audit reported every open Dependabot alert, even when
the alert named a manifest that is not in the checkout. For example,
issue #16345 cited alert #1369 for `loadtest/uv.lock`, which does not
exist on main. The certifi and pillow fixes were already in `uv.lock`,
but the finding came back on each run.
After: The audit drops a Dependabot alert when its manifest path is not
a file in the checkout. An alert with no manifest path stays, and so
does every alert if the repo root cannot be found.
What it does: stops stale Dependabot alerts from becoming audit findings
and tracking issues.
How: `auditDependabot` calls the new `dropMissingManifests` after
parsing the alerts. It resolves each manifest path against the git root.
Table-driven tests cover the filter, and one test runs `auditDependabot`
with a fake `gh`. Existing test fixtures now create the `pyproject.toml`
their alert cites.
## Not done
- Installed-version check: the tool has no lockfile reader that returns
the installed version of an arbitrary package. The OSV scan only reports
vulnerable packages. I did not add a new parser. An alert for a manifest
that exists, with a fix already installed, still shows until Dependabot
closes it.
- "Also failed on this run" replies: `PlanAnnouncement` only knows the
advisory-to-thread map. It has no record of earlier replies, and the
existing tests expect the reply on each failing run. Skipping it needs
new persistent state, so I made no change.
## How Has This Been Tested?
`go build ./... && go vet ./... && go test ./...` in `tools/ods` all
pass. `pre-commit` is not installed in this environment, so it did not
run.
Linear: none. No Linear link in this PR.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01E2K27DFf4J3iAGXYpe4xtR)_
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Stops the CVE audit from turning stale Dependabot alerts into findings
and tracking issues when the alert's manifest no longer exists in the
checkout.
- The audit drops alerts whose manifest path is not a file in the git
root; alerts without a manifest path, and all alerts when the repo root
cannot be found, are kept.
- No installed-version check was added: an alert for a manifest that
exists still shows even when the fix is already installed.
<sup>Written for commit
ods/v0.13.14
|
||
|
Raunak Bhagat
|
a6d3b6bc05 |
feat(opal): move between segments with the arrow keys (#16322)
## Description
Segmented fields (`InputTime`, `InputDatePicker`) now move between
segments with the arrow keys. ArrowLeft at the start of a segment
focuses the previous segment, with the caret at its end. ArrowRight at
the end focuses the next segment, with the caret at its start.
Shift+Arrow and caret moves inside a segment stay native.
The change is in the shared `makeSegmentKeyDownHandler`
(`inputs/segmented.tsx`), which now takes an optional next-segment ref.
Both pickers pass it.
First PR of a stack: this → `InputCurrency` → cost-overrides modal.
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Adds arrow key navigation between segments in `InputTime` and
`InputDatePicker`. ArrowLeft at a segment's start moves focus to the
previous segment's end, and ArrowRight at the end moves to the next
segment's start.
Only a plain arrow with a collapsed caret crosses an edge;
Shift/Ctrl/Meta/Alt arrows and mid-segment moves stay native. Handled in
the shared `makeSegmentKeyDownHandler`, which now takes an optional
next-segment ref, and tests cover edge crossing and all guard cases.
<sup>Written for commit
ods/v0.13.13
|
||
|
claude[bot]andClaude
|
ad36cbfadb |
fix(deps): bump pillow to 13.0.0 for GHSA-8g5g-8vmr-9782 (#16342)
<!-- claude-slack-attribution -->
_Requested by **Nikolas Garza** · [Slack
thread](https://onyx-company.slack.com/archives/C0C5SMAM4TT/p1792538621965719?thread_ts=1792538621.965719&cid=C0C5SMAM4TT)_
## Description
**Before:** Pillow was locked at 12.3.0, which has a critical heap
buffer overflow in the JPEG decoder (GHSA-8g5g-8vmr-9782). The daily CVE
audit fails on main because of it (#16335).
**After:** Pillow is locked at 13.0.0, the first release with the fix.
The audit finding should clear.
This bumps pillow from 12.3.0 to 13.0.0 in the Python lockfile, the
exported requirements files and the sandbox image pins.
**How:** Pillow is not pinned in `pyproject.toml` (it comes in through
other dependencies), so no constraint change was needed. I ran `uv lock
--upgrade-package pillow` (only pillow changed in `uv.lock`) and re-ran
the repo's pre-commit `uv export` commands for
`backend/requirements/{default,dev}.txt` (`ee.txt` and
`model_server.txt` don't include pillow). I also hand-edited the pillow
pin in
`backend/onyx/server/features/build/sandbox/image/initial-requirements.txt`,
after checking that `uv pip compile` still resolves with pillow 13.0.0
alongside matplotlib, pdfplumber and python-pptx. 13.0.0 is a new major
version, so CI should be watched for API breakage. A grep of `backend/`
found no uses of the usual removed Pillow APIs, but this was not run
against tests.
Fixes #16335
The certifi finding (#16334) is not part of this PR; it already has
#16337.
## How Has This Been Tested?
Lockfile and requirements regenerated locally with uv. Relying on CI for
the test run.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01E2K27DFf4J3iAGXYpe4xtR)_
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Bumps `pillow` from 12.3.0 to 13.0.0 to fix GHSA-8g5g-8vmr-9782, a
critical heap buffer overflow in the JPEG decoder that has been failing
the daily CVE audit on main (#16335). The uv lockfile, exported
requirements files, and sandbox image pins are all updated; only
`pillow` changed.
**Dependencies**
- `pillow` is not pinned in `pyproject.toml`, so no constraint change
was needed.
- 13.0.0 is a major version bump; no removed Pillow APIs were found in
`backend/`, but CI should still be watched.
<sup>Written for commit
|
||
|
onyx-cherry-pick[bot]
|
792fe3bfe2 |
fix(deps): pin certifi to 2026.10.19 (#16337)
Pins `certifi` to `2026.10.19` in every manifest and lockfile for
[PYSEC-2026-4320](https://osv.dev/vulnerability/PYSEC-2026-4320). Opened
by the [CVE Alerts
run](https://github.com/onyx-dot-app/onyx/actions/runs/39513310537).
Closes #16334
PYSEC-2026-4320 (GHSA-6q48-6c7w-9gw2) is a trust-store problem, not a
code bug. certifi before 2026.10.19 ships the `emSign ECC Root CA - G3`
root, and eight intermediates under it have leaked private keys. Anyone
holding one of those keys who can also intercept traffic can pose as any
HTTPS server to a client that trusts the bundle. The fix is to lock
certifi at 2026.10.19, which is on the same date-based version line as
the current 2025.11.12. I read the root `pyproject.toml`,
`tools/ods/pyproject.toml` and `cli/pyproject.toml` and none of them pin
certifi; it comes in only through `uv.lock` as a dependency of other
packages, so I edited no files. I had no search tool, so I could not
check the whole repo for other pins or for code that points TLS at a
different CA bundle. I am calling Onyx affected because the backend
makes many outbound HTTPS calls (LLM providers, connectors, webhooks)
through `requests`, `httpx` and `urllib3`, which use the certifi bundle
by default. An attack still needs both a leaked key and a network
position, and images built from the old lock need rebuilding to pick up
the fix.
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Pins `certifi` to `2026.10.19` in the backend requirement files and
`uv.lock` to fix PYSEC-2026-4320, a trust-store vulnerability.
- certifi before 2026.10.19 ships the `emSign ECC Root CA - G3` root,
and eight intermediates under it have leaked private keys.
- An attacker holding a leaked key who can intercept traffic can pose as
any HTTPS server to clients trusting that bundle.
- Images built from the old lock need rebuilding so the backend's
outbound HTTPS calls (LLM providers, connectors, webhooks) pick up the
new bundle.
<sup>Written for commit
|
||
|
claude[bot]andClaude
|
3a50ee8f32 |
fix(cve-alerts): allow a major bump when it is the only fix (#16343)
<!-- claude-slack-attribution -->
_Requested by **Nikolas Garza** · [Slack
thread](https://onyx-company.slack.com/archives/C0C5SMAM4TT/p1792538621965719?thread_ts=1792538621.965719&cid=C0C5SMAM4TT)_
## Description
Before: When the only fix for an advisory was a new major version, the
CVE Alerts workflow changed nothing. It left the issue open for a manual
bump.
After: In that case the workflow bumps to the lowest fixed release of
the lowest major that has one, and starts the note with "MAJOR bump".
The reviewer then checks the PR for breaking changes.
This change lets the CVE Alerts workflow open a fix PR for a major
version bump when no fix exists on the current major line.
How: The rule lived only in the triage prompt,
`.github/prompts/cve-fix.md`, so only that file changes. The workflow
and `cve-check-manifests.ts` do not check major versions. The PR body
already includes Claude's note, so the "MAJOR bump" flag shows in the
PR. The motivating case is pillow 12.3.0 for GHSA-8g5g-8vmr-9782. The
only fix is 13.0.0, so the workflow changed nothing and left issue
#16335 (see PR #16342).
## How Has This Been Tested?
I checked that no other workflow or script code enforces the old rule. I
did not run the workflow. The `pre-commit` hooks could not run in my
environment because the `shellcheck_py` build failed.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01E2K27DFf4J3iAGXYpe4xtR)_
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Allows the CVE Alerts workflow to open a fix PR for a major version bump
when no fix exists on the current major line.
Previously, when the only fix for an advisory was a new major version,
the workflow changed nothing and left the issue open for a manual bump.
Now it bumps to the lowest fixed release of the lowest major that has
one and starts the note with "MAJOR bump" so the reviewer checks the PR
for breaking changes. This change only touches the triage prompt in
`.github/prompts/cve-fix.md`; no workflow or script code enforces the
old rule. The motivating case is pillow 12.3.0 for GHSA-8g5g-8vmr-9782,
whose only fix is 13.0.0.
<sup>Written for commit
|
||
|
Evan LohnandClaude Opus 5.5
|
ed82e5a623 |
feat(box): re-fetch failed documents by id (Resolver) (#16099)
## Description
`BoxConnector` now implements `Resolver.reindex`. Targeted reindex can
re-fetch Box documents by id, without a crawl or a checkpoint.
**Document types covered**
- Files (`box-file-<id>`): fetched with `files.get_file_by_id`.
- Web links (`box-weblink-<id>`): fetched with
`web_links.get_web_link_by_id`, only when `include_web_links` is on.
The Box connector emits no other document types, so all types are
covered.
**How it works**
- The item's `path_collection` gives its folder chain. The connector
finds the outermost configured entry folder in that chain and rebuilds
the crawl's frontier entries from that folder down to the parent. The
`path` metadata, `parent_hierarchy_raw_node_id` and folder
`HierarchyNode`s match the crawl.
- Documents are built with the crawl's own `_convert_file` /
`_convert_web_link`. The folder `HierarchyNode` builder moved into a
shared `_folder_hierarchy_node` helper.
- Folder entries are cached for the whole call, so targets in the same
folder share upstream calls. Each folder node is yielded once.
**Failures** (each keyed on the input document id)
- Ids that do not parse as `box-file-<digits>` or
`box-weblink-<digits>`.
- Items that are not found (404), not readable (403), trashed, or
deleted.
- Items outside the configured folders.
- Web links when `include_web_links` is off.
- Files that the crawl skips (unsupported type, over the size threshold,
empty download), and conversion errors.
A `BoxAPIError` on one target does not stop the other targets.
Credential errors (HTTP 401, or OAuth `invalid_client` /
`unauthorized_client`) are raised again. Entity failures are skipped, as
in Confluence and Google Drive.
**Permissions**
With `include_permissions=True`, access uses the same EE helpers as the
perm-sync crawl: `resolve_box_ancestor_access` for the folders above the
entry folder, `resolve_box_folder_access` for each folder down to the
parent, then `resolve_box_file_access` / `resolve_box_web_link_access`
for the item. Folder nodes carry the same access too.
## How Has This Been Tested?
New daily test
`backend/tests/daily/connectors/box/test_box_resolver.py`, run against
the Box CI enterprise:
```
uv run --env-file .vscode/.env pytest -xv backend/tests/daily/connectors/box/test_box_resolver.py
13 passed
```
It covers: one file, all files plus the web link,
`include_permissions=True` (the test asserts `external_access` on docs
and nodes equals the perm-sync crawl's), a disabled web link, a file
outside the configured folder, bogus file and web link ids, unparseable
ids, a bad target next to a good one, empty input, and entity-only
input.
Also run: `backend/tests/daily/connectors/box/test_box_basic.py` (6
passed) and `backend/tests/unit/onyx/connectors/box` (61 passed).
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
- [x] [Optional] Override Linear Check
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
`BoxConnector` now implements `Resolver.reindex`, so failed documents
can be re-fetched by ID without a crawl or checkpoint.
- File (`box-file-<id>`) and web link (`box-weblink-<id>`) targets are
fetched via the Box API; web links only when `include_web_links` is on.
- The connector rebuilds the crawl's folder chain from the innermost
configured entry folder down to the item's parent, matching how the
crawl roots nested configured folders, and reuses `_convert_file` /
`_convert_web_link` for metadata, `HierarchyNode`s, and permissions.
- Failures are keyed on the input document ID for unparseable IDs,
trashed or inaccessible items, out-of-scope folders, disabled web links,
and items the crawl would skip. One bad target does not abort the
others; credential errors, including from downloads and permission
reads, are re-raised.
- With `include_permissions=True`, folder and document access match the
perm-sync crawl via the existing EE helpers.
- Adds daily and unit tests covering success paths, permission parity,
nested configured folders, and each failure case.
<sup>Written for commit
|
||
|
Evan LohnandClaude Opus 5.5
|
79088479c7 |
chore: dev worker queues match supervisord (#16158)
## Description
The light and heavy Celery workers in the dev launch configs
(`.vscode/launch.json`, `.zed/debug.json`, `.zed/tasks.json`,
`.zed/process-compose.k8s.yaml`) are missing queues that
`backend/supervisord.conf` runs:
- light: `chat_ttl_deletion`, `capability_checks_draft`
- heavy: `connector_hierarchy_fetching`, `capability_checks`
A dev stack started from these configs, including `ods stack up`, never
runs connector checks: draft runs on the create form stay queued. It
also skips chat TTL deletion and hierarchy fetching. This PR adds the
queues so the lists match supervisord.
## How Has This Been Tested?
Started a stack with `ods stack up --profile full` and read each
worker's `[queues]` banner in its log. Before the change, the light
worker had no `capability_checks_draft` and the heavy worker had no
`capability_checks`.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
- [x] [Optional] Override Linear Check
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Aligns the Celery worker queue lists in the dev launch configs
(`.vscode/launch.json`, `.zed/debug.json`, `.zed/tasks.json`,
`.zed/process-compose.k8s.yaml`) with `backend/supervisord.conf`, so dev
stacks started from these configs (including `ods stack up`) now run
connector checks, chat TTL deletion, and hierarchy fetching that were
previously skipped.
- Adds `chat_ttl_deletion` and `capability_checks_draft` to the light
worker queue list.
- Adds `connector_hierarchy_fetching` and `capability_checks` to the
heavy worker queue list.
<sup>Written for commit
|
||
|
Evan LohnandClaude Opus 5.5
|
e516bc7ae3 |
fix: follow longer PR stacks in the merge-queue tested-base check (#16303)
## Description
The merge-queue check `check-tested-base.sh` follows the chain of open
PRs from a stacked PR's tested parent down to the target branch. It
stopped after 10 PRs. The connector-checks stack (#16156 → #16200) is 12
PRs deep, so the `required` and `playwright-required` jobs failed in the
merge queue although the chain reaches `main` ([run
39354693529](https://github.com/onyx-dot-app/onyx/actions/runs/39354693529/job/118406689215)).
This PR raises the limit to 30 and names it `MAX_CHAIN_DEPTH`. Each
level is one `gh pr list` call, so the cost stays small.
## How Has This Been Tested?
- Ran the edited `reaches_target` read-only against the live stack from
`feat/checks-linear` with `TARGET=main`. It follows all 11 links and
reaches `main`. With the old limit it stops at
`feat/checks-incomplete-status`, as in the failed run.
- `bash -n` and pre-commit (shellcheck) pass.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
- [x] [Optional] Override Linear Check
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Fixes the merge-queue tested-base check so it follows chains of open PRs
up to 30 deep instead of stopping at 10, which caused 12-PR stacks to
fail even when their chain reached `main`. Raising the limit from 10 to
30 costs one `gh pr list` call per level, so it stays cheap.
<sup>Written for commit
|
||
|
Justin Tahara
|
cfa3f23daa |
fix(celery): resolve the broker client through Sentinel when Sentinel is on (#16313)
## Description
With `REDIS_SENTINEL_HOSTS` set, Celery's `broker_url` is a kombu
`sentinel://` URL (`configs/base.py`). `celery_get_broker_client` passed
that URL to `Redis.from_url`. redis-py only accepts `redis://`,
`rediss://` and `unix://`, so every caller raised:
```
ValueError: Redis URL must specify one of the following schemes (redis://, rediss://, unix://)
```
This breaks every task that reads Celery queue depth on Sentinel
deployments:
- the user file processing beat check (the safety net that re-queues
files stuck in PROCESSING; upload-time enqueue goes through kombu and
still works)
- user file delete and project sync
- pruning
- connector deletion
- doc permission sync and external group sync
- the monitoring tasks
The bug is present in every release since Sentinel support (#12347).
The fix: when `REDIS_SENTINEL_HOSTS` is set and the broker URL is
`sentinel://`, build the client from `RedisPool.create_pool` on
`REDIS_DB_NUMBER_CELERY`. This reuses the Sentinel auth, TLS, timeout
and retry settings that the app Redis pool already uses.
All other URLs keep the `Redis.from_url` path. The URL check also covers
one odd setup: Sentinel hosts set, plus a `CELERY_BROKER_URL` override
that points at plain Redis. That setup keeps the URL it was given.
## How Has This Been Tested?
- Unit tests in `test_celery_redis.py`:
- A Sentinel URL resolves through
`RedisPool.create_pool(db=REDIS_DB_NUMBER_CELERY, replica=False)` and
never calls `from_url`.
- A `redis://` URL still uses `from_url` when Sentinel hosts are set.
- Live check: a local `redis:7` master plus a `redis:7` Sentinel, with
`REDIS_SENTINEL_HOSTS` set and a Celery app built from
`configs/base.py`.
- Broker URL: `sentinel://127.0.0.1:26391/15`.
- The old `Redis.from_url` path raises the error above.
- The new client pings the master, uses DB 15, returns the correct
`celery_get_queue_length`, and the second call reuses the cached client.
## Additional Options
- [x] [Optional] Please cherry-pick this PR to the latest release
version.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Fixes `celery_get_broker_client` crashing with `ValueError` when
Celery's broker URL is a kombu `sentinel://` URL. This broke every task
that reads Celery queue depth on Sentinel deployments, blocking user
file processing, pruning, connector deletion, permission sync, and
monitoring.
When `REDIS_SENTINEL_HOSTS` is set and the broker URL starts with
`sentinel://`, the client is now built from `RedisPool.create_pool` on
`REDIS_DB_NUMBER_CELERY`, reusing the Sentinel auth, TLS, and timeout
settings of the app Redis pool. All other URLs keep the `Redis.from_url`
path.
<sup>Written for commit
|
||
|
devin-ai-integration[bot]andraunak
|
b8c31c5b70 |
fix(web): clamp agents table cells and stop drag-reorder reshuffle (#16282)
## Description
These are fixes to the admin Agents table (`AgentsTable`), plus a new
opt-in column option in `@opal/Table`.
**1. Text overflow.** Long names and descriptions wrapped past the fixed
row height and overlapped the next row.
- The Name and Description cells now use `@opal/OverflowText` in place
of the legacy `refresh-components` `Text`. The text is clamped to 1
line. When the text is cut, hover it to see the full text in a tooltip.
- The `Content` cells now clamp to 1 line. Created By sets
`titleMaxLines={1}`. Access sets `titleMaxLines={1}` and
`descriptionMaxLines={1}`.
- `@opal/Table` data columns get a new option, `truncate?: boolean`.
When it is set, `TableCell` adds `w-0 min-w-full` to `.tbl-cell-inner`.
Under `table-layout: auto`, the content then cannot widen the column, so
long content truncates. Before this change, the Agents table grew to
2844px inside an 840px container. All Agents data columns set `truncate:
true`. The option is off by default, so other tables do not change. For
example, the Users `GroupsCell` measures its own width and needs the old
behavior.
**2. Random reshuffle after a drag-reorder.** The causes:
- `GET /api/admin/persona` (`get_persona_snapshots_for_user`) has no
`ORDER BY`, so the refetch after a save came back in arbitrary DB order.
- `handleReorder` sent only the moved range, as indexes into the visible
list. Other agents kept `null` or stale priorities, which collided with
the new values.
- The row snapped back until the refetch finished.
The fix:
- The table sorts agents client-side the way the paginated endpoint
does: `display_priority` ascending with nulls last, then `|id|`.
- `buildDisplayPriorityMap` merges the reordered visible IDs back into
the full agent list. It then sends a priority for every agent whose
position changed.
- The new priorities go into the SWR cache before the PATCH.
- `@opal/Table` now passes the `query`-filtered rows to
`useDraggableRows`, not all `items`. A drag while a search is active no
longer shifts hidden rows.
## How Has This Been Tested?
- `bun run types:check`, `oxlint`, `oxfmt` and the `@opal/Table` jest
tests pass.
- I tested in the browser on a local stack with 8 agents; see the
comment below. The checks were: no horizontal overflow at 840px and
1600px, ellipsis plus hover tooltip on long cells, and normal, far-below
and search-filtered drags. Each order persisted across reload and
matched the DB priorities.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
Link to Devin session:
https://app.devin.ai/sessions/0d13205be9bb4560a7a0e55e880edd52
Open in Devin Desktop:
https://app.devin.ai/desktop/session/0d13205be9bb4560a7a0e55e880edd52?variant=devin
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Fixes two bugs in the admin Agents table: long text no longer overlaps
the next row, and rows stay put after a drag-reorder.
- Data columns now accept an opt-in `truncate` flag; truncated columns
never widen the column, clamp content to one line, and show the full
value in a hover tooltip.
- The Name, Description, Created By, and Access cells clamp this way,
using `OverflowText` or one-line `Content` with max-line limits.
- Drag-reorder operates on query-visible rows only, so hidden rows keep
their positions.
- On drop, the full agent list is renumbered from the new visible order
and the priorities are written to the SWR cache before the PATCH, so
rows don't snap back while saving; nothing is saved when no priority
changed, and a failed save refreshes the table and shows an error.
- The table sorts agents client-side to match the server's order
(`display_priority` ascending with nulls last, then `|id|`).
<sup>Written for commit
|
||
|
devin-ai-integration[bot]
|
ecaa977d4d |
chore(web): remove unused useContentSize hook (#16304)
## Description
This PR removes `web/src/hooks/useContentSize.ts`. Its only caller was
`web/src/refresh-components/buttons/SelectButton.tsx`, and #16301
deleted that file. Now `git grep useContentSize` on `main` finds matches
only in the hook's own file. No barrel file re-exports the hook.
## How Has This Been Tested?
- `bun run types:check` passes with no type errors.
- The pre-commit hooks pass. These include type coverage, oxlint, oxfmt
and feature-map integrity.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
Link to Devin session:
https://app.devin.ai/sessions/3efa311f9e28496fa1d729253e27b8d2
Open in Devin Desktop:
https://app.devin.ai/desktop/session/3efa311f9e28496fa1d729253e27b8d2?variant=devin
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Removes the unused `useContentSize` hook from
`web/src/hooks/useContentSize.ts`. Its only caller was
`SelectButton.tsx`, which was deleted in #16301, so `git grep
useContentSize` on `main` now finds matches only in the hook's own file,
and no barrel file re-exports it. No other code is affected.
<sup>Written for commit
nightly-latest-20261020
|
||
|
Wenxi
|
b80667827f |
fix(plugins): manage default Chat selection in plugin settings (#16281)
## Description Move MCP and OpenAPI default Chat settings out of Chat Preferences and into Plugins. Chat Preferences keeps built-in controls and shows a translated notice linking to Plugins. OpenAPI Configuration now has an **Enable in default Chat** toggle. Changes are saved with the form. Turning it off removes the plugin from default Chat while preserving custom-agent use. OpenAPI tools remain read-only; there are no per-operation switches. Admin directory status reflects default Chat selection. OpenAPI plugins no longer have a global disable control. Delete the plugin to remove it from the workspace. Previously disabled records remain off default Chat, but configured custom agents can use them, subject to access and authentication. Explicit default Chat selection normalizes the legacy flag. This uses existing fields and associations, with no migration. New full-admin setup defaults the Chat toggle on; scoped creators cannot change workspace Chat defaults. MCP retains its server-level default Chat switch and global per-tool controls. Removing the Chat Preferences cards intentionally removes individual MCP tool selection specifically for default Chat. OpenAPI retains its whole-plugin default Chat choice in Configuration. Terraform `onyx_custom_tool.enabled` now controls default Chat selection. The provider reads and writes `default_chat_enabled` through the custom-tool API. Metadata and selection save atomically, parallel resources preserve unrelated selections, and omitted OAuth settings remain unchanged. Upgrade the provider alongside the server; older versions use the removed global status writes. No Terraform state schema migration is required. Update the feature map and simplify the OpenAPI tools list to shared name/description rows with consistent search and counts. Rows retain composite method/path/name keys. ## How Has This Been Tested? - 149 frontend tests passed across plugin forms, member details, and the admin directory. - Backend availability unit tests passed, including permission checks, legacy selections, and omitted versus explicitly cleared OAuth settings. Additional MCP/default Chat unit checks passed. - 13 PostgreSQL tests passed against a disposable ODS database. Coverage includes legacy OpenAPI construction, atomic API create/update/read, rollback, and concurrent selections. - Terraform provider `go test ./...` passed and generated documentation is current. Live Terraform acceptance tests were not run. - Repository pre-commit checks passed, including type checks, formatting, lint, translations, and feature-map integrity. `git diff --check` passed. ## Additional Options - [ ] [Optional] Please cherry-pick this PR to the latest release version. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added per-plugin availability controls for Default Chat, including MCP servers and OpenAPI actions. * OpenAPI actions disabled for Default Chat remain available to configured custom agents. * Added permission-aware controls so only workspace administrators can change Default Chat availability. * Added support for managing these settings through Terraform custom tools. * **Improvements** * Chat Preferences now focuses on built-in tools, with links directing administrators to Plugins for MCP and OpenAPI configuration. * Improved OpenAPI action search and display. * **Localization** * Added translated availability guidance across supported languages. <!-- end of auto-generated comment: release notes by coderabbit.ai -->v5.0.0-beta.2 |
||
|
Wenxi
|
e1dda7a89e |
feat(plugins): align directory ordering and availability indicators (#16273)
## Description Align admin and member plugin directory ordering and availability indicators. Member indicators show Main Chat, custom agents, and Craft in clockwise order, with one gray dot when workspace setup is required. Unsupported surfaces appear in the tooltip. Include default Chat availability in the Craft MCP catalog so sorting and indicators use known state. Keep indicator geometry consistent in RTL locales. ## How Has This Been Tested? 31 member directory tests and 11 backend availability tests passed. Type, lint, formatting, and feature-map checks passed. ## Additional Options - [ ] [Optional] Please cherry-pick this PR to the latest release version. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Plugin directories show separate availability indicators for Chat, custom agents, and Craft, with detailed status tooltips. - Plugin cards are sorted by availability, type, and locale-aware name. - Availability labels for disabled, unavailable, custom-agent, and unknown states are translated across supported languages. - Craft MCP listings indicate whether each server is available in default Chat. - **Bug Fixes** - Indicators and sorting more accurately reflect disabled skills, connection issues, unsupported surfaces, and Craft availability. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Wenxi
|
26e288019a |
feat(plugins): unify member connection and settings modals (#16272)
## Description Unify member plugin connection, tools, approvals, and skill settings in persistent modals. Availability and skill changes remain drafts until Save, and previews reuse the existing modal. Keep the backdrop and active content in one modal root so long content scrolls through connection transitions. Show connection and skill issues in separate cards. Skill-name conflicts now offer explicit replacement confirmation without discarding other edits. ## How Has This Been Tested? 87 directory, detail, and connection tests plus 7 skill-panel tests passed. Coverage includes conflict confirmation, draft retention, stable backdrop identity, native wheel scrolling, and outside-click behavior. The scrolling regression fails against the old nested modal roots. Type, lint, formatting, translation, and feature-map checks passed. ## Additional Options - [ ] [Optional] Please cherry-pick this PR to the latest release version. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Plugin details now group tool availability and approval settings under Tools, with editable preferences saved or discarded together. * Craft app skill settings support previews, draft enablement changes, save/cancel controls, and conflict handling when enabling a skill requires replacement. * Connection details show readiness, setup issues, connect/disconnect controls, and warnings when associated skills are disabled. * Plugin setup guidance clarifies availability across default Chat, Craft sessions, and scheduled tasks, including when custom agents may still use a plugin. * **Documentation** * Updated plugin guidance and terminology across supported languages. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Wenxi
|
7fbe49a24b |
feat(plugins): unify admin plugin configuration forms (#16271)
## Description Unify admin MCP, OpenAPI, and Craft App configuration forms. Availability changes remain drafts until Save. MCP access and availability controls appear on the first setup page; authentication stays on the second. OpenAPI Configuration combines schema, optional display description, schema-backed agent description, and workspace availability. Creation keeps one modal backdrop across steps. Preserve automatic OAuth setup after authentication and protect refreshed credentials when saving setup metadata. Unrelated saves preserve other administrators’ availability changes. Pending Chat enablement can retry discovery without changing authentication. ## How Has This Been Tested? Targeted MCP, OpenAPI, and Craft App component suites passed, including draft retention, access checks, retry behavior, concurrent availability edits, and implicit form submission. OAuth backend regressions passed, including concurrent token refresh preservation. Type, lint, formatting, translation, and feature-map checks passed. ## Additional Options - [ ] [Optional] Please cherry-pick this PR to the latest release version. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Configure app availability for Chat, Craft, and the workspace, with changes applied when saved. * Choose Chat and Craft availability during MCP setup; OAuth setup choices are remembered. * Edit OpenAPI schema, descriptions, and availability together in Configuration. * Create apps and tools as enabled or disabled. * **Changes** * Availability indicators are read-only; manage settings in configuration. * MCP approval settings are separate from server configuration, and tool counts include locked tools. * MCP setup validates access before connection; unchanged credentials cannot be resubmitted unnecessarily. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Raunak Bhagat
|
3370ae6478 |
fix(connectors): redact auth headers and cookies from check error details (#16268)
## Description
Stacked on #16287, the top of the connector checks stack (GitHub stack
#16195). It fixes a credential leak that #16160 introduced.
**Problem**
- `JIRAError.__str__` prints the request and response headers. These
hold `Authorization: Basic <base64(email:token)>` and the session
cookies.
- `_translate_errors` used `str(e)` as the message of `JiraApiError`.
The SDK error also stays in the exception chain, so the stored stack
trace prints it.
- Redaction removes only the raw credential strings. The credential is
base64-encoded in the header, so it was not removed.
- Result: an unexpected Jira error, such as a 5xx, stored the encoded
credential in the check report. Curators can read that report.
**Fix**
- `JiraApiError` gets a message without headers: `JiraError HTTP
<status>`.
- `JiraApiError` is raised `from None`, so the `JIRAError` and its
header dump never reach the stored stack trace. Status, method, URL and
Jira's error body stay on `JiraApiError`, so error details keep them.
- The existing `scrub_sensitive_values` covers the rest. There are no
custom header regexes: the only source of printed headers is closed.
- `_translate_errors` no longer raises `AttributeError` when a
`JIRAError` response has no request. That error replaced the
`JiraApiError` that callers catch.
**Not in this PR**
- Reports stored before this fix can still hold leaked traces. This code
is not on `main`, so only test deployments of the stack are affected.
- The result `message` is still not scrubbed for a raw
`requests.HTTPError`.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
- [x] [Optional] Override Linear Check
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Fixes a credential leak where Jira check errors stored the
base64-encoded Basic auth header and session cookies in error details.
- The leak came from `JIRAError.__str__` printing request and response
headers; `JiraApiError` now uses a header-free message (`JiraError HTTP
<status>`) and drops the SDK error from its chain, so neither the
message nor the stored stack trace holds the headers.
- Reading the request method now null-guards `e.response.request`, so an
error without a request is no longer replaced by an `AttributeError`.
Reports stored before this fix may still contain leaked traces; raw
`requests.HTTPError` messages remain unscrubbed.
<sup>Written for commit
|
||
|
Evan LohnandClaude Opus 5.5
|
a7c83a2a23 |
fix: review follow-ups for the connector checks stack (#16287)
## Description
This PR fixes review comments on the connector checks stack. Each fix is
in its own commit. Each fix has a test that fails without it. It sits on
top of #16200 so that the PRs below it do not have to run CI again.
**Error details (#16160)**
- Remove the `;params` of every URL path segment, not only the last one.
A server can put a session token there.
- Never keep part of a secret where the response body is cut before
redaction. The cut moves to the start of a secret that it would split.
- When a `SourceApiError` has no status, URL or body, read the HTTP
error that it wraps.
- Simplify the error-detail test helper. It does not need to raise the
error first.
**Check dependencies at creation (#16157)**
- A check whose dependency did not finish in the blocking budget is now
unfinished too. Its early result does not block the pairing. The
background run runs it after its dependency. A stored report shows it as
not run.
**Row text (#16167)**
- A test fails when a check row description can have more than three
lines.
**Confluence (#16169)**
- Before Data Center 9.1, read anonymous access from the JSON-RPC
permissions, as permission sync does.
- Read user emails counts each Cloud account once, also when it has
several grants.
- Read group members names the number of users it read.
**Jira (#16193)**
- The query check fails when the search finds an issue id but the fetch
returns no issue.
**Google Drive (#16194)**
- The shared drives row says "No shared drives visible" when the listing
is empty, also when Shared Drive URLs is set.
- A denied read of the primary admin's directory entry shows the
directory-read text, not the user-listing text.
- Shared Drive URLs or Folder URLs that give no ID fail. They do not
pass as "All 0 selected ... found".
## How Has This Been Tested?
- Unit tests for capability_checks, confluence, jira, google_drive and
server: 3446 passed.
- External-dependency test for an unfinished dependency in proposed
pairing validation, plus the creation and dry-run test files: 31 passed.
- `ty` on the changed files; pre-commit on every commit.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
- [x] [Optional] Override Linear Check
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Fixes review comments on the connector checks stack: check dependencies,
error details, row text limits, and the Confluence, Jira, and Google
Drive capability checks. Each fix has a test that fails without it.
**Bug Fixes**
- A check whose dependency did not finish in the blocking budget is now
unfinished too, so its early result no longer blocks the pairing and the
report shows it as not run.
- Error details strip `;params` from every URL path segment, move the
redaction cut to the start of a secret the bound would split, and read
the HTTP error a bare `SourceApiError` wraps.
- Confluence before Data Center 9.1 reads anonymous access from the
JSON-RPC permissions, counts each Cloud user once across grants, and
names the number of users it read.
- The Jira query check fails when the search finds an issue id that the
fetch does not return.
- Google Drive shows "No shared drives visible" for an empty listing
even with Shared Drive URLs set, a denied admin directory read shows the
directory-read text, and Shared Drive URLs or Folder URLs that give no
ID fail.
**Tests**
- A new test reads the check modules and fails when a row description
can have more than three lines.
- Coverage for Confluence now also runs a 9.1+ site whose REST
permissions answer, so the anonymous-access check exercises its REST
endpoint call.
<sup>Written for commit
|
||
|
Evan LohnandClaude Opus 5.5
|
5d283b7228 |
feat(connectors): HubSpot checks follow the design's rows, copy and states (#16200)
## Description
Backend PR for the connector checks redesign, stacked on #16199. HubSpot
follows the design's rows, copy and states.
**HubSpot**
- **Rows:**
- The 9 checks are the design's 9 rows, with its names: "Sign in to
HubSpot", "Read tickets", "Read companies", "Read deals", "Read
contacts", "List linked records", "Sample notes", "List users" and "Read
record viewers".
- Dependencies follow the design's run order, with one change. "List
linked records" and "Sample notes" run after the sign-in, not after the
four read checks. A dependency that finds nothing to test stops the
checks after it. Many portals have no tickets, so "No tickets in
HubSpot" would stop both checks, but they can test a record of another
type. The two checks sample the first type that lists a record, and they
leave out a denied type: its read check reports it.
- **Copy:**
- Every failure gives the design's line and description.
- A denial names the private app scope that HubSpot needs, e.g. "In
HubSpot, add `crm.objects.deals.read` to the private app's scopes."
- Passing checks give what they found: "Signed in to account
{portal_id}", "Found {n} tickets" (the count from the Search API),
"Listed contacts linked to tickets", "Read a note linked to a ticket",
"{n} users found", "{n} users can view a sample ticket". Singular and
plural forms are correct ("Found 1 company").
- **Honest passes.** These checks passed while testing nothing:
- A type with no records is now Skipped ("No tickets in HubSpot").
- When no selected type has a record, the connector would index nothing.
The first read check now fails ("No records in the selected types").
This blocks Create.
- No record in scope is now Skipped for "List linked records", "Sample
notes" and "Read record viewers" ("No records in scope").
- A sample record without notes is now Incomplete ("No notes found to
test").
- No listed user is now Incomplete ("No users found to test").
**Fixes from the design's review**
- **Rate limits.** After the rate limiter's 5 retries, a check raises
`RateLimitedValidationError`, which stops the run. Before, the limit was
an unexpected error.
- **Sign-in.** It now reports a missing token ("Account has no API
token"), an unreachable HubSpot ("Cannot reach HubSpot") and a timeout
("Sign-in timed out"). On a form run, an unreachable HubSpot or a
timeout fails sign-in. A 5xx or an answer that is not JSON is
"Unexpected response from HubSpot" (Incomplete).
- **Error Details.** `HubSpotApiError` is a `SourceApiError`. The
gateway adds the method, the URL and HubSpot's error body, for SDK and
REST calls, so Error Details show them.
- **Connection errors.** The gateway raises `HubSpotConnectionError`
(with `timed_out`) for a connection failure or a timeout, from the SDK
and from REST calls. The checks cannot import `requests` (the SDK import
fence).
- **Permission sync validation.** `validate_hubspot_perm_sync` accepts a
portal with no records. Before, the viewers check passed silently there;
now it raises `NothingToCheckError`, which would pause the connector.
**Not in this PR:**
- `validates_binding` on "Sign in to HubSpot". The design says sign-in
unlocks the form, but HubSpot's config has no credential-bound fields,
and the framework requires a binding class for a binding check
(`test_binding_checks_require_only_bound_fields`). GitHub and Google
Drive also leave it out.
- The design's HubSpot text for the 10-minute hang guard ("The check
stopped after 10 minutes without an answer from HubSpot."). The runner
writes this row text for every source. A per-source text needs a
framework change.
## How Has This Been Tested?
- New unit tests (`test_hubspot_check_row_text.py`): the sign-in lines,
the form-run policy (unreachable and timeout), the rate-limit stop, the
dependency stops, Error Details, the empty-type and empty-scope states,
the samples' Skipped and Incomplete states, and that "List linked
records" runs past an empty type.
- New gateway tests: SDK and REST errors carry the request and body; a
connection failure says whether it timed out; a search keeps its match
count.
- Existing HubSpot check tests updated to the new states.
- Connector, server and EE unit tests: 5519 passed, 25 skipped, 1
failed. The failed test is a web connector test
(`test_browser_lifecycle.py`). It also fails without this change. `ty`
and pre-commit are clean.
- HubSpot has no daily live capability-check test, so there is nothing
to update there.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
- [x] [Optional] Override Linear Check
---
_Generated by [Claude
Code](https://claude.ai/code/session_01KFMazQbuG8ui3jxBxm47sC)_
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Rewrites the HubSpot capability checks to follow the checks design: nine
rows with the design's names, run order, copy, and outcome states.
Checks now report what they tested and what they found instead of
passing silently.
- "List linked records" and "Sample notes" run after sign-in only, so an
empty or denied type no longer stops them.
- Read checks report Search API counts with singular and plural forms; a
full users page reports a "100+" lower bound.
- Empty types are Skipped, a record without notes or no listed user is
Incomplete, and a portal with no records of any selected type fails the
first read check to block Create.
- Rate limits stop the run after retries with a clear message; sign-in
reports missing tokens, unreachable HubSpot, and timeouts, failing a
form run on the latter two.
- HubSpot API errors now carry the request method, URL, and response
body so Error Details show them; connection failures raise a dedicated
`HubSpotConnectionError`.
- Permission sync validation accepts a portal with no records instead of
raising.
- Row copy uses the shared `CheckRowText` model, and the check tests
share a helpers module.
<sup>Written for commit
|
||
|
devin-ai-integration[bot]
|
9e65247dc6 |
chore(web): remove unused refresh-components SelectButton (#16301)
## Description
#16242 moved `sections/knowledge/SourceHierarchyBrowser.tsx` to Opal
`SelectButton`. That file was the last caller of
`@/refresh-components/buttons/SelectButton`, so nothing imports the
refresh component now. The only references left were its own story and
lint config.
This PR removes:
- `web/src/refresh-components/buttons/SelectButton.tsx` and
`SelectButton.stories.tsx`.
- Its two baseline entries in `web/.oxlintrc.json`.
- Its row in
`web/tools/oxlint/opal/rules/no-replaced-refresh-import.ts`. The rule's
comment says to delete the row when the refresh module is deleted.
- The `src/refresh-components/buttons` floor in
`web/.type-coverage-baseline.yaml`, lowered from 99.7 to 99.6. The
deleted file was fully typed, so the directory ratio fell by a rounding
step.
## How Has This Been Tested?
- `bun run types:check`: no type errors.
- `bunx oxlint`: no new errors.
- Pre-commit hooks (oxfmt, oxlint, type coverage, feature-map integrity)
pass.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
Link to Devin session:
https://app.devin.ai/sessions/ba1242cc5338467c8db2fdfa02b81d10
Open in Devin Desktop:
https://app.devin.ai/desktop/session/ba1242cc5338467c8db2fdfa02b81d10?variant=devin
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Removes the now-unused `SelectButton` from `refresh-components` after
#16242 moved the last caller to Opal's component.
- Deletes `SelectButton.tsx` and its story file, along with its lint
baseline entries.
- Removes the `SelectButton` row from the `no-replaced-refresh-import`
rule.
- Lowers the type coverage baseline for `src/refresh-components/buttons`
from 99.7 to 99.6, since the removed file was fully typed.
<sup>Written for commit
|
