mirror of
https://github.com/onyx-dot-app/onyx.git
synced 2026-10-22 19:32:55 -05:00
Evan LohnandClaude Opus 5.5
ed82e5a623
## Description
`BoxConnector` now implements `Resolver.reindex`. Targeted reindex can
re-fetch Box documents by id, without a crawl or a checkpoint.
**Document types covered**
- Files (`box-file-<id>`): fetched with `files.get_file_by_id`.
- Web links (`box-weblink-<id>`): fetched with
`web_links.get_web_link_by_id`, only when `include_web_links` is on.
The Box connector emits no other document types, so all types are
covered.
**How it works**
- The item's `path_collection` gives its folder chain. The connector
finds the outermost configured entry folder in that chain and rebuilds
the crawl's frontier entries from that folder down to the parent. The
`path` metadata, `parent_hierarchy_raw_node_id` and folder
`HierarchyNode`s match the crawl.
- Documents are built with the crawl's own `_convert_file` /
`_convert_web_link`. The folder `HierarchyNode` builder moved into a
shared `_folder_hierarchy_node` helper.
- Folder entries are cached for the whole call, so targets in the same
folder share upstream calls. Each folder node is yielded once.
**Failures** (each keyed on the input document id)
- Ids that do not parse as `box-file-<digits>` or
`box-weblink-<digits>`.
- Items that are not found (404), not readable (403), trashed, or
deleted.
- Items outside the configured folders.
- Web links when `include_web_links` is off.
- Files that the crawl skips (unsupported type, over the size threshold,
empty download), and conversion errors.
A `BoxAPIError` on one target does not stop the other targets.
Credential errors (HTTP 401, or OAuth `invalid_client` /
`unauthorized_client`) are raised again. Entity failures are skipped, as
in Confluence and Google Drive.
**Permissions**
With `include_permissions=True`, access uses the same EE helpers as the
perm-sync crawl: `resolve_box_ancestor_access` for the folders above the
entry folder, `resolve_box_folder_access` for each folder down to the
parent, then `resolve_box_file_access` / `resolve_box_web_link_access`
for the item. Folder nodes carry the same access too.
## How Has This Been Tested?
New daily test
`backend/tests/daily/connectors/box/test_box_resolver.py`, run against
the Box CI enterprise:
```
uv run --env-file .vscode/.env pytest -xv backend/tests/daily/connectors/box/test_box_resolver.py
13 passed
```
It covers: one file, all files plus the web link,
`include_permissions=True` (the test asserts `external_access` on docs
and nodes equals the perm-sync crawl's), a disabled web link, a file
outside the configured folder, bogus file and web link ids, unparseable
ids, a bad target next to a good one, empty input, and entity-only
input.
Also run: `backend/tests/daily/connectors/box/test_box_basic.py` (6
passed) and `backend/tests/unit/onyx/connectors/box` (61 passed).
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
- [x] [Optional] Override Linear Check
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
`BoxConnector` now implements `Resolver.reindex`, so failed documents
can be re-fetched by ID without a crawl or checkpoint.
- File (`box-file-<id>`) and web link (`box-weblink-<id>`) targets are
fetched via the Box API; web links only when `include_web_links` is on.
- The connector rebuilds the crawl's folder chain from the innermost
configured entry folder down to the item's parent, matching how the
crawl roots nested configured folders, and reuses `_convert_file` /
`_convert_web_link` for metadata, `HierarchyNode`s, and permissions.
- Failures are keyed on the input document ID for unparseable IDs,
trashed or inaccessible items, out-of-scope folders, disabled web links,
and items the crawl would skip. One bad target does not abort the
others; credential errors, including from downloads and permission
reads, are re-raised.
- With `include_permissions=True`, folder and document access match the
perm-sync crawl via the existing EE helpers.
- Adds daily and unit tests covering success paths, permission parity,
nested configured folders, and each failure case.
<sup>Written for commit c9c7d09c90.
Summary will update on new commits.</sup>
<a
href="https://cubic.dev/pr/onyx-dot-app/onyx/pull/16099?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="View guided diff"
src="https://www.cubic.dev/buttons/review-in-cubic-light.svg"></picture></a>
<a
href="https://www.cubic.dev/action/auto-fix/pr/onyx-dot-app/onyx/16099?returnTo=https%3A%2F%2Fgithub.com%2Fonyx-dot-app%2Fonyx%2Fpull%2F16099&source=description"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/turn-on-auto-fix-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/turn-on-auto-fix-light.svg"><img
alt="Turn on auto-fix"
src="https://www.cubic.dev/buttons/turn-on-auto-fix-light.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **New Features**
* Added targeted reindexing for Box files and web links using document
IDs.
* Reindexed items retain their metadata, folder hierarchy, and
permissions.
* Missing parent folders can be rebuilt during reindexing.
* Multiple items can be processed while reporting individual successes
and failures.
* Clear failures are provided for malformed, unknown, inactive,
out-of-scope, or unavailable items.
* Reindexing now respects Box item status and configured folder or
web-link scope.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
