Files
DanelegendandClaude Opus 5 769ea3f7af feat(indexing): re-chunk tabular documents indexed by the old chunker (#16153) to release v4.8 (#16310)
Cherry-pick of #16153 to release v4.8 — the migration that re-chunks
tabular documents already indexed by the old chunker.

Upstream commit: e6772d6f41

**Conflict resolution** — mostly neighbouring main-only entries this
branch does not have:

- `app_base.py`, `constants.py`, `beat_schedule.py`: kept this branch's
entries and added only the `tabular_rechunk` ones. Upstream's
`cc_pair_ids_backfill` task, KV key and lock are not on this branch and
were not carried over.
- env templates and `values.yaml`: took the `TABULAR_RECHUNK_ENABLED`
entries, dropped the `SKIP_SPREADSHEET_CONTENT` ones (not a setting on
this branch).
- `Chart.yaml`: bumped this branch's version 0.8.28 → 0.8.29 and
replaced the change list, per the convention on this branch.
- `db/document.py`: kept both sides of the import block
(`KGEntity`/`KGRelationship` plus the new `FileRecord`), and added
`true` to the SQLAlchemy imports — `get_oversized_tabular_documents`
joins with it, and nothing on this branch imported it yet. The
cherry-pick also pulled in `PortedScope` and
`sample_ported_document_ids` as context; those belong to #14463, which
is not on this branch, so they were removed.

**Adapted to this branch's document index**, which still has both Vespa
and OpenSearch:

- `run_tabular_rechunk.py` imports `DocumentSectionRequest` from
`interfaces_new` (upstream's `interfaces` is the post-#15336 name).
- It resolves the index with `get_default_document_index(...)` instead
of `build_opensearch_document_index(...)`, typed `DocumentIndex`.
Hardcoding OpenSearch would read chunk heads from an index a
Vespa-serving deployment never populates, and every document would be
skipped as "not re-chunkable". `id_based_retrieval` has the same
signature on both backends. `test_run_tabular_rechunk.py` patches the
new name.
- `tabular_rechunk/tasks.py`: dropped the `SKIP_SPREADSHEET_CONTENT`
gate from `_blocked_reason`, since that setting does not exist here. The
other gates (disabled vector DB, reindex in flight, port backfilling)
are unchanged.

Verified: `tests/unit/onyx/background` and `tests/unit/onyx/indexing`
pass (303 passed, 6 skipped); `ruff check` clean on every file this
touches. `test_oversized_tabular_scan.py` came across unchanged and was
not run here (it needs Postgres).

**Stacked backport, merge last.** Order: #16296 → #16298 → #16308 →
#16309 → this PR.

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Backports #16153, the one-time migration that re-chunks tabular
documents indexed by the old chunker, to v4.8. With
`TABULAR_RECHUNK_ENABLED` set, a beat task re-chunks oversized tabular
documents in place from the stored workbook, without re-crawling the
source; a KV record stores the cursor and completion state, so it
resumes across ticks and never re-runs once finished.

**Backport adaptations**

- Left out upstream content not on this branch: the
`cc_pair_ids_backfill` task, KV key, lock, and the `PortedScope` pieces
from #14463.
- The run imports `DocumentSectionRequest` from `interfaces_new`
(upstream's `interfaces` is the post-#15336 name), and resolves the
index with `get_default_document_index(...)` typed `DocumentIndex`
instead of `build_opensearch_document_index(...)`: this branch serves
both Vespa and OpenSearch, and hardcoding OpenSearch would skip every
document under a Vespa-serving deployment.
- The `SKIP_SPREADSHEET_CONTENT` gate was dropped from `_blocked_reason`
because that setting does not exist here; the remaining gates (disabled
vector DB, reindex in flight, port backfilling) are unchanged.
- `db/document.py` keeps both import blocks and adds `true` to the
SQLAlchemy imports; `miscellaneous_utils.py` gains
`experts_from_stores_representations` for the owner round-trip.

Verified: `tests/unit/onyx/background` and `tests/unit/onyx/indexing`
pass (303 passed, 6 skipped), and `ruff check` is clean on every touched
file. `test_oversized_tabular_scan.py` needs Postgres and was not run.

<sup>Written for commit 8e1c69b270.
Summary will update on new commits.</sup>

<a
href="https://cubic.dev/pr/onyx-dot-app/onyx/pull/16310?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="View guided diff"
src="https://www.cubic.dev/buttons/review-in-cubic-light.svg"></picture></a>
<a
href="https://www.cubic.dev/action/auto-fix/pr/onyx-dot-app/onyx/16310?returnTo=https%3A%2F%2Fgithub.com%2Fonyx-dot-app%2Fonyx%2Fpull%2F16310&source=description"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/turn-on-auto-fix-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/turn-on-auto-fix-light.svg"><img
alt="Turn on auto-fix"
src="https://www.cubic.dev/buttons/turn-on-auto-fix-light.svg"></picture></a>

<!-- End of auto-generated description by cubic. -->

- [x] [Optional] Override Linear Check

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-21 02:18:17 +00:00
..

Onyx CLI

Release CLI PyPI

A CLI for querying enterprise knowledge from Onyx. Includes an interactive chat TUI for humans and non-interactive commands for AI agents and scripts.

Installation

pip install onyx-cli

Or with uv:

uv pip install onyx-cli

Standalone binaries for Linux, macOS, and Windows (amd64/arm64) are also attached to each cli/v* GitHub release.

Setup

Run the interactive chat TUI — on first launch it will guide you through setup:

onyx-cli chat

This prompts for your Onyx server URL and personal access token (PAT), tests the connection, and saves config to ~/.config/onyx-cli/config.json (or $XDG_CONFIG_HOME/onyx-cli/config.json if set). To reconfigure later, use the /configure command inside the TUI.

Environment variables override config file values:

Variable Required Description
ONYX_SERVER_URL No Server origin or already-prefixed API base (default: https://cloud.onyx.app)
ONYX_API_PREFIX No API path prefix (default: /api); set to empty for direct backend access
ONYX_PAT No Personal access token for authentication (required if no config file)
ONYX_PERSONA_ID No Default agent/persona ID
ONYX_STREAM_MARKDOWN No Enable/disable progressive markdown rendering (true/false)
ONYX_SSH_HOST_KEY No Path to SSH host key for serve command

Usage

Interactive chat

onyx-cli chat
onyx-cli chat --no-stream-markdown
onyx-cli chat --agent-name "Support Agent"
Flag Description
--no-stream-markdown Disable progressive markdown rendering during streaming
--agent-id <int> Agent ID to use for this session
--agent-name <string> Agent name to use for this session (exact or unique substring; mutually exclusive with --agent-id)

One-shot question

onyx-cli ask "What is our company's PTO policy?"
onyx-cli ask --agent-id 5 "Summarize this topic"
onyx-cli ask --agent-name "Support Agent" "hello"
onyx-cli ask --json "Hello"
Flag Description
--agent-id <int> Agent ID to use (overrides default)
--agent-name <string> Agent name to use (exact or unique substring; mutually exclusive with --agent-id)
--json Output NDJSON stream events instead of plain text
--prompt <string> Question text (use with piped stdin context)
--quiet Buffer output and print once at end
--max-output <int> Max bytes before truncating (0 to disable)

List agents

onyx-cli agents
onyx-cli agents --json

Serve over SSH

# Start a public SSH endpoint for the CLI TUI
onyx-cli serve --host 0.0.0.0 --port 2222

# Connect as a client
ssh your-host -p 2222

Clients can either:

  • paste a personal access token (PAT) at the login prompt, or
  • skip the prompt by sending ONYX_PAT over SSH:
export ONYX_PAT=your-pat
ssh -o SendEnv=ONYX_PAT your-host -p 2222

Useful hardening flags:

  • --host-key (default ~/.config/onyx-cli/host_ed25519)
  • --idle-timeout (default 15m)
  • --max-session-timeout (default 8h)
  • --rate-limit-per-minute (default 20)
  • --rate-limit-burst (default 40)
  • --rate-limit-cache (default 4096)

Commands

Command Mode Description
chat Interactive Launch the interactive chat TUI (requires terminal)
ask Agent / Script Ask a question and print the answer to stdout
agents Agent / Script List available agents (ID, name, description)
validate-config Agent / Script Check CLI configuration and server connectivity
install-skill Agent / Script Install the Onyx CLI agent skill file
experiments Agent / Script List experimental features and their status
serve Interactive Serve the Onyx TUI over SSH

Global Flags

Flag Description
--version, -v Print client and server version information
--debug Run in debug mode (verbose logging)

Agent / Non-Interactive Use

When called without a TTY (e.g., by an AI agent or piped into another command), onyx-cli adjusts its behavior:

  • No subcommand: prints help and exits 0 (instead of launching the TUI)
  • Results to stdout, progress/errors to stderr
  • No ANSI codes or interactive prompts
  • ask output truncated to 50000 bytes by default; full response saved to a temp file. Use --max-output 0 to disable.
  • search stdout stays valid JSON: over the limit, whole results are dropped and a truncation object carries metadata plus the temp file path of the full response.

Configuration

If a human has already run onyx-cli chat (which includes first-time setup), the CLI works out of the box — no additional setup needed. Environment variables can override the config file or serve as an alternative when no config file exists:

export ONYX_SERVER_URL="https://your-onyx-server.com"
export ONYX_PAT="your-pat"

Exit Codes

Code Name When
0 Success Command completed
1 General Unknown error
2 BadRequest Invalid arguments
3 NotConfigured Missing config/PAT
4 AuthFailure Invalid PAT (401/403)
5 Unreachable Server unreachable
6 RateLimited Server returned 429
7 Timeout Request timed out
8 ServerError Server returned 5xx
9 NotAvailable Feature/endpoint doesn't exist

Skill File

Install the bundled SKILL.md so AI coding agents can discover the CLI:

onyx-cli install-skill
onyx-cli install-skill --global
onyx-cli install-skill --copy
onyx-cli install-skill --agent claude-code
Flag Description
--global, -g Install to home directory instead of project
--copy Copy files instead of symlinking
--agent, -a Target specific agents (e.g. claude-code; can be repeated)

Slash Commands (in TUI)

Command Description
/help Show help message
/clear Clear chat and start a new session
/agent List and switch agents (by ID or name)
/attach <path> Attach a file to next message
/sessions List recent chat sessions
/configure Re-run connection setup
/connectors Open connectors in browser
/settings Open settings in browser
/quit Exit Onyx CLI

Keyboard Shortcuts

Key Action
Enter Send message
Escape Cancel current generation
Ctrl+O Toggle source citations
Ctrl+D Quit (press twice)
Scroll / Shift+Up/Down Scroll chat history
Page Up / Page Down Scroll half page

Building from Source

Requires Go 1.24+.

cd cli
go build -o onyx-cli .

Development

# Run tests
go test ./...

# Build
go build -o onyx-cli .

# Lint
golangci-lint run ./...

Publishing to PyPI

The CLI is distributed as a Python package via PyPI. The build system uses hatchling with manygo to cross-compile Go binaries into platform-specific wheels.

Tag a release and push — the release-cli.yml workflow builds wheels for all platforms and publishes to PyPI automatically. ods release cli calculates and pushes the next tag for you:

ods release cli              # bumps the patch version
ods release cli --bump minor
ods release cli --version 1.5.0
ods release cli --dry-run    # computes the version, tags nothing

To do this manually:

git tag cli/v0.1.0
git push origin cli/v0.1.0

Important

ods release tags HEAD, not main, and does not check which branch you are on. Compare git rev-parse HEAD with git rev-parse origin/main first. Tagging from a stale branch publishes that branch under the new version.

The workflow builds wheels for:

  • linux/amd64 manylinux
  • linux/amd64 musllinux
  • linux/arm64 manylinux
  • linux/arm64 musllinux
  • darwin/amd64
  • darwin/arm64
  • windows/amd64
  • windows/arm64

Manual release

Build a wheel locally with uv. Set GOOS and GOARCH to cross-compile for other platforms (Go handles this natively — no cross-compiler needed):

# Build for current platform
uv build --wheel

# Cross-compile for a different platform
GOOS=linux GOARCH=amd64 uv build --wheel

# Build a musllinux-tagged Linux wheel for Alpine/musl environments
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 \
  ONYX_CLI_WHEEL_PLATFORM_TAG=musllinux_1_2_x86_64 \
  uv build --wheel

# Upload to PyPI
uv publish

Build constraints

The release workflows build every wheel against tools/requirements/build-constraints.txt, shared by cli, tools/ods and tools/ods-audit. They pass it as UV_BUILD_CONSTRAINT with UV_REQUIRE_HASHES, so the build environment (hatchling, go-bin, manygo and their dependencies) is pinned and verified.

A plain local uv build resolves those dependencies freely. Set the same variables to build against the pinned closure:

UV_BUILD_CONSTRAINT="$(git rev-parse --show-toplevel)/tools/requirements/build-constraints.txt" \
  UV_REQUIRE_HASHES=true GOTOOLCHAIN=local \
  uv build --wheel

tools/requirements/build-constraints.txt is compiled from its .in by the pip-compile pre-commit hook, so a plain pre-commit run pip-compile refreshes it. To upgrade a pinned build dependency, edit build-constraints.in and the matching [build-system] requires in all three pyproject.toml files; the build-constraints-drift hook fails if they disagree.

go-bin ships the Go toolchain used to compile the binary. Bumping Go means moving it everywhere at once: the go directive in go.mod, cli/Dockerfile, the setup-go and GO_VERSION pins across .github/workflows/, .devcontainer/Dockerfile, the go-bin pin in all three pyproject.toml files and in tools/requirements/build-constraints.in, and the compiled build-constraints.txt.

Versioning

Versions are derived from git tags with the cli/ prefix (e.g. cli/v0.1.0). The tag is parsed by internal/_version.py and injected into the Go binary via -ldflags at build time.