mirror of
https://github.com/onyx-dot-app/onyx.git
synced 2026-10-22 11:22:49 -05:00
## Description
Craft development required repeated image builds, copied cluster
settings, and editor-specific Telepresence commands. Startup could
report success before services were usable.
- Share configuration resolution and launch commands between Make and VS
Code. Read deployed ConfigMaps and Secrets in memory, with personal
`.env.k8s` overrides.
- Add separate infrastructure and application readiness checks,
`craft-doctor`, and shared Telepresence connection/intercept handling.
- Reuse images, detect source changes, and provide checkout-scoped
cleanup previews. Add pause/resume and migration commands with
database-target checks.
- Default Kubernetes sandbox callbacks to the internal API Service.
Preserve explicit `ONYX_SERVER_URL` overrides and bump the Helm chart to
0.9.20.
- Update the local Kubernetes guide, Craft development skill, and
feature map.
- Give Helm chart CI a 100 GB runner disk. The previous runner exhausted
disk while unpacking OpenSearch.
- Upgrade the Storybook package family to 10.6.2 to fix the critical
audit finding GHSA-f2ph-cmg5-wxc8.
Existing `.env.k8s` files remain untouched; old service entries continue
to override chart settings. Image refresh updates the PodTemplate for
future sandboxes; existing sandbox pods retain their images.
## Review guide
[Read or download the Markdown review
guide](https://github.com/onyx-dot-app/onyx/blob/d3cbc42d89bfb150d1cfc815638645043c34b69e/docs/craft/dev/kubernetes-review-guide.md).
## How Has This Been Tested?
- Live kind and Telepresence: startup, intercept reuse, doctor, API and
worker hot reload, readiness success/failure, and migration
status/upgrade.
- Three real Craft smoke tests: sandbox provisioning, file
upload/readback, and web app preview through the frontend. No real LLM
calls were exercised.
- 76 Craft Helm tests and 27 development-tool regression tests passed.
- Pre-commit checks, explicit development-tool type checking, frontend
type coverage, and skill validation passed.
- The local web audit passes its critical-severity gate without
suppressions. The Storybook production build passes.
- Removed the disposable test cluster, local processes, and temporary
files.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Harden local Craft Kubernetes development by sharing configuration and
launch logic between Make and VS Code, and by checking service readiness
before startup reports success.
**What’s included**
- Resolve deployed ConfigMaps, Secrets, and container env in memory
through a shared launcher; personal `.env.k8s` values take precedence,
and resolved secrets never touch disk or terminal output.
- Add shared lifecycle commands (`craft-doctor`, staged
infrastructure/application readiness, Telepresence connect and
intercept, pause, resume, migration, cleanup previews) used by both Make
and VS Code.
- Reuse built images when source is unchanged, force a rebuild with
`--rebuild-sandbox-image`, and keep probe and command invocations
bounded while preserving actionable errors.
- Default sandbox callbacks to the internal API Service, preserving
explicit `ONYX_SERVER_URL` overrides; drop the now-unneeded validation
and bump the Helm chart to 0.9.20.
- Give the Helm CI runner a 100 GB disk, since Docker and kind each
store images and OpenSearch exhausted the default disk. Bump Storybook
tooling to 10.6.2 to patch a development server vulnerability.
- Update the local Kubernetes guide, Craft development skill, feature
map, and add a downloadable PR review guide.
**Validation**
- Passed 76 Craft Helm tests, 27 development-tool regression tests, and
new unit tests for the shared runtime and lifecycle CLI.
- Tested startup, readiness, Telepresence, hot reload, migrations, and
three Craft smoke tests against kind; no real LLM calls were exercised.
<sup>Written for commit d3cbc42d89.
Summary will update on new commits.</sup>
<a
href="https://cubic.dev/pr/onyx-dot-app/onyx/pull/16376?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="View guided diff"
src="https://www.cubic.dev/buttons/review-in-cubic-light.svg"></picture></a>
<a
href="https://www.cubic.dev/action/auto-fix/pr/onyx-dot-app/onyx/16376?returnTo=https%3A%2F%2Fgithub.com%2Fonyx-dot-app%2Fonyx%2Fpull%2F16376&source=description"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/turn-on-auto-fix-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/turn-on-auto-fix-light.svg"><img
alt="Turn on auto-fix"
src="https://www.cubic.dev/buttons/turn-on-auto-fix-light.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
- **New Features**
- Added a unified local Kubernetes workflow for Craft, including API and
worker launching, connectivity, readiness checks, migrations,
pause/resume, diagnostics, cleanup, and disk management.
- Added sandbox image reuse, rebuilding, loading, and configurable image
selection.
- Sandbox callbacks now default to the internal API service URL, with
support for explicit overrides.
- **Documentation**
- Added guidance for local Kubernetes setup, configuration, lifecycle
management, troubleshooting, and verification.
- **Bug Fixes**
- Improved readiness validation and stopped sensitive generated
credentials from being printed.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
74 lines
2.4 KiB
Python
74 lines
2.4 KiB
Python
#!/usr/bin/env python3
|
|
"""Launch a local Craft process with the deployed cluster configuration."""
|
|
|
|
import argparse
|
|
import os
|
|
import runpy
|
|
import subprocess
|
|
import sys
|
|
from pathlib import Path
|
|
|
|
from runtime import Cluster
|
|
|
|
ROOT = Path(__file__).resolve().parents[3]
|
|
|
|
|
|
def main() -> None:
|
|
"""Resolve cluster configuration and launch the selected local application or readiness check."""
|
|
parser = argparse.ArgumentParser(description=__doc__)
|
|
parser.add_argument(
|
|
"--cluster-name", default=os.environ.get("CLUSTER_NAME", "onyx-dev")
|
|
)
|
|
parser.add_argument("--namespace", default=os.environ.get("NAMESPACE", "onyx"))
|
|
parser.add_argument("--sandbox-namespace", default="onyx-sandboxes")
|
|
parser.add_argument("--env-file", type=Path)
|
|
parser.add_argument("role", choices=("api", "celery", "ready"))
|
|
parser.add_argument("args", nargs=argparse.REMAINDER)
|
|
args = parser.parse_args()
|
|
cluster = Cluster(ROOT, args.cluster_name, args.namespace, args.sandbox_namespace)
|
|
if args.role == "ready":
|
|
cluster.application_ready(
|
|
os.environ.get("CRAFT_FRONTEND_URL", "http://localhost:3000")
|
|
)
|
|
return
|
|
env = cluster.environment(args.env_file)
|
|
# Preserve debugger-provided process settings over personal file defaults.
|
|
for key in ("LOG_LEVEL", "PYTHONUNBUFFERED", "ENABLE_MULTIPASS_INDEXING"):
|
|
if key in os.environ:
|
|
env[key] = os.environ[key]
|
|
os.environ.update(env)
|
|
backend = ROOT / "backend"
|
|
os.chdir(backend)
|
|
sys.path.insert(0, str(backend))
|
|
forwarded = args.args[1:] if args.args[:1] == ["--"] else args.args
|
|
if args.role == "api":
|
|
sys.argv = [
|
|
"uvicorn",
|
|
"onyx.main:app",
|
|
"--reload",
|
|
"--reload-exclude",
|
|
str(backend / "onyx/skills/builtin"),
|
|
"--reload-exclude",
|
|
str(backend / "onyx/server/features/build/sandbox/image"),
|
|
"--port",
|
|
"8080",
|
|
*forwarded,
|
|
]
|
|
runpy.run_module("uvicorn", run_name="__main__")
|
|
else:
|
|
sys.argv = [str(backend / "scripts/dev_celery_reload.py"), *forwarded]
|
|
runpy.run_path(sys.argv[0], run_name="__main__")
|
|
|
|
|
|
if __name__ == "__main__":
|
|
try:
|
|
main()
|
|
except (
|
|
ValueError,
|
|
OSError,
|
|
subprocess.CalledProcessError,
|
|
subprocess.TimeoutExpired,
|
|
) as exc:
|
|
print(f"error: {exc}", file=sys.stderr)
|
|
raise SystemExit(1) from exc
|