feat(devcontainer): CLAUDE.md overlay + Playwright e2e bootstrap docs (#11658)

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Jamison LahmanandClaude Opus 4.8 authored and GitHub committed 2026-06-03 17:37:13 +00:00
1 parent 7e8e5e7705
commit 2c3a048c7c
6 files changed
+88 -63

No files matched your search

+28 -60
View File
@@ -5,6 +5,10 @@ description: Write and maintain Playwright end-to-end tests for the Onyx applica
# Playwright E2E Tests
**Spec-authoring rules live in `web/tests/e2e/README.md`** — the Page Object Model, locator
priority, and auto-retrying matchers. Read it before adding or changing a spec. This skill covers
the surrounding workflow: layout, environment, running tests, auth, and Onyx-specific utilities.
## Project Layout
- **Tests**: `web/tests/e2e/` — organized by feature (`auth/`, `admin/`, `chat/`, `assistants/`, `connectors/`, `mcp/`)
@@ -26,15 +30,33 @@ import { TEST_ADMIN_CREDENTIALS } from "@tests/e2e/constants";
All new files should be `.ts`, not `.js`.
## Environment
Run Playwright against your **local dev servers** so the tests exercise your working-tree changes:
start `ods web dev` (frontend, `:3000`) and `ods backend api` (backend, `:8080`), then use the
config default `BASE_URL=http://localhost:3000` — no override needed. In dev, `next dev` proxies
`/api/*` to the backend itself (the route handler `web/src/app/api/[...path]/route.ts`), so the UI
and `/api` both live on `:3000`; global setup needs the backend up too, since it registers/logs in
users over `/api/auth/*`.
Readiness check: `curl -s -o /dev/null -w '%{http_code}' http://localhost:3000/api/auth/type` → `200`.
Other bootstrap notes:
- The devcontainer image bakes in Playwright's OS deps + Node, but **not** the Chromium browser binary. If a run fails with a missing-browser error, install it once: `bunx playwright install chromium`.
- Run commands from the `web/` directory; `global-setup.ts` writes auth-state files (`admin_auth.json`, …) to the cwd. It's idempotent, so setup is fast.
- `web/.vscode/.env` may be **absent** — `dotenv` skips it silently (fine for most tests). One exception: `tests/e2e/mcp/mcp_oauth_flow.spec.ts` **throws at import** without `MCP_OAUTH_*` vars, which also breaks a whole-suite `--list`. Scope runs to specific files/dirs to avoid it.
## Running Tests
```bash
# Run a specific test file
npx playwright test web/tests/e2e/chat/default_assistant.spec.ts
cd web # from the repo root
# Run a specific project
npx playwright test --project admin
npx playwright test --project exclusive
# Run a specific test file
bunx playwright test tests/e2e/chat/default_assistant.spec.ts
# Narrow to a single test by title (fast smoke check)
bunx playwright test tests/e2e/chat/welcome_page.spec.ts \
-g "chat input is visible and focusable" --project admin
```
## Test Projects
@@ -163,60 +185,6 @@ for (const theme of THEMES) {
- `TOOL_IDS` — centralized `data-testid` selectors for tool options
- `openActionManagement(page)` — opens the tool management popover
## Locator Strategy
Use locators in this priority order:
1. **`data-testid` / `aria-label`** — preferred for Onyx components
```typescript
page.getByTestId("AppSidebar/new-session")
page.getByLabel("admin-page-title")
```
2. **Role-based** — for standard HTML elements
```typescript
page.getByRole("button", { name: "Create" })
page.getByRole("dialog")
```
3. **Text/Label** — for visible text content
```typescript
page.getByText("Custom Assistant")
page.getByLabel("Email")
```
4. **CSS selectors** — last resort, only when above won't work
```typescript
page.locator('input[name="name"]')
page.locator("#onyx-chat-input-textarea")
```
**Never use** `page.locator` with complex CSS/XPath when a built-in locator works.
## Assertions
Use web-first assertions — they auto-retry until the condition is met:
```typescript
// Visibility
await expect(page.getByTestId("onyx-logo")).toBeVisible({ timeout: 5000 });
// Text content
await expect(page.getByTestId("assistant-name-display")).toHaveText("My Assistant");
// Count
await expect(page.locator('[data-testid="onyx-ai-message"]')).toHaveCount(2, { timeout: 30000 });
// URL
await expect(page).toHaveURL(/chatId=/);
// Element state
await expect(toggle).toBeChecked();
await expect(button).toBeEnabled();
```
**Never use** `assert` statements or hardcoded `page.waitForTimeout()`.
## Waiting Strategy
```typescript
@@ -240,7 +208,7 @@ await page.waitForResponse(resp => resp.url().includes("/api/chat") && resp.stat
2. **API-first setup** — use `OnyxApiClient` for backend state; reserve UI interactions for the behavior under test
3. **User isolation** — tests that modify visible app state (sidebar, chat history) should run as the worker-specific user via `loginAsWorkerUser(page, testInfo.workerIndex)` (not admin) and clean up resources in `afterAll`. Each parallel worker gets its own user, preventing cross-contamination. Reserve `loginAsRandomUser` for flows that require a brand-new user (e.g. onboarding)
4. **DRY helpers** — extract reusable logic into `utils/` with JSDoc comments
5. **No hardcoded waits** — use `waitFor`, `waitForLoadState`, or web-first assertions
5. **No hardcoded waits** — use the auto-retrying matchers (`web/tests/e2e/README.md`), `waitFor`, or `waitForLoadState`; never `waitForTimeout`
6. **Parallel-safe** — no shared mutable state between tests. Prefer static, human-readable names (e.g. `"E2E-CMD Chat 1"`) and clean up resources by ID in `afterAll`. This keeps screenshots deterministic and avoids needing to mask/hide dynamic text. Only fall back to timestamps (`\`test-${Date.now()}\``) when resources cannot be reliably cleaned up or when name collisions across parallel workers would cause functional failures
7. **Error context** — catch and re-throw with useful debug info (page text, URL, etc.)
8. **Tag slow tests** — mark serial/slow tests with `@exclusive` in the test title
+12
View File
@@ -73,6 +73,18 @@ user has read/write access to the bind-mounted workspace:
To override the auto-detection, set `DEVCONTAINER_REMOTE_USER` before running
`ods dev up`.
## Claude Code memory (devcontainer overlay)
`.devcontainer/claude-code/CLAUDE.md` holds Claude Code instructions that apply **only inside
the container** (e.g. service hostnames, "no Docker daemon in here"). It is bind-mounted
read-only to `/etc/claude-code/CLAUDE.md` — Claude Code's managed-policy memory location — so it
loads automatically alongside the project's root `CLAUDE.md`.
Because it is a live bind mount, editing the file in the repo takes effect on the next Claude
Code session — no image rebuild or container restart required. The directory (rather than the
single file) is mounted so that atomic-save editors don't detach the mount, and so additional
managed config (e.g. `managed-settings.json`) can be dropped in alongside it later.
## Firewall
The container ships with an **opt-in** default-deny firewall (`init-firewall.sh`).
+35
View File
@@ -0,0 +1,35 @@
# DEVCONTAINER OVERLAY
Running **inside the Onyx dev container**. These notes are additive to the root
`/workspace/CLAUDE.md`; on conflict with a host-oriented instruction there, prefer these.
## No Docker daemon in here
Don't use `docker` / `docker exec` / `docker compose`. Onyx services run as sibling
containers on the `onyx_default` network, reachable directly by hostname — so the root
guide's `docker exec -it onyx-relational_db-1 psql ...` won't work. Use
`psql -h relational_db -U postgres -c "<SQL>"` instead.
## Service hostnames (`onyx_default` network)
Each is also exported as an env var:
- Postgres: `relational_db` (`POSTGRES_HOST`)
- Redis: `cache` (`REDIS_HOST`)
- Vespa: `index` (`VESPA_HOST`)
- Model server: `inference_model_server` (`MODEL_SERVER_HOST`)
- OpenSearch: `opensearch` (`OPENSEARCH_HOST`)
- MinIO / S3: `minio:9000` (`S3_ENDPOINT_URL=http://minio:9000`)
## Running the app (web UI + API)
The supporting services above run as sibling containers, but the **frontend and backend are not
started for you** — run them in this container (both hot-reload):
- `ods web dev` — Next.js frontend on `localhost:3000`
- `ods backend api` — FastAPI backend (uvicorn) on `localhost:8080`
In dev mode the frontend proxies `/api/*` straight to the backend (the dev-only catch-all route
handler at `web/src/app/api/[...path]/route.ts`), so **`localhost:3000` serves both the UI and
`/api`** — no reverse proxy needed. You can also hit the backend directly at `localhost:8080` (note:
**no** `/api` prefix there — e.g. `/health`, `/auth/type`).
+1
View File
@@ -9,6 +9,7 @@
"mounts": [
"source=${localEnv:HOME}/.claude,target=/home/dev/.claude,type=bind",
"source=${localEnv:HOME}/.claude.json,target=/home/dev/.claude.json,type=bind",
"source=${localWorkspaceFolder}/.devcontainer/claude-code,target=/etc/claude-code,type=bind,readonly",
"source=${localEnv:HOME}/.zshrc,target=/home/dev/.zshrc.host,type=bind,readonly",
"source=${localEnv:HOME}/.gitconfig,target=/home/dev/.gitconfig,type=bind,readonly",
"source=${localEnv:HOME}/.config/nvim,target=/home/dev/.config/nvim,type=bind,readonly",
+3 -3
View File
@@ -4,8 +4,8 @@ This file provides guidance to AI agents when working with code in this reposito
## KEY NOTES
- If you run into any missing python dependency errors, try running your command with `source .venv/bin/activate` \
to assume the python venv.
- Python deps live in a `uv`-managed virtualenv at `.venv` (repo root). If it doesn't exist yet, create it \
with `uv sync --frozen`, then `source .venv/bin/activate`.
- To make tests work, check the `.env` file at the root of the project to find an OpenAI key.
- If using `playwright` to explore the frontend, you can usually log in with username `a@example.com` and password
`a`. The app can be accessed at `http://localhost:3000`.
@@ -195,7 +195,7 @@ Write the migration manually and place it in the file that alembic creates when
## Testing Strategy
First, you must activate the virtual environment with `source .venv/bin/activate`.
First, activate the virtualenv: `source .venv/bin/activate`. If `.venv` doesn't exist yet, create it first with `uv sync --frozen`.
There are 4 main types of tests within Onyx:
+9
View File
@@ -29,6 +29,15 @@ await expect(page.locator(".message")).toContainText("hello");
**Why:** specs that read like a description of user behavior are easier to scan, review, and refactor. Locator churn changes one POM method, not every spec that touched the surface.
**Locator priority** — when defining locators on a page object, prefer in this order:
1. `data-testid` / `aria-label` (`getByTestId`, `getByLabel`) — preferred for Onyx components.
2. Role-based (`getByRole`) — standard HTML elements.
3. Text / label (`getByText`, `getByLabel`) — visible text content.
4. CSS selectors (`locator(...)`) — last resort, only when nothing above works.
Never reach for complex CSS/XPath when a built-in locator fits.
## 2. Use auto-retrying matchers — never `getAttribute` / `evaluate` for async state
Playwright's `expect(locator).*` matchers retry until the assertion passes or the timeout expires. `locator.getAttribute()` and `page.evaluate()` are single snapshots — they read the DOM exactly once and fail immediately on a stale read.