diff --git a/.cursor/skills/playwright/SKILL.md b/.cursor/skills/playwright/SKILL.md index 137921901c..1b1f7df5ac 100644 --- a/.cursor/skills/playwright/SKILL.md +++ b/.cursor/skills/playwright/SKILL.md @@ -5,6 +5,10 @@ description: Write and maintain Playwright end-to-end tests for the Onyx applica # Playwright E2E Tests +**Spec-authoring rules live in `web/tests/e2e/README.md`** — the Page Object Model, locator +priority, and auto-retrying matchers. Read it before adding or changing a spec. This skill covers +the surrounding workflow: layout, environment, running tests, auth, and Onyx-specific utilities. + ## Project Layout - **Tests**: `web/tests/e2e/` — organized by feature (`auth/`, `admin/`, `chat/`, `assistants/`, `connectors/`, `mcp/`) @@ -26,15 +30,33 @@ import { TEST_ADMIN_CREDENTIALS } from "@tests/e2e/constants"; All new files should be `.ts`, not `.js`. +## Environment + +Run Playwright against your **local dev servers** so the tests exercise your working-tree changes: +start `ods web dev` (frontend, `:3000`) and `ods backend api` (backend, `:8080`), then use the +config default `BASE_URL=http://localhost:3000` — no override needed. In dev, `next dev` proxies +`/api/*` to the backend itself (the route handler `web/src/app/api/[...path]/route.ts`), so the UI +and `/api` both live on `:3000`; global setup needs the backend up too, since it registers/logs in +users over `/api/auth/*`. + +Readiness check: `curl -s -o /dev/null -w '%{http_code}' http://localhost:3000/api/auth/type` → `200`. + +Other bootstrap notes: +- The devcontainer image bakes in Playwright's OS deps + Node, but **not** the Chromium browser binary. If a run fails with a missing-browser error, install it once: `bunx playwright install chromium`. +- Run commands from the `web/` directory; `global-setup.ts` writes auth-state files (`admin_auth.json`, …) to the cwd. It's idempotent, so setup is fast. +- `web/.vscode/.env` may be **absent** — `dotenv` skips it silently (fine for most tests). One exception: `tests/e2e/mcp/mcp_oauth_flow.spec.ts` **throws at import** without `MCP_OAUTH_*` vars, which also breaks a whole-suite `--list`. Scope runs to specific files/dirs to avoid it. + ## Running Tests ```bash -# Run a specific test file -npx playwright test web/tests/e2e/chat/default_assistant.spec.ts +cd web # from the repo root -# Run a specific project -npx playwright test --project admin -npx playwright test --project exclusive +# Run a specific test file +bunx playwright test tests/e2e/chat/default_assistant.spec.ts + +# Narrow to a single test by title (fast smoke check) +bunx playwright test tests/e2e/chat/welcome_page.spec.ts \ + -g "chat input is visible and focusable" --project admin ``` ## Test Projects @@ -163,60 +185,6 @@ for (const theme of THEMES) { - `TOOL_IDS` — centralized `data-testid` selectors for tool options - `openActionManagement(page)` — opens the tool management popover -## Locator Strategy - -Use locators in this priority order: - -1. **`data-testid` / `aria-label`** — preferred for Onyx components - ```typescript - page.getByTestId("AppSidebar/new-session") - page.getByLabel("admin-page-title") - ``` - -2. **Role-based** — for standard HTML elements - ```typescript - page.getByRole("button", { name: "Create" }) - page.getByRole("dialog") - ``` - -3. **Text/Label** — for visible text content - ```typescript - page.getByText("Custom Assistant") - page.getByLabel("Email") - ``` - -4. **CSS selectors** — last resort, only when above won't work - ```typescript - page.locator('input[name="name"]') - page.locator("#onyx-chat-input-textarea") - ``` - -**Never use** `page.locator` with complex CSS/XPath when a built-in locator works. - -## Assertions - -Use web-first assertions — they auto-retry until the condition is met: - -```typescript -// Visibility -await expect(page.getByTestId("onyx-logo")).toBeVisible({ timeout: 5000 }); - -// Text content -await expect(page.getByTestId("assistant-name-display")).toHaveText("My Assistant"); - -// Count -await expect(page.locator('[data-testid="onyx-ai-message"]')).toHaveCount(2, { timeout: 30000 }); - -// URL -await expect(page).toHaveURL(/chatId=/); - -// Element state -await expect(toggle).toBeChecked(); -await expect(button).toBeEnabled(); -``` - -**Never use** `assert` statements or hardcoded `page.waitForTimeout()`. - ## Waiting Strategy ```typescript @@ -240,7 +208,7 @@ await page.waitForResponse(resp => resp.url().includes("/api/chat") && resp.stat 2. **API-first setup** — use `OnyxApiClient` for backend state; reserve UI interactions for the behavior under test 3. **User isolation** — tests that modify visible app state (sidebar, chat history) should run as the worker-specific user via `loginAsWorkerUser(page, testInfo.workerIndex)` (not admin) and clean up resources in `afterAll`. Each parallel worker gets its own user, preventing cross-contamination. Reserve `loginAsRandomUser` for flows that require a brand-new user (e.g. onboarding) 4. **DRY helpers** — extract reusable logic into `utils/` with JSDoc comments -5. **No hardcoded waits** — use `waitFor`, `waitForLoadState`, or web-first assertions +5. **No hardcoded waits** — use the auto-retrying matchers (`web/tests/e2e/README.md`), `waitFor`, or `waitForLoadState`; never `waitForTimeout` 6. **Parallel-safe** — no shared mutable state between tests. Prefer static, human-readable names (e.g. `"E2E-CMD Chat 1"`) and clean up resources by ID in `afterAll`. This keeps screenshots deterministic and avoids needing to mask/hide dynamic text. Only fall back to timestamps (`\`test-${Date.now()}\``) when resources cannot be reliably cleaned up or when name collisions across parallel workers would cause functional failures 7. **Error context** — catch and re-throw with useful debug info (page text, URL, etc.) 8. **Tag slow tests** — mark serial/slow tests with `@exclusive` in the test title diff --git a/.devcontainer/README.md b/.devcontainer/README.md index b64b1205f4..0872ad5cbd 100644 --- a/.devcontainer/README.md +++ b/.devcontainer/README.md @@ -73,6 +73,18 @@ user has read/write access to the bind-mounted workspace: To override the auto-detection, set `DEVCONTAINER_REMOTE_USER` before running `ods dev up`. +## Claude Code memory (devcontainer overlay) + +`.devcontainer/claude-code/CLAUDE.md` holds Claude Code instructions that apply **only inside +the container** (e.g. service hostnames, "no Docker daemon in here"). It is bind-mounted +read-only to `/etc/claude-code/CLAUDE.md` — Claude Code's managed-policy memory location — so it +loads automatically alongside the project's root `CLAUDE.md`. + +Because it is a live bind mount, editing the file in the repo takes effect on the next Claude +Code session — no image rebuild or container restart required. The directory (rather than the +single file) is mounted so that atomic-save editors don't detach the mount, and so additional +managed config (e.g. `managed-settings.json`) can be dropped in alongside it later. + ## Firewall The container ships with an **opt-in** default-deny firewall (`init-firewall.sh`). diff --git a/.devcontainer/claude-code/CLAUDE.md b/.devcontainer/claude-code/CLAUDE.md new file mode 100644 index 0000000000..8cd51d6b70 --- /dev/null +++ b/.devcontainer/claude-code/CLAUDE.md @@ -0,0 +1,35 @@ +# DEVCONTAINER OVERLAY + +Running **inside the Onyx dev container**. These notes are additive to the root +`/workspace/CLAUDE.md`; on conflict with a host-oriented instruction there, prefer these. + +## No Docker daemon in here + +Don't use `docker` / `docker exec` / `docker compose`. Onyx services run as sibling +containers on the `onyx_default` network, reachable directly by hostname — so the root +guide's `docker exec -it onyx-relational_db-1 psql ...` won't work. Use +`psql -h relational_db -U postgres -c ""` instead. + +## Service hostnames (`onyx_default` network) + +Each is also exported as an env var: + +- Postgres: `relational_db` (`POSTGRES_HOST`) +- Redis: `cache` (`REDIS_HOST`) +- Vespa: `index` (`VESPA_HOST`) +- Model server: `inference_model_server` (`MODEL_SERVER_HOST`) +- OpenSearch: `opensearch` (`OPENSEARCH_HOST`) +- MinIO / S3: `minio:9000` (`S3_ENDPOINT_URL=http://minio:9000`) + +## Running the app (web UI + API) + +The supporting services above run as sibling containers, but the **frontend and backend are not +started for you** — run them in this container (both hot-reload): + +- `ods web dev` — Next.js frontend on `localhost:3000` +- `ods backend api` — FastAPI backend (uvicorn) on `localhost:8080` + +In dev mode the frontend proxies `/api/*` straight to the backend (the dev-only catch-all route +handler at `web/src/app/api/[...path]/route.ts`), so **`localhost:3000` serves both the UI and +`/api`** — no reverse proxy needed. You can also hit the backend directly at `localhost:8080` (note: +**no** `/api` prefix there — e.g. `/health`, `/auth/type`). diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json index 471b3a8a9c..e2a0737c9f 100644 --- a/.devcontainer/devcontainer.json +++ b/.devcontainer/devcontainer.json @@ -9,6 +9,7 @@ "mounts": [ "source=${localEnv:HOME}/.claude,target=/home/dev/.claude,type=bind", "source=${localEnv:HOME}/.claude.json,target=/home/dev/.claude.json,type=bind", + "source=${localWorkspaceFolder}/.devcontainer/claude-code,target=/etc/claude-code,type=bind,readonly", "source=${localEnv:HOME}/.zshrc,target=/home/dev/.zshrc.host,type=bind,readonly", "source=${localEnv:HOME}/.gitconfig,target=/home/dev/.gitconfig,type=bind,readonly", "source=${localEnv:HOME}/.config/nvim,target=/home/dev/.config/nvim,type=bind,readonly", diff --git a/AGENTS.md b/AGENTS.md index 411bf1ad86..c8eeedc229 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,8 +4,8 @@ This file provides guidance to AI agents when working with code in this reposito ## KEY NOTES -- If you run into any missing python dependency errors, try running your command with `source .venv/bin/activate` \ - to assume the python venv. +- Python deps live in a `uv`-managed virtualenv at `.venv` (repo root). If it doesn't exist yet, create it \ + with `uv sync --frozen`, then `source .venv/bin/activate`. - To make tests work, check the `.env` file at the root of the project to find an OpenAI key. - If using `playwright` to explore the frontend, you can usually log in with username `a@example.com` and password `a`. The app can be accessed at `http://localhost:3000`. @@ -195,7 +195,7 @@ Write the migration manually and place it in the file that alembic creates when ## Testing Strategy -First, you must activate the virtual environment with `source .venv/bin/activate`. +First, activate the virtualenv: `source .venv/bin/activate`. If `.venv` doesn't exist yet, create it first with `uv sync --frozen`. There are 4 main types of tests within Onyx: diff --git a/web/tests/e2e/README.md b/web/tests/e2e/README.md index db61e3cbd8..0512cb9b63 100644 --- a/web/tests/e2e/README.md +++ b/web/tests/e2e/README.md @@ -29,6 +29,15 @@ await expect(page.locator(".message")).toContainText("hello"); **Why:** specs that read like a description of user behavior are easier to scan, review, and refactor. Locator churn changes one POM method, not every spec that touched the surface. +**Locator priority** — when defining locators on a page object, prefer in this order: + +1. `data-testid` / `aria-label` (`getByTestId`, `getByLabel`) — preferred for Onyx components. +2. Role-based (`getByRole`) — standard HTML elements. +3. Text / label (`getByText`, `getByLabel`) — visible text content. +4. CSS selectors (`locator(...)`) — last resort, only when nothing above works. + +Never reach for complex CSS/XPath when a built-in locator fits. + ## 2. Use auto-retrying matchers — never `getAttribute` / `evaluate` for async state Playwright's `expect(locator).*` matchers retry until the assertion passes or the timeout expires. `locator.getAttribute()` and `page.evaluate()` are single snapshots — they read the DOM exactly once and fail immediately on a stale read.