Files
onyx/tools/ods/internal/devstack/envfile.go
roshan a8033533d1 refactor(ods): unify launchers and protect shared infrastructure (#16246)
## Description

Make ODS reuse shared development infrastructure by default, with
`--isolated` for separate datasets.

- Use the same service specifications and launchers for `stack`, `web
dev`, and `backend` commands. Remove the duplicate `stack
--frontend-only` option.
- Enforce project ownership across checkouts and resolve database
commands to the actual managed database.
- Stop services before pending migrations, preserve them when already at
head, and reject manual migrations beneath managed services.
- Harden startup cancellation, shutdown grace, descendant cleanup,
terminal input, and frontend API validation. Preserve Windows foreground
commands.
- Consolidate environment helpers, protect symlinked environment files,
propagate restore errors, and update help and documentation.

This PR contains only ODS and its development-stack skill. It does not
include the plugins implementation.

## How Has This Been Tested?

- Targeted command and regression tests using fake tools and disposable
Git repositories.
- Race tests for runtime targeting, ownership, migration ordering,
process cleanup, and terminal input.
- `go vet ./...`, pre-commit checks, native builds, and Windows
cross-builds.
- The full Go suite was run. Its remaining failures are the existing
macOS devcontainer SSH-mount fixture expectations in
`TestDevcontainerCommands` and `TestRunDevRebuild`.

No real database migrations, restores, or development-stack restarts
ran. Native Windows execution remains untested.

## Additional Options

- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
- [x] [Optional] Override Linear Check

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
ODS now shares development infrastructure across worktrees by default,
with `--isolated` for separate datasets. `stack`, `web dev`, and
`backend` use the same service specs and launchers, and the duplicate
`stack --frontend-only` option is removed.

- Direct launches run the same dependency-prepare step as stack startup,
so `backend` and `web dev` work with fresh or stale Python environments.
- A stopped preview discards its former project state before retargeting
to another API owner.
- Enforces project ownership across checkouts and resolves database
commands to the managed database. New tests reject conflicting targets
before any mutation.
- Stops services before pending migrations, preserves them when already
at head, and rejects manual migrations beneath managed services.
Migration probe failures fail closed.
- Hardens startup cancellation, shutdown grace, and descendant cleanup;
ODS keeps the terminal foreground group so Ctrl-C still honors shutdown
timeouts, and tmux pane identifiers no longer change service identity.
- Database passwords reach tools through the child environment only,
never in process arguments; Windows foreground commands are preserved.
- Consolidates environment helpers, protects symlinked env files,
propagates restore errors, and refreshes help and documentation. Test
commits exercise the new migration-target and preview-launch paths.

<sup>Written for commit 14500cb051.
Summary will update on new commits.</sup>

<a
href="https://cubic.dev/pr/onyx-dot-app/onyx/pull/16246?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="View guided diff"
src="https://www.cubic.dev/buttons/review-in-cubic-light.svg"></picture></a>
<a
href="https://www.cubic.dev/action/auto-fix/pr/onyx-dot-app/onyx/16246?returnTo=https%3A%2F%2Fgithub.com%2Fonyx-dot-app%2Fonyx%2Fpull%2F16246&source=description"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/turn-on-auto-fix-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/turn-on-auto-fix-light.svg"><img
alt="Turn on auto-fix"
src="https://www.cubic.dev/buttons/turn-on-auto-fix-light.svg"></picture></a>

<!-- End of auto-generated description by cubic. -->

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added the global `--isolated` option for separate development stacks;
shared infrastructure is the default, with clearer project ownership and
status information.
* Added `ods web dev` previews that can use a healthy managed API on
macOS and Linux, plus foreground startup on Windows.
* Expanded command guidance for development workflows, database
targeting, and dump formats.
* **Bug Fixes**
* Pending migrations now stop managed application services first;
healthy services remain running when no migrations are needed.
* Database operations verify project ownership and target selection
before making changes. Restore errors now fail the command, SQL restores
stop on errors, and temporary files are cleaned up after failures.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-19 20:07:02 +00:00

178 lines
4.8 KiB
Go

package devstack
import (
"errors"
"fmt"
"io/fs"
"os"
"path/filepath"
"regexp"
"sort"
"strings"
log "github.com/sirupsen/logrus"
"github.com/onyx-dot-app/onyx/tools/ods/internal/dotenv"
)
var placeholderLine = regexp.MustCompile(`^([A-Z][A-Z0-9_]*)=<REPLACE(?: THIS|_THIS)>$`)
// PrepareEnvFiles makes sure .vscode/.env and .vscode/.env.web exist as
// regular files owned by this worktree. A symlink is replaced by a copy of
// its target so a port update never edits another checkout's file. A missing
// file is created from the tracked template with placeholders blanked.
// Values are never logged.
func PrepareEnvFiles(root string) (created, localized []string, err error) {
vscode := filepath.Join(root, ".vscode")
for _, f := range []struct{ name, template string }{
{".env", "env_template.txt"},
{".env.web", "env.web_template.txt"},
} {
path := filepath.Join(vscode, f.name)
wasLink, err := localizeSymlink(path)
if err != nil {
return created, localized, err
}
if wasLink {
localized = append(localized, path)
}
if _, err := os.Lstat(path); err == nil {
continue
} else if !errors.Is(err, fs.ErrNotExist) {
return created, localized, err
}
template, err := os.ReadFile(filepath.Join(vscode, f.template))
if err != nil {
return created, localized, fmt.Errorf("read env template: %w", err)
}
if err := os.MkdirAll(vscode, 0o755); err != nil {
return created, localized, err
}
if err := writeFileAtomic(path, []byte(fillTemplate(string(template), root)), 0o600); err != nil {
return created, localized, err
}
log.Infof("Created %s from %s", path, f.template)
created = append(created, path)
}
return created, localized, nil
}
// fillTemplate blanks the <REPLACE THIS> placeholders, points SAML_CONF_DIR
// at this checkout, and turns tracing off, so the file loads without edits.
func fillTemplate(template, root string) string {
var out []string
for _, line := range strings.Split(template, "\n") {
switch {
case placeholderLine.MatchString(line):
out = append(out, placeholderLine.FindStringSubmatch(line)[1]+"=")
case strings.HasPrefix(line, "SAML_CONF_DIR="):
out = append(out, "SAML_CONF_DIR="+filepath.Join(root, "backend", "ee", "onyx", "configs", "saml_config"))
case strings.HasPrefix(line, "LANGSMITH_TRACING="):
out = append(out, "LANGSMITH_TRACING=false")
default:
out = append(out, line)
}
}
return strings.Join(out, "\n")
}
// localizeSymlink replaces a symlink at path with a regular file holding the
// target's content. A dangling symlink is removed.
func localizeSymlink(path string) (bool, error) {
info, err := os.Lstat(path)
if errors.Is(err, fs.ErrNotExist) {
return false, nil
}
if err != nil {
return false, err
}
if info.Mode()&fs.ModeSymlink == 0 {
return false, nil
}
content, err := os.ReadFile(path)
if errors.Is(err, fs.ErrNotExist) {
log.Warnf("Removing dangling symlink %s", path)
return true, os.Remove(path)
}
if err != nil {
return false, err
}
if err := os.Remove(path); err != nil {
return false, err
}
if err := writeFileAtomic(path, content, 0o600); err != nil {
return false, err
}
log.Infof("Converted %s into a worktree-local file", path)
return true, nil
}
// LoadEnvFile parses a dotenv file. A missing file yields no entries.
func LoadEnvFile(path string) ([]string, error) {
entries, err := dotenv.Load(path)
if errors.Is(err, fs.ErrNotExist) {
return nil, nil
}
return entries, err
}
// UpsertEnvFile sets values in a dotenv file.
func UpsertEnvFile(path string, values map[string]string) error {
if len(values) == 0 {
return nil
}
if _, err := localizeSymlink(path); err != nil {
return err
}
return dotenv.Upsert(path, values)
}
// MergeEnv returns base with overrides applied: an override replaces the
// base entry with the same key.
func MergeEnv(base, overrides []string) []string {
index := map[string]int{}
merged := make([]string, 0, len(base)+len(overrides))
for _, entry := range base {
key, _, _ := strings.Cut(entry, "=")
if i, ok := index[key]; ok {
merged[i] = entry
continue
}
index[key] = len(merged)
merged = append(merged, entry)
}
for _, entry := range overrides {
key, _, _ := strings.Cut(entry, "=")
if i, ok := index[key]; ok {
merged[i] = entry
continue
}
index[key] = len(merged)
merged = append(merged, entry)
}
return merged
}
// EnvValue returns the value of key in env, or "".
func EnvValue(env []string, key string) string {
for i := len(env) - 1; i >= 0; i-- {
if v, ok := strings.CutPrefix(env[i], key+"="); ok {
return v
}
}
return ""
}
func mapToEnv(m map[string]string) []string {
keys := make([]string, 0, len(m))
for k := range m {
keys = append(keys, k)
}
sort.Strings(keys)
env := make([]string, 0, len(keys))
for _, k := range keys {
env = append(env, k+"="+m[k])
}
return env
}