mirror of
https://github.com/onyx-dot-app/onyx.git
synced 2026-10-22 19:32:55 -05:00
## Description
Make ODS reuse shared development infrastructure by default, with
`--isolated` for separate datasets.
- Use the same service specifications and launchers for `stack`, `web
dev`, and `backend` commands. Remove the duplicate `stack
--frontend-only` option.
- Enforce project ownership across checkouts and resolve database
commands to the actual managed database.
- Stop services before pending migrations, preserve them when already at
head, and reject manual migrations beneath managed services.
- Harden startup cancellation, shutdown grace, descendant cleanup,
terminal input, and frontend API validation. Preserve Windows foreground
commands.
- Consolidate environment helpers, protect symlinked environment files,
propagate restore errors, and update help and documentation.
This PR contains only ODS and its development-stack skill. It does not
include the plugins implementation.
## How Has This Been Tested?
- Targeted command and regression tests using fake tools and disposable
Git repositories.
- Race tests for runtime targeting, ownership, migration ordering,
process cleanup, and terminal input.
- `go vet ./...`, pre-commit checks, native builds, and Windows
cross-builds.
- The full Go suite was run. Its remaining failures are the existing
macOS devcontainer SSH-mount fixture expectations in
`TestDevcontainerCommands` and `TestRunDevRebuild`.
No real database migrations, restores, or development-stack restarts
ran. Native Windows execution remains untested.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
- [x] [Optional] Override Linear Check
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
ODS now shares development infrastructure across worktrees by default,
with `--isolated` for separate datasets. `stack`, `web dev`, and
`backend` use the same service specs and launchers, and the duplicate
`stack --frontend-only` option is removed.
- Direct launches run the same dependency-prepare step as stack startup,
so `backend` and `web dev` work with fresh or stale Python environments.
- A stopped preview discards its former project state before retargeting
to another API owner.
- Enforces project ownership across checkouts and resolves database
commands to the managed database. New tests reject conflicting targets
before any mutation.
- Stops services before pending migrations, preserves them when already
at head, and rejects manual migrations beneath managed services.
Migration probe failures fail closed.
- Hardens startup cancellation, shutdown grace, and descendant cleanup;
ODS keeps the terminal foreground group so Ctrl-C still honors shutdown
timeouts, and tmux pane identifiers no longer change service identity.
- Database passwords reach tools through the child environment only,
never in process arguments; Windows foreground commands are preserved.
- Consolidates environment helpers, protects symlinked env files,
propagates restore errors, and refreshes help and documentation. Test
commits exercise the new migration-target and preview-launch paths.
<sup>Written for commit 14500cb051.
Summary will update on new commits.</sup>
<a
href="https://cubic.dev/pr/onyx-dot-app/onyx/pull/16246?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="View guided diff"
src="https://www.cubic.dev/buttons/review-in-cubic-light.svg"></picture></a>
<a
href="https://www.cubic.dev/action/auto-fix/pr/onyx-dot-app/onyx/16246?returnTo=https%3A%2F%2Fgithub.com%2Fonyx-dot-app%2Fonyx%2Fpull%2F16246&source=description"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/turn-on-auto-fix-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/turn-on-auto-fix-light.svg"><img
alt="Turn on auto-fix"
src="https://www.cubic.dev/buttons/turn-on-auto-fix-light.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added the global `--isolated` option for separate development stacks;
shared infrastructure is the default, with clearer project ownership and
status information.
* Added `ods web dev` previews that can use a healthy managed API on
macOS and Linux, plus foreground startup on Windows.
* Expanded command guidance for development workflows, database
targeting, and dump formats.
* **Bug Fixes**
* Pending migrations now stop managed application services first;
healthy services remain running when no migrations are needed.
* Database operations verify project ownership and target selection
before making changes. Restore errors now fail the command, SQL restores
stop on errors, and temporary files are cleaned up after failures.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
178 lines
4.8 KiB
Go
178 lines
4.8 KiB
Go
package devstack
|
|
|
|
import (
|
|
"errors"
|
|
"fmt"
|
|
"io/fs"
|
|
"os"
|
|
"path/filepath"
|
|
"regexp"
|
|
"sort"
|
|
"strings"
|
|
|
|
log "github.com/sirupsen/logrus"
|
|
|
|
"github.com/onyx-dot-app/onyx/tools/ods/internal/dotenv"
|
|
)
|
|
|
|
var placeholderLine = regexp.MustCompile(`^([A-Z][A-Z0-9_]*)=<REPLACE(?: THIS|_THIS)>$`)
|
|
|
|
// PrepareEnvFiles makes sure .vscode/.env and .vscode/.env.web exist as
|
|
// regular files owned by this worktree. A symlink is replaced by a copy of
|
|
// its target so a port update never edits another checkout's file. A missing
|
|
// file is created from the tracked template with placeholders blanked.
|
|
// Values are never logged.
|
|
func PrepareEnvFiles(root string) (created, localized []string, err error) {
|
|
vscode := filepath.Join(root, ".vscode")
|
|
for _, f := range []struct{ name, template string }{
|
|
{".env", "env_template.txt"},
|
|
{".env.web", "env.web_template.txt"},
|
|
} {
|
|
path := filepath.Join(vscode, f.name)
|
|
wasLink, err := localizeSymlink(path)
|
|
if err != nil {
|
|
return created, localized, err
|
|
}
|
|
if wasLink {
|
|
localized = append(localized, path)
|
|
}
|
|
if _, err := os.Lstat(path); err == nil {
|
|
continue
|
|
} else if !errors.Is(err, fs.ErrNotExist) {
|
|
return created, localized, err
|
|
}
|
|
template, err := os.ReadFile(filepath.Join(vscode, f.template))
|
|
if err != nil {
|
|
return created, localized, fmt.Errorf("read env template: %w", err)
|
|
}
|
|
if err := os.MkdirAll(vscode, 0o755); err != nil {
|
|
return created, localized, err
|
|
}
|
|
if err := writeFileAtomic(path, []byte(fillTemplate(string(template), root)), 0o600); err != nil {
|
|
return created, localized, err
|
|
}
|
|
log.Infof("Created %s from %s", path, f.template)
|
|
created = append(created, path)
|
|
}
|
|
return created, localized, nil
|
|
}
|
|
|
|
// fillTemplate blanks the <REPLACE THIS> placeholders, points SAML_CONF_DIR
|
|
// at this checkout, and turns tracing off, so the file loads without edits.
|
|
func fillTemplate(template, root string) string {
|
|
var out []string
|
|
for _, line := range strings.Split(template, "\n") {
|
|
switch {
|
|
case placeholderLine.MatchString(line):
|
|
out = append(out, placeholderLine.FindStringSubmatch(line)[1]+"=")
|
|
case strings.HasPrefix(line, "SAML_CONF_DIR="):
|
|
out = append(out, "SAML_CONF_DIR="+filepath.Join(root, "backend", "ee", "onyx", "configs", "saml_config"))
|
|
case strings.HasPrefix(line, "LANGSMITH_TRACING="):
|
|
out = append(out, "LANGSMITH_TRACING=false")
|
|
default:
|
|
out = append(out, line)
|
|
}
|
|
}
|
|
return strings.Join(out, "\n")
|
|
}
|
|
|
|
// localizeSymlink replaces a symlink at path with a regular file holding the
|
|
// target's content. A dangling symlink is removed.
|
|
func localizeSymlink(path string) (bool, error) {
|
|
info, err := os.Lstat(path)
|
|
if errors.Is(err, fs.ErrNotExist) {
|
|
return false, nil
|
|
}
|
|
if err != nil {
|
|
return false, err
|
|
}
|
|
if info.Mode()&fs.ModeSymlink == 0 {
|
|
return false, nil
|
|
}
|
|
content, err := os.ReadFile(path)
|
|
if errors.Is(err, fs.ErrNotExist) {
|
|
log.Warnf("Removing dangling symlink %s", path)
|
|
return true, os.Remove(path)
|
|
}
|
|
if err != nil {
|
|
return false, err
|
|
}
|
|
if err := os.Remove(path); err != nil {
|
|
return false, err
|
|
}
|
|
if err := writeFileAtomic(path, content, 0o600); err != nil {
|
|
return false, err
|
|
}
|
|
log.Infof("Converted %s into a worktree-local file", path)
|
|
return true, nil
|
|
}
|
|
|
|
// LoadEnvFile parses a dotenv file. A missing file yields no entries.
|
|
func LoadEnvFile(path string) ([]string, error) {
|
|
entries, err := dotenv.Load(path)
|
|
if errors.Is(err, fs.ErrNotExist) {
|
|
return nil, nil
|
|
}
|
|
return entries, err
|
|
}
|
|
|
|
// UpsertEnvFile sets values in a dotenv file.
|
|
func UpsertEnvFile(path string, values map[string]string) error {
|
|
if len(values) == 0 {
|
|
return nil
|
|
}
|
|
if _, err := localizeSymlink(path); err != nil {
|
|
return err
|
|
}
|
|
return dotenv.Upsert(path, values)
|
|
}
|
|
|
|
// MergeEnv returns base with overrides applied: an override replaces the
|
|
// base entry with the same key.
|
|
func MergeEnv(base, overrides []string) []string {
|
|
index := map[string]int{}
|
|
merged := make([]string, 0, len(base)+len(overrides))
|
|
for _, entry := range base {
|
|
key, _, _ := strings.Cut(entry, "=")
|
|
if i, ok := index[key]; ok {
|
|
merged[i] = entry
|
|
continue
|
|
}
|
|
index[key] = len(merged)
|
|
merged = append(merged, entry)
|
|
}
|
|
for _, entry := range overrides {
|
|
key, _, _ := strings.Cut(entry, "=")
|
|
if i, ok := index[key]; ok {
|
|
merged[i] = entry
|
|
continue
|
|
}
|
|
index[key] = len(merged)
|
|
merged = append(merged, entry)
|
|
}
|
|
return merged
|
|
}
|
|
|
|
// EnvValue returns the value of key in env, or "".
|
|
func EnvValue(env []string, key string) string {
|
|
for i := len(env) - 1; i >= 0; i-- {
|
|
if v, ok := strings.CutPrefix(env[i], key+"="); ok {
|
|
return v
|
|
}
|
|
}
|
|
return ""
|
|
}
|
|
|
|
func mapToEnv(m map[string]string) []string {
|
|
keys := make([]string, 0, len(m))
|
|
for k := range m {
|
|
keys = append(keys, k)
|
|
}
|
|
sort.Strings(keys)
|
|
env := make([]string, 0, len(keys))
|
|
for _, k := range keys {
|
|
env = append(env, k+"="+m[k])
|
|
}
|
|
return env
|
|
}
|