mirror of
https://github.com/onyx-dot-app/onyx.git
synced 2026-10-23 11:53:08 -05:00
## Description
Make ODS reuse shared development infrastructure by default, with
`--isolated` for separate datasets.
- Use the same service specifications and launchers for `stack`, `web
dev`, and `backend` commands. Remove the duplicate `stack
--frontend-only` option.
- Enforce project ownership across checkouts and resolve database
commands to the actual managed database.
- Stop services before pending migrations, preserve them when already at
head, and reject manual migrations beneath managed services.
- Harden startup cancellation, shutdown grace, descendant cleanup,
terminal input, and frontend API validation. Preserve Windows foreground
commands.
- Consolidate environment helpers, protect symlinked environment files,
propagate restore errors, and update help and documentation.
This PR contains only ODS and its development-stack skill. It does not
include the plugins implementation.
## How Has This Been Tested?
- Targeted command and regression tests using fake tools and disposable
Git repositories.
- Race tests for runtime targeting, ownership, migration ordering,
process cleanup, and terminal input.
- `go vet ./...`, pre-commit checks, native builds, and Windows
cross-builds.
- The full Go suite was run. Its remaining failures are the existing
macOS devcontainer SSH-mount fixture expectations in
`TestDevcontainerCommands` and `TestRunDevRebuild`.
No real database migrations, restores, or development-stack restarts
ran. Native Windows execution remains untested.
## Additional Options
- [ ] [Optional] Please cherry-pick this PR to the latest release
version.
- [x] [Optional] Override Linear Check
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
ODS now shares development infrastructure across worktrees by default,
with `--isolated` for separate datasets. `stack`, `web dev`, and
`backend` use the same service specs and launchers, and the duplicate
`stack --frontend-only` option is removed.
- Direct launches run the same dependency-prepare step as stack startup,
so `backend` and `web dev` work with fresh or stale Python environments.
- A stopped preview discards its former project state before retargeting
to another API owner.
- Enforces project ownership across checkouts and resolves database
commands to the managed database. New tests reject conflicting targets
before any mutation.
- Stops services before pending migrations, preserves them when already
at head, and rejects manual migrations beneath managed services.
Migration probe failures fail closed.
- Hardens startup cancellation, shutdown grace, and descendant cleanup;
ODS keeps the terminal foreground group so Ctrl-C still honors shutdown
timeouts, and tmux pane identifiers no longer change service identity.
- Database passwords reach tools through the child environment only,
never in process arguments; Windows foreground commands are preserved.
- Consolidates environment helpers, protects symlinked env files,
propagates restore errors, and refreshes help and documentation. Test
commits exercise the new migration-target and preview-launch paths.
<sup>Written for commit 14500cb051.
Summary will update on new commits.</sup>
<a
href="https://cubic.dev/pr/onyx-dot-app/onyx/pull/16246?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="View guided diff"
src="https://www.cubic.dev/buttons/review-in-cubic-light.svg"></picture></a>
<a
href="https://www.cubic.dev/action/auto-fix/pr/onyx-dot-app/onyx/16246?returnTo=https%3A%2F%2Fgithub.com%2Fonyx-dot-app%2Fonyx%2Fpull%2F16246&source=description"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/turn-on-auto-fix-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/turn-on-auto-fix-light.svg"><img
alt="Turn on auto-fix"
src="https://www.cubic.dev/buttons/turn-on-auto-fix-light.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added the global `--isolated` option for separate development stacks;
shared infrastructure is the default, with clearer project ownership and
status information.
* Added `ods web dev` previews that can use a healthy managed API on
macOS and Linux, plus foreground startup on Windows.
* Expanded command guidance for development workflows, database
targeting, and dump formats.
* **Bug Fixes**
* Pending migrations now stop managed application services first;
healthy services remain running when no migrations are needed.
* Database operations verify project ownership and target selection
before making changes. Restore errors now fail the command, SQL restores
stop on errors, and temporary files are cleaned up after failures.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
222 lines
6.0 KiB
Go
222 lines
6.0 KiB
Go
// Package devstack runs local Onyx shared, isolated, and frontend stacks: it leases
|
|
// host ports, starts the application services as managed process groups,
|
|
// keeps their state in the worktree's git metadata, and reconciles a partial
|
|
// start on the next run.
|
|
package devstack
|
|
|
|
import (
|
|
"crypto/sha256"
|
|
"encoding/hex"
|
|
"encoding/json"
|
|
"errors"
|
|
"fmt"
|
|
"os"
|
|
"os/exec"
|
|
"path/filepath"
|
|
"strings"
|
|
"time"
|
|
|
|
"github.com/onyx-dot-app/onyx/tools/ods/internal/docker"
|
|
)
|
|
|
|
const (
|
|
stateFileName = "state.json"
|
|
lockFileName = "lock"
|
|
logsDirName = "logs"
|
|
|
|
// SchemaVersion is bumped when State changes shape.
|
|
SchemaVersion = 1
|
|
)
|
|
|
|
// Phases of a stack, as recorded in State.Phase.
|
|
const (
|
|
PhasePreparing = "preparing"
|
|
PhaseInfra = "infra"
|
|
PhaseMigrating = "migrating"
|
|
PhaseStarting = "starting"
|
|
PhaseReady = "ready"
|
|
PhaseDegraded = "degraded"
|
|
PhaseStopped = "stopped"
|
|
PhaseFailed = "failed"
|
|
)
|
|
|
|
// Mode selects infrastructure and process ownership.
|
|
type Mode string
|
|
|
|
const (
|
|
Shared Mode = "shared"
|
|
Isolated Mode = "isolated"
|
|
Frontend Mode = "frontend"
|
|
)
|
|
|
|
// Stack identifies a checkout and its selected lifecycle state.
|
|
type Stack struct {
|
|
// Root is the worktree root with symlinks resolved.
|
|
Root string
|
|
// StateDir holds state, locks, and logs in shared or worktree Git metadata.
|
|
StateDir string
|
|
// Project is the Docker Compose project name.
|
|
Project string
|
|
Mode Mode
|
|
ProjectExplicit bool
|
|
// Registry holds the host-wide port leases.
|
|
Registry *Registry
|
|
}
|
|
|
|
// Open resolves the selected mode and retains its saved project unless overridden.
|
|
func Open(dir, project string, mode Mode) (*Stack, error) {
|
|
if mode != Shared && mode != Isolated && mode != Frontend {
|
|
return nil, fmt.Errorf("unknown stack mode %q", mode)
|
|
}
|
|
root, err := gitOutput(dir, "rev-parse", "--show-toplevel")
|
|
if err != nil {
|
|
return nil, fmt.Errorf("%s is not inside a git worktree: %w", dir, err)
|
|
}
|
|
if root, err = filepath.EvalSymlinks(root); err != nil {
|
|
return nil, err
|
|
}
|
|
commonDir, err := gitOutput(root, "rev-parse", "--path-format=absolute", "--git-common-dir")
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
commonDir, err = filepath.EvalSymlinks(commonDir)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
stateDir := filepath.Join(commonDir, "ods-stack-shared")
|
|
projectRoot := filepath.Dir(commonDir)
|
|
if mode != Shared {
|
|
stateDir, err = gitOutput(root, "rev-parse", "--path-format=absolute", "--git-path", "ods-stack")
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
projectRoot = root
|
|
if mode == Frontend {
|
|
stateDir += "-frontend"
|
|
}
|
|
}
|
|
s := &Stack{Root: root, StateDir: stateDir, Mode: mode, ProjectExplicit: project != ""}
|
|
st, err := s.ReadState()
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
if project == "" {
|
|
if st != nil && st.Project != "" {
|
|
project = st.Project
|
|
} else {
|
|
project = ProjectName(projectRoot)
|
|
}
|
|
}
|
|
s.Project = docker.NormalizeProjectName(project)
|
|
return s, nil
|
|
}
|
|
|
|
// ProjectName derives a Compose project name that is unique per worktree
|
|
// path: the normalized directory name plus a short hash of the full path.
|
|
func ProjectName(root string) string {
|
|
var b strings.Builder
|
|
// Compose accepts lowercase ASCII letters, digits, '-' and '_' only.
|
|
for _, r := range strings.ToLower(filepath.Base(root)) {
|
|
if (r >= 'a' && r <= 'z') || (r >= '0' && r <= '9') || r == '-' || r == '_' {
|
|
b.WriteRune(r)
|
|
}
|
|
}
|
|
name := strings.Trim(b.String(), "-_")
|
|
if name == "" {
|
|
name = "onyx"
|
|
}
|
|
if len(name) > 40 {
|
|
name = name[:40]
|
|
}
|
|
sum := sha256.Sum256([]byte(root))
|
|
return name + "-" + hex.EncodeToString(sum[:])[:8]
|
|
}
|
|
|
|
func gitOutput(dir string, args ...string) (string, error) {
|
|
cmd := exec.Command("git", args...)
|
|
cmd.Dir = dir
|
|
out, err := cmd.Output()
|
|
if err != nil {
|
|
return "", err
|
|
}
|
|
return strings.TrimSpace(string(out)), nil
|
|
}
|
|
|
|
// State is the persisted record of a stack.
|
|
type State struct {
|
|
SettleTime time.Duration `json:"settle_time,omitempty"`
|
|
SchemaVersion int `json:"schema_version"`
|
|
Root string `json:"root"`
|
|
Project string `json:"project"`
|
|
Profile string `json:"profile"`
|
|
Phase string `json:"phase"`
|
|
Ports map[string]int `json:"ports"`
|
|
Services []ServiceRecord `json:"services"`
|
|
Errors []string `json:"errors,omitempty"`
|
|
UpdatedAt time.Time `json:"updated_at"`
|
|
Extra map[string]string `json:"extra,omitempty"`
|
|
}
|
|
|
|
// ReadState returns the persisted state, or nil when the stack has none.
|
|
func (s *Stack) ReadState() (*State, error) {
|
|
data, err := os.ReadFile(filepath.Join(s.StateDir, stateFileName))
|
|
if errors.Is(err, os.ErrNotExist) {
|
|
return nil, nil
|
|
}
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
var st State
|
|
if err := json.Unmarshal(data, &st); err != nil {
|
|
return nil, fmt.Errorf("parse %s: %w", filepath.Join(s.StateDir, stateFileName), err)
|
|
}
|
|
if st.Project != "" {
|
|
st.Project = docker.NormalizeProjectName(st.Project)
|
|
}
|
|
return &st, nil
|
|
}
|
|
|
|
// WriteState persists st atomically.
|
|
func (s *Stack) WriteState(st *State) error {
|
|
st.SchemaVersion = SchemaVersion
|
|
if st.Root == "" {
|
|
st.Root = s.Root
|
|
}
|
|
st.Project = s.Project
|
|
st.UpdatedAt = time.Now().UTC()
|
|
if st.Ports == nil {
|
|
st.Ports = map[string]int{}
|
|
}
|
|
if err := os.MkdirAll(s.StateDir, 0o755); err != nil {
|
|
return err
|
|
}
|
|
data, err := json.MarshalIndent(st, "", " ")
|
|
if err != nil {
|
|
return err
|
|
}
|
|
return writeFileAtomic(filepath.Join(s.StateDir, stateFileName), append(data, '\n'), 0o644)
|
|
}
|
|
|
|
// LogDir is where service logs are written.
|
|
func (s *Stack) LogDir() string {
|
|
return filepath.Join(s.StateDir, logsDirName)
|
|
}
|
|
|
|
func writeFileAtomic(path string, data []byte, perm os.FileMode) error {
|
|
tmp, err := os.CreateTemp(filepath.Dir(path), "."+filepath.Base(path)+".tmp-*")
|
|
if err != nil {
|
|
return err
|
|
}
|
|
tmpName := tmp.Name()
|
|
defer func() { _ = os.Remove(tmpName) }()
|
|
_, writeErr := tmp.Write(data)
|
|
closeErr := tmp.Close()
|
|
if err := errors.Join(writeErr, closeErr); err != nil {
|
|
return err
|
|
}
|
|
if err := os.Chmod(tmpName, perm); err != nil {
|
|
return err
|
|
}
|
|
return os.Rename(tmpName, path)
|
|
}
|