mirror of
https://github.com/vrtmrz/obsidian-livesync.git
synced 2026-10-19 01:31:09 -05:00
Integrate opt-in NFC path identity and combined regression coverage
This commit is contained in:
commit
e6794ca967
39 files changed
+2246
-96
No files matched your search
@@ -0,0 +1,135 @@
|
||||
---
|
||||
date: 2026-10-17
|
||||
commonlib-version: "0.1.42"
|
||||
self-hosted-livesync-version: "1.0.37"
|
||||
status: unreleased
|
||||
---
|
||||
|
||||
# Architectural Decision Record: Unicode Path Identity for CouchDB
|
||||
|
||||
## Status
|
||||
|
||||
Accepted on 2026-10-17. The feature remains unreleased. Local validation
|
||||
evidence and platform limits are recorded in the design document.
|
||||
|
||||
## Context
|
||||
|
||||
Self-hosted LiveSync derives normal-file Metadata document IDs from Vault paths.
|
||||
Unicode permits visually identical text to use different code-point sequences.
|
||||
For example, `é` can be represented as one precomposed character or as `e`
|
||||
followed by a combining acute accent. Filesystems and applications can expose
|
||||
these forms differently. Without a shared canonical form, two clients can
|
||||
derive different document IDs for the same logical path, or fail to resolve an
|
||||
existing document when another client reports an equivalent spelling.
|
||||
|
||||
Changing only a lookup performed when receiving a document is insufficient.
|
||||
New documents, edits, deletions, local provenance, and resumable Fetch
|
||||
checkpoints must all use the same path identity. Applying the change to a
|
||||
populated database in place would
|
||||
also leave old document IDs and revision histories behind. Distinct files or
|
||||
Metadata documents can collapse to one identity under NFC, so automatic
|
||||
selection or merging could lose user data.
|
||||
|
||||
The Metadata path and file content have separate roles: Metadata stores file
|
||||
properties and references to Chunks, while Chunks store the content. A path
|
||||
identity change must not require physical files to be renamed or content to be
|
||||
rewritten in Metadata.
|
||||
|
||||
## Decision
|
||||
|
||||
Add the opt-in `pathNormalizationVersion` setting. An absent value and `0`
|
||||
retain the existing path-to-ID behaviour. Version `1` applies Unicode
|
||||
Normalization Form C (NFC) to the logical path used for Metadata IDs and path
|
||||
comparison keys. Existing case handling remains an independent rule; CLI path
|
||||
resolution uses NFC equivalence regardless of the `handleFilenameCaseSensitive`
|
||||
setting. NFC changes logical identity only: it does not rewrite the stored
|
||||
`Metadata.path`, rename a physical file, or change Chunk IDs or content. It is
|
||||
independent of `idDerivationVersion` and does not require encryption.
|
||||
|
||||
When the CLI resolves a path, it uses the actual spelling of each existing
|
||||
file and directory. A missing component is created under the actual resolved
|
||||
parent using the spelling received for that component. Obsidian resolves paths
|
||||
through its `TFile` and Vault APIs. Its collision checks are limited to the
|
||||
paths those APIs expose and do not promise to find raw physical aliases hidden
|
||||
by the Obsidian index.
|
||||
|
||||
The initial support boundary is CLI and Obsidian clients using CouchDB.
|
||||
Journal and P2P reject version `1`; Direct file access refuses a remote which
|
||||
declares the NFC feature. Other host adapters are not supported for version `1`
|
||||
until they provide equivalent path resolution and ambiguity handling within
|
||||
their supported storage listing.
|
||||
|
||||
Version `1` is represented by the required `nfc-path-identity-v1` feature in
|
||||
the existing generation-13 database version document. Every client must agree
|
||||
on the path identity before it can write Metadata. A mode `0` client cannot
|
||||
join a database requiring this feature. An existing database without the
|
||||
feature cannot acquire it through an ordinary Metadata write while it already
|
||||
contains Metadata. These admission checks remain active when ordinary
|
||||
configuration-mismatch warnings are disabled.
|
||||
|
||||
`pathNormalizationVersion` is a required Tweak even when encryption is
|
||||
disabled. It does not replace or modify `idDerivationVersion`.
|
||||
|
||||
The supported migration is reconstruction, not an in-place ID rewrite:
|
||||
|
||||
1. Update every participating client and stop synchronisation.
|
||||
2. Select a complete, authoritative Vault and resolve NFC-equivalent files or
|
||||
Metadata documents which would produce the same ID.
|
||||
3. Set version `1` on the source client and Rebuild both databases from that
|
||||
Vault.
|
||||
4. Set version `1` on each remaining client and Fetch the rebuilt remote.
|
||||
5. Resume synchronisation only after every client has the same path identity.
|
||||
|
||||
In Obsidian, the setting remains pending through ordinary settings saves, the
|
||||
settings-only Apply fallback, and reload. It becomes active only when the
|
||||
existing Rebuild or Fetch operation is successfully reserved. A `remoteOnly`
|
||||
operation cannot apply a pending identity change, and the settings-only
|
||||
fallback is not offered. The Setup URI shares
|
||||
`pathNormalizationVersion` so a new client receives the intended value;
|
||||
importing that value does not replace the required Rebuild or Fetch operation.
|
||||
|
||||
The CLI exposes explicit `rebuild` and `fetch` commands. Both require
|
||||
`--force` before application start-up because they replace local or remote
|
||||
database state. `--force` confirms the requested operation; it does not bypass
|
||||
identity checks, collision preflight, compatibility checks, or required human
|
||||
remediation. The commands use the existing Commonlib Rebuilder; this change
|
||||
does not introduce a second migration engine.
|
||||
|
||||
The migration procedure also uses `--write-settings` to retain the selected
|
||||
local database and settings across subsequent CLI invocations. Collision
|
||||
preflight uses the supported host listing and local Metadata. It refuses
|
||||
ambiguous file or parent-directory candidates before database reconstruction;
|
||||
it does not claim to detect physical aliases which a host does not expose.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Use NFC by default:** rejected because existing clients and databases must
|
||||
retain their current IDs until their owners deliberately migrate them.
|
||||
- **Normalise only lookups:** rejected because new Metadata writes and deletes
|
||||
could continue using legacy IDs and create duplicate documents.
|
||||
- **Change `idDerivationVersion`:** rejected because it controls cryptographic
|
||||
ID derivation, not Unicode canonicalisation, and must remain meaningful when
|
||||
encryption is disabled.
|
||||
- **Rename files to NFC:** rejected because the decision concerns logical
|
||||
identity, while physical spelling belongs to the Vault and its filesystem.
|
||||
- **Rewrite remote IDs automatically or merge collisions:** rejected because
|
||||
changing document IDs cannot preserve CouchDB revision ancestry, and the
|
||||
system cannot choose safely between distinct file histories.
|
||||
|
||||
## Consequences
|
||||
|
||||
Legacy mode remains unchanged, and NFC identity requires a coordinated,
|
||||
destructive Rebuild and Fetch. An ambiguity found in the supported storage
|
||||
listing or local Metadata stops reconstruction before either database is
|
||||
reset; the user must resolve the names or synchronisation scope. The check is
|
||||
bounded by each host's listing contract: Obsidian uses its `TFile` and Vault
|
||||
APIs and does not scan the physical Vault for aliases hidden by that index.
|
||||
No claim is made that raw physical aliases outside a host's supported listing
|
||||
are detected.
|
||||
|
||||
## References
|
||||
|
||||
- [Unicode path identity design](../design_docs/unicode_path_identity.md)
|
||||
- [Setting details](../settings.md#unicode-path-identity)
|
||||
- [Commonlib Unicode path identity contract](https://github.com/vrtmrz/livesync-commonlib/blob/main/docs/unicode-path-identity.md)
|
||||
- [Project glossary](../glossary.md)
|
||||
@@ -0,0 +1,263 @@
|
||||
---
|
||||
date: 2026-10-17
|
||||
commonlib-version: "0.1.42"
|
||||
self-hosted-livesync-version: "1.0.37"
|
||||
status: unreleased
|
||||
---
|
||||
|
||||
# Unicode path identity
|
||||
|
||||
## Purpose and baseline
|
||||
|
||||
This design defines how Self-hosted LiveSync opts into Unicode NFC for normal
|
||||
file Metadata document IDs, how hosts preserve the Vault's physical names, and
|
||||
how existing databases move to the new identity. The accepted rationale is in
|
||||
the [Unicode path identity ADR](../adr/2026_10_unicode_path_identity.md).
|
||||
Commonlib's [consumer contract](https://github.com/vrtmrz/livesync-commonlib/blob/main/docs/unicode-path-identity.md)
|
||||
defines the shared package boundary.
|
||||
|
||||
This records an unreleased change to Self-hosted LiveSync `1.0.37`, using the
|
||||
published Commonlib `0.1.42` package as an exact dependency.
|
||||
|
||||
## Scope
|
||||
|
||||
| Area | Contract |
|
||||
| --- | --- |
|
||||
| Logical identity | Version `1` uses NFC for Metadata ID and path comparison keys. Existing case handling remains a separate rule. Missing and `0` retain legacy behaviour. |
|
||||
| Stored paths and content | Keep the spelling in `Metadata.path` and existing physical Vault names. NFC does not change Chunk IDs or content. |
|
||||
| Initial hosts and remote | Support CLI and Obsidian with CouchDB. |
|
||||
| Other transports | Journal and P2P reject version `1`; Direct file access refuses a remote declaring the NFC feature. Other host adapters remain outside the support matrix. |
|
||||
| Migration | Rebuild both databases from one complete authoritative Vault, then Fetch each remaining client. Do not rewrite IDs or merge histories in place. |
|
||||
|
||||
## Path identity contract
|
||||
|
||||
`pathNormalizationVersion` selects the logical path used by Commonlib's
|
||||
`PathService` for normal-file Metadata IDs and path comparison keys:
|
||||
|
||||
| Value | Path identity |
|
||||
| --- | --- |
|
||||
| Missing or `0` | Preserve the legacy path and ID behaviour exactly. |
|
||||
| `1` | Apply Unicode NFC to the logical path, then apply the existing case handling and ID derivation rules. |
|
||||
| Any other value | Reject it; do not fall back to legacy mode. |
|
||||
|
||||
NFC changes only logical identity and comparison keys. It is not a physical
|
||||
filename conversion, a rewrite of `Metadata.path`, or a Unicode compatibility
|
||||
transformation such as NFKC. Existing case handling remains governed by
|
||||
`handleFilenameCaseSensitive`; it is independent of NFC equivalence resolution
|
||||
in the CLI. `idDerivationVersion` continues to select its separate
|
||||
cryptographic ID construction. NFC path identity works with or without
|
||||
encryption and does not change Chunk IDs or content.
|
||||
|
||||
The version is a required Tweak, even when encryption is disabled, represented
|
||||
by `nfc-path-identity-v1` in the existing generation-13 database version
|
||||
document's `used_features` list. The Setup URI includes
|
||||
`pathNormalizationVersion`, decodes an absent value as `0`, and rejects
|
||||
unsupported values. Sharing the setting communicates the target identity;
|
||||
setup still requires an explicit Rebuild or Fetch choice.
|
||||
|
||||
Fast Fetch checkpoints include the path identity version. A checkpoint created
|
||||
under legacy identity cannot be resumed under version `1`, or vice versa. This
|
||||
reuses the existing checkpoint identity and does not add a separate recovery
|
||||
store.
|
||||
|
||||
## Admission and database compatibility
|
||||
|
||||
Clients validate local and remote database identity before accepting ordinary
|
||||
replication, including when configuration-mismatch warnings are disabled. The
|
||||
checks cover both directions:
|
||||
|
||||
| Client setting | Database feature | Result |
|
||||
| --- | --- | --- |
|
||||
| Missing or `0` | No NFC feature | Continue with legacy identity. |
|
||||
| `1` | NFC feature present | Continue with NFC identity. |
|
||||
| `1` | Neither database has the feature or Metadata | A fresh database may declare the feature before its first Metadata mutation. |
|
||||
| `1` | A database has Metadata without the feature | Reject ordinary writes; require Rebuild. |
|
||||
| Missing or `0` | A local or remote database has the NFC feature | Reject; the database requires a version-`1` client. |
|
||||
|
||||
The feature is independent of encryption. Receiving an incompatible version
|
||||
document records the incompatibility and stops the active synchronisation
|
||||
session. This is a compatibility fence, not a rollback of operations already
|
||||
admitted elsewhere; every device must be stopped before migration begins.
|
||||
|
||||
## Host path resolution and collision preflight
|
||||
|
||||
Commonlib owns logical ID derivation and the Rebuilder's preflight contract.
|
||||
Each host maps that logical path to entries in its supported storage listing.
|
||||
|
||||
The CLI resolves existing files and directories using NFC comparison keys,
|
||||
regardless of `handleFilenameCaseSensitive`. It carries the actual spelling
|
||||
reported for each existing component through reads, writes, deletes, stat,
|
||||
directory creation, and rename operations. If a path has missing components,
|
||||
the CLI uses the actual existing parent and preserves the received spelling for
|
||||
each new component. A scan enumerates each directory once and stats the raw
|
||||
paths returned by that enumeration; it does not repeat canonical lookup for
|
||||
each listed entry. No persistent index or recovery store is added.
|
||||
|
||||
If more than one entry resolves to the same logical comparison key after NFC
|
||||
and the configured case rule at any component, including a parent directory,
|
||||
the path is ambiguous. The CLI warns and refuses the operation; it must not
|
||||
select a candidate based on directory enumeration order, merge the entries, or
|
||||
rename them. The same rule applies during Rebuild preflight.
|
||||
|
||||
Obsidian resolution remains within its `TFile` and Vault API boundary. Its
|
||||
collision preflight checks paths exposed through those APIs; it does not walk
|
||||
the physical Vault independently and cannot promise to detect raw filesystem
|
||||
aliases hidden by Obsidian's index. Other host adapters remain outside the
|
||||
version-`1` support matrix until their path resolution and ambiguity handling
|
||||
are implemented and verified within their own supported listings.
|
||||
|
||||
Before resetting either database, Rebuild checks the supported storage
|
||||
listing and local Metadata for paths which would resolve to the same identity
|
||||
under NFC and the configured case rule. When a local database is not open,
|
||||
preflight checks the storage paths returned by the host listing. If multiple
|
||||
candidate files or Metadata documents collide, or a parent component is
|
||||
ambiguous, the operation stops before local or remote mutation. It does not
|
||||
choose a winner, merge revision histories, or rename a Vault file. The user
|
||||
must resolve the ambiguous names or exclude one file from the synchronisation
|
||||
scope, then retry the reconstruction. This check is bounded by the host's
|
||||
listing contract and does not imply that every raw physical alias is visible.
|
||||
|
||||
## Reconstruction and client operations
|
||||
|
||||
Mode changes are never applied by an ordinary Metadata write. In Obsidian, the
|
||||
Advanced page offers **Legacy path identity** (`0`) and **Unicode NFC path
|
||||
identity** (`1`) for CouchDB. A changed `pathNormalizationVersion` remains
|
||||
pending through ordinary settings saves, the settings-only Apply fallback, and
|
||||
reload. It becomes active only after the existing Rebuild or Fetch operation
|
||||
has been successfully reserved; cancellation or failed reservation leaves it
|
||||
pending. A `remoteOnly` operation cannot apply a pending identity change, and
|
||||
the settings-only fallback is not offered.
|
||||
|
||||
Migration uses this ordered procedure:
|
||||
|
||||
1. Update every CLI and Obsidian client which will use the database. Stop
|
||||
synchronisation and editing on all devices, and keep a complete backup.
|
||||
2. Select the Vault whose complete file set is authoritative. Resolve any
|
||||
collision reported by Rebuild before continuing.
|
||||
3. Select version `1` on that source. In Obsidian, choose **Overwrite Server
|
||||
Data with This Device's Files**; the new mode becomes active only when the
|
||||
Rebuild reservation succeeds. In the CLI, use settings with
|
||||
`pathNormalizationVersion: 1` and run `rebuild` with `--force
|
||||
--write-settings`. Both local and remote databases are reconstructed from
|
||||
the specified physical Vault.
|
||||
4. Select version `1` on each remaining client. In Obsidian, choose **Reset
|
||||
Synchronisation on This Device**; the new mode becomes active only when the
|
||||
Fetch reservation succeeds. In the CLI, use settings with
|
||||
`pathNormalizationVersion: 1` and run `fetch` with `--force
|
||||
--write-settings`. Each operation replaces that client's local database
|
||||
from the rebuilt remote.
|
||||
5. Confirm that every participating client is using version `1`, then resume
|
||||
synchronisation.
|
||||
|
||||
The CLI checks `--force` before core start-up so an omitted confirmation cannot
|
||||
trigger database access or destructive preparation. Neither `--force` nor a
|
||||
headless default choice overrides an NFC collision, a feature mismatch, or a
|
||||
preflight failure. If a required setup decision cannot be made in a headless
|
||||
CLI invocation, the command must fail with a clear remediation message rather
|
||||
than report success after cancellation.
|
||||
|
||||
`--write-settings` retains the Rebuilder's selected local database name and
|
||||
lasting settings changes for subsequent invocations. Without this option, the
|
||||
existing CLI policy leaves the settings file unchanged, so a later command can
|
||||
reopen the previously selected database.
|
||||
|
||||
## State which follows the selected identity
|
||||
|
||||
The path identity is also used by two existing resume mechanisms:
|
||||
|
||||
- **File-reflection provenance:** the existing device-local provenance storage
|
||||
interface remains an exact-key store. The private
|
||||
`ServiceFileHandlerBase.getProvenancePath()` method canonicalises provenance
|
||||
keys with NFC in mode `1` for reads, writes, deletes, raw snapshot lookup,
|
||||
and untracked revision recording. Mode `0` retains the old keys. Rebuild and
|
||||
Fetch use the existing local-state reconstruction and replacement lifecycle;
|
||||
no new persistent field or in-place key migration is introduced.
|
||||
- **Fast Fetch checkpoints:** the path version participates in checkpoint
|
||||
identity so an interrupted transfer cannot resume using a different
|
||||
path-to-ID rule. Existing checkpoint persistence and retry handling remain
|
||||
in control.
|
||||
|
||||
These rules keep a physical NFD filename addressable when an NFC-equivalent
|
||||
Metadata path arrives, while using one canonical identity for subsequent
|
||||
updates and deletions.
|
||||
|
||||
## Compatibility matrix
|
||||
|
||||
| Client and remote | Version `0` | Version `1` |
|
||||
| --- | --- | --- |
|
||||
| CLI with CouchDB | Supported with legacy identity. | Supported with NFC path resolution. |
|
||||
| Obsidian with CouchDB | Supported with legacy identity. | Supported with NFC path resolution. |
|
||||
| Journal or P2P | Existing behaviour. | Rejected. |
|
||||
| Direct file access | Existing behaviour. | Refuses a remote requiring `nfc-path-identity-v1`. |
|
||||
| Other host adapters, including WebApp | Existing behaviour, subject to their existing support. | Outside the initial support matrix until physical-path handling is verified. |
|
||||
| Older client without this feature | Can use a legacy database. | Cannot join a database declaring `nfc-path-identity-v1`. |
|
||||
|
||||
## Verification contract and limits
|
||||
|
||||
The implementation acceptance matrix covers:
|
||||
|
||||
- unchanged legacy IDs for missing and explicit version `0`, plus matching
|
||||
ASCII IDs in both modes;
|
||||
- identical normal-file Metadata IDs for NFC and NFD spellings in version `1`,
|
||||
with and without path obfuscation, and with the existing case rule applied;
|
||||
- Metadata creation, update, deletion, and file-reflection provenance using the
|
||||
same canonical identity without renaming the physical file;
|
||||
- CLI reads, writes, deletes, stat, directory creation, and rename through the
|
||||
actual spelling of existing files and parents; preservation of the received
|
||||
spelling for new components; and resolution independent of the case setting;
|
||||
- refusal to write when an existing legacy database has Metadata, refusal of
|
||||
version-`0` clients against an NFC database, and enforcement when ordinary
|
||||
mismatch warnings are disabled;
|
||||
- URI round trips, checkpoint invalidation across path versions, and transport
|
||||
rejection for Journal, P2P, and Direct file access;
|
||||
- refusal for ambiguous file or parent-directory candidates before local or
|
||||
remote reset, preserving both databases on rejection, within the paths
|
||||
returned by each host's supported listing; and
|
||||
- CLI `rebuild` and `fetch` command dispatch, pre-start `--force` enforcement,
|
||||
and end-to-end creation, update, and deletion through CouchDB between CLI and
|
||||
Obsidian.
|
||||
|
||||
Focused unit and package checks do not substitute for the real CouchDB and
|
||||
Obsidian round trip using the exact published Commonlib artefact. Obsidian checks
|
||||
are limited to paths exposed through its `TFile` and Vault APIs; they do not
|
||||
establish detection of physical aliases hidden by the Obsidian index. No result
|
||||
for raw physical filesystem aliases outside a host's supported listing is
|
||||
claimed.
|
||||
|
||||
Local validation on Linux with real Obsidian `1.12.7` and the published
|
||||
Commonlib `0.1.42` package passed after the CLI path resolution, case-aware
|
||||
collision scan, and pending-setting changes. It covered:
|
||||
|
||||
- NFC identity with unencrypted Metadata;
|
||||
- NFC identity with E2EE and Path Obfuscation;
|
||||
- two legacy CLI databases migrated through `rebuild` and `fetch`, followed by
|
||||
the encrypted Obsidian round trip; and
|
||||
- the existing encrypted ASCII scenario in legacy mode.
|
||||
|
||||
In those NFC scenarios, an NFD directory and mixed-form filename were retained
|
||||
while checking content, Metadata creation time, remote updates, absence of an
|
||||
NFC duplicate, and remote deletion through the polling CLI daemon. The `mirror`
|
||||
command retains tombstoned local files by default; its update behaviour was
|
||||
tested separately from daemon deletion reflection.
|
||||
|
||||
## Non-goals
|
||||
|
||||
This design does not:
|
||||
|
||||
- change Chunk IDs, E2EE, Properties encryption, or the Metadata and Chunk
|
||||
document formats;
|
||||
- convert or rename existing physical files;
|
||||
- migrate remote document IDs or revision histories in place;
|
||||
- choose among ambiguous NFC-equivalent files or parents, or merge their
|
||||
content;
|
||||
- change the default away from legacy path identity;
|
||||
- enable version `1` for Journal, P2P, Direct file access, WebApp, or an
|
||||
unverified host adapter; or
|
||||
- claim native iOS or macOS filesystem compatibility.
|
||||
|
||||
## References
|
||||
|
||||
- [Unicode path identity ADR](../adr/2026_10_unicode_path_identity.md)
|
||||
- [Setting details](../settings.md#unicode-path-identity)
|
||||
- [Commonlib Unicode path identity contract](https://github.com/vrtmrz/livesync-commonlib/blob/main/docs/unicode-path-identity.md)
|
||||
- [Metadata and Chunks glossary](../glossary.md#user-facing-and-operational-terms)
|
||||
@@ -1050,6 +1050,18 @@ When this setting is disabled, changing only the letter case of a file name with
|
||||
|
||||
New Vaults use case-insensitive handling for cross-platform compatibility. Existing settings with an explicit value preserve that choice. Earlier releases also followed the case-insensitive branch when this value was absent, so 1.0 saves a missing legacy value as `false` without requiring a compatibility review or database rebuild.
|
||||
|
||||
#### Unicode path identity
|
||||
|
||||
Setting key: pathNormalizationVersion
|
||||
|
||||
Version `0` preserves legacy file identity and is the default for existing and new settings. Version `1` uses Unicode NFC when deriving document IDs, so equivalent composed and decomposed paths identify the same file across the CLI and Obsidian. Existing physical file names are retained. This mode currently supports CouchDB only.
|
||||
|
||||
Before enabling version `1`, update every participating client, stop synchronisation, and back up a complete authoritative Vault. Rebuild both databases from that Vault, then select version `1` and Fetch on each remaining device. Changing this setting without reconstruction is refused when it would mix database identities. Do not resume an older client against the migrated remote.
|
||||
|
||||
The selected value stays pending through ordinary settings saves and reloads. It becomes active only when the Rebuild or Fetch operation is successfully reserved; cancellation leaves it pending.
|
||||
|
||||
Distinct files or Metadata documents which derive the same identity stop reconstruction before either database is reset. This check uses the host's supported file listing; Obsidian checks the paths exposed by its Vault APIs and cannot detect physical aliases hidden by its index. Resolve the duplicate names or synchronisation scope first; Self-hosted LiveSync does not merge them automatically. Case-insensitive handling also applies the existing letter-case rule.
|
||||
|
||||
### 4. Compatibility (Internal API Usage)
|
||||
|
||||
#### Scan changes on customisation sync
|
||||
|
||||
+45
-3
@@ -64,9 +64,10 @@ livesync-cli [database-path] [command] [args...]
|
||||
|
||||
- `database-path`: Path to the directory where `.livesync` folder and `settings.json` are (or will be) located.
|
||||
- Note: In previous versions, this was referred to as the "vault" path. Now it is clearly distinguished from the actual vault (the directory containing your `.md` files).
|
||||
- `--vault <path>` / `-V <path>`: (daemon/mirror only) Path to the vault directory containing `.md` files.
|
||||
- `--vault <path>` / `-V <path>`: (daemon/mirror/rebuild/fetch) Path to the Vault directory containing `.md` files.
|
||||
- Allows the PouchDB database directory and the actual vault directory to be different locations.
|
||||
- For `mirror` command, the positional `[vault-path]` argument takes precedence over `--vault`.
|
||||
- `--force` / `-f`: Required by `rebuild` and `fetch` because they replace a database. It also retains its existing `init-settings` overwrite behaviour.
|
||||
- `--write-settings`: Write setting migrations and other lasting changes after the command succeeds.
|
||||
- `init-settings` writes its target file. `setup`, `remote-add`, `remote-rm`, `remote-set`, and `remote-activate` write their settings changes without this option.
|
||||
- All remaining commands leave the settings file unchanged by default.
|
||||
@@ -82,6 +83,9 @@ livesync-cli [database-path] [command] [args...]
|
||||
- If `vault-path` is provided, the CLI will synchronise the database with files in the vault directory.
|
||||
- If `vault-path` is omitted, it defaults to `database-path` (compatibility mode).
|
||||
- Use this command to keep your local `.md` files in sync with the database.
|
||||
- `rebuild`: Rebuild the local and remote databases from the selected Vault. This replaces existing local and remote database state and requires `--force`.
|
||||
- This command may select a new local database suffix. Add `--write-settings` if later CLI commands should use the resulting database selection; otherwise the existing settings file remains unchanged.
|
||||
- `fetch`: Replace the local database with the configured remote state, then reflect fetched files to the selected Vault. Existing local database state is discarded, so this command requires `--force`.
|
||||
- `ls [prefix]`: List files currently stored in the local database.
|
||||
- `push <src> <dst>`: Push a local file `<src>` into the database at path `<dst>`.
|
||||
- `pull <src> <dst>`: Pull a file `<src>` from the database into local file `<dst>`.
|
||||
@@ -110,6 +114,12 @@ livesync-cli ./my-db sync
|
||||
# Mirroring to your actual Obsidian vault
|
||||
livesync-cli ./my-db mirror /path/to/obsidian-vault
|
||||
|
||||
# Rebuild local and remote databases from an authoritative Vault
|
||||
livesync-cli ./my-db --vault /path/to/obsidian-vault rebuild --force --write-settings
|
||||
|
||||
# Replace the local database and Vault files from the configured remote
|
||||
livesync-cli ./my-db --vault /path/to/obsidian-vault fetch --force --write-settings
|
||||
|
||||
# Manual file operations
|
||||
livesync-cli ./my-db push ./note.md folder/note.md
|
||||
livesync-cli ./my-db pull folder/note.md ./note.md
|
||||
@@ -369,6 +379,36 @@ The CLI uses the same settings format as the Obsidian plug-in. Create a `.livesy
|
||||
- `couchDB_DBNAME`: Database name
|
||||
- `isConfigured`: Set to `true` after configuration
|
||||
|
||||
#### Unicode path identity
|
||||
|
||||
The `pathNormalizationVersion` setting controls how file paths identify
|
||||
Metadata documents when using CouchDB. An absent value or `0` keeps the legacy
|
||||
identity. Version `1` uses Unicode Normalization Form C (NFC) for Metadata IDs
|
||||
and path comparison keys. It does not rewrite `Metadata.path`, rename existing
|
||||
files, or change Chunks or file content. `handleFilenameCaseSensitive` remains
|
||||
an independent case-handling setting; the CLI resolves NFC-equivalent names
|
||||
even when case-sensitive handling is enabled.
|
||||
|
||||
When the CLI resolves an existing path, it uses the actual spelling of each
|
||||
matching file or directory for reads, writes, deletes, `stat` calls, directory
|
||||
creation, and renames. If a path component is missing, it uses the actual
|
||||
existing parent and preserves the spelling received for the new component. If
|
||||
a file or any parent directory has multiple candidates for the same logical
|
||||
key after NFC and the existing case rule, the CLI warns and refuses the
|
||||
operation. It does not choose a candidate based on directory order, merge
|
||||
entries, or rename them.
|
||||
|
||||
Version `1` requires a coordinated Rebuild and Fetch; changing the setting by
|
||||
itself does not migrate a database. Update every participating client, stop
|
||||
synchronisation, and back up a complete authoritative Vault. Set
|
||||
`pathNormalizationVersion` to `1` on the source client, then run `rebuild`
|
||||
with `--force --write-settings`. Set the same value on each remaining CLI
|
||||
client, then run `fetch` with `--force --write-settings`. Use **Overwrite
|
||||
Server Data with This Device's Files** and **Reset Synchronisation on This
|
||||
Device** for the corresponding Obsidian Rebuild and Fetch operations. Resolve
|
||||
any ambiguous names before reconstruction. Version `1` is supported with
|
||||
CouchDB only; do not resume a legacy client against the rebuilt database.
|
||||
|
||||
### Command-line Reference
|
||||
|
||||
```
|
||||
@@ -381,11 +421,11 @@ Arguments:
|
||||
|
||||
Options:
|
||||
--settings, -s <path> Path to settings file (default: .livesync/settings.json in local database directory)
|
||||
--force, -f Overwrite existing file on init-settings
|
||||
--force, -f Required by rebuild/fetch; overwrite existing file on init-settings
|
||||
--verbose, -v Enable verbose logging
|
||||
--debug, -d Enable debug logging (includes verbose)
|
||||
--interval <N>, -i <N> (daemon only) Poll CouchDB every N seconds instead of using the _changes feed
|
||||
--vault <path>, -V <path> (daemon/mirror) Path to vault directory, decoupled from database-path
|
||||
--vault <path>, -V <path> (daemon/mirror/rebuild/fetch) Path to vault directory, decoupled from database-path
|
||||
--write-settings Write setting changes after a successful command
|
||||
--compat-remote-admin-exit-zero
|
||||
Preserve the former zero exit code when remote-administration verification fails
|
||||
@@ -398,6 +438,8 @@ Commands:
|
||||
p2p-peers <timeout> Show discovered peers as [peer]<TAB><peer-id><TAB><peer-name>
|
||||
p2p-sync <peer> <timeout> Synchronise with specified peer-id or peer-name
|
||||
p2p-host Start P2P host mode and wait until interrupted (Ctrl+C)
|
||||
rebuild Rebuild local and remote databases from the selected Vault (requires --force)
|
||||
fetch Replace the local database with remote state and reflect it to the Vault (requires --force)
|
||||
push <src> <dst> Push local file <src> into local database path <dst>
|
||||
pull <src> <dst> Pull file <src> from local database into local file <dst>
|
||||
pull-rev <src> <dst> <rev> Pull specific revision <rev> into local file <dst>
|
||||
|
||||
@@ -3,7 +3,7 @@ import type { IFileSystemAdapter } from "@vrtmrz/livesync-commonlib/compat/servi
|
||||
import { NodePathAdapter } from "./NodePathAdapter";
|
||||
import { NodeTypeGuardAdapter } from "./NodeTypeGuardAdapter";
|
||||
import { NodeConversionAdapter } from "./NodeConversionAdapter";
|
||||
import { NodeStorageAdapter } from "@vrtmrz/livesync-commonlib/node";
|
||||
import { AmbiguousNormalisedPathError, UnicodeResolvingNodeStorageAdapter } from "./UnicodeResolvingNodeStorageAdapter";
|
||||
import { NodeVaultAdapter } from "./NodeVaultAdapter";
|
||||
import type { NodeFile, NodeFolder, NodeStat } from "./NodeTypes";
|
||||
import { path } from "@vrtmrz/livesync-commonlib/node";
|
||||
@@ -16,19 +16,21 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
|
||||
readonly path: NodePathAdapter;
|
||||
readonly typeGuard: NodeTypeGuardAdapter;
|
||||
readonly conversion: NodeConversionAdapter;
|
||||
readonly storage: NodeStorageAdapter;
|
||||
readonly storage: UnicodeResolvingNodeStorageAdapter;
|
||||
readonly vault: NodeVaultAdapter;
|
||||
|
||||
private fileCache = new Map<string, NodeFile>();
|
||||
|
||||
constructor(
|
||||
private basePath: string,
|
||||
private reportDiagnostic: CliDiagnosticReporter = () => undefined
|
||||
private reportDiagnostic: CliDiagnosticReporter = () => undefined,
|
||||
getPathNormalizationVersion: () => number = () => 0,
|
||||
isCaseInsensitive: () => boolean = () => false
|
||||
) {
|
||||
this.path = new NodePathAdapter();
|
||||
this.typeGuard = new NodeTypeGuardAdapter();
|
||||
this.conversion = new NodeConversionAdapter();
|
||||
this.storage = new NodeStorageAdapter(basePath);
|
||||
this.storage = new UnicodeResolvingNodeStorageAdapter(basePath, getPathNormalizationVersion, isCaseInsensitive);
|
||||
this.vault = new NodeVaultAdapter(this.storage);
|
||||
}
|
||||
|
||||
@@ -58,6 +60,14 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
|
||||
|
||||
async getAbstractFileByPath(p: FilePath | string): Promise<NodeFile | null> {
|
||||
const pathStr = this.normalisePath(p);
|
||||
if (this.storage.isUnicodePathNormalizationEnabled) {
|
||||
const resolvedPath = await this.storage.resolveLogicalPath(pathStr, false, false);
|
||||
if (resolvedPath === null) {
|
||||
this.fileCache.delete(pathStr);
|
||||
return null;
|
||||
}
|
||||
return await this.refreshResolvedFile(resolvedPath);
|
||||
}
|
||||
if (!this.fileCache.has(pathStr) && !(await this.hasExactPathCase(pathStr))) {
|
||||
this.fileCache.delete(pathStr);
|
||||
return null;
|
||||
@@ -68,6 +78,15 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
|
||||
async getAbstractFileByPathInsensitive(p: FilePath | string): Promise<NodeFile | null> {
|
||||
const pathStr = this.normalisePath(p);
|
||||
|
||||
if (this.storage.isUnicodePathNormalizationEnabled) {
|
||||
const resolvedPath = await this.storage.resolveLogicalPath(pathStr, true, false);
|
||||
if (resolvedPath === null) {
|
||||
this.fileCache.delete(pathStr);
|
||||
return null;
|
||||
}
|
||||
return await this.refreshResolvedFile(resolvedPath);
|
||||
}
|
||||
|
||||
const exact = await this.getAbstractFileByPath(pathStr);
|
||||
if (exact) {
|
||||
return exact;
|
||||
@@ -115,17 +134,34 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
|
||||
// This is used by Obsidian to sync internal file metadata
|
||||
}
|
||||
|
||||
async refreshFile(p: string): Promise<NodeFile | null> {
|
||||
async refreshFile(p: string, caseInsensitive = false): Promise<NodeFile | null> {
|
||||
const pathStr = this.normalisePath(p);
|
||||
try {
|
||||
const stat = await this.storage.stat(pathStr);
|
||||
if (stat?.type !== "file") {
|
||||
const resolvedPath = this.storage.isUnicodePathNormalizationEnabled
|
||||
? await this.storage.resolveLogicalPath(pathStr, caseInsensitive, false)
|
||||
: pathStr;
|
||||
if (resolvedPath === null) {
|
||||
this.fileCache.delete(pathStr);
|
||||
return null;
|
||||
}
|
||||
return await this.refreshResolvedFile(resolvedPath);
|
||||
} catch (error) {
|
||||
if (error instanceof AmbiguousNormalisedPathError) throw error;
|
||||
this.fileCache.delete(pathStr);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private async refreshResolvedFile(resolvedPath: string): Promise<NodeFile | null> {
|
||||
try {
|
||||
const stat = await this.storage.statExact(resolvedPath);
|
||||
if (stat?.type !== "file") {
|
||||
this.fileCache.delete(resolvedPath);
|
||||
return null;
|
||||
}
|
||||
|
||||
const file: NodeFile = {
|
||||
path: pathStr as FilePath,
|
||||
path: resolvedPath as FilePath,
|
||||
stat: {
|
||||
size: stat.size,
|
||||
mtime: stat.mtime,
|
||||
@@ -133,11 +169,12 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
|
||||
type: "file",
|
||||
},
|
||||
};
|
||||
this.fileCache.set(pathStr, file);
|
||||
this.fileCache.set(resolvedPath, file);
|
||||
return file;
|
||||
} catch {
|
||||
} catch (error) {
|
||||
if (error instanceof AmbiguousNormalisedPathError) throw error;
|
||||
// Evict so a deleted file is not returned by subsequent cache scans.
|
||||
this.fileCache.delete(pathStr);
|
||||
this.fileCache.delete(resolvedPath);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
@@ -146,18 +183,21 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
|
||||
* Helper method to recursively scan directory and populate file cache
|
||||
*/
|
||||
async scanDirectory(relativePath: string = ""): Promise<void> {
|
||||
await this.scanDirectoryInto(relativePath, this.fileCache);
|
||||
const resolvedPath = await this.storage.resolveLogicalPath(relativePath, false, false);
|
||||
if (resolvedPath !== null) await this.scanDirectoryInto(resolvedPath, this.fileCache);
|
||||
}
|
||||
|
||||
private async scanDirectoryInto(relativePath: string, files: Map<string, NodeFile>): Promise<void> {
|
||||
const fullPath = this.resolvePath(relativePath);
|
||||
try {
|
||||
const directoryStat = await this.storage.stat(relativePath);
|
||||
const directoryStat = this.storage.isUnicodePathNormalizationEnabled
|
||||
? await this.storage.statExact(relativePath)
|
||||
: await this.storage.stat(relativePath, false);
|
||||
if (directoryStat?.type !== "folder") throw new Error(`Directory does not exist: ${fullPath}`);
|
||||
const entries = await this.storage.list(relativePath);
|
||||
const entries = await this.storage.listExact(relativePath);
|
||||
|
||||
for (const entryPath of entries.files) {
|
||||
const stat = await this.storage.stat(entryPath);
|
||||
const stat = await this.storage.statExact(entryPath);
|
||||
if (stat?.type !== "file") continue;
|
||||
const file: NodeFile = {
|
||||
path: entryPath as FilePath,
|
||||
@@ -169,6 +209,7 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
|
||||
await this.scanDirectoryInto(entryPath, files);
|
||||
}
|
||||
} catch (error) {
|
||||
if (error instanceof AmbiguousNormalisedPathError) throw error;
|
||||
// Directory doesn't exist or is not readable
|
||||
this.reportDiagnostic(`Error scanning directory ${fullPath}:`, error);
|
||||
}
|
||||
|
||||
@@ -1,5 +1,12 @@
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { fsPromises as fs, os, path } from "@vrtmrz/livesync-commonlib/node";
|
||||
import { fsPromises as fs, NodeStorageAdapter, os, path } from "@vrtmrz/livesync-commonlib/node";
|
||||
import { DEFAULT_SETTINGS, type FilePath } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import {
|
||||
ServiceRebuilder,
|
||||
type ServiceRebuilderDependencies,
|
||||
} from "@vrtmrz/livesync-commonlib/compat/serviceModules/Rebuilder";
|
||||
import { PathServiceCompat } from "@vrtmrz/livesync-commonlib/compat/services/implements/injectable/InjectablePathService";
|
||||
import { ServiceContext } from "@vrtmrz/livesync-commonlib/compat/services/base/ServiceBase";
|
||||
import { NodeFileSystemAdapter } from "./NodeFileSystemAdapter";
|
||||
|
||||
describe("NodeFileSystemAdapter file enumeration", () => {
|
||||
@@ -116,3 +123,382 @@ describe("NodeFileSystemAdapter file enumeration", () => {
|
||||
await expect(adapter.getFiles()).resolves.toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("NodeFileSystemAdapter Unicode path resolution", () => {
|
||||
const tempDirs: string[] = [];
|
||||
|
||||
async function createVault() {
|
||||
const directory = await fs.mkdtemp(path.join(os.tmpdir(), "livesync-cli-unicode-"));
|
||||
tempDirs.push(directory);
|
||||
return directory;
|
||||
}
|
||||
|
||||
function createNormalisingAdapter(directory: string, caseInsensitive = false) {
|
||||
return new NodeFileSystemAdapter(
|
||||
directory,
|
||||
undefined,
|
||||
() => 1,
|
||||
() => caseInsensitive
|
||||
);
|
||||
}
|
||||
|
||||
afterEach(async () => {
|
||||
await Promise.all(tempDirs.splice(0).map((directory) => fs.rm(directory, { recursive: true, force: true })));
|
||||
});
|
||||
|
||||
it.each([0, 1])("keeps an NFD-only file in the storage listing in identity mode %s", async (version) => {
|
||||
const directory = await createVault();
|
||||
const originalPath = "café.md".normalize("NFD");
|
||||
await fs.writeFile(path.join(directory, originalPath), "original");
|
||||
const report = vi.fn();
|
||||
const adapter = new NodeFileSystemAdapter(directory, report, () => version);
|
||||
|
||||
expect((await adapter.getFiles()).map((file) => file.path)).toEqual([originalPath]);
|
||||
expect(report).not.toHaveBeenCalled();
|
||||
await expect(fs.readFile(path.join(directory, originalPath), "utf8")).resolves.toBe("original");
|
||||
expect(await fs.readdir(directory)).toEqual([originalPath]);
|
||||
});
|
||||
|
||||
it.each([false, true])(
|
||||
"enumerates nested mixed names once per directory with case-insensitive matching %s",
|
||||
async (caseInsensitive) => {
|
||||
const directory = await createVault();
|
||||
const parent = "café".normalize("NFD");
|
||||
const child = `${parent}/Étage`;
|
||||
await fs.mkdir(path.join(directory, child), { recursive: true });
|
||||
const names = Array.from(
|
||||
{ length: 12 },
|
||||
(_, index) =>
|
||||
`${child}/${index}-résumé.md`
|
||||
.normalize(index % 2 ? "NFC" : "NFD")
|
||||
.split("/")
|
||||
.pop()!
|
||||
);
|
||||
for (const name of names) await fs.writeFile(path.join(directory, child, name), "content");
|
||||
const adapter = createNormalisingAdapter(directory, caseInsensitive);
|
||||
const list = vi.spyOn(NodeStorageAdapter.prototype, "list");
|
||||
try {
|
||||
const files = await adapter.getFiles();
|
||||
expect(files.map((file) => file.path).sort()).toEqual(names.map((name) => `${child}/${name}`).sort());
|
||||
expect(list).toHaveBeenCalledTimes(3);
|
||||
} finally {
|
||||
list.mockRestore();
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
it("uses the existing mixed parent during directory preparation and preserves the new leaf spelling", async () => {
|
||||
const directory = await createVault();
|
||||
const parent = "café".normalize("NFD");
|
||||
const child = `${parent}/Étage`;
|
||||
await fs.mkdir(path.join(directory, child), { recursive: true });
|
||||
const adapter = createNormalisingAdapter(directory);
|
||||
await adapter.storage.mkdir("café");
|
||||
await adapter.storage.mkdir("café/Étage");
|
||||
const newName = "résumé.md".normalize("NFD");
|
||||
const file = await adapter.vault.create(`café/Étage/${newName}`, "created");
|
||||
|
||||
expect(file.path).toBe(`${child}/${newName}`);
|
||||
expect(await fs.readdir(directory)).toEqual([parent]);
|
||||
expect(await fs.readdir(path.join(directory, parent))).toEqual(["Étage"]);
|
||||
expect(await fs.readdir(path.join(directory, child))).toEqual([newName]);
|
||||
await expect(adapter.storage.stat(`café/Étage/${newName.normalize("NFC")}`)).resolves.toMatchObject({
|
||||
size: 7,
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects an NFC-equivalent file and directory before scanning or writing a child", async () => {
|
||||
const directory = await createVault();
|
||||
const composed = "café";
|
||||
const decomposed = composed.normalize("NFD");
|
||||
await fs.writeFile(path.join(directory, composed), "file");
|
||||
await fs.mkdir(path.join(directory, decomposed));
|
||||
const adapter = createNormalisingAdapter(directory);
|
||||
|
||||
await expect(adapter.getFiles()).rejects.toThrow(/ambiguous/i);
|
||||
await expect(adapter.storage.write(`${composed}/new.md`, "new")).rejects.toThrow(/ambiguous/i);
|
||||
await expect(fs.readFile(path.join(directory, composed), "utf8")).resolves.toBe("file");
|
||||
expect(await fs.readdir(path.join(directory, decomposed))).toEqual([]);
|
||||
});
|
||||
|
||||
it.each([false, true])(
|
||||
"enumerates each directory once for a flat NFC scan with case-insensitive matching %s",
|
||||
async (caseInsensitive) => {
|
||||
const directory = await createVault();
|
||||
for (let index = 0; index < 12; index++) {
|
||||
await fs.writeFile(path.join(directory, `${index}.md`), "content");
|
||||
}
|
||||
const adapter = createNormalisingAdapter(directory, caseInsensitive);
|
||||
const list = vi.spyOn(NodeStorageAdapter.prototype, "list");
|
||||
try {
|
||||
expect(await adapter.getFiles()).toHaveLength(12);
|
||||
expect(list).toHaveBeenCalledTimes(1);
|
||||
} finally {
|
||||
list.mockRestore();
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
it("reads, updates, and deletes an NFD file through its NFC path", async () => {
|
||||
const directory = await createVault();
|
||||
const nfcFolder = "café";
|
||||
const nfdFolder = nfcFolder.normalize("NFD");
|
||||
const nfcName = "Ångström.md";
|
||||
const nfdName = nfcName.normalize("NFD");
|
||||
await fs.mkdir(path.join(directory, nfdFolder));
|
||||
await fs.writeFile(path.join(directory, nfdFolder, nfdName), "before", "utf8");
|
||||
const adapter = createNormalisingAdapter(directory);
|
||||
|
||||
const file = await adapter.getAbstractFileByPath(`${nfcFolder}/${nfcName}`);
|
||||
expect(file?.path).toBe(`${nfdFolder}/${nfdName}`);
|
||||
if (!file) throw new Error("Expected the NFC path to resolve to the NFD file");
|
||||
await expect(adapter.vault.read(file)).resolves.toBe("before");
|
||||
await adapter.vault.modify(file, "after");
|
||||
await expect(fs.readFile(path.join(directory, nfdFolder, nfdName), "utf8")).resolves.toBe("after");
|
||||
expect(await fs.readdir(directory)).toEqual([nfdFolder]);
|
||||
expect(await fs.readdir(path.join(directory, nfdFolder))).toEqual([nfdName]);
|
||||
|
||||
await adapter.vault.delete(file);
|
||||
|
||||
await expect(fs.readdir(path.join(directory, nfdFolder))).resolves.toEqual([]);
|
||||
});
|
||||
|
||||
it("creates a new file inside an existing NFD directory using that directory spelling", async () => {
|
||||
const directory = await createVault();
|
||||
const nfdFolder = "café".normalize("NFD");
|
||||
await fs.mkdir(path.join(directory, nfdFolder));
|
||||
const adapter = createNormalisingAdapter(directory);
|
||||
|
||||
const file = await adapter.vault.create("café/new.md", "content");
|
||||
|
||||
expect(file.path).toBe(`${nfdFolder}/new.md`);
|
||||
await expect(fs.readFile(path.join(directory, nfdFolder, "new.md"), "utf8")).resolves.toBe("content");
|
||||
expect(await fs.readdir(directory)).toEqual([nfdFolder]);
|
||||
});
|
||||
|
||||
it("renames a file by case inside an NFD directory while preserving that directory spelling", async () => {
|
||||
const directory = await createVault();
|
||||
const nfcFolder = "Caf\u00e9";
|
||||
const nfdFolder = nfcFolder.normalize("NFD");
|
||||
await fs.mkdir(path.join(directory, nfdFolder));
|
||||
await fs.writeFile(path.join(directory, nfdFolder, "Note.md"), "content", "utf8");
|
||||
const adapter = createNormalisingAdapter(directory, true);
|
||||
const file = await adapter.getAbstractFileByPathInsensitive(`${nfcFolder}/Note.md`);
|
||||
if (!file) throw new Error("Expected the NFC path to resolve to the NFD file");
|
||||
|
||||
const renamedFile = await adapter.renameFile(file, `${nfcFolder}/note.md`);
|
||||
|
||||
expect(renamedFile.path).toBe(`${nfdFolder}/note.md`);
|
||||
expect(await fs.readdir(directory)).toEqual([nfdFolder]);
|
||||
await expect(fs.readdir(path.join(directory, nfdFolder))).resolves.toEqual(["note.md"]);
|
||||
await expect(fs.readFile(path.join(directory, nfdFolder, "note.md"), "utf8")).resolves.toBe("content");
|
||||
});
|
||||
|
||||
it("rejects an NFC/NFD file collision for lookup and vault mutations", async () => {
|
||||
const directory = await createVault();
|
||||
const nfcName = "café.md";
|
||||
const nfdName = nfcName.normalize("NFD");
|
||||
await fs.writeFile(path.join(directory, nfcName), "NFC", "utf8");
|
||||
await fs.writeFile(path.join(directory, nfdName), "NFD", "utf8");
|
||||
const adapter = createNormalisingAdapter(directory);
|
||||
const exactFile = {
|
||||
path: nfcName as FilePath,
|
||||
stat: { ctime: 1, mtime: 2, size: 3, type: "file" as const },
|
||||
};
|
||||
|
||||
await expect(adapter.getAbstractFileByPath(nfcName)).rejects.toThrow(/ambiguous/i);
|
||||
await expect(adapter.vault.modify(exactFile, "changed")).rejects.toThrow(/ambiguous/i);
|
||||
await expect(adapter.vault.delete(exactFile)).rejects.toThrow(/ambiguous/i);
|
||||
|
||||
await expect(fs.readFile(path.join(directory, nfcName), "utf8")).resolves.toBe("NFC");
|
||||
await expect(fs.readFile(path.join(directory, nfdName), "utf8")).resolves.toBe("NFD");
|
||||
expect((await fs.readdir(directory)).sort()).toEqual([nfcName, nfdName].sort());
|
||||
});
|
||||
|
||||
it("rejects an NFC/NFD directory collision before creating a child", async () => {
|
||||
const directory = await createVault();
|
||||
const nfcFolder = "café";
|
||||
const nfdFolder = nfcFolder.normalize("NFD");
|
||||
await fs.mkdir(path.join(directory, nfcFolder));
|
||||
await fs.mkdir(path.join(directory, nfdFolder));
|
||||
await fs.writeFile(path.join(directory, nfcFolder, "inside.md"), "NFC", "utf8");
|
||||
await fs.writeFile(path.join(directory, nfdFolder, "inside.md"), "NFD", "utf8");
|
||||
const adapter = createNormalisingAdapter(directory);
|
||||
|
||||
await expect(adapter.getAbstractFileByPath(`${nfcFolder}/inside.md`)).rejects.toThrow(/ambiguous/i);
|
||||
await expect(adapter.storage.write(`${nfcFolder}/created.md`, "created")).rejects.toThrow(/ambiguous/i);
|
||||
|
||||
await expect(fs.readdir(path.join(directory, nfcFolder))).resolves.toEqual(["inside.md"]);
|
||||
await expect(fs.readdir(path.join(directory, nfdFolder))).resolves.toEqual(["inside.md"]);
|
||||
await expect(fs.readFile(path.join(directory, nfcFolder, "inside.md"), "utf8")).resolves.toBe("NFC");
|
||||
await expect(fs.readFile(path.join(directory, nfdFolder, "inside.md"), "utf8")).resolves.toBe("NFD");
|
||||
});
|
||||
|
||||
it("rejects enumeration when NFC-equivalent directories contain different files", async () => {
|
||||
const directory = await createVault();
|
||||
const nfcFolder = "café";
|
||||
const nfdFolder = nfcFolder.normalize("NFD");
|
||||
await fs.mkdir(path.join(directory, nfcFolder));
|
||||
await fs.mkdir(path.join(directory, nfdFolder));
|
||||
await fs.writeFile(path.join(directory, nfcFolder, "a.md"), "NFC", "utf8");
|
||||
await fs.writeFile(path.join(directory, nfdFolder, "b.md"), "NFD", "utf8");
|
||||
const adapter = createNormalisingAdapter(directory);
|
||||
|
||||
await expect(adapter.getFiles()).rejects.toThrow(/ambiguous/i);
|
||||
await expect(adapter.scanDirectory()).rejects.toThrow(/ambiguous/i);
|
||||
|
||||
await expect(fs.readFile(path.join(directory, nfcFolder, "a.md"), "utf8")).resolves.toBe("NFC");
|
||||
await expect(fs.readFile(path.join(directory, nfdFolder, "b.md"), "utf8")).resolves.toBe("NFD");
|
||||
});
|
||||
|
||||
it("rechecks the directory after a unique file is cached", async () => {
|
||||
const directory = await createVault();
|
||||
const nfcName = "café.md";
|
||||
const nfdName = nfcName.normalize("NFD");
|
||||
await fs.writeFile(path.join(directory, nfdName), "NFD", "utf8");
|
||||
const adapter = createNormalisingAdapter(directory);
|
||||
|
||||
await expect(adapter.getAbstractFileByPath(nfcName)).resolves.toMatchObject({ path: nfdName });
|
||||
await fs.writeFile(path.join(directory, nfcName), "NFC", "utf8");
|
||||
|
||||
await expect(adapter.getAbstractFileByPath(nfcName)).rejects.toThrow(/ambiguous/i);
|
||||
await expect(fs.readFile(path.join(directory, nfcName), "utf8")).resolves.toBe("NFC");
|
||||
await expect(fs.readFile(path.join(directory, nfdName), "utf8")).resolves.toBe("NFD");
|
||||
});
|
||||
|
||||
it.each([
|
||||
["Foo", "foo"],
|
||||
["Café", "café".normalize("NFD")],
|
||||
])("rejects case-insensitive parent aliases %s and %s containing different files", async (first, second) => {
|
||||
const directory = await createVault();
|
||||
for (const [parent, name] of [
|
||||
[first, "one.md"],
|
||||
[second, "two.md"],
|
||||
]) {
|
||||
await fs.mkdir(path.join(directory, parent));
|
||||
await fs.writeFile(path.join(directory, parent, name), name);
|
||||
}
|
||||
const adapter = createNormalisingAdapter(directory, true);
|
||||
|
||||
await expect(adapter.getFiles()).rejects.toThrow(/ambiguous/i);
|
||||
await expect(adapter.scanDirectory()).rejects.toThrow(/ambiguous/i);
|
||||
await expect(fs.readFile(path.join(directory, first, "one.md"), "utf8")).resolves.toBe("one.md");
|
||||
await expect(fs.readFile(path.join(directory, second, "two.md"), "utf8")).resolves.toBe("two.md");
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ version: 0, caseInsensitive: true },
|
||||
{ version: 1, caseInsensitive: false },
|
||||
])(
|
||||
"compatibility: retains distinct case paths in mode $version with case-insensitive matching $caseInsensitive",
|
||||
async ({ version, caseInsensitive }) => {
|
||||
const directory = await createVault();
|
||||
const names = ["Café/one.md", "café/two.md".normalize("NFD")];
|
||||
for (const name of names) {
|
||||
await fs.mkdir(path.dirname(path.join(directory, name)));
|
||||
await fs.writeFile(path.join(directory, name), name);
|
||||
}
|
||||
const adapter = new NodeFileSystemAdapter(
|
||||
directory,
|
||||
undefined,
|
||||
() => version,
|
||||
() => caseInsensitive
|
||||
);
|
||||
|
||||
expect((await adapter.getFiles()).map((file) => file.path).sort()).toEqual([...names].sort());
|
||||
for (const name of names) await expect(adapter.storage.read(name)).resolves.toBe(name);
|
||||
}
|
||||
);
|
||||
|
||||
it("refuses reconstruction before starting remote activity when parent aliases are ambiguous", async () => {
|
||||
const directory = await createVault();
|
||||
for (const name of ["Café/one.md", "café/two.md".normalize("NFD")]) {
|
||||
await fs.mkdir(path.dirname(path.join(directory, name)));
|
||||
await fs.writeFile(path.join(directory, name), name);
|
||||
}
|
||||
const adapter = createNormalisingAdapter(directory, true);
|
||||
const settings = {
|
||||
...DEFAULT_SETTINGS,
|
||||
pathNormalizationVersion: 1 as const,
|
||||
handleFilenameCaseSensitive: false,
|
||||
};
|
||||
const setting = { currentSettings: () => settings, suspendAllSync: { addHandler: vi.fn() } };
|
||||
const runBoundedRemoteActivity = vi.fn(async () => undefined);
|
||||
const rebuilder = new ServiceRebuilder({
|
||||
setting,
|
||||
storageAccess: { getFileNames: async () => (await adapter.getFiles()).map((file) => file.path) },
|
||||
database: {
|
||||
onDatabaseReset: { addHandler: vi.fn() },
|
||||
isDatabaseReady: () => true,
|
||||
localDatabase: { findAllDocs: async function* () {} },
|
||||
},
|
||||
vault: { isTargetFile: async () => true },
|
||||
path: new PathServiceCompat(new ServiceContext(), { settingService: setting as never }),
|
||||
replicator: { runBoundedRemoteActivity },
|
||||
UI: { showMarkdownDialog: vi.fn(async () => undefined) },
|
||||
} as unknown as ServiceRebuilderDependencies);
|
||||
|
||||
await expect(rebuilder.$rebuildEverything()).rejects.toThrow(/ambiguous/i);
|
||||
expect(runBoundedRemoteActivity).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("uses fresh listings after external additions, renames, deletions, and collisions", async () => {
|
||||
const directory = await createVault();
|
||||
const nfdName = "café.md".normalize("NFD");
|
||||
await fs.writeFile(path.join(directory, nfdName), "original");
|
||||
const adapter = createNormalisingAdapter(directory);
|
||||
expect((await adapter.getFiles()).map((file) => file.path)).toEqual([nfdName]);
|
||||
|
||||
const renamed = "résumé.md".normalize("NFD");
|
||||
await fs.rename(path.join(directory, nfdName), path.join(directory, renamed));
|
||||
await fs.writeFile(path.join(directory, "new.md"), "new");
|
||||
expect((await adapter.getFiles()).map((file) => file.path).sort()).toEqual(["new.md", renamed]);
|
||||
|
||||
await fs.rm(path.join(directory, "new.md"));
|
||||
expect((await adapter.getFiles()).map((file) => file.path)).toEqual([renamed]);
|
||||
await fs.writeFile(path.join(directory, renamed.normalize("NFC")), "collision");
|
||||
await expect(adapter.getFiles()).rejects.toThrow(/ambiguous/i);
|
||||
});
|
||||
|
||||
it("retains the storage boundary for exact metadata reads and directory scans", async () => {
|
||||
const directory = await createVault();
|
||||
const outside = await createVault();
|
||||
await fs.writeFile(path.join(outside, "note.md"), "outside");
|
||||
await fs.symlink(outside, path.join(directory, "linked"), "dir");
|
||||
const adapter = createNormalisingAdapter(directory);
|
||||
|
||||
await expect(adapter.getFiles()).resolves.toEqual([]);
|
||||
await expect(adapter.storage.statExact("linked/note.md")).rejects.toThrow(/symbolic links/i);
|
||||
await expect(adapter.storage.write("linked/note.md", "changed")).rejects.toThrow(/symbolic links/i);
|
||||
await expect(fs.readFile(path.join(outside, "note.md"), "utf8")).resolves.toBe("outside");
|
||||
});
|
||||
|
||||
it("detects NFC and case collisions when lookup is case-insensitive", async () => {
|
||||
const directory = await createVault();
|
||||
const upperName = "Café.md";
|
||||
const lowerNfdName = "café.md".normalize("NFD");
|
||||
await fs.writeFile(path.join(directory, upperName), "upper", "utf8");
|
||||
await fs.writeFile(path.join(directory, lowerNfdName), "lower", "utf8");
|
||||
const adapter = createNormalisingAdapter(directory, true);
|
||||
|
||||
await expect(adapter.getAbstractFileByPathInsensitive("CAFÉ.md")).rejects.toThrow(/ambiguous/i);
|
||||
await expect(adapter.getFiles()).rejects.toThrow(/ambiguous/i);
|
||||
await expect(adapter.scanDirectory()).rejects.toThrow(/ambiguous/i);
|
||||
await expect(fs.readFile(path.join(directory, upperName), "utf8")).resolves.toBe("upper");
|
||||
await expect(fs.readFile(path.join(directory, lowerNfdName), "utf8")).resolves.toBe("lower");
|
||||
});
|
||||
|
||||
it("retains exact-path behaviour when path normalisation is disabled", async () => {
|
||||
const directory = await createVault();
|
||||
const nfcName = "café.md";
|
||||
const nfdName = nfcName.normalize("NFD");
|
||||
await fs.writeFile(path.join(directory, nfdName), "NFD", "utf8");
|
||||
const adapter = new NodeFileSystemAdapter(directory);
|
||||
|
||||
await expect(adapter.getAbstractFileByPath(nfcName)).resolves.toBeNull();
|
||||
await adapter.storage.write(nfcName, "NFC");
|
||||
|
||||
await expect(fs.readFile(path.join(directory, nfcName), "utf8")).resolves.toBe("NFC");
|
||||
await expect(fs.readFile(path.join(directory, nfdName), "utf8")).resolves.toBe("NFD");
|
||||
});
|
||||
});
|
||||
@@ -2,6 +2,7 @@ import type { FilePath, UXDataWriteOptions } from "@vrtmrz/livesync-commonlib/co
|
||||
import type { IVaultAdapter } from "@vrtmrz/livesync-commonlib/compat/serviceModules/adapters";
|
||||
import type { NodeFile, NodeFolder } from "./NodeTypes";
|
||||
import { NodeStorageAdapter } from "@vrtmrz/livesync-commonlib/node";
|
||||
import { UnicodeResolvingNodeStorageAdapter } from "./UnicodeResolvingNodeStorageAdapter";
|
||||
|
||||
/**
|
||||
* Vault adapter implementation for Node.js
|
||||
@@ -73,8 +74,16 @@ export class NodeVaultAdapter implements IVaultAdapter<NodeFile> {
|
||||
}
|
||||
|
||||
private async toNodeFile(path: string): Promise<NodeFile> {
|
||||
const stat = await this.storage.stat(path);
|
||||
const resolvedPath =
|
||||
this.storage instanceof UnicodeResolvingNodeStorageAdapter
|
||||
? await this.storage.resolveLogicalPath(path, undefined, false)
|
||||
: path;
|
||||
if (resolvedPath === null) throw new Error(`Could not read created file metadata: ${path}`);
|
||||
const stat =
|
||||
this.storage instanceof UnicodeResolvingNodeStorageAdapter
|
||||
? await this.storage.statExact(resolvedPath)
|
||||
: await this.storage.stat(resolvedPath);
|
||||
if (stat?.type !== "file") throw new Error(`Could not read created file metadata: ${path}`);
|
||||
return { path: path as FilePath, stat };
|
||||
return { path: resolvedPath as FilePath, stat };
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,205 @@
|
||||
import { NodeStorageAdapter, validateStoragePath } from "@vrtmrz/livesync-commonlib/node";
|
||||
|
||||
export class AmbiguousNormalisedPathError extends Error {
|
||||
constructor(path: string) {
|
||||
super(`Ambiguous path after Unicode normalisation: ${path}`);
|
||||
this.name = "AmbiguousNormalisedPathError";
|
||||
}
|
||||
}
|
||||
|
||||
/** Resolves NFC-equivalent logical paths against current Node storage listings. */
|
||||
export class UnicodeResolvingNodeStorageAdapter extends NodeStorageAdapter {
|
||||
constructor(
|
||||
rootPath: string,
|
||||
private readonly getPathNormalizationVersion: () => number = () => 0,
|
||||
private readonly isCaseInsensitive: () => boolean = () => false
|
||||
) {
|
||||
super(rootPath);
|
||||
}
|
||||
|
||||
get isUnicodePathNormalizationEnabled(): boolean {
|
||||
return this.getPathNormalizationVersion() === 1;
|
||||
}
|
||||
|
||||
/** Read metadata for an entry already resolved or obtained from a storage listing. */
|
||||
async statExact(path: string) {
|
||||
return await super.trystat(path);
|
||||
}
|
||||
|
||||
/** Inspect a physical directory once before descending into its enumerated entries. */
|
||||
async listExact(path: string) {
|
||||
const entries = await super.list(path);
|
||||
if (this.isUnicodePathNormalizationEnabled) {
|
||||
const caseInsensitive = this.isCaseInsensitive();
|
||||
const names = new Map<string, string>();
|
||||
for (const entry of [...entries.files, ...entries.folders]) {
|
||||
const key = this.comparisonKey(entry, caseInsensitive);
|
||||
if (names.has(key)) throw new AmbiguousNormalisedPathError(entry);
|
||||
names.set(key, entry);
|
||||
}
|
||||
}
|
||||
return entries;
|
||||
}
|
||||
|
||||
async resolveLogicalPath(
|
||||
path: string,
|
||||
caseInsensitive = this.isCaseInsensitive(),
|
||||
allowMissing = true
|
||||
): Promise<string | null> {
|
||||
if (!this.isUnicodePathNormalizationEnabled) return path;
|
||||
|
||||
validateStoragePath(path, true);
|
||||
if (path === "") return "";
|
||||
|
||||
const segments = path.split("/");
|
||||
let resolvedParent = "";
|
||||
|
||||
for (let index = 0; index < segments.length; index++) {
|
||||
const segment = segments[index];
|
||||
const entries = await super.list(resolvedParent);
|
||||
const candidates = [...entries.files, ...entries.folders];
|
||||
const requestedKey = this.comparisonKey(segment, caseInsensitive);
|
||||
const matches = candidates.filter((entryPath) => {
|
||||
const entryName = entryPath.slice(resolvedParent === "" ? 0 : resolvedParent.length + 1);
|
||||
return this.comparisonKey(entryName, caseInsensitive) === requestedKey;
|
||||
});
|
||||
|
||||
if (matches.length > 1) throw new AmbiguousNormalisedPathError(path);
|
||||
if (matches.length === 1) {
|
||||
resolvedParent = matches[0];
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!allowMissing) return null;
|
||||
const remainingPath = segments.slice(index).join("/");
|
||||
return resolvedParent === "" ? remainingPath : `${resolvedParent}/${remainingPath}`;
|
||||
}
|
||||
|
||||
return resolvedParent;
|
||||
}
|
||||
|
||||
override async exists(path: string, caseInsensitive = this.isCaseInsensitive()): Promise<boolean> {
|
||||
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, false);
|
||||
if (resolvedPath === null) return false;
|
||||
return await super.exists(resolvedPath);
|
||||
}
|
||||
|
||||
override async trystat(path: string, caseInsensitive = this.isCaseInsensitive()) {
|
||||
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, false);
|
||||
if (resolvedPath === null) return null;
|
||||
return await super.trystat(resolvedPath);
|
||||
}
|
||||
|
||||
override async stat(path: string, caseInsensitive = this.isCaseInsensitive()) {
|
||||
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, false);
|
||||
if (resolvedPath === null) return null;
|
||||
return await super.trystat(resolvedPath);
|
||||
}
|
||||
|
||||
override async mkdir(path: string, caseInsensitive = this.isCaseInsensitive()): Promise<void> {
|
||||
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, true);
|
||||
if (resolvedPath === null) throw new Error(`Could not resolve path: ${path}`);
|
||||
await super.mkdir(resolvedPath);
|
||||
}
|
||||
|
||||
override async remove(path: string, caseInsensitive = this.isCaseInsensitive()): Promise<void> {
|
||||
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, false);
|
||||
await super.remove(resolvedPath ?? path);
|
||||
}
|
||||
|
||||
override async read(path: string, caseInsensitive = this.isCaseInsensitive()): Promise<string> {
|
||||
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, false);
|
||||
return await super.read(resolvedPath ?? path);
|
||||
}
|
||||
|
||||
override async readBinary(path: string, caseInsensitive = this.isCaseInsensitive()): Promise<ArrayBuffer> {
|
||||
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, false);
|
||||
return await super.readBinary(resolvedPath ?? path);
|
||||
}
|
||||
|
||||
override async write(
|
||||
path: string,
|
||||
data: string,
|
||||
options?: Parameters<NodeStorageAdapter["write"]>[2],
|
||||
caseInsensitive = this.isCaseInsensitive()
|
||||
): Promise<void> {
|
||||
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, true);
|
||||
if (resolvedPath === null) throw new Error(`Could not resolve path: ${path}`);
|
||||
await super.write(resolvedPath, data, options);
|
||||
}
|
||||
|
||||
override async writeBinary(
|
||||
path: string,
|
||||
data: ArrayBuffer,
|
||||
options?: Parameters<NodeStorageAdapter["writeBinary"]>[2],
|
||||
caseInsensitive = this.isCaseInsensitive()
|
||||
): Promise<void> {
|
||||
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, true);
|
||||
if (resolvedPath === null) throw new Error(`Could not resolve path: ${path}`);
|
||||
await super.writeBinary(resolvedPath, data, options);
|
||||
}
|
||||
|
||||
override async append(
|
||||
path: string,
|
||||
data: string,
|
||||
options?: Parameters<NodeStorageAdapter["append"]>[2],
|
||||
caseInsensitive = this.isCaseInsensitive()
|
||||
): Promise<void> {
|
||||
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, true);
|
||||
if (resolvedPath === null) throw new Error(`Could not resolve path: ${path}`);
|
||||
await super.append(resolvedPath, data, options);
|
||||
}
|
||||
|
||||
override async rename(
|
||||
sourcePath: string,
|
||||
targetPath: string,
|
||||
caseInsensitive = this.isCaseInsensitive()
|
||||
): Promise<void> {
|
||||
const resolvedSource = await this.resolveLogicalPath(sourcePath, caseInsensitive, false);
|
||||
if (resolvedSource === null) return await super.rename(sourcePath, targetPath);
|
||||
|
||||
const resolvedTarget = await this.resolveLogicalPath(targetPath, caseInsensitive, true);
|
||||
if (resolvedTarget === null) throw new Error(`Could not resolve path: ${targetPath}`);
|
||||
|
||||
// Preserve explicit case-only or Unicode-spelling renames when the target resolves to the source.
|
||||
const sourceKey = this.comparisonKey(resolvedSource, caseInsensitive);
|
||||
const targetKey = this.comparisonKey(resolvedTarget, caseInsensitive);
|
||||
const target =
|
||||
sourceKey === targetKey && resolvedTarget === resolvedSource && targetPath !== resolvedSource
|
||||
? await this.resolveRenameTargetSpelling(targetPath, caseInsensitive)
|
||||
: resolvedTarget;
|
||||
|
||||
await super.rename(resolvedSource, target);
|
||||
}
|
||||
|
||||
override async list(basePath: string, caseInsensitive = this.isCaseInsensitive()) {
|
||||
const resolvedPath = await this.resolveLogicalPath(basePath, caseInsensitive, false);
|
||||
if (resolvedPath === null) return { files: [], folders: [] };
|
||||
return await super.list(resolvedPath);
|
||||
}
|
||||
|
||||
private async resolveRenameTargetSpelling(path: string, caseInsensitive: boolean): Promise<string> {
|
||||
validateStoragePath(path, false);
|
||||
const segments = path.split("/");
|
||||
const leaf = segments.pop();
|
||||
if (leaf === undefined) throw new Error(`Could not resolve path: ${path}`);
|
||||
const parentPath = segments.join("/");
|
||||
const resolvedParent = await this.resolveLogicalPath(parentPath, caseInsensitive, true);
|
||||
if (resolvedParent === null) throw new Error(`Could not resolve path: ${path}`);
|
||||
|
||||
const entries = await super.list(resolvedParent);
|
||||
const requestedKey = this.comparisonKey(leaf, caseInsensitive);
|
||||
const matches = [...entries.files, ...entries.folders].filter((entryPath) => {
|
||||
const entryName = entryPath.slice(resolvedParent === "" ? 0 : resolvedParent.length + 1);
|
||||
return this.comparisonKey(entryName, caseInsensitive) === requestedKey;
|
||||
});
|
||||
if (matches.length > 1) throw new AmbiguousNormalisedPathError(path);
|
||||
|
||||
return resolvedParent === "" ? leaf : `${resolvedParent}/${leaf}`;
|
||||
}
|
||||
|
||||
private comparisonKey(segment: string, caseInsensitive: boolean): string {
|
||||
const normalisedSegment = segment.normalize("NFC");
|
||||
return caseInsensitive ? normalisedSegment.toLowerCase() : normalisedSegment;
|
||||
}
|
||||
}
|
||||
@@ -6,6 +6,7 @@ import {
|
||||
type ObsidianLiveSyncSettings,
|
||||
type EntryDoc,
|
||||
} from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { isRemediationModeActive } from "@vrtmrz/livesync-commonlib/compat/common/utils";
|
||||
import { ConnectionStringParser } from "@vrtmrz/livesync-commonlib/compat/common/ConnectionString";
|
||||
import {
|
||||
activateRemoteConfiguration,
|
||||
@@ -13,7 +14,7 @@ import {
|
||||
} from "@vrtmrz/livesync-commonlib/remote-configurations";
|
||||
import { stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
|
||||
import type { CLICommandContext, CLIOptions } from "./types";
|
||||
import { toArrayBuffer, toDatabaseRelativePath } from "./utils";
|
||||
import { getForceRequirementError, toArrayBuffer, toDatabaseRelativePath } from "./utils";
|
||||
import { collectPeers, openP2PHost, parseTimeoutSeconds, syncWithPeer } from "./p2p";
|
||||
import { compatGlobal } from "@vrtmrz/livesync-commonlib/compat/common/coreEnvFunctions";
|
||||
import { fsPromises as fs, getNodeFileCreationTime, path } from "@vrtmrz/livesync-commonlib/node";
|
||||
@@ -37,8 +38,25 @@ function redactConnectionString(uri: string): string {
|
||||
}
|
||||
|
||||
export async function runCommand(options: CLIOptions, context: CLICommandContext): Promise<boolean> {
|
||||
const { databasePath, core, replicationScheduling, settingsPath } = context;
|
||||
const { core } = context;
|
||||
const { standardIo } = core.services.context;
|
||||
const forceRequirementError = getForceRequirementError(options.command, options.force);
|
||||
if (forceRequirementError) {
|
||||
writeStderrLine(standardIo, `[Error] ${forceRequirementError}`);
|
||||
return false;
|
||||
}
|
||||
if (
|
||||
options.command === "fetch" &&
|
||||
isRemediationModeActive(core.services.setting.currentSettings())
|
||||
) {
|
||||
writeStderrLine(
|
||||
standardIo,
|
||||
`[Error] The fetch command cannot run while remediation mode is active. Clear the modification-time limit before fetching.`
|
||||
);
|
||||
return false;
|
||||
}
|
||||
|
||||
const { databasePath, replicationScheduling, settingsPath } = context;
|
||||
const vaultPath = context.vaultPath || databasePath;
|
||||
|
||||
await core.services.control.activated;
|
||||
@@ -519,6 +537,29 @@ export async function runCommand(options: CLIOptions, context: CLICommandContext
|
||||
return true;
|
||||
}
|
||||
|
||||
if (options.command === "rebuild") {
|
||||
writeStderrLine(standardIo, "[Command] rebuild");
|
||||
try {
|
||||
await core.serviceModules.rebuilder.$rebuildEverything();
|
||||
return true;
|
||||
} catch (error) {
|
||||
writeStderrLine(standardIo, "[Error] Rebuild failed:", error);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (options.command === "fetch") {
|
||||
writeStderrLine(standardIo, "[Command] fetch");
|
||||
try {
|
||||
// Fetch replaces the local database from remote; do not persist legacy local files first.
|
||||
await core.serviceModules.rebuilder.$fetchLocal(false, true);
|
||||
return true;
|
||||
} catch (error) {
|
||||
writeStderrLine(standardIo, "[Error] Fetch failed:", error);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (options.command === "remote-add") {
|
||||
if (options.commandArgs.length < 2) {
|
||||
throw new Error("remote-add requires two arguments: <name> <connstr>");
|
||||
|
||||
@@ -118,6 +118,10 @@ function createCoreMock() {
|
||||
fetch: vi.fn(async () => undefined),
|
||||
storeContent: vi.fn(async () => true),
|
||||
},
|
||||
rebuilder: {
|
||||
$rebuildEverything: vi.fn(async () => undefined),
|
||||
$fetchLocal: vi.fn(async () => undefined),
|
||||
},
|
||||
},
|
||||
} as any;
|
||||
}
|
||||
@@ -268,6 +272,73 @@ describe("runCommand abnormal cases", () => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
it.each(["rebuild", "fetch"] as const)("requires --force before activating for %s", async (command) => {
|
||||
const core = createCoreMock();
|
||||
const activated = vi.fn(() => Promise.resolve());
|
||||
Object.defineProperty(core.services.control, "activated", { get: activated });
|
||||
|
||||
await expect(runCommand(makeOptions(command, []), { ...context, core })).resolves.toBe(false);
|
||||
|
||||
expect(activated).not.toHaveBeenCalled();
|
||||
expect(core.serviceModules.rebuilder.$rebuildEverything).not.toHaveBeenCalled();
|
||||
expect(core.serviceModules.rebuilder.$fetchLocal).not.toHaveBeenCalled();
|
||||
expect(core.services.context.standardIo.writeStderr).toHaveBeenCalledWith(
|
||||
expect.stringContaining("requires --force")
|
||||
);
|
||||
});
|
||||
|
||||
it("dispatches a forced rebuild to the existing Rebuilder service", async () => {
|
||||
const core = createCoreMock();
|
||||
const options = makeOptions("rebuild", []);
|
||||
options.force = true;
|
||||
|
||||
await expect(runCommand(options, { ...context, core })).resolves.toBe(true);
|
||||
|
||||
expect(core.serviceModules.rebuilder.$rebuildEverything).toHaveBeenCalledOnce();
|
||||
expect(core.serviceModules.rebuilder.$fetchLocal).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("dispatches a forced fetch without saving legacy Vault files first", async () => {
|
||||
const core = createCoreMock();
|
||||
const options = makeOptions("fetch", []);
|
||||
options.force = true;
|
||||
|
||||
await expect(runCommand(options, { ...context, core })).resolves.toBe(true);
|
||||
|
||||
expect(core.serviceModules.rebuilder.$fetchLocal).toHaveBeenCalledOnce();
|
||||
expect(core.serviceModules.rebuilder.$fetchLocal).toHaveBeenCalledWith(false, true);
|
||||
expect(core.serviceModules.rebuilder.$rebuildEverything).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("rejects fetch in remediation mode even when forced", async () => {
|
||||
const core = createCoreMock();
|
||||
core.services.setting.currentSettings().maxMTimeForReflectEvents = 1;
|
||||
const options = makeOptions("fetch", []);
|
||||
options.force = true;
|
||||
|
||||
await expect(runCommand(options, { ...context, core })).resolves.toBe(false);
|
||||
|
||||
expect(core.serviceModules.rebuilder.$fetchLocal).not.toHaveBeenCalled();
|
||||
expect(core.services.context.standardIo.writeStderr).toHaveBeenCalledWith(
|
||||
expect.stringContaining("remediation mode")
|
||||
);
|
||||
});
|
||||
|
||||
it("surfaces an internal rebuild refusal as a failed command", async () => {
|
||||
const core = createCoreMock();
|
||||
core.serviceModules.rebuilder.$rebuildEverything.mockRejectedValue(
|
||||
new Error("Rebuilding is not available while remediation mode is active.")
|
||||
);
|
||||
const options = makeOptions("rebuild", []);
|
||||
options.force = true;
|
||||
|
||||
await expect(runCommand(options, { ...context, core })).resolves.toBe(false);
|
||||
|
||||
expect(core.services.context.standardIo.writeStderr).toHaveBeenCalledWith(
|
||||
expect.stringContaining("Rebuilding is not available while remediation mode is active")
|
||||
);
|
||||
});
|
||||
|
||||
it("retains visible progress for the interactive sync command", async () => {
|
||||
const core = createCoreMock();
|
||||
|
||||
|
||||
@@ -22,6 +22,8 @@ export type CLICommand =
|
||||
| "rm"
|
||||
| "resolve"
|
||||
| "mirror"
|
||||
| "rebuild"
|
||||
| "fetch"
|
||||
| "remote-add"
|
||||
| "remote-rm"
|
||||
| "remote-ls"
|
||||
@@ -100,6 +102,8 @@ export const VALID_COMMANDS = new Set([
|
||||
"rm",
|
||||
"resolve",
|
||||
"mirror",
|
||||
"rebuild",
|
||||
"fetch",
|
||||
"remote-add",
|
||||
"remote-rm",
|
||||
"remote-ls",
|
||||
|
||||
@@ -1,4 +1,16 @@
|
||||
import { path } from "@vrtmrz/livesync-commonlib/node";
|
||||
import type { CLICommand } from "./types";
|
||||
|
||||
export function getForceRequirementError(command: CLICommand, force = false): string | undefined {
|
||||
if (force) return undefined;
|
||||
if (command === "rebuild") {
|
||||
return "Command 'rebuild' requires --force because it replaces the local and remote databases.";
|
||||
}
|
||||
if (command === "fetch") {
|
||||
return "Command 'fetch' requires --force because it replaces the local database with remote state.";
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
export function toArrayBuffer(data: Buffer): ArrayBuffer {
|
||||
return data.buffer.slice(data.byteOffset, data.byteOffset + data.byteLength) as ArrayBuffer;
|
||||
|
||||
@@ -1,6 +1,20 @@
|
||||
import { path } from "@vrtmrz/livesync-commonlib/node";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { toDatabaseRelativePath } from "./utils";
|
||||
import { getForceRequirementError, toDatabaseRelativePath } from "./utils";
|
||||
|
||||
describe("getForceRequirementError", () => {
|
||||
it.each([
|
||||
["rebuild", "Command 'rebuild' requires --force because it replaces the local and remote databases."],
|
||||
["fetch", "Command 'fetch' requires --force because it replaces the local database with remote state."],
|
||||
] as const)("requires force for %s", (command, expected) => {
|
||||
expect(getForceRequirementError(command, false)).toBe(expected);
|
||||
expect(getForceRequirementError(command, true)).toBeUndefined();
|
||||
});
|
||||
|
||||
it("does not require force for ordinary commands", () => {
|
||||
expect(getForceRequirementError("sync", false)).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe("toDatabaseRelativePath", () => {
|
||||
const databasePath = path.resolve("/tmp/livesync-vault");
|
||||
|
||||
@@ -3,6 +3,7 @@ import { appendFile, mkdtemp, mkdir, readFile, rm, stat, writeFile } from "node:
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import * as chokidar from "chokidar";
|
||||
import * as offlineScanner from "@vrtmrz/livesync-commonlib/compat/serviceFeatures/offlineScanner";
|
||||
import { ControlService } from "@vrtmrz/livesync-commonlib/compat/services/base/ControlService";
|
||||
import {
|
||||
AutoAccepting,
|
||||
@@ -20,6 +21,7 @@ import { cloneSettings } from "./settingsPersistence";
|
||||
import type { CLICommand } from "./commands/types";
|
||||
|
||||
vi.mock("chokidar", { spy: true });
|
||||
vi.mock("@vrtmrz/livesync-commonlib/compat/serviceFeatures/offlineScanner", { spy: true });
|
||||
|
||||
function createStandardIoMock() {
|
||||
return {
|
||||
@@ -41,6 +43,7 @@ describe("CLI database preparation", () => {
|
||||
|
||||
beforeEach(async () => {
|
||||
vi.mocked(chokidar.watch).mockClear();
|
||||
vi.mocked(offlineScanner.useOfflineScanner).mockClear();
|
||||
directory = await mkdtemp(join(tmpdir(), "livesync-cli-bootstrap-"));
|
||||
vaultPath = join(directory, "vault");
|
||||
settingsPath = join(directory, "settings.json");
|
||||
@@ -69,8 +72,18 @@ describe("CLI database preparation", () => {
|
||||
await rm(directory, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
async function start(command: CLICommand, runner: CliCommandRunner, exitCode = 1) {
|
||||
process.argv = ["node", "livesync-cli", directory, "--vault", vaultPath, "--settings", settingsPath, command];
|
||||
async function start(command: CLICommand, runner: CliCommandRunner, exitCode = 1, force = false) {
|
||||
process.argv = [
|
||||
"node",
|
||||
"livesync-cli",
|
||||
directory,
|
||||
...(force ? ["--force"] : []),
|
||||
"--vault",
|
||||
vaultPath,
|
||||
"--settings",
|
||||
settingsPath,
|
||||
command,
|
||||
];
|
||||
// Daemon probes return false so the real core unloads without keeping a daemon alive.
|
||||
await expect(main(standardIo, runner)).rejects.toThrow(`__EXIT__:${exitCode}`);
|
||||
}
|
||||
@@ -255,6 +268,49 @@ describe("CLI database preparation", () => {
|
||||
expect(watch).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it.each(["rebuild", "fetch"] as const)("does not scan or mirror legacy Vault files before %s", async (command) => {
|
||||
const storedPaths: string[] = [];
|
||||
const runner = vi.fn<CliCommandRunner>(async (_options, { core }) => {
|
||||
for await (const doc of core.services.database.localDatabase.findAllNormalDocs()) {
|
||||
storedPaths.push(doc.path);
|
||||
}
|
||||
return false;
|
||||
});
|
||||
|
||||
await start(command, runner, 1, true);
|
||||
|
||||
expect(runner).toHaveBeenCalledOnce();
|
||||
expect(storedPaths).toEqual([]);
|
||||
expect(await readFile(join(vaultPath, "notes/local.md"), "utf-8")).toBe("local content");
|
||||
expect(vi.mocked(chokidar.watch)).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("registers the Rebuild scanner after readiness and makes its Vault scan available", async () => {
|
||||
const enumerate = vi.spyOn(ServiceFileAccessCLI.prototype, "getFiles");
|
||||
const useScanner = vi.mocked(offlineScanner.useOfflineScanner);
|
||||
|
||||
let scansBeforeCommand = -1;
|
||||
let scannerRegistrationsBeforeCommand = -1;
|
||||
let storedPaths: string[] = [];
|
||||
const runner = vi.fn<CliCommandRunner>(async (_options, { core }) => {
|
||||
scansBeforeCommand = enumerate.mock.calls.length;
|
||||
scannerRegistrationsBeforeCommand = useScanner.mock.calls.length;
|
||||
await core.services.vault.scanVault(true, true);
|
||||
for await (const doc of core.services.database.localDatabase.findAllNormalDocs()) {
|
||||
storedPaths.push(doc.path);
|
||||
}
|
||||
return false;
|
||||
});
|
||||
|
||||
await start("rebuild", runner, 1, true);
|
||||
|
||||
expect(useScanner).toHaveBeenCalledOnce();
|
||||
expect(scannerRegistrationsBeforeCommand).toBe(1);
|
||||
expect(scansBeforeCommand).toBe(0);
|
||||
expect(enumerate).toHaveBeenCalledOnce();
|
||||
expect(storedPaths).toEqual(["notes/local.md"]);
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ command: "daemon" as const, commandRuns: true },
|
||||
{ command: "mirror" as const, commandRuns: false },
|
||||
|
||||
+29
-2
@@ -21,6 +21,7 @@ import {
|
||||
import { runCommand } from "./commands/runCommand";
|
||||
import { isCLICommand } from "./commands/types";
|
||||
import type { CLICommand, CLICommandContext, CLIOptions } from "./commands/types";
|
||||
import { getForceRequirementError } from "./commands/utils";
|
||||
import { getPathFromUXFileInfo } from "@vrtmrz/livesync-commonlib/compat/common/typeUtils";
|
||||
import { stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
|
||||
import { IgnoreRules } from "./serviceModules/IgnoreRules";
|
||||
@@ -47,6 +48,7 @@ const SETTINGS_FILE = ".livesync/settings.json";
|
||||
interface CLIVaultSyncMode {
|
||||
readonly watchFiles: boolean;
|
||||
readonly reflectReplicationResults: boolean;
|
||||
readonly scanOnStartup?: boolean;
|
||||
readonly startupDatabaseOptions: StartupDatabaseOptions;
|
||||
}
|
||||
|
||||
@@ -62,6 +64,18 @@ const VAULT_SYNC_MODES: Readonly<Partial<Record<CLICommand, CLIVaultSyncMode>>>
|
||||
reflectReplicationResults: false,
|
||||
startupDatabaseOptions: { ignoreSuspending: true, continueOnFileFailure: false },
|
||||
},
|
||||
rebuild: {
|
||||
watchFiles: false,
|
||||
reflectReplicationResults: false,
|
||||
scanOnStartup: false,
|
||||
startupDatabaseOptions: { ignoreSuspending: false, continueOnFileFailure: false },
|
||||
},
|
||||
fetch: {
|
||||
watchFiles: false,
|
||||
reflectReplicationResults: true,
|
||||
scanOnStartup: false,
|
||||
startupDatabaseOptions: { ignoreSuspending: false, continueOnFileFailure: false },
|
||||
},
|
||||
};
|
||||
|
||||
ensureGlobalNodeLocalStorage();
|
||||
@@ -90,6 +104,8 @@ Commands:
|
||||
p2p-sync <peer> <timeout>
|
||||
Sync with the specified peer-id or peer-name
|
||||
p2p-host Start P2P host mode and wait until interrupted
|
||||
rebuild Rebuild local and remote databases from the selected Vault (requires --force)
|
||||
fetch Replace the local database with remote state and reflect it to the Vault (requires --force)
|
||||
push <src> <dst> Push local file <src> into local database path <dst>
|
||||
pull <src> <dst> Pull file <src> from local database into local file <dst>
|
||||
pull-rev <src> <dst> <rev> Pull file <src> at specific revision <rev> into local file <dst>
|
||||
@@ -122,8 +138,9 @@ Commands:
|
||||
Show remote database status
|
||||
|
||||
Options:
|
||||
--vault <path>, -V <path> (daemon/mirror) Path to the vault directory containing .md files
|
||||
--vault <path>, -V <path> (daemon/mirror/rebuild/fetch) Path to the vault directory containing .md files
|
||||
(defaults to database-path; allows separate PouchDB and vault dirs)
|
||||
--force, -f Required for rebuild and fetch because they replace databases
|
||||
--interval <N>, -i <N> (daemon only) Poll CouchDB every N seconds instead of using the _changes feed
|
||||
--write-settings Write setting changes after a successful command
|
||||
--compat-remote-admin-exit-zero
|
||||
@@ -319,6 +336,12 @@ export async function main(
|
||||
commandRunner: CliCommandRunner = runCommand
|
||||
) {
|
||||
const options = parseArgs(standardIo);
|
||||
const forceRequirementError = getForceRequirementError(options.command, options.force);
|
||||
if (forceRequirementError) {
|
||||
writeStderrLine(standardIo, `[Error] ${forceRequirementError}`);
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
const vaultSyncMode = VAULT_SYNC_MODES[options.command];
|
||||
if (options.interval && options.command !== "daemon") {
|
||||
writeStderrLine(
|
||||
@@ -521,7 +544,7 @@ export async function main(
|
||||
() => [], // No add-ons
|
||||
(core, coreFeatureViews) => {
|
||||
replicationScheduling = coreFeatureViews.replicationScheduling;
|
||||
if (vaultSyncMode) {
|
||||
if (vaultSyncMode && vaultSyncMode.scanOnStartup !== false) {
|
||||
useOfflineScanner(core);
|
||||
}
|
||||
// Register P2P replicator feature.
|
||||
@@ -641,6 +664,10 @@ export async function main(
|
||||
if (!readyResult) {
|
||||
throw new Error("Failed to initialise LiveSync.");
|
||||
}
|
||||
if (vaultSyncMode?.scanOnStartup === false) {
|
||||
// Keep the startup reset clean, then enable scans for the explicit Rebuild or Fetch operation.
|
||||
useOfflineScanner(core);
|
||||
}
|
||||
const settingsBeforeCommand = cloneSettings(core.services.setting.currentSettings());
|
||||
const transientSettingKeys = changedSettingKeys(settingsBeforeSuspend, settingsAfterSuspend);
|
||||
for (const key of CLI_RUNTIME_ONLY_SETTING_KEYS) {
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { parseArgs as parseCliArgs } from "./main";
|
||||
import { main as runCliMain, parseArgs as parseCliArgs } from "./main";
|
||||
|
||||
function createStandardIoMock() {
|
||||
return {
|
||||
@@ -19,6 +19,7 @@ function mockProcessExit() {
|
||||
|
||||
describe("CLI parseArgs", () => {
|
||||
const originalArgv = process.argv.slice();
|
||||
const originalExitCode = process.exitCode;
|
||||
let standardIo: ReturnType<typeof createStandardIoMock>;
|
||||
|
||||
beforeEach(() => {
|
||||
@@ -29,6 +30,7 @@ describe("CLI parseArgs", () => {
|
||||
|
||||
afterEach(() => {
|
||||
process.argv = originalArgv.slice();
|
||||
process.exitCode = originalExitCode;
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
@@ -99,6 +101,28 @@ describe("CLI parseArgs", () => {
|
||||
expect(parsed.commandArgs).toEqual([]);
|
||||
});
|
||||
|
||||
it.each(["rebuild", "fetch"] as const)("parses %s and its force flag", (command) => {
|
||||
process.argv = ["node", "livesync-cli", "./databasePath", "--force", command];
|
||||
const parsed = parseArgs();
|
||||
|
||||
expect(parsed.command).toBe(command);
|
||||
expect(parsed.force).toBe(true);
|
||||
expect(parsed.commandArgs).toEqual([]);
|
||||
});
|
||||
|
||||
it.each(["rebuild", "fetch"] as const)("rejects %s without force before database access", async (command) => {
|
||||
process.argv = ["node", "livesync-cli", "./missing-database", command];
|
||||
process.exitCode = undefined;
|
||||
const runner = vi.fn(async () => true);
|
||||
|
||||
await expect(runCliMain(standardIo, runner as any)).resolves.toBeUndefined();
|
||||
|
||||
expect(runner).not.toHaveBeenCalled();
|
||||
expect(process.exitCode).toBe(1);
|
||||
expect(standardIo.writeStderr).toHaveBeenCalledWith(expect.stringContaining("requires --force"));
|
||||
expect(standardIo.writeStderr).not.toHaveBeenCalledWith(expect.stringContaining("Database directory"));
|
||||
});
|
||||
|
||||
it("parses remote-add command", () => {
|
||||
process.argv = [
|
||||
"node",
|
||||
|
||||
@@ -99,10 +99,10 @@ export function initialiseServiceModulesCLI(
|
||||
|
||||
// Rebuilder (platform-independent)
|
||||
const rebuilder = new ServiceRebuilder({
|
||||
path: services.path,
|
||||
events: services.context.events,
|
||||
API: services.API,
|
||||
database: services.database,
|
||||
path: services.path,
|
||||
appLifecycle: services.appLifecycle,
|
||||
setting: services.setting,
|
||||
remote: services.remote,
|
||||
|
||||
@@ -1,6 +1,10 @@
|
||||
import { FileAccessBase, type FileAccessBaseDependencies } from "@vrtmrz/livesync-commonlib/compat/serviceModules/FileAccessBase";
|
||||
import {
|
||||
FileAccessBase,
|
||||
type FileAccessBaseDependencies,
|
||||
} from "@vrtmrz/livesync-commonlib/compat/serviceModules/FileAccessBase";
|
||||
import { NodeFileSystemAdapter } from "@/apps/cli/adapters/NodeFileSystemAdapter";
|
||||
import { LOG_LEVEL_NOTICE } from "octagonal-wheels/common/logger";
|
||||
import { getPathNormalizationVersion } from "@vrtmrz/livesync-commonlib/settings";
|
||||
|
||||
/**
|
||||
* CLI-specific implementation of FileAccessBase
|
||||
@@ -8,12 +12,23 @@ import { LOG_LEVEL_NOTICE } from "octagonal-wheels/common/logger";
|
||||
*/
|
||||
export class FileAccessCLI extends FileAccessBase<NodeFileSystemAdapter> {
|
||||
constructor(basePath: string, dependencies: FileAccessBaseDependencies) {
|
||||
const adapter = new NodeFileSystemAdapter(basePath, (message, detail) => {
|
||||
dependencies.APIService.addLog(message, LOG_LEVEL_NOTICE);
|
||||
if (detail !== undefined) {
|
||||
dependencies.APIService.addLog(detail, LOG_LEVEL_NOTICE);
|
||||
}
|
||||
});
|
||||
const pathNormalizationVersion = () =>
|
||||
getPathNormalizationVersion(dependencies.settingService.currentSettings());
|
||||
const isCaseInsensitive = () => {
|
||||
const settings = dependencies.settingService.currentSettings();
|
||||
return !settings.handleFilenameCaseSensitive || dependencies.vaultService.isStorageInsensitive();
|
||||
};
|
||||
const adapter = new NodeFileSystemAdapter(
|
||||
basePath,
|
||||
(message, detail) => {
|
||||
dependencies.APIService.addLog(message, LOG_LEVEL_NOTICE);
|
||||
if (detail !== undefined) {
|
||||
dependencies.APIService.addLog(detail, LOG_LEVEL_NOTICE);
|
||||
}
|
||||
},
|
||||
pathNormalizationVersion,
|
||||
isCaseInsensitive
|
||||
);
|
||||
super(adapter, dependencies);
|
||||
}
|
||||
|
||||
|
||||
@@ -90,10 +90,10 @@ export function initialiseServiceModulesFSAPI(
|
||||
|
||||
// Rebuilder (platform-independent)
|
||||
const rebuilder = new ServiceRebuilder({
|
||||
path: services.path,
|
||||
events: services.context.events,
|
||||
API: services.API,
|
||||
database: services.database,
|
||||
path: services.path,
|
||||
appLifecycle: services.appLifecycle,
|
||||
setting: services.setting,
|
||||
remote: services.remote,
|
||||
|
||||
@@ -139,6 +139,7 @@
|
||||
"Copy Report to clipboard": "Copy Report to clipboard",
|
||||
"CouchDB Configuration": "CouchDB Configuration",
|
||||
"CouchDB Connection Tweak": "CouchDB Connection Tweak",
|
||||
"CouchDB only. Version 0 preserves legacy path identity; version 1 uses Unicode NFC. Physical file names stay unchanged. Update every device and stop synchronisation, then Rebuild both databases from a complete authoritative Vault and Fetch on the other devices. Resolve colliding file names first. P2P and Object Storage do not support version 1.": "CouchDB only. Version 0 preserves legacy path identity; version 1 uses Unicode NFC. Physical file names stay unchanged. Update every device and stop synchronisation, then Rebuild both databases from a complete authoritative Vault and Fetch on the other devices. Resolve colliding file names first. P2P and Object Storage do not support version 1.",
|
||||
"CouchDB validates the database name when you connect. The name must not be empty.": "CouchDB validates the database name when you connect. The name must not be empty.",
|
||||
"Create or connect to database and continue": "Create or connect to database and continue",
|
||||
"Create P2P remote": "Create P2P remote",
|
||||
@@ -404,6 +405,7 @@
|
||||
"lang-zh": "简体中文",
|
||||
"lang-zh-tw": "繁體中文",
|
||||
"Later": "Later",
|
||||
"Legacy path identity": "Legacy path identity",
|
||||
"Limit: {datetime} ({timestamp})": "Limit: {datetime} ({timestamp})",
|
||||
"live": "live",
|
||||
"LiveSync could not handle multiple vaults which have same name without different prefix, This should be automatically configured.": "LiveSync could not handle multiple vaults which have same name without different prefix, This should be automatically configured.",
|
||||
@@ -792,6 +794,7 @@
|
||||
"Paste a connection string": "Paste a connection string",
|
||||
"Paste the Setup URI generated from one of your active devices.": "Paste the Setup URI generated from one of your active devices.",
|
||||
"Path": "Path",
|
||||
"Path identity mode": "Path identity mode",
|
||||
"Path Obfuscation": "Path Obfuscation",
|
||||
"Patterns to match files for overwriting instead of merging": "Patterns to match files for overwriting instead of merging",
|
||||
"Patterns to match files for syncing": "Patterns to match files for syncing",
|
||||
@@ -1449,6 +1452,7 @@
|
||||
"Ui.SetupWizard.SetupRemote.ProceedCouchDb": "Continue to CouchDB setup",
|
||||
"Ui.SetupWizard.SetupRemote.ProceedP2P": "Continue to P2P setup",
|
||||
"Ui.SetupWizard.SetupRemote.Title": "Choose a synchronisation remote",
|
||||
"Unicode NFC path identity": "Unicode NFC path identity",
|
||||
"Unique name between all synchronized devices. To edit this setting, please disable customization sync once.": "Unique name between all synchronized devices. To edit this setting, please disable customization sync once.",
|
||||
"Unless you are certain, selecting this options is bit dangerous. It assumes that the server configuration is compatible with this device. If this is not the case, data loss may occur. Please ensure you know what you are doing.": "Unless you are certain, selecting this options is bit dangerous. It assumes that the server configuration is compatible with this device. If this is not the case, data loss may occur. Please ensure you know what you are doing.",
|
||||
"Updating list...": "Updating list...",
|
||||
|
||||
@@ -169,6 +169,8 @@ Copy: Copy
|
||||
Copy Report to clipboard: Copy Report to clipboard
|
||||
CouchDB Configuration: CouchDB Configuration
|
||||
CouchDB Connection Tweak: CouchDB Connection Tweak
|
||||
"CouchDB only. Version 0 preserves legacy path identity; version 1 uses Unicode NFC. Physical file names stay unchanged. Update every device and stop synchronisation, then Rebuild both databases from a complete authoritative Vault and Fetch on the other devices. Resolve colliding file names first. P2P and Object Storage do not support version 1.":
|
||||
"CouchDB only. Version 0 preserves legacy path identity; version 1 uses Unicode NFC. Physical file names stay unchanged. Update every device and stop synchronisation, then Rebuild both databases from a complete authoritative Vault and Fetch on the other devices. Resolve colliding file names first. P2P and Object Storage do not support version 1."
|
||||
CouchDB validates the database name when you connect. The name must not be empty.:
|
||||
CouchDB validates the database name when you connect. The name must not be
|
||||
empty.
|
||||
@@ -556,6 +558,7 @@ lang-ru: Русский
|
||||
lang-zh: 简体中文
|
||||
lang-zh-tw: 繁體中文
|
||||
Later: Later
|
||||
Legacy path identity: Legacy path identity
|
||||
"Limit: {datetime} ({timestamp})": "Limit: {datetime} ({timestamp})"
|
||||
live: live
|
||||
LiveSync could not handle multiple vaults which have same name without different prefix, This should be automatically configured.:
|
||||
@@ -1251,6 +1254,7 @@ password: password
|
||||
Password: Password
|
||||
Paste a connection string: Paste a connection string
|
||||
Path: Path
|
||||
Path identity mode: Path identity mode
|
||||
Path Obfuscation: Path Obfuscation
|
||||
Patterns to match files for overwriting instead of merging: Patterns to match files for overwriting instead of merging
|
||||
Patterns to match files for syncing: Patterns to match files for syncing
|
||||
@@ -2042,6 +2046,7 @@ TweakMismatchResolve:
|
||||
TweakResolving: Configuration Mismatch Detected
|
||||
UseRemoteConfig: Use Remote Configuration
|
||||
AutoAcceptCompatible: Auto-Accept Available
|
||||
Unicode NFC path identity: Unicode NFC path identity
|
||||
Unique name between all synchronized devices. To edit this setting, please disable customization sync once.:
|
||||
Unique name between all synchronized devices. To edit this setting, please
|
||||
disable customization sync once.
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import type { RemoteDBSettings } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { usesEncryptedInternalMetadata } from "@vrtmrz/livesync-commonlib/replication";
|
||||
import { getPathNormalizationVersion } from "@vrtmrz/livesync-commonlib/settings";
|
||||
|
||||
type EndpointProjection = readonly [kind: "url" | "invalid-url", value: string];
|
||||
|
||||
@@ -44,17 +45,18 @@ function projectHeaders(value: string): readonly (readonly [name: string, value:
|
||||
|
||||
function projectRemoteSecurity(settings: RemoteDBSettings) {
|
||||
return [
|
||||
getPathNormalizationVersion(settings),
|
||||
settings.idDerivationVersion,
|
||||
settings.idDerivationKey,
|
||||
settings.encrypt
|
||||
? ([
|
||||
"encrypted",
|
||||
settings.passphrase,
|
||||
settings.useDynamicIterationCount,
|
||||
settings.E2EEAlgorithm,
|
||||
settings.permitEmptyPassphrase,
|
||||
] as const)
|
||||
: (["plain"] as const),
|
||||
? ([
|
||||
"encrypted",
|
||||
settings.passphrase,
|
||||
settings.useDynamicIterationCount,
|
||||
settings.E2EEAlgorithm,
|
||||
settings.permitEmptyPassphrase,
|
||||
] as const)
|
||||
: (["plain"] as const),
|
||||
] as const;
|
||||
}
|
||||
|
||||
|
||||
@@ -7,6 +7,30 @@ import {
|
||||
} from "./replicatorConfigurationIdentity";
|
||||
|
||||
describe("active Replicator configuration identity", () => {
|
||||
it("replaces a connection when the path identity mode changes", () => {
|
||||
const previous = configuredSettings({ pathNormalizationVersion: 0 });
|
||||
const next = { ...previous, pathNormalizationVersion: 1 as const };
|
||||
expect(getCouchDBReplicatorConfigurationIdentity(next)).not.toBe(
|
||||
getCouchDBReplicatorConfigurationIdentity(previous)
|
||||
);
|
||||
expect(getObjectStorageReplicatorConfigurationIdentity(next)).not.toBe(
|
||||
getObjectStorageReplicatorConfigurationIdentity(previous)
|
||||
);
|
||||
});
|
||||
|
||||
it("treats a missing path identity mode as version 0", () => {
|
||||
const missingVersion = configuredSettings();
|
||||
Reflect.deleteProperty(missingVersion, "pathNormalizationVersion");
|
||||
const explicitVersion0 = { ...missingVersion, pathNormalizationVersion: 0 as const };
|
||||
|
||||
expect(getCouchDBReplicatorConfigurationIdentity(explicitVersion0)).toBe(
|
||||
getCouchDBReplicatorConfigurationIdentity(missingVersion)
|
||||
);
|
||||
expect(getObjectStorageReplicatorConfigurationIdentity(explicitVersion0)).toBe(
|
||||
getObjectStorageReplicatorConfigurationIdentity(missingVersion)
|
||||
);
|
||||
});
|
||||
|
||||
function configuredSettings(overrides: Partial<ObsidianLiveSyncSettings> = {}): ObsidianLiveSyncSettings {
|
||||
return Object.assign(createNewVaultSettings(), {
|
||||
activeConfigurationId: "profile-a",
|
||||
|
||||
+6
-2
@@ -112,10 +112,10 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
|
||||
fileReflectionProvenance: createFileReflectionProvenance(services.keyValueDB),
|
||||
});
|
||||
const rebuilder = new ServiceRebuilder({
|
||||
path: services.path,
|
||||
events: services.context.events,
|
||||
API: services.API,
|
||||
database: services.database,
|
||||
path: services.path,
|
||||
appLifecycle: services.appLifecycle,
|
||||
setting: services.setting,
|
||||
remote: services.remote,
|
||||
@@ -185,7 +185,11 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
|
||||
core,
|
||||
(_compatibilityReplicator, p2p) => createInteractiveP2PReplication(p2p),
|
||||
createOpenRebuildUI(this.app),
|
||||
{ prepareP2PSettings: useP2PSettingsPreparation(core.services.API.webCompatFetch.bind(core.services.API)) }
|
||||
{
|
||||
prepareP2PSettings: useP2PSettingsPreparation(
|
||||
core.services.API.webCompatFetch.bind(core.services.API)
|
||||
),
|
||||
}
|
||||
);
|
||||
setupManager.registerP2PSetupConnectionProbe(replicator.connectionProbe);
|
||||
useP2PReplicatorCommands(core, replicator);
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { ChunkAlgorithmNames } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { $msg } from "@/common/translation";
|
||||
import type { SettingSpecGroup } from "./SettingSpec.ts";
|
||||
|
||||
export type AdvancedSettingSpecContext = {
|
||||
@@ -22,6 +23,18 @@ export function createAdvancedSettingSpecGroups({
|
||||
control: { type: "dropdown", options: () => ChunkAlgorithmNames },
|
||||
},
|
||||
{ key: "customChunkSize", control: { type: "number", min: 0, allowZero: true } },
|
||||
{
|
||||
key: "pathNormalizationVersion",
|
||||
holdValue: true,
|
||||
control: {
|
||||
type: "numeric-dropdown",
|
||||
options: () => ({
|
||||
"0": $msg("Legacy path identity"),
|
||||
"1": $msg("Unicode NFC path identity"),
|
||||
}),
|
||||
},
|
||||
visible: isCouchDB,
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
|
||||
@@ -243,6 +243,29 @@ export class LiveSyncSetting extends Setting {
|
||||
});
|
||||
return this;
|
||||
}
|
||||
|
||||
autoWireNumericDropDown(key: AllNumericItemKey, opt: AutoWireOption & { options: Record<string, string> }) {
|
||||
const conf = this.autoWireSetting(key, opt);
|
||||
this.addDropdown((dropdown) => {
|
||||
this.autoWiredComponent = dropdown;
|
||||
const setValue = wrapMemo((value: string) => {
|
||||
dropdown.setValue(value);
|
||||
});
|
||||
|
||||
dropdown.addOptions(opt.options);
|
||||
this.invalidateValue = () => setValue(`${LiveSyncSetting.env.editingSettings[key] ?? ""}`);
|
||||
this.invalidateValue();
|
||||
dropdown.onChange(async (value) => {
|
||||
const numericValue = Number(value);
|
||||
if (!Number.isInteger(numericValue) || !Object.prototype.hasOwnProperty.call(opt.options, value)) {
|
||||
return false;
|
||||
}
|
||||
await this.commitValue(numericValue);
|
||||
});
|
||||
this.autoWireComponent(this.autoWiredComponent, conf, opt);
|
||||
});
|
||||
return this;
|
||||
}
|
||||
addApplyButton(keys: AllSettingItemKey[], text?: string) {
|
||||
this.addButton((button) => {
|
||||
this.applyButtonComponent = button;
|
||||
|
||||
@@ -133,7 +133,11 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
|
||||
applySetting(keys: AllSettingItemKey[]) {
|
||||
for (const k of keys) {
|
||||
if (!this.isDirty(k)) continue;
|
||||
if (this.isConfigurationEncryptionSetting(k) || k in OnDialogSettingsDefault) {
|
||||
if (
|
||||
this.isReconstructionRequiredSetting(k) ||
|
||||
this.isConfigurationEncryptionSetting(k) ||
|
||||
k in OnDialogSettingsDefault
|
||||
) {
|
||||
// //@ts-ignore
|
||||
// this.initialSettings[k] = this.editingSettings[k];
|
||||
continue;
|
||||
@@ -147,8 +151,8 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
|
||||
});
|
||||
}
|
||||
async applyAllSettings() {
|
||||
const changedKeys = (Object.keys(this.editingSettings ?? {}) as AllSettingItemKey[]).filter((e) =>
|
||||
this.isDirty(e)
|
||||
const changedKeys = (Object.keys(this.editingSettings ?? {}) as AllSettingItemKey[]).filter(
|
||||
(e) => this.isDirty(e) && !this.isReconstructionRequiredSetting(e)
|
||||
);
|
||||
await this.saveConfigurationEncryptionIfDirty(changedKeys);
|
||||
this.applySetting(changedKeys);
|
||||
@@ -168,7 +172,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
|
||||
*/
|
||||
async saveSettings(keys: AllSettingItemKey[]) {
|
||||
let hasChanged = false;
|
||||
const appliedKeys = keys.filter((k) => this.isDirty(k));
|
||||
const appliedKeys = keys.filter((k) => this.isDirty(k) && !this.isReconstructionRequiredSetting(k));
|
||||
await this.saveConfigurationEncryptionIfDirty(appliedKeys);
|
||||
for (const k of appliedKeys) {
|
||||
if (this.isConfigurationEncryptionSetting(k)) continue;
|
||||
@@ -193,11 +197,16 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
|
||||
await Promise.all(handlers);
|
||||
// }
|
||||
keys.forEach((e) => {
|
||||
if (this.isReconstructionRequiredSetting(e) && this.isDirty(e)) return;
|
||||
if (this.isConfigurationEncryptionSetting(e) && this.isDirty(e)) return;
|
||||
this.refreshSetting(e);
|
||||
});
|
||||
}
|
||||
|
||||
private isReconstructionRequiredSetting(key: AllSettingItemKey): boolean {
|
||||
return key === "pathNormalizationVersion";
|
||||
}
|
||||
|
||||
private isConfigurationEncryptionSetting(key: AllSettingItemKey): boolean {
|
||||
return key === "configPassphraseStore" || key === "configPassphrase";
|
||||
}
|
||||
@@ -225,8 +234,8 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
|
||||
* @param keys setting keys for applying
|
||||
*/
|
||||
async saveAllDirtySettings() {
|
||||
const changedKeys = (Object.keys(this.editingSettings ?? {}) as AllSettingItemKey[]).filter((e) =>
|
||||
this.isDirty(e)
|
||||
const changedKeys = (Object.keys(this.editingSettings ?? {}) as AllSettingItemKey[]).filter(
|
||||
(e) => this.isDirty(e) && !this.isReconstructionRequiredSetting(e)
|
||||
);
|
||||
await this.saveSettings(changedKeys);
|
||||
this.reloadAllSettings();
|
||||
@@ -299,6 +308,13 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
|
||||
reloadAllSettings(skipUpdate: boolean = false) {
|
||||
const previousSettings = this._editingSettings;
|
||||
const previousSnapshot = this.initialSettings;
|
||||
const pendingPathNormalizationVersion =
|
||||
previousSettings && previousSnapshot && this.isDirty("pathNormalizationVersion")
|
||||
? {
|
||||
value: previousSettings.pathNormalizationVersion,
|
||||
appliedValue: previousSnapshot.pathNormalizationVersion,
|
||||
}
|
||||
: undefined;
|
||||
const pendingConfigurationEncryption =
|
||||
previousSettings && previousSnapshot
|
||||
? {
|
||||
@@ -326,6 +342,10 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
|
||||
if (pendingConfigurationEncryption?.configPassphrase) {
|
||||
this._editingSettings.configPassphrase = pendingConfigurationEncryption.configPassphrase.value;
|
||||
}
|
||||
if (pendingPathNormalizationVersion) {
|
||||
this._editingSettings.pathNormalizationVersion = pendingPathNormalizationVersion.value;
|
||||
this.initialSettings.pathNormalizationVersion = pendingPathNormalizationVersion.appliedValue;
|
||||
}
|
||||
if (!skipUpdate) this.requestUpdate();
|
||||
}
|
||||
|
||||
@@ -333,6 +353,10 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
|
||||
* Reread each setting and request invalidate
|
||||
*/
|
||||
refreshSetting(key: AllSettingItemKey) {
|
||||
if (this.isReconstructionRequiredSetting(key) && this.isDirty(key)) {
|
||||
this.requestUpdate();
|
||||
return;
|
||||
}
|
||||
const localSetting = this.reloadAllLocalSettings();
|
||||
if (key in this.core.settings) {
|
||||
if (key in localSetting) {
|
||||
@@ -627,6 +651,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
|
||||
"passphrase",
|
||||
"useDynamicIterationCount",
|
||||
"usePathObfuscation",
|
||||
"pathNormalizationVersion",
|
||||
"encrypt",
|
||||
// "remoteType",
|
||||
]);
|
||||
@@ -637,6 +662,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
|
||||
"passphrase",
|
||||
"useDynamicIterationCount",
|
||||
"usePathObfuscation",
|
||||
"pathNormalizationVersion",
|
||||
"encrypt",
|
||||
]);
|
||||
}
|
||||
@@ -682,16 +708,23 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
|
||||
|
||||
override getControlValue(key: string): unknown {
|
||||
const spec = this.getDeclarativeSettingSpec(key);
|
||||
return this.editingSettings[spec.key];
|
||||
const value = this.editingSettings[spec.key];
|
||||
return spec.control.type === "numeric-dropdown" ? `${value}` : value;
|
||||
}
|
||||
|
||||
override async setControlValue(key: string, value: unknown): Promise<void> {
|
||||
const spec = this.getDeclarativeSettingSpec(key);
|
||||
if (!isValidSettingSpecValue(spec, value)) {
|
||||
const persistedValue =
|
||||
spec.control.type === "numeric-dropdown" && typeof value === "string" ? Number(value) : value;
|
||||
if (!isValidSettingSpecValue(spec, persistedValue)) {
|
||||
throw new TypeError(`Invalid value for declarative setting ${key}`);
|
||||
}
|
||||
Reflect.set(this.editingSettings, spec.key, value);
|
||||
await this.saveSettings([spec.key]);
|
||||
Reflect.set(this.editingSettings, spec.key, persistedValue);
|
||||
if (spec.holdValue) {
|
||||
this.requestUpdate();
|
||||
} else {
|
||||
await this.saveSettings([spec.key]);
|
||||
}
|
||||
}
|
||||
|
||||
private createRebuildRequiredAction(): SettingDefinitionAction {
|
||||
@@ -1059,6 +1092,15 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
|
||||
};
|
||||
|
||||
rebuildDB = async (method: "localOnly" | "remoteOnly" | "rebuildBothByThisDevice" | "localOnlyWithChunks") => {
|
||||
if (method === "remoteOnly" && this.isDirty("pathNormalizationVersion")) {
|
||||
Logger(
|
||||
$msg(
|
||||
"CouchDB only. Version 0 preserves legacy path identity; version 1 uses Unicode NFC. Physical file names stay unchanged. Update every device and stop synchronisation, then Rebuild both databases from a complete authoritative Vault and Fetch on the other devices. Resolve colliding file names first. P2P and Object Storage do not support version 1."
|
||||
),
|
||||
LOG_LEVEL_NOTICE
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (this.editingSettings.encrypt && this.editingSettings.passphrase == "") {
|
||||
Logger($msg("obsidianLiveSyncSettingTab.logEncryptionNoPassphrase"), LOG_LEVEL_NOTICE);
|
||||
return;
|
||||
@@ -1109,6 +1151,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
|
||||
this.editingSettings.passphrase = "";
|
||||
}
|
||||
await this.saveAllDirtySettings();
|
||||
await this.applyReconstructionRequiredSettings();
|
||||
},
|
||||
});
|
||||
if (result.result === "scheduled") {
|
||||
@@ -1118,6 +1161,9 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
|
||||
if (result.result === "failed") {
|
||||
return;
|
||||
}
|
||||
if (this.isDirty("pathNormalizationVersion")) {
|
||||
return;
|
||||
}
|
||||
|
||||
const applyWithoutInitialisation = $msg(
|
||||
"Ui.SetupWizard.ApplySettingsInitialisation.ApplyWithoutInitialisation"
|
||||
@@ -1136,6 +1182,23 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
|
||||
}
|
||||
}
|
||||
|
||||
private async applyReconstructionRequiredSettings(): Promise<void> {
|
||||
const key = "pathNormalizationVersion";
|
||||
if (!this.isDirty(key)) return;
|
||||
|
||||
const previousValue = this.core.settings.pathNormalizationVersion;
|
||||
this.copySettingValue(this.core.settings, this.editingSettings, key);
|
||||
try {
|
||||
await this.services.setting.saveSettingData();
|
||||
} catch (error) {
|
||||
this.core.settings.pathNormalizationVersion = previousValue;
|
||||
throw error;
|
||||
}
|
||||
|
||||
this.copySettingValue(this.initialSettings, this.core.settings, key);
|
||||
this.refreshSetting(key);
|
||||
}
|
||||
|
||||
// The imperative renderer remains required by the declared Obsidian 1.7.2 minimum version.
|
||||
override display(): void {
|
||||
this.displayImperative();
|
||||
|
||||
@@ -316,6 +316,117 @@ describe("ObsidianLiveSyncSettingTab pending-setting initialisation", () => {
|
||||
};
|
||||
}
|
||||
|
||||
it("requires local and remote rebuilds when path identity changes", () => {
|
||||
const { tab } = createSettingsTab();
|
||||
const initialSettings = tab.initialSettings!;
|
||||
initialSettings.handleFilenameCaseSensitive = false;
|
||||
tab.editingSettings.handleFilenameCaseSensitive = false;
|
||||
Reflect.set(initialSettings, "pathNormalizationVersion", 0);
|
||||
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 1);
|
||||
|
||||
expect(tab.isNeedRebuildLocal()).toBe(true);
|
||||
expect(tab.isNeedRebuildRemote()).toBe(true);
|
||||
});
|
||||
|
||||
it("does not offer settings-only fallback when path identity changes", async () => {
|
||||
const { tab, saveSettingData, confirmWithMessage } = createSettingsTab();
|
||||
const initialSettings = tab.initialSettings!;
|
||||
Reflect.set(tab.core.settings, "pathNormalizationVersion", 0);
|
||||
Reflect.set(initialSettings, "pathNormalizationVersion", 0);
|
||||
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 1);
|
||||
settingsInitialisationMocks.applySettingsWithInitialisationChoice.mockResolvedValueOnce({
|
||||
result: "cancelled",
|
||||
});
|
||||
|
||||
await tab.confirmRebuild();
|
||||
|
||||
expect(confirmWithMessage).not.toHaveBeenCalled();
|
||||
expect(saveSettingData).not.toHaveBeenCalled();
|
||||
expect(Reflect.get(tab.core.settings, "pathNormalizationVersion")).toBe(0);
|
||||
expect(Reflect.get(tab.editingSettings, "pathNormalizationVersion")).toBe(1);
|
||||
expect(Reflect.get(initialSettings, "pathNormalizationVersion")).toBe(0);
|
||||
expect(tab.isDirty("pathNormalizationVersion")).toBe(true);
|
||||
});
|
||||
|
||||
it.each([
|
||||
["regular-expression Apply", "syncOnlyRegEx", "notes/.*"],
|
||||
["preset Apply", "preset", "LIVESYNC"],
|
||||
] as const)("keeps path identity pending during %s", async (_label, setting, value) => {
|
||||
const { tab } = createSettingsTab();
|
||||
const initialSettings = tab.initialSettings!;
|
||||
Reflect.set(tab.core.settings, "pathNormalizationVersion", 0);
|
||||
Reflect.set(initialSettings, "pathNormalizationVersion", 0);
|
||||
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 1);
|
||||
Reflect.set(tab.editingSettings, setting, value);
|
||||
if (setting === "preset") {
|
||||
tab.addOnSaved("preset", async () => {
|
||||
tab.editingSettings.liveSync = true;
|
||||
await tab.saveAllDirtySettings();
|
||||
});
|
||||
}
|
||||
|
||||
await tab.saveAllDirtySettings();
|
||||
|
||||
expect(Reflect.get(tab.core.settings, "pathNormalizationVersion")).toBe(0);
|
||||
expect(Reflect.get(tab.initialSettings!, "pathNormalizationVersion")).toBe(0);
|
||||
expect(Reflect.get(tab.editingSettings, "pathNormalizationVersion")).toBe(1);
|
||||
expect(tab.isDirty("pathNormalizationVersion")).toBe(true);
|
||||
if (setting === "syncOnlyRegEx") {
|
||||
expect(tab.core.settings.syncOnlyRegEx).toBe(value);
|
||||
} else {
|
||||
expect(tab.core.settings.liveSync).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it("excludes path identity from direct save operations", async () => {
|
||||
const { tab, saveSettingData } = createSettingsTab();
|
||||
const initialSettings = tab.initialSettings!;
|
||||
Reflect.set(tab.core.settings, "pathNormalizationVersion", 0);
|
||||
Reflect.set(initialSettings, "pathNormalizationVersion", 0);
|
||||
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 1);
|
||||
|
||||
await tab.saveSettings(["pathNormalizationVersion"]);
|
||||
|
||||
expect(saveSettingData).not.toHaveBeenCalled();
|
||||
expect(Reflect.get(tab.core.settings, "pathNormalizationVersion")).toBe(0);
|
||||
expect(Reflect.get(tab.initialSettings!, "pathNormalizationVersion")).toBe(0);
|
||||
expect(Reflect.get(tab.editingSettings, "pathNormalizationVersion")).toBe(1);
|
||||
expect(tab.isDirty("pathNormalizationVersion")).toBe(true);
|
||||
});
|
||||
|
||||
it("excludes path identity from ordinary Apply operations", async () => {
|
||||
const { tab, saveSettingData } = createSettingsTab();
|
||||
const initialSettings = tab.initialSettings!;
|
||||
Reflect.set(tab.core.settings, "pathNormalizationVersion", 0);
|
||||
Reflect.set(initialSettings, "pathNormalizationVersion", 0);
|
||||
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 1);
|
||||
|
||||
await tab.applyAllSettings();
|
||||
|
||||
expect(saveSettingData).not.toHaveBeenCalled();
|
||||
expect(Reflect.get(tab.core.settings, "pathNormalizationVersion")).toBe(0);
|
||||
expect(Reflect.get(tab.initialSettings!, "pathNormalizationVersion")).toBe(0);
|
||||
expect(Reflect.get(tab.editingSettings, "pathNormalizationVersion")).toBe(1);
|
||||
expect(tab.isDirty("pathNormalizationVersion")).toBe(true);
|
||||
});
|
||||
|
||||
it("retains a pending path identity edit and its applied baseline after reload", () => {
|
||||
const { tab } = createSettingsTab();
|
||||
const initialSettings = tab.initialSettings!;
|
||||
Reflect.set(tab.core.settings, "pathNormalizationVersion", 0);
|
||||
Reflect.set(initialSettings, "pathNormalizationVersion", 0);
|
||||
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 1);
|
||||
|
||||
tab.reloadAllSettings();
|
||||
|
||||
expect(Reflect.get(tab.core.settings, "pathNormalizationVersion")).toBe(0);
|
||||
expect(Reflect.get(tab.initialSettings!, "pathNormalizationVersion")).toBe(0);
|
||||
expect(Reflect.get(tab.editingSettings, "pathNormalizationVersion")).toBe(1);
|
||||
expect(tab.isDirty("pathNormalizationVersion")).toBe(true);
|
||||
expect(tab.isNeedRebuildLocal()).toBe(true);
|
||||
expect(tab.isNeedRebuildRemote()).toBe(true);
|
||||
});
|
||||
|
||||
it("keeps pending settings in the editing buffer when initialisation and the fallback are cancelled", async () => {
|
||||
const { tab, saveSettingData, confirmWithMessage, closeSetting } = createSettingsTab();
|
||||
settingsInitialisationMocks.applySettingsWithInitialisationChoice.mockResolvedValueOnce({
|
||||
@@ -353,8 +464,11 @@ describe("ObsidianLiveSyncSettingTab pending-setting initialisation", () => {
|
||||
expect(closeSetting).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("closes settings only after initialisation has been scheduled", async () => {
|
||||
it("applies path identity only from a successful initialisation reservation callback", async () => {
|
||||
const { tab, saveSettingData, confirmWithMessage, closeSetting } = createSettingsTab();
|
||||
Reflect.set(tab.core.settings, "pathNormalizationVersion", 0);
|
||||
Reflect.set(tab.initialSettings!, "pathNormalizationVersion", 0);
|
||||
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 1);
|
||||
settingsInitialisationMocks.applySettingsWithInitialisationChoice.mockImplementationOnce(
|
||||
async ({ applySettings }: { applySettings: () => Promise<void> }) => {
|
||||
await applySettings();
|
||||
@@ -364,9 +478,118 @@ describe("ObsidianLiveSyncSettingTab pending-setting initialisation", () => {
|
||||
|
||||
await tab.confirmRebuild();
|
||||
|
||||
expect(saveSettingData).toHaveBeenCalledOnce();
|
||||
expect(saveSettingData).toHaveBeenCalledTimes(2);
|
||||
expect(confirmWithMessage).not.toHaveBeenCalled();
|
||||
expect(closeSetting).toHaveBeenCalledOnce();
|
||||
expect(Reflect.get(tab.core.settings, "pathNormalizationVersion")).toBe(1);
|
||||
expect(Reflect.get(tab.initialSettings!, "pathNormalizationVersion")).toBe(1);
|
||||
expect(Reflect.get(tab.editingSettings, "pathNormalizationVersion")).toBe(1);
|
||||
expect(tab.isDirty("pathNormalizationVersion")).toBe(false);
|
||||
});
|
||||
|
||||
it("rolls path identity back and keeps it pending when saving after reservation fails", async () => {
|
||||
const { tab, saveSettingData, closeSetting } = createSettingsTab();
|
||||
const initialSettings = tab.initialSettings!;
|
||||
Reflect.set(tab.core.settings, "pathNormalizationVersion", 0);
|
||||
Reflect.set(initialSettings, "pathNormalizationVersion", 0);
|
||||
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 1);
|
||||
tab.editingSettings.handleFilenameCaseSensitive = false;
|
||||
const failure = new Error("settings could not be saved");
|
||||
saveSettingData.mockRejectedValueOnce(failure);
|
||||
settingsInitialisationMocks.applySettingsWithInitialisationChoice.mockImplementationOnce(
|
||||
async ({ applySettings }: { applySettings: () => Promise<void> }) => {
|
||||
await applySettings();
|
||||
return { result: "scheduled", mode: "rebuild" };
|
||||
}
|
||||
);
|
||||
|
||||
await expect(tab.confirmRebuild()).rejects.toBe(failure);
|
||||
|
||||
expect(saveSettingData).toHaveBeenCalledOnce();
|
||||
expect(Reflect.get(tab.core.settings, "pathNormalizationVersion")).toBe(0);
|
||||
expect(Reflect.get(tab.initialSettings!, "pathNormalizationVersion")).toBe(0);
|
||||
expect(Reflect.get(tab.editingSettings, "pathNormalizationVersion")).toBe(1);
|
||||
expect(tab.isDirty("pathNormalizationVersion")).toBe(true);
|
||||
expect(closeSetting).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("retains pending path identity when fetch validation fails", async () => {
|
||||
const { tab } = createSettingsTab();
|
||||
const initialSettings = tab.initialSettings!;
|
||||
Reflect.set(tab.core.settings, "pathNormalizationVersion", 0);
|
||||
Reflect.set(initialSettings, "pathNormalizationVersion", 0);
|
||||
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 1);
|
||||
vi.spyOn(tab, "checkWorkingPassphrase").mockResolvedValue(false);
|
||||
settingsInitialisationMocks.applySettingsWithInitialisationChoice.mockImplementationOnce(
|
||||
async ({ validateChoice }: { validateChoice: (mode: "fetch" | "rebuild") => Promise<boolean> }) => {
|
||||
expect(await validateChoice("fetch")).toBe(false);
|
||||
return { result: "failed", mode: "fetch" };
|
||||
}
|
||||
);
|
||||
|
||||
await tab.confirmRebuild();
|
||||
|
||||
expect(Reflect.get(tab.core.settings, "pathNormalizationVersion")).toBe(0);
|
||||
expect(Reflect.get(tab.initialSettings!, "pathNormalizationVersion")).toBe(0);
|
||||
expect(Reflect.get(tab.editingSettings, "pathNormalizationVersion")).toBe(1);
|
||||
expect(tab.isDirty("pathNormalizationVersion")).toBe(true);
|
||||
expect(tab.closeSetting).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("retains pending path identity when initialisation reservation fails", async () => {
|
||||
const { tab, saveSettingData } = createSettingsTab();
|
||||
const initialSettings = tab.initialSettings!;
|
||||
Reflect.set(tab.core.settings, "pathNormalizationVersion", 0);
|
||||
Reflect.set(initialSettings, "pathNormalizationVersion", 0);
|
||||
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 1);
|
||||
settingsInitialisationMocks.applySettingsWithInitialisationChoice.mockResolvedValueOnce({
|
||||
result: "failed",
|
||||
mode: "rebuild",
|
||||
});
|
||||
|
||||
await tab.confirmRebuild();
|
||||
|
||||
expect(saveSettingData).not.toHaveBeenCalled();
|
||||
expect(Reflect.get(tab.core.settings, "pathNormalizationVersion")).toBe(0);
|
||||
expect(Reflect.get(tab.initialSettings!, "pathNormalizationVersion")).toBe(0);
|
||||
expect(Reflect.get(tab.editingSettings, "pathNormalizationVersion")).toBe(1);
|
||||
expect(tab.isDirty("pathNormalizationVersion")).toBe(true);
|
||||
expect(tab.closeSetting).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("refuses remote-only overwrite with pending path identity before side effects", async () => {
|
||||
const { tab, saveSettingData, suspendAllSync, suspendExtraSync, performRebuildDB, closeSetting } =
|
||||
createSettingsTab();
|
||||
const initialSettings = tab.initialSettings!;
|
||||
Reflect.set(tab.core.settings, "pathNormalizationVersion", 0);
|
||||
Reflect.set(initialSettings, "pathNormalizationVersion", 0);
|
||||
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 1);
|
||||
tab.editingSettings.encrypt = false;
|
||||
const applyAllSettings = vi.spyOn(tab, "applyAllSettings");
|
||||
const reloadAllSettings = vi.spyOn(tab, "reloadAllSettings");
|
||||
|
||||
await tab.rebuildDB("remoteOnly");
|
||||
|
||||
expect(applyAllSettings).not.toHaveBeenCalled();
|
||||
expect(suspendAllSync).not.toHaveBeenCalled();
|
||||
expect(suspendExtraSync).not.toHaveBeenCalled();
|
||||
expect(reloadAllSettings).not.toHaveBeenCalled();
|
||||
expect(saveSettingData).not.toHaveBeenCalled();
|
||||
expect(performRebuildDB).not.toHaveBeenCalled();
|
||||
expect(closeSetting).not.toHaveBeenCalled();
|
||||
expect(Reflect.get(tab.core.settings, "pathNormalizationVersion")).toBe(0);
|
||||
expect(Reflect.get(tab.initialSettings!, "pathNormalizationVersion")).toBe(0);
|
||||
expect(Reflect.get(tab.editingSettings, "pathNormalizationVersion")).toBe(1);
|
||||
expect(tab.isDirty("pathNormalizationVersion")).toBe(true);
|
||||
expect(vi.mocked($msg)).toHaveBeenCalledWith(
|
||||
"CouchDB only. Version 0 preserves legacy path identity; version 1 uses Unicode NFC. Physical file names stay unchanged. Update every device and stop synchronisation, then Rebuild both databases from a complete authoritative Vault and Fetch on the other devices. Resolve colliding file names first. P2P and Object Storage do not support version 1."
|
||||
);
|
||||
expect(loggerMocks.Logger).toHaveBeenCalledWith(
|
||||
expect.stringContaining(
|
||||
"Rebuild both databases from a complete authoritative Vault and Fetch on the other devices."
|
||||
),
|
||||
LOG_LEVEL_NOTICE
|
||||
);
|
||||
});
|
||||
|
||||
it("does not offer the settings-only fallback after an initialisation failure", async () => {
|
||||
@@ -489,6 +712,33 @@ describe("ObsidianLiveSyncSettingTab declarative settings boundary", () => {
|
||||
expect(saveSettings).toHaveBeenCalledWith(["hashCacheMaxCount"]);
|
||||
});
|
||||
|
||||
it("keeps a path identity dropdown edit pending until the rebuild flow applies it", async () => {
|
||||
const { tab, saveSettingData } = createSettingsTab();
|
||||
const initialSettings = tab.initialSettings!;
|
||||
const saveSettings = vi.spyOn(tab, "saveSettings");
|
||||
vi.spyOn(tab, "requestUpdate").mockImplementation(() => undefined);
|
||||
Reflect.set(tab.core.settings, "pathNormalizationVersion", 0);
|
||||
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 0);
|
||||
Reflect.set(initialSettings, "pathNormalizationVersion", 0);
|
||||
|
||||
expect(tab.getControlValue("pathNormalizationVersion")).toBe("0");
|
||||
|
||||
await tab.setControlValue("pathNormalizationVersion", "1");
|
||||
|
||||
expect(Reflect.get(tab.editingSettings, "pathNormalizationVersion")).toBe(1);
|
||||
expect(tab.getControlValue("pathNormalizationVersion")).toBe("1");
|
||||
expect(Reflect.get(tab.core.settings, "pathNormalizationVersion")).toBe(0);
|
||||
expect(Reflect.get(initialSettings, "pathNormalizationVersion")).toBe(0);
|
||||
expect(saveSettings).not.toHaveBeenCalled();
|
||||
expect(tab.requestUpdate).toHaveBeenCalledOnce();
|
||||
expect(saveSettingData).not.toHaveBeenCalled();
|
||||
expect(tab.isNeedRebuildLocal()).toBe(true);
|
||||
expect(tab.isNeedRebuildRemote()).toBe(true);
|
||||
await expect(tab.setControlValue("pathNormalizationVersion", "0.5")).rejects.toThrow(
|
||||
/Invalid value for declarative setting/u
|
||||
);
|
||||
});
|
||||
|
||||
it("rejects unregistered declarative control keys", async () => {
|
||||
const { tab } = createSettingsTab();
|
||||
|
||||
|
||||
@@ -43,7 +43,7 @@ describe("paneAdvanced", () => {
|
||||
};
|
||||
|
||||
paneAdvanced.call(host as never, {} as HTMLElement, { addPanel } as never);
|
||||
await vi.waitFor(() => expect(settingHarness.rendered).toHaveLength(9));
|
||||
await vi.waitFor(() => expect(settingHarness.rendered).toHaveLength(10));
|
||||
|
||||
expect(addPanel.mock.calls.map(([, heading]) => heading)).toEqual([
|
||||
"Memory cache",
|
||||
@@ -55,6 +55,7 @@ describe("paneAdvanced", () => {
|
||||
panelElements.get("Memory cache"),
|
||||
panelElements.get("Local Database Tweak"),
|
||||
panelElements.get("Local Database Tweak"),
|
||||
panelElements.get("Local Database Tweak"),
|
||||
panelElements.get("Transfer Tweak"),
|
||||
panelElements.get("Transfer Tweak"),
|
||||
panelElements.get("Transfer Tweak"),
|
||||
@@ -66,6 +67,7 @@ describe("paneAdvanced", () => {
|
||||
"hashCacheMaxCount",
|
||||
"chunkSplitterVersion",
|
||||
"customChunkSize",
|
||||
"pathNormalizationVersion",
|
||||
"readChunksOnline",
|
||||
"useOnlyLocalChunk",
|
||||
"concurrencyOfReadChunksOnline",
|
||||
|
||||
@@ -11,6 +11,7 @@ export type PersistedSettingKey = PersistedBooleanSettingKey | PersistedStringSe
|
||||
type SettingSpecBase<K extends PersistedSettingKey, C> = {
|
||||
key: K;
|
||||
control: C;
|
||||
holdValue?: boolean;
|
||||
visible?: () => boolean;
|
||||
disabled?: () => boolean;
|
||||
aliases?: string[];
|
||||
@@ -34,6 +35,14 @@ type NumberSettingSpec = SettingSpecBase<
|
||||
}
|
||||
>;
|
||||
|
||||
type NumericDropdownSettingSpec = SettingSpecBase<
|
||||
PersistedNumericSettingKey,
|
||||
{
|
||||
type: "numeric-dropdown";
|
||||
options: () => Record<string, string>;
|
||||
}
|
||||
>;
|
||||
|
||||
type DropdownSettingSpec = SettingSpecBase<
|
||||
PersistedStringSettingKey,
|
||||
{
|
||||
@@ -42,7 +51,7 @@ type DropdownSettingSpec = SettingSpecBase<
|
||||
}
|
||||
>;
|
||||
|
||||
export type SettingSpec = ToggleSettingSpec | NumberSettingSpec | DropdownSettingSpec;
|
||||
export type SettingSpec = ToggleSettingSpec | NumberSettingSpec | NumericDropdownSettingSpec | DropdownSettingSpec;
|
||||
|
||||
/** An existing Advanced-page panel and the standard controls rendered within it. */
|
||||
export type SettingSpecGroup = {
|
||||
@@ -54,7 +63,7 @@ export type SettingSpecMessages = {
|
||||
valueShouldBeInRange: (range: { min?: number; max?: number }) => string;
|
||||
};
|
||||
|
||||
type LegacySettingOptions = Pick<AutoWireOption, "onUpdate">;
|
||||
type LegacySettingOptions = Pick<AutoWireOption, "holdValue" | "onUpdate">;
|
||||
|
||||
export type LegacySettingBinding =
|
||||
| {
|
||||
@@ -71,6 +80,11 @@ export type LegacySettingBinding =
|
||||
acceptZero?: boolean;
|
||||
};
|
||||
}
|
||||
| {
|
||||
type: "numeric-dropdown";
|
||||
key: PersistedNumericSettingKey;
|
||||
options: LegacySettingOptions & { options: Record<string, string> };
|
||||
}
|
||||
| {
|
||||
type: "dropdown";
|
||||
key: PersistedStringSettingKey;
|
||||
@@ -83,6 +97,10 @@ export interface LegacySettingSpecRenderer {
|
||||
key: AllNumericItemKey,
|
||||
options: AutoWireOption & { clampMin?: number; clampMax?: number; acceptZero?: boolean }
|
||||
): unknown;
|
||||
autoWireNumericDropDown(
|
||||
key: AllNumericItemKey,
|
||||
options: AutoWireOption & { options: Record<string, string> }
|
||||
): unknown;
|
||||
autoWireDropDown(key: AllStringItemKey, options: AutoWireOption & { options: Record<string, string> }): unknown;
|
||||
}
|
||||
|
||||
@@ -94,15 +112,21 @@ function isNumberSettingSpec(spec: SettingSpec): spec is NumberSettingSpec {
|
||||
return spec.control.type === "number";
|
||||
}
|
||||
|
||||
function isNumericDropdownSettingSpec(spec: SettingSpec): spec is NumericDropdownSettingSpec {
|
||||
return spec.control.type === "numeric-dropdown";
|
||||
}
|
||||
|
||||
function toLegacyUpdateOptions(spec: SettingSpec): LegacySettingOptions {
|
||||
if (!spec.visible && !spec.disabled) {
|
||||
return {};
|
||||
}
|
||||
return {
|
||||
onUpdate: (): OnUpdateResult => ({
|
||||
...(spec.visible ? { visibility: spec.visible() } : {}),
|
||||
...(spec.disabled ? { disabled: spec.disabled() } : {}),
|
||||
}),
|
||||
...(spec.holdValue ? { holdValue: spec.holdValue } : {}),
|
||||
...(!spec.visible && !spec.disabled
|
||||
? {}
|
||||
: {
|
||||
onUpdate: (): OnUpdateResult => ({
|
||||
...(spec.visible ? { visibility: spec.visible() } : {}),
|
||||
...(spec.disabled ? { disabled: spec.disabled() } : {}),
|
||||
}),
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -131,6 +155,16 @@ export function toLegacySettingBinding(spec: SettingSpec): LegacySettingBinding
|
||||
},
|
||||
};
|
||||
}
|
||||
if (isNumericDropdownSettingSpec(spec)) {
|
||||
return {
|
||||
type: "numeric-dropdown",
|
||||
key: spec.key,
|
||||
options: {
|
||||
...updateOptions,
|
||||
options: spec.control.options(),
|
||||
},
|
||||
};
|
||||
}
|
||||
return {
|
||||
type: "dropdown",
|
||||
key: spec.key,
|
||||
@@ -151,6 +185,9 @@ export function renderLegacySettingSpec(renderer: LegacySettingSpecRenderer, spe
|
||||
case "number":
|
||||
renderer.autoWireNumeric(binding.key, binding.options);
|
||||
return;
|
||||
case "numeric-dropdown":
|
||||
renderer.autoWireNumericDropDown(binding.key, binding.options);
|
||||
return;
|
||||
case "dropdown":
|
||||
renderer.autoWireDropDown(binding.key, binding.options);
|
||||
}
|
||||
@@ -176,6 +213,12 @@ export function isValidSettingSpecValue(spec: SettingSpec, value: unknown): bool
|
||||
return typeof value === "boolean";
|
||||
case "number":
|
||||
return typeof value === "number" && !numberIsOutOfRange(value, spec.control);
|
||||
case "numeric-dropdown":
|
||||
return (
|
||||
typeof value === "number" &&
|
||||
Number.isInteger(value) &&
|
||||
Object.prototype.hasOwnProperty.call(spec.control.options(), `${value}`)
|
||||
);
|
||||
case "dropdown":
|
||||
return typeof value === "string" && Object.prototype.hasOwnProperty.call(spec.control.options(), value);
|
||||
}
|
||||
@@ -229,6 +272,14 @@ export function toObsidianSettingDefinition(
|
||||
};
|
||||
break;
|
||||
}
|
||||
case "numeric-dropdown":
|
||||
control = {
|
||||
type: "dropdown",
|
||||
key: spec.key,
|
||||
options: spec.control.options(),
|
||||
...(spec.disabled ? { disabled: spec.disabled } : {}),
|
||||
};
|
||||
break;
|
||||
case "dropdown":
|
||||
control = {
|
||||
type: "dropdown",
|
||||
|
||||
@@ -2,6 +2,7 @@ import { describe, expect, expectTypeOf, it, vi } from "vitest";
|
||||
import { statusDisplay, type ConfigurationItem } from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import { getConfig, type OnDialogSettings } from "./settingConstants.ts";
|
||||
import {
|
||||
isValidSettingSpecValue,
|
||||
renderLegacySettingSpec,
|
||||
toLegacySettingBinding,
|
||||
toObsidianSettingDefinition,
|
||||
@@ -16,7 +17,7 @@ const rangeMessages = {
|
||||
};
|
||||
|
||||
describe("Advanced setting specifications", () => {
|
||||
it("lists only the nine controls currently exposed by the Advanced page", () => {
|
||||
it("lists only the ten controls currently exposed by the Advanced page", () => {
|
||||
const groups = createAdvancedSettingSpecGroups({ isCouchDB: () => true });
|
||||
|
||||
expect(groups.map(({ heading }) => heading)).toEqual([
|
||||
@@ -29,6 +30,7 @@ describe("Advanced setting specifications", () => {
|
||||
"hashCacheMaxCount",
|
||||
"chunkSplitterVersion",
|
||||
"customChunkSize",
|
||||
"pathNormalizationVersion",
|
||||
"readChunksOnline",
|
||||
"useOnlyLocalChunk",
|
||||
"concurrencyOfReadChunksOnline",
|
||||
@@ -38,12 +40,13 @@ describe("Advanced setting specifications", () => {
|
||||
]);
|
||||
});
|
||||
|
||||
it("keeps CouchDB visibility on the four remote chunk controls", () => {
|
||||
it("keeps path identity and the four remote chunk controls CouchDB-only", () => {
|
||||
let couchDB = false;
|
||||
const groups = createAdvancedSettingSpecGroups({ isCouchDB: () => couchDB });
|
||||
const conditional = groups.flatMap(({ items }) => items).filter(({ visible }) => visible !== undefined);
|
||||
|
||||
expect(conditional.map(({ key }) => key)).toEqual([
|
||||
"pathNormalizationVersion",
|
||||
"readChunksOnline",
|
||||
"useOnlyLocalChunk",
|
||||
"concurrencyOfReadChunksOnline",
|
||||
@@ -55,6 +58,44 @@ describe("Advanced setting specifications", () => {
|
||||
expect(conditional.every(({ visible }) => visible?.() === true)).toBe(true);
|
||||
});
|
||||
|
||||
it("exposes CouchDB path identity as a two-choice numeric dropdown", () => {
|
||||
let couchDB = false;
|
||||
const spec = createAdvancedSettingSpecGroups({ isCouchDB: () => couchDB })
|
||||
.flatMap(({ items }) => items)
|
||||
.find(({ key }) => key === "pathNormalizationVersion");
|
||||
if (!spec) throw new Error("Path identity mode is not exposed on the Advanced page");
|
||||
if (spec.control.type !== "numeric-dropdown") throw new Error("Expected a numeric dropdown");
|
||||
|
||||
expect(spec.control).toMatchObject({ type: "numeric-dropdown", options: expect.any(Function) });
|
||||
expect(spec.visible?.()).toBe(false);
|
||||
couchDB = true;
|
||||
expect(spec.visible?.()).toBe(true);
|
||||
expect(spec.control.options()).toEqual({
|
||||
"0": "Legacy path identity",
|
||||
"1": "Unicode NFC path identity",
|
||||
});
|
||||
expect(isValidSettingSpecValue(spec, 0)).toBe(true);
|
||||
expect(isValidSettingSpecValue(spec, 1)).toBe(true);
|
||||
expect(isValidSettingSpecValue(spec, 0.5)).toBe(false);
|
||||
expect(isValidSettingSpecValue(spec, 2)).toBe(false);
|
||||
|
||||
const definition = toObsidianSettingDefinition(spec, { name: "Path identity mode" }, rangeMessages);
|
||||
expect(definition.control).toEqual({
|
||||
type: "dropdown",
|
||||
key: "pathNormalizationVersion",
|
||||
options: { "0": "Legacy path identity", "1": "Unicode NFC path identity" },
|
||||
});
|
||||
expect(toLegacySettingBinding(spec)).toEqual({
|
||||
type: "numeric-dropdown",
|
||||
key: "pathNormalizationVersion",
|
||||
options: {
|
||||
holdValue: true,
|
||||
onUpdate: expect.any(Function),
|
||||
options: { "0": "Legacy path identity", "1": "Unicode NFC path identity" },
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("excludes transient settings-dialogue keys from the persisted key type", () => {
|
||||
type OnDialogOverlap = Extract<keyof OnDialogSettings, PersistedSettingKey>;
|
||||
expectTypeOf<OnDialogOverlap>().toEqualTypeOf<never>();
|
||||
@@ -225,6 +266,7 @@ describe("SettingSpec conversion", () => {
|
||||
const renderer = {
|
||||
autoWireToggle: vi.fn(),
|
||||
autoWireNumeric: vi.fn(),
|
||||
autoWireNumericDropDown: vi.fn(),
|
||||
autoWireDropDown: vi.fn(),
|
||||
};
|
||||
const visible = vi.fn(() => false);
|
||||
@@ -241,6 +283,7 @@ describe("SettingSpec conversion", () => {
|
||||
expect(options.onUpdate()).toEqual({ visibility: false });
|
||||
expect(visible).toHaveBeenCalledOnce();
|
||||
expect(renderer.autoWireNumeric).not.toHaveBeenCalled();
|
||||
expect(renderer.autoWireNumericDropDown).not.toHaveBeenCalled();
|
||||
expect(renderer.autoWireDropDown).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -86,6 +86,7 @@ describe("settings page catalogue", () => {
|
||||
"hashCacheMaxCount",
|
||||
"chunkSplitterVersion",
|
||||
"customChunkSize",
|
||||
"pathNormalizationVersion",
|
||||
"readChunksOnline",
|
||||
"useOnlyLocalChunk",
|
||||
"concurrencyOfReadChunksOnline",
|
||||
|
||||
@@ -1,4 +1,9 @@
|
||||
import { assessRemoteFeatureDocument, describeRemoteFeatureRejection } from "@vrtmrz/livesync-commonlib/replication";
|
||||
import {
|
||||
assessRemoteFeatureDocument,
|
||||
describeRemoteFeatureRejection,
|
||||
NFC_PATH_IDENTITY_FEATURE,
|
||||
} from "@vrtmrz/livesync-commonlib/replication";
|
||||
import { getPathNormalizationVersion } from "@vrtmrz/livesync-commonlib/settings";
|
||||
import {
|
||||
SYNCINFO_ID,
|
||||
VERSIONING_DOCID,
|
||||
@@ -33,7 +38,7 @@ const REPROCESS_BATCH_SIZE = 100;
|
||||
type ReplicateResultProcessorSettings = Pick<
|
||||
ObsidianLiveSyncSettings,
|
||||
"maxMTimeForReflectEvents" | "suspendParseReplicationResult"
|
||||
>;
|
||||
> & { pathNormalizationVersion?: 0 | 1 };
|
||||
type ReplicateResultProcessorServices = Pick<
|
||||
LiveSyncBaseCore["services"],
|
||||
"appLifecycle" | "database" | "path" | "replication" | "vault"
|
||||
@@ -285,13 +290,20 @@ export class ReplicateResultProcessor {
|
||||
if (change._id === VERSIONING_DOCID || change.type === "versioninfo") {
|
||||
this.log(`Version info document received: ${change._id}`, LOG_LEVEL_VERBOSE);
|
||||
const assessment = assessRemoteFeatureDocument(change);
|
||||
if (assessment.status !== "supported" && assessment.status !== "older-generation") {
|
||||
const pathIdentityDoesNotMatch =
|
||||
assessment.status === "supported" &&
|
||||
assessment.usedFeatures.includes(NFC_PATH_IDENTITY_FEATURE) !==
|
||||
(getPathNormalizationVersion(this.context.currentSettings()) === 1);
|
||||
if (
|
||||
(assessment.status !== "supported" && assessment.status !== "older-generation") ||
|
||||
pathIdentityDoesNotMatch
|
||||
) {
|
||||
// Fence and retire the active publication through its owner.
|
||||
this.context.requestActiveReplicatorRetirement();
|
||||
this.log(
|
||||
`${describeRemoteFeatureRejection(assessment)} Update Self-hosted LiveSync before synchronising.`,
|
||||
LOG_LEVEL_NOTICE
|
||||
);
|
||||
const rejection = pathIdentityDoesNotMatch
|
||||
? "Local and remote path identity settings do not match. Select the matching mode and Fetch the migrated database before synchronising."
|
||||
: `${describeRemoteFeatureRejection(assessment)} Update Self-hosted LiveSync before synchronising.`;
|
||||
this.log(rejection, LOG_LEVEL_NOTICE);
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
@@ -6,6 +6,7 @@ import {
|
||||
type EntryDoc,
|
||||
type FilePathWithPrefix,
|
||||
} from "@vrtmrz/livesync-commonlib/compat/common/types";
|
||||
import * as replicationCompatibility from "@vrtmrz/livesync-commonlib/replication";
|
||||
import { ENCRYPTED_INTERNAL_METADATA_FEATURE, REMOTE_FEATURE_GENERATION } from "@vrtmrz/livesync-commonlib/replication";
|
||||
import {
|
||||
isValidFilenameInAndroid,
|
||||
@@ -46,6 +47,7 @@ type SetupOptions = {
|
||||
localVersionInfo?: unknown;
|
||||
isTargetFile?: (path: string) => Promise<boolean>;
|
||||
databaseId?: string;
|
||||
pathNormalizationVersion?: number;
|
||||
};
|
||||
|
||||
function setup(options: SetupOptions = {}) {
|
||||
@@ -96,6 +98,7 @@ function setup(options: SetupOptions = {}) {
|
||||
currentSettings: () => ({
|
||||
maxMTimeForReflectEvents: options.maxMTimeForReflectEvents ?? 0,
|
||||
suspendParseReplicationResult: false,
|
||||
pathNormalizationVersion: options.pathNormalizationVersion ?? 0,
|
||||
}),
|
||||
getKeyValueDB: () => core.kvDB,
|
||||
getLocalDatabase: () => core.localDatabase,
|
||||
@@ -322,6 +325,66 @@ describe("ReplicateResultProcessor", () => {
|
||||
expect(onCloseActiveReplication).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it.each([
|
||||
["local version 0 and remote version 1", 0, ["nfc-path-identity-v1"], true],
|
||||
["local version 1 and remote version 0", 1, [], true],
|
||||
["both use version 1", 1, ["nfc-path-identity-v1"], false],
|
||||
["both use version 0", 0, [], false],
|
||||
] as const)(
|
||||
"checks path identity compatibility for %s",
|
||||
(_description, localVersion, remoteFeatures, shouldRetire) => {
|
||||
const originalAssessment = replicationCompatibility.assessRemoteFeatureDocument;
|
||||
const assessmentSpy = vi
|
||||
.spyOn(replicationCompatibility, "assessRemoteFeatureDocument")
|
||||
.mockImplementation((document) => {
|
||||
const assessment = originalAssessment(document);
|
||||
if (
|
||||
assessment.status !== "unknown-features" ||
|
||||
assessment.identifiers.length !== 1 ||
|
||||
assessment.identifiers[0] !== "nfc-path-identity-v1"
|
||||
) {
|
||||
return assessment;
|
||||
}
|
||||
return { status: "supported", usedFeatures: ["nfc-path-identity-v1"] };
|
||||
});
|
||||
|
||||
try {
|
||||
const { onCloseActiveReplication, processor } = setup({ pathNormalizationVersion: localVersion });
|
||||
processor.enqueueAll([
|
||||
{
|
||||
_id: VERSIONING_DOCID,
|
||||
_rev: "2-path-identity",
|
||||
type: "versioninfo",
|
||||
version: REMOTE_FEATURE_GENERATION,
|
||||
used_features: [...remoteFeatures],
|
||||
} as PouchDB.Core.ExistingDocument<EntryDoc>,
|
||||
]);
|
||||
|
||||
if (shouldRetire) {
|
||||
expect(onCloseActiveReplication).toHaveBeenCalledOnce();
|
||||
} else {
|
||||
expect(onCloseActiveReplication).not.toHaveBeenCalled();
|
||||
}
|
||||
} finally {
|
||||
assessmentSpy.mockRestore();
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
it("keeps the existing older-generation gate for local path identity version 1", () => {
|
||||
const { onCloseActiveReplication, processor } = setup({ pathNormalizationVersion: 1 });
|
||||
processor.enqueueAll([
|
||||
{
|
||||
_id: VERSIONING_DOCID,
|
||||
_rev: "1-legacy-path-identity",
|
||||
type: "versioninfo",
|
||||
version: 11,
|
||||
} as PouchDB.Core.ExistingDocument<EntryDoc>,
|
||||
]);
|
||||
|
||||
expect(onCloseActiveReplication).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("reports unknown feature identifiers and requests Replicator retirement", () => {
|
||||
const logger = vi.fn();
|
||||
setGlobalLogFunction(logger);
|
||||
|
||||
@@ -293,6 +293,8 @@ describe("file provenance across multiple devices and real CouchDB", () => {
|
||||
const { keep, content } = await resolveAndReplicate(f);
|
||||
const index = f.revisions.findIndex((rev, i) => i > 0 && rev !== keep);
|
||||
const device = f.devices[index];
|
||||
const storage = device.getStorage();
|
||||
expect(await storage.body.text()).not.toBe(content);
|
||||
const record = device.reflection.get(path);
|
||||
expect(record?.revision).toBe(f.revisions[index]);
|
||||
expect(record?.contentHash).toMatch(/^sha256:[0-9a-f]{64}$/u);
|
||||
@@ -305,12 +307,15 @@ describe("file provenance across multiple devices and real CouchDB", () => {
|
||||
expect((await device.db.info()).update_seq).toBe(before);
|
||||
expect(await device.getStorage().body.text()).toBe(content);
|
||||
expect(device.reflection.get(path)?.revision).toBe(keep);
|
||||
await expect(device.handler.storeFileToDB(path)).resolves.toBe(true);
|
||||
expect((await device.db.info()).update_seq).toBe(before);
|
||||
await device.db.replicate.to(f.remote);
|
||||
for (const peer of f.devices) {
|
||||
await peer.db.replicate.from(f.remote);
|
||||
expect((await leaves(peer.db)).map((doc) => doc._rev)).toEqual([keep]);
|
||||
expect(await revisionContent(peer, keep)).toBe(content);
|
||||
}
|
||||
expect((await leaves(f.remote)).map((doc) => doc._rev)).toEqual([keep]);
|
||||
}, 60_000);
|
||||
|
||||
it.each(["missing record", "compacted legacy base", "compacted hashed base with local edits"] as const)(
|
||||
|
||||
+1
-1
@@ -116,7 +116,7 @@ Each simulated device owns a separate real PouchDB database, `LiveSyncLocalDB` m
|
||||
| Four editing devices | Repeat the same sequence with four independently edited branches. | All four contents are initially preserved, and the same unchanged-save and convergence guarantees hold. |
|
||||
| A genuine edit on a losing device | After three-way resolution reaches its DB, a device adds content to its still-unreflected losing file, then saves and replicates it. | The new revision extends that device's recorded branch. The selected result and the new edit remain readable on every replica. |
|
||||
| Missing provenance | After four-way resolution, remove a losing device's provenance record and set its file to the original historical ancestor's content. Save, remove provenance again, and repeat the save. | The file becomes one fresh independent root, distinct from the historical root. Both contents remain readable after replication, and repeated saves create no duplicates. CLI `resolve` can select the independent root and propagate its resolution. |
|
||||
| Compacted base with verified unchanged content | After four-way resolution, retain a losing device's content hash, compact its local DB, and confirm that its recorded revision body is unavailable. Save the unchanged file. | The selected content is reflected without adding a database revision or conflict, and all replicas retain only the selected branch. |
|
||||
| Compacted base with verified unchanged content | After four-way resolution, retain a losing device's content hash, compact its local DB, and confirm that its recorded revision body is unavailable. Process the unchanged file twice. | The selected content is reflected without adding a database revision or conflict, provenance advances to that revision, and all replicas retain only the selected branch. |
|
||||
| Compacted base with legacy provenance | After four-way resolution, retain only the losing device's revision and observed modification time, compact its local DB, and confirm that the recorded revision body is unavailable. Save the remaining file, then repeat after removing provenance. | The file is preserved as one independent conflict. Both contents remain readable on every replica, repeated saves create no duplicates, and CLI `resolve` can select the independent root and propagate its resolution. |
|
||||
| Compacted base with a genuine edit | After four-way resolution and local compaction, retain the losing device's content hash but edit its file without changing its byte length or modification time. | The changed content is preserved as one independent conflict, remains readable on every replica, and can be selected through CLI `resolve`. Repeated saves after provenance loss create no duplicates. |
|
||||
|
||||
|
||||
@@ -170,6 +170,10 @@ Set `E2E_OBSIDIAN_ONLY_DIFFERENT_CHUNK_ID_KEYS=true` to run the focused case whe
|
||||
|
||||
`test:e2e:obsidian:cli-to-obsidian-sync` is the cross-runtime compatibility check for the official LiveSync CLI and the real Obsidian plug-in. Build the plug-in first, and build the local CLI too when no external CLI command is selected. The script uses E2EE, Path Obfuscation, and the current preferred chunk settings to create and synchronise a note through the CLI, starts real Obsidian with an isolated Vault and profile, synchronises the same CouchDB database, and verifies that the plug-in materialises identical note content. This covers the boundary that CLI-only and plug-in-only round trips do not exercise.
|
||||
|
||||
Set `E2E_OBSIDIAN_UNICODE_PATHS=true` to exercise path identity version 1 on Linux. This variant uses an NFD directory and a mixed NFC/NFD file name, checks reflection at Obsidian's NFC path, and returns an update and deletion to the original CLI physical path. Updates use `sync` and `mirror`; deletions use the polling daemon, because `mirror` retains tombstoned local files by default. It uses unencrypted Metadata by default to match issue #1259; add `E2E_OBSIDIAN_UNICODE_ENCRYPTION=true` for the encrypted variant. These checks do not establish native macOS or iOS filesystem behaviour.
|
||||
|
||||
Set `E2E_OBSIDIAN_UNICODE_MIGRATION=true` to begin with two CLI databases using legacy path identity. The source runs `rebuild --force --write-settings` after selecting version 1, and the second client runs `fetch --force --write-settings`. The scenario checks that the old document ID is replaced, the NFC feature is declared, and both physical NFD filenames are preserved before continuing the real Obsidian update and deletion checks.
|
||||
|
||||
The isolated Obsidian session starts with its CouchDB settings and device-local compatibility acknowledgement already in place. This keeps the scenario focused on cross-runtime data compatibility; unconfigured start-up and visible CouchDB onboarding are covered by their dedicated workflows.
|
||||
|
||||
By default, the compatibility check runs `node src/apps/cli/dist/index.cjs`. Set `LIVESYNC_CLI_COMMAND` to test another CLI build or distribution. The value may be a quoted command line or a JSON array of executable and prefix arguments; the scenario arguments are appended without going through a shell.
|
||||
|
||||
@@ -28,7 +28,11 @@ process.env.E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS ??= "30000";
|
||||
process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ??= "30000";
|
||||
|
||||
const liveSyncCli = resolve("src/apps/cli/dist/index.cjs");
|
||||
const notePath = "E2E/cli-to-obsidian.md";
|
||||
const migrateUnicodePaths = process.env.E2E_OBSIDIAN_UNICODE_MIGRATION === "true";
|
||||
const unicodePaths = migrateUnicodePaths || process.env.E2E_OBSIDIAN_UNICODE_PATHS === "true";
|
||||
const useEncryption = !unicodePaths || process.env.E2E_OBSIDIAN_UNICODE_ENCRYPTION === "true";
|
||||
const notePath = unicodePaths ? "Cafe\u0301/r\u00e9sume\u0301.md" : "E2E/cli-to-obsidian.md";
|
||||
const receivedNotePath = unicodePaths ? notePath.normalize("NFC") : notePath;
|
||||
const noteContent = [
|
||||
"# CLI to real Obsidian",
|
||||
"",
|
||||
@@ -183,7 +187,8 @@ async function configureLiveSyncCli(
|
||||
command: LiveSyncCliCommand,
|
||||
settingsPath: string,
|
||||
couchDb: Awaited<ReturnType<typeof loadCouchDbConfig>>,
|
||||
dbName: string
|
||||
dbName: string,
|
||||
pathNormalizationVersion = unicodePaths ? 1 : 0
|
||||
): Promise<void> {
|
||||
await runLiveSyncCli(command, ["init-settings", "--force", settingsPath]);
|
||||
const settings = JSON.parse(await readFile(settingsPath, "utf-8")) as Record<string, unknown>;
|
||||
@@ -210,16 +215,61 @@ async function configureLiveSyncCli(
|
||||
handleFilenameCaseSensitive: false,
|
||||
doNotUseFixedRevisionForChunks: true,
|
||||
E2EEAlgorithm: "v2",
|
||||
encrypt: true,
|
||||
encrypt: useEncryption,
|
||||
passphrase: e2eePassphrase,
|
||||
usePathObfuscation: true,
|
||||
encryptInternalMetadata: true,
|
||||
usePathObfuscation: useEncryption,
|
||||
encryptInternalMetadata: useEncryption,
|
||||
pathNormalizationVersion,
|
||||
doctorProcessedVersion: "0.25.27",
|
||||
isConfigured: true,
|
||||
});
|
||||
await writeFile(settingsPath, `${JSON.stringify(settings, null, 2)}\n`, "utf-8");
|
||||
}
|
||||
|
||||
async function startLiveSyncCliDaemon(command: LiveSyncCliCommand, databasePath: string, settingsPath: string) {
|
||||
const child = spawn(
|
||||
command.executable,
|
||||
[...command.prefixArgs, databasePath, "--settings", settingsPath, "--interval", "1", "daemon"],
|
||||
{ cwd: process.cwd(), env: process.env, stdio: ["ignore", "pipe", "pipe"] }
|
||||
);
|
||||
let output = "";
|
||||
let spawnError: Error | undefined;
|
||||
child.stdout.on("data", (chunk: Buffer) => {
|
||||
output += chunk.toString("utf-8");
|
||||
});
|
||||
child.stderr.on("data", (chunk: Buffer) => {
|
||||
output += chunk.toString("utf-8");
|
||||
});
|
||||
child.on("error", (error) => {
|
||||
spawnError = error;
|
||||
});
|
||||
const stop = async () => {
|
||||
if (child.exitCode !== null || child.signalCode !== null || spawnError) return;
|
||||
await new Promise<void>((resolvePromise) => {
|
||||
const timeout = setTimeout(() => child.kill("SIGKILL"), 10000);
|
||||
child.once("exit", () => {
|
||||
clearTimeout(timeout);
|
||||
resolvePromise();
|
||||
});
|
||||
child.kill("SIGTERM");
|
||||
});
|
||||
};
|
||||
const deadline = Date.now() + Number(process.env.E2E_LIVESYNC_CLI_TIMEOUT_MS ?? 60000);
|
||||
try {
|
||||
while (!output.includes("[Daemon] Polling mode:")) {
|
||||
if (spawnError) throw spawnError;
|
||||
if (child.exitCode !== null || child.signalCode !== null || Date.now() >= deadline) {
|
||||
throw new Error(`The CLI daemon did not become ready.\n${output}`);
|
||||
}
|
||||
await new Promise((resolvePromise) => setTimeout(resolvePromise, 100));
|
||||
}
|
||||
return { stop, getOutput: () => output };
|
||||
} catch (error) {
|
||||
await stop();
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
async function writeCliNote(
|
||||
command: LiveSyncCliCommand,
|
||||
databasePath: string,
|
||||
@@ -280,6 +330,86 @@ async function waitForVaultContent(
|
||||
throw new Error(`Timed out waiting for CLI-created note at ${fullPath}. Last content:\n${lastContent}`);
|
||||
}
|
||||
|
||||
async function migrateCliDatabases(
|
||||
command: LiveSyncCliCommand,
|
||||
cliState: string,
|
||||
databasePath: string,
|
||||
settingsPath: string,
|
||||
couchDb: Awaited<ReturnType<typeof loadCouchDbConfig>>,
|
||||
dbName: string,
|
||||
legacyFileInfo: CliFileInfo
|
||||
): Promise<CliFileInfo> {
|
||||
const secondaryDatabasePath = join(cliState, "secondary-database");
|
||||
const secondarySettingsPath = join(cliState, "secondary-settings.json");
|
||||
await mkdir(secondaryDatabasePath, { recursive: true });
|
||||
await configureLiveSyncCli(command, secondarySettingsPath, couchDb, dbName, 0);
|
||||
await runLiveSyncCli(command, [secondaryDatabasePath, "--settings", secondarySettingsPath, "sync"]);
|
||||
await runLiveSyncCli(command, [secondaryDatabasePath, "--settings", secondarySettingsPath, "mirror"]);
|
||||
assertEqual(
|
||||
await readFile(join(secondaryDatabasePath, notePath), "utf-8"),
|
||||
editedNoteContent,
|
||||
"The secondary CLI did not materialise the legacy NFD note before migration."
|
||||
);
|
||||
|
||||
for (const path of [settingsPath, secondarySettingsPath]) {
|
||||
const settings = JSON.parse(await readFile(path, "utf-8"));
|
||||
settings.pathNormalizationVersion = 1;
|
||||
await writeFile(path, `${JSON.stringify(settings, null, 2)}\n`, "utf-8");
|
||||
}
|
||||
await runLiveSyncCli(command, [databasePath, "--settings", settingsPath, "rebuild", "--force", "--write-settings"]);
|
||||
const info = await runLiveSyncCli(command, [databasePath, "--settings", settingsPath, "info", notePath]);
|
||||
const fileInfo = JSON.parse(info.stdout) as CliFileInfo;
|
||||
if (fileInfo.id === legacyFileInfo.id) {
|
||||
throw new Error("Rebuild retained the legacy NFD document ID.");
|
||||
}
|
||||
await waitForCouchDbDocs(couchDb, dbName, (docs) => {
|
||||
const ids = new Set(docs.map((doc) => doc._id));
|
||||
return (
|
||||
!ids.has(legacyFileInfo.id) &&
|
||||
ids.has(fileInfo.id) &&
|
||||
fileInfo.children.every((childId) => ids.has(childId)) &&
|
||||
docs.some((doc) => Array.isArray(doc.used_features) && doc.used_features.includes("nfc-path-identity-v1"))
|
||||
);
|
||||
});
|
||||
await runLiveSyncCli(command, [
|
||||
secondaryDatabasePath,
|
||||
"--settings",
|
||||
secondarySettingsPath,
|
||||
"fetch",
|
||||
"--force",
|
||||
"--write-settings",
|
||||
]);
|
||||
const secondaryInfo = await runLiveSyncCli(command, [
|
||||
secondaryDatabasePath,
|
||||
"--settings",
|
||||
secondarySettingsPath,
|
||||
"info",
|
||||
notePath,
|
||||
]);
|
||||
assertEqual(
|
||||
(JSON.parse(secondaryInfo.stdout) as CliFileInfo).id,
|
||||
fileInfo.id,
|
||||
"Fetch did not replace the secondary CLI's legacy document identity."
|
||||
);
|
||||
for (const path of [databasePath, secondaryDatabasePath]) {
|
||||
assertEqual(
|
||||
await readFile(join(path, notePath), "utf-8"),
|
||||
editedNoteContent,
|
||||
"Migration changed the content of the original NFD physical file."
|
||||
);
|
||||
await access(join(path, receivedNotePath)).then(
|
||||
() => {
|
||||
throw new Error("Migration created a second NFC physical file.");
|
||||
},
|
||||
(error: NodeJS.ErrnoException) => {
|
||||
if (error.code !== "ENOENT") throw error;
|
||||
}
|
||||
);
|
||||
}
|
||||
console.log("CLI Rebuild and Fetch migrated legacy document IDs while preserving both NFD physical files.");
|
||||
return { ...fileInfo, expectedCtime: legacyFileInfo.expectedCtime };
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const liveSyncCliCommand = resolveLiveSyncCliCommand();
|
||||
if (process.env.LIVESYNC_CLI_COMMAND === undefined) {
|
||||
@@ -304,12 +434,19 @@ async function main(): Promise<void> {
|
||||
const cliSourcePath = join(cliState, "source", "cli-to-obsidian.md");
|
||||
const vault = await createTemporaryVault();
|
||||
let session: ObsidianLiveSyncSession | undefined;
|
||||
let daemon: Awaited<ReturnType<typeof startLiveSyncCliDaemon>> | undefined;
|
||||
|
||||
try {
|
||||
await assertCouchDbReachable(couchDb);
|
||||
await createCouchDbDatabase(couchDb, dbName);
|
||||
await mkdir(cliDatabasePath, { recursive: true });
|
||||
await configureLiveSyncCli(liveSyncCliCommand, cliSettingsPath, couchDb, dbName);
|
||||
await configureLiveSyncCli(
|
||||
liveSyncCliCommand,
|
||||
cliSettingsPath,
|
||||
couchDb,
|
||||
dbName,
|
||||
migrateUnicodePaths ? 0 : undefined
|
||||
);
|
||||
|
||||
if (process.env.LIVESYNC_CLI_COMMAND === undefined) {
|
||||
console.log(`Using locally built LiveSync CLI: ${liveSyncCli}`);
|
||||
@@ -323,7 +460,21 @@ async function main(): Promise<void> {
|
||||
console.log(`Temporary Obsidian vault: ${vault.path}`);
|
||||
console.log(`Temporary CouchDB database: ${dbName}`);
|
||||
|
||||
const cliFileInfo = await writeCliNote(liveSyncCliCommand, cliDatabasePath, cliSettingsPath, cliSourcePath);
|
||||
let cliFileInfo = await writeCliNote(liveSyncCliCommand, cliDatabasePath, cliSettingsPath, cliSourcePath);
|
||||
if (migrateUnicodePaths) {
|
||||
cliFileInfo = await migrateCliDatabases(
|
||||
liveSyncCliCommand,
|
||||
cliState,
|
||||
cliDatabasePath,
|
||||
cliSettingsPath,
|
||||
couchDb,
|
||||
dbName,
|
||||
cliFileInfo
|
||||
);
|
||||
}
|
||||
if (unicodePaths && !useEncryption) {
|
||||
assertEqual(cliFileInfo.id, receivedNotePath.toLowerCase(), "The CLI did not derive an NFC document ID.");
|
||||
}
|
||||
await waitForCouchDbDocs(couchDb, dbName, (docs) => {
|
||||
const ids = new Set(docs.map((doc) => doc._id));
|
||||
return ids.has(cliFileInfo.id) && cliFileInfo.children.every((childId) => ids.has(childId));
|
||||
@@ -342,10 +493,11 @@ async function main(): Promise<void> {
|
||||
dbName,
|
||||
},
|
||||
{
|
||||
encrypt: true,
|
||||
encrypt: useEncryption,
|
||||
passphrase: e2eePassphrase,
|
||||
usePathObfuscation: true,
|
||||
encryptInternalMetadata: true,
|
||||
usePathObfuscation: useEncryption,
|
||||
encryptInternalMetadata: useEncryption,
|
||||
pathNormalizationVersion: unicodePaths ? 1 : 0,
|
||||
E2EEAlgorithm: "v2",
|
||||
}
|
||||
),
|
||||
@@ -355,7 +507,7 @@ async function main(): Promise<void> {
|
||||
await prepareRemote(obsidianCli.binary, session.cliEnv);
|
||||
await pushLocalChanges(obsidianCli.binary, session.cliEnv);
|
||||
|
||||
const received = await waitForVaultContent(vault.path, notePath);
|
||||
const received = await waitForVaultContent(vault.path, receivedNotePath);
|
||||
assertEqual(received, editedNoteContent, "The real Obsidian plug-in did not materialise the CLI-edited note.");
|
||||
if (cliFileInfo.expectedCtime !== undefined) {
|
||||
const receivedCtime = await evalObsidianJson<number>(
|
||||
@@ -363,7 +515,7 @@ async function main(): Promise<void> {
|
||||
[
|
||||
"(async()=>{",
|
||||
"const core=app.plugins.plugins['obsidian-livesync'].core;",
|
||||
`const entry=await core.localDatabase.getDBEntry(${JSON.stringify(notePath)},undefined,false,true);`,
|
||||
`const entry=await core.localDatabase.getDBEntry(${JSON.stringify(receivedNotePath)},undefined,false,true);`,
|
||||
"return JSON.stringify(entry.ctime);",
|
||||
"})()",
|
||||
].join(""),
|
||||
@@ -376,12 +528,78 @@ async function main(): Promise<void> {
|
||||
);
|
||||
console.log("The CLI edit retained the filesystem creation time in the real Obsidian client's Metadata.");
|
||||
} else {
|
||||
console.log("Creation-time verification skipped: native stats did not expose distinct creation and change times.");
|
||||
console.log(
|
||||
"Creation-time verification skipped: native stats did not expose distinct creation and change times."
|
||||
);
|
||||
}
|
||||
console.log(
|
||||
"CLI-edited encrypted note was retrieved by the real Obsidian plug-in with identical content."
|
||||
);
|
||||
if (unicodePaths) {
|
||||
const returnedContent = `${editedNoteContent}\nUpdated in Obsidian.\n`;
|
||||
await evalObsidianJson(
|
||||
obsidianCli.binary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const path=${JSON.stringify(receivedNotePath)};`,
|
||||
"const file=app.vault.getAbstractFileByPath(path);",
|
||||
"await new Promise(resolve=>setTimeout(resolve,Math.max(0,file.stat.mtime+2500-Date.now())));",
|
||||
`await app.vault.modify(file,${JSON.stringify(returnedContent)});`,
|
||||
"return JSON.stringify(true);})()",
|
||||
].join(""),
|
||||
session.cliEnv
|
||||
);
|
||||
await pushLocalChanges(obsidianCli.binary, session.cliEnv);
|
||||
await runLiveSyncCli(liveSyncCliCommand, [cliDatabasePath, "--settings", cliSettingsPath, "sync"]);
|
||||
await runLiveSyncCli(liveSyncCliCommand, [cliDatabasePath, "--settings", cliSettingsPath, "mirror"]);
|
||||
assertEqual(
|
||||
await readFile(join(cliDatabasePath, notePath), "utf-8"),
|
||||
returnedContent,
|
||||
"The Obsidian update did not reach the original NFD physical file."
|
||||
);
|
||||
await access(join(cliDatabasePath, receivedNotePath)).then(
|
||||
() => {
|
||||
throw new Error("The CLI created a second NFC physical file.");
|
||||
},
|
||||
(error: NodeJS.ErrnoException) => {
|
||||
if (error.code !== "ENOENT") throw error;
|
||||
}
|
||||
);
|
||||
// The mirror command intentionally keeps tombstoned local files. The daemon applies remote deletions.
|
||||
daemon = await startLiveSyncCliDaemon(liveSyncCliCommand, cliDatabasePath, cliSettingsPath);
|
||||
await evalObsidianJson(
|
||||
obsidianCli.binary,
|
||||
[
|
||||
"(async()=>{",
|
||||
`const file=app.vault.getAbstractFileByPath(${JSON.stringify(receivedNotePath)});`,
|
||||
"await new Promise(resolve=>setTimeout(resolve,Math.max(0,file.stat.mtime+2500-Date.now())));",
|
||||
"await app.vault.delete(file);",
|
||||
"return JSON.stringify(true);})()",
|
||||
].join(""),
|
||||
session.cliEnv
|
||||
);
|
||||
await pushLocalChanges(obsidianCli.binary, session.cliEnv);
|
||||
const deletionDeadline = Date.now() + Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS);
|
||||
for (;;) {
|
||||
try {
|
||||
await access(join(cliDatabasePath, notePath));
|
||||
} catch (error) {
|
||||
if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error;
|
||||
break;
|
||||
}
|
||||
if (Date.now() >= deletionDeadline) {
|
||||
throw new Error(
|
||||
`The Obsidian deletion did not remove the original NFD physical file.\n${daemon.getOutput()}`
|
||||
);
|
||||
}
|
||||
await new Promise((resolvePromise) => setTimeout(resolvePromise, 100));
|
||||
}
|
||||
await daemon.stop();
|
||||
daemon = undefined;
|
||||
console.log(
|
||||
"NFC path identity preserved the NFD physical name through creation, remote update, and deletion."
|
||||
);
|
||||
}
|
||||
console.log("CLI-edited note was retrieved by the real Obsidian plug-in with identical content.");
|
||||
} finally {
|
||||
await daemon?.stop();
|
||||
if (session) {
|
||||
await session.app.stop();
|
||||
}
|
||||
|
||||
@@ -14,6 +14,12 @@ Earlier releases remain available in the 1.0 release history, the 1.0 preview hi
|
||||
|
||||
### Synchronisation
|
||||
|
||||
#### Added
|
||||
|
||||
- We can opt into NFC path identity for CouchDB synchronisation between the CLI and Obsidian. This prevents decomposed or mixed Unicode file names from becoming inaccessible through a receiving device's normalised path. (#1259)
|
||||
- Existing Vaults retain their current identity rules. Migration requires updated clients, a Rebuild from a complete authoritative Vault, and a Fetch on the other devices.
|
||||
- Reconstruction stops when the supported host file listing or local Metadata exposes ambiguous files or parent folders under NFC and the configured case handling. The CLI retains existing physical file names. P2P and Object Storage do not support the new mode.
|
||||
|
||||
#### Fixed
|
||||
|
||||
- We can synchronise file deletions and renames when **Do not keep metadata of deleted files.** is enabled.
|
||||
|
||||
Reference in new issue
Block a user