Add opt-in Unicode NFC path identity for CouchDB

This commit is contained in:
vorotamoroz committed 2026-10-17 13:31:19 +00:00
1 parent 72dc88c7ef
commit ddb4d52931
39 files changed
+2245 -97

No files matched your search

+135
View File
@@ -0,0 +1,135 @@
---
date: 2026-10-17
commonlib-version: "0.1.42"
self-hosted-livesync-version: "1.0.37"
status: unreleased
---
# Architectural Decision Record: Unicode Path Identity for CouchDB
## Status
Accepted on 2026-10-17. The feature remains unreleased. Local validation
evidence and platform limits are recorded in the design document.
## Context
Self-hosted LiveSync derives normal-file Metadata document IDs from Vault paths.
Unicode permits visually identical text to use different code-point sequences.
For example, `é` can be represented as one precomposed character or as `e`
followed by a combining acute accent. Filesystems and applications can expose
these forms differently. Without a shared canonical form, two clients can
derive different document IDs for the same logical path, or fail to resolve an
existing document when another client reports an equivalent spelling.
Changing only a lookup performed when receiving a document is insufficient.
New documents, edits, deletions, local provenance, and resumable Fetch
checkpoints must all use the same path identity. Applying the change to a
populated database in place would
also leave old document IDs and revision histories behind. Distinct files or
Metadata documents can collapse to one identity under NFC, so automatic
selection or merging could lose user data.
The Metadata path and file content have separate roles: Metadata stores file
properties and references to Chunks, while Chunks store the content. A path
identity change must not require physical files to be renamed or content to be
rewritten in Metadata.
## Decision
Add the opt-in `pathNormalizationVersion` setting. An absent value and `0`
retain the existing path-to-ID behaviour. Version `1` applies Unicode
Normalization Form C (NFC) to the logical path used for Metadata IDs and path
comparison keys. Existing case handling remains an independent rule; CLI path
resolution uses NFC equivalence regardless of the `handleFilenameCaseSensitive`
setting. NFC changes logical identity only: it does not rewrite the stored
`Metadata.path`, rename a physical file, or change Chunk IDs or content. It is
independent of `idDerivationVersion` and does not require encryption.
When the CLI resolves a path, it uses the actual spelling of each existing
file and directory. A missing component is created under the actual resolved
parent using the spelling received for that component. Obsidian resolves paths
through its `TFile` and Vault APIs. Its collision checks are limited to the
paths those APIs expose and do not promise to find raw physical aliases hidden
by the Obsidian index.
The initial support boundary is CLI and Obsidian clients using CouchDB.
Journal and P2P reject version `1`; Direct file access refuses a remote which
declares the NFC feature. Other host adapters are not supported for version `1`
until they provide equivalent path resolution and ambiguity handling within
their supported storage listing.
Version `1` is represented by the required `nfc-path-identity-v1` feature in
the existing generation-13 database version document. Every client must agree
on the path identity before it can write Metadata. A mode `0` client cannot
join a database requiring this feature. An existing database without the
feature cannot acquire it through an ordinary Metadata write while it already
contains Metadata. These admission checks remain active when ordinary
configuration-mismatch warnings are disabled.
`pathNormalizationVersion` is a required Tweak even when encryption is
disabled. It does not replace or modify `idDerivationVersion`.
The supported migration is reconstruction, not an in-place ID rewrite:
1. Update every participating client and stop synchronisation.
2. Select a complete, authoritative Vault and resolve NFC-equivalent files or
Metadata documents which would produce the same ID.
3. Set version `1` on the source client and Rebuild both databases from that
Vault.
4. Set version `1` on each remaining client and Fetch the rebuilt remote.
5. Resume synchronisation only after every client has the same path identity.
In Obsidian, the setting remains pending through ordinary settings saves, the
settings-only Apply fallback, and reload. It becomes active only when the
existing Rebuild or Fetch operation is successfully reserved. A `remoteOnly`
operation cannot apply a pending identity change, and the settings-only
fallback is not offered. The Setup URI shares
`pathNormalizationVersion` so a new client receives the intended value;
importing that value does not replace the required Rebuild or Fetch operation.
The CLI exposes explicit `rebuild` and `fetch` commands. Both require
`--force` before application start-up because they replace local or remote
database state. `--force` confirms the requested operation; it does not bypass
identity checks, collision preflight, compatibility checks, or required human
remediation. The commands use the existing Commonlib Rebuilder; this change
does not introduce a second migration engine.
The migration procedure also uses `--write-settings` to retain the selected
local database and settings across subsequent CLI invocations. Collision
preflight uses the supported host listing and local Metadata. It refuses
ambiguous file or parent-directory candidates before database reconstruction;
it does not claim to detect physical aliases which a host does not expose.
## Alternatives considered
- **Use NFC by default:** rejected because existing clients and databases must
retain their current IDs until their owners deliberately migrate them.
- **Normalise only lookups:** rejected because new Metadata writes and deletes
could continue using legacy IDs and create duplicate documents.
- **Change `idDerivationVersion`:** rejected because it controls cryptographic
ID derivation, not Unicode canonicalisation, and must remain meaningful when
encryption is disabled.
- **Rename files to NFC:** rejected because the decision concerns logical
identity, while physical spelling belongs to the Vault and its filesystem.
- **Rewrite remote IDs automatically or merge collisions:** rejected because
changing document IDs cannot preserve CouchDB revision ancestry, and the
system cannot choose safely between distinct file histories.
## Consequences
Legacy mode remains unchanged, and NFC identity requires a coordinated,
destructive Rebuild and Fetch. An ambiguity found in the supported storage
listing or local Metadata stops reconstruction before either database is
reset; the user must resolve the names or synchronisation scope. The check is
bounded by each host's listing contract: Obsidian uses its `TFile` and Vault
APIs and does not scan the physical Vault for aliases hidden by that index.
No claim is made that raw physical aliases outside a host's supported listing
are detected.
## References
- [Unicode path identity design](../design_docs/unicode_path_identity.md)
- [Setting details](../settings.md#unicode-path-identity)
- [Commonlib Unicode path identity contract](https://github.com/vrtmrz/livesync-commonlib/blob/main/docs/unicode-path-identity.md)
- [Project glossary](../glossary.md)
+263
View File
@@ -0,0 +1,263 @@
---
date: 2026-10-17
commonlib-version: "0.1.42"
self-hosted-livesync-version: "1.0.37"
status: unreleased
---
# Unicode path identity
## Purpose and baseline
This design defines how Self-hosted LiveSync opts into Unicode NFC for normal
file Metadata document IDs, how hosts preserve the Vault's physical names, and
how existing databases move to the new identity. The accepted rationale is in
the [Unicode path identity ADR](../adr/2026_10_unicode_path_identity.md).
Commonlib's [consumer contract](https://github.com/vrtmrz/livesync-commonlib/blob/main/docs/unicode-path-identity.md)
defines the shared package boundary.
This records an unreleased change to Self-hosted LiveSync `1.0.37`, using the
published Commonlib `0.1.42` package as an exact dependency.
## Scope
| Area | Contract |
| --- | --- |
| Logical identity | Version `1` uses NFC for Metadata ID and path comparison keys. Existing case handling remains a separate rule. Missing and `0` retain legacy behaviour. |
| Stored paths and content | Keep the spelling in `Metadata.path` and existing physical Vault names. NFC does not change Chunk IDs or content. |
| Initial hosts and remote | Support CLI and Obsidian with CouchDB. |
| Other transports | Journal and P2P reject version `1`; Direct file access refuses a remote declaring the NFC feature. Other host adapters remain outside the support matrix. |
| Migration | Rebuild both databases from one complete authoritative Vault, then Fetch each remaining client. Do not rewrite IDs or merge histories in place. |
## Path identity contract
`pathNormalizationVersion` selects the logical path used by Commonlib's
`PathService` for normal-file Metadata IDs and path comparison keys:
| Value | Path identity |
| --- | --- |
| Missing or `0` | Preserve the legacy path and ID behaviour exactly. |
| `1` | Apply Unicode NFC to the logical path, then apply the existing case handling and ID derivation rules. |
| Any other value | Reject it; do not fall back to legacy mode. |
NFC changes only logical identity and comparison keys. It is not a physical
filename conversion, a rewrite of `Metadata.path`, or a Unicode compatibility
transformation such as NFKC. Existing case handling remains governed by
`handleFilenameCaseSensitive`; it is independent of NFC equivalence resolution
in the CLI. `idDerivationVersion` continues to select its separate
cryptographic ID construction. NFC path identity works with or without
encryption and does not change Chunk IDs or content.
The version is a required Tweak, even when encryption is disabled, represented
by `nfc-path-identity-v1` in the existing generation-13 database version
document's `used_features` list. The Setup URI includes
`pathNormalizationVersion`, decodes an absent value as `0`, and rejects
unsupported values. Sharing the setting communicates the target identity;
setup still requires an explicit Rebuild or Fetch choice.
Fast Fetch checkpoints include the path identity version. A checkpoint created
under legacy identity cannot be resumed under version `1`, or vice versa. This
reuses the existing checkpoint identity and does not add a separate recovery
store.
## Admission and database compatibility
Clients validate local and remote database identity before accepting ordinary
replication, including when configuration-mismatch warnings are disabled. The
checks cover both directions:
| Client setting | Database feature | Result |
| --- | --- | --- |
| Missing or `0` | No NFC feature | Continue with legacy identity. |
| `1` | NFC feature present | Continue with NFC identity. |
| `1` | Neither database has the feature or Metadata | A fresh database may declare the feature before its first Metadata mutation. |
| `1` | A database has Metadata without the feature | Reject ordinary writes; require Rebuild. |
| Missing or `0` | A local or remote database has the NFC feature | Reject; the database requires a version-`1` client. |
The feature is independent of encryption. Receiving an incompatible version
document records the incompatibility and stops the active synchronisation
session. This is a compatibility fence, not a rollback of operations already
admitted elsewhere; every device must be stopped before migration begins.
## Host path resolution and collision preflight
Commonlib owns logical ID derivation and the Rebuilder's preflight contract.
Each host maps that logical path to entries in its supported storage listing.
The CLI resolves existing files and directories using NFC comparison keys,
regardless of `handleFilenameCaseSensitive`. It carries the actual spelling
reported for each existing component through reads, writes, deletes, stat,
directory creation, and rename operations. If a path has missing components,
the CLI uses the actual existing parent and preserves the received spelling for
each new component. A scan enumerates each directory once and stats the raw
paths returned by that enumeration; it does not repeat canonical lookup for
each listed entry. No persistent index or recovery store is added.
If more than one entry resolves to the same logical comparison key after NFC
and the configured case rule at any component, including a parent directory,
the path is ambiguous. The CLI warns and refuses the operation; it must not
select a candidate based on directory enumeration order, merge the entries, or
rename them. The same rule applies during Rebuild preflight.
Obsidian resolution remains within its `TFile` and Vault API boundary. Its
collision preflight checks paths exposed through those APIs; it does not walk
the physical Vault independently and cannot promise to detect raw filesystem
aliases hidden by Obsidian's index. Other host adapters remain outside the
version-`1` support matrix until their path resolution and ambiguity handling
are implemented and verified within their own supported listings.
Before resetting either database, Rebuild checks the supported storage
listing and local Metadata for paths which would resolve to the same identity
under NFC and the configured case rule. When a local database is not open,
preflight checks the storage paths returned by the host listing. If multiple
candidate files or Metadata documents collide, or a parent component is
ambiguous, the operation stops before local or remote mutation. It does not
choose a winner, merge revision histories, or rename a Vault file. The user
must resolve the ambiguous names or exclude one file from the synchronisation
scope, then retry the reconstruction. This check is bounded by the host's
listing contract and does not imply that every raw physical alias is visible.
## Reconstruction and client operations
Mode changes are never applied by an ordinary Metadata write. In Obsidian, the
Advanced page offers **Legacy path identity** (`0`) and **Unicode NFC path
identity** (`1`) for CouchDB. A changed `pathNormalizationVersion` remains
pending through ordinary settings saves, the settings-only Apply fallback, and
reload. It becomes active only after the existing Rebuild or Fetch operation
has been successfully reserved; cancellation or failed reservation leaves it
pending. A `remoteOnly` operation cannot apply a pending identity change, and
the settings-only fallback is not offered.
Migration uses this ordered procedure:
1. Update every CLI and Obsidian client which will use the database. Stop
synchronisation and editing on all devices, and keep a complete backup.
2. Select the Vault whose complete file set is authoritative. Resolve any
collision reported by Rebuild before continuing.
3. Select version `1` on that source. In Obsidian, choose **Overwrite Server
Data with This Device's Files**; the new mode becomes active only when the
Rebuild reservation succeeds. In the CLI, use settings with
`pathNormalizationVersion: 1` and run `rebuild` with `--force
--write-settings`. Both local and remote databases are reconstructed from
the specified physical Vault.
4. Select version `1` on each remaining client. In Obsidian, choose **Reset
Synchronisation on This Device**; the new mode becomes active only when the
Fetch reservation succeeds. In the CLI, use settings with
`pathNormalizationVersion: 1` and run `fetch` with `--force
--write-settings`. Each operation replaces that client's local database
from the rebuilt remote.
5. Confirm that every participating client is using version `1`, then resume
synchronisation.
The CLI checks `--force` before core start-up so an omitted confirmation cannot
trigger database access or destructive preparation. Neither `--force` nor a
headless default choice overrides an NFC collision, a feature mismatch, or a
preflight failure. If a required setup decision cannot be made in a headless
CLI invocation, the command must fail with a clear remediation message rather
than report success after cancellation.
`--write-settings` retains the Rebuilder's selected local database name and
lasting settings changes for subsequent invocations. Without this option, the
existing CLI policy leaves the settings file unchanged, so a later command can
reopen the previously selected database.
## State which follows the selected identity
The path identity is also used by two existing resume mechanisms:
- **File-reflection provenance:** the existing device-local provenance storage
interface remains an exact-key store. The private
`ServiceFileHandlerBase.getProvenancePath()` method canonicalises provenance
keys with NFC in mode `1` for reads, writes, deletes, raw snapshot lookup,
and untracked revision recording. Mode `0` retains the old keys. Rebuild and
Fetch use the existing local-state reconstruction and replacement lifecycle;
no new persistent field or in-place key migration is introduced.
- **Fast Fetch checkpoints:** the path version participates in checkpoint
identity so an interrupted transfer cannot resume using a different
path-to-ID rule. Existing checkpoint persistence and retry handling remain
in control.
These rules keep a physical NFD filename addressable when an NFC-equivalent
Metadata path arrives, while using one canonical identity for subsequent
updates and deletions.
## Compatibility matrix
| Client and remote | Version `0` | Version `1` |
| --- | --- | --- |
| CLI with CouchDB | Supported with legacy identity. | Supported with NFC path resolution. |
| Obsidian with CouchDB | Supported with legacy identity. | Supported with NFC path resolution. |
| Journal or P2P | Existing behaviour. | Rejected. |
| Direct file access | Existing behaviour. | Refuses a remote requiring `nfc-path-identity-v1`. |
| Other host adapters, including WebApp | Existing behaviour, subject to their existing support. | Outside the initial support matrix until physical-path handling is verified. |
| Older client without this feature | Can use a legacy database. | Cannot join a database declaring `nfc-path-identity-v1`. |
## Verification contract and limits
The implementation acceptance matrix covers:
- unchanged legacy IDs for missing and explicit version `0`, plus matching
ASCII IDs in both modes;
- identical normal-file Metadata IDs for NFC and NFD spellings in version `1`,
with and without path obfuscation, and with the existing case rule applied;
- Metadata creation, update, deletion, and file-reflection provenance using the
same canonical identity without renaming the physical file;
- CLI reads, writes, deletes, stat, directory creation, and rename through the
actual spelling of existing files and parents; preservation of the received
spelling for new components; and resolution independent of the case setting;
- refusal to write when an existing legacy database has Metadata, refusal of
version-`0` clients against an NFC database, and enforcement when ordinary
mismatch warnings are disabled;
- URI round trips, checkpoint invalidation across path versions, and transport
rejection for Journal, P2P, and Direct file access;
- refusal for ambiguous file or parent-directory candidates before local or
remote reset, preserving both databases on rejection, within the paths
returned by each host's supported listing; and
- CLI `rebuild` and `fetch` command dispatch, pre-start `--force` enforcement,
and end-to-end creation, update, and deletion through CouchDB between CLI and
Obsidian.
Focused unit and package checks do not substitute for the real CouchDB and
Obsidian round trip using the exact published Commonlib artefact. Obsidian checks
are limited to paths exposed through its `TFile` and Vault APIs; they do not
establish detection of physical aliases hidden by the Obsidian index. No result
for raw physical filesystem aliases outside a host's supported listing is
claimed.
Local validation on Linux with real Obsidian `1.12.7` and the published
Commonlib `0.1.42` package passed after the CLI path resolution, case-aware
collision scan, and pending-setting changes. It covered:
- NFC identity with unencrypted Metadata;
- NFC identity with E2EE and Path Obfuscation;
- two legacy CLI databases migrated through `rebuild` and `fetch`, followed by
the encrypted Obsidian round trip; and
- the existing encrypted ASCII scenario in legacy mode.
In those NFC scenarios, an NFD directory and mixed-form filename were retained
while checking content, Metadata creation time, remote updates, absence of an
NFC duplicate, and remote deletion through the polling CLI daemon. The `mirror`
command retains tombstoned local files by default; its update behaviour was
tested separately from daemon deletion reflection.
## Non-goals
This design does not:
- change Chunk IDs, E2EE, Properties encryption, or the Metadata and Chunk
document formats;
- convert or rename existing physical files;
- migrate remote document IDs or revision histories in place;
- choose among ambiguous NFC-equivalent files or parents, or merge their
content;
- change the default away from legacy path identity;
- enable version `1` for Journal, P2P, Direct file access, WebApp, or an
unverified host adapter; or
- claim native iOS or macOS filesystem compatibility.
## References
- [Unicode path identity ADR](../adr/2026_10_unicode_path_identity.md)
- [Setting details](../settings.md#unicode-path-identity)
- [Commonlib Unicode path identity contract](https://github.com/vrtmrz/livesync-commonlib/blob/main/docs/unicode-path-identity.md)
- [Metadata and Chunks glossary](../glossary.md#user-facing-and-operational-terms)
+12
View File
@@ -1050,6 +1050,18 @@ When this setting is disabled, changing only the letter case of a file name with
New Vaults use case-insensitive handling for cross-platform compatibility. Existing settings with an explicit value preserve that choice. Earlier releases also followed the case-insensitive branch when this value was absent, so 1.0 saves a missing legacy value as `false` without requiring a compatibility review or database rebuild.
#### Unicode path identity
Setting key: pathNormalizationVersion
Version `0` preserves legacy file identity and is the default for existing and new settings. Version `1` uses Unicode NFC when deriving document IDs, so equivalent composed and decomposed paths identify the same file across the CLI and Obsidian. Existing physical file names are retained. This mode currently supports CouchDB only.
Before enabling version `1`, update every participating client, stop synchronisation, and back up a complete authoritative Vault. Rebuild both databases from that Vault, then select version `1` and Fetch on each remaining device. Changing this setting without reconstruction is refused when it would mix database identities. Do not resume an older client against the migrated remote.
The selected value stays pending through ordinary settings saves and reloads. It becomes active only when the Rebuild or Fetch operation is successfully reserved; cancellation leaves it pending.
Distinct files or Metadata documents which derive the same identity stop reconstruction before either database is reset. This check uses the host's supported file listing; Obsidian checks the paths exposed by its Vault APIs and cannot detect physical aliases hidden by its index. Resolve the duplicate names or synchronisation scope first; Self-hosted LiveSync does not merge them automatically. Case-insensitive handling also applies the existing letter-case rule.
### 4. Compatibility (Internal API Usage)
#### Scan changes on customisation sync
+4 -4
View File
@@ -23,7 +23,7 @@
"@smithy/types": "^4.14.3",
"@smithy/util-retry": "^4.4.5",
"@vrtmrz/browser-ui-kit": "0.1.0",
"@vrtmrz/livesync-commonlib": "0.1.40",
"@vrtmrz/livesync-commonlib": "0.1.42",
"@vrtmrz/obsidian-plugin-kit": "0.1.4",
"@vrtmrz/ui-interactions": "0.1.2",
"diff-match-patch": "^1.0.5",
@@ -4389,9 +4389,9 @@
}
},
"node_modules/@vrtmrz/livesync-commonlib": {
"version": "0.1.40",
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.40.tgz",
"integrity": "sha512-GVskcVFoQIpb45rR+QT9Wjtdqc6oAxYz5IB2WeJwusWZ8lTmqVrba+6YD9FTtsbzqbPOyJ/LdQxnET7SUexVtw==",
"version": "0.1.42",
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.42.tgz",
"integrity": "sha512-LyWl0Mkrj9UsQXV0nOUaYJybcN3wLkpAAxCeHMZYOEizaUTQ7ezN+FgzN+OQxDRA8UXWnoatxh719NWzOwBpBw==",
"license": "MIT",
"dependencies": {
"@aws-sdk/client-s3": "^3.808.0",
+1 -1
View File
@@ -191,7 +191,7 @@
"@smithy/types": "^4.14.3",
"@smithy/util-retry": "^4.4.5",
"@vrtmrz/browser-ui-kit": "0.1.0",
"@vrtmrz/livesync-commonlib": "0.1.40",
"@vrtmrz/livesync-commonlib": "0.1.42",
"@vrtmrz/obsidian-plugin-kit": "0.1.4",
"@vrtmrz/ui-interactions": "0.1.2",
"diff-match-patch": "^1.0.5",
+45 -3
View File
@@ -64,9 +64,10 @@ livesync-cli [database-path] [command] [args...]
- `database-path`: Path to the directory where `.livesync` folder and `settings.json` are (or will be) located.
- Note: In previous versions, this was referred to as the "vault" path. Now it is clearly distinguished from the actual vault (the directory containing your `.md` files).
- `--vault <path>` / `-V <path>`: (daemon/mirror only) Path to the vault directory containing `.md` files.
- `--vault <path>` / `-V <path>`: (daemon/mirror/rebuild/fetch) Path to the Vault directory containing `.md` files.
- Allows the PouchDB database directory and the actual vault directory to be different locations.
- For `mirror` command, the positional `[vault-path]` argument takes precedence over `--vault`.
- `--force` / `-f`: Required by `rebuild` and `fetch` because they replace a database. It also retains its existing `init-settings` overwrite behaviour.
- `--write-settings`: Write setting migrations and other lasting changes after the command succeeds.
- `init-settings` writes its target file. `setup`, `remote-add`, `remote-rm`, `remote-set`, and `remote-activate` write their settings changes without this option.
- All remaining commands leave the settings file unchanged by default.
@@ -82,6 +83,9 @@ livesync-cli [database-path] [command] [args...]
- If `vault-path` is provided, the CLI will synchronise the database with files in the vault directory.
- If `vault-path` is omitted, it defaults to `database-path` (compatibility mode).
- Use this command to keep your local `.md` files in sync with the database.
- `rebuild`: Rebuild the local and remote databases from the selected Vault. This replaces existing local and remote database state and requires `--force`.
- This command may select a new local database suffix. Add `--write-settings` if later CLI commands should use the resulting database selection; otherwise the existing settings file remains unchanged.
- `fetch`: Replace the local database with the configured remote state, then reflect fetched files to the selected Vault. Existing local database state is discarded, so this command requires `--force`.
- `ls [prefix]`: List files currently stored in the local database.
- `push <src> <dst>`: Push a local file `<src>` into the database at path `<dst>`.
- `pull <src> <dst>`: Pull a file `<src>` from the database into local file `<dst>`.
@@ -110,6 +114,12 @@ livesync-cli ./my-db sync
# Mirroring to your actual Obsidian vault
livesync-cli ./my-db mirror /path/to/obsidian-vault
# Rebuild local and remote databases from an authoritative Vault
livesync-cli ./my-db --vault /path/to/obsidian-vault rebuild --force --write-settings
# Replace the local database and Vault files from the configured remote
livesync-cli ./my-db --vault /path/to/obsidian-vault fetch --force --write-settings
# Manual file operations
livesync-cli ./my-db push ./note.md folder/note.md
livesync-cli ./my-db pull folder/note.md ./note.md
@@ -369,6 +379,36 @@ The CLI uses the same settings format as the Obsidian plug-in. Create a `.livesy
- `couchDB_DBNAME`: Database name
- `isConfigured`: Set to `true` after configuration
#### Unicode path identity
The `pathNormalizationVersion` setting controls how file paths identify
Metadata documents when using CouchDB. An absent value or `0` keeps the legacy
identity. Version `1` uses Unicode Normalization Form C (NFC) for Metadata IDs
and path comparison keys. It does not rewrite `Metadata.path`, rename existing
files, or change Chunks or file content. `handleFilenameCaseSensitive` remains
an independent case-handling setting; the CLI resolves NFC-equivalent names
even when case-sensitive handling is enabled.
When the CLI resolves an existing path, it uses the actual spelling of each
matching file or directory for reads, writes, deletes, `stat` calls, directory
creation, and renames. If a path component is missing, it uses the actual
existing parent and preserves the spelling received for the new component. If
a file or any parent directory has multiple candidates for the same logical
key after NFC and the existing case rule, the CLI warns and refuses the
operation. It does not choose a candidate based on directory order, merge
entries, or rename them.
Version `1` requires a coordinated Rebuild and Fetch; changing the setting by
itself does not migrate a database. Update every participating client, stop
synchronisation, and back up a complete authoritative Vault. Set
`pathNormalizationVersion` to `1` on the source client, then run `rebuild`
with `--force --write-settings`. Set the same value on each remaining CLI
client, then run `fetch` with `--force --write-settings`. Use **Overwrite
Server Data with This Device's Files** and **Reset Synchronisation on This
Device** for the corresponding Obsidian Rebuild and Fetch operations. Resolve
any ambiguous names before reconstruction. Version `1` is supported with
CouchDB only; do not resume a legacy client against the rebuilt database.
### Command-line Reference
```
@@ -381,11 +421,11 @@ Arguments:
Options:
--settings, -s <path> Path to settings file (default: .livesync/settings.json in local database directory)
--force, -f Overwrite existing file on init-settings
--force, -f Required by rebuild/fetch; overwrite existing file on init-settings
--verbose, -v Enable verbose logging
--debug, -d Enable debug logging (includes verbose)
--interval <N>, -i <N> (daemon only) Poll CouchDB every N seconds instead of using the _changes feed
--vault <path>, -V <path> (daemon/mirror) Path to vault directory, decoupled from database-path
--vault <path>, -V <path> (daemon/mirror/rebuild/fetch) Path to vault directory, decoupled from database-path
--write-settings Write setting changes after a successful command
--compat-remote-admin-exit-zero
Preserve the former zero exit code when remote-administration verification fails
@@ -398,6 +438,8 @@ Commands:
p2p-peers <timeout> Show discovered peers as [peer]<TAB><peer-id><TAB><peer-name>
p2p-sync <peer> <timeout> Synchronise with specified peer-id or peer-name
p2p-host Start P2P host mode and wait until interrupted (Ctrl+C)
rebuild Rebuild local and remote databases from the selected Vault (requires --force)
fetch Replace the local database with remote state and reflect it to the Vault (requires --force)
push <src> <dst> Push local file <src> into local database path <dst>
pull <src> <dst> Pull file <src> from local database into local file <dst>
pull-rev <src> <dst> <rev> Pull specific revision <rev> into local file <dst>
+56 -15
View File
@@ -3,7 +3,7 @@ import type { IFileSystemAdapter } from "@vrtmrz/livesync-commonlib/compat/servi
import { NodePathAdapter } from "./NodePathAdapter";
import { NodeTypeGuardAdapter } from "./NodeTypeGuardAdapter";
import { NodeConversionAdapter } from "./NodeConversionAdapter";
import { NodeStorageAdapter } from "@vrtmrz/livesync-commonlib/node";
import { AmbiguousNormalisedPathError, UnicodeResolvingNodeStorageAdapter } from "./UnicodeResolvingNodeStorageAdapter";
import { NodeVaultAdapter } from "./NodeVaultAdapter";
import type { NodeFile, NodeFolder, NodeStat } from "./NodeTypes";
import { path } from "@vrtmrz/livesync-commonlib/node";
@@ -16,19 +16,21 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
readonly path: NodePathAdapter;
readonly typeGuard: NodeTypeGuardAdapter;
readonly conversion: NodeConversionAdapter;
readonly storage: NodeStorageAdapter;
readonly storage: UnicodeResolvingNodeStorageAdapter;
readonly vault: NodeVaultAdapter;
private fileCache = new Map<string, NodeFile>();
constructor(
private basePath: string,
private reportDiagnostic: CliDiagnosticReporter = () => undefined
private reportDiagnostic: CliDiagnosticReporter = () => undefined,
getPathNormalizationVersion: () => number = () => 0,
isCaseInsensitive: () => boolean = () => false
) {
this.path = new NodePathAdapter();
this.typeGuard = new NodeTypeGuardAdapter();
this.conversion = new NodeConversionAdapter();
this.storage = new NodeStorageAdapter(basePath);
this.storage = new UnicodeResolvingNodeStorageAdapter(basePath, getPathNormalizationVersion, isCaseInsensitive);
this.vault = new NodeVaultAdapter(this.storage);
}
@@ -58,6 +60,14 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
async getAbstractFileByPath(p: FilePath | string): Promise<NodeFile | null> {
const pathStr = this.normalisePath(p);
if (this.storage.isUnicodePathNormalizationEnabled) {
const resolvedPath = await this.storage.resolveLogicalPath(pathStr, false, false);
if (resolvedPath === null) {
this.fileCache.delete(pathStr);
return null;
}
return await this.refreshResolvedFile(resolvedPath);
}
if (!this.fileCache.has(pathStr) && !(await this.hasExactPathCase(pathStr))) {
this.fileCache.delete(pathStr);
return null;
@@ -68,6 +78,15 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
async getAbstractFileByPathInsensitive(p: FilePath | string): Promise<NodeFile | null> {
const pathStr = this.normalisePath(p);
if (this.storage.isUnicodePathNormalizationEnabled) {
const resolvedPath = await this.storage.resolveLogicalPath(pathStr, true, false);
if (resolvedPath === null) {
this.fileCache.delete(pathStr);
return null;
}
return await this.refreshResolvedFile(resolvedPath);
}
const exact = await this.getAbstractFileByPath(pathStr);
if (exact) {
return exact;
@@ -115,17 +134,34 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
// This is used by Obsidian to sync internal file metadata
}
async refreshFile(p: string): Promise<NodeFile | null> {
async refreshFile(p: string, caseInsensitive = false): Promise<NodeFile | null> {
const pathStr = this.normalisePath(p);
try {
const stat = await this.storage.stat(pathStr);
if (stat?.type !== "file") {
const resolvedPath = this.storage.isUnicodePathNormalizationEnabled
? await this.storage.resolveLogicalPath(pathStr, caseInsensitive, false)
: pathStr;
if (resolvedPath === null) {
this.fileCache.delete(pathStr);
return null;
}
return await this.refreshResolvedFile(resolvedPath);
} catch (error) {
if (error instanceof AmbiguousNormalisedPathError) throw error;
this.fileCache.delete(pathStr);
return null;
}
}
private async refreshResolvedFile(resolvedPath: string): Promise<NodeFile | null> {
try {
const stat = await this.storage.statExact(resolvedPath);
if (stat?.type !== "file") {
this.fileCache.delete(resolvedPath);
return null;
}
const file: NodeFile = {
path: pathStr as FilePath,
path: resolvedPath as FilePath,
stat: {
size: stat.size,
mtime: stat.mtime,
@@ -133,11 +169,12 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
type: "file",
},
};
this.fileCache.set(pathStr, file);
this.fileCache.set(resolvedPath, file);
return file;
} catch {
} catch (error) {
if (error instanceof AmbiguousNormalisedPathError) throw error;
// Evict so a deleted file is not returned by subsequent cache scans.
this.fileCache.delete(pathStr);
this.fileCache.delete(resolvedPath);
return null;
}
}
@@ -146,18 +183,21 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
* Helper method to recursively scan directory and populate file cache
*/
async scanDirectory(relativePath: string = ""): Promise<void> {
await this.scanDirectoryInto(relativePath, this.fileCache);
const resolvedPath = await this.storage.resolveLogicalPath(relativePath, false, false);
if (resolvedPath !== null) await this.scanDirectoryInto(resolvedPath, this.fileCache);
}
private async scanDirectoryInto(relativePath: string, files: Map<string, NodeFile>): Promise<void> {
const fullPath = this.resolvePath(relativePath);
try {
const directoryStat = await this.storage.stat(relativePath);
const directoryStat = this.storage.isUnicodePathNormalizationEnabled
? await this.storage.statExact(relativePath)
: await this.storage.stat(relativePath, false);
if (directoryStat?.type !== "folder") throw new Error(`Directory does not exist: ${fullPath}`);
const entries = await this.storage.list(relativePath);
const entries = await this.storage.listExact(relativePath);
for (const entryPath of entries.files) {
const stat = await this.storage.stat(entryPath);
const stat = await this.storage.statExact(entryPath);
if (stat?.type !== "file") continue;
const file: NodeFile = {
path: entryPath as FilePath,
@@ -169,6 +209,7 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
await this.scanDirectoryInto(entryPath, files);
}
} catch (error) {
if (error instanceof AmbiguousNormalisedPathError) throw error;
// Directory doesn't exist or is not readable
this.reportDiagnostic(`Error scanning directory ${fullPath}:`, error);
}
@@ -1,5 +1,12 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import { fsPromises as fs, os, path } from "@vrtmrz/livesync-commonlib/node";
import { fsPromises as fs, NodeStorageAdapter, os, path } from "@vrtmrz/livesync-commonlib/node";
import { DEFAULT_SETTINGS, type FilePath } from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
ServiceRebuilder,
type ServiceRebuilderDependencies,
} from "@vrtmrz/livesync-commonlib/compat/serviceModules/Rebuilder";
import { PathServiceCompat } from "@vrtmrz/livesync-commonlib/compat/services/implements/injectable/InjectablePathService";
import { ServiceContext } from "@vrtmrz/livesync-commonlib/compat/services/base/ServiceBase";
import { NodeFileSystemAdapter } from "./NodeFileSystemAdapter";
describe("NodeFileSystemAdapter file enumeration", () => {
@@ -116,3 +123,382 @@ describe("NodeFileSystemAdapter file enumeration", () => {
await expect(adapter.getFiles()).resolves.toEqual([]);
});
});
describe("NodeFileSystemAdapter Unicode path resolution", () => {
const tempDirs: string[] = [];
async function createVault() {
const directory = await fs.mkdtemp(path.join(os.tmpdir(), "livesync-cli-unicode-"));
tempDirs.push(directory);
return directory;
}
function createNormalisingAdapter(directory: string, caseInsensitive = false) {
return new NodeFileSystemAdapter(
directory,
undefined,
() => 1,
() => caseInsensitive
);
}
afterEach(async () => {
await Promise.all(tempDirs.splice(0).map((directory) => fs.rm(directory, { recursive: true, force: true })));
});
it.each([0, 1])("keeps an NFD-only file in the storage listing in identity mode %s", async (version) => {
const directory = await createVault();
const originalPath = "café.md".normalize("NFD");
await fs.writeFile(path.join(directory, originalPath), "original");
const report = vi.fn();
const adapter = new NodeFileSystemAdapter(directory, report, () => version);
expect((await adapter.getFiles()).map((file) => file.path)).toEqual([originalPath]);
expect(report).not.toHaveBeenCalled();
await expect(fs.readFile(path.join(directory, originalPath), "utf8")).resolves.toBe("original");
expect(await fs.readdir(directory)).toEqual([originalPath]);
});
it.each([false, true])(
"enumerates nested mixed names once per directory with case-insensitive matching %s",
async (caseInsensitive) => {
const directory = await createVault();
const parent = "café".normalize("NFD");
const child = `${parent}/Étage`;
await fs.mkdir(path.join(directory, child), { recursive: true });
const names = Array.from(
{ length: 12 },
(_, index) =>
`${child}/${index}-résumé.md`
.normalize(index % 2 ? "NFC" : "NFD")
.split("/")
.pop()!
);
for (const name of names) await fs.writeFile(path.join(directory, child, name), "content");
const adapter = createNormalisingAdapter(directory, caseInsensitive);
const list = vi.spyOn(NodeStorageAdapter.prototype, "list");
try {
const files = await adapter.getFiles();
expect(files.map((file) => file.path).sort()).toEqual(names.map((name) => `${child}/${name}`).sort());
expect(list).toHaveBeenCalledTimes(3);
} finally {
list.mockRestore();
}
}
);
it("uses the existing mixed parent during directory preparation and preserves the new leaf spelling", async () => {
const directory = await createVault();
const parent = "café".normalize("NFD");
const child = `${parent}/Étage`;
await fs.mkdir(path.join(directory, child), { recursive: true });
const adapter = createNormalisingAdapter(directory);
await adapter.storage.mkdir("café");
await adapter.storage.mkdir("café/Étage");
const newName = "résumé.md".normalize("NFD");
const file = await adapter.vault.create(`café/Étage/${newName}`, "created");
expect(file.path).toBe(`${child}/${newName}`);
expect(await fs.readdir(directory)).toEqual([parent]);
expect(await fs.readdir(path.join(directory, parent))).toEqual(["Étage"]);
expect(await fs.readdir(path.join(directory, child))).toEqual([newName]);
await expect(adapter.storage.stat(`café/Étage/${newName.normalize("NFC")}`)).resolves.toMatchObject({
size: 7,
});
});
it("rejects an NFC-equivalent file and directory before scanning or writing a child", async () => {
const directory = await createVault();
const composed = "café";
const decomposed = composed.normalize("NFD");
await fs.writeFile(path.join(directory, composed), "file");
await fs.mkdir(path.join(directory, decomposed));
const adapter = createNormalisingAdapter(directory);
await expect(adapter.getFiles()).rejects.toThrow(/ambiguous/i);
await expect(adapter.storage.write(`${composed}/new.md`, "new")).rejects.toThrow(/ambiguous/i);
await expect(fs.readFile(path.join(directory, composed), "utf8")).resolves.toBe("file");
expect(await fs.readdir(path.join(directory, decomposed))).toEqual([]);
});
it.each([false, true])(
"enumerates each directory once for a flat NFC scan with case-insensitive matching %s",
async (caseInsensitive) => {
const directory = await createVault();
for (let index = 0; index < 12; index++) {
await fs.writeFile(path.join(directory, `${index}.md`), "content");
}
const adapter = createNormalisingAdapter(directory, caseInsensitive);
const list = vi.spyOn(NodeStorageAdapter.prototype, "list");
try {
expect(await adapter.getFiles()).toHaveLength(12);
expect(list).toHaveBeenCalledTimes(1);
} finally {
list.mockRestore();
}
}
);
it("reads, updates, and deletes an NFD file through its NFC path", async () => {
const directory = await createVault();
const nfcFolder = "café";
const nfdFolder = nfcFolder.normalize("NFD");
const nfcName = "Ångström.md";
const nfdName = nfcName.normalize("NFD");
await fs.mkdir(path.join(directory, nfdFolder));
await fs.writeFile(path.join(directory, nfdFolder, nfdName), "before", "utf8");
const adapter = createNormalisingAdapter(directory);
const file = await adapter.getAbstractFileByPath(`${nfcFolder}/${nfcName}`);
expect(file?.path).toBe(`${nfdFolder}/${nfdName}`);
if (!file) throw new Error("Expected the NFC path to resolve to the NFD file");
await expect(adapter.vault.read(file)).resolves.toBe("before");
await adapter.vault.modify(file, "after");
await expect(fs.readFile(path.join(directory, nfdFolder, nfdName), "utf8")).resolves.toBe("after");
expect(await fs.readdir(directory)).toEqual([nfdFolder]);
expect(await fs.readdir(path.join(directory, nfdFolder))).toEqual([nfdName]);
await adapter.vault.delete(file);
await expect(fs.readdir(path.join(directory, nfdFolder))).resolves.toEqual([]);
});
it("creates a new file inside an existing NFD directory using that directory spelling", async () => {
const directory = await createVault();
const nfdFolder = "café".normalize("NFD");
await fs.mkdir(path.join(directory, nfdFolder));
const adapter = createNormalisingAdapter(directory);
const file = await adapter.vault.create("café/new.md", "content");
expect(file.path).toBe(`${nfdFolder}/new.md`);
await expect(fs.readFile(path.join(directory, nfdFolder, "new.md"), "utf8")).resolves.toBe("content");
expect(await fs.readdir(directory)).toEqual([nfdFolder]);
});
it("renames a file by case inside an NFD directory while preserving that directory spelling", async () => {
const directory = await createVault();
const nfcFolder = "Caf\u00e9";
const nfdFolder = nfcFolder.normalize("NFD");
await fs.mkdir(path.join(directory, nfdFolder));
await fs.writeFile(path.join(directory, nfdFolder, "Note.md"), "content", "utf8");
const adapter = createNormalisingAdapter(directory, true);
const file = await adapter.getAbstractFileByPathInsensitive(`${nfcFolder}/Note.md`);
if (!file) throw new Error("Expected the NFC path to resolve to the NFD file");
const renamedFile = await adapter.renameFile(file, `${nfcFolder}/note.md`);
expect(renamedFile.path).toBe(`${nfdFolder}/note.md`);
expect(await fs.readdir(directory)).toEqual([nfdFolder]);
await expect(fs.readdir(path.join(directory, nfdFolder))).resolves.toEqual(["note.md"]);
await expect(fs.readFile(path.join(directory, nfdFolder, "note.md"), "utf8")).resolves.toBe("content");
});
it("rejects an NFC/NFD file collision for lookup and vault mutations", async () => {
const directory = await createVault();
const nfcName = "café.md";
const nfdName = nfcName.normalize("NFD");
await fs.writeFile(path.join(directory, nfcName), "NFC", "utf8");
await fs.writeFile(path.join(directory, nfdName), "NFD", "utf8");
const adapter = createNormalisingAdapter(directory);
const exactFile = {
path: nfcName as FilePath,
stat: { ctime: 1, mtime: 2, size: 3, type: "file" as const },
};
await expect(adapter.getAbstractFileByPath(nfcName)).rejects.toThrow(/ambiguous/i);
await expect(adapter.vault.modify(exactFile, "changed")).rejects.toThrow(/ambiguous/i);
await expect(adapter.vault.delete(exactFile)).rejects.toThrow(/ambiguous/i);
await expect(fs.readFile(path.join(directory, nfcName), "utf8")).resolves.toBe("NFC");
await expect(fs.readFile(path.join(directory, nfdName), "utf8")).resolves.toBe("NFD");
expect((await fs.readdir(directory)).sort()).toEqual([nfcName, nfdName].sort());
});
it("rejects an NFC/NFD directory collision before creating a child", async () => {
const directory = await createVault();
const nfcFolder = "café";
const nfdFolder = nfcFolder.normalize("NFD");
await fs.mkdir(path.join(directory, nfcFolder));
await fs.mkdir(path.join(directory, nfdFolder));
await fs.writeFile(path.join(directory, nfcFolder, "inside.md"), "NFC", "utf8");
await fs.writeFile(path.join(directory, nfdFolder, "inside.md"), "NFD", "utf8");
const adapter = createNormalisingAdapter(directory);
await expect(adapter.getAbstractFileByPath(`${nfcFolder}/inside.md`)).rejects.toThrow(/ambiguous/i);
await expect(adapter.storage.write(`${nfcFolder}/created.md`, "created")).rejects.toThrow(/ambiguous/i);
await expect(fs.readdir(path.join(directory, nfcFolder))).resolves.toEqual(["inside.md"]);
await expect(fs.readdir(path.join(directory, nfdFolder))).resolves.toEqual(["inside.md"]);
await expect(fs.readFile(path.join(directory, nfcFolder, "inside.md"), "utf8")).resolves.toBe("NFC");
await expect(fs.readFile(path.join(directory, nfdFolder, "inside.md"), "utf8")).resolves.toBe("NFD");
});
it("rejects enumeration when NFC-equivalent directories contain different files", async () => {
const directory = await createVault();
const nfcFolder = "café";
const nfdFolder = nfcFolder.normalize("NFD");
await fs.mkdir(path.join(directory, nfcFolder));
await fs.mkdir(path.join(directory, nfdFolder));
await fs.writeFile(path.join(directory, nfcFolder, "a.md"), "NFC", "utf8");
await fs.writeFile(path.join(directory, nfdFolder, "b.md"), "NFD", "utf8");
const adapter = createNormalisingAdapter(directory);
await expect(adapter.getFiles()).rejects.toThrow(/ambiguous/i);
await expect(adapter.scanDirectory()).rejects.toThrow(/ambiguous/i);
await expect(fs.readFile(path.join(directory, nfcFolder, "a.md"), "utf8")).resolves.toBe("NFC");
await expect(fs.readFile(path.join(directory, nfdFolder, "b.md"), "utf8")).resolves.toBe("NFD");
});
it("rechecks the directory after a unique file is cached", async () => {
const directory = await createVault();
const nfcName = "café.md";
const nfdName = nfcName.normalize("NFD");
await fs.writeFile(path.join(directory, nfdName), "NFD", "utf8");
const adapter = createNormalisingAdapter(directory);
await expect(adapter.getAbstractFileByPath(nfcName)).resolves.toMatchObject({ path: nfdName });
await fs.writeFile(path.join(directory, nfcName), "NFC", "utf8");
await expect(adapter.getAbstractFileByPath(nfcName)).rejects.toThrow(/ambiguous/i);
await expect(fs.readFile(path.join(directory, nfcName), "utf8")).resolves.toBe("NFC");
await expect(fs.readFile(path.join(directory, nfdName), "utf8")).resolves.toBe("NFD");
});
it.each([
["Foo", "foo"],
["Café", "café".normalize("NFD")],
])("rejects case-insensitive parent aliases %s and %s containing different files", async (first, second) => {
const directory = await createVault();
for (const [parent, name] of [
[first, "one.md"],
[second, "two.md"],
]) {
await fs.mkdir(path.join(directory, parent));
await fs.writeFile(path.join(directory, parent, name), name);
}
const adapter = createNormalisingAdapter(directory, true);
await expect(adapter.getFiles()).rejects.toThrow(/ambiguous/i);
await expect(adapter.scanDirectory()).rejects.toThrow(/ambiguous/i);
await expect(fs.readFile(path.join(directory, first, "one.md"), "utf8")).resolves.toBe("one.md");
await expect(fs.readFile(path.join(directory, second, "two.md"), "utf8")).resolves.toBe("two.md");
});
it.each([
{ version: 0, caseInsensitive: true },
{ version: 1, caseInsensitive: false },
])(
"compatibility: retains distinct case paths in mode $version with case-insensitive matching $caseInsensitive",
async ({ version, caseInsensitive }) => {
const directory = await createVault();
const names = ["Café/one.md", "café/two.md".normalize("NFD")];
for (const name of names) {
await fs.mkdir(path.dirname(path.join(directory, name)));
await fs.writeFile(path.join(directory, name), name);
}
const adapter = new NodeFileSystemAdapter(
directory,
undefined,
() => version,
() => caseInsensitive
);
expect((await adapter.getFiles()).map((file) => file.path).sort()).toEqual([...names].sort());
for (const name of names) await expect(adapter.storage.read(name)).resolves.toBe(name);
}
);
it("refuses reconstruction before starting remote activity when parent aliases are ambiguous", async () => {
const directory = await createVault();
for (const name of ["Café/one.md", "café/two.md".normalize("NFD")]) {
await fs.mkdir(path.dirname(path.join(directory, name)));
await fs.writeFile(path.join(directory, name), name);
}
const adapter = createNormalisingAdapter(directory, true);
const settings = {
...DEFAULT_SETTINGS,
pathNormalizationVersion: 1 as const,
handleFilenameCaseSensitive: false,
};
const setting = { currentSettings: () => settings, suspendAllSync: { addHandler: vi.fn() } };
const runBoundedRemoteActivity = vi.fn(async () => undefined);
const rebuilder = new ServiceRebuilder({
setting,
storageAccess: { getFileNames: async () => (await adapter.getFiles()).map((file) => file.path) },
database: {
onDatabaseReset: { addHandler: vi.fn() },
isDatabaseReady: () => true,
localDatabase: { findAllDocs: async function* () {} },
},
vault: { isTargetFile: async () => true },
path: new PathServiceCompat(new ServiceContext(), { settingService: setting as never }),
replicator: { runBoundedRemoteActivity },
UI: { showMarkdownDialog: vi.fn(async () => undefined) },
} as unknown as ServiceRebuilderDependencies);
await expect(rebuilder.$rebuildEverything()).rejects.toThrow(/ambiguous/i);
expect(runBoundedRemoteActivity).not.toHaveBeenCalled();
});
it("uses fresh listings after external additions, renames, deletions, and collisions", async () => {
const directory = await createVault();
const nfdName = "café.md".normalize("NFD");
await fs.writeFile(path.join(directory, nfdName), "original");
const adapter = createNormalisingAdapter(directory);
expect((await adapter.getFiles()).map((file) => file.path)).toEqual([nfdName]);
const renamed = "résumé.md".normalize("NFD");
await fs.rename(path.join(directory, nfdName), path.join(directory, renamed));
await fs.writeFile(path.join(directory, "new.md"), "new");
expect((await adapter.getFiles()).map((file) => file.path).sort()).toEqual(["new.md", renamed]);
await fs.rm(path.join(directory, "new.md"));
expect((await adapter.getFiles()).map((file) => file.path)).toEqual([renamed]);
await fs.writeFile(path.join(directory, renamed.normalize("NFC")), "collision");
await expect(adapter.getFiles()).rejects.toThrow(/ambiguous/i);
});
it("retains the storage boundary for exact metadata reads and directory scans", async () => {
const directory = await createVault();
const outside = await createVault();
await fs.writeFile(path.join(outside, "note.md"), "outside");
await fs.symlink(outside, path.join(directory, "linked"), "dir");
const adapter = createNormalisingAdapter(directory);
await expect(adapter.getFiles()).resolves.toEqual([]);
await expect(adapter.storage.statExact("linked/note.md")).rejects.toThrow(/symbolic links/i);
await expect(adapter.storage.write("linked/note.md", "changed")).rejects.toThrow(/symbolic links/i);
await expect(fs.readFile(path.join(outside, "note.md"), "utf8")).resolves.toBe("outside");
});
it("detects NFC and case collisions when lookup is case-insensitive", async () => {
const directory = await createVault();
const upperName = "Café.md";
const lowerNfdName = "café.md".normalize("NFD");
await fs.writeFile(path.join(directory, upperName), "upper", "utf8");
await fs.writeFile(path.join(directory, lowerNfdName), "lower", "utf8");
const adapter = createNormalisingAdapter(directory, true);
await expect(adapter.getAbstractFileByPathInsensitive("CAFÉ.md")).rejects.toThrow(/ambiguous/i);
await expect(adapter.getFiles()).rejects.toThrow(/ambiguous/i);
await expect(adapter.scanDirectory()).rejects.toThrow(/ambiguous/i);
await expect(fs.readFile(path.join(directory, upperName), "utf8")).resolves.toBe("upper");
await expect(fs.readFile(path.join(directory, lowerNfdName), "utf8")).resolves.toBe("lower");
});
it("retains exact-path behaviour when path normalisation is disabled", async () => {
const directory = await createVault();
const nfcName = "café.md";
const nfdName = nfcName.normalize("NFD");
await fs.writeFile(path.join(directory, nfdName), "NFD", "utf8");
const adapter = new NodeFileSystemAdapter(directory);
await expect(adapter.getAbstractFileByPath(nfcName)).resolves.toBeNull();
await adapter.storage.write(nfcName, "NFC");
await expect(fs.readFile(path.join(directory, nfcName), "utf8")).resolves.toBe("NFC");
await expect(fs.readFile(path.join(directory, nfdName), "utf8")).resolves.toBe("NFD");
});
});
+11 -2
View File
@@ -2,6 +2,7 @@ import type { FilePath, UXDataWriteOptions } from "@vrtmrz/livesync-commonlib/co
import type { IVaultAdapter } from "@vrtmrz/livesync-commonlib/compat/serviceModules/adapters";
import type { NodeFile, NodeFolder } from "./NodeTypes";
import { NodeStorageAdapter } from "@vrtmrz/livesync-commonlib/node";
import { UnicodeResolvingNodeStorageAdapter } from "./UnicodeResolvingNodeStorageAdapter";
/**
* Vault adapter implementation for Node.js
@@ -73,8 +74,16 @@ export class NodeVaultAdapter implements IVaultAdapter<NodeFile> {
}
private async toNodeFile(path: string): Promise<NodeFile> {
const stat = await this.storage.stat(path);
const resolvedPath =
this.storage instanceof UnicodeResolvingNodeStorageAdapter
? await this.storage.resolveLogicalPath(path, undefined, false)
: path;
if (resolvedPath === null) throw new Error(`Could not read created file metadata: ${path}`);
const stat =
this.storage instanceof UnicodeResolvingNodeStorageAdapter
? await this.storage.statExact(resolvedPath)
: await this.storage.stat(resolvedPath);
if (stat?.type !== "file") throw new Error(`Could not read created file metadata: ${path}`);
return { path: path as FilePath, stat };
return { path: resolvedPath as FilePath, stat };
}
}
@@ -0,0 +1,205 @@
import { NodeStorageAdapter, validateStoragePath } from "@vrtmrz/livesync-commonlib/node";
export class AmbiguousNormalisedPathError extends Error {
constructor(path: string) {
super(`Ambiguous path after Unicode normalisation: ${path}`);
this.name = "AmbiguousNormalisedPathError";
}
}
/** Resolves NFC-equivalent logical paths against current Node storage listings. */
export class UnicodeResolvingNodeStorageAdapter extends NodeStorageAdapter {
constructor(
rootPath: string,
private readonly getPathNormalizationVersion: () => number = () => 0,
private readonly isCaseInsensitive: () => boolean = () => false
) {
super(rootPath);
}
get isUnicodePathNormalizationEnabled(): boolean {
return this.getPathNormalizationVersion() === 1;
}
/** Read metadata for an entry already resolved or obtained from a storage listing. */
async statExact(path: string) {
return await super.trystat(path);
}
/** Inspect a physical directory once before descending into its enumerated entries. */
async listExact(path: string) {
const entries = await super.list(path);
if (this.isUnicodePathNormalizationEnabled) {
const caseInsensitive = this.isCaseInsensitive();
const names = new Map<string, string>();
for (const entry of [...entries.files, ...entries.folders]) {
const key = this.comparisonKey(entry, caseInsensitive);
if (names.has(key)) throw new AmbiguousNormalisedPathError(entry);
names.set(key, entry);
}
}
return entries;
}
async resolveLogicalPath(
path: string,
caseInsensitive = this.isCaseInsensitive(),
allowMissing = true
): Promise<string | null> {
if (!this.isUnicodePathNormalizationEnabled) return path;
validateStoragePath(path, true);
if (path === "") return "";
const segments = path.split("/");
let resolvedParent = "";
for (let index = 0; index < segments.length; index++) {
const segment = segments[index];
const entries = await super.list(resolvedParent);
const candidates = [...entries.files, ...entries.folders];
const requestedKey = this.comparisonKey(segment, caseInsensitive);
const matches = candidates.filter((entryPath) => {
const entryName = entryPath.slice(resolvedParent === "" ? 0 : resolvedParent.length + 1);
return this.comparisonKey(entryName, caseInsensitive) === requestedKey;
});
if (matches.length > 1) throw new AmbiguousNormalisedPathError(path);
if (matches.length === 1) {
resolvedParent = matches[0];
continue;
}
if (!allowMissing) return null;
const remainingPath = segments.slice(index).join("/");
return resolvedParent === "" ? remainingPath : `${resolvedParent}/${remainingPath}`;
}
return resolvedParent;
}
override async exists(path: string, caseInsensitive = this.isCaseInsensitive()): Promise<boolean> {
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, false);
if (resolvedPath === null) return false;
return await super.exists(resolvedPath);
}
override async trystat(path: string, caseInsensitive = this.isCaseInsensitive()) {
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, false);
if (resolvedPath === null) return null;
return await super.trystat(resolvedPath);
}
override async stat(path: string, caseInsensitive = this.isCaseInsensitive()) {
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, false);
if (resolvedPath === null) return null;
return await super.trystat(resolvedPath);
}
override async mkdir(path: string, caseInsensitive = this.isCaseInsensitive()): Promise<void> {
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, true);
if (resolvedPath === null) throw new Error(`Could not resolve path: ${path}`);
await super.mkdir(resolvedPath);
}
override async remove(path: string, caseInsensitive = this.isCaseInsensitive()): Promise<void> {
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, false);
await super.remove(resolvedPath ?? path);
}
override async read(path: string, caseInsensitive = this.isCaseInsensitive()): Promise<string> {
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, false);
return await super.read(resolvedPath ?? path);
}
override async readBinary(path: string, caseInsensitive = this.isCaseInsensitive()): Promise<ArrayBuffer> {
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, false);
return await super.readBinary(resolvedPath ?? path);
}
override async write(
path: string,
data: string,
options?: Parameters<NodeStorageAdapter["write"]>[2],
caseInsensitive = this.isCaseInsensitive()
): Promise<void> {
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, true);
if (resolvedPath === null) throw new Error(`Could not resolve path: ${path}`);
await super.write(resolvedPath, data, options);
}
override async writeBinary(
path: string,
data: ArrayBuffer,
options?: Parameters<NodeStorageAdapter["writeBinary"]>[2],
caseInsensitive = this.isCaseInsensitive()
): Promise<void> {
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, true);
if (resolvedPath === null) throw new Error(`Could not resolve path: ${path}`);
await super.writeBinary(resolvedPath, data, options);
}
override async append(
path: string,
data: string,
options?: Parameters<NodeStorageAdapter["append"]>[2],
caseInsensitive = this.isCaseInsensitive()
): Promise<void> {
const resolvedPath = await this.resolveLogicalPath(path, caseInsensitive, true);
if (resolvedPath === null) throw new Error(`Could not resolve path: ${path}`);
await super.append(resolvedPath, data, options);
}
override async rename(
sourcePath: string,
targetPath: string,
caseInsensitive = this.isCaseInsensitive()
): Promise<void> {
const resolvedSource = await this.resolveLogicalPath(sourcePath, caseInsensitive, false);
if (resolvedSource === null) return await super.rename(sourcePath, targetPath);
const resolvedTarget = await this.resolveLogicalPath(targetPath, caseInsensitive, true);
if (resolvedTarget === null) throw new Error(`Could not resolve path: ${targetPath}`);
// Preserve explicit case-only or Unicode-spelling renames when the target resolves to the source.
const sourceKey = this.comparisonKey(resolvedSource, caseInsensitive);
const targetKey = this.comparisonKey(resolvedTarget, caseInsensitive);
const target =
sourceKey === targetKey && resolvedTarget === resolvedSource && targetPath !== resolvedSource
? await this.resolveRenameTargetSpelling(targetPath, caseInsensitive)
: resolvedTarget;
await super.rename(resolvedSource, target);
}
override async list(basePath: string, caseInsensitive = this.isCaseInsensitive()) {
const resolvedPath = await this.resolveLogicalPath(basePath, caseInsensitive, false);
if (resolvedPath === null) return { files: [], folders: [] };
return await super.list(resolvedPath);
}
private async resolveRenameTargetSpelling(path: string, caseInsensitive: boolean): Promise<string> {
validateStoragePath(path, false);
const segments = path.split("/");
const leaf = segments.pop();
if (leaf === undefined) throw new Error(`Could not resolve path: ${path}`);
const parentPath = segments.join("/");
const resolvedParent = await this.resolveLogicalPath(parentPath, caseInsensitive, true);
if (resolvedParent === null) throw new Error(`Could not resolve path: ${path}`);
const entries = await super.list(resolvedParent);
const requestedKey = this.comparisonKey(leaf, caseInsensitive);
const matches = [...entries.files, ...entries.folders].filter((entryPath) => {
const entryName = entryPath.slice(resolvedParent === "" ? 0 : resolvedParent.length + 1);
return this.comparisonKey(entryName, caseInsensitive) === requestedKey;
});
if (matches.length > 1) throw new AmbiguousNormalisedPathError(path);
return resolvedParent === "" ? leaf : `${resolvedParent}/${leaf}`;
}
private comparisonKey(segment: string, caseInsensitive: boolean): string {
const normalisedSegment = segment.normalize("NFC");
return caseInsensitive ? normalisedSegment.toLowerCase() : normalisedSegment;
}
}
+43 -2
View File
@@ -6,6 +6,7 @@ import {
type ObsidianLiveSyncSettings,
type EntryDoc,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { isRemediationModeActive } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { ConnectionStringParser } from "@vrtmrz/livesync-commonlib/compat/common/ConnectionString";
import {
activateRemoteConfiguration,
@@ -13,7 +14,7 @@ import {
} from "@vrtmrz/livesync-commonlib/remote-configurations";
import { stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import type { CLICommandContext, CLIOptions } from "./types";
import { toArrayBuffer, toDatabaseRelativePath } from "./utils";
import { getForceRequirementError, toArrayBuffer, toDatabaseRelativePath } from "./utils";
import { collectPeers, openP2PHost, parseTimeoutSeconds, syncWithPeer } from "./p2p";
import { compatGlobal } from "@vrtmrz/livesync-commonlib/compat/common/coreEnvFunctions";
import { fsPromises as fs, getNodeFileCreationTime, path } from "@vrtmrz/livesync-commonlib/node";
@@ -37,8 +38,25 @@ function redactConnectionString(uri: string): string {
}
export async function runCommand(options: CLIOptions, context: CLICommandContext): Promise<boolean> {
const { databasePath, core, replicationScheduling, settingsPath } = context;
const { core } = context;
const { standardIo } = core.services.context;
const forceRequirementError = getForceRequirementError(options.command, options.force);
if (forceRequirementError) {
writeStderrLine(standardIo, `[Error] ${forceRequirementError}`);
return false;
}
if (
options.command === "fetch" &&
isRemediationModeActive(core.services.setting.currentSettings())
) {
writeStderrLine(
standardIo,
`[Error] The fetch command cannot run while remediation mode is active. Clear the modification-time limit before fetching.`
);
return false;
}
const { databasePath, replicationScheduling, settingsPath } = context;
const vaultPath = context.vaultPath || databasePath;
await core.services.control.activated;
@@ -519,6 +537,29 @@ export async function runCommand(options: CLIOptions, context: CLICommandContext
return true;
}
if (options.command === "rebuild") {
writeStderrLine(standardIo, "[Command] rebuild");
try {
await core.serviceModules.rebuilder.$rebuildEverything();
return true;
} catch (error) {
writeStderrLine(standardIo, "[Error] Rebuild failed:", error);
return false;
}
}
if (options.command === "fetch") {
writeStderrLine(standardIo, "[Command] fetch");
try {
// Fetch replaces the local database from remote; do not persist legacy local files first.
await core.serviceModules.rebuilder.$fetchLocal(false, true);
return true;
} catch (error) {
writeStderrLine(standardIo, "[Error] Fetch failed:", error);
return false;
}
}
if (options.command === "remote-add") {
if (options.commandArgs.length < 2) {
throw new Error("remote-add requires two arguments: <name> <connstr>");
@@ -118,6 +118,10 @@ function createCoreMock() {
fetch: vi.fn(async () => undefined),
storeContent: vi.fn(async () => true),
},
rebuilder: {
$rebuildEverything: vi.fn(async () => undefined),
$fetchLocal: vi.fn(async () => undefined),
},
},
} as any;
}
@@ -268,6 +272,73 @@ describe("runCommand abnormal cases", () => {
vi.restoreAllMocks();
});
it.each(["rebuild", "fetch"] as const)("requires --force before activating for %s", async (command) => {
const core = createCoreMock();
const activated = vi.fn(() => Promise.resolve());
Object.defineProperty(core.services.control, "activated", { get: activated });
await expect(runCommand(makeOptions(command, []), { ...context, core })).resolves.toBe(false);
expect(activated).not.toHaveBeenCalled();
expect(core.serviceModules.rebuilder.$rebuildEverything).not.toHaveBeenCalled();
expect(core.serviceModules.rebuilder.$fetchLocal).not.toHaveBeenCalled();
expect(core.services.context.standardIo.writeStderr).toHaveBeenCalledWith(
expect.stringContaining("requires --force")
);
});
it("dispatches a forced rebuild to the existing Rebuilder service", async () => {
const core = createCoreMock();
const options = makeOptions("rebuild", []);
options.force = true;
await expect(runCommand(options, { ...context, core })).resolves.toBe(true);
expect(core.serviceModules.rebuilder.$rebuildEverything).toHaveBeenCalledOnce();
expect(core.serviceModules.rebuilder.$fetchLocal).not.toHaveBeenCalled();
});
it("dispatches a forced fetch without saving legacy Vault files first", async () => {
const core = createCoreMock();
const options = makeOptions("fetch", []);
options.force = true;
await expect(runCommand(options, { ...context, core })).resolves.toBe(true);
expect(core.serviceModules.rebuilder.$fetchLocal).toHaveBeenCalledOnce();
expect(core.serviceModules.rebuilder.$fetchLocal).toHaveBeenCalledWith(false, true);
expect(core.serviceModules.rebuilder.$rebuildEverything).not.toHaveBeenCalled();
});
it("rejects fetch in remediation mode even when forced", async () => {
const core = createCoreMock();
core.services.setting.currentSettings().maxMTimeForReflectEvents = 1;
const options = makeOptions("fetch", []);
options.force = true;
await expect(runCommand(options, { ...context, core })).resolves.toBe(false);
expect(core.serviceModules.rebuilder.$fetchLocal).not.toHaveBeenCalled();
expect(core.services.context.standardIo.writeStderr).toHaveBeenCalledWith(
expect.stringContaining("remediation mode")
);
});
it("surfaces an internal rebuild refusal as a failed command", async () => {
const core = createCoreMock();
core.serviceModules.rebuilder.$rebuildEverything.mockRejectedValue(
new Error("Rebuilding is not available while remediation mode is active.")
);
const options = makeOptions("rebuild", []);
options.force = true;
await expect(runCommand(options, { ...context, core })).resolves.toBe(false);
expect(core.services.context.standardIo.writeStderr).toHaveBeenCalledWith(
expect.stringContaining("Rebuilding is not available while remediation mode is active")
);
});
it("retains visible progress for the interactive sync command", async () => {
const core = createCoreMock();
+4
View File
@@ -22,6 +22,8 @@ export type CLICommand =
| "rm"
| "resolve"
| "mirror"
| "rebuild"
| "fetch"
| "remote-add"
| "remote-rm"
| "remote-ls"
@@ -100,6 +102,8 @@ export const VALID_COMMANDS = new Set([
"rm",
"resolve",
"mirror",
"rebuild",
"fetch",
"remote-add",
"remote-rm",
"remote-ls",
+12
View File
@@ -1,4 +1,16 @@
import { path } from "@vrtmrz/livesync-commonlib/node";
import type { CLICommand } from "./types";
export function getForceRequirementError(command: CLICommand, force = false): string | undefined {
if (force) return undefined;
if (command === "rebuild") {
return "Command 'rebuild' requires --force because it replaces the local and remote databases.";
}
if (command === "fetch") {
return "Command 'fetch' requires --force because it replaces the local database with remote state.";
}
return undefined;
}
export function toArrayBuffer(data: Buffer): ArrayBuffer {
return data.buffer.slice(data.byteOffset, data.byteOffset + data.byteLength) as ArrayBuffer;
+15 -1
View File
@@ -1,6 +1,20 @@
import { path } from "@vrtmrz/livesync-commonlib/node";
import { describe, expect, it } from "vitest";
import { toDatabaseRelativePath } from "./utils";
import { getForceRequirementError, toDatabaseRelativePath } from "./utils";
describe("getForceRequirementError", () => {
it.each([
["rebuild", "Command 'rebuild' requires --force because it replaces the local and remote databases."],
["fetch", "Command 'fetch' requires --force because it replaces the local database with remote state."],
] as const)("requires force for %s", (command, expected) => {
expect(getForceRequirementError(command, false)).toBe(expected);
expect(getForceRequirementError(command, true)).toBeUndefined();
});
it("does not require force for ordinary commands", () => {
expect(getForceRequirementError("sync", false)).toBeUndefined();
});
});
describe("toDatabaseRelativePath", () => {
const databasePath = path.resolve("/tmp/livesync-vault");
+58 -2
View File
@@ -3,6 +3,7 @@ import { appendFile, mkdtemp, mkdir, readFile, rm, stat, writeFile } from "node:
import { tmpdir } from "node:os";
import { join } from "node:path";
import * as chokidar from "chokidar";
import * as offlineScanner from "@vrtmrz/livesync-commonlib/compat/serviceFeatures/offlineScanner";
import { ControlService } from "@vrtmrz/livesync-commonlib/compat/services/base/ControlService";
import {
AutoAccepting,
@@ -20,6 +21,7 @@ import { cloneSettings } from "./settingsPersistence";
import type { CLICommand } from "./commands/types";
vi.mock("chokidar", { spy: true });
vi.mock("@vrtmrz/livesync-commonlib/compat/serviceFeatures/offlineScanner", { spy: true });
function createStandardIoMock() {
return {
@@ -41,6 +43,7 @@ describe("CLI database preparation", () => {
beforeEach(async () => {
vi.mocked(chokidar.watch).mockClear();
vi.mocked(offlineScanner.useOfflineScanner).mockClear();
directory = await mkdtemp(join(tmpdir(), "livesync-cli-bootstrap-"));
vaultPath = join(directory, "vault");
settingsPath = join(directory, "settings.json");
@@ -69,8 +72,18 @@ describe("CLI database preparation", () => {
await rm(directory, { recursive: true, force: true });
});
async function start(command: CLICommand, runner: CliCommandRunner, exitCode = 1) {
process.argv = ["node", "livesync-cli", directory, "--vault", vaultPath, "--settings", settingsPath, command];
async function start(command: CLICommand, runner: CliCommandRunner, exitCode = 1, force = false) {
process.argv = [
"node",
"livesync-cli",
directory,
...(force ? ["--force"] : []),
"--vault",
vaultPath,
"--settings",
settingsPath,
command,
];
// Daemon probes return false so the real core unloads without keeping a daemon alive.
await expect(main(standardIo, runner)).rejects.toThrow(`__EXIT__:${exitCode}`);
}
@@ -255,6 +268,49 @@ describe("CLI database preparation", () => {
expect(watch).not.toHaveBeenCalled();
});
it.each(["rebuild", "fetch"] as const)("does not scan or mirror legacy Vault files before %s", async (command) => {
const storedPaths: string[] = [];
const runner = vi.fn<CliCommandRunner>(async (_options, { core }) => {
for await (const doc of core.services.database.localDatabase.findAllNormalDocs()) {
storedPaths.push(doc.path);
}
return false;
});
await start(command, runner, 1, true);
expect(runner).toHaveBeenCalledOnce();
expect(storedPaths).toEqual([]);
expect(await readFile(join(vaultPath, "notes/local.md"), "utf-8")).toBe("local content");
expect(vi.mocked(chokidar.watch)).not.toHaveBeenCalled();
});
it("registers the Rebuild scanner after readiness and makes its Vault scan available", async () => {
const enumerate = vi.spyOn(ServiceFileAccessCLI.prototype, "getFiles");
const useScanner = vi.mocked(offlineScanner.useOfflineScanner);
let scansBeforeCommand = -1;
let scannerRegistrationsBeforeCommand = -1;
let storedPaths: string[] = [];
const runner = vi.fn<CliCommandRunner>(async (_options, { core }) => {
scansBeforeCommand = enumerate.mock.calls.length;
scannerRegistrationsBeforeCommand = useScanner.mock.calls.length;
await core.services.vault.scanVault(true, true);
for await (const doc of core.services.database.localDatabase.findAllNormalDocs()) {
storedPaths.push(doc.path);
}
return false;
});
await start("rebuild", runner, 1, true);
expect(useScanner).toHaveBeenCalledOnce();
expect(scannerRegistrationsBeforeCommand).toBe(1);
expect(scansBeforeCommand).toBe(0);
expect(enumerate).toHaveBeenCalledOnce();
expect(storedPaths).toEqual(["notes/local.md"]);
});
it.each([
{ command: "daemon" as const, commandRuns: true },
{ command: "mirror" as const, commandRuns: false },
+29 -2
View File
@@ -21,6 +21,7 @@ import {
import { runCommand } from "./commands/runCommand";
import { isCLICommand } from "./commands/types";
import type { CLICommand, CLICommandContext, CLIOptions } from "./commands/types";
import { getForceRequirementError } from "./commands/utils";
import { getPathFromUXFileInfo } from "@vrtmrz/livesync-commonlib/compat/common/typeUtils";
import { stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import { IgnoreRules } from "./serviceModules/IgnoreRules";
@@ -47,6 +48,7 @@ const SETTINGS_FILE = ".livesync/settings.json";
interface CLIVaultSyncMode {
readonly watchFiles: boolean;
readonly reflectReplicationResults: boolean;
readonly scanOnStartup?: boolean;
readonly startupDatabaseOptions: StartupDatabaseOptions;
}
@@ -62,6 +64,18 @@ const VAULT_SYNC_MODES: Readonly<Partial<Record<CLICommand, CLIVaultSyncMode>>>
reflectReplicationResults: false,
startupDatabaseOptions: { ignoreSuspending: true, continueOnFileFailure: false },
},
rebuild: {
watchFiles: false,
reflectReplicationResults: false,
scanOnStartup: false,
startupDatabaseOptions: { ignoreSuspending: false, continueOnFileFailure: false },
},
fetch: {
watchFiles: false,
reflectReplicationResults: true,
scanOnStartup: false,
startupDatabaseOptions: { ignoreSuspending: false, continueOnFileFailure: false },
},
};
ensureGlobalNodeLocalStorage();
@@ -90,6 +104,8 @@ Commands:
p2p-sync <peer> <timeout>
Sync with the specified peer-id or peer-name
p2p-host Start P2P host mode and wait until interrupted
rebuild Rebuild local and remote databases from the selected Vault (requires --force)
fetch Replace the local database with remote state and reflect it to the Vault (requires --force)
push <src> <dst> Push local file <src> into local database path <dst>
pull <src> <dst> Pull file <src> from local database into local file <dst>
pull-rev <src> <dst> <rev> Pull file <src> at specific revision <rev> into local file <dst>
@@ -122,8 +138,9 @@ Commands:
Show remote database status
Options:
--vault <path>, -V <path> (daemon/mirror) Path to the vault directory containing .md files
--vault <path>, -V <path> (daemon/mirror/rebuild/fetch) Path to the vault directory containing .md files
(defaults to database-path; allows separate PouchDB and vault dirs)
--force, -f Required for rebuild and fetch because they replace databases
--interval <N>, -i <N> (daemon only) Poll CouchDB every N seconds instead of using the _changes feed
--write-settings Write setting changes after a successful command
--compat-remote-admin-exit-zero
@@ -319,6 +336,12 @@ export async function main(
commandRunner: CliCommandRunner = runCommand
) {
const options = parseArgs(standardIo);
const forceRequirementError = getForceRequirementError(options.command, options.force);
if (forceRequirementError) {
writeStderrLine(standardIo, `[Error] ${forceRequirementError}`);
process.exitCode = 1;
return;
}
const vaultSyncMode = VAULT_SYNC_MODES[options.command];
if (options.interval && options.command !== "daemon") {
writeStderrLine(
@@ -521,7 +544,7 @@ export async function main(
() => [], // No add-ons
(core, coreFeatureViews) => {
replicationScheduling = coreFeatureViews.replicationScheduling;
if (vaultSyncMode) {
if (vaultSyncMode && vaultSyncMode.scanOnStartup !== false) {
useOfflineScanner(core);
}
// Register P2P replicator feature.
@@ -641,6 +664,10 @@ export async function main(
if (!readyResult) {
throw new Error("Failed to initialise LiveSync.");
}
if (vaultSyncMode?.scanOnStartup === false) {
// Keep the startup reset clean, then enable scans for the explicit Rebuild or Fetch operation.
useOfflineScanner(core);
}
const settingsBeforeCommand = cloneSettings(core.services.setting.currentSettings());
const transientSettingKeys = changedSettingKeys(settingsBeforeSuspend, settingsAfterSuspend);
for (const key of CLI_RUNTIME_ONLY_SETTING_KEYS) {
+25 -1
View File
@@ -1,5 +1,5 @@
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { parseArgs as parseCliArgs } from "./main";
import { main as runCliMain, parseArgs as parseCliArgs } from "./main";
function createStandardIoMock() {
return {
@@ -19,6 +19,7 @@ function mockProcessExit() {
describe("CLI parseArgs", () => {
const originalArgv = process.argv.slice();
const originalExitCode = process.exitCode;
let standardIo: ReturnType<typeof createStandardIoMock>;
beforeEach(() => {
@@ -29,6 +30,7 @@ describe("CLI parseArgs", () => {
afterEach(() => {
process.argv = originalArgv.slice();
process.exitCode = originalExitCode;
vi.restoreAllMocks();
});
@@ -99,6 +101,28 @@ describe("CLI parseArgs", () => {
expect(parsed.commandArgs).toEqual([]);
});
it.each(["rebuild", "fetch"] as const)("parses %s and its force flag", (command) => {
process.argv = ["node", "livesync-cli", "./databasePath", "--force", command];
const parsed = parseArgs();
expect(parsed.command).toBe(command);
expect(parsed.force).toBe(true);
expect(parsed.commandArgs).toEqual([]);
});
it.each(["rebuild", "fetch"] as const)("rejects %s without force before database access", async (command) => {
process.argv = ["node", "livesync-cli", "./missing-database", command];
process.exitCode = undefined;
const runner = vi.fn(async () => true);
await expect(runCliMain(standardIo, runner as any)).resolves.toBeUndefined();
expect(runner).not.toHaveBeenCalled();
expect(process.exitCode).toBe(1);
expect(standardIo.writeStderr).toHaveBeenCalledWith(expect.stringContaining("requires --force"));
expect(standardIo.writeStderr).not.toHaveBeenCalledWith(expect.stringContaining("Database directory"));
});
it("parses remote-add command", () => {
process.argv = [
"node",
@@ -99,6 +99,7 @@ export function initialiseServiceModulesCLI(
// Rebuilder (platform-independent)
const rebuilder = new ServiceRebuilder({
path: services.path,
events: services.context.events,
API: services.API,
database: services.database,
+22 -7
View File
@@ -1,6 +1,10 @@
import { FileAccessBase, type FileAccessBaseDependencies } from "@vrtmrz/livesync-commonlib/compat/serviceModules/FileAccessBase";
import {
FileAccessBase,
type FileAccessBaseDependencies,
} from "@vrtmrz/livesync-commonlib/compat/serviceModules/FileAccessBase";
import { NodeFileSystemAdapter } from "@/apps/cli/adapters/NodeFileSystemAdapter";
import { LOG_LEVEL_NOTICE } from "octagonal-wheels/common/logger";
import { getPathNormalizationVersion } from "@vrtmrz/livesync-commonlib/settings";
/**
* CLI-specific implementation of FileAccessBase
@@ -8,12 +12,23 @@ import { LOG_LEVEL_NOTICE } from "octagonal-wheels/common/logger";
*/
export class FileAccessCLI extends FileAccessBase<NodeFileSystemAdapter> {
constructor(basePath: string, dependencies: FileAccessBaseDependencies) {
const adapter = new NodeFileSystemAdapter(basePath, (message, detail) => {
dependencies.APIService.addLog(message, LOG_LEVEL_NOTICE);
if (detail !== undefined) {
dependencies.APIService.addLog(detail, LOG_LEVEL_NOTICE);
}
});
const pathNormalizationVersion = () =>
getPathNormalizationVersion(dependencies.settingService.currentSettings());
const isCaseInsensitive = () => {
const settings = dependencies.settingService.currentSettings();
return !settings.handleFilenameCaseSensitive || dependencies.vaultService.isStorageInsensitive();
};
const adapter = new NodeFileSystemAdapter(
basePath,
(message, detail) => {
dependencies.APIService.addLog(message, LOG_LEVEL_NOTICE);
if (detail !== undefined) {
dependencies.APIService.addLog(detail, LOG_LEVEL_NOTICE);
}
},
pathNormalizationVersion,
isCaseInsensitive
);
super(adapter, dependencies);
}
@@ -90,6 +90,7 @@ export function initialiseServiceModulesFSAPI(
// Rebuilder (platform-independent)
const rebuilder = new ServiceRebuilder({
path: services.path,
events: services.context.events,
API: services.API,
database: services.database,
+4
View File
@@ -139,6 +139,7 @@
"Copy Report to clipboard": "Copy Report to clipboard",
"CouchDB Configuration": "CouchDB Configuration",
"CouchDB Connection Tweak": "CouchDB Connection Tweak",
"CouchDB only. Version 0 preserves legacy path identity; version 1 uses Unicode NFC. Physical file names stay unchanged. Update every device and stop synchronisation, then Rebuild both databases from a complete authoritative Vault and Fetch on the other devices. Resolve colliding file names first. P2P and Object Storage do not support version 1.": "CouchDB only. Version 0 preserves legacy path identity; version 1 uses Unicode NFC. Physical file names stay unchanged. Update every device and stop synchronisation, then Rebuild both databases from a complete authoritative Vault and Fetch on the other devices. Resolve colliding file names first. P2P and Object Storage do not support version 1.",
"CouchDB validates the database name when you connect. The name must not be empty.": "CouchDB validates the database name when you connect. The name must not be empty.",
"Create or connect to database and continue": "Create or connect to database and continue",
"Create P2P remote": "Create P2P remote",
@@ -404,6 +405,7 @@
"lang-zh": "简体中文",
"lang-zh-tw": "繁體中文",
"Later": "Later",
"Legacy path identity": "Legacy path identity",
"Limit: {datetime} ({timestamp})": "Limit: {datetime} ({timestamp})",
"live": "live",
"LiveSync could not handle multiple vaults which have same name without different prefix, This should be automatically configured.": "LiveSync could not handle multiple vaults which have same name without different prefix, This should be automatically configured.",
@@ -792,6 +794,7 @@
"Paste a connection string": "Paste a connection string",
"Paste the Setup URI generated from one of your active devices.": "Paste the Setup URI generated from one of your active devices.",
"Path": "Path",
"Path identity mode": "Path identity mode",
"Path Obfuscation": "Path Obfuscation",
"Patterns to match files for overwriting instead of merging": "Patterns to match files for overwriting instead of merging",
"Patterns to match files for syncing": "Patterns to match files for syncing",
@@ -1449,6 +1452,7 @@
"Ui.SetupWizard.SetupRemote.ProceedCouchDb": "Continue to CouchDB setup",
"Ui.SetupWizard.SetupRemote.ProceedP2P": "Continue to P2P setup",
"Ui.SetupWizard.SetupRemote.Title": "Choose a synchronisation remote",
"Unicode NFC path identity": "Unicode NFC path identity",
"Unique name between all synchronized devices. To edit this setting, please disable customization sync once.": "Unique name between all synchronized devices. To edit this setting, please disable customization sync once.",
"Unless you are certain, selecting this options is bit dangerous. It assumes that the server configuration is compatible with this device. If this is not the case, data loss may occur. Please ensure you know what you are doing.": "Unless you are certain, selecting this options is bit dangerous. It assumes that the server configuration is compatible with this device. If this is not the case, data loss may occur. Please ensure you know what you are doing.",
"Updating list...": "Updating list...",
+5
View File
@@ -169,6 +169,8 @@ Copy: Copy
Copy Report to clipboard: Copy Report to clipboard
CouchDB Configuration: CouchDB Configuration
CouchDB Connection Tweak: CouchDB Connection Tweak
"CouchDB only. Version 0 preserves legacy path identity; version 1 uses Unicode NFC. Physical file names stay unchanged. Update every device and stop synchronisation, then Rebuild both databases from a complete authoritative Vault and Fetch on the other devices. Resolve colliding file names first. P2P and Object Storage do not support version 1.":
"CouchDB only. Version 0 preserves legacy path identity; version 1 uses Unicode NFC. Physical file names stay unchanged. Update every device and stop synchronisation, then Rebuild both databases from a complete authoritative Vault and Fetch on the other devices. Resolve colliding file names first. P2P and Object Storage do not support version 1."
CouchDB validates the database name when you connect. The name must not be empty.:
CouchDB validates the database name when you connect. The name must not be
empty.
@@ -556,6 +558,7 @@ lang-ru: Русский
lang-zh: 简体中文
lang-zh-tw: 繁體中文
Later: Later
Legacy path identity: Legacy path identity
"Limit: {datetime} ({timestamp})": "Limit: {datetime} ({timestamp})"
live: live
LiveSync could not handle multiple vaults which have same name without different prefix, This should be automatically configured.:
@@ -1251,6 +1254,7 @@ password: password
Password: Password
Paste a connection string: Paste a connection string
Path: Path
Path identity mode: Path identity mode
Path Obfuscation: Path Obfuscation
Patterns to match files for overwriting instead of merging: Patterns to match files for overwriting instead of merging
Patterns to match files for syncing: Patterns to match files for syncing
@@ -2042,6 +2046,7 @@ TweakMismatchResolve:
TweakResolving: Configuration Mismatch Detected
UseRemoteConfig: Use Remote Configuration
AutoAcceptCompatible: Auto-Accept Available
Unicode NFC path identity: Unicode NFC path identity
Unique name between all synchronized devices. To edit this setting, please disable customization sync once.:
Unique name between all synchronized devices. To edit this setting, please
disable customization sync once.
+10 -8
View File
@@ -1,5 +1,6 @@
import type { RemoteDBSettings } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { usesEncryptedInternalMetadata } from "@vrtmrz/livesync-commonlib/replication";
import { getPathNormalizationVersion } from "@vrtmrz/livesync-commonlib/settings";
type EndpointProjection = readonly [kind: "url" | "invalid-url", value: string];
@@ -44,17 +45,18 @@ function projectHeaders(value: string): readonly (readonly [name: string, value:
function projectRemoteSecurity(settings: RemoteDBSettings) {
return [
getPathNormalizationVersion(settings),
settings.idDerivationVersion,
settings.idDerivationKey,
settings.encrypt
? ([
"encrypted",
settings.passphrase,
settings.useDynamicIterationCount,
settings.E2EEAlgorithm,
settings.permitEmptyPassphrase,
] as const)
: (["plain"] as const),
? ([
"encrypted",
settings.passphrase,
settings.useDynamicIterationCount,
settings.E2EEAlgorithm,
settings.permitEmptyPassphrase,
] as const)
: (["plain"] as const),
] as const;
}
@@ -7,6 +7,30 @@ import {
} from "./replicatorConfigurationIdentity";
describe("active Replicator configuration identity", () => {
it("replaces a connection when the path identity mode changes", () => {
const previous = configuredSettings({ pathNormalizationVersion: 0 });
const next = { ...previous, pathNormalizationVersion: 1 as const };
expect(getCouchDBReplicatorConfigurationIdentity(next)).not.toBe(
getCouchDBReplicatorConfigurationIdentity(previous)
);
expect(getObjectStorageReplicatorConfigurationIdentity(next)).not.toBe(
getObjectStorageReplicatorConfigurationIdentity(previous)
);
});
it("treats a missing path identity mode as version 0", () => {
const missingVersion = configuredSettings();
Reflect.deleteProperty(missingVersion, "pathNormalizationVersion");
const explicitVersion0 = { ...missingVersion, pathNormalizationVersion: 0 as const };
expect(getCouchDBReplicatorConfigurationIdentity(explicitVersion0)).toBe(
getCouchDBReplicatorConfigurationIdentity(missingVersion)
);
expect(getObjectStorageReplicatorConfigurationIdentity(explicitVersion0)).toBe(
getObjectStorageReplicatorConfigurationIdentity(missingVersion)
);
});
function configuredSettings(overrides: Partial<ObsidianLiveSyncSettings> = {}): ObsidianLiveSyncSettings {
return Object.assign(createNewVaultSettings(), {
activeConfigurationId: "profile-a",
+6 -1
View File
@@ -112,6 +112,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
fileReflectionProvenance: createFileReflectionProvenance(services.keyValueDB),
});
const rebuilder = new ServiceRebuilder({
path: services.path,
events: services.context.events,
API: services.API,
database: services.database,
@@ -184,7 +185,11 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
core,
(_compatibilityReplicator, p2p) => createInteractiveP2PReplication(p2p),
createOpenRebuildUI(this.app),
{ prepareP2PSettings: useP2PSettingsPreparation(core.services.API.webCompatFetch.bind(core.services.API)) }
{
prepareP2PSettings: useP2PSettingsPreparation(
core.services.API.webCompatFetch.bind(core.services.API)
),
}
);
setupManager.registerP2PSetupConnectionProbe(replicator.connectionProbe);
useP2PReplicatorCommands(core, replicator);
@@ -1,4 +1,5 @@
import { ChunkAlgorithmNames } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { $msg } from "@/common/translation";
import type { SettingSpecGroup } from "./SettingSpec.ts";
export type AdvancedSettingSpecContext = {
@@ -22,6 +23,18 @@ export function createAdvancedSettingSpecGroups({
control: { type: "dropdown", options: () => ChunkAlgorithmNames },
},
{ key: "customChunkSize", control: { type: "number", min: 0, allowZero: true } },
{
key: "pathNormalizationVersion",
holdValue: true,
control: {
type: "numeric-dropdown",
options: () => ({
"0": $msg("Legacy path identity"),
"1": $msg("Unicode NFC path identity"),
}),
},
visible: isCouchDB,
},
],
},
{
@@ -243,6 +243,29 @@ export class LiveSyncSetting extends Setting {
});
return this;
}
autoWireNumericDropDown(key: AllNumericItemKey, opt: AutoWireOption & { options: Record<string, string> }) {
const conf = this.autoWireSetting(key, opt);
this.addDropdown((dropdown) => {
this.autoWiredComponent = dropdown;
const setValue = wrapMemo((value: string) => {
dropdown.setValue(value);
});
dropdown.addOptions(opt.options);
this.invalidateValue = () => setValue(`${LiveSyncSetting.env.editingSettings[key] ?? ""}`);
this.invalidateValue();
dropdown.onChange(async (value) => {
const numericValue = Number(value);
if (!Number.isInteger(numericValue) || !Object.prototype.hasOwnProperty.call(opt.options, value)) {
return false;
}
await this.commitValue(numericValue);
});
this.autoWireComponent(this.autoWiredComponent, conf, opt);
});
return this;
}
addApplyButton(keys: AllSettingItemKey[], text?: string) {
this.addButton((button) => {
this.applyButtonComponent = button;
@@ -133,7 +133,11 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
applySetting(keys: AllSettingItemKey[]) {
for (const k of keys) {
if (!this.isDirty(k)) continue;
if (this.isConfigurationEncryptionSetting(k) || k in OnDialogSettingsDefault) {
if (
this.isReconstructionRequiredSetting(k) ||
this.isConfigurationEncryptionSetting(k) ||
k in OnDialogSettingsDefault
) {
// //@ts-ignore
// this.initialSettings[k] = this.editingSettings[k];
continue;
@@ -147,8 +151,8 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
});
}
async applyAllSettings() {
const changedKeys = (Object.keys(this.editingSettings ?? {}) as AllSettingItemKey[]).filter((e) =>
this.isDirty(e)
const changedKeys = (Object.keys(this.editingSettings ?? {}) as AllSettingItemKey[]).filter(
(e) => this.isDirty(e) && !this.isReconstructionRequiredSetting(e)
);
await this.saveConfigurationEncryptionIfDirty(changedKeys);
this.applySetting(changedKeys);
@@ -168,7 +172,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
*/
async saveSettings(keys: AllSettingItemKey[]) {
let hasChanged = false;
const appliedKeys = keys.filter((k) => this.isDirty(k));
const appliedKeys = keys.filter((k) => this.isDirty(k) && !this.isReconstructionRequiredSetting(k));
await this.saveConfigurationEncryptionIfDirty(appliedKeys);
for (const k of appliedKeys) {
if (this.isConfigurationEncryptionSetting(k)) continue;
@@ -193,11 +197,16 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
await Promise.all(handlers);
// }
keys.forEach((e) => {
if (this.isReconstructionRequiredSetting(e) && this.isDirty(e)) return;
if (this.isConfigurationEncryptionSetting(e) && this.isDirty(e)) return;
this.refreshSetting(e);
});
}
private isReconstructionRequiredSetting(key: AllSettingItemKey): boolean {
return key === "pathNormalizationVersion";
}
private isConfigurationEncryptionSetting(key: AllSettingItemKey): boolean {
return key === "configPassphraseStore" || key === "configPassphrase";
}
@@ -225,8 +234,8 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
* @param keys setting keys for applying
*/
async saveAllDirtySettings() {
const changedKeys = (Object.keys(this.editingSettings ?? {}) as AllSettingItemKey[]).filter((e) =>
this.isDirty(e)
const changedKeys = (Object.keys(this.editingSettings ?? {}) as AllSettingItemKey[]).filter(
(e) => this.isDirty(e) && !this.isReconstructionRequiredSetting(e)
);
await this.saveSettings(changedKeys);
this.reloadAllSettings();
@@ -299,6 +308,13 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
reloadAllSettings(skipUpdate: boolean = false) {
const previousSettings = this._editingSettings;
const previousSnapshot = this.initialSettings;
const pendingPathNormalizationVersion =
previousSettings && previousSnapshot && this.isDirty("pathNormalizationVersion")
? {
value: previousSettings.pathNormalizationVersion,
appliedValue: previousSnapshot.pathNormalizationVersion,
}
: undefined;
const pendingConfigurationEncryption =
previousSettings && previousSnapshot
? {
@@ -326,6 +342,10 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
if (pendingConfigurationEncryption?.configPassphrase) {
this._editingSettings.configPassphrase = pendingConfigurationEncryption.configPassphrase.value;
}
if (pendingPathNormalizationVersion) {
this._editingSettings.pathNormalizationVersion = pendingPathNormalizationVersion.value;
this.initialSettings.pathNormalizationVersion = pendingPathNormalizationVersion.appliedValue;
}
if (!skipUpdate) this.requestUpdate();
}
@@ -333,6 +353,10 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
* Reread each setting and request invalidate
*/
refreshSetting(key: AllSettingItemKey) {
if (this.isReconstructionRequiredSetting(key) && this.isDirty(key)) {
this.requestUpdate();
return;
}
const localSetting = this.reloadAllLocalSettings();
if (key in this.core.settings) {
if (key in localSetting) {
@@ -627,6 +651,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
"passphrase",
"useDynamicIterationCount",
"usePathObfuscation",
"pathNormalizationVersion",
"encrypt",
// "remoteType",
]);
@@ -637,6 +662,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
"passphrase",
"useDynamicIterationCount",
"usePathObfuscation",
"pathNormalizationVersion",
"encrypt",
]);
}
@@ -682,16 +708,23 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
override getControlValue(key: string): unknown {
const spec = this.getDeclarativeSettingSpec(key);
return this.editingSettings[spec.key];
const value = this.editingSettings[spec.key];
return spec.control.type === "numeric-dropdown" ? `${value}` : value;
}
override async setControlValue(key: string, value: unknown): Promise<void> {
const spec = this.getDeclarativeSettingSpec(key);
if (!isValidSettingSpecValue(spec, value)) {
const persistedValue =
spec.control.type === "numeric-dropdown" && typeof value === "string" ? Number(value) : value;
if (!isValidSettingSpecValue(spec, persistedValue)) {
throw new TypeError(`Invalid value for declarative setting ${key}`);
}
Reflect.set(this.editingSettings, spec.key, value);
await this.saveSettings([spec.key]);
Reflect.set(this.editingSettings, spec.key, persistedValue);
if (spec.holdValue) {
this.requestUpdate();
} else {
await this.saveSettings([spec.key]);
}
}
private createRebuildRequiredAction(): SettingDefinitionAction {
@@ -1059,6 +1092,15 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
};
rebuildDB = async (method: "localOnly" | "remoteOnly" | "rebuildBothByThisDevice" | "localOnlyWithChunks") => {
if (method === "remoteOnly" && this.isDirty("pathNormalizationVersion")) {
Logger(
$msg(
"CouchDB only. Version 0 preserves legacy path identity; version 1 uses Unicode NFC. Physical file names stay unchanged. Update every device and stop synchronisation, then Rebuild both databases from a complete authoritative Vault and Fetch on the other devices. Resolve colliding file names first. P2P and Object Storage do not support version 1."
),
LOG_LEVEL_NOTICE
);
return;
}
if (this.editingSettings.encrypt && this.editingSettings.passphrase == "") {
Logger($msg("obsidianLiveSyncSettingTab.logEncryptionNoPassphrase"), LOG_LEVEL_NOTICE);
return;
@@ -1109,6 +1151,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
this.editingSettings.passphrase = "";
}
await this.saveAllDirtySettings();
await this.applyReconstructionRequiredSettings();
},
});
if (result.result === "scheduled") {
@@ -1118,6 +1161,9 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
if (result.result === "failed") {
return;
}
if (this.isDirty("pathNormalizationVersion")) {
return;
}
const applyWithoutInitialisation = $msg(
"Ui.SetupWizard.ApplySettingsInitialisation.ApplyWithoutInitialisation"
@@ -1136,6 +1182,23 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
}
}
private async applyReconstructionRequiredSettings(): Promise<void> {
const key = "pathNormalizationVersion";
if (!this.isDirty(key)) return;
const previousValue = this.core.settings.pathNormalizationVersion;
this.copySettingValue(this.core.settings, this.editingSettings, key);
try {
await this.services.setting.saveSettingData();
} catch (error) {
this.core.settings.pathNormalizationVersion = previousValue;
throw error;
}
this.copySettingValue(this.initialSettings, this.core.settings, key);
this.refreshSetting(key);
}
// The imperative renderer remains required by the declared Obsidian 1.7.2 minimum version.
override display(): void {
this.displayImperative();
@@ -316,6 +316,117 @@ describe("ObsidianLiveSyncSettingTab pending-setting initialisation", () => {
};
}
it("requires local and remote rebuilds when path identity changes", () => {
const { tab } = createSettingsTab();
const initialSettings = tab.initialSettings!;
initialSettings.handleFilenameCaseSensitive = false;
tab.editingSettings.handleFilenameCaseSensitive = false;
Reflect.set(initialSettings, "pathNormalizationVersion", 0);
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 1);
expect(tab.isNeedRebuildLocal()).toBe(true);
expect(tab.isNeedRebuildRemote()).toBe(true);
});
it("does not offer settings-only fallback when path identity changes", async () => {
const { tab, saveSettingData, confirmWithMessage } = createSettingsTab();
const initialSettings = tab.initialSettings!;
Reflect.set(tab.core.settings, "pathNormalizationVersion", 0);
Reflect.set(initialSettings, "pathNormalizationVersion", 0);
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 1);
settingsInitialisationMocks.applySettingsWithInitialisationChoice.mockResolvedValueOnce({
result: "cancelled",
});
await tab.confirmRebuild();
expect(confirmWithMessage).not.toHaveBeenCalled();
expect(saveSettingData).not.toHaveBeenCalled();
expect(Reflect.get(tab.core.settings, "pathNormalizationVersion")).toBe(0);
expect(Reflect.get(tab.editingSettings, "pathNormalizationVersion")).toBe(1);
expect(Reflect.get(initialSettings, "pathNormalizationVersion")).toBe(0);
expect(tab.isDirty("pathNormalizationVersion")).toBe(true);
});
it.each([
["regular-expression Apply", "syncOnlyRegEx", "notes/.*"],
["preset Apply", "preset", "LIVESYNC"],
] as const)("keeps path identity pending during %s", async (_label, setting, value) => {
const { tab } = createSettingsTab();
const initialSettings = tab.initialSettings!;
Reflect.set(tab.core.settings, "pathNormalizationVersion", 0);
Reflect.set(initialSettings, "pathNormalizationVersion", 0);
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 1);
Reflect.set(tab.editingSettings, setting, value);
if (setting === "preset") {
tab.addOnSaved("preset", async () => {
tab.editingSettings.liveSync = true;
await tab.saveAllDirtySettings();
});
}
await tab.saveAllDirtySettings();
expect(Reflect.get(tab.core.settings, "pathNormalizationVersion")).toBe(0);
expect(Reflect.get(tab.initialSettings!, "pathNormalizationVersion")).toBe(0);
expect(Reflect.get(tab.editingSettings, "pathNormalizationVersion")).toBe(1);
expect(tab.isDirty("pathNormalizationVersion")).toBe(true);
if (setting === "syncOnlyRegEx") {
expect(tab.core.settings.syncOnlyRegEx).toBe(value);
} else {
expect(tab.core.settings.liveSync).toBe(true);
}
});
it("excludes path identity from direct save operations", async () => {
const { tab, saveSettingData } = createSettingsTab();
const initialSettings = tab.initialSettings!;
Reflect.set(tab.core.settings, "pathNormalizationVersion", 0);
Reflect.set(initialSettings, "pathNormalizationVersion", 0);
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 1);
await tab.saveSettings(["pathNormalizationVersion"]);
expect(saveSettingData).not.toHaveBeenCalled();
expect(Reflect.get(tab.core.settings, "pathNormalizationVersion")).toBe(0);
expect(Reflect.get(tab.initialSettings!, "pathNormalizationVersion")).toBe(0);
expect(Reflect.get(tab.editingSettings, "pathNormalizationVersion")).toBe(1);
expect(tab.isDirty("pathNormalizationVersion")).toBe(true);
});
it("excludes path identity from ordinary Apply operations", async () => {
const { tab, saveSettingData } = createSettingsTab();
const initialSettings = tab.initialSettings!;
Reflect.set(tab.core.settings, "pathNormalizationVersion", 0);
Reflect.set(initialSettings, "pathNormalizationVersion", 0);
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 1);
await tab.applyAllSettings();
expect(saveSettingData).not.toHaveBeenCalled();
expect(Reflect.get(tab.core.settings, "pathNormalizationVersion")).toBe(0);
expect(Reflect.get(tab.initialSettings!, "pathNormalizationVersion")).toBe(0);
expect(Reflect.get(tab.editingSettings, "pathNormalizationVersion")).toBe(1);
expect(tab.isDirty("pathNormalizationVersion")).toBe(true);
});
it("retains a pending path identity edit and its applied baseline after reload", () => {
const { tab } = createSettingsTab();
const initialSettings = tab.initialSettings!;
Reflect.set(tab.core.settings, "pathNormalizationVersion", 0);
Reflect.set(initialSettings, "pathNormalizationVersion", 0);
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 1);
tab.reloadAllSettings();
expect(Reflect.get(tab.core.settings, "pathNormalizationVersion")).toBe(0);
expect(Reflect.get(tab.initialSettings!, "pathNormalizationVersion")).toBe(0);
expect(Reflect.get(tab.editingSettings, "pathNormalizationVersion")).toBe(1);
expect(tab.isDirty("pathNormalizationVersion")).toBe(true);
expect(tab.isNeedRebuildLocal()).toBe(true);
expect(tab.isNeedRebuildRemote()).toBe(true);
});
it("keeps pending settings in the editing buffer when initialisation and the fallback are cancelled", async () => {
const { tab, saveSettingData, confirmWithMessage, closeSetting } = createSettingsTab();
settingsInitialisationMocks.applySettingsWithInitialisationChoice.mockResolvedValueOnce({
@@ -353,8 +464,11 @@ describe("ObsidianLiveSyncSettingTab pending-setting initialisation", () => {
expect(closeSetting).not.toHaveBeenCalled();
});
it("closes settings only after initialisation has been scheduled", async () => {
it("applies path identity only from a successful initialisation reservation callback", async () => {
const { tab, saveSettingData, confirmWithMessage, closeSetting } = createSettingsTab();
Reflect.set(tab.core.settings, "pathNormalizationVersion", 0);
Reflect.set(tab.initialSettings!, "pathNormalizationVersion", 0);
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 1);
settingsInitialisationMocks.applySettingsWithInitialisationChoice.mockImplementationOnce(
async ({ applySettings }: { applySettings: () => Promise<void> }) => {
await applySettings();
@@ -364,9 +478,118 @@ describe("ObsidianLiveSyncSettingTab pending-setting initialisation", () => {
await tab.confirmRebuild();
expect(saveSettingData).toHaveBeenCalledOnce();
expect(saveSettingData).toHaveBeenCalledTimes(2);
expect(confirmWithMessage).not.toHaveBeenCalled();
expect(closeSetting).toHaveBeenCalledOnce();
expect(Reflect.get(tab.core.settings, "pathNormalizationVersion")).toBe(1);
expect(Reflect.get(tab.initialSettings!, "pathNormalizationVersion")).toBe(1);
expect(Reflect.get(tab.editingSettings, "pathNormalizationVersion")).toBe(1);
expect(tab.isDirty("pathNormalizationVersion")).toBe(false);
});
it("rolls path identity back and keeps it pending when saving after reservation fails", async () => {
const { tab, saveSettingData, closeSetting } = createSettingsTab();
const initialSettings = tab.initialSettings!;
Reflect.set(tab.core.settings, "pathNormalizationVersion", 0);
Reflect.set(initialSettings, "pathNormalizationVersion", 0);
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 1);
tab.editingSettings.handleFilenameCaseSensitive = false;
const failure = new Error("settings could not be saved");
saveSettingData.mockRejectedValueOnce(failure);
settingsInitialisationMocks.applySettingsWithInitialisationChoice.mockImplementationOnce(
async ({ applySettings }: { applySettings: () => Promise<void> }) => {
await applySettings();
return { result: "scheduled", mode: "rebuild" };
}
);
await expect(tab.confirmRebuild()).rejects.toBe(failure);
expect(saveSettingData).toHaveBeenCalledOnce();
expect(Reflect.get(tab.core.settings, "pathNormalizationVersion")).toBe(0);
expect(Reflect.get(tab.initialSettings!, "pathNormalizationVersion")).toBe(0);
expect(Reflect.get(tab.editingSettings, "pathNormalizationVersion")).toBe(1);
expect(tab.isDirty("pathNormalizationVersion")).toBe(true);
expect(closeSetting).not.toHaveBeenCalled();
});
it("retains pending path identity when fetch validation fails", async () => {
const { tab } = createSettingsTab();
const initialSettings = tab.initialSettings!;
Reflect.set(tab.core.settings, "pathNormalizationVersion", 0);
Reflect.set(initialSettings, "pathNormalizationVersion", 0);
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 1);
vi.spyOn(tab, "checkWorkingPassphrase").mockResolvedValue(false);
settingsInitialisationMocks.applySettingsWithInitialisationChoice.mockImplementationOnce(
async ({ validateChoice }: { validateChoice: (mode: "fetch" | "rebuild") => Promise<boolean> }) => {
expect(await validateChoice("fetch")).toBe(false);
return { result: "failed", mode: "fetch" };
}
);
await tab.confirmRebuild();
expect(Reflect.get(tab.core.settings, "pathNormalizationVersion")).toBe(0);
expect(Reflect.get(tab.initialSettings!, "pathNormalizationVersion")).toBe(0);
expect(Reflect.get(tab.editingSettings, "pathNormalizationVersion")).toBe(1);
expect(tab.isDirty("pathNormalizationVersion")).toBe(true);
expect(tab.closeSetting).not.toHaveBeenCalled();
});
it("retains pending path identity when initialisation reservation fails", async () => {
const { tab, saveSettingData } = createSettingsTab();
const initialSettings = tab.initialSettings!;
Reflect.set(tab.core.settings, "pathNormalizationVersion", 0);
Reflect.set(initialSettings, "pathNormalizationVersion", 0);
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 1);
settingsInitialisationMocks.applySettingsWithInitialisationChoice.mockResolvedValueOnce({
result: "failed",
mode: "rebuild",
});
await tab.confirmRebuild();
expect(saveSettingData).not.toHaveBeenCalled();
expect(Reflect.get(tab.core.settings, "pathNormalizationVersion")).toBe(0);
expect(Reflect.get(tab.initialSettings!, "pathNormalizationVersion")).toBe(0);
expect(Reflect.get(tab.editingSettings, "pathNormalizationVersion")).toBe(1);
expect(tab.isDirty("pathNormalizationVersion")).toBe(true);
expect(tab.closeSetting).not.toHaveBeenCalled();
});
it("refuses remote-only overwrite with pending path identity before side effects", async () => {
const { tab, saveSettingData, suspendAllSync, suspendExtraSync, performRebuildDB, closeSetting } =
createSettingsTab();
const initialSettings = tab.initialSettings!;
Reflect.set(tab.core.settings, "pathNormalizationVersion", 0);
Reflect.set(initialSettings, "pathNormalizationVersion", 0);
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 1);
tab.editingSettings.encrypt = false;
const applyAllSettings = vi.spyOn(tab, "applyAllSettings");
const reloadAllSettings = vi.spyOn(tab, "reloadAllSettings");
await tab.rebuildDB("remoteOnly");
expect(applyAllSettings).not.toHaveBeenCalled();
expect(suspendAllSync).not.toHaveBeenCalled();
expect(suspendExtraSync).not.toHaveBeenCalled();
expect(reloadAllSettings).not.toHaveBeenCalled();
expect(saveSettingData).not.toHaveBeenCalled();
expect(performRebuildDB).not.toHaveBeenCalled();
expect(closeSetting).not.toHaveBeenCalled();
expect(Reflect.get(tab.core.settings, "pathNormalizationVersion")).toBe(0);
expect(Reflect.get(tab.initialSettings!, "pathNormalizationVersion")).toBe(0);
expect(Reflect.get(tab.editingSettings, "pathNormalizationVersion")).toBe(1);
expect(tab.isDirty("pathNormalizationVersion")).toBe(true);
expect(vi.mocked($msg)).toHaveBeenCalledWith(
"CouchDB only. Version 0 preserves legacy path identity; version 1 uses Unicode NFC. Physical file names stay unchanged. Update every device and stop synchronisation, then Rebuild both databases from a complete authoritative Vault and Fetch on the other devices. Resolve colliding file names first. P2P and Object Storage do not support version 1."
);
expect(loggerMocks.Logger).toHaveBeenCalledWith(
expect.stringContaining(
"Rebuild both databases from a complete authoritative Vault and Fetch on the other devices."
),
LOG_LEVEL_NOTICE
);
});
it("does not offer the settings-only fallback after an initialisation failure", async () => {
@@ -489,6 +712,33 @@ describe("ObsidianLiveSyncSettingTab declarative settings boundary", () => {
expect(saveSettings).toHaveBeenCalledWith(["hashCacheMaxCount"]);
});
it("keeps a path identity dropdown edit pending until the rebuild flow applies it", async () => {
const { tab, saveSettingData } = createSettingsTab();
const initialSettings = tab.initialSettings!;
const saveSettings = vi.spyOn(tab, "saveSettings");
vi.spyOn(tab, "requestUpdate").mockImplementation(() => undefined);
Reflect.set(tab.core.settings, "pathNormalizationVersion", 0);
Reflect.set(tab.editingSettings, "pathNormalizationVersion", 0);
Reflect.set(initialSettings, "pathNormalizationVersion", 0);
expect(tab.getControlValue("pathNormalizationVersion")).toBe("0");
await tab.setControlValue("pathNormalizationVersion", "1");
expect(Reflect.get(tab.editingSettings, "pathNormalizationVersion")).toBe(1);
expect(tab.getControlValue("pathNormalizationVersion")).toBe("1");
expect(Reflect.get(tab.core.settings, "pathNormalizationVersion")).toBe(0);
expect(Reflect.get(initialSettings, "pathNormalizationVersion")).toBe(0);
expect(saveSettings).not.toHaveBeenCalled();
expect(tab.requestUpdate).toHaveBeenCalledOnce();
expect(saveSettingData).not.toHaveBeenCalled();
expect(tab.isNeedRebuildLocal()).toBe(true);
expect(tab.isNeedRebuildRemote()).toBe(true);
await expect(tab.setControlValue("pathNormalizationVersion", "0.5")).rejects.toThrow(
/Invalid value for declarative setting/u
);
});
it("rejects unregistered declarative control keys", async () => {
const { tab } = createSettingsTab();
@@ -43,7 +43,7 @@ describe("paneAdvanced", () => {
};
paneAdvanced.call(host as never, {} as HTMLElement, { addPanel } as never);
await vi.waitFor(() => expect(settingHarness.rendered).toHaveLength(9));
await vi.waitFor(() => expect(settingHarness.rendered).toHaveLength(10));
expect(addPanel.mock.calls.map(([, heading]) => heading)).toEqual([
"Memory cache",
@@ -55,6 +55,7 @@ describe("paneAdvanced", () => {
panelElements.get("Memory cache"),
panelElements.get("Local Database Tweak"),
panelElements.get("Local Database Tweak"),
panelElements.get("Local Database Tweak"),
panelElements.get("Transfer Tweak"),
panelElements.get("Transfer Tweak"),
panelElements.get("Transfer Tweak"),
@@ -66,6 +67,7 @@ describe("paneAdvanced", () => {
"hashCacheMaxCount",
"chunkSplitterVersion",
"customChunkSize",
"pathNormalizationVersion",
"readChunksOnline",
"useOnlyLocalChunk",
"concurrencyOfReadChunksOnline",
@@ -11,6 +11,7 @@ export type PersistedSettingKey = PersistedBooleanSettingKey | PersistedStringSe
type SettingSpecBase<K extends PersistedSettingKey, C> = {
key: K;
control: C;
holdValue?: boolean;
visible?: () => boolean;
disabled?: () => boolean;
aliases?: string[];
@@ -34,6 +35,14 @@ type NumberSettingSpec = SettingSpecBase<
}
>;
type NumericDropdownSettingSpec = SettingSpecBase<
PersistedNumericSettingKey,
{
type: "numeric-dropdown";
options: () => Record<string, string>;
}
>;
type DropdownSettingSpec = SettingSpecBase<
PersistedStringSettingKey,
{
@@ -42,7 +51,7 @@ type DropdownSettingSpec = SettingSpecBase<
}
>;
export type SettingSpec = ToggleSettingSpec | NumberSettingSpec | DropdownSettingSpec;
export type SettingSpec = ToggleSettingSpec | NumberSettingSpec | NumericDropdownSettingSpec | DropdownSettingSpec;
/** An existing Advanced-page panel and the standard controls rendered within it. */
export type SettingSpecGroup = {
@@ -54,7 +63,7 @@ export type SettingSpecMessages = {
valueShouldBeInRange: (range: { min?: number; max?: number }) => string;
};
type LegacySettingOptions = Pick<AutoWireOption, "onUpdate">;
type LegacySettingOptions = Pick<AutoWireOption, "holdValue" | "onUpdate">;
export type LegacySettingBinding =
| {
@@ -71,6 +80,11 @@ export type LegacySettingBinding =
acceptZero?: boolean;
};
}
| {
type: "numeric-dropdown";
key: PersistedNumericSettingKey;
options: LegacySettingOptions & { options: Record<string, string> };
}
| {
type: "dropdown";
key: PersistedStringSettingKey;
@@ -83,6 +97,10 @@ export interface LegacySettingSpecRenderer {
key: AllNumericItemKey,
options: AutoWireOption & { clampMin?: number; clampMax?: number; acceptZero?: boolean }
): unknown;
autoWireNumericDropDown(
key: AllNumericItemKey,
options: AutoWireOption & { options: Record<string, string> }
): unknown;
autoWireDropDown(key: AllStringItemKey, options: AutoWireOption & { options: Record<string, string> }): unknown;
}
@@ -94,15 +112,21 @@ function isNumberSettingSpec(spec: SettingSpec): spec is NumberSettingSpec {
return spec.control.type === "number";
}
function isNumericDropdownSettingSpec(spec: SettingSpec): spec is NumericDropdownSettingSpec {
return spec.control.type === "numeric-dropdown";
}
function toLegacyUpdateOptions(spec: SettingSpec): LegacySettingOptions {
if (!spec.visible && !spec.disabled) {
return {};
}
return {
onUpdate: (): OnUpdateResult => ({
...(spec.visible ? { visibility: spec.visible() } : {}),
...(spec.disabled ? { disabled: spec.disabled() } : {}),
}),
...(spec.holdValue ? { holdValue: spec.holdValue } : {}),
...(!spec.visible && !spec.disabled
? {}
: {
onUpdate: (): OnUpdateResult => ({
...(spec.visible ? { visibility: spec.visible() } : {}),
...(spec.disabled ? { disabled: spec.disabled() } : {}),
}),
}),
};
}
@@ -131,6 +155,16 @@ export function toLegacySettingBinding(spec: SettingSpec): LegacySettingBinding
},
};
}
if (isNumericDropdownSettingSpec(spec)) {
return {
type: "numeric-dropdown",
key: spec.key,
options: {
...updateOptions,
options: spec.control.options(),
},
};
}
return {
type: "dropdown",
key: spec.key,
@@ -151,6 +185,9 @@ export function renderLegacySettingSpec(renderer: LegacySettingSpecRenderer, spe
case "number":
renderer.autoWireNumeric(binding.key, binding.options);
return;
case "numeric-dropdown":
renderer.autoWireNumericDropDown(binding.key, binding.options);
return;
case "dropdown":
renderer.autoWireDropDown(binding.key, binding.options);
}
@@ -176,6 +213,12 @@ export function isValidSettingSpecValue(spec: SettingSpec, value: unknown): bool
return typeof value === "boolean";
case "number":
return typeof value === "number" && !numberIsOutOfRange(value, spec.control);
case "numeric-dropdown":
return (
typeof value === "number" &&
Number.isInteger(value) &&
Object.prototype.hasOwnProperty.call(spec.control.options(), `${value}`)
);
case "dropdown":
return typeof value === "string" && Object.prototype.hasOwnProperty.call(spec.control.options(), value);
}
@@ -229,6 +272,14 @@ export function toObsidianSettingDefinition(
};
break;
}
case "numeric-dropdown":
control = {
type: "dropdown",
key: spec.key,
options: spec.control.options(),
...(spec.disabled ? { disabled: spec.disabled } : {}),
};
break;
case "dropdown":
control = {
type: "dropdown",
@@ -2,6 +2,7 @@ import { describe, expect, expectTypeOf, it, vi } from "vitest";
import { statusDisplay, type ConfigurationItem } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { getConfig, type OnDialogSettings } from "./settingConstants.ts";
import {
isValidSettingSpecValue,
renderLegacySettingSpec,
toLegacySettingBinding,
toObsidianSettingDefinition,
@@ -16,7 +17,7 @@ const rangeMessages = {
};
describe("Advanced setting specifications", () => {
it("lists only the nine controls currently exposed by the Advanced page", () => {
it("lists only the ten controls currently exposed by the Advanced page", () => {
const groups = createAdvancedSettingSpecGroups({ isCouchDB: () => true });
expect(groups.map(({ heading }) => heading)).toEqual([
@@ -29,6 +30,7 @@ describe("Advanced setting specifications", () => {
"hashCacheMaxCount",
"chunkSplitterVersion",
"customChunkSize",
"pathNormalizationVersion",
"readChunksOnline",
"useOnlyLocalChunk",
"concurrencyOfReadChunksOnline",
@@ -38,12 +40,13 @@ describe("Advanced setting specifications", () => {
]);
});
it("keeps CouchDB visibility on the four remote chunk controls", () => {
it("keeps path identity and the four remote chunk controls CouchDB-only", () => {
let couchDB = false;
const groups = createAdvancedSettingSpecGroups({ isCouchDB: () => couchDB });
const conditional = groups.flatMap(({ items }) => items).filter(({ visible }) => visible !== undefined);
expect(conditional.map(({ key }) => key)).toEqual([
"pathNormalizationVersion",
"readChunksOnline",
"useOnlyLocalChunk",
"concurrencyOfReadChunksOnline",
@@ -55,6 +58,44 @@ describe("Advanced setting specifications", () => {
expect(conditional.every(({ visible }) => visible?.() === true)).toBe(true);
});
it("exposes CouchDB path identity as a two-choice numeric dropdown", () => {
let couchDB = false;
const spec = createAdvancedSettingSpecGroups({ isCouchDB: () => couchDB })
.flatMap(({ items }) => items)
.find(({ key }) => key === "pathNormalizationVersion");
if (!spec) throw new Error("Path identity mode is not exposed on the Advanced page");
if (spec.control.type !== "numeric-dropdown") throw new Error("Expected a numeric dropdown");
expect(spec.control).toMatchObject({ type: "numeric-dropdown", options: expect.any(Function) });
expect(spec.visible?.()).toBe(false);
couchDB = true;
expect(spec.visible?.()).toBe(true);
expect(spec.control.options()).toEqual({
"0": "Legacy path identity",
"1": "Unicode NFC path identity",
});
expect(isValidSettingSpecValue(spec, 0)).toBe(true);
expect(isValidSettingSpecValue(spec, 1)).toBe(true);
expect(isValidSettingSpecValue(spec, 0.5)).toBe(false);
expect(isValidSettingSpecValue(spec, 2)).toBe(false);
const definition = toObsidianSettingDefinition(spec, { name: "Path identity mode" }, rangeMessages);
expect(definition.control).toEqual({
type: "dropdown",
key: "pathNormalizationVersion",
options: { "0": "Legacy path identity", "1": "Unicode NFC path identity" },
});
expect(toLegacySettingBinding(spec)).toEqual({
type: "numeric-dropdown",
key: "pathNormalizationVersion",
options: {
holdValue: true,
onUpdate: expect.any(Function),
options: { "0": "Legacy path identity", "1": "Unicode NFC path identity" },
},
});
});
it("excludes transient settings-dialogue keys from the persisted key type", () => {
type OnDialogOverlap = Extract<keyof OnDialogSettings, PersistedSettingKey>;
expectTypeOf<OnDialogOverlap>().toEqualTypeOf<never>();
@@ -225,6 +266,7 @@ describe("SettingSpec conversion", () => {
const renderer = {
autoWireToggle: vi.fn(),
autoWireNumeric: vi.fn(),
autoWireNumericDropDown: vi.fn(),
autoWireDropDown: vi.fn(),
};
const visible = vi.fn(() => false);
@@ -241,6 +283,7 @@ describe("SettingSpec conversion", () => {
expect(options.onUpdate()).toEqual({ visibility: false });
expect(visible).toHaveBeenCalledOnce();
expect(renderer.autoWireNumeric).not.toHaveBeenCalled();
expect(renderer.autoWireNumericDropDown).not.toHaveBeenCalled();
expect(renderer.autoWireDropDown).not.toHaveBeenCalled();
});
});
@@ -86,6 +86,7 @@ describe("settings page catalogue", () => {
"hashCacheMaxCount",
"chunkSplitterVersion",
"customChunkSize",
"pathNormalizationVersion",
"readChunksOnline",
"useOnlyLocalChunk",
"concurrencyOfReadChunksOnline",
@@ -1,4 +1,9 @@
import { assessRemoteFeatureDocument, describeRemoteFeatureRejection } from "@vrtmrz/livesync-commonlib/replication";
import {
assessRemoteFeatureDocument,
describeRemoteFeatureRejection,
NFC_PATH_IDENTITY_FEATURE,
} from "@vrtmrz/livesync-commonlib/replication";
import { getPathNormalizationVersion } from "@vrtmrz/livesync-commonlib/settings";
import {
SYNCINFO_ID,
VERSIONING_DOCID,
@@ -33,7 +38,7 @@ const REPROCESS_BATCH_SIZE = 100;
type ReplicateResultProcessorSettings = Pick<
ObsidianLiveSyncSettings,
"maxMTimeForReflectEvents" | "suspendParseReplicationResult"
>;
> & { pathNormalizationVersion?: 0 | 1 };
type ReplicateResultProcessorServices = Pick<
LiveSyncBaseCore["services"],
"appLifecycle" | "database" | "path" | "replication" | "vault"
@@ -285,13 +290,20 @@ export class ReplicateResultProcessor {
if (change._id === VERSIONING_DOCID || change.type === "versioninfo") {
this.log(`Version info document received: ${change._id}`, LOG_LEVEL_VERBOSE);
const assessment = assessRemoteFeatureDocument(change);
if (assessment.status !== "supported" && assessment.status !== "older-generation") {
const pathIdentityDoesNotMatch =
assessment.status === "supported" &&
assessment.usedFeatures.includes(NFC_PATH_IDENTITY_FEATURE) !==
(getPathNormalizationVersion(this.context.currentSettings()) === 1);
if (
(assessment.status !== "supported" && assessment.status !== "older-generation") ||
pathIdentityDoesNotMatch
) {
// Fence and retire the active publication through its owner.
this.context.requestActiveReplicatorRetirement();
this.log(
`${describeRemoteFeatureRejection(assessment)} Update Self-hosted LiveSync before synchronising.`,
LOG_LEVEL_NOTICE
);
const rejection = pathIdentityDoesNotMatch
? "Local and remote path identity settings do not match. Select the matching mode and Fetch the migrated database before synchronising."
: `${describeRemoteFeatureRejection(assessment)} Update Self-hosted LiveSync before synchronising.`;
this.log(rejection, LOG_LEVEL_NOTICE);
}
return true;
}
@@ -6,6 +6,7 @@ import {
type EntryDoc,
type FilePathWithPrefix,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import * as replicationCompatibility from "@vrtmrz/livesync-commonlib/replication";
import { ENCRYPTED_INTERNAL_METADATA_FEATURE, REMOTE_FEATURE_GENERATION } from "@vrtmrz/livesync-commonlib/replication";
import {
isValidFilenameInAndroid,
@@ -46,6 +47,7 @@ type SetupOptions = {
localVersionInfo?: unknown;
isTargetFile?: (path: string) => Promise<boolean>;
databaseId?: string;
pathNormalizationVersion?: number;
};
function setup(options: SetupOptions = {}) {
@@ -96,6 +98,7 @@ function setup(options: SetupOptions = {}) {
currentSettings: () => ({
maxMTimeForReflectEvents: options.maxMTimeForReflectEvents ?? 0,
suspendParseReplicationResult: false,
pathNormalizationVersion: options.pathNormalizationVersion ?? 0,
}),
getKeyValueDB: () => core.kvDB,
getLocalDatabase: () => core.localDatabase,
@@ -322,6 +325,66 @@ describe("ReplicateResultProcessor", () => {
expect(onCloseActiveReplication).not.toHaveBeenCalled();
});
it.each([
["local version 0 and remote version 1", 0, ["nfc-path-identity-v1"], true],
["local version 1 and remote version 0", 1, [], true],
["both use version 1", 1, ["nfc-path-identity-v1"], false],
["both use version 0", 0, [], false],
] as const)(
"checks path identity compatibility for %s",
(_description, localVersion, remoteFeatures, shouldRetire) => {
const originalAssessment = replicationCompatibility.assessRemoteFeatureDocument;
const assessmentSpy = vi
.spyOn(replicationCompatibility, "assessRemoteFeatureDocument")
.mockImplementation((document) => {
const assessment = originalAssessment(document);
if (
assessment.status !== "unknown-features" ||
assessment.identifiers.length !== 1 ||
assessment.identifiers[0] !== "nfc-path-identity-v1"
) {
return assessment;
}
return { status: "supported", usedFeatures: ["nfc-path-identity-v1"] };
});
try {
const { onCloseActiveReplication, processor } = setup({ pathNormalizationVersion: localVersion });
processor.enqueueAll([
{
_id: VERSIONING_DOCID,
_rev: "2-path-identity",
type: "versioninfo",
version: REMOTE_FEATURE_GENERATION,
used_features: [...remoteFeatures],
} as PouchDB.Core.ExistingDocument<EntryDoc>,
]);
if (shouldRetire) {
expect(onCloseActiveReplication).toHaveBeenCalledOnce();
} else {
expect(onCloseActiveReplication).not.toHaveBeenCalled();
}
} finally {
assessmentSpy.mockRestore();
}
}
);
it("keeps the existing older-generation gate for local path identity version 1", () => {
const { onCloseActiveReplication, processor } = setup({ pathNormalizationVersion: 1 });
processor.enqueueAll([
{
_id: VERSIONING_DOCID,
_rev: "1-legacy-path-identity",
type: "versioninfo",
version: 11,
} as PouchDB.Core.ExistingDocument<EntryDoc>,
]);
expect(onCloseActiveReplication).not.toHaveBeenCalled();
});
it("reports unknown feature identifiers and requests Replicator retirement", () => {
const logger = vi.fn();
setGlobalLogFunction(logger);
+4
View File
@@ -170,6 +170,10 @@ Set `E2E_OBSIDIAN_ONLY_DIFFERENT_CHUNK_ID_KEYS=true` to run the focused case whe
`test:e2e:obsidian:cli-to-obsidian-sync` is the cross-runtime compatibility check for the official LiveSync CLI and the real Obsidian plug-in. Build the plug-in first, and build the local CLI too when no external CLI command is selected. The script uses E2EE, Path Obfuscation, and the current preferred chunk settings to create and synchronise a note through the CLI, starts real Obsidian with an isolated Vault and profile, synchronises the same CouchDB database, and verifies that the plug-in materialises identical note content. This covers the boundary that CLI-only and plug-in-only round trips do not exercise.
Set `E2E_OBSIDIAN_UNICODE_PATHS=true` to exercise path identity version 1 on Linux. This variant uses an NFD directory and a mixed NFC/NFD file name, checks reflection at Obsidian's NFC path, and returns an update and deletion to the original CLI physical path. Updates use `sync` and `mirror`; deletions use the polling daemon, because `mirror` retains tombstoned local files by default. It uses unencrypted Metadata by default to match issue #1259; add `E2E_OBSIDIAN_UNICODE_ENCRYPTION=true` for the encrypted variant. These checks do not establish native macOS or iOS filesystem behaviour.
Set `E2E_OBSIDIAN_UNICODE_MIGRATION=true` to begin with two CLI databases using legacy path identity. The source runs `rebuild --force --write-settings` after selecting version 1, and the second client runs `fetch --force --write-settings`. The scenario checks that the old document ID is replaced, the NFC feature is declared, and both physical NFD filenames are preserved before continuing the real Obsidian update and deletion checks.
The isolated Obsidian session starts with its CouchDB settings and device-local compatibility acknowledgement already in place. This keeps the scenario focused on cross-runtime data compatibility; unconfigured start-up and visible CouchDB onboarding are covered by their dedicated workflows.
By default, the compatibility check runs `node src/apps/cli/dist/index.cjs`. Set `LIVESYNC_CLI_COMMAND` to test another CLI build or distribution. The value may be a quoted command line or a JSON array of executable and prefix arguments; the scenario arguments are appended without going through a shell.
+234 -16
View File
@@ -28,7 +28,11 @@ process.env.E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS ??= "30000";
process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ??= "30000";
const liveSyncCli = resolve("src/apps/cli/dist/index.cjs");
const notePath = "E2E/cli-to-obsidian.md";
const migrateUnicodePaths = process.env.E2E_OBSIDIAN_UNICODE_MIGRATION === "true";
const unicodePaths = migrateUnicodePaths || process.env.E2E_OBSIDIAN_UNICODE_PATHS === "true";
const useEncryption = !unicodePaths || process.env.E2E_OBSIDIAN_UNICODE_ENCRYPTION === "true";
const notePath = unicodePaths ? "Cafe\u0301/r\u00e9sume\u0301.md" : "E2E/cli-to-obsidian.md";
const receivedNotePath = unicodePaths ? notePath.normalize("NFC") : notePath;
const noteContent = [
"# CLI to real Obsidian",
"",
@@ -183,7 +187,8 @@ async function configureLiveSyncCli(
command: LiveSyncCliCommand,
settingsPath: string,
couchDb: Awaited<ReturnType<typeof loadCouchDbConfig>>,
dbName: string
dbName: string,
pathNormalizationVersion = unicodePaths ? 1 : 0
): Promise<void> {
await runLiveSyncCli(command, ["init-settings", "--force", settingsPath]);
const settings = JSON.parse(await readFile(settingsPath, "utf-8")) as Record<string, unknown>;
@@ -210,16 +215,61 @@ async function configureLiveSyncCli(
handleFilenameCaseSensitive: false,
doNotUseFixedRevisionForChunks: true,
E2EEAlgorithm: "v2",
encrypt: true,
encrypt: useEncryption,
passphrase: e2eePassphrase,
usePathObfuscation: true,
encryptInternalMetadata: true,
usePathObfuscation: useEncryption,
encryptInternalMetadata: useEncryption,
pathNormalizationVersion,
doctorProcessedVersion: "0.25.27",
isConfigured: true,
});
await writeFile(settingsPath, `${JSON.stringify(settings, null, 2)}\n`, "utf-8");
}
async function startLiveSyncCliDaemon(command: LiveSyncCliCommand, databasePath: string, settingsPath: string) {
const child = spawn(
command.executable,
[...command.prefixArgs, databasePath, "--settings", settingsPath, "--interval", "1", "daemon"],
{ cwd: process.cwd(), env: process.env, stdio: ["ignore", "pipe", "pipe"] }
);
let output = "";
let spawnError: Error | undefined;
child.stdout.on("data", (chunk: Buffer) => {
output += chunk.toString("utf-8");
});
child.stderr.on("data", (chunk: Buffer) => {
output += chunk.toString("utf-8");
});
child.on("error", (error) => {
spawnError = error;
});
const stop = async () => {
if (child.exitCode !== null || child.signalCode !== null || spawnError) return;
await new Promise<void>((resolvePromise) => {
const timeout = setTimeout(() => child.kill("SIGKILL"), 10000);
child.once("exit", () => {
clearTimeout(timeout);
resolvePromise();
});
child.kill("SIGTERM");
});
};
const deadline = Date.now() + Number(process.env.E2E_LIVESYNC_CLI_TIMEOUT_MS ?? 60000);
try {
while (!output.includes("[Daemon] Polling mode:")) {
if (spawnError) throw spawnError;
if (child.exitCode !== null || child.signalCode !== null || Date.now() >= deadline) {
throw new Error(`The CLI daemon did not become ready.\n${output}`);
}
await new Promise((resolvePromise) => setTimeout(resolvePromise, 100));
}
return { stop, getOutput: () => output };
} catch (error) {
await stop();
throw error;
}
}
async function writeCliNote(
command: LiveSyncCliCommand,
databasePath: string,
@@ -280,6 +330,86 @@ async function waitForVaultContent(
throw new Error(`Timed out waiting for CLI-created note at ${fullPath}. Last content:\n${lastContent}`);
}
async function migrateCliDatabases(
command: LiveSyncCliCommand,
cliState: string,
databasePath: string,
settingsPath: string,
couchDb: Awaited<ReturnType<typeof loadCouchDbConfig>>,
dbName: string,
legacyFileInfo: CliFileInfo
): Promise<CliFileInfo> {
const secondaryDatabasePath = join(cliState, "secondary-database");
const secondarySettingsPath = join(cliState, "secondary-settings.json");
await mkdir(secondaryDatabasePath, { recursive: true });
await configureLiveSyncCli(command, secondarySettingsPath, couchDb, dbName, 0);
await runLiveSyncCli(command, [secondaryDatabasePath, "--settings", secondarySettingsPath, "sync"]);
await runLiveSyncCli(command, [secondaryDatabasePath, "--settings", secondarySettingsPath, "mirror"]);
assertEqual(
await readFile(join(secondaryDatabasePath, notePath), "utf-8"),
editedNoteContent,
"The secondary CLI did not materialise the legacy NFD note before migration."
);
for (const path of [settingsPath, secondarySettingsPath]) {
const settings = JSON.parse(await readFile(path, "utf-8"));
settings.pathNormalizationVersion = 1;
await writeFile(path, `${JSON.stringify(settings, null, 2)}\n`, "utf-8");
}
await runLiveSyncCli(command, [databasePath, "--settings", settingsPath, "rebuild", "--force", "--write-settings"]);
const info = await runLiveSyncCli(command, [databasePath, "--settings", settingsPath, "info", notePath]);
const fileInfo = JSON.parse(info.stdout) as CliFileInfo;
if (fileInfo.id === legacyFileInfo.id) {
throw new Error("Rebuild retained the legacy NFD document ID.");
}
await waitForCouchDbDocs(couchDb, dbName, (docs) => {
const ids = new Set(docs.map((doc) => doc._id));
return (
!ids.has(legacyFileInfo.id) &&
ids.has(fileInfo.id) &&
fileInfo.children.every((childId) => ids.has(childId)) &&
docs.some((doc) => Array.isArray(doc.used_features) && doc.used_features.includes("nfc-path-identity-v1"))
);
});
await runLiveSyncCli(command, [
secondaryDatabasePath,
"--settings",
secondarySettingsPath,
"fetch",
"--force",
"--write-settings",
]);
const secondaryInfo = await runLiveSyncCli(command, [
secondaryDatabasePath,
"--settings",
secondarySettingsPath,
"info",
notePath,
]);
assertEqual(
(JSON.parse(secondaryInfo.stdout) as CliFileInfo).id,
fileInfo.id,
"Fetch did not replace the secondary CLI's legacy document identity."
);
for (const path of [databasePath, secondaryDatabasePath]) {
assertEqual(
await readFile(join(path, notePath), "utf-8"),
editedNoteContent,
"Migration changed the content of the original NFD physical file."
);
await access(join(path, receivedNotePath)).then(
() => {
throw new Error("Migration created a second NFC physical file.");
},
(error: NodeJS.ErrnoException) => {
if (error.code !== "ENOENT") throw error;
}
);
}
console.log("CLI Rebuild and Fetch migrated legacy document IDs while preserving both NFD physical files.");
return { ...fileInfo, expectedCtime: legacyFileInfo.expectedCtime };
}
async function main(): Promise<void> {
const liveSyncCliCommand = resolveLiveSyncCliCommand();
if (process.env.LIVESYNC_CLI_COMMAND === undefined) {
@@ -304,12 +434,19 @@ async function main(): Promise<void> {
const cliSourcePath = join(cliState, "source", "cli-to-obsidian.md");
const vault = await createTemporaryVault();
let session: ObsidianLiveSyncSession | undefined;
let daemon: Awaited<ReturnType<typeof startLiveSyncCliDaemon>> | undefined;
try {
await assertCouchDbReachable(couchDb);
await createCouchDbDatabase(couchDb, dbName);
await mkdir(cliDatabasePath, { recursive: true });
await configureLiveSyncCli(liveSyncCliCommand, cliSettingsPath, couchDb, dbName);
await configureLiveSyncCli(
liveSyncCliCommand,
cliSettingsPath,
couchDb,
dbName,
migrateUnicodePaths ? 0 : undefined
);
if (process.env.LIVESYNC_CLI_COMMAND === undefined) {
console.log(`Using locally built LiveSync CLI: ${liveSyncCli}`);
@@ -323,7 +460,21 @@ async function main(): Promise<void> {
console.log(`Temporary Obsidian vault: ${vault.path}`);
console.log(`Temporary CouchDB database: ${dbName}`);
const cliFileInfo = await writeCliNote(liveSyncCliCommand, cliDatabasePath, cliSettingsPath, cliSourcePath);
let cliFileInfo = await writeCliNote(liveSyncCliCommand, cliDatabasePath, cliSettingsPath, cliSourcePath);
if (migrateUnicodePaths) {
cliFileInfo = await migrateCliDatabases(
liveSyncCliCommand,
cliState,
cliDatabasePath,
cliSettingsPath,
couchDb,
dbName,
cliFileInfo
);
}
if (unicodePaths && !useEncryption) {
assertEqual(cliFileInfo.id, receivedNotePath.toLowerCase(), "The CLI did not derive an NFC document ID.");
}
await waitForCouchDbDocs(couchDb, dbName, (docs) => {
const ids = new Set(docs.map((doc) => doc._id));
return ids.has(cliFileInfo.id) && cliFileInfo.children.every((childId) => ids.has(childId));
@@ -342,10 +493,11 @@ async function main(): Promise<void> {
dbName,
},
{
encrypt: true,
encrypt: useEncryption,
passphrase: e2eePassphrase,
usePathObfuscation: true,
encryptInternalMetadata: true,
usePathObfuscation: useEncryption,
encryptInternalMetadata: useEncryption,
pathNormalizationVersion: unicodePaths ? 1 : 0,
E2EEAlgorithm: "v2",
}
),
@@ -355,7 +507,7 @@ async function main(): Promise<void> {
await prepareRemote(obsidianCli.binary, session.cliEnv);
await pushLocalChanges(obsidianCli.binary, session.cliEnv);
const received = await waitForVaultContent(vault.path, notePath);
const received = await waitForVaultContent(vault.path, receivedNotePath);
assertEqual(received, editedNoteContent, "The real Obsidian plug-in did not materialise the CLI-edited note.");
if (cliFileInfo.expectedCtime !== undefined) {
const receivedCtime = await evalObsidianJson<number>(
@@ -363,7 +515,7 @@ async function main(): Promise<void> {
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
`const entry=await core.localDatabase.getDBEntry(${JSON.stringify(notePath)},undefined,false,true);`,
`const entry=await core.localDatabase.getDBEntry(${JSON.stringify(receivedNotePath)},undefined,false,true);`,
"return JSON.stringify(entry.ctime);",
"})()",
].join(""),
@@ -376,12 +528,78 @@ async function main(): Promise<void> {
);
console.log("The CLI edit retained the filesystem creation time in the real Obsidian client's Metadata.");
} else {
console.log("Creation-time verification skipped: native stats did not expose distinct creation and change times.");
console.log(
"Creation-time verification skipped: native stats did not expose distinct creation and change times."
);
}
console.log(
"CLI-edited encrypted note was retrieved by the real Obsidian plug-in with identical content."
);
if (unicodePaths) {
const returnedContent = `${editedNoteContent}\nUpdated in Obsidian.\n`;
await evalObsidianJson(
obsidianCli.binary,
[
"(async()=>{",
`const path=${JSON.stringify(receivedNotePath)};`,
"const file=app.vault.getAbstractFileByPath(path);",
"await new Promise(resolve=>setTimeout(resolve,Math.max(0,file.stat.mtime+2500-Date.now())));",
`await app.vault.modify(file,${JSON.stringify(returnedContent)});`,
"return JSON.stringify(true);})()",
].join(""),
session.cliEnv
);
await pushLocalChanges(obsidianCli.binary, session.cliEnv);
await runLiveSyncCli(liveSyncCliCommand, [cliDatabasePath, "--settings", cliSettingsPath, "sync"]);
await runLiveSyncCli(liveSyncCliCommand, [cliDatabasePath, "--settings", cliSettingsPath, "mirror"]);
assertEqual(
await readFile(join(cliDatabasePath, notePath), "utf-8"),
returnedContent,
"The Obsidian update did not reach the original NFD physical file."
);
await access(join(cliDatabasePath, receivedNotePath)).then(
() => {
throw new Error("The CLI created a second NFC physical file.");
},
(error: NodeJS.ErrnoException) => {
if (error.code !== "ENOENT") throw error;
}
);
// The mirror command intentionally keeps tombstoned local files. The daemon applies remote deletions.
daemon = await startLiveSyncCliDaemon(liveSyncCliCommand, cliDatabasePath, cliSettingsPath);
await evalObsidianJson(
obsidianCli.binary,
[
"(async()=>{",
`const file=app.vault.getAbstractFileByPath(${JSON.stringify(receivedNotePath)});`,
"await new Promise(resolve=>setTimeout(resolve,Math.max(0,file.stat.mtime+2500-Date.now())));",
"await app.vault.delete(file);",
"return JSON.stringify(true);})()",
].join(""),
session.cliEnv
);
await pushLocalChanges(obsidianCli.binary, session.cliEnv);
const deletionDeadline = Date.now() + Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS);
for (;;) {
try {
await access(join(cliDatabasePath, notePath));
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error;
break;
}
if (Date.now() >= deletionDeadline) {
throw new Error(
`The Obsidian deletion did not remove the original NFD physical file.\n${daemon.getOutput()}`
);
}
await new Promise((resolvePromise) => setTimeout(resolvePromise, 100));
}
await daemon.stop();
daemon = undefined;
console.log(
"NFC path identity preserved the NFD physical name through creation, remote update, and deletion."
);
}
console.log("CLI-edited note was retrieved by the real Obsidian plug-in with identical content.");
} finally {
await daemon?.stop();
if (session) {
await session.app.stop();
}
+6
View File
@@ -12,6 +12,12 @@ Earlier releases remain available in the 1.0 release history, the 1.0 preview hi
## Unreleased
### Fixed
- We can opt into NFC path identity for CouchDB synchronisation between the CLI and Obsidian. This prevents decomposed or mixed Unicode file names from becoming inaccessible through a receiving device's normalised path. (#1259)
- Existing Vaults retain their current identity rules. Migration requires updated clients, a Rebuild from a complete authoritative Vault, and a Fetch on the other devices.
- Reconstruction stops when the supported host file listing or local Metadata exposes ambiguous files or parent folders under NFC and the configured case handling. The CLI retains existing physical file names. P2P and Object Storage do not support the new mode.
## 1.0.37
16th October, 2026