starred/CloakBrowser
1
0 Fork 0
mirror of https://github.com/CloakHQ/CloakBrowser.git synced 2026-10-18 23:07:13 -05:00
Code Issues Packages Projects Releases 10 Wiki Activity
Files
v0.6.0
CloakBrowser/cloakbrowser/download.py
T
ShubhamandCloakHQ 9fb05e0fc8 fix(download): make concurrent first-run downloads safe (#555) 
* fix(download): make concurrent first-run downloads safe

_extract_archive deleted the install dir and then extracted into it in
place. When several processes or threads start on an empty cache, a
second caller can delete the first caller's install mid-extraction, or
pass the "binary exists and is executable" check on a binary that is
still being written. On Linux that surfaces as ETXTBSY on spawn; on
macOS as dlopen failures and "Directory not empty", and the cache can
be left permanently incomplete.

Extract into a uniquely named sibling staging dir and rename it into
place. If the destination already holds a complete install, another
caller won and the staged copy is discarded; an incomplete leftover is
still replaced as before.

* fix(js): make concurrent first-run downloads safe

Same race as the Python wrapper: extractArchive deleted the install dir
and extracted into it in place, so concurrent ensureBinary() calls on an
empty cache could delete each other's install or return a binary that
was still being written (spawn ETXTBSY on Linux).

Extract into a uniquely named sibling staging dir and rename it into
place, keeping an existing complete install if another caller won.
Also name the temporary archive with randomUUID() instead of
Date.now(), which collided for calls made in the same millisecond.

* fix(dotnet): make concurrent first-run downloads safe

Same race as the Python and JS wrappers: ExtractArchive deleted the
install dir and extracted into it in place, so concurrent
EnsureBinaryAsync() calls on an empty cache failed with "already
exists" / DirectoryNotFoundException or saw a half-written install.

Extract into a uniquely named sibling staging dir and move it into
place, keeping an existing complete install if another caller won.

* fix(download): replace a partial install safely under concurrent callers

When an incomplete install dir was left over from an interrupted run,
every concurrent caller saw it as incomplete and deleted it in place.
The first to finish renamed its staging dir into place; the others'
unguarded second rename then failed (ENOTEMPTY, FileNotFoundError,
DirectoryNotFoundException), and a late delete could remove files from
an install another caller had just moved in. With 8 concurrent callers,
7 raised in each round.

Rename the leftover aside to a unique <install>.stale-<id> dir and
delete it from there, then retry the staging rename a bounded number
of times, keeping a complete install if another caller got there
first. Applies to the Python, JS and .NET clients, each with a
regression test that starts several concurrent callers on a partial
install. The JS test uses separate processes, since callers in one
process cannot interleave inside the synchronous rename block.

* fix(download): retry directory renames Windows transiently denies

Antivirus and indexers on Windows briefly hold freshly extracted files
open, which fails the directory rename with access denied. Retry that
rename for about 2.5 seconds while the destination does not exist, in
all three clients. Cleanup of the staging and stale dirs is now best
effort so a failed delete never masks the error being raised.

---------

Co-authored-by: CloakHQ <cloakhq@pm.me>
2026-10-08 00:53:57 +02:00

1331 lines
50 KiB
Python
Raw Permalink Blame History

"""Binary download and cache management for cloakbrowser.
Downloads the patched Chromium binary on first use, caches it locally.
Similar to how Playwright downloads its own bundled Chromium.
"""
from __future__ import annotations
import hashlib
import logging
import os
import shutil
import platform
import stat
import subprocess
import sys
import tarfile
import tempfile
import threading
import time
import uuid
from pathlib import Path
import httpx
from ._version import __version__ as _wrapper_version
from .config import (
BINARY_SIGNING_PUBKEYS,
CHROMIUM_VERSION,
DOWNLOAD_BASE_URL,
GITHUB_API_URL,
GITHUB_DOWNLOAD_BASE_URL,
_version_newer,
check_platform_available,
get_archive_ext,
get_archive_name,
get_binary_dir,
get_binary_path,
get_cache_dir,
get_chromium_version,
get_download_url,
get_effective_version,
get_fallback_download_url,
get_local_binary_override,
get_platform_tag,
normalize_release_channel,
normalize_requested_version,
)
logger = logging.getLogger("cloakbrowser")
class BinaryVerificationError(RuntimeError):
"""A downloaded binary could not be authenticated (bad/missing signature,
version mismatch, or checksum failure).
Distinct from transient download/network errors: a verification failure is
a tampering signal and MUST surface, never silently fall back to another
binary. The Pro routing in ensure_binary re-raises this rather than
downgrading to the free tier.
"""
# Timeout for download (large binary, allow 10 min)
DOWNLOAD_TIMEOUT = httpx.Timeout(connect=10.0, read=60.0, write=10.0, pool=10.0)
# Auto-update check interval (1 hour)
UPDATE_CHECK_INTERVAL = 3600
# Free-tier welcome banner re-show interval (1 day). Free users see the
# "get the latest free" invite again after this gap; Pro users see it only once
# (see _show_welcome).
WELCOME_FREE_INTERVAL = 24 * 3600
# Pro Chromium major shown in the free-tier welcome banner. Bump at each Pro
# major release (there is no local constant to derive it from — the live Pro
# version comes from the network, which we don't call just to print a banner).
PRO_MAJOR = "154"
# Attempts to rename an extraction into place while concurrent callers are
# replacing the same partial install (see _extract_archive).
INSTALL_RENAME_ATTEMPTS = 5
# Retries for a directory rename that Windows transiently denies (see
# _rename_dir_with_retry): about 2.5 seconds in total.
TRANSIENT_RENAME_ATTEMPTS = 10
TRANSIENT_RENAME_DELAY = 0.25
def _welcome_due(marker: Path, pro: bool) -> bool:
"""Whether the welcome banner should be shown now.
Pro: once ever (only when the marker is absent). Free: re-show when the
marker is absent or its timestamp is older than WELCOME_FREE_INTERVAL.
Unreadable or legacy empty markers count as stale (due).
"""
if not marker.exists():
return True
if pro:
return False
try:
last = int(marker.read_text().strip())
except (OSError, ValueError):
return True
return (time.time() - last) >= WELCOME_FREE_INTERVAL
_preview_fallback_warned = False
def _warn_preview_fallback() -> None:
"""Tell the user, once per process, that a requested Preview build does not
exist for this platform and the Stable build is being used instead."""
global _preview_fallback_warned
if _preview_fallback_warned:
return
_preview_fallback_warned = True
# Write straight to stderr (like the welcome banner) so an app's logging
# config can't silence it.
sys.stderr.write(
"[cloakbrowser] Preview channel requested, but no preview build is "
f"available for {get_platform_tag()}; using the stable build.\n"
)
def _emit(text: str) -> None:
"""Write a cosmetic line to stderr, swallowing any error.
On a legacy Windows console sys.stderr is cp1252/strict; an unencodable
glyph raises UnicodeEncodeError. Because the welcome banner runs on the
binary-download path, that would abort the whole launch (ticket 2354).
A banner is decoration - never let it propagate.
"""
try:
sys.stderr.write(text)
except Exception:
pass
def _show_welcome(tier: str = "keyless") -> None:
"""Show welcome message on launch. A marker file gates the cadence: a paid
(Pro) key shows once ever; free (keyless or a free GitHub key) re-shows every
WELCOME_FREE_INTERVAL.
tier:
"pro" — a paid license key (Pro banner + support address)
"free" — a free GitHub key (latest binary, one concurrent session)
"keyless" — no key, running the older free binary (invite the free login)
"""
marker = get_cache_dir() / ".welcome_shown"
if not _welcome_due(marker, pro=(tier == "pro")):
return
# ASCII-only banner + a swallow-everything writer: on a legacy Windows
# console sys.stderr is cp1252/strict, so a non-ASCII glyph (or any IO
# error) here would raise and abort the whole binary-download path. A
# cosmetic banner must never be able to kill a launch.
_emit("\n")
_emit(" CloakBrowser - stealth Chromium for automation\n")
_emit(" https://github.com/CloakHQ/CloakBrowser\n")
_emit("\n")
if tier == "pro":
_emit(
f" CloakBrowser Pro active (v{PRO_MAJOR}) - latest binary, newest patches.\n"
)
_emit(" Pro support -> support@cloakbrowser.dev\n")
elif tier == "free":
_emit(
f" CloakBrowser free (v{PRO_MAJOR}): the latest binary, 1 concurrent session.\n"
)
_emit(
" For more than one concurrent session -> https://cloakbrowser.dev\n"
)
else:
free_major = CHROMIUM_VERSION.split(".")[0]
_emit(
f" Running the free binary (v{free_major}). "
f"The latest binary (v{PRO_MAJOR}) is free too, with 1 concurrent session.\n"
)
_emit(
" Get your key: run cloakbrowser login or visit https://cloakbrowser.dev/free\n"
)
_emit(
" For more than one concurrent session -> https://cloakbrowser.dev\n"
)
_emit(" Star us if CloakBrowser helps your project!\n")
_emit("\n")
try:
marker.parent.mkdir(parents=True, exist_ok=True)
marker.write_text(str(int(time.time())))
except OSError:
pass
def ensure_binary(
license_key: str | None = None,
browser_version: str | None = None,
release_channel: str | None = None,
) -> str:
"""Ensure the stealth Chromium binary is available. Download if needed.
Returns the path to the chrome executable as a string.
Args:
license_key: Pro license key. Also reads from CLOAKBROWSER_LICENSE_KEY env var.
browser_version: Exact Chromium version pin. Also reads from CLOAKBROWSER_VERSION.
release_channel: Stable (default) or Preview binary channel.
Set CLOAKBROWSER_BINARY_PATH to skip download and use a local build.
"""
# Check for local override first
local_override = get_local_binary_override()
if local_override:
path = Path(local_override)
if not path.exists():
raise FileNotFoundError(
f"CLOAKBROWSER_BINARY_PATH set to '{local_override}' but file does not exist"
)
logger.info("Using local binary override: %s", local_override)
return str(path)
requested_version = normalize_requested_version(browser_version)
# Pro license key check (custom download URL overrides Pro path)
from .license import (
CloakBrowserLicenseError,
resolve_license_key,
validate_license,
)
key = resolve_license_key(license_key)
if os.environ.get("CLOAKBROWSER_DOWNLOAD_URL"):
key = None
if key:
info = validate_license(key)
if info and info.valid:
# Free tier always gets the latest build. Drop any version pin: the
# server force-serves latest to free keys, so fetching a pinned
# version's signed manifest would mismatch the served bytes and fail
# checksum verification. Paid keys keep pinning/rollback.
if info.plan == "free":
requested_version = None
# A valid license is entitled to Pro, so Pro failures surface loudly
# rather than silently substituting the older free binary. (A blip
# during a routine update never reaches here: _ensure_pro_binary
# returns the cached Pro binary and updates in the background.)
try:
return _ensure_pro_binary(
key,
requested_version=requested_version,
plan=info.plan,
release_channel=release_channel,
)
except BinaryVerificationError:
# Authenticity could not be confirmed — surface verbatim.
raise
except Exception as e:
# Transient failure with no cached Pro binary to use — surface a
# clear error rather than silently downloading the free binary.
raise RuntimeError(
f"Pro binary unavailable: {e}. Your license is valid but the "
f"Pro binary could not be downloaded right now. Retry in a "
f"moment. To use the free binary instead, unset "
f"CLOAKBROWSER_LICENSE_KEY."
) from e
elif info:
# Key supplied but rejected — abort, never downgrade to free.
raise CloakBrowserLicenseError(
f"CloakBrowser Pro: license key is invalid or expired "
f"(plan={info.plan}). Check CLOAKBROWSER_LICENSE_KEY, or unset it "
f"to use the free binary."
)
else:
# Key supplied but unvalidatable (server down, no cache) — abort.
raise CloakBrowserLicenseError(
"CloakBrowser Pro: license could not be validated (server "
"unreachable and no cached validation). Retry in a moment, or "
"unset CLOAKBROWSER_LICENSE_KEY to use the free binary."
)
# Fail fast if no binary available for this platform
check_platform_available()
if requested_version:
binary_path = get_binary_path(requested_version)
if binary_path.exists() and _is_executable(binary_path):
logger.debug(
"Pinned binary found in cache: %s (version %s)",
binary_path,
requested_version,
)
_show_welcome()
return str(binary_path)
logger.info(
"Stealth Chromium %s not found. Downloading for %s...",
requested_version,
get_platform_tag(),
)
_download_and_extract(requested_version)
if not (binary_path.exists() and _is_executable(binary_path)):
raise RuntimeError(
f"Pinned download completed but binary not found at expected path: {binary_path}. "
f"This may indicate a packaging issue. Please report at "
f"https://github.com/CloakHQ/cloakbrowser/issues"
)
_show_welcome()
return str(binary_path)
# Check for auto-updated version first, then fall back to hardcoded
effective = get_effective_version()
binary_path = get_binary_path(effective)
if binary_path.exists() and _is_executable(binary_path):
logger.debug("Binary found in cache: %s (version %s)", binary_path, effective)
_show_welcome()
_maybe_trigger_update_check()
return str(binary_path)
# Fall back to platform's hardcoded version if effective version binary doesn't exist
platform_version = get_chromium_version()
if effective != platform_version:
fallback_path = get_binary_path()
if fallback_path.exists() and _is_executable(fallback_path):
logger.debug("Binary found in cache: %s", fallback_path)
_maybe_trigger_update_check()
return str(fallback_path)
# Download platform's hardcoded version
logger.info(
"Stealth Chromium %s not found. Downloading for %s...",
platform_version,
get_platform_tag(),
)
_download_and_extract()
binary_path = get_binary_path()
if not binary_path.exists():
raise RuntimeError(
f"Download completed but binary not found at expected path: {binary_path}. "
f"This may indicate a packaging issue. Please report at "
f"https://github.com/CloakHQ/cloakbrowser/issues"
)
_maybe_trigger_update_check()
return str(binary_path)
def _download_and_extract(version: str | None = None) -> None:
"""Download the binary archive and extract to cache directory.
Tries the primary server (cloakbrowser.dev) first, falls back to
GitHub Releases if the primary is unreachable or returns an error.
Verifies SHA-256 checksum before extraction when available.
"""
primary_url = get_download_url(version)
fallback_url = get_fallback_download_url(version)
binary_dir = get_binary_dir(version)
binary_path = get_binary_path(version)
# Create cache dir
binary_dir.parent.mkdir(parents=True, exist_ok=True)
# Download to temp file first (atomic — no partial downloads in cache)
with tempfile.NamedTemporaryFile(suffix=get_archive_ext(), delete=False) as tmp:
tmp_path = Path(tmp.name)
try:
# Try primary, fall back to GitHub Releases (skip fallback if custom URL)
try:
_download_file(primary_url, tmp_path)
except Exception as primary_err:
if os.environ.get("CLOAKBROWSER_DOWNLOAD_URL"):
raise
logger.warning(
"Primary download failed (%s), trying GitHub Releases...",
primary_err,
)
_download_file(fallback_url, tmp_path)
# Verify the download before extraction. On the official path this is a
# mandatory, non-bypassable Ed25519 signature check (see
# _verify_download_checksum); the skip flag only applies to custom
# self-hosted CLOAKBROWSER_DOWNLOAD_URL setups.
_verify_download_checksum(tmp_path, version)
_extract_archive(tmp_path, binary_dir, binary_path)
_show_welcome()
finally:
# Clean up temp file
tmp_path.unlink(missing_ok=True)
def _pro_binary_ready(version: str | None) -> bool:
"""True when a cached, executable Pro binary exists for ``version``."""
if not version:
return False
path = get_binary_path(version, pro=True)
return path.exists() and _is_executable(path)
def _ensure_pro_binary(
license_key: str,
requested_version: str | None = None,
plan: str | None = None,
release_channel: str | None = None,
) -> str:
"""Ensure the latest (keyed) binary is downloaded and cached. Returns its path.
Used for both free-tier GitHub keys (plan="free") and paid keys — both fetch
the same latest build; the server enforces the concurrency cap. A valid key
NEVER falls back to the older free binary: if the latest build cannot be
resolved or downloaded and none is cached, the error is raised.
"""
from .license import get_pro_latest_release, get_pro_latest_version
# Banner tier: a free GitHub key gets the free banner, paid keys the Pro one.
welcome_tier = "free" if plan == "free" else "pro"
# --- Pinned: launch the exact requested version, no server cross-check, no
# marker write (a rollback pin must not stick future unpinned launches). ---
if requested_version:
if _pro_binary_ready(requested_version):
binary_path = get_binary_path(requested_version, pro=True)
logger.debug(
"Pinned Pro binary found in cache: %s (version %s)",
binary_path,
requested_version,
)
_show_welcome(welcome_tier)
return str(binary_path)
logger.info(
"Downloading Pro Chromium %s for %s...", requested_version, get_platform_tag()
)
_download_pro_binary(requested_version, license_key)
binary_path = get_binary_path(requested_version, pro=True)
if not binary_path.exists():
raise RuntimeError(
f"Pro download completed but binary not found at: {binary_path}"
)
_show_welcome(welcome_tier)
return str(binary_path)
# --- Unpinned: track the server's latest selected channel. ---
effective = get_effective_version(pro=True, release_channel=release_channel)
# Honor CLOAKBROWSER_AUTO_UPDATE=false the way the free path does: if the user
# froze updates AND a Pro build is already cached, keep it and skip the server
# check. With no cached build we must still fetch one — a valid Pro license can
# never launch the free binary. (The `update` CLI ignores this and always acts.)
frozen = os.environ.get("CLOAKBROWSER_AUTO_UPDATE", "").lower() == "false"
if frozen and _pro_binary_ready(effective):
logger.debug("Pro auto-update disabled; using cached %s", effective)
_show_welcome(welcome_tier)
return str(get_binary_path(effective, pro=True))
# get_pro_latest_version() is rate-limited to one network call per hour and
# returns a cached string in between, so this foreground check stays cheap on
# steady-state launches while still landing the selected release after a version gap.
latest = get_pro_latest_version(release_channel)
# Tell the user when they asked for Preview but the server has no preview
# build for this platform, so the Stable build is used instead. The lookup is
# a cache hit (get_pro_latest_version above already populated it), so this
# only touches the network on the once-an-hour refresh.
if normalize_release_channel(release_channel) == "preview":
release = get_pro_latest_release(release_channel)
if release is not None and release.fallback:
_warn_preview_fallback()
# Prefer the server's latest when it is newer than — or replaces a missing —
# the cached build. Otherwise stay on the cached Pro binary (fast, offline-ok).
if latest and (
effective is None
or not _pro_binary_ready(effective)
or _version_newer(latest, effective)
):
version: str | None = latest
else:
version = effective
if version is None:
# Valid Pro license but nothing resolvable (server unreachable AND no
# cached Pro build). Never downgrade to the free binary — fail loudly.
raise RuntimeError("Could not determine latest Pro version from server")
if _pro_binary_ready(version):
binary_path = get_binary_path(version, pro=True)
# Advance the marker if this cached build is newer than what the marker names,
# so `info` (and a later server-outage launch) reflect the build we actually
# launch — never a stale marker.
if version != effective:
try:
_write_pro_version_marker(version, release_channel)
except OSError:
pass
logger.debug("Pro binary found in cache: %s (version %s)", binary_path, version)
_show_welcome(welcome_tier)
return str(binary_path)
# `version` (the server latest) needs downloading. On failure, fall back to a
# cached Pro build if we have one — never the free binary.
try:
logger.info(
"Downloading Pro Chromium %s for %s...", version, get_platform_tag()
)
_download_pro_binary(version, license_key)
except BinaryVerificationError:
# A tampering signal must surface verbatim — never mask it behind the
# cached-Pro fallback, which is only for transient download failures.
raise
except Exception:
if _pro_binary_ready(effective):
logger.warning(
"Pro update to %s failed; launching cached Pro binary %s",
version,
effective,
)
_show_welcome(welcome_tier)
return str(get_binary_path(effective, pro=True))
raise
binary_path = get_binary_path(version, pro=True)
if not binary_path.exists():
raise RuntimeError(
f"Pro download completed but binary not found at: {binary_path}"
)
# Advance the marker so future unpinned launches use this build.
try:
_write_pro_version_marker(version, release_channel)
except OSError:
pass
_show_welcome(welcome_tier)
return str(binary_path)
def _download_pro_binary(version: str, license_key: str) -> None:
"""Download a Pro binary from cloakbrowser.dev with license key auth.
Requests the explicit version so the served archive matches the signed
manifest verified in _verify_pro_download.
"""
download_url = f"{DOWNLOAD_BASE_URL}/api/download/{version}"
binary_dir = get_binary_dir(version, pro=True)
binary_path = get_binary_path(version, pro=True)
platform_tag = get_platform_tag()
binary_dir.parent.mkdir(parents=True, exist_ok=True)
with tempfile.NamedTemporaryFile(suffix=get_archive_ext(), delete=False) as tmp:
tmp_path = Path(tmp.name)
try:
_download_file(
download_url,
tmp_path,
headers={
"Authorization": f"Bearer {license_key}",
"X-Platform": platform_tag,
},
)
# Pro binaries come from cloakbrowser.dev — the same origin as free
# downloads — so the M1 attack the Ed25519 signature defends against
# applies equally. Verify with the same non-bypassable signature check;
# CLOAKBROWSER_SKIP_CHECKSUM does NOT bypass it (parity with the
# official free path).
_verify_pro_download(tmp_path, version)
_extract_archive(tmp_path, binary_dir, binary_path)
finally:
tmp_path.unlink(missing_ok=True)
def _verify_pro_download(file_path: Path, version: str) -> None:
"""Verify a Pro archive with the same non-bypassable Ed25519 signature check
as official free downloads.
Pro binaries are served from cloakbrowser.dev (same origin as the free
tier), so a tampered same-origin SHA256SUMS could otherwise certify a
tampered binary (M1, #308). Fetch the Pro SHA256SUMS + detached
SHA256SUMS.sig, verify the signature against the pinned keys FIRST, bind the
manifest to the requested version, then verify the archive's SHA-256.
An invalid signature, checksum, or version mismatch raises
BinaryVerificationError (a tampering signal the router surfaces verbatim);
CLOAKBROWSER_SKIP_CHECKSUM cannot bypass it. A failed manifest FETCH is
transient — nothing was validated — and raises a plain RuntimeError. A
valid-license user is never silently downgraded to the free binary.
"""
base = f"{DOWNLOAD_BASE_URL}/releases/pro/chromium-v{version}"
try:
manifest_resp = httpx.get(
f"{base}/SHA256SUMS", follow_redirects=True, timeout=10.0
)
manifest_resp.raise_for_status()
sig_resp = httpx.get(
f"{base}/SHA256SUMS.sig", follow_redirects=True, timeout=10.0
)
sig_resp.raise_for_status()
except Exception as exc:
# Fetch failure is transient, not tampering — raise a plain RuntimeError
# (the router reports it as "unavailable, retry") rather than a
# BinaryVerificationError (which it surfaces as a tampering signal).
raise RuntimeError(
f"Could not fetch the signed SHA256SUMS for Pro {version} ({exc})"
) from exc
manifest_bytes = manifest_resp.content
# _verify_signature / _verify_checksum raise plain RuntimeError; convert to
# BinaryVerificationError so the Pro router treats them as tampering signals
# (re-raise) rather than transient failures (fall back to free).
try:
_verify_signature(manifest_bytes, sig_resp.content)
except RuntimeError as exc:
raise BinaryVerificationError(str(exc)) from exc
manifest_text = manifest_bytes.decode("utf-8")
# Version binding: same forced-downgrade defense as the official path.
declared = _parse_manifest_version(manifest_text)
if declared != version:
raise BinaryVerificationError(
f"Version mismatch in signed Pro SHA256SUMS: requested {version}, "
f"manifest declares {declared or 'none'}. Refusing (possible downgrade)."
)
tarball_name = get_archive_name()
expected = _parse_checksums(manifest_text).get(tarball_name)
if expected is None:
raise BinaryVerificationError(
f"Signature-verified Pro SHA256SUMS has no entry for {tarball_name} — "
f"cannot confirm binary integrity."
)
try:
_verify_checksum(file_path, expected)
except RuntimeError as exc:
raise BinaryVerificationError(str(exc)) from exc
def _verify_download_checksum(file_path: Path, version: str | None = None) -> None:
"""Verify the downloaded archive's integrity and authenticity.
Official path (cloakbrowser.dev / GitHub Releases): fetch SHA256SUMS plus
its detached Ed25519 signature SHA256SUMS.sig, verify the signature against
the pinned public keys FIRST, then verify the archive's SHA-256 against the
now-authenticated manifest. Mandatory and non-bypassable — a same-origin
manifest can no longer certify a tampered binary (#308).
Custom self-hosted path (CLOAKBROWSER_DOWNLOAD_URL set): the pinned keys do
not apply to a third-party server, so fall back to the plain same-origin
SHA256SUMS check, which CLOAKBROWSER_SKIP_CHECKSUM may bypass.
"""
tarball_name = get_archive_name()
if os.environ.get("CLOAKBROWSER_DOWNLOAD_URL"):
# Self-hosted mirror: signature scheme does not apply. Preserve the
# legacy same-origin checksum behavior, skippable as before.
if os.environ.get("CLOAKBROWSER_SKIP_CHECKSUM", "").lower() == "true":
logger.warning(
"CLOAKBROWSER_SKIP_CHECKSUM set — skipping verification for custom download URL"
)
return
checksums = _fetch_checksums(version)
if checksums is None:
logger.warning(
"SHA256SUMS not available from custom URL — skipping checksum verification"
)
return
expected = checksums.get(tarball_name)
if expected is None:
logger.warning(
"SHA256SUMS found but no entry for %s — skipping verification",
tarball_name,
)
return
_verify_checksum(file_path, expected)
return
# Official path: signature is the trust root and is non-bypassable.
manifest = _fetch_signed_manifest(version)
if manifest is None:
raise RuntimeError(
"Could not fetch a signed SHA256SUMS (SHA256SUMS + SHA256SUMS.sig) "
"for this release — refusing to use an unverified binary. "
"Retry, or report at https://github.com/CloakHQ/cloakbrowser/issues"
)
manifest_bytes, sig_bytes = manifest
_verify_signature(manifest_bytes, sig_bytes)
manifest_text = manifest_bytes.decode("utf-8")
# Version binding: the signed manifest must declare the version we asked for.
# The signature proves "we made this manifest", not "this is the version you
# requested" — without this check a mirror could serve a genuinely-signed
# older release in place of the requested one (forced downgrade).
requested = version or get_chromium_version()
declared = _parse_manifest_version(manifest_text)
if declared != requested:
raise RuntimeError(
f"Version mismatch in signed SHA256SUMS: requested {requested}, "
f"manifest declares {declared or 'none'}. Refusing (possible downgrade)."
)
checksums = _parse_checksums(manifest_text)
expected = checksums.get(tarball_name)
if expected is None:
raise RuntimeError(
f"Signature-verified SHA256SUMS has no entry for {tarball_name} — "
f"cannot confirm binary integrity."
)
_verify_checksum(file_path, expected)
def _parse_manifest_version(text: str) -> str | None:
"""Read the 'version=<v>' line from a signed manifest. None if absent.
The line has no internal whitespace so older wrappers' SHA256SUMS parsers
ignore it (they only accept '<hash> <filename>' lines).
"""
for line in text.splitlines():
line = line.strip()
if line.startswith("version="):
return line[len("version=") :].strip()
return None
def _fetch_signed_manifest(version: str | None = None) -> tuple[bytes, bytes] | None:
"""Fetch (SHA256SUMS, SHA256SUMS.sig) raw bytes for a version, or None.
Both files are fetched from the SAME origin so the signature always matches
the exact manifest bytes it certifies. The primary origin is tried first,
then the GitHub Releases mirror. follow_redirects mirrors _fetch_checksums:
cloakbrowser.dev 301-redirects /chromium-v* to GitHub Releases.
"""
v = version or get_chromium_version()
bases = [
f"{DOWNLOAD_BASE_URL}/chromium-v{v}",
f"{GITHUB_DOWNLOAD_BASE_URL}/chromium-v{v}",
]
for base in bases:
try:
manifest_resp = httpx.get(
f"{base}/SHA256SUMS", follow_redirects=True, timeout=10.0
)
manifest_resp.raise_for_status()
sig_resp = httpx.get(
f"{base}/SHA256SUMS.sig", follow_redirects=True, timeout=10.0
)
sig_resp.raise_for_status()
return manifest_resp.content, sig_resp.content
except Exception:
continue
return None
def _verify_signature(manifest_bytes: bytes, sig_b64: bytes) -> None:
"""Verify a detached Ed25519 signature over the raw manifest bytes.
sig_b64 is the base64 of the 64-byte raw signature. Tries each pinned key
in BINARY_SIGNING_PUBKEYS; succeeds if any validates. Raises RuntimeError
if the signature is malformed or no pinned key validates it.
"""
import base64
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey
try:
signature = base64.b64decode(sig_b64.strip(), validate=True)
except Exception as exc:
raise RuntimeError(
f"Malformed SHA256SUMS.sig (not valid base64): {exc}"
) from exc
for pubkey_b64 in BINARY_SIGNING_PUBKEYS:
try:
pub = Ed25519PublicKey.from_public_bytes(base64.b64decode(pubkey_b64))
except Exception:
# Skip an unparsable pinned key (e.g. the placeholder) rather than
# aborting — another pinned key may still validate.
continue
try:
pub.verify(signature, manifest_bytes)
logger.info("SHA256SUMS signature verified: Ed25519 OK")
return
except Exception:
# InvalidSignature, or a malformed/wrong-length signature that makes
# verify raise something else — either way this key didn't match,
# so try the next pinned key (and ultimately fail closed below).
continue
raise RuntimeError(
"SHA256SUMS signature verification failed — no pinned key validated the "
"manifest. The binary's authenticity could not be confirmed. "
"Report at https://github.com/CloakHQ/cloakbrowser/issues"
)
def _fetch_checksums(version: str | None = None) -> dict[str, str] | None:
"""Fetch SHA256SUMS file for a version. Returns {filename: hash} or None."""
v = version or get_chromium_version()
has_custom_url = os.environ.get("CLOAKBROWSER_DOWNLOAD_URL")
# Build URL list — respect custom URL contract (no GitHub fallback)
urls = [f"{DOWNLOAD_BASE_URL}/chromium-v{v}/SHA256SUMS"]
if not has_custom_url:
urls.append(f"{GITHUB_DOWNLOAD_BASE_URL}/chromium-v{v}/SHA256SUMS")
for url in urls:
try:
resp = httpx.get(url, follow_redirects=True, timeout=10.0)
resp.raise_for_status()
return _parse_checksums(resp.text)
except Exception:
continue
return None
def _parse_checksums(text: str) -> dict[str, str]:
"""Parse SHA256SUMS format: '<64-hex sha256> filename' per line.
Only lines whose first token is a 64-character hex digest are accepted
(matches the JS parser); blank lines, the version= line, and any other
junk are ignored.
"""
result = {}
for line in text.strip().splitlines():
parts = line.strip().split(None, 1)
if len(parts) != 2:
continue
hash_val, filename = parts
hash_val = hash_val.lower()
if len(hash_val) != 64 or any(c not in "0123456789abcdef" for c in hash_val):
continue
result[filename.lstrip("*")] = hash_val
return result
def _verify_checksum(file_path: Path, expected_hash: str) -> None:
"""Verify SHA-256 of a file. Raises RuntimeError on mismatch."""
sha256 = hashlib.sha256()
try:
with open(file_path, "rb") as f:
for chunk in iter(lambda: f.read(8192), b""):
sha256.update(chunk)
except OSError:
logger.error("Failed to read downloaded file for checksum: %s", file_path)
raise
actual = sha256.hexdigest().lower()
if actual != expected_hash:
raise RuntimeError(
f"Checksum verification failed!\n"
f" Expected: {expected_hash}\n"
f" Got: {actual}\n"
f" File may be corrupted or tampered with. "
f"Please retry or report at https://github.com/CloakHQ/cloakbrowser/issues"
)
logger.info("Checksum verified: SHA-256 OK")
def _download_file(url: str, dest: Path, headers: dict[str, str] | None = None) -> None:
"""Download a file with progress logging."""
logger.info("Downloading from %s", url)
with httpx.stream(
"GET",
url,
follow_redirects=True,
timeout=DOWNLOAD_TIMEOUT,
headers=headers or {},
) as response:
response.raise_for_status()
try:
total = int(response.headers.get("content-length", 0))
except (TypeError, ValueError):
total = 0
downloaded = 0
last_logged_pct = -1
try:
with open(dest, "wb") as f:
for chunk in response.iter_bytes(chunk_size=8192):
f.write(chunk)
downloaded += len(chunk)
if total > 0:
pct = downloaded * 100 // total
# Log every 10%
if pct >= last_logged_pct + 10:
last_logged_pct = pct
logger.info(
"Download progress: %d%% (%d/%d MB)",
pct,
downloaded // (1024 * 1024),
total // (1024 * 1024),
)
except OSError:
logger.error("Failed to write downloaded file: %s", dest)
raise
logger.info("Download complete: %d MB", dest.stat().st_size // (1024 * 1024))
def _extract_archive(
archive_path: Path, dest_dir: Path, binary_path: Path | None = None
) -> None:
"""Extract tar.gz or zip archive to destination directory."""
logger.info("Extracting to %s", dest_dir)
bp = binary_path or get_binary_path()
# Extract into a private sibling dir, then rename it into place. Concurrent
# first runs (in this process or others) each get their own staging dir, so
# none of them can see or delete another's half-written install.
staging_dir = dest_dir.with_name(f"{dest_dir.name}.partial-{uuid.uuid4().hex}")
staging_dir.mkdir()
stale_dirs: list[Path] = []
try:
if str(archive_path).endswith(".zip"):
_extract_zip(archive_path, staging_dir)
else:
_extract_tar(archive_path, staging_dir)
# If extracted into a single subdirectory, flatten it
# (e.g. fingerprint-chromium-142-custom-v2/chrome → chrome)
# But never flatten .app bundles: macOS needs the bundle structure intact
_flatten_single_subdir(staging_dir)
# Make binary executable
staged_binary_path = staging_dir / bp.relative_to(dest_dir)
if staged_binary_path.exists():
_make_executable(staged_binary_path)
# macOS: remove quarantine/provenance xattrs to prevent Gatekeeper prompts
if platform.system() == "Darwin":
_remove_quarantine(staging_dir)
for attempt in range(1, INSTALL_RENAME_ATTEMPTS + 1):
try:
_rename_dir_with_retry(staging_dir, dest_dir)
break
except OSError:
# dest_dir is in the way. Keep it if another caller finished a complete
# install; replace it if it is an incomplete leftover, such as an
# interrupted in-place extraction from an older release.
if bp.exists() and _is_executable(bp):
break
if attempt == INSTALL_RENAME_ATTEMPTS:
raise
# Rename the leftover aside before deleting it. Deleting it in place races
# other callers renaming their extraction into dest_dir.
stale_dir = dest_dir.with_name(f"{dest_dir.name}.stale-{uuid.uuid4().hex}")
try:
_rename_dir_with_retry(dest_dir, stale_dir)
except FileNotFoundError:
continue # another caller moved it aside first
stale_dirs.append(stale_dir)
finally:
shutil.rmtree(staging_dir, ignore_errors=True)
for stale_dir in stale_dirs:
shutil.rmtree(stale_dir, ignore_errors=True)
if bp.exists():
logger.info("Binary ready: %s", bp)
def _rename_dir_with_retry(source: Path, destination: Path) -> None:
"""Rename a directory, retrying while Windows reports it as in use."""
for attempt in range(1, TRANSIENT_RENAME_ATTEMPTS + 1):
try:
source.rename(destination)
return
except PermissionError:
# Windows antivirus briefly holds freshly written files open, which fails the
# rename with access denied. An existing destination is not transient.
if (
platform.system() != "Windows"
or attempt == TRANSIENT_RENAME_ATTEMPTS
or destination.exists()
):
raise
time.sleep(TRANSIENT_RENAME_DELAY)
def _extract_tar(archive_path: Path, dest_dir: Path) -> None:
"""Extract tar.gz. Archive is signature-verified before extraction, so unpack
as-is. filter="fully_trusted" pins legacy behavior (3.14 defaults to "data",
which mangles macOS .app symlinks/perms); fallback covers Python < 3.12."""
with tarfile.open(archive_path, "r:gz") as tar:
try:
tar.extractall(dest_dir, filter="fully_trusted")
except TypeError:
tar.extractall(dest_dir)
def _extract_zip(archive_path: Path, dest_dir: Path) -> None:
"""Extract zip. Signature-verified before extraction, so unpack as-is."""
import zipfile
with zipfile.ZipFile(archive_path, "r") as zf:
zf.extractall(dest_dir)
def _flatten_single_subdir(dest_dir: Path) -> None:
"""If extraction created a single subdirectory, move its contents up.
Many tar archives wrap files in a top-level directory (e.g.
fingerprint-chromium-142-custom-v2/chrome). We want chrome at dest_dir/chrome.
"""
entries = list(dest_dir.iterdir())
if len(entries) == 1 and entries[0].is_dir():
subdir = entries[0]
# Never flatten .app bundles — macOS needs the bundle structure
if subdir.name.endswith(".app"):
logger.debug("Keeping .app bundle intact: %s", subdir.name)
return
logger.debug("Flattening single subdirectory: %s", subdir.name)
try:
for item in subdir.iterdir():
shutil.move(str(item), str(dest_dir / item.name))
subdir.rmdir()
except OSError:
logger.error("Failed to flatten extracted directory: %s", subdir)
raise
def _is_executable(path: Path) -> bool:
"""Check if a file is executable."""
try:
return os.access(path, os.X_OK)
except OSError:
logger.error("Failed to check executable permissions: %s", path)
return False
def _make_executable(path: Path) -> None:
"""Make a file executable (chmod +x). Skipped on Windows (no-op / AV lock risk)."""
if platform.system() == "Windows":
return
current = path.stat().st_mode
path.chmod(current | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH)
def _remove_quarantine(path: Path) -> None:
"""Remove macOS quarantine/provenance xattrs so Gatekeeper doesn't block the binary."""
try:
subprocess.run(
["xattr", "-cr", str(path)],
capture_output=True,
timeout=30,
)
logger.debug("Removed quarantine attributes from %s", path)
except Exception:
logger.debug("Failed to remove quarantine attributes", exc_info=True)
def clear_cache() -> None:
"""Remove all cached binaries. Forces re-download on next launch."""
from .config import get_cache_dir
cache_dir = get_cache_dir()
if cache_dir.exists():
try:
shutil.rmtree(cache_dir)
except OSError:
logger.error("Failed to clear cache: %s", cache_dir)
raise
logger.info("Cache cleared: %s", cache_dir)
def binary_info(
browser_version: str | None = None,
release_channel: str | None = None,
) -> dict:
"""Return info about the current binary installation.
tier reflects what is actually installed on disk, not merely whether a
license is cached — a cached license with no Pro binary downloaded yet is
still effectively running the free binary, and the active key may differ
from the cached one.
browser_version (or CLOAKBROWSER_VERSION) pins the reported version so the
info matches what a pinned launch actually runs, instead of latest.
"""
requested = normalize_requested_version(browser_version)
# Prefer Pro only if a Pro binary actually exists on disk. get_effective_version
# returns None for Pro when nothing is cached (it never falls back to free).
pro_version = requested or get_effective_version(
pro=True, release_channel=release_channel
)
pro = _pro_binary_ready(pro_version) # already false for a None version
if pro:
effective = pro_version
binary_path = get_binary_path(pro_version, pro=True)
else:
effective = requested or get_effective_version()
binary_path = get_binary_path(effective)
if pro:
download_url = f"{DOWNLOAD_BASE_URL}/api/download/latest"
if normalize_release_channel(release_channel) == "preview":
download_url += "?channel=preview"
else:
download_url = get_download_url(effective)
return {
"version": effective,
"tier": "pro" if pro else "free",
"bundled_version": CHROMIUM_VERSION,
"platform": get_platform_tag(),
"binary_path": str(binary_path),
"installed": binary_path.exists(),
"cache_dir": str(get_binary_dir(effective, pro=pro)),
"download_url": download_url,
}
# ---------------------------------------------------------------------------
# Auto-update
# ---------------------------------------------------------------------------
def check_for_update() -> str | None:
"""Manually check for a newer Chromium version. Returns new version or None.
This is the public API for triggering an update check. Unlike the
background check in ensure_binary(), this blocks until complete.
"""
latest = _get_latest_chromium_version()
if latest is None:
return None
if not _version_newer(latest, get_chromium_version()):
return None
binary_dir = get_binary_dir(latest)
if binary_dir.exists():
# Already downloaded
_write_version_marker(latest)
return latest
logger.info("Downloading Chromium %s...", latest)
_download_and_extract(version=latest)
_write_version_marker(latest)
return latest
def check_for_pro_update(
license_key: str, release_channel: str | None = None
) -> str | None:
"""Move a Pro install to the selected channel's latest. Blocks until complete.
Returns the new version when a newer Pro build is downloaded or an
already-cached newer build is activated, else None (already up to date or the
server could not be reached). Requires a valid Pro license key.
"""
from .license import get_pro_latest_version
latest = get_pro_latest_version(release_channel)
if not latest:
return None
effective = get_effective_version(pro=True, release_channel=release_channel)
if effective and not _version_newer(latest, effective) and _pro_binary_ready(
effective
):
# Already on the latest cached Pro build.
return None
if not _pro_binary_ready(latest):
logger.info("Downloading Pro Chromium %s...", latest)
_download_pro_binary(latest, license_key)
binary_path = get_binary_path(latest, pro=True)
if not binary_path.exists():
raise RuntimeError(
f"Pro download completed but binary not found at: {binary_path}"
)
_write_pro_version_marker(latest, release_channel)
return latest
def _should_check_for_update() -> bool:
"""Check if auto-update is enabled and rate limit hasn't been hit."""
if os.environ.get("CLOAKBROWSER_AUTO_UPDATE", "").lower() == "false":
return False
if get_local_binary_override():
return False
if os.environ.get("CLOAKBROWSER_DOWNLOAD_URL"):
return False
check_file = get_cache_dir() / ".last_update_check"
if check_file.exists():
try:
last_check = float(check_file.read_text().strip())
if time.time() - last_check < UPDATE_CHECK_INTERVAL:
return False
except (ValueError, OSError):
pass
return True
def _get_latest_chromium_version() -> str | None:
"""Hit GitHub Releases API, return latest chromium-v* version for this platform.
Checks that the release has a binary asset for the current platform,
so Linux-only releases won't be offered to macOS users.
"""
try:
resp = httpx.get(GITHUB_API_URL, params={"per_page": 10}, timeout=10.0)
resp.raise_for_status()
platform_tarball = get_archive_name()
for release in resp.json():
tag = release.get("tag_name", "")
if tag.startswith("chromium-v") and not release.get("draft"):
asset_names = {a["name"] for a in release.get("assets", [])}
if platform_tarball in asset_names:
return tag.removeprefix("chromium-v")
return None
except Exception:
logger.debug("Auto-update check failed", exc_info=True)
return None
def _write_version_marker(version: str) -> None:
"""Write the latest version marker for this platform to cache dir."""
cache_dir = get_cache_dir()
cache_dir.mkdir(parents=True, exist_ok=True)
marker = cache_dir / f"latest_version_{get_platform_tag()}"
# Write to temp file then rename for atomicity
tmp = marker.with_suffix(".tmp")
tmp.write_text(version)
tmp.rename(marker)
def _write_pro_version_marker(
version: str, release_channel: str | None = None
) -> None:
"""Atomically write the selected channel's Pro version marker."""
cache_dir = get_cache_dir()
cache_dir.mkdir(parents=True, exist_ok=True)
channel = normalize_release_channel(release_channel)
marker_prefix = (
"latest_pro_version_preview"
if channel == "preview"
else "latest_pro_version"
)
marker = cache_dir / f"{marker_prefix}_{get_platform_tag()}"
tmp = marker.with_suffix(".tmp")
tmp.write_text(version)
os.replace(str(tmp), str(marker))
_wrapper_update_checked = False
def _check_wrapper_update() -> None:
"""Check PyPI for a newer wrapper version. Runs once per process."""
global _wrapper_update_checked
if _wrapper_update_checked:
return
_wrapper_update_checked = True
if os.environ.get("CLOAKBROWSER_AUTO_UPDATE", "").lower() == "false":
return
if os.environ.get("CLOAKBROWSER_DOWNLOAD_URL"):
return
try:
resp = httpx.get(
"https://pypi.org/pypi/cloakbrowser/json",
timeout=5.0,
)
resp.raise_for_status()
latest = resp.json()["info"]["version"]
if _version_newer(latest, _wrapper_version):
logger.warning(
"Update available: cloakbrowser %s -> %s. "
"Run: pip install --upgrade cloakbrowser",
_wrapper_version,
latest,
)
except Exception:
logger.debug("Wrapper update check failed", exc_info=True)
def _check_and_download_update() -> None:
"""Background task: check for newer binary, download if available."""
try:
# Record check timestamp first (rate limiting)
check_file = get_cache_dir() / ".last_update_check"
check_file.parent.mkdir(parents=True, exist_ok=True)
check_file.write_text(str(time.time()))
platform_version = get_chromium_version()
latest = _get_latest_chromium_version()
if latest is None:
return
if not _version_newer(latest, platform_version):
return
# Already downloaded?
if get_binary_dir(latest).exists():
_write_version_marker(latest)
return
logger.info(
"Newer Chromium available: %s (current: %s). Downloading in background...",
latest,
platform_version,
)
_download_and_extract(version=latest)
_write_version_marker(latest)
logger.info(
"Background update complete: Chromium %s ready. Will use on next launch.",
latest,
)
except Exception:
logger.debug("Background update failed", exc_info=True)
def _maybe_trigger_update_check() -> None:
"""Fire-and-forget update check in a daemon thread."""
# Wrapper update: once per process, not rate-limited
if not _wrapper_update_checked:
t = threading.Thread(target=_check_wrapper_update, daemon=True)
t.start()
# Binary update: rate-limited to once per hour
if not _should_check_for_update():
return
t = threading.Thread(target=_check_and_download_update, daemon=True)
t.start()
Reference in new issue View Git Blame Copy Permalink