mirror of
https://github.com/storytold/lightcraft.git
synced 2026-10-19 02:10:25 -05:00
Merge PR #1614
Resolved against rb-base-36: docs combine the 1 px clip neighbourhood and frame order with the base's moving-highlight rule and 2048 px deghost. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
commit
3c0300fcbb
5 files changed
+149
-15
No files matched your search
+73
-13
@@ -1,18 +1,20 @@
|
||||
//! HDR merge of exposure brackets.
|
||||
//!
|
||||
//! 1. **Order and reference.** Frames are ordered by brightness (EXIF exposure, or median
|
||||
//! luminance without EXIF); the middle one is the reference (geometry, colour and the output's
|
||||
//! default brightness come from it).
|
||||
//! luminance without EXIF; ties by content); the middle one is the reference (geometry, colour
|
||||
//! and the output's default brightness come from it). Everything below runs in that order, so
|
||||
//! the result is the same whatever order the frames come in.
|
||||
//! 2. **Auto align** ([`crate::align`]): each frame is registered to the reference and resampled
|
||||
//! into its geometry.
|
||||
//! 3. **Exposure normalisation.** The exposure ratio between brightness-adjacent frames is
|
||||
//! measured robustly — the median of log ratios over pixels well exposed in both — and chained
|
||||
//! to the reference; EXIF is the fallback when too few pixels overlap. Radiance
|
||||
//! `Rᵢ = vᵢ / kᵢ` is expressed in reference units.
|
||||
//! 4. **Weights.** `wᵢ = kᵢ · (1 − smoothstep(0.80·clip, 0.96·clip, max channel))`: longer
|
||||
//! exposures get more weight (shot-noise optimal for a linear sensor), saturated pixels none
|
||||
//! (all channels together, so a clipped channel never shifts colour). The darkest frame keeps a
|
||||
//! tiny weight so fully saturated pixels still get a value. Where the reference is at or near its clip, the scene is
|
||||
//! 4. **Weights.** `wᵢ = kᵢ · (1 − smoothstep(0.80·clip, 0.96·clip, max channel within 1 px))`:
|
||||
//! longer exposures get more weight (shot-noise optimal for a linear sensor), saturated pixels
|
||||
//! none (all channels together, so a clipped channel never shifts colour), nor their neighbours,
|
||||
//! which demosaicing interpolated from clipped photosites. The darkest frame keeps a tiny weight
|
||||
//! so fully saturated pixels still get a value. Where the reference is at or near its clip, the scene is
|
||||
//! at least that bright: a frame reading far darker there saw something that moved (a star, the moon, a light) and is
|
||||
//! left out, so a clipped highlight of the reference is never replaced by content from another moment.
|
||||
//! 5. **Deghost** (None/Low/Medium/High): on a ≤ 2048 px version, a per-pixel consistency test
|
||||
@@ -133,9 +135,25 @@ pub fn merge_hdr(mut frames: Vec<Frame>, opts: &HdrOptions, progress: &Progress)
|
||||
Some(e) if e.iter().any(|v| (v - e[0]).abs() > 0.1) => e.clone(),
|
||||
_ => medians.iter().map(|m| (*m as f64).log2()).collect(),
|
||||
};
|
||||
let mut order: Vec<usize> = (0..n).collect();
|
||||
order.sort_by(|&a, &b| brightness[a].total_cmp(&brightness[b]).then(a.cmp(&b)));
|
||||
let reference = order[n / 2];
|
||||
// ties (equal brightness) by content, never by position
|
||||
let fingerprint: Vec<f64> =
|
||||
frames.par_iter().map(|f| f.image.data.iter().step_by(997).map(|p| p[0] as f64 + 2.0 * p[1] as f64 + 3.0 * p[2] as f64).sum()).collect();
|
||||
let mut input: Vec<usize> = (0..n).collect();
|
||||
input.sort_by(|&a, &b| {
|
||||
brightness[a].total_cmp(&brightness[b]).then(medians[a].total_cmp(&medians[b])).then(fingerprint[a].total_cmp(&fingerprint[b]))
|
||||
});
|
||||
// From here on frames are in brightness order (`input[p]` is the caller's index of position `p`): alignment seeds,
|
||||
// sums and deghost masks no longer depend on the order the caller passed. Per-frame results go back at the end.
|
||||
let mut slots: Vec<Option<Frame>> = frames.into_iter().map(Some).collect();
|
||||
let mut frames: Vec<Frame> = input.iter().filter_map(|&i| slots.get_mut(i).and_then(Option::take)).collect();
|
||||
if frames.len() != n {
|
||||
return Err(MergeError::Mismatch("HDR merge lost a frame while ordering them".into()));
|
||||
}
|
||||
let exif: Option<Vec<f64>> = exif.map(|e| input.iter().map(|&i| e[i]).collect());
|
||||
let medians: Vec<f32> = input.iter().map(|&i| medians[i]).collect();
|
||||
let brightness: Vec<f64> = input.iter().map(|&i| brightness[i]).collect();
|
||||
let order: Vec<usize> = (0..n).collect();
|
||||
let reference = n / 2;
|
||||
|
||||
// 2. alignment
|
||||
let mut alignments = vec![Alignment::identity(); n];
|
||||
@@ -206,6 +224,8 @@ pub fn merge_hdr(mut frames: Vec<Frame>, opts: &HdrOptions, progress: &Progress)
|
||||
|
||||
let darkest = order[0];
|
||||
let clips: Vec<f32> = frames.iter().map(|f| f.clip).collect();
|
||||
// how close each pixel of each frame is to saturation: its brightest channel within 1 px (see `saturation`)
|
||||
let sat: Vec<Vec<f32>> = frames.par_iter().map(|f| saturation(&f.image)).collect();
|
||||
let mut out = Rgb32f::new(w, h);
|
||||
let fr = &frames;
|
||||
let vd = &valid;
|
||||
@@ -223,7 +243,7 @@ pub fn merge_hdr(mut frames: Vec<Frame>, opts: &HdrOptions, progress: &Progress)
|
||||
continue;
|
||||
}
|
||||
let v = fr[i].image.data[i_px];
|
||||
let m = maxc(v);
|
||||
let m = sat[i][i_px];
|
||||
let mut wt = k[i] * (1.0 - smoothstep(0.80 * clips[i], 0.96 * clips[i], m));
|
||||
if i == darkest {
|
||||
wt += 1e-6;
|
||||
@@ -272,14 +292,28 @@ pub fn merge_hdr(mut frames: Vec<Frame>, opts: &HdrOptions, progress: &Progress)
|
||||
}
|
||||
None => Plane::new(0, 0),
|
||||
};
|
||||
// per-frame results in the caller's order
|
||||
let mut ev_in = vec![0.0; n];
|
||||
let mut alignments_in = vec![Alignment::identity(); n];
|
||||
for (p, &i) in input.iter().enumerate() {
|
||||
ev_in[i] = ev[p];
|
||||
alignments_in[i] = alignments[p].clone();
|
||||
}
|
||||
let ev_exif = ev_exif.map(|e: Vec<f64>| {
|
||||
let mut v = vec![0.0; n];
|
||||
for (p, &i) in input.iter().enumerate() {
|
||||
v[i] = e[p];
|
||||
}
|
||||
v
|
||||
});
|
||||
let rf = &frames[reference];
|
||||
Ok(HdrResult {
|
||||
radiance: out,
|
||||
ghost,
|
||||
reference,
|
||||
ev,
|
||||
reference: input[reference],
|
||||
ev: ev_in,
|
||||
ev_exif,
|
||||
alignments,
|
||||
alignments: alignments_in,
|
||||
color: rf.color.clone(),
|
||||
raw: rf.raw,
|
||||
orientation: rf.orientation,
|
||||
@@ -461,3 +495,29 @@ fn dilate(p: &Plane, r: usize) -> Plane {
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// A frame's saturation per pixel: the brightest channel within 1 px (3 × 3). A demosaiced pixel next to clipped
|
||||
/// photosites is interpolated from them and reads low in the channel that clipped (green, in camera RGB), so it loses
|
||||
/// its weight with them. On real ARW brackets the brightest frame's green was 3–5 % low within 2 px of clipped pixels
|
||||
/// (and linear to 1 % elsewhere), and as that frame carries the most weight, bright windows got magenta fringes.
|
||||
fn saturation(img: &Rgb32f) -> Vec<f32> {
|
||||
let (w, h) = (img.width, img.height);
|
||||
let m: Vec<f32> = img.data.iter().map(|p| maxc(*p)).collect();
|
||||
if w == 0 || h == 0 {
|
||||
return m;
|
||||
}
|
||||
let mut rows = vec![0f32; w * h];
|
||||
rows.par_chunks_mut(w).zip(m.par_chunks(w)).for_each(|(out, src)| {
|
||||
for (x, o) in out.iter_mut().enumerate() {
|
||||
*o = src[x.saturating_sub(1)..=(x + 1).min(w - 1)].iter().copied().fold(0.0, f32::max);
|
||||
}
|
||||
});
|
||||
let mut out = vec![0f32; w * h];
|
||||
out.par_chunks_mut(w).enumerate().for_each(|(y, row)| {
|
||||
let (a, b) = (y.saturating_sub(1), (y + 1).min(h - 1));
|
||||
for (x, v) in row.iter_mut().enumerate() {
|
||||
*v = (a..=b).map(|yy| rows[yy * w + x]).fold(0.0, f32::max);
|
||||
}
|
||||
});
|
||||
out
|
||||
}
|
||||
@@ -125,6 +125,69 @@ fn bracket_merge_recovers_radiance_and_alignment() {
|
||||
assert!(rmed < 0.05, "recovered highlights: median log error {rmed}");
|
||||
}
|
||||
|
||||
/// The result doesn't depend on the order the frames come in: every permutation of a bracket gives the same radiance and
|
||||
/// deghost overlay, bit for bit, and the same per-frame results for each frame. (Real brackets merged in the app's
|
||||
/// selection order and in the CLI's file order differed on 0.6 % of their pixels: alignment seeds followed the input
|
||||
/// positions.)
|
||||
#[test]
|
||||
fn merge_does_not_depend_on_the_frame_order() {
|
||||
let t = truth();
|
||||
let shifts = [(3.25f32, -2.0f32), (0.0, 0.0), (-4.0, 1.5)];
|
||||
let ks = [0.25f32, 1.0, 4.0];
|
||||
let shot: Vec<Frame> = (0..3).map(|i| shoot(&t, ks[i], shifts[i].0, shifts[i].1, i as u64 + 1)).collect();
|
||||
let opts = HdrOptions { align: true, deghost: Deghost::Medium };
|
||||
let base = merge_hdr(shot.clone(), &opts, &no_progress).unwrap();
|
||||
assert_eq!(base.reference, 1);
|
||||
for perm in [[2usize, 0, 1], [1, 2, 0], [0, 2, 1]] {
|
||||
let r = merge_hdr(perm.iter().map(|&i| shot[i].clone()).collect(), &opts, &no_progress).unwrap();
|
||||
assert!(r.radiance.data == base.radiance.data, "{perm:?}: the radiance differs");
|
||||
assert!(r.ghost.data == base.ghost.data, "{perm:?}: the deghost overlay differs");
|
||||
assert_eq!(perm[r.reference], base.reference, "{perm:?}");
|
||||
for (p, &i) in perm.iter().enumerate() {
|
||||
assert_eq!(r.ev[p], base.ev[i], "{perm:?}");
|
||||
assert_eq!(r.alignments[p].h, base.alignments[i].h, "{perm:?}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Pixels demosaiced next to clipped photosites read low in the clipped channel (real ARW brackets: the brightest
|
||||
/// frame's green 3–5 % low within 2 px of clipped pixels, linear to 1 % elsewhere); the merge must not take them from
|
||||
/// that frame, or bright windows get magenta fringes. Here the +2 EV frame's green is 5 % low around its clipped areas.
|
||||
#[test]
|
||||
fn pixels_next_to_clipped_ones_do_not_tint_the_merge() {
|
||||
let t = truth();
|
||||
let ks = [0.25f32, 1.0, 4.0];
|
||||
let mut frames: Vec<Frame> = (0..3).map(|i| shoot(&t, ks[i], 0.0, 0.0, i as u64 + 1)).collect();
|
||||
let (w, h) = (frames[2].image.width, frames[2].image.height);
|
||||
let clipped = |img: &Rgb32f, x: usize, y: usize| img.data[y * w + x].iter().any(|&v| v >= 1.0);
|
||||
let mut ring = vec![false; w * h];
|
||||
for y in 1..h - 1 {
|
||||
for x in 1..w - 1 {
|
||||
let img = &frames[2].image;
|
||||
ring[y * w + x] = !clipped(img, x, y) && (y - 1..=y + 1).any(|yy| (x - 1..=x + 1).any(|xx| clipped(img, xx, yy)));
|
||||
}
|
||||
}
|
||||
for (p, &r) in frames[2].image.data.iter_mut().zip(&ring) {
|
||||
if r {
|
||||
p[1] *= 0.95;
|
||||
}
|
||||
}
|
||||
let r = merge_hdr(frames, &HdrOptions { align: false, deghost: Deghost::None }, &no_progress).unwrap();
|
||||
let ratio = |p: [f32; 3]| p[1] / (p[0] + p[2]);
|
||||
let mut devs = Vec::new();
|
||||
for y in 1..h - 1 {
|
||||
for x in 1..w - 1 {
|
||||
let want = t.get(x + M, y + M);
|
||||
if ring[y * w + x] && want[0] > 0.01 && want[2] > 0.01 {
|
||||
devs.push(ratio(r.radiance.get(x, y)) / ratio(want) - 1.0);
|
||||
}
|
||||
}
|
||||
}
|
||||
assert!(devs.len() > 500, "ring pixels: {}", devs.len());
|
||||
let med = pct(devs, 0.5);
|
||||
assert!(med.abs() < 0.01, "green against red and blue on the ring: {:+.2} %", 100.0 * med);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deghost_removes_a_moving_object() {
|
||||
let t = truth();
|
||||
|
||||
+4
-1
@@ -30,7 +30,10 @@ reference. Auto Align: features on exposure-normalised luminance, RANSAC transla
|
||||
homography (the simplest model that explains ≥ 97 % of the matches). Exposure ratios are measured
|
||||
(median log ratio of well-exposed pixels between brightness neighbours), EXIF is the fallback.
|
||||
Radiance is a weighted mean in linear light, weight `k · (1 − smoothstep(0.8·clip, 0.96·clip, max
|
||||
channel))`. Where the reference is at or near its clip the scene is at least that bright, so a frame that reads far
|
||||
channel within 1 px))`: a pixel next to clipped photosites was demosaiced from them and reads low in the clipped
|
||||
channel, so it goes with them (on real ARW brackets the brightest frame's green was 3–5 % low there, and bright
|
||||
windows took a magenta tinge). The frames are taken in brightness order (ties by content), so the result doesn't
|
||||
depend on the order they come in. Where the reference is at or near its clip the scene is at least that bright, so a frame that reads far
|
||||
darker there (an object moved: a star, the moon, a passing light) is left out and the clipped highlight stays (always
|
||||
on: a moving star otherwise left a black hole). Deghost: a per-pixel consistency test on ≤ 2048 px versions against the
|
||||
reference radiance (with a noise floor; the reference predicts down to 0.1 % of its clip, so a brighter frame's ghost in
|
||||
|
||||
@@ -353,7 +353,7 @@ differences and missing bindings are in [ui-parity.md](ui-parity.md#shortcuts).
|
||||
|
||||
| Id | Feature | Tier | Status | Evidence | Missing / notes |
|
||||
|---|---|---|---|---|---|
|
||||
| LR-MERGE-HDR | HDR merge | P2 | 🟡 | `cmd:merge.hdr`, `cmd:dialog.mergeHdr`, `crates/merge/src/frame.rs` (`Frame::lens`), `crates/merge/src/hdr.rs` | auto align, deghost None–High + overlay, auto settings, float DNG, Create Stack; the DNG keeps the reference frame's embedded lens corrections (tested by `hdr_merge_keeps_the_reference_frames_lens_corrections`); raws without colour matrices of their own (CR2, NEF, ARW…) keep the colour they open with, written as DNG tags (`starting_colour_tags`; the look's chroma curve has no tag; panoramas too), issue #965. Tested on real Sony A7R VI brackets (67 MP, 2026-10-14): frames agree with the merged radiance within ~1 % in all channels, exposure steps match EXIF within 0.03 EV. A clipped highlight of the reference that moved in other frames (a star at 200 mm) stays instead of becoming a black hole, and deghost (≤ 2048 px maps) catches a brighter frame's ghost in a dark sky (`a_clipped_star_of_the_reference_is_not_erased_by_frames_where_it_moved`, `deghost_removes_a_bright_frames_ghost_in_a_dark_sky`). 🟡 because: AI Denoise doesn't run on merged DNGs (Bayer only; Lightroom's does), JPEG brackets are treated as linear (no camera response curve), and every frame is held as 32-bit float (~0.86 GB per 67 MP frame, so 7 frames need ~7 GB); gap 29 |
|
||||
| LR-MERGE-HDR | HDR merge | P2 | 🟡 | `cmd:merge.hdr`, `cmd:dialog.mergeHdr`, `crates/merge/src/frame.rs` (`Frame::lens`), `crates/merge/src/hdr.rs` | auto align, deghost None–High + overlay, auto settings, float DNG, Create Stack; the DNG keeps the reference frame's embedded lens corrections (tested by `hdr_merge_keeps_the_reference_frames_lens_corrections`); the same result whatever order the frames come in (`merge_does_not_depend_on_the_frame_order`); pixels next to clipped ones lose their weight with them (`pixels_next_to_clipped_ones_do_not_tint_the_merge`); raws without colour matrices of their own (CR2, NEF, ARW…) keep the colour they open with, written as DNG tags (`starting_colour_tags`; the look's chroma curve has no tag; panoramas too), issue #965. Tested on real Sony A7R VI brackets (67 MP, 2026-10-14): frames agree with the merged radiance within ~1 % in all channels, exposure steps match EXIF within 0.03 EV. A clipped highlight of the reference that moved in other frames (a star at 200 mm) stays instead of becoming a black hole, and deghost (≤ 2048 px maps) catches a brighter frame's ghost in a dark sky (`a_clipped_star_of_the_reference_is_not_erased_by_frames_where_it_moved`, `deghost_removes_a_bright_frames_ghost_in_a_dark_sky`). 🟡 because: AI Denoise doesn't run on merged DNGs (Bayer only; Lightroom's does), JPEG brackets are treated as linear (no camera response curve), and every frame is held as 32-bit float (~0.86 GB per 67 MP frame, so 7 frames need ~7 GB); gap 29 |
|
||||
| LR-MERGE-PANO | Panorama | P2 | ✅ | `cmd:merge.panorama`, `cmd:dialog.mergePanorama` | spherical/cylindrical/perspective + auto, boundary warp, auto crop, fill edges (diffusion), DNG; no lens model / 360° wrap |
|
||||
| LR-MERGE-HDRPANO | HDR panorama | P2 | ✅ | `cmd:merge.hdrPanorama`, `cmd:dialog.mergeHdrPanorama` | brackets grouped by EXIF |
|
||||
| LR-MERGE-HEADLESS | Merge with last settings | P2 | ✅ | `cmd:merge.hdrLast`, `cmd:merge.panoramaLast`, `cmd:merge.hdrPanoramaLast`, `cmd:merge.hdrBatch`, `crates/catalog/src/stacks.rs` (`bracket_sets`) | no dialog; the options of the last merge of that kind (defaults the first time). HDR for Each Bracket Set merges a batch like Lightroom's merge of stacks: one HDR per stack among the photos, then per bracket (photos taken less than a second apart, after the shutter time, at exposures ≥ ½ EV apart; bursts at one exposure are skipped), one after the other in the activity stack (cancelling one stops the rest; a failed set doesn't); CLI `merge hdr-batch FILES…` |
|
||||
@@ -797,6 +797,7 @@ Latin stays upright. Advanced Japanese composition remains open. Regression cove
|
||||
|
||||
| Date | Change | Summary |
|
||||
|---|---|---|
|
||||
| 2026-10-14 | patch | LR-MERGE-HDR: the merge no longer depends on the order of its frames, and pixels next to clipped photosites lose their weight with them (no magenta fringes along clipped windows); status unchanged (🟡) |
|
||||
| 2026-10-14 | minor | LR-IMP-CAMERA-COVERAGE: 17 uncompressed Olympus and Samsung Adobe models verified on one CC0 file each; status unchanged (🟡) |
|
||||
| 2026-10-13 | minor | LR-EDIT-SECTION-SOLO: Solo Mode for Edit sections (Window ▸ Edit Sections, section-header menu, ⌥-click); KEYC-PANELS notes it |
|
||||
| 2026-10-14 | minor | LR-MERGE-HDR ✅ → 🟡: tested on real 67 MP Sony brackets; moving clipped highlights (black-hole stars) and dark-sky ghosts fixed, merge memory cut ~25 %; AI Denoise on merged DNGs, JPEG response curves and per-frame memory remain (gap 29). LR-MERGE-HEADLESS, MENU-PHOTO-MERGE: HDR for Each Bracket Set (`merge.hdrBatch`) |
|
||||
|
||||
@@ -121,6 +121,13 @@
|
||||
- A spot with Feather at 0 no longer draws black pixels exactly on its radius. This also fixes spots at 0% Opacity
|
||||
changing pixels along their edge. The hard edge now leaves pixels on the radius unchanged.
|
||||
|
||||
### Cleaner HDR merges
|
||||
- Bright windows in an HDR made with Photo ▸ Photo Merge ▸ HDR no longer take a magenta tinge from the brighter
|
||||
photos along the areas they clipped: right next to a clipped area, a photo's colour isn't used. It showed most when
|
||||
the highlights were darkened or saturated.
|
||||
- The same photos give the same HDR whatever order they are selected in; before, a different order could change a
|
||||
few pixels.
|
||||
|
||||
### HDR merges keep the white balance
|
||||
- An HDR merged from raws that LightCraft colours from the camera's JPEG (CR2, NEF, ARW…) now opens at the same
|
||||
As Shot temperature and tint as its photos, and changing the white balance no longer turns it green. Sony ARW
|
||||
|
||||
Reference in new issue
Block a user