Complete rustpython-unicode isolation: case mapping, casing predicates, sre parity - #8237

crates/unicode/src/case.rs (1)

139-144: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Extract shared surrogate-passthrough logic to reduce duplication.

The surrogate handling in capitalize_wtf8 (lines 139-144) duplicates the three-line pattern in map_wtf8 (lines 192-196), differing only by the first = false side-effect. Extracting a helper would eliminate the copy.

Proposed refactor

+fn push_surrogate(out: &mut Vec<u8>, c: CodePoint) {
+    let mut buf = Wtf8Buf::new();
+    buf.push(c);
+    out.extend_from_slice(buf.as_bytes());
+}
+
 fn capitalize_wtf8(text: &Wtf8) -> Wtf8Buf {
     // ...
             Wtf8Chunk::Surrogate(c) => {
                 first = false;
-                let mut buf = Wtf8Buf::new();
-                buf.push(c);
-                out.extend_from_slice(buf.as_bytes());
+                push_surrogate(&mut out, c);
             }
+
+fn map_wtf8(text: &Wtf8, f: impl Fn(&str, &mut FmtWriter<'_>)) -> Wtf8Buf {
+    // ...
+            Wtf8Chunk::Surrogate(c) => {
-                let mut buf = Wtf8Buf::new();
-                buf.push(c);
-                out.extend_from_slice(buf.as_bytes());
+                push_surrogate(&mut out, c);
             }

Also applies to: 192-196

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/unicode/src/case.rs` around lines 139 - 144, The surrogate passthrough
logic is duplicated in capitalize_wtf8 and map_wtf8; extract the shared
Wtf8Chunk::Surrogate handling into a helper so both paths reuse it, while
preserving the first = false side-effect only in capitalize_wtf8. Update the
branches in those functions to call the shared helper and keep the existing
behavior identical otherwise.

crates/unicode/tests/differential.rs (1)

312-346: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Add regression check to regen_mapping_version_skew.

regen_version_skew (line 202-208) refuses to record cpython=true/crate=false divergences as version skew, treating them as regressions (with a KNOWN_RECATEGORIZATIONS exception). regen_mapping_version_skew has no equivalent guard — a mapping regression (CPython maps a code point, crate doesn't) would be silently written to the skew file without warning.

The simple_mappings_match_cpython_except_documented_version_skew test is the ultimate safety net, but adding a parallel assertion here would catch regressions at regen time, consistent with the predicate path.

Proposed fix

     let reference = parse_mappings(MAPPINGS);
     let divergences = all_mapping_divergences(&reference);
 
+    // A `cpython maps, crate doesn't` divergence means a code point lost its
+    // mapping in a later Unicode release — a real regression, not version skew.
+    let regressions: Vec<_> = divergences
+        .iter()
+        .filter(|(name, cp)| {
+            let expected = reference.get(name).and_then(|t| t.get(cp)).copied().unwrap_or(*cp);
+            expected != *cp && crate_mapping(name, *cp) == *cp
+        })
+        .collect();
+    assert!(
+        regressions.is_empty(),
+        "refusing to record {} mapping regression(s) — these are regressions, \
+         not version skew: {:?}",
+        regressions.len(),
+        &regressions[..regressions.len().min(20)]
+    );
+
     let mut by_mapping: BTreeMap<String, BTreeSet<u32>> = BTreeMap::new();

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/unicode/tests/differential.rs` around lines 312 - 346, Add the same
regression guard used by regen_version_skew to regen_mapping_version_skew so
mapping regressions are not silently written into the skew file. In
regen_mapping_version_skew, before building the output body, inspect the
divergences from all_mapping_divergences and assert that any
cpython=true/crate=false cases are either absent or explicitly covered by the
existing KNOWN_RECATEGORIZATIONS-style exception, then fail fast with a clear
message if a true regression is detected. Use the existing regen_version_skew
logic and simple_mappings_match_cpython_except_documented_version_skew predicate
as the reference for the expected behavior.

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes