singe/thirdparty/SDL3_ttf/external/harfbuzz/AVAR2-INSTANCING.md

avar2 Partial Instancing for HarfBuzz Subset Library

Context

The HarfBuzz subset library currently rejects partial instancing of fonts with avar2 tables ("Partial-instancing avar2 table is not supported."). This work ports the "Offset Compensation" algorithm from fonttools (design doc: partial-instancing-avar2-design.md in the fonttools repo) to HarfBuzz.

Key architectural difference: In fonttools, the subsetter and instancer are separate. In HarfBuzz, they are fused -- each table's subset() handles both glyph subsetting and variation instancing in a single pass. This means we can't simply "skip" tables; we must route them to their existing non-instancing code paths that still handle glyph subsetting.

Scope: Full implementation including self-contained axis detection and variation culling.

Algorithm Summary

When partial-instancing a font with avar2:

1. Normalize using avar v1 only -- compute intermediate-space axis locations
2. Rebase avar2 IVS regions via standard rebaseTent solver
3. Detect self-contained pinned axes -- whose final coordinate is constant
4. Add offset compensation deltas to the IVS so final coordinates remain in old coordinate space
5. Skip variation instancing in other tables for non-self-contained axes
6. Run standard instancing for self-contained axes at their old-space final coordinates
7. Cull dead variations in gvar/cvar/HVAR/VVAR/MVAR/GDEF whose regions fall outside reachable old-space final-coord ranges

See partial-instancing-avar2-design.md in the fonttools repo for the full mathematical proof.

Implementation Plan

Chunk 1: Plan Members and Normalization

Files: src/hb-subset-plan-member-list.hh, src/hb-subset-plan-var.cc, src/hb-ot-var-avar-table.hh

1a. New plan members (hb-subset-plan-member-list.hh)

Add after axes_triple_distances (line 131):

HB_SUBSET_PLAN_MEMBER (bool, has_avar2)
HB_SUBSET_PLAN_MEMBER (hb_hashmap_t E(<hb_tag_t, Triple>), old_intermediates)

• has_avar2: true when font has avar2 data AND partial instancing is requested (not all axes pinned).
• old_intermediates: maps axis tag -> Triple(a_i, d_i, b_i) -- old intermediate coords at new min/default/max. These are the coordinates needed for offset compensation.

1b. avar v1-only mapping (hb-ot-var-avar-table.hh)

Add a v1_only argument to map_coords_2_14(). When set, return after the avar v1 segment maps and omit the avar v2 tail processing:

bool map_coords_2_14 (float *coords, unsigned int coords_length,
                      bool v1_only = false) const
{
    // Existing avar v1 mapping...
    if (v1_only)
        return true;
    // Existing avar v2 mapping...
}

1c. Normalization changes (hb-subset-plan-var.cc, normalize_axes_location())

Replace the avar2 early-exit return false (lines 254-258) with avar2-aware normalization:

1. When avar2 partial instancing is detected (has_v2_data() && !all_axes_pinned):
   • Set plan->has_avar2 = true
   • Call map_coords_2_14(..., true) for mins, defaults, and maxs arrays
   • Store the v1-only mapped values in plan->old_intermediates
   • Compute plan->avar2_reachable_ranges and detect plan->avar2_self_contained axes (see Chunks 7 and 8)
   • Keep ALL axes in axes_index_map (including pinned ones), so pinned axes remain in fvar as hidden -- EXCEPT self-contained pinned axes, which are removed entirely
   • Set plan->axes_location and plan->normalized_coords to the self-contained pins ONLY (in old final-coordinate space); empty when there are none. The rest of the subsetter thereby sees an ordinary partial instancing of just those axes
2. Else: use existing path (map_coords_2_14() with full avar v1+v2)

Why v1-only: old_intermediates must be in intermediate coordinate space (post-fvar, post-avar-v1, pre-avar-v2) because:

• avar2 IVS regions are in intermediate space -- rebaseTent needs matching limits
• SegmentMaps::subset() uses it and unmaps through avar v1 -- correct with v1-only values
• Other tables must NOT use these values for instancing; they consume axes_location/normalized_coords, which under avar2 hold only the self-contained pins in final space (see Chunks 4 and 7)

Chunk 2: avar2 Subsetting (Core Algorithm)

File: src/hb-ot-var-avar-table.hh, avar::subset() and new helper

2a. Version and segment maps

Modify avar::subset() to:

• Output version 2.0 when plan->has_avar2 is true
• For avar2 pinned axes: serialize identity segment maps {-1->-1, 0->0, 1->1} (compile() needs entries for all fvar axes)
• For avar2 restricted axes: use existing SegmentMaps::subset() (works correctly with intermediate-space axes_location); its optional out_mappings parameter hands the instantiated mappings to _subset_avar2(), which needs them to locate the new mappings' kinks

2b. avar2 IVS instancing (_subset_avar2() helper)

Add helper method _subset_avar2(hb_subset_context_t *c):

1. Locate original avar2 data: walk past all SegmentMaps to find avarV2Tail, resolve varIdxMap and varStore.

2. Compute default deltas: evaluate original VarStore at old-default intermediate coordinates (d_1, d_2, ..., d_n). For each varIdx, default_delta = varStore.get_delta(varIdx, default_coords).

3. Rebase IVS regions: call item_vars.create_from_item_varstore() then item_vars.instantiate_tuple_vars() using plan->axes_location (intermediate-space limits). The standard instancing subtracts default deltas via merge_tuple_variations().

4. Detect self-contained pinned axes: After IVS rebasing, for each pinned axis, check if any remaining TupleVariation has a non-zero delta at that axis's inner position. If no variation remains: axis is self-contained. Its final coord = d_i + default_delta_i. These axes can be removed from fvar and their contributions folded into gvar/HVAR/etc.

5. Privatize shared varIdx delta rows: the avar2 VarIdxMap may map several fvar axes to the SAME IVS delta row (Amstelvar does: GRAD+XOPQ, and YOPQ+YTLC+YTUC). Writing one axis's offset-compensation deltas into a shared row would corrupt every other axis reading that row. Ref-count varIdxes across all fvar axes; for each offset-receiving axis (in user_axes_location) whose row is shared, append a copy of its row within the same VarData via item_vars.duplicate_row() and repoint its varIdx. Sharers keep the clean row; the varstore optimization pass re-merges identical rows afterwards.

6. Add offset compensation tuples: For each fvar axis:

   • Restricted or non-self-contained pinned axis (in user_axes_location):
     • If varIdx == NO_VARIATION_INDEX: create new VarData via item_vars.add_vardata(1), create/update VarIdxMap entry
     • Add bias tuple: empty region, delta = d_int + round(defaultDelta)
     • If pinned, that bias is all that's needed. Otherwise, encode the piecewise-linear function offset(z) = inv_renorm(z) - z as tents:
       • Collect knots: z = -1 -> a_i + 1, z = 0 -> d_i, z = +1 -> b_i - 1. If the axis default MOVED, inv_renorm also kinks where the old default lands in the new space: add z = z_old -> -z_old when interior.
       • Collect interior avar v1 breakpoints: each in-range old breakpoint kinks the new mapping at its output coordinate z; add that z with its old intermediate value (computed through user space and the old fvar + avar v1 chain) so offset(z) is reproduced at every kink. Keep this larger knot set only if it does not increase the residual estimated by a 257-point sweep; emit a DEBUG_MSG when the residual exceeds 8 F2Dot14 units (a steep retained segment that F2Dot14 cannot reproduce bit-exactly).
       • Synthesize one tent per non-zero knot: peak at z, extending to the adjacent knots (or the axis end for the outermost knots). Adjacent tents vanish at each other's peaks, so each delta is simply round((offset(z) - d_i) * 16384); the base value d_i is carried by the empty-region bias. This reduces to the classic two tents (-1,-1,0) / (0,+1,+1) when the default is unchanged and there are no interior knots. (Functionally equivalent to the 1-D VariationModel synthesis fonttools uses; the serialized regions differ when two or more knots lie strictly on one side of 0 -- fonttools extends an inner tent past the next knot and cascades its deltas, while HarfBuzz emits adjacent-knot tents with independent deltas, which rounds slightly better. Both realize the same piecewise-linear function.)
   • Self-contained pinned axis: skip offset compensation (will be folded)
   • Free/private axis (not in user_axes_location):
     • If round(defaultDelta) != 0: add bias tuple with round(defaultDelta)
     • Skip varIdxes already processed (multiple free axes can share a row)

7. Remove self-contained axes from axis order before building VarStore, so VarRegionList matches post-removal fvar axis count.

8. Finalize: call item_vars.build_region_list() + item_vars.as_item_varstore(optimize=true, use_no_variation_idx=false).

9. Serialize avarV2Tail: serialize DeltaSetIndexMap (with updated varIdx entries after optimization via item_vars.get_varidx_map()) and ItemVariationStore. Handle Offset32To resolution relative to avar table start.

10. Store self-contained axes info in plan for use by other tables.

Chunk 3: item_variations_t Extensions

File: src/hb-ot-var-common.hh

Add public methods to item_variations_t:

// Pass false to add tuples before explicitly building the region list.
bool instantiate_tuple_vars (
    const hb_hashmap_t<hb_tag_t, Triple>& axes_location,
    const hb_hashmap_t<hb_tag_t, TripleDistances>& axes_triple_distances,
    bool build_regions = true)

// Add a new VarData subtable. Returns outer index.
unsigned add_vardata (unsigned item_count)

// Get item count for a VarData subtable.
unsigned get_item_count (unsigned outer) const

// Add a tuple with a single non-zero delta at position inner.
void add_tuple (unsigned outer,
                hb_hashmap_t<hb_tag_t, Triple>&& axis_tuples,
                unsigned inner, int delta, unsigned item_count)

// Duplicate the delta row at position inner within VarData outer,
// appending the copy as a new item. Returns the new inner index.
// Used to privatize shared varIdx rows before offset compensation.
unsigned duplicate_row (unsigned outer, unsigned inner)

These methods allow the avar2 subsetting code to manipulate the IVS representation between rebasing and finalization.

Chunk 4: Instancing in Other Tables

Under avar2, the plan presents only the self-contained pins (in old final-coordinate space) through plan->axes_location and plan->normalized_coords; both are EMPTY when there are none. The ordinary if (c->plan->normalized_coords) instancing guards therefore need no avar2 special-casing at all: with no self-contained axes the tables take their existing non-instancing glyph-subsetting paths (variation data preserved in old final space), and with self-contained axes the standard instancing machinery runs and pins exactly those axes -- which is correct, because their final coordinates are constants in the space the variation tables live in. This applies uniformly to glyf/head/maxp/hmtx/OS/2/post bounds and metrics recomputation, GPOS value-format optimization, gvar, HVAR/VVAR, GDEF, COLR (including the paint instancer), BASE, CFF2, cvar (including cvt delta application), and MVAR (cvar and MVAR fall back to passthrough when there are no self-contained axes).

The avar2 variation culling (Chunk 8) hooks both flavors: the byte-level paths (gvar verbatim copy, plain VarStore serialization) and the decompiled paths (tuple_variations_t::cull_unreachable, called from item_variations_t::instantiate and gvar's glyph_variations_t).

Chunk 5: fvar Changes

File: src/hb-ot-var-fvar-table.hh

AxisRecord::subset

For avar2 pinned axes: set HIDDEN flag and min=default=max=pinned_value:

if (c->plan->has_avar2 && axis_limit->is_point())
{
    out->minValue.set_float (axis_limit->middle);
    out->defaultValue.set_float (axis_limit->middle);
    out->maxValue.set_float (axis_limit->middle);
    out->flags = out->flags | AxisRecord::AXIS_FLAG_HIDDEN;
}

InstanceRecord::subset

For avar2: don't skip pinned axes when writing instance coordinates (they're still in fvar):

if (axis_limit->is_point () && !c->plan->has_avar2) continue;

Chunk 6: Table Ordering

File: src/hb-subset.cc, _dependencies_satisfied()

Add avar as a dependency for variation tables when avar2 is present. The avar2 subsetting step computes self-contained axes info that other tables need.

case HB_TAG('g','v','a','r'):
case HB_TAG('c','v','a','r'):
case HB_TAG('H','V','A','R'):
case HB_TAG('V','V','A','R'):
case HB_TAG('M','V','A','R'):
case HB_TAG('G','D','E','F'):
case HB_TAG('C','F','F','2'):
    return !plan->has_avar2 || !pending_subset_tags.has(HB_TAG('a','v','a','r'));

Chunk 7: Self-Contained Axis Instancing

A pinned axis is self-contained when its final coordinate is CONSTANT over the retained box: its varIdx is NO_VARIATION_INDEX, or its avar2 delta row is constant over the box (detected at plan time with the same interval arithmetic as Chunk 8; a constant delta interval has zero width). Its final coordinate is d_i + delta.

Plan members:

HB_SUBSET_PLAN_MEMBER (hb_hashmap_t E(<hb_tag_t, double>), avar2_self_contained)

Self-contained axes are removed from axes_index_map/axis_tags (dropped from fvar and avar entirely; _subset_avar2 skips them and writes the DeltaSetIndexMap over the retained axes only). The plan presents them to every other table as an ordinary partial pin at the constant final coordinate through axes_location/normalized_coords (see Chunk 4); the intermediate-space ranges avar itself needs remain in old_intermediates. This mirrors fontTools' selfContainedAxes + _instantiateFvarForAvar2 + second instancing pass, folded into HarfBuzz's single-pass architecture.

Detection is suppressed (such pins stay in fvar as ordinary hidden axes) when the face has a CFF2 table -- CFF2 has no partial-pin instancing path; its pinned path flattens ALL blends and drops the CFF2 VariationStore -- or a VARC table, which passes through verbatim with explicit fvar axis indices that axis removal would desynchronize. The CFF2 limitation (which also silently flattens non-avar2 partial instancing of CFF2 fonts) is harfbuzz/harfbuzz#4710; fontTools implements proper partial CFF2 instancing in fonttools/fonttools#3506 (instantiateCFF2: lift blend deltas into the CFF2 VarStore, instantiate it, read back the rewritten deltas), which is the design to port. Lifting the suppression then just means deleting the CFF2 check here.

Chunk 8: Variation Culling

Cull dead variations in gvar/HVAR/VVAR/GDEF/COLR/BASE whose axis regions fall outside the reachable old-space final-coord ranges (cvar/MVAR: TODO; they are culled only via their instancing paths when self-contained axes exist).

Reachable ranges are computed at plan time (avar2_reachable_ranges): with offset compensation, the instance's final coordinates equal the original font's over the retained user box, so bound final_i = intermediate_i + delta_i over the box of retained old-intermediate ranges (restricted axes their [a_i, b_i], free public axes [-1, +1], pinned axes their d_i, hidden axes 0) with interval arithmetic over the original avar2 VarStore. This is the same quantity fontTools bounds via getExtremes on the instanced store, computed from the original store instead; exact (tighter) for pinned and hidden axes.

A TupleVariation/region is dead if some axis's tent is provably zero over the axis's entire reachable range (invalid tents evaluate as constant 1 at runtime and never kill); ranges are padded by one F2Dot14 unit. Hidden axes are assumed to sit at their default, the same semantic assumption fontTools makes.

Verification

1. Differential API test (test/api/test-subset-avar2.c, using test/api/fonts/TestAvar2Instance.ttf): the final normalized coordinates (hb_font_get_var_coords_normalized, i.e. fvar + avar v1 + avar v2) of a partial instance must match the original font's at the same user-space location, within 2 F2Dot14 units (5 for the steepest quantization-limited case, chosen one unit below what dropping the interior-breakpoint collection produces), over grids of locations including the exact kink positions. Rendering-level outputs (advance width and glyph extents, exercising gvar and HVAR through culling and self-contained instancing) are compared as well. Covers: same-default and moved-default restriction (both directions), shared-varIdx rows (restricted, pinned, and both-restricted), pinning at and off the default, a pinned axis driven by a free axis, a NO_VARIATION_INDEX axis (fresh VarData), interior avar v1 breakpoints, steep moved-far-from-min defaults, variation culling via a hidden driven axis, and self-contained pins (alone and combined with a range restriction).
2. Take an avar2 font (e.g., RobotoFlex-avar2.ttf), partial-instance with both fonttools and HarfBuzz, compare avar2 VarStore output
3. Render glyphs at various coordinates using the instanced font -- verify visual match with the original
4. Test full pinning (all axes) still works (existing path)
5. Test with NO_VARIATION_INDEX axes, fonts without explicit VarIdxMap, multiple restricted axes
6. Test self-contained axis detection and removal
7. Test variation culling produces smaller output without visual changes