youmuliu/stgsharp

Fork 0

Files

T

Youmu Liu f0b6a4038e remove useless node from source gen pipeline. add source gen logic

2026-08-21 18:31:31 +08:00

35 KiB

Raw Blame History

L4 Software Cache Design Specification

Project: StgSharp
Author: Nitload
Version: Draft v0.3
Date: 2026-03-10

1. Purpose

1.1 Problem Background

StgSharp's matrix computation system is migrating from a "single global context table" to a "per-thread context" architecture to adapt to the scheduling characteristics of hybrid-architecture CPUs (e.g., Intel P-Core / E-Core). After the migration, each thread independently holds its own computation context, but matrix data may still be allocated by the HLSF allocator or global malloc on a remote NUMA node, resulting in significantly increased cross-node access latency.

At the same time, in matrix multiplication (FMA), the core row data of matrix A in the checkerboard-shaped kernel grid layout must be repacked (Packed) into a micro-kernel-friendly contiguous arrangement. In libraries such as OpenBLAS, this Pack operation is re-executed on every tile switch, whereas L4 can cache the Pack results to avoid repeated reordering.

Additionally, the segmented accumulation (segment) of matrix multiplication requires storing intermediate partial sums locally to avoid cross-NUMA writes to the target matrix on every segment. L4 can serve as a local accumulation buffer for partial sums, writing them back in a single operation after all segments complete locally.

1.2 Solution

The L4 cache is a purely software-implemented data caching layer logically positioned between the CPU hardware caches (L1/L2/L3) and main memory. Its core responsibilities:

Pack result caching: Cache the repack results of FMA core rows in matrix A to avoid repeated Pack operations
NUMA migration cache: Migrate data from remote NUMA nodes to local memory and cache it, eliminating cross-node latency
Segment accumulation buffer: Temporarily store partial sums from FMA segmented computation to reduce remote write operations
PLU intermediate data buffer: Provide a local contiguous workspace for PLU decomposition

1.3 Design Goals

Provide NUMA-friendly local data access for per-thread matrix computation contexts
Fixed capacity (64 MB), no runtime expansion, deterministic latency
Transparent pointer redirection: upper-layer code queries via original pointers; L4 returns local cache pointers
Cache eviction policy based on RRPV mask + windowed OPT prediction
Support both StreamTransparent and CacheBuffer working modes
Support writable cache lines that invoke registered callback functions for pre-processing and write-back upon eviction or explicit flush

2. Overall Architecture

2.1 Three-Component Model

┌���─────────────────────────────────────────────────────────────────┐
│                              L4                                  │
│                                                                  │
│  ┌──────────────┐  ┌────────────────────┐  ┌──────────────────┐  │
│  │  CacheLine    │  │     L4 Core        │  │    Predictor     │  │
│  │  Handle       │  │     Cache          │  │    Prefetch      │  │
│  │              │  │                    │  │    Predictor     │  │
│  │ • local addr  │  │ • Swiss Table      │  │ • 2048-entry buf │  │
│  │ • ref count   │  │ • Slab Allocator   │  │ • threshold 1024 │  │
│  │ • Dispose     │  │ • RRPV mask        │  │ • windowed OPT   │  │
│  │   drops ref   │  │ • SIMD eviction    │  │ • active key set │  │
│  │              │  │ • writeback mgmt   │  │                  │  │
│  └──────────────┘  └────────────────────┘  └──────────────────┘  │
│                                                                  │
└──────────────────────────────────────────────────────────────────┘

2.2 Data Access Path

Compute thread accessing data:

  ┌─────────────────────────────────────────────────────────┐
  │  Read-only path: L4.Map(originKey)                      │
  │     ├─ StreamTransparent + no repack needed?            │
  │     │  └─ Yes → return original pointer directly        │
  │     │          (zero overhead)                          │
  │     │                                                   │
  │     └─ CacheBuffer mode                                 │
  │        ├─ Swiss Table lookup (2.6ns, lock-free)         │
  │        ├─ Hit → update RRPV=0, return CacheLine handle  │
  │        └─ Miss → SpinWait for Predictor prefetch        │
  ├─────────────────────────────────────────────────────────┤
  │  Writable path: L4.MapWritable(originKey, preprocess, writeback) │
  │     ├─ Hit → return existing partial sum cache line     │
  │     └─ Miss → allocate zero-initialized cache line,     │
  │        register writeback callbacks                     │
  │        cache line marked dirty (dirty=1)                │
  │        eviction invokes preprocess + writeback          │
  ├─────────────────────────────────────────────────────────┤
  │  After computation:                                     │
  │  3. CacheLine.Dispose() (refCount--)                    │
  │  4. L4.StepComplete()                                   │
  │     ├─ Predictor.AdvanceExecutionPoint()                │
  │     └─ Predictor.NeedsPrefetch?                         │
  │        └─ Yes → CAS-acquire lock, execute prefetch      │
  └─────────────────────────────────────────────────────────┘

2.3 Key Parameters

Parameter	Value	Description
Total cache capacity	64 MB	Per-NUMA-node contiguous memory
Cache line size	8192 Bytes	= 64 × `double MAT_KERNEL`
Cache line count	8192	64MB / 8192B
Mapping table type	Swiss Table	1024 buckets × 16 slots, SIMD-accelerated
Cache line pool	Slab Allocator	Non-concurrent, fixed capacity, no auto-expansion
Eviction policy	RRPV mask + windowed OPT	SoA layout + SSE2 batch scan
Prefetch policy	Predictor windowed OPT	2048-entry buffer, 1024-step lookahead
Writeback policy	Callback-driven	Invokes registered callbacks on dirty eviction/flush

3. Key Components

3.1 Swiss Table (Pointer Mapping Table)

Responsibility: Maintains the original pointer → cache pointer mapping.

Key: original pointer nint, pointing to data allocated by HLSF/malloc on a remote node
Value: cache pointer nint, pointing to locally Slab-allocated cache line
Hash: hardware CRC32C instruction on the 64-bit pointer
- Low 10 bits → bucket index (1024 buckets)
- Bits 10-16 → H2 control byte (used for SIMD pre-filtering)
Scan: SSE2 pmovmskb instruction for parallel comparison of 16 control bytes
Tombstone mechanism: deleted entries set ctrl to 0xFE, preserving probe chain integrity
Capacity: 1024 × 16 = 16,384 mappings, significantly larger than total cache line count (8192 lines), ensuring extremely low hash collision rate

Measured Performance (Ryzen 7 6800H, .NET 8, BenchmarkDotNet):

Operation	Swiss Table	Dictionary	Speedup
Zipfian Lookup (8192×)	20.0 μs	76.1 μs	3.8×
Uniform Lookup (8192×)	18.6 μs	50.6 μs	2.7×
Insert (4096 entries)	18.5 μs	46.5 μs	2.5×
SteadyState (4096 ops)	25.3 μs	68.3 μs	2.7×

Single lookup latency ≈ 2.6 ns, GC allocation = 0.

3.2 Slab Allocator (Cache Line Pool)

Responsibility: Manages allocation and reclamation of fixed-size (8192B) cache lines.

Mode: Non-concurrent, fixed capacity (FixedSlabAllocator<T> mode), reusing the existing SlabAllocator.FromBuffer<T>() interface
No auto-expansion: capacity is fixed at initialization (64MB / 8192B = 8192 lines); allocation failure triggers eviction rather than expansion
Allocation complexity: O(1), pops from free stack
Reclamation complexity: O(1), pushes back to free stack
Alignment: all cache lines are 64B aligned, ensuring no cross-hardware-cache-line boundaries

3.3 Eviction Metadata (SoA Layout + RRPV Mask + Writeback Callbacks)

Responsibility: Tracks per-cache-line reference state, heat, and writeback information to determine eviction priority.

Replaces the four-level circular linked list from v0.1 with parallel arrays in Structure-of-Arrays layout:

SoA Layout — Parallel Arrays

  _refFlags[i]       : byte      Reference count snapshot (0=free, non-zero=held)
  _evictionFlags[i]  : byte      dirty(1bit) + RRPV(2bit) + costClass(1bit)
  _originKeys[i]     : nuint     Original pointer (reverse-deregisters Swiss Table entry on eviction)
  _preprocess[i]     : nint      Writeback pre-processing function pointer (called before dirty eviction)
  _writeback[i]      : nint      Writeback function pointer (called during dirty eviction)
  _callbackCtx[i]    : nuint     Callback context (passed to preprocess/writeback)

Eviction Flag Bit Layout (_evictionFlags[i]):

The RRPV array uses byte storage for each cache line's usage state; the low 2 bits encode the RRPV value.

RRPV Semantics:

RRPV	Meaning	Eviction Priority
0	Will be re-accessed very soon	Lowest (do not evict)
1	Will be accessed shortly	Low
2	Will be accessed in a while	Medium
3	May never be accessed again	Highest (evict first)

RRPV Update Events:

Event	Operation	Description
Load (insert)	RRPV = 0	Freshly loaded data will be used soon
Hit	`_evictionFlags[i] = 0`	Bits 1-2 cleared, RRPV reset to 0
Eviction scan	Find lines with RRPV=3 to evict	If none found, increment all RRPV +1 (aging)

Advantages over the v0.1 four-level circular linked list:

Comparison	v0.1 Circular List	v0.3 RRPV Mask + Callbacks
Data structure	4 doubly-linked lists + next ptrs	Byte array + function pointer arrays
Warm-up operation	Unlink + relink (4 pointer writes)	`+= 1` (1-byte write)
Aging operation	Per-node list migration	SIMD batch +1 (16 entries/cycle)
Eviction scan	Iterate list head (pointer chasing)	SIMD pattern matching (16/cycle)
Cache friendliness	Poor (pointer chasing)	Good (contiguous memory scan)
Reference protection	None	Lines with refCount > 0 not evicted
Writeback support	None	Callback-function driven

3.4 Writeback Callback Mechanism

Responsibility: Provides flexible writeback strategies for writable cache lines, using native function pointers to avoid managed overhead.

Design Principle: All writable cache lines are treated as dirty by default. On eviction or explicit flush, L4 itself does not manage the writeback strategy; instead, the bound predictor is responsible for implementing it.

The predictor is defined via the following interface:

    public interface IPrefetchPredict
    {

        int Predict(
            Span<CacheLinePrediction> node
        );

        bool Prefetch(
             nuint origin,
             ulong policy,
             nuint cache
        );

        void WriteBack(
             nuint origin,
             ulong policy,
             nuint cache
        );

    }

Typical Callback Implementations:

Scenario 1: Segment accumulation (FMA partial sum)
  preprocess = null  (no pre-processing needed; data is already a double array)
  writeback  = AccumulateAdd  (element-wise dst[i] += src[i], AVX2-vectorized)

Scenario 2: Writeback of packed intermediate results
  preprocess = UnpackKernelRow  (reverse-repack from Pack format to row-major order)
  writeback  = AccumulateAdd

Scenario 3: PLU decomposition intermediate data
  preprocess = null
  writeback  = DirectCopy  (direct overwrite memcpy)

3.5 CacheLine (Cache Line Handle)

Responsibility: A cache line access handle held by the compute thread; prevents in-use lines from being evicted via reference counting.

public ref struct CacheLine : IDisposable
{
    private ref int _activeCount;    // L4 global active count
    private ref int _refCount;       // Per-line reference count
    private nuint   _handle;         // Local cache data address

    // Construction: Interlocked.Increment(ref _activeCount)
    //               Interlocked.Increment(ref _refCount)
    // Dispose:      Interlocked.Decrement on both counters
}

ref struct ensures it cannot escape to the heap
Reference counts use Interlocked atomic operations for thread safety
Lines with _refCount > 0 are skipped during eviction scans
StreamTransparent mode returns a CacheLine that does not occupy a cache slot
Both read-only and writable paths return the same CacheLine type, callers decide read/write behavior via .Address

3.6 Predictor (Prefetch Predictor)

Responsibility: Provides deterministic prefetch predictions for L4 based on the windowed OPT strategy.

Core Principle: Matrix computation task sequences are deterministic — the scheduler pre-allocates task blocks (kr, kc, zz) for each thread, allowing precise computation of the cache line keys required at each future step. This gives L4 a predictive capability that hardware caches cannot possess: direct visibility into the future, rather than inferring the future from history as strategies like Hawkeye do.

Predictor Sliding Window:

  Task sequence: ... [T_now] [T+1] [T+2] ... [T+1023] ... [T+2047]
                      ↑ execution point                  ↑ buffer tail

  Window capacity      = 2048 prediction entries
  Prefetch trigger threshold = trigger refill when prefetch frontier
                               leads execution point by fewer than 1024 steps
  Refill batch size    = 64 entries (avoids excessive single-refill latency)

数据结构：

internal class Predictor
{
    private const int WindowCapacity = 2048;
    private const int RefillThreshold = 1024;

    private PrefetchEntry[] _buffer;      // Circular prediction buffer
    private int _head;                     // Prefetch frontier (prefetched up to here)
    private int _tail;                     // Production position (scheduler fills up to here)
    private int _executed;                 // Execution point (compute thread has completed up to here)

    private HashSet<nuint> _activeKeys;    // All active keys within the window (deduplicated)
}

struct PrefetchEntry
{
    public nuint Key;              // Cache line identifier
    public nuint SourceAddress;    // Source data address
    public bool  NeedsRepack;      // Whether non-contiguous repacking is needed
}

Cooperation with Eviction Policy:

Eviction priority (from highest to lowest priority):

  1. refCount==0, not in window, RRPV==3, dirty==0  ← First (zero-cost eviction)
  2. refCount==0, not in window, RRPV==3, dirty==1  ← Second (needs writeback but cold)
  3. refCount==0, not in window, RRPV<3,  dirty==0  ← Ordered by RRPV
  4. refCount==0, not in window, RRPV<3,  dirty==1  ← Ordered by RRPV
  5. refCount==0, in window                          ← Avoid evicting if possible
  6. refCount>0                                      ← Never evict

Windowed OPT means most eviction decisions do not depend on RRPV — "not in window" alone is sufficient to determine evictability. RRPV provides fine-grained differentiation among out-of-window lines; the dirty bit preferentially evicts clean lines at equal RRPV to avoid writeback overhead.

3.7 Concurrency Model

L4 uses a no-extra-threads design; all operations are performed by compute threads themselves:

Operation	Executor	Concurrency
Lookup (Swiss Table read)	Any compute thread	✅ Fully parallel, lock-free
HIT RRPV update	Any compute thread	✅ Single-byte write, no sync needed
refCount increment/decrement	Any compute thread	✅ Interlocked atomic operations
Prefetch refresh (eviction + load)	CAS-winner thread	Serial (only one thread enters)
Dirty writeback	CAS-winner thread	Serial (within eviction flow)
FlushAllDirty	Single thread after computation	Serial (no concurrency)

Prefetch Refresh CAS Lock:

private int _refreshLock;  // 0=free, 1=locked

private void TryRefreshPrefetch()
{
    if (Interlocked.CompareExchange(ref _refreshLock, 1, 0) != 0)
        return;  // Another thread is doing it; current thread continues computing
    try { /* eviction + prefetch */ }
    finally { Volatile.Write(ref _refreshLock, 0); }
}

Swiss Table Concurrent Read Safety Guarantee: The prefetch refresh thread follows strict write ordering on insertion: write Slab data first, then write eviction metadata and callback pointers, sfence, then write the Swiss Table control byte and key/value, ensuring that when a compute thread reads the control byte, the data is already fully visible.

4. Operation Flows

4.1 Cache Lookup (Read-Only Hit)

Compute CRC32C hash of the original pointer
Swiss Table lookup: SIMD scan ctrl → compare key → return cache pointer
Update RRPV: _evictionFlags[slot] &= 0xF9 (RRPV zeroed, dirty bit unchanged)
Construct CacheLine handle (refCount++)
Return CacheLine; upper-layer code accesses local data via .Address

4.2 Cache Lookup (Writable Hit)

Swiss Table lookup hits
Update RRPV: _evictionFlags[slot] &= 0xF9
Construct CacheLine handle (refCount++)
Return CacheLine; upper-layer code reads/writes the cache line via .Address (e.g., partial sum +=)

4.3 Cache Miss (Read-Only)

Swiss Table lookup misses
Compute thread enters Thread.Sleep(0) spin loop
Wait for Predictor-driven prefetch refresh to load the line into L4
Exit wait loop when Swiss Table shows a hit
Construct CacheLine handle and return

Under normal operation, the Predictor always leads the execution point by 1024+ steps, misses occur only during cold start or extreme bursts.

4.4 Cache Miss (Writable)

Swiss Table lookup misses
Allocate a cache line from Slab
If Slab has no free slots → trigger eviction (see 4.6)
Zero-initialize the cache line (Unsafe.InitBlock(dest, 0, 8192))
Register writeback callbacks: _preprocess[slot], _writeback[slot], _callbackCtx[slot]
Set eviction flags: dirty=1, RRPV=0
Write _originKeys[slot]
sfence
Write Swiss Table mapping
Construct CacheLine handle and return

4.5 Prefetch Refresh

Triggered by compute threads in StepComplete(); CAS lock ensures single-thread execution:

Synchronize refFlags snapshot (_refCounts[i] > 0 → _refFlags[i] = 1)
Retrieve pending prefetch entries from Predictor (up to 64 at a time)
For each entry:
- Swiss Table hit → mark RRPV=0 (hot-line protection)
- Swiss Table miss → execute load: a. Allocate cache line from Slab b. If Slab has no free slots → trigger eviction (see 4.6) c. Execute Pack reorder or memcpy based on NeedsRepack flag d. Write eviction metadata (originKey, RRPV=0, dirty=0, callbacks=null) e. sfence f. Write Swiss Table mapping

4.6 Cache Eviction (SIMD Scan + Writeback)

Eviction scan pseudocode:

  Pass 1: Find refCount==0, RRPV==3, dirty==0 (zero-cost eviction)
    for i in range(scanCursor, CacheLineCount), step=16:
      refChunk = SSE2.Load(refFlags + i)
      refFreeMask = SSE2.MoveMask(SSE2.CompareEqual(refChunk, zero))

      evChunk = SSE2.Load(evictionFlags + i)
      // RRPV==3 and dirty==0 → byte & 0x07 == 0x06
      masked = SSE2.And(evChunk, 0x07)
      coldCleanMask = SSE2.MoveMask(SSE2.CompareEqual(masked, 0x06))

      candidates = refFreeMask & coldCleanMask
      while candidates != 0:
        bit = TrailingZeroCount(candidates)
        idx = i + bit
        if originKeys[idx] not in Predictor.ActiveKeys:
          evict(idx)    // No writeback, discard directly
          return
        candidates &= candidates - 1

  Pass 2: Find refCount==0, RRPV==3, dirty==1 (needs writeback)
    Same as above, but match byte & 0x07 == 0x07
    evict triggers writeback callbacks

  Pass 3: Increment all RRPV+1 (aging), retry
    Up to 3 rounds

逐出执行：

private void Evict(int slot)
{
    byte flags = _evictionFlags[slot];

    // Dirty writeback
    if ((flags & 0x01) != 0)
    {
        nuint origin = _originKeys[slot];
        nuint cached = SlotAddressFromIndex(slot);
        nuint ctx    = _callbackCtx[slot];

        // 1. Pre-processing (optional)
        var preprocess = (delegate* unmanaged<nuint, nuint, void>)_preprocess[slot];
        if (preprocess != null)
            preprocess(cached, ctx);

        // 2. Writeback
        var writeback = (delegate* unmanaged<nuint, nuint, nuint, void>)_writeback[slot];
        if (writeback != null)
            writeback(origin, cached, ctx);
    }

    // Standard eviction
    _map.TryRemove(_originKeys[slot], out _);
    _slab.Free(SlotAddressFromIndex(slot));
    _evictionFlags[slot] = 0;
    _originKeys[slot] = 0;
    _preprocess[slot] = 0;
    _writeback[slot] = 0;
    _callbackCtx[slot] = 0;
}

4.7 显式刷新

/// <summary>
/// Explicitly writes back the specified dirty cache line. Called after all segments complete.
/// </summary>
public void FlushDirty(nuint originKey)
{
    if (_map.TryGet(originKey, out nuint addr))
    {
        int slot = SlotIndex(addr);
        if ((_evictionFlags[slot] & 0x01) != 0)
        {
            // Execute pre-processing + writeback
            InvokeWriteBack(slot);
            _evictionFlags[slot] &= 0xFE;  // Clear dirty bit
        }
    }
}

/// <summary>
/// Writes back all dirty cache lines in bulk. Called after entire matrix computation completes.
/// </summary>
public void FlushAllDirty()
{
    for (int i = 0; i < CacheLineCount; i++)
    {
        if ((_evictionFlags[i] & 0x01) != 0 && _originKeys[i] != 0)
        {
            InvokeWriteBack(i);
            _evictionFlags[i] &= 0xFE;
        }
    }
}

4.8 Working Mode Switch

public enum L4Mode
{
    /// <summary>
    /// Pass-through mode: Map() returns the original pointer directly; L4 does not intervene.
    /// Suitable for single-NUMA + no-repack scenarios (e.g., vector addition, scalar multiplication).
    /// </summary>
    StreamTransparent,

    /// <summary>
    /// Buffer mode: full lookup/Pack/cache/eviction pipeline.
    /// Suitable for FMA scenarios requiring repacking or multi-NUMA scenarios.
    /// </summary>
    CacheBuffer
}

Mode is determined at construction time and does not change at runtime. Decision logic:

Requires Pack reordering (FMA core rows of matrix A) → CacheBuffer
Multiple NUMA nodes → CacheBuffer
Requires segment accumulation → CacheBuffer
Single NUMA + no reordering + no segment → StreamTransparent

5. Segment Accumulation Scenario

5.1 Problem

For matrix multiplication C = A × B when the K dimension is large, segmented accumulation is required:

segment 0:  C_partial  = A[kr, 0..KC]    × B[0..KC, kc]
segment 1:  C_partial += A[kr, KC..2KC]  × B[KC..2KC, kc]
...
segment N:  C_partial += A[kr, N*KC..K]  × B[N*KC..K, kc]

Final: C[kr, kc] = Σ all segment partial sums

If each segment's result is += directly to the C matrix, and C resides on a remote NUMA node, every segment incurs cross-node write latency.

5.2 L4 Solution

Temporarily store C's partial sums in L4's writable cache lines:

segment 0: c = l4.MapWritable(C_key, null, &AccumulateAdd)
           // First miss → allocate zero-initialized cache line
           FMA(c.Address, a.Address, b.Address)  // c += a*b
           c.Dispose()

segment 1: c = l4.MapWritable(C_key, null, &AccumulateAdd)
           // Hit! Returns existing partial sum cache line
           FMA(c.Address, a.Address, b.Address)  // c += a*b
           c.Dispose()

... after all segments complete ...

l4.FlushDirty(C_key)
// Calls AccumulateAdd(C_origin, cacheline, ctx)
// → Adds local partial sum back to remote C matrix in one shot

5.3 Benefits

Without segment buffering (OpenBLAS approach):
  Each segment writes C matrix across NUMA
  Remote writes per output tile = Zk times
  Per-write latency = ~200-300ns (cross-node)

With L4 segment buffering:
  All segment += operations complete locally in L4 cache lines
  Partial sums likely remain in L1/L2 (8KB << L1 capacity)
  Remote writes per output tile = 1 (final FlushDirty)
  Writeback uses AVX2-vectorized stores: 8192B = 32 × 256-bit stores

  Reduction ratio = Zk : 1

6. Uniqueness and Advantages

6.1 Deterministic Prefetch (Windowed OPT)

Essential difference from hardware cache replacement policies (LRU, RRIP, Hawkeye):

Policy	Information Source	Prediction Method	Accuracy
LRU	Past access order	Least recently used	Moderate
RRIP	Insert/hit markers	Predict reuse distance	Moderate-High
Hawkeye	OPTgen retrospection	Learn past to infer future	High
L4 Predictor	Future task queue	Direct future visibility	Perfect (within window)

L4's Predictor is a deterministic Oracle — it does not predict; it knows. Because matrix computation task sequences are fully determined at scheduler allocation time.

6.2 NUMA Adaptability

The same code automatically gains NUMA benefits on different hardware configurations:

Hardware Configuration	L4 Behavior
Single-socket CPU	Pack cache + segment accumulation buffer
Dual-socket CPU	Pack cache + NUMA migration cache + segment accum.
Hybrid-architecture CPU	Pack cache + cross-domain migration cache + segment accum.

6.3 Callback-Driven Writeback Strategy

Using native function pointers for writeback avoids:

Virtual method call overhead
Interface dispatch overhead
Delegate GC pressure
Hard-coding specific writeback logic

Callers register preprocess and writeback callbacks at MapWritable time, L4 invokes them directly through function pointers on eviction or Flush, zero managed overhead, zero virtual dispatch, performance equivalent to C function calls.

6.4 Reference Count Protection

Through CacheLine handle reference counting, cache lines actively in use by compute threads are absolutely never evicted (including writeback not being triggered). The SIMD eviction scan skips lines with refCount > 0 immediately via the _refFlags array, without checking linked lists or acquiring locks.

6.5 Cache Line Size Aligned to Matrix Kernel

The 8192B cache line size is not arbitrary:

1 double MAT_KERNEL (4×4 double matrix) = 128 Bytes
1 cache line = 64 MAT_KERNEL units
Matrix multiplication micro-kernels operate on data sizes perfectly aligned to cache lines
Compute kernel functions operate on cache lines as units; no cross-cache-line accesses occur

6.6 Fixed-Capacity Determinism

Slab allocator never expands → memory footprint upper bound is known (64 MB)
SIMD eviction scan → latency upper bound is known (at most 3 aging rounds × 512 SIMD comparisons)
Swiss Table fixed 16,384 slots → lookup latency upper bound is known
Writeback callback latency is controlled by the caller → predictable
Worst-case L4 latency = lookup + eviction (incl. writeback) + allocation + memcpy, all statically computable

6.7 Reuse of Existing StgSharp Infrastructure

L4 Component	Reused StgSharp Infrastructure
Mapping table	`SwissTable` (implemented and benchmark-validated)
Mapping table SIMD	`IBucketScanner` + `SIMDID` architecture detection
Cache line alloc	`SlabAllocator.FromBuffer<T>()`
Hash computation	Hardware `CRC32C` instruction
Memory allocation	`NativeMemory.AlignedAlloc`
Future multi-platform	`IBucketScanner` + NRGA compilation pipeline

6.8 Originality Analysis

After surveying existing mainstream numerical computation libraries, L4's design has no direct precedent or equivalent in known public implementations.

NUMA Strategy Comparison with Existing Libraries

Library / Framework	NUMA Strategy	Software Cache Layer	Mapping Table	Eviction Policy	Dirty Writeback
Intel MKL	Thread pinning + `numactl`	None	None	None	None
OpenBLAS	Thread affinity + block alignment	None	None	None	None
BLIS	Multi-level block alignment L1/L2/L3	None	None	None	None
Eigen	No explicit NUMA handling	None	None	None	None
cuBLAS / rocBLAS	GPU memory (hardware-solved)	None	None	None	None
StgSharp L4	Software-managed thread-local cache	Yes	Swiss Table	RRPV + windowed OPT	Callback-driven

7. 从 v0.1 到 v0.3 的变更摘要

项目	v0.1	v0.3
逐出数据结构	4 级环形双向链表	RRPV 2-bit 掩码 + 8KB SoA 数组
逐出扫描方式	链表头指针遍历	SSE2 批量模式匹配（16 行/周期）
热��更新	链表节点摘插（4 指针写）	单字节位运算 `&= 0x03`
老化操作	逐节点移链表	SIMD 批量 +1
引用保护	无	refCount > 0 时 SIMD 扫描跳过
预取策略	无	Predictor 窗口 OPT（2048 条缓冲）
线程模型	L4 服务线程 + Command Queue	无额外线程，CAS 抢锁
命令队列	链式 CommandCache 分段	不需要（预取刷新替代命令驱动）
缓存行元数据	Slab 分配的 CacheLineHead 结构	SoA 平行数组 + 函数指针数组
工作模式	单一模式	StreamTransparent / CacheBuffer 切换
写回策略	无（只读缓存）	dirty 位 + preprocess/writeback 回调
写回机制	无	本机函数指针，零托管开销

8. 未来工作

L4 根类型重构（组合 Swiss Table、Slab、SoA 元数据、Predictor）
Predictor 与矩阵乘法调度器的集成
Segment 累加写回的 AVX2 向量化实现
SIMD 逐出扫描的 AVX2 升级（32 行/周期）
Swiss Table ARM NEON 实现
Swiss Table AVX-512 升级（64 槽扫描从 16 次降至 4 次）
墓碑累积监控与自动 rehash 策略
性能基准测试（NUMA 本地 vs 远端访问延迟对比）
Segment 累加 vs 直接写回的 benchmark 对比
与逐线程矩阵上下文的集成验证
双路 6242R 环境下的 NUMA 搬运性能验证
远期：PPC64LE 平台的 VSX 原生 Swiss Table 扫描支持

9. 参考

Google Abseil Swiss Tables Design Notes
HP Labs RRIP (Re-Reference Interval Prediction) 论文
Hawkeye: Efficient Cache Replacement via OPTgen (ISCA 2017)
StgSharp HLSF Allocator 设计文档
StgSharp SIMD Architecture (SIMDID 体系)
StgSharp NRGA Roadmap
StgSharp SlabAllocator<T> 实现
StgSharp SwissTable 实现及 Benchmark

35 KiB Raw Blame History Unescape Escape