@@ -1,762 +0,0 @@
# L4 Software Cache Design Specification
> Project: StgSharp
> Author: Nitload
> Version: Draft v0.3
> Date: 2026-03-10
---
## 1. Purpose
### 1.1 Problem Background
StgSharp's matrix computation system is migrating from a "single global context table" to a
"per-thread context" architecture to adapt to the scheduling characteristics of hybrid-architecture
CPUs (e.g., Intel P-Core / E-Core). After the migration, each thread independently holds its own
computation context, but matrix data may still be allocated by the HLSF allocator or global
`malloc` on a remote NUMA node, resulting in significantly increased cross-node access latency.
At the same time, in matrix multiplication (FMA), the core row data of matrix A in the
checkerboard-shaped kernel grid layout must be repacked (Packed) into a micro-kernel-friendly
contiguous arrangement. In libraries such as OpenBLAS, this Pack operation is re-executed on
every tile switch, whereas L4 can cache the Pack results to avoid repeated reordering.
Additionally, the segmented accumulation (segment) of matrix multiplication requires storing
intermediate partial sums locally to avoid cross-NUMA writes to the target matrix on every
segment. L4 can serve as a local accumulation buffer for partial sums, writing them back in a
single operation after all segments complete locally.
### 1.2 Solution
The L4 cache is a purely software-implemented **data caching layer ** logically positioned
between the CPU hardware caches (L1/L2/L3) and main memory. Its core responsibilities:
1. **Pack result caching ** : Cache the repack results of FMA core rows in matrix A to avoid repeated Pack operations
2. **NUMA migration cache ** : Migrate data from remote NUMA nodes to local memory and cache it, eliminating cross-node latency
3. **Segment accumulation buffer ** : Temporarily store partial sums from FMA segmented computation to reduce remote write operations
4. **PLU intermediate data buffer ** : Provide a local contiguous workspace for PLU decomposition
### 1.3 Design Goals
- Provide NUMA-friendly local data access for per-thread matrix computation contexts
- Fixed capacity (64 MB), no runtime expansion, deterministic latency
- Transparent pointer redirection: upper-layer code queries via original pointers; L4 returns local cache pointers
- Cache eviction policy based on **RRPV mask + windowed OPT prediction **
- Support both **StreamTransparent ** and **CacheBuffer ** working modes
- Support **writable cache lines ** that invoke registered callback functions for pre-processing and write-back upon eviction or explicit flush
---
## 2. Overall Architecture
### 2.1 Three-Component Model
```
┌� � � ─────────────────────────────────────────────────────────────────┐
│ L4 │
│ │
│ ┌──────────────┐ ┌────────────────────┐ ┌──────────────────┐ │
│ │ CacheLine │ │ L4 Core │ │ Predictor │ │
│ │ Handle │ │ Cache │ │ Prefetch │ │
│ │ │ │ │ │ Predictor │ │
│ │ • local addr │ │ • Swiss Table │ │ • 2048-entry buf │ │
│ │ • ref count │ │ • Slab Allocator │ │ • threshold 1024 │ │
│ │ • Dispose │ │ • RRPV mask │ │ • windowed OPT │ │
│ │ drops ref │ │ • SIMD eviction │ │ • active key set │ │
│ │ │ │ • writeback mgmt │ │ │ │
│ └──────────────┘ └────────────────────┘ └──────────────────┘ │
│ │
└──────────────────────────────────────────────────────────────────┘
```
### 2.2 Data Access Path
```
Compute thread accessing data:
┌─────────────────────────────────────────────────────────┐
│ Read-only path: L4.Map(originKey) │
│ ├─ StreamTransparent + no repack needed? │
│ │ └─ Yes → return original pointer directly │
│ │ (zero overhead) │
│ │ │
│ └─ CacheBuffer mode │
│ ├─ Swiss Table lookup (2.6ns, lock-free) │
│ ├─ Hit → update RRPV=0, return CacheLine handle │
│ └─ Miss → SpinWait for Predictor prefetch │
├─────────────────────────────────────────────────────────┤
│ Writable path: L4.MapWritable(originKey, preprocess, writeback) │
│ ├─ Hit → return existing partial sum cache line │
│ └─ Miss → allocate zero-initialized cache line, │
│ register writeback callbacks │
│ cache line marked dirty (dirty=1) │
│ eviction invokes preprocess + writeback │
├─────────────────────────────────────────────────────────┤
│ After computation: │
│ 3. CacheLine.Dispose() (refCount--) │
│ 4. L4.StepComplete() │
│ ├─ Predictor.AdvanceExecutionPoint() │
│ └─ Predictor.NeedsPrefetch? │
│ └─ Yes → CAS-acquire lock, execute prefetch │
└─────────────────────────────────────────────────────────┘
```
### 2.3 Key Parameters
| Parameter | Value | Description |
|------------------------|--------------------------|------------------------------------------------------|
| Total cache capacity | 64 MB | Per-NUMA-node contiguous memory |
| Cache line size | 8192 Bytes | = 64 × `double MAT_KERNEL` |
| Cache line count | 8192 | 64MB / 8192B |
| Mapping table type | Swiss Table | 1024 buckets ×
| Cache line pool | Slab Allocator | Non-concurrent, fixed capacity, no auto-expansion |
| Eviction policy | RRPV mask + windowed OPT | SoA layout + SSE2 batch scan |
| Prefetch policy | Predictor windowed OPT | 2048-entry buffer, 1024-step lookahead |
| Writeback policy | Callback-driven | Invokes registered callbacks on dirty eviction/flush |
---
## 3. Key Components
### 3.1 Swiss Table (Pointer Mapping Table)
**Responsibility ** : Maintains the `original pointer → cache pointer` mapping.
- Key: original pointer `nint` , pointing to data allocated by HLSF/malloc on a remote node
- Value: cache pointer `nint` , pointing to locally Slab-allocated cache line
- Hash: hardware `CRC32C` instruction on the 64-bit pointer
- Low 10 bits → bucket index (1024 buckets)
- Bits 10-16 → H2 control byte (used for SIMD pre-filtering)
- Scan: SSE2 `pmovmskb` instruction for parallel comparison of 16 control bytes
- Tombstone mechanism: deleted entries set ctrl to `0xFE` , preserving probe chain integrity
- Capacity: 1024 ×
ensuring extremely low hash collision rate
**Measured Performance ** (Ryzen 7 6800H, .NET 8, BenchmarkDotNet):
| Operation | Swiss Table | Dictionary | Speedup |
|--------------------------|-------------|-------------|---------|
| Zipfian Lookup (8192× ×
| Uniform Lookup (8192× ×
| Insert (4096 entries) | 18.5 μs | 46.5 μs | 2.5×
| SteadyState (4096 ops) | 25.3 μs | 68.3 μs | 2.7×
Single lookup latency ≈ 2.6 ns, GC allocation = 0.
### 3.2 Slab Allocator (Cache Line Pool)
**Responsibility ** : Manages allocation and reclamation of fixed-size (8192B) cache lines.
- Mode: **Non-concurrent, fixed capacity ** (`FixedSlabAllocator<T>` mode),
reusing the existing `SlabAllocator.FromBuffer<T>()` interface
- No auto-expansion: capacity is fixed at initialization (64MB / 8192B = 8192 lines);
allocation failure triggers eviction rather than expansion
- Allocation complexity: O(1), pops from free stack
- Reclamation complexity: O(1), pushes back to free stack
- Alignment: all cache lines are 64B aligned, ensuring no cross-hardware-cache-line boundaries
### 3.3 Eviction Metadata (SoA Layout + RRPV Mask + Writeback Callbacks)
**Responsibility ** : Tracks per-cache-line reference state, heat, and writeback information to determine eviction priority.
Replaces the four-level circular linked list from v0.1 with parallel arrays in Structure-of-Arrays layout:
```
SoA Layout — Parallel Arrays
_refFlags[i] : byte Reference count snapshot (0=free, non-zero=held)
_evictionFlags[i] : byte dirty(1bit) + RRPV(2bit) + costClass(1bit)
_originKeys[i] : nuint Original pointer (reverse-deregisters Swiss Table entry on eviction)
_preprocess[i] : nint Writeback pre-processing function pointer (called before dirty eviction)
_writeback[i] : nint Writeback function pointer (called during dirty eviction)
_callbackCtx[i] : nuint Callback context (passed to preprocess/writeback)
```
**Eviction Flag Bit Layout ** (`_evictionFlags[i]` ):
The RRPV array uses byte storage for each cache line's usage state; the low 2 bits encode the RRPV value.
**RRPV Semantics ** :
| RRPV | Meaning | Eviction Priority |
|------|---------------------------------|---------------------------|
| 0 | Will be re-accessed very soon | Lowest (do not evict) |
| 1 | Will be accessed shortly | Low |
| 2 | Will be accessed in a while | Medium |
| 3 | May never be accessed again | Highest (evict first) |
**RRPV Update Events ** :
| Event | Operation | Description |
|--------------------|---------------------------------|------------------------------------------------------|
| Load (insert) | RRPV = 0 | Freshly loaded data will be used soon |
| Hit | `_evictionFlags[i] = 0` | Bits 1-2 cleared, RRPV reset to 0 |
| Eviction scan | Find lines with RRPV=3 to evict | If none found, increment all RRPV +1 (aging) |
**Advantages over the v0.1 four-level circular linked list ** :
| Comparison | v0.1 Circular List | v0.3 RRPV Mask + Callbacks |
|----------------------|-------------------------------------|--------------------------------------|
| Data structure | 4 doubly-linked lists + next ptrs | Byte array + function pointer arrays |
| Warm-up operation | Unlink + relink (4 pointer writes) | `+= 1` (1-byte write) |
| Aging operation | Per-node list migration | SIMD batch +1 (16 entries/cycle) |
| Eviction scan | Iterate list head (pointer chasing) | SIMD pattern matching (16/cycle) |
| Cache friendliness | Poor (pointer chasing) | Good (contiguous memory scan) |
| Reference protection | None | Lines with refCount > 0 not evicted |
| Writeback support | None | Callback-function driven |
### 3.4 Writeback Callback Mechanism
**Responsibility ** : Provides flexible writeback strategies for writable cache lines, using native function pointers to avoid managed overhead.
**Design Principle ** : All writable cache lines are treated as dirty by default. On eviction or explicit flush,
L4 itself does not manage the writeback strategy; instead, the bound predictor is responsible for implementing it.
The predictor is defined via the following interface:
```
public interface IPrefetchPredict
{
int Predict(
Span<CacheLinePrediction> node
);
bool Prefetch(
nuint origin,
ulong policy,
nuint cache
);
void WriteBack(
nuint origin,
ulong policy,
nuint cache
);
}
```
**Typical Callback Implementations ** :
```
Scenario 1: Segment accumulation (FMA partial sum)
preprocess = null (no pre-processing needed; data is already a double array)
writeback = AccumulateAdd (element-wise dst[i] += src[i], AVX2-vectorized)
Scenario 2: Writeback of packed intermediate results
preprocess = UnpackKernelRow (reverse-repack from Pack format to row-major order)
writeback = AccumulateAdd
Scenario 3: PLU decomposition intermediate data
preprocess = null
writeback = DirectCopy (direct overwrite memcpy)
```
### 3.5 CacheLine (Cache Line Handle)
**Responsibility ** : A cache line access handle held by the compute thread; prevents in-use lines from being evicted via reference counting.
``` csharp
public ref struct CacheLine : IDisposable
{
private ref int _activeCount ; // L4 global active count
private ref int _refCount ; // Per-line reference count
private nuint _handle ; // Local cache data address
// Construction: Interlocked.Increment(ref _activeCount)
// Interlocked.Increment(ref _refCount)
// Dispose: Interlocked.Decrement on both counters
}
```
- `ref struct` ensures it cannot escape to the heap
- Reference counts use `Interlocked` atomic operations for thread safety
- Lines with `_refCount > 0` are skipped during eviction scans
- StreamTransparent mode returns a CacheLine that does not occupy a cache slot
- Both read-only and writable paths return the same CacheLine type,
callers decide read/write behavior via `.Address`
### 3.6 Predictor (Prefetch Predictor)
**Responsibility ** : Provides deterministic prefetch predictions for L4 based on the windowed OPT strategy.
**Core Principle ** : Matrix computation task sequences are deterministic — the scheduler pre-allocates
task blocks `(kr, kc, zz)` for each thread, allowing precise computation of the cache line keys
required at each future step. This gives L4 a predictive capability that hardware caches cannot
possess: **direct visibility into the future ** , rather than inferring the future from history as
strategies like Hawkeye do.
```
Predictor Sliding Window:
Task sequence: ... [T_now] [T+1] [T+2] ... [T+1023] ... [T+2047]
↑ execution point ↑ buffer tail
Window capacity = 2048 prediction entries
Prefetch trigger threshold = trigger refill when prefetch frontier
leads execution point by fewer than 1024 steps
Refill batch size = 64 entries (avoids excessive single-refill latency)
```
**数据结构 ** ：
``` csharp
internal class Predictor
{
private const int WindowCapacity = 2048 ;
private const int RefillThreshold = 1024 ;
private PrefetchEntry [ ] _buffer ; // Circular prediction buffer
private int _head ; // Prefetch frontier (prefetched up to here)
private int _tail ; // Production position (scheduler fills up to here)
private int _executed ; // Execution point (compute thread has completed up to here)
private HashSet < nuint > _activeKeys ; // All active keys within the window (deduplicated)
}
struct PrefetchEntry
{
public nuint Key ; // Cache line identifier
public nuint SourceAddress ; // Source data address
public bool NeedsRepack ; // Whether non-contiguous repacking is needed
}
```
**Cooperation with Eviction Policy ** :
```
Eviction priority (from highest to lowest priority):
1. refCount==0, not in window, RRPV==3, dirty==0 ← First (zero-cost eviction)
2. refCount==0, not in window, RRPV==3, dirty==1 ← Second (needs writeback but cold)
3. refCount==0, not in window, RRPV<3, dirty==0 ← Ordered by RRPV
4. refCount==0, not in window, RRPV<3, dirty==1 ← Ordered by RRPV
5. refCount==0, in window ← Avoid evicting if possible
6. refCount>0 ← Never evict
```
Windowed OPT means most eviction decisions do not depend on RRPV — "not in window" alone is
sufficient to determine evictability. RRPV provides fine-grained differentiation among
out-of-window lines; the dirty bit preferentially evicts clean lines at equal RRPV to avoid writeback overhead.
### 3.7 Concurrency Model
L4 uses a **no-extra-threads ** design; all operations are performed by compute threads themselves:
| Operation | Executor | Concurrency |
|------------------------------------|---------------------------------|--------------------------------------|
| Lookup (Swiss Table read) | Any compute thread | ✅ Fully parallel, lock-free |
| HIT RRPV update | Any compute thread | ✅ Single-byte write, no sync needed |
| refCount increment/decrement | Any compute thread | ✅ Interlocked atomic operations |
| Prefetch refresh (eviction + load) | CAS-winner thread | Serial (only one thread enters) |
| Dirty writeback | CAS-winner thread | Serial (within eviction flow) |
| FlushAllDirty | Single thread after computation | Serial (no concurrency) |
**Prefetch Refresh CAS Lock ** :
``` csharp
private int _refreshLock ; // 0=free, 1=locked
private void TryRefreshPrefetch ( )
{
if ( Interlocked . CompareExchange ( ref _refreshLock , 1 , 0 ) ! = 0 )
return ; // Another thread is doing it; current thread continues computing
try { /* eviction + prefetch */ }
finally { Volatile . Write ( ref _refreshLock , 0 ) ; }
}
```
**Swiss Table Concurrent Read Safety Guarantee ** : The prefetch refresh thread follows strict write ordering
on insertion: write Slab data first, then write eviction metadata and callback pointers, `sfence` , then
write the Swiss Table control byte and key/value, ensuring that when a compute thread reads the control
byte, the data is already fully visible.
---
## 4. Operation Flows
### 4.1 Cache Lookup (Read-Only Hit)
1. Compute CRC32C hash of the original pointer
2. Swiss Table lookup: SIMD scan ctrl → compare key → return cache pointer
3. Update RRPV: `_evictionFlags[slot] &= 0xF9` (RRPV zeroed, dirty bit unchanged)
4. Construct CacheLine handle (refCount++)
5. Return CacheLine; upper-layer code accesses local data via `.Address`
### 4.2 Cache Lookup (Writable Hit)
1. Swiss Table lookup hits
2. Update RRPV: `_evictionFlags[slot] &= 0xF9`
3. Construct CacheLine handle (refCount++)
4. Return CacheLine; upper-layer code reads/writes the cache line via `.Address` (e.g., partial sum +=)
### 4.3 Cache Miss (Read-Only)
1. Swiss Table lookup misses
2. Compute thread enters `Thread.Sleep(0)` spin loop
3. Wait for Predictor-driven prefetch refresh to load the line into L4
4. Exit wait loop when Swiss Table shows a hit
5. Construct CacheLine handle and return
Under normal operation, the Predictor always leads the execution point by 1024+ steps,
misses occur only during cold start or extreme bursts.
### 4.4 Cache Miss (Writable)
1. Swiss Table lookup misses
2. Allocate a cache line from Slab
3. If Slab has no free slots → trigger eviction (see 4.6)
4. Zero-initialize the cache line (`Unsafe.InitBlock(dest, 0, 8192)` )
5. Register writeback callbacks: `_preprocess[slot]` , `_writeback[slot]` , `_callbackCtx[slot]`
6. Set eviction flags: `dirty=1, RRPV=0`
7. Write `_originKeys[slot]`
8. `sfence`
9. Write Swiss Table mapping
10. Construct CacheLine handle and return
### 4.5 Prefetch Refresh
Triggered by compute threads in `StepComplete()` ; CAS lock ensures single-thread execution:
1. Synchronize refFlags snapshot (`_refCounts[i] > 0 → _refFlags[i] = 1` )
2. Retrieve pending prefetch entries from Predictor (up to 64 at a time)
3. For each entry:
- Swiss Table hit → mark RRPV=0 (hot-line protection)
- Swiss Table miss → execute load:
a. Allocate cache line from Slab
b. If Slab has no free slots → trigger eviction (see 4.6)
c. Execute Pack reorder or memcpy based on `NeedsRepack` flag
d. Write eviction metadata (originKey, RRPV=0, dirty=0, callbacks=null)
e. `sfence`
f. Write Swiss Table mapping
### 4.6 Cache Eviction (SIMD Scan + Writeback)
```
Eviction scan pseudocode:
Pass 1: Find refCount==0, RRPV==3, dirty==0 (zero-cost eviction)
for i in range(scanCursor, CacheLineCount), step=16:
refChunk = SSE2.Load(refFlags + i)
refFreeMask = SSE2.MoveMask(SSE2.CompareEqual(refChunk, zero))
evChunk = SSE2.Load(evictionFlags + i)
// RRPV==3 and dirty==0 → byte & 0x07 == 0x06
masked = SSE2.And(evChunk, 0x07)
coldCleanMask = SSE2.MoveMask(SSE2.CompareEqual(masked, 0x06))
candidates = refFreeMask & coldCleanMask
while candidates != 0:
bit = TrailingZeroCount(candidates)
idx = i + bit
if originKeys[idx] not in Predictor.ActiveKeys:
evict(idx) // No writeback, discard directly
return
candidates &= candidates - 1
Pass 2: Find refCount==0, RRPV==3, dirty==1 (needs writeback)
Same as above, but match byte & 0x07 == 0x07
evict triggers writeback callbacks
Pass 3: Increment all RRPV+1 (aging), retry
Up to 3 rounds
```
**逐出执行 ** ：
``` csharp
private void Evict ( int slot )
{
byte flags = _evictionFlags [ slot ] ;
// Dirty writeback
if ( ( flags & 0x01 ) ! = 0 )
{
nuint origin = _originKeys [ slot ] ;
nuint cached = SlotAddressFromIndex ( slot ) ;
nuint ctx = _callbackCtx [ slot ] ;
// 1. Pre-processing (optional)
var preprocess = ( delegate * unmanaged < nuint , nuint , void > ) _preprocess [ slot ] ;
if ( preprocess ! = null )
preprocess ( cached , ctx ) ;
// 2. Writeback
var writeback = ( delegate * unmanaged < nuint , nuint , nuint , void > ) _writeback [ slot ] ;
if ( writeback ! = null )
writeback ( origin , cached , ctx ) ;
}
// Standard eviction
_map . TryRemove ( _originKeys [ slot ] , out _ ) ;
_slab . Free ( SlotAddressFromIndex ( slot ) ) ;
_evictionFlags [ slot ] = 0 ;
_originKeys [ slot ] = 0 ;
_preprocess [ slot ] = 0 ;
_writeback [ slot ] = 0 ;
_callbackCtx [ slot ] = 0 ;
}
```
### 4.7 显式刷新
``` csharp
/// <summary>
/// Explicitly writes back the specified dirty cache line. Called after all segments complete.
/// </summary>
public void FlushDirty ( nuint originKey )
{
if ( _map . TryGet ( originKey , out nuint addr ) )
{
int slot = SlotIndex ( addr ) ;
if ( ( _evictionFlags [ slot ] & 0x01 ) ! = 0 )
{
// Execute pre-processing + writeback
InvokeWriteBack ( slot ) ;
_evictionFlags [ slot ] & = 0xFE ; // Clear dirty bit
}
}
}
/// <summary>
/// Writes back all dirty cache lines in bulk. Called after entire matrix computation completes.
/// </summary>
public void FlushAllDirty ( )
{
for ( int i = 0 ; i < CacheLineCount ; i + + )
{
if ( ( _evictionFlags [ i ] & 0x01 ) ! = 0 & & _originKeys [ i ] ! = 0 )
{
InvokeWriteBack ( i ) ;
_evictionFlags [ i ] & = 0xFE ;
}
}
}
```
### 4.8 Working Mode Switch
``` csharp
public enum L4Mode
{
/// <summary>
/// Pass-through mode: Map() returns the original pointer directly; L4 does not intervene.
/// Suitable for single-NUMA + no-repack scenarios (e.g., vector addition, scalar multiplication).
/// </summary>
StreamTransparent ,
/// <summary>
/// Buffer mode: full lookup/Pack/cache/eviction pipeline.
/// Suitable for FMA scenarios requiring repacking or multi-NUMA scenarios.
/// </summary>
CacheBuffer
}
```
Mode is determined at construction time and does not change at runtime. Decision logic:
- Requires Pack reordering (FMA core rows of matrix A) → CacheBuffer
- Multiple NUMA nodes → CacheBuffer
- Requires segment accumulation → CacheBuffer
- Single NUMA + no reordering + no segment → StreamTransparent
---
## 5. Segment Accumulation Scenario
### 5.1 Problem
For matrix multiplication `C = A × when the K dimension is large, segmented accumulation is required:
```
segment 0: C_partial = A[kr, 0..KC] ×
segment 1: C_partial += A[kr, KC..2KC] ×
...
segment N: C_partial += A[kr, N*KC..K] ×
Final: C[kr, kc] = Σ all segment partial sums
```
If each segment's result is `+=` directly to the C matrix,
and C resides on a remote NUMA node, every segment incurs cross-node write latency.
### 5.2 L4 Solution
Temporarily store C's partial sums in L4's writable cache lines:
```
segment 0: c = l4.MapWritable(C_key, null, &AccumulateAdd)
// First miss → allocate zero-initialized cache line
FMA(c.Address, a.Address, b.Address) // c += a*b
c.Dispose()
segment 1: c = l4.MapWritable(C_key, null, &AccumulateAdd)
// Hit! Returns existing partial sum cache line
FMA(c.Address, a.Address, b.Address) // c += a*b
c.Dispose()
... after all segments complete ...
l4.FlushDirty(C_key)
// Calls AccumulateAdd(C_origin, cacheline, ctx)
// → Adds local partial sum back to remote C matrix in one shot
```
### 5.3 Benefits
```
Without segment buffering (OpenBLAS approach):
Each segment writes C matrix across NUMA
Remote writes per output tile = Zk times
Per-write latency = ~200-300ns (cross-node)
With L4 segment buffering:
All segment += operations complete locally in L4 cache lines
Partial sums likely remain in L1/L2 (8KB << L1 capacity)
Remote writes per output tile = 1 (final FlushDirty)
Writeback uses AVX2-vectorized stores: 8192B = 32 ×
Reduction ratio = Zk : 1
```
---
## 6. Uniqueness and Advantages
### 6.1 Deterministic Prefetch (Windowed OPT)
Essential difference from hardware cache replacement policies (LRU, RRIP, Hawkeye):
| Policy | Information Source | Prediction Method | Accuracy |
|---------------------|-------------------------|-------------------------------|---------------------------|
| LRU | Past access order | Least recently used | Moderate |
| RRIP | Insert/hit markers | Predict reuse distance | Moderate-High |
| Hawkeye | OPTgen retrospection | Learn past to infer future | High |
| **L4 Predictor ** | **Future task queue ** | **Direct future visibility ** | **Perfect ** (within window) |
L4's Predictor is a **deterministic Oracle ** — it does not predict; it knows.
Because matrix computation task sequences are fully determined at scheduler allocation time.
### 6.2 NUMA Adaptability
The same code automatically gains NUMA benefits on different hardware configurations:
| Hardware Configuration | L4 Behavior |
|-------------------------|-------------------------------------------------------------|
| Single-socket CPU | Pack cache + segment accumulation buffer |
| Dual-socket CPU | Pack cache + NUMA migration cache + segment accum. |
| Hybrid-architecture CPU | Pack cache + cross-domain migration cache + segment accum. |
### 6.3 Callback-Driven Writeback Strategy
Using native function pointers for writeback avoids:
- Virtual method call overhead
- Interface dispatch overhead
- Delegate GC pressure
- Hard-coding specific writeback logic
Callers register `preprocess` and `writeback` callbacks at `MapWritable` time,
L4 invokes them directly through function pointers on eviction or Flush,
**zero managed overhead, zero virtual dispatch, performance equivalent to C function calls ** .
### 6.4 Reference Count Protection
Through `CacheLine` handle reference counting, cache lines actively in use by compute threads are
absolutely never evicted (including writeback not being triggered). The SIMD eviction scan skips
lines with `refCount > 0` immediately via the `_refFlags` array, without checking linked lists or acquiring locks.
### 6.5 Cache Line Size Aligned to Matrix Kernel
The 8192B cache line size is not arbitrary:
- 1 `double MAT_KERNEL` (4×
- 1 cache line = 64 `MAT_KERNEL` units
- Matrix multiplication micro-kernels operate on data sizes perfectly aligned to cache lines
- Compute kernel functions operate on cache lines as units; no cross-cache-line accesses occur
### 6.6 Fixed-Capacity Determinism
- Slab allocator never expands → memory footprint upper bound is known (64 MB)
- SIMD eviction scan → latency upper bound is known (at most 3 aging rounds ×
- Swiss Table fixed 16,384 slots → lookup latency upper bound is known
- Writeback callback latency is controlled by the caller → predictable
- Worst-case L4 latency = lookup + eviction (incl. writeback) + allocation + memcpy, all statically computable
### 6.7 Reuse of Existing StgSharp Infrastructure
| L4 Component | Reused StgSharp Infrastructure |
|-----------------------|-------------------------------------------------------|
| Mapping table | `SwissTable` (implemented and benchmark-validated) |
| Mapping table SIMD | `IBucketScanner` + `SIMDID` architecture detection |
| Cache line alloc | `SlabAllocator.FromBuffer<T>()` |
| Hash computation | Hardware `CRC32C` instruction |
| Memory allocation | `NativeMemory.AlignedAlloc` |
| Future multi-platform | `IBucketScanner` + NRGA compilation pipeline |
### 6.8 Originality Analysis
After surveying existing mainstream numerical computation libraries, L4's design has
**no direct precedent or equivalent in known public implementations ** .
#### NUMA Strategy Comparison with Existing Libraries
| Library / Framework | NUMA Strategy | Software Cache Layer | Mapping Table | Eviction Policy | Dirty Writeback |
|---------------------|-------------------------------------|---------------------|-----------------|-------------------------|--------------------|
| Intel MKL | Thread pinning + `numactl` | None | None | None | None |
| OpenBLAS | Thread affinity + block alignment | None | None | None | None |
| BLIS | Multi-level block alignment L1/L2/L3| None | None | None | None |
| Eigen | No explicit NUMA handling | None | None | None | None |
| cuBLAS / rocBLAS | GPU memory (hardware-solved) | None | None | None | None |
| **StgSharp L4 ** | **Software-managed thread-local cache ** | **Yes ** | **Swiss Table ** | **RRPV + windowed OPT ** | **Callback-driven ** |
---
## 7. 从 v0.1 到 v0.3 的变更摘要
| 项目 | v0.1 | v0.3 |
|-----------------|-------------------------------|---------------------------------------|
| 逐出数据结构 | 4 级环形双向链表 | RRPV 2-bit 掩码 + 8KB SoA 数组 |
| 逐出扫描方式 | 链表头指针遍历 | SSE2 批量模式匹配（16 行/周期） |
| 热� � � 更新 | 链表节点摘插（4 指针写） | 单字节位运算 `&= 0x03` |
| 老化操作 | 逐节点移链表 | SIMD 批量 +1 |
| 引用保护 | 无 | refCount > 0 时 SIMD 扫描跳过 |
| 预取策略 | 无 | Predictor 窗口 OPT（
| 线程模型 | L4 服务线程 + Command Queue | 无额外线程，CAS 抢锁 |
| 命令队列 | 链式 CommandCache 分段 | 不需要（预取刷新替代命令驱动） |
| 缓存行元数据 | Slab 分配的 CacheLineHead 结构 | SoA 平行数组 + 函数指针数组 |
| 工作模式 | 单一模式 | StreamTransparent / CacheBuffer 切换 |
| 写回策略 | 无（只读缓存） | dirty 位 + preprocess/writeback 回调 |
| 写回机制 | 无 | 本机函数指针，零托管开销 |
---
## 8. 未来工作
- [ ] L4 根类型重构（组合 Swiss Table、Slab、SoA 元数据、Predictor）
- [ ] Predictor 与矩阵乘法调度器的集成
- [ ] Segment 累加写回的 AVX2 向量化实现
- [ ] SIMD 逐出扫描的 AVX2 升级（32 行/周期）
- [ ] Swiss Table ARM NEON 实现
- [ ] Swiss Table AVX-512 升级（64 槽扫描从 16 次降至 4 次）
- [ ] 墓碑累积监控与自动 rehash 策略
- [ ] 性能基准测试（NUMA 本地 vs 远端访问延迟对比）
- [ ] Segment 累加 vs 直接写回的 benchmark 对比
- [ ] 与逐线程矩阵上下文的集成验证
- [ ] 双路 6242R 环境下的 NUMA 搬运性能验证
- [ ] 远期：PPC64LE 平台的 VSX 原生 Swiss Table 扫描支持
---
## 9. 参考
- Google Abseil Swiss Tables Design Notes
- HP Labs RRIP (Re-Reference Interval Prediction) 论文
- Hawkeye: Efficient Cache Replacement via OPTgen (ISCA 2017)
- StgSharp HLSF Allocator 设计文档
- StgSharp SIMD Architecture (`SIMDID` 体系)
- StgSharp NRGA Roadmap
- StgSharp `SlabAllocator<T>` 实现
- StgSharp `SwissTable` 实现及 Benchmark
Youmu Liu