diff --git a/StgSharp.Common/HighPerformance/Memory/L4_design.md b/StgSharp.Common/HighPerformance/Memory/L4_design.md index 7bbbae4e..4f46c369 100644 --- a/StgSharp.Common/HighPerformance/Memory/L4_design.md +++ b/StgSharp.Common/HighPerformance/Memory/L4_design.md @@ -1,215 +1,219 @@ -# L4 软件缓存器设计方案 +# L4 Software Cache Design Specification -> 项目：StgSharp -> 作者：Nitload -> 版本：草案 v0.3 -> 日期：2026-03-10 +> Project: StgSharp +> Author: Nitload +> Version: Draft v0.3 +> Date: 2026-03-10 --- -## 1. 目的 +## 1. Purpose -### 1.1 问题背景 +### 1.1 Problem Background -StgSharp 的矩阵计算系统正从"全局单一上下文表"迁移为"逐线程上下文"架构， -以适配混合架构 CPU（如 Intel P-Core / E-Core）的调度特性。 -迁移后每个线程独立持有计算上下文，但矩阵数据仍可能由 HLSF 分配器 -或全局 `malloc` 分配在远端 NUMA 节点上，导致跨节点访问延迟显著增大。 +StgSharp's matrix computation system is migrating from a "single global context table" to a +"per-thread context" architecture to adapt to the scheduling characteristics of hybrid-architecture +CPUs (e.g., Intel P-Core / E-Core). After the migration, each thread independently holds its own +computation context, but matrix data may still be allocated by the HLSF allocator or global +`malloc` on a remote NUMA node, resulting in significantly increased cross-node access latency. -同时，矩阵乘法（FMA）中 A 矩阵的核行数据需要从棋盘形 kernel grid 布局 -重排（Pack）为微内核友好的连续排列。此 Pack 操作在 OpenBLAS 等库中 -每次 tile 切换都重新执行，而 L4 可缓存 Pack 结果避免重复重排。 +At the same time, in matrix multiplication (FMA), the core row data of matrix A in the +checkerboard-shaped kernel grid layout must be repacked (Packed) into a micro-kernel-friendly +contiguous arrangement. In libraries such as OpenBLAS, this Pack operation is re-executed on +every tile switch, whereas L4 can cache the Pack results to avoid repeated reordering. -此外，矩阵乘法的分段累加（segment）需要将中间 partial sum 暂存在本地， -避免每个 segment 都跨 NUMA 写回目标矩阵。L4 可作为 partial sum 的 -本地累加缓冲区，所有 segment 在本地完成后一次性写回。 +Additionally, the segmented accumulation (segment) of matrix multiplication requires storing +intermediate partial sums locally to avoid cross-NUMA writes to the target matrix on every +segment. L4 can serve as a local accumulation buffer for partial sums, writing them back in a +single operation after all segments complete locally. -### 1.2 解决方案 +### 1.2 Solution -L4 缓存器是一个纯软件实现的**数据缓存层**， -位于 CPU 硬件缓存（L1/L2/L3）与主存之间的逻辑位置。 -其核心职责： +The L4 cache is a purely software-implemented **data caching layer** logically positioned +between the CPU hardware caches (L1/L2/L3) and main memory. Its core responsibilities: -1. **Pack 结果缓存**：缓存 FMA 中 A 核行的重排结果，避免重复 Pack -2. **NUMA 搬运缓存**：将远端 NUMA 数据搬运到本地并缓存，消除跨节点延迟 -3. **Segment 累加缓冲**：暂存 FMA 分段计算的 partial sum，减少远端写操作 -4. **PLU 中间数据缓冲**：为 PLU 分解提供本地连续工作区 +1. **Pack result caching**: Cache the repack results of FMA core rows in matrix A to avoid repeated Pack operations +2. **NUMA migration cache**: Migrate data from remote NUMA nodes to local memory and cache it, eliminating cross-node latency +3. **Segment accumulation buffer**: Temporarily store partial sums from FMA segmented computation to reduce remote write operations +4. **PLU intermediate data buffer**: Provide a local contiguous workspace for PLU decomposition -### 1.3 设计目标 +### 1.3 Design Goals -- 为逐线程矩阵计算上下文提供 NUMA 友好的本地数据访问 -- 固定容量（64 MB），不做运行时扩容，确定性延迟 -- 透明的指针重定向：上层代码通过原始指针查找，L4 返回本地缓存指针 -- 基于 **RRPV 掩码 + 窗口 OPT 预测** 的缓存逐出策略 -- 支持 **StreamTransparent（直通）** 和 **CacheBuffer（缓冲区）** 两种工作模式 -- 支持**可写缓存行**，逐出或显式刷新时通过回调函数预处理并写回 +- Provide NUMA-friendly local data access for per-thread matrix computation contexts +- Fixed capacity (64 MB), no runtime expansion, deterministic latency +- Transparent pointer redirection: upper-layer code queries via original pointers; L4 returns local cache pointers +- Cache eviction policy based on **RRPV mask + windowed OPT prediction** +- Support both **StreamTransparent** and **CacheBuffer** working modes +- Support **writable cache lines** that invoke registered callback functions for pre-processing and write-back upon eviction or explicit flush --- -## 2. 整体架构 +## 2. Overall Architecture -### 2.1 三组件模型 +### 2.1 Three-Component Model ``` ┌���─────────────────────────────────────────────────────────────────┐ │ L4 │ │ │ │ ┌──────────────┐ ┌────────────────────┐ ┌──────────────────┐ │ -│ │ CacheLine │ │ L4 本体 │ │ Predictor │ │ -│ │ 缓存行句柄 │ │ 缓存器 │ │ 预取预测器 │ │ -│ │ │ │ │ │ │ │ -│ │ • 本地地址 │ │ • Swiss Table │ │ • 2048 条缓冲 │ │ -│ │ • 引用计数 │ │ • Slab 分配器 │ │ • 1024 阈值 │ │ -│ │ • Dispose │ │ • RRPV 掩码 │ │ • 窗口 OPT │ │ -│ │ 释放引用 │ │ • SIMD 逐出 │ │ • 活跃 key 集 │ │ -│ │ │ │ • 写回回调管理 │ │ │ │ +│ │ CacheLine │ │ L4 Core │ │ Predictor │ │ +│ │ Handle │ │ Cache │ │ Prefetch │ │ +│ │ │ │ │ │ Predictor │ │ +│ │ • local addr │ │ • Swiss Table │ │ • 2048-entry buf │ │ +│ │ • ref count │ │ • Slab Allocator │ │ • threshold 1024 │ │ +│ │ • Dispose │ │ • RRPV mask │ │ • windowed OPT │ │ +│ │ drops ref │ │ • SIMD eviction │ │ • active key set │ │ +│ │ │ │ • writeback mgmt │ │ │ │ │ └──────────────┘ └────────────────────┘ └──────────────────┘ │ │ │ └──────────────────────────────────────────────────────────────────┘ ``` -### 2.2 数据访问路径 +### 2.2 Data Access Path ``` -计算线程访问数据： +Compute thread accessing data: ┌─────────────────────────────────────────────────────────┐ - │ 只读路径：L4.Map(originKey) │ - │ ├─ StreamTransparent + 无需重排？ │ - │ │ └─ 是 → 直接返回原始指针（零开销） │ + │ Read-only path: L4.Map(originKey) │ + │ ├─ StreamTransparent + no repack needed? │ + │ │ └─ Yes → return original pointer directly │ + │ │ (zero overhead) │ │ │ │ - │ └─ CacheBuffer 模式 │ - │ ├─ Swiss Table 查找（2.6ns，无锁） │ - │ ├─ 命中 → 更新 RRPV=0，返回 CacheLine 句柄 │ - │ └─ 未命中 → SpinWait 等待 Predictor 预取完成 │ + │ └─ CacheBuffer mode │ + │ ├─ Swiss Table lookup (2.6ns, lock-free) │ + │ ├─ Hit → update RRPV=0, return CacheLine handle │ + │ └─ Miss → SpinWait for Predictor prefetch │ ├─────────────────────────────────────────────────────────┤ - │ 可写路径：L4.MapWritable(originKey, preprocess, writeback) │ - │ ├─ 命中 → 返回已有的 partial sum 缓存行 │ - │ └─ 未命中 → 分配零初始化缓存行，注册写回回调 │ - │ 缓存行标记为脏（dirty=1） │ - │ 逐出时自动调用 preprocess + writeback │ + │ Writable path: L4.MapWritable(originKey, preprocess, writeback) │ + │ ├─ Hit → return existing partial sum cache line │ + │ └─ Miss → allocate zero-initialized cache line, │ + │ register writeback callbacks │ + │ cache line marked dirty (dirty=1) │ + │ eviction invokes preprocess + writeback │ ├─────────────────────────────────────────────────────────┤ - │ 计算完成后： │ - │ 3. CacheLine.Dispose()（refCount--） │ + │ After computation: │ + │ 3. CacheLine.Dispose() (refCount--) │ │ 4. L4.StepComplete() │ │ ├─ Predictor.AdvanceExecutionPoint() │ │ └─ Predictor.NeedsPrefetch? │ - │ └─ 是 → CAS 抢锁，执行预取刷新 │ + │ └─ Yes → CAS-acquire lock, execute prefetch │ └─────────────────────────────────────────────────────────┘ ``` -### 2.3 关键参数 +### 2.3 Key Parameters -| 参数 | 值 | 说明 | -|---------------------|------------------------|-----------------------------------| -| 缓存区总容量 | 64 MB | per-NUMA-node 连续内存 | -| 缓存行大小 | 8192 Bytes | = 64 个 `double MAT_KERNEL` 的大小 | -| 缓存行总数 | 8192 | 64MB / 8192B | -| 映射表类型 | Swiss Table | 1024 桶 × 16 槽，SIMD 加速 | -| 缓存行存储池 | Slab Allocator | 非并发、固定容量、不自动扩容 | -| 逐出策略 | RRPV 掩码 + 窗口 OPT | SoA 布局 + SSE2 批量扫描 | -| 预取策略 | Predictor 窗口 OPT | 2048 条缓冲，超前 1024 步 | -| 写回策略 | 回调驱动 | 脏行逐出/显式刷新时调用注册的回调 | +| Parameter | Value | Description | +|------------------------|--------------------------|------------------------------------------------------| +| Total cache capacity | 64 MB | Per-NUMA-node contiguous memory | +| Cache line size | 8192 Bytes | = 64 × `double MAT_KERNEL` | +| Cache line count | 8192 | 64MB / 8192B | +| Mapping table type | Swiss Table | 1024 buckets × 16 slots, SIMD-accelerated | +| Cache line pool | Slab Allocator | Non-concurrent, fixed capacity, no auto-expansion | +| Eviction policy | RRPV mask + windowed OPT | SoA layout + SSE2 batch scan | +| Prefetch policy | Predictor windowed OPT | 2048-entry buffer, 1024-step lookahead | +| Writeback policy | Callback-driven | Invokes registered callbacks on dirty eviction/flush | --- -## 3. 重要组件 +## 3. Key Components -### 3.1 Swiss Table（指针映射表） +### 3.1 Swiss Table (Pointer Mapping Table) -**职责**：维护 `原始指针 → 缓存指针` 的映射关系。 +**Responsibility**: Maintains the `original pointer → cache pointer` mapping. -- 键（Key）：原始指针 `nint`，指向 HLSF/malloc 分配的远端数据 -- 值（Value）：缓存指针 `nint`，指向本地 Slab 分配的缓存行 -- 哈希：使用硬件 `CRC32C` 指令对 64 位指针计算哈希 - - 低 10 位 → 桶索引（1024 个桶） - - 第 10-16 位 → H2 控制字节（用于 SIMD 预筛选） -- 扫描：SSE2 `pmovmskb` 指令并行比较 16 个控制字节 -- 墓碑机制：删除时 ctrl 设为 `0xFE`，保护探测链完整性 -- 容量：1024 × 16 = 16,384 条映射，远大于缓存行总数（8192 行）， - 保证极低的哈希碰撞率 +- Key: original pointer `nint`, pointing to data allocated by HLSF/malloc on a remote node +- Value: cache pointer `nint`, pointing to locally Slab-allocated cache line +- Hash: hardware `CRC32C` instruction on the 64-bit pointer + - Low 10 bits → bucket index (1024 buckets) + - Bits 10-16 → H2 control byte (used for SIMD pre-filtering) +- Scan: SSE2 `pmovmskb` instruction for parallel comparison of 16 control bytes +- Tombstone mechanism: deleted entries set ctrl to `0xFE`, preserving probe chain integrity +- Capacity: 1024 × 16 = 16,384 mappings, significantly larger than total cache line count (8192 lines), + ensuring extremely low hash collision rate -**性能实测**（Ryzen 7 6800H, .NET 8, BenchmarkDotNet）： +**Measured Performance** (Ryzen 7 6800H, .NET 8, BenchmarkDotNet): -| 操作 | Swiss Table | Dictionary | 倍速 | -|------------------------|-------------|-------------|--------| -| Zipfian Lookup (8192次) | 20.0 μs | 76.1 μs | 3.8x | -| Uniform Lookup (8192次) | 18.6 μs | 50.6 μs | 2.7x | -| Insert (4096 条) | 18.5 μs | 46.5 μs | 2.5x | -| SteadyState (4096 ops) | 25.3 μs | 68.3 μs | 2.7x | +| Operation | Swiss Table | Dictionary | Speedup | +|--------------------------|-------------|-------------|---------| +| Zipfian Lookup (8192×) | 20.0 μs | 76.1 μs | 3.8× | +| Uniform Lookup (8192×) | 18.6 μs | 50.6 μs | 2.7× | +| Insert (4096 entries) | 18.5 μs | 46.5 μs | 2.5× | +| SteadyState (4096 ops) | 25.3 μs | 68.3 μs | 2.7× | -单次查找延迟 ≈ 2.6 ns，GC 分配 = 0。 +Single lookup latency ≈ 2.6 ns, GC allocation = 0. -### 3.2 Slab Allocator（缓存行存储池） +### 3.2 Slab Allocator (Cache Line Pool) -**职责**：管理固定大小（8192B）缓存行的分配与回收。 +**Responsibility**: Manages allocation and reclamation of fixed-size (8192B) cache lines. -- 模式：**非并发、固定容量**（`FixedSlabAllocator` 模式）， - 复用现有 `SlabAllocator.FromBuffer()` 接口 -- 不自动扩容：容量在初始化时确定（64MB / 8192B = 8192 行）， - 分配失败时触发逐出流程而非扩容 -- 分配复杂度：O(1)，从空闲栈弹出 -- 回收复杂度：O(1)，压回空闲栈 -- 对齐：所有缓存行 64B 对齐，确保不跨硬件缓存行边界 +- Mode: **Non-concurrent, fixed capacity** (`FixedSlabAllocator` mode), + reusing the existing `SlabAllocator.FromBuffer()` interface +- No auto-expansion: capacity is fixed at initialization (64MB / 8192B = 8192 lines); + allocation failure triggers eviction rather than expansion +- Allocation complexity: O(1), pops from free stack +- Reclamation complexity: O(1), pushes back to free stack +- Alignment: all cache lines are 64B aligned, ensuring no cross-hardware-cache-line boundaries -### 3.3 逐出元数据（SoA 布局 + RRPV 掩码 + 写回回调） +### 3.3 Eviction Metadata (SoA Layout + RRPV Mask + Writeback Callbacks) -**职责**：追踪缓存行的引用状态、热度和写回信息，决定逐出优先级。 +**Responsibility**: Tracks per-cache-line reference state, heat, and writeback information to determine eviction priority. -取代 v0.1 中的 4 级环形链表，采用 Structure-of-Arrays 布局的平行数组： +Replaces the four-level circular linked list from v0.1 with parallel arrays in Structure-of-Arrays layout: ``` -SoA 布局 — 平行数组 +SoA Layout — Parallel Arrays - _refFlags[i] : byte 引用计数快照（0=空闲, 非0=被持有） + _refFlags[i] : byte Reference count snapshot (0=free, non-zero=held) _evictionFlags[i] : byte dirty(1bit) + RRPV(2bit) + costClass(1bit) - _originKeys[i] : nuint 原始指针（逐出时反向注销 Swiss Table） - _preprocess[i] : nint 写回预处理函数指针（脏行逐出前调用） - _writeback[i] : nint 写回函数指针（脏行逐出时调用） - _callbackCtx[i] : nuint 回调上下文（传递给 preprocess/writeback） + _originKeys[i] : nuint Original pointer (reverse-deregisters Swiss Table entry on eviction) + _preprocess[i] : nint Writeback pre-processing function pointer (called before dirty eviction) + _writeback[i] : nint Writeback function pointer (called during dirty eviction) + _callbackCtx[i] : nuint Callback context (passed to preprocess/writeback) ``` -**逐出标志位布局**（`_evictionFlags[i]`）： +**Eviction Flag Bit Layout** (`_evictionFlags[i]`): -RRPV数组使用byte存储每一个cacheline的使用情况，其低二位表示如下逻辑 +The RRPV array uses byte storage for each cache line's usage state; the low 2 bits encode the RRPV value. -**RRPV 语义**： +**RRPV Semantics**: -| RRPV | 含义 | 逐出优先级 | -|------|--------------------------|-----------| -| 0 | 马上会被再次访问 | 最低（不逐出）| -| 1 | 一会儿会被访问 | 低 | -| 2 | 比较久以后才会被访问 | 中 | -| 3 | 可能再也不会被访问了 | 最高（优先逐出）| +| RRPV | Meaning | Eviction Priority | +|------|---------------------------------|---------------------------| +| 0 | Will be re-accessed very soon | Lowest (do not evict) | +| 1 | Will be accessed shortly | Low | +| 2 | Will be accessed in a while | Medium | +| 3 | May never be accessed again | Highest (evict first) | -**RRPV 操作时机**： +**RRPV Update Events**: -| 时机 | 操作 | 说明 | -|---------------|-----------------------------------|-----------------------------| -| 加载时（插入） | RRPV = 0 | 刚加载的数据马上要用 | -| 命中时 | `_evictionFlags[i] = 0` | bit 1-2 清零，RRPV 归零 | -| 逐出扫描时 | 找 RRPV=3 的行逐出 | 找不到则全体 RRPV+1（老化） | +| Event | Operation | Description | +|--------------------|---------------------------------|------------------------------------------------------| +| Load (insert) | RRPV = 0 | Freshly loaded data will be used soon | +| Hit | `_evictionFlags[i] = 0` | Bits 1-2 cleared, RRPV reset to 0 | +| Eviction scan | Find lines with RRPV=3 to evict | If none found, increment all RRPV +1 (aging) | -**相比 v0.1 的 4 级环形链表的优势**： +**Advantages over the v0.1 four-level circular linked list**: -| 对比项 | v0.1 环形链表 | v0.3 RRPV 掩码 + 回调 | -|---------------|------------------------|------------------------------| -| 数据结构 | 4 条双向链表 + next 指针 | byte 数组 + 函数指针数组 | -| 升温操作 | 摘节点 + 插节点（4 指针）| `+= 1`（1 字节写入） | -| 老化操作 | 逐节点移链表 | SIMD 批量 +1（16 个/周期） | -| 逐出扫描 | 遍历链表头（指针追逐） | SIMD 模式匹配（16 个/周期） | -| 缓存友好性 | 差（指针跳跃） | 好（连续内存扫描） | -| 引用保护 | 无 | refCount > 0 时不可逐出 | -| 写回支持 | 无 | 回调函数驱动 | +| Comparison | v0.1 Circular List | v0.3 RRPV Mask + Callbacks | +|----------------------|-------------------------------------|--------------------------------------| +| Data structure | 4 doubly-linked lists + next ptrs | Byte array + function pointer arrays | +| Warm-up operation | Unlink + relink (4 pointer writes) | `+= 1` (1-byte write) | +| Aging operation | Per-node list migration | SIMD batch +1 (16 entries/cycle) | +| Eviction scan | Iterate list head (pointer chasing) | SIMD pattern matching (16/cycle) | +| Cache friendliness | Poor (pointer chasing) | Good (contiguous memory scan) | +| Reference protection | None | Lines with refCount > 0 not evicted | +| Writeback support | None | Callback-function driven | -### 3.4 写回回调机制 +### 3.4 Writeback Callback Mechanism -**职责**：为可写缓存行提供灵活的写回策略，通过本机函数指针避免托管开销。 +**Responsibility**: Provides flexible writeback strategies for writable cache lines, using native function pointers to avoid managed overhead. -**设计原则**：所有可写缓存行默认视为脏数据。逐出或显式刷新时， -L4 本身不负责管理写回策略，写回策略由绑定的预测器负责实现。 +**Design Principle**: All writable cache lines are treated as dirty by default. On eviction or explicit flush, +L4 itself does not manage the writeback strategy; instead, the bound predictor is responsible for implementing it. -预测器通过如下的接口进行定义 +The predictor is defined via the following interface: ``` public interface IPrefetchPredict @@ -234,64 +238,66 @@ L4 本身不负责管理写回策略，写回策略由绑定的预测器负责 } ``` -**典型回调实现**： +**Typical Callback Implementations**: ``` -场景 1：Segment 累加（FMA partial sum） - preprocess = null（无需预处理，数据已经是 double 数组） - writeback = AccumulateAdd（逐元素 dst[i] += src[i]，AVX2 向量化） +Scenario 1: Segment accumulation (FMA partial sum) + preprocess = null (no pre-processing needed; data is already a double array) + writeback = AccumulateAdd (element-wise dst[i] += src[i], AVX2-vectorized) -场景 2：Pack 重排的中间结果写回 - preprocess = UnpackKernelRow（将 Pack 格式反向重排为行主序） +Scenario 2: Writeback of packed intermediate results + preprocess = UnpackKernelRow (reverse-repack from Pack format to row-major order) writeback = AccumulateAdd -场景 3：PLU 分解中间数据 +Scenario 3: PLU decomposition intermediate data preprocess = null - writeback = DirectCopy（直接覆写 memcpy） + writeback = DirectCopy (direct overwrite memcpy) ``` -### 3.5 CacheLine（缓存行句柄） +### 3.5 CacheLine (Cache Line Handle) -**职责**：计算线程持有的缓存行访问句柄，通过引用计数防止正在使用的行被逐出。 +**Responsibility**: A cache line access handle held by the compute thread; prevents in-use lines from being evicted via reference counting. ```csharp public ref struct CacheLine : IDisposable { - private ref int _activeCount; // L4 全局活跃计数 - private ref int _refCount; // 该缓存行的引用计数 - private nuint _handle; // 本地缓存数据地址 + private ref int _activeCount; // L4 global active count + private ref int _refCount; // Per-line reference count + private nuint _handle; // Local cache data address - // 构造时：Interlocked.Increment(ref _activeCount) - // Interlocked.Increment(ref _refCount) - // Dispose：Interlocked.Decrement 两个计数器 + // Construction: Interlocked.Increment(ref _activeCount) + // Interlocked.Increment(ref _refCount) + // Dispose: Interlocked.Decrement on both counters } ``` -- `ref struct` 确保不会逃逸到堆上 -- 引用计数通过 `Interlocked` 原子操作保证线程安全 -- `_refCount > 0` 的缓存行在逐出扫描中被跳过 -- StreamTransparent 模式下返回的 CacheLine 不占用缓存槽位 -- 只读路径和可写路径返回相同的 CacheLine 类型， - 调用者通过 `.Address` 决定读写行为 +- `ref struct` ensures it cannot escape to the heap +- Reference counts use `Interlocked` atomic operations for thread safety +- Lines with `_refCount > 0` are skipped during eviction scans +- StreamTransparent mode returns a CacheLine that does not occupy a cache slot +- Both read-only and writable paths return the same CacheLine type, + callers decide read/write behavior via `.Address` -### 3.6 Predictor（预取预测器） +### 3.6 Predictor (Prefetch Predictor) -**职责**：基于窗口 OPT 策略，为 L4 提供确定性的预取预测。 +**Responsibility**: Provides deterministic prefetch predictions for L4 based on the windowed OPT strategy. -**核心原理**：矩阵计算的任务序列是确定性的——调度器预先分配了每个线程的 -任务块 `(kr, kc, zz)`，因此可以精确计算未来每步任务需要的缓存行 key。 -这使得 L4 拥有硬件缓存不可能具备的预测能力：**直接看到未来**， -而非像 Hawkeye 等策略那样从历史推断未来。 +**Core Principle**: Matrix computation task sequences are deterministic — the scheduler pre-allocates +task blocks `(kr, kc, zz)` for each thread, allowing precise computation of the cache line keys +required at each future step. This gives L4 a predictive capability that hardware caches cannot +possess: **direct visibility into the future**, rather than inferring the future from history as +strategies like Hawkeye do. ``` -Predictor 滑动窗口： +Predictor Sliding Window: - 任务序列：... [T_now] [T+1] [T+2] ... [T+1023] ... [T+2047] - ↑ 执行位点 ↑ 缓冲区尾部 + Task sequence: ... [T_now] [T+1] [T+2] ... [T+1023] ... [T+2047] + ↑ execution point ↑ buffer tail - 窗口容量 = 2048 条预测条目 - 预取触发阈值 = 预取前沿领先执行位点不足 1024 步时触发刷新 - 每次刷新批量 = 64 条（避免单次刷新耗时过长） + Window capacity = 2048 prediction entries + Prefetch trigger threshold = trigger refill when prefetch frontier + leads execution point by fewer than 1024 steps + Refill batch size = 64 entries (avoids excessive single-refill latency) ``` **数据结构**： @@ -302,53 +308,53 @@ internal class Predictor private const int WindowCapacity = 2048; private const int RefillThreshold = 1024; - private PrefetchEntry[] _buffer; // 环形预测缓冲区 - private int _head; // 预取前沿（已预取到此） - private int _tail; // 生产位置（调度器填充到此） - private int _executed; // 执行位点（计算线程已完成到此） + private PrefetchEntry[] _buffer; // Circular prediction buffer + private int _head; // Prefetch frontier (prefetched up to here) + private int _tail; // Production position (scheduler fills up to here) + private int _executed; // Execution point (compute thread has completed up to here) - private HashSet _activeKeys; // 窗口内所有活跃 key（去重） + private HashSet _activeKeys; // All active keys within the window (deduplicated) } struct PrefetchEntry { - public nuint Key; // 缓存行标识 - public nuint SourceAddress; // 源数据地址 - public bool NeedsRepack; // 是否需要非连续重排 + public nuint Key; // Cache line identifier + public nuint SourceAddress; // Source data address + public bool NeedsRepack; // Whether non-contiguous repacking is needed } ``` -**与逐出策略的协作**： +**Cooperation with Eviction Policy**: ``` -逐出优先级（从先逐出到后逐出）： +Eviction priority (from highest to lowest priority): - 1. refCount==0, 不在窗口中, RRPV==3, dirty==0 ← 最先（零成本逐出） - 2. refCount==0, 不在窗口中, RRPV==3, dirty==1 ← 其次（需写回但已冷） - 3. refCount==0, 不在窗口中, RRPV<3, dirty==0 ← 按 RRPV 排序 - 4. refCount==0, 不在窗口中, RRPV<3, dirty==1 ← 按 RRPV 排序 - 5. refCount==0, 在窗口中 ← 尽量不逐出 - 6. refCount>0 ← 绝不逐出 + 1. refCount==0, not in window, RRPV==3, dirty==0 ← First (zero-cost eviction) + 2. refCount==0, not in window, RRPV==3, dirty==1 ← Second (needs writeback but cold) + 3. refCount==0, not in window, RRPV<3, dirty==0 ← Ordered by RRPV + 4. refCount==0, not in window, RRPV<3, dirty==1 ← Ordered by RRPV + 5. refCount==0, in window ← Avoid evicting if possible + 6. refCount>0 ← Never evict ``` -窗口 OPT 使得大多数逐出决策无需依赖 RRPV——"不在窗口中"已足以判定可逐出。 -RRPV 在窗口外的行中做细粒度区分；dirty 位在同等 RRPV 下优先逐出 -干净行以避免写回开销。 +Windowed OPT means most eviction decisions do not depend on RRPV — "not in window" alone is +sufficient to determine evictability. RRPV provides fine-grained differentiation among +out-of-window lines; the dirty bit preferentially evicts clean lines at equal RRPV to avoid writeback overhead. -### 3.7 并发模型 +### 3.7 Concurrency Model -L4 采用**无额外线程**设计，所有操作由计算线程自身完成： +L4 uses a **no-extra-threads** design; all operations are performed by compute threads themselves: -| 操作 | 执行者 | 并发性 | -|------------------------|-------------------|-------------------------------| -| Lookup（Swiss Table 读）| 各计算线程 | ✅ 完全并行，无锁 | -| HIT 更新 RRPV | 各计算线程 | ✅ 单字节写，无需同步 | -| refCount 增减 | 各计算线程 | ✅ Interlocked 原子操作 | -| 预取刷新（逐出+加载） | CAS 抢锁的单个线程 | 串行（仅一个线程进入） | -| 脏行写回 | CAS 抢锁的线程 | 串行（在逐出流程内执行） | -| FlushAllDirty | 计算结束后单线程 | 串行（无并发） | +| Operation | Executor | Concurrency | +|------------------------------------|---------------------------------|--------------------------------------| +| Lookup (Swiss Table read) | Any compute thread | ✅ Fully parallel, lock-free | +| HIT RRPV update | Any compute thread | ✅ Single-byte write, no sync needed | +| refCount increment/decrement | Any compute thread | ✅ Interlocked atomic operations | +| Prefetch refresh (eviction + load) | CAS-winner thread | Serial (only one thread enters) | +| Dirty writeback | CAS-winner thread | Serial (within eviction flow) | +| FlushAllDirty | Single thread after computation | Serial (no concurrency) | -**预取刷新 CAS 锁**： +**Prefetch Refresh CAS Lock**: ```csharp private int _refreshLock; // 0=free, 1=locked @@ -356,87 +362,88 @@ private int _refreshLock; // 0=free, 1=locked private void TryRefreshPrefetch() { if (Interlocked.CompareExchange(ref _refreshLock, 1, 0) != 0) - return; // 另一个线程在做了，当前线程继续计算 - try { /* 逐出 + 预取 */ } + return; // Another thread is doing it; current thread continues computing + try { /* eviction + prefetch */ } finally { Volatile.Write(ref _refreshLock, 0); } } ``` -**Swiss Table 并发读安全保证**：预取刷新线程插入时遵守严格写顺序： -先写 Slab 数据，再写逐出元数据和回调指针，`sfence`，最后写 Swiss Table 的 -控制字节和 key/value，确保计算线程读到控制字节时数据已完整可见。 +**Swiss Table Concurrent Read Safety Guarantee**: The prefetch refresh thread follows strict write ordering +on insertion: write Slab data first, then write eviction metadata and callback pointers, `sfence`, then +write the Swiss Table control byte and key/value, ensuring that when a compute thread reads the control +byte, the data is already fully visible. --- -## 4. 操作流程 +## 4. Operation Flows -### 4.1 缓存查找（只读命中） +### 4.1 Cache Lookup (Read-Only Hit) -1. 计算原始指针的 CRC32C 哈希 -2. Swiss Table 查找：SIMD 扫描 ctrl → 比较 key → 返回缓存指针 -3. 更新 RRPV：`_evictionFlags[slot] &= 0xF9`（RRPV 归零，dirty 位不变） -4. 构造 CacheLine 句柄（refCount++） -5. 返回 CacheLine，上层代码通过 `.Address` 直接访问本��数据 +1. Compute CRC32C hash of the original pointer +2. Swiss Table lookup: SIMD scan ctrl → compare key → return cache pointer +3. Update RRPV: `_evictionFlags[slot] &= 0xF9` (RRPV zeroed, dirty bit unchanged) +4. Construct CacheLine handle (refCount++) +5. Return CacheLine; upper-layer code accesses local data via `.Address` -### 4.2 缓存查找（可写命中） +### 4.2 Cache Lookup (Writable Hit) -1. Swiss Table 查找命中 -2. 更新 RRPV：`_evictionFlags[slot] &= 0xF9` -3. 构造 CacheLine 句柄（refCount++） -4. 返回 CacheLine，上层代码通过 `.Address` 读写缓存行（如 partial sum +=） +1. Swiss Table lookup hits +2. Update RRPV: `_evictionFlags[slot] &= 0xF9` +3. Construct CacheLine handle (refCount++) +4. Return CacheLine; upper-layer code reads/writes the cache line via `.Address` (e.g., partial sum +=) -### 4.3 缓存未命中（只读） +### 4.3 Cache Miss (Read-Only) -1. Swiss Table 查找未命中 -2. 计算线程进入 Thread.Sleep(0) 等待循环 -3. 等待 Predictor 驱动的预取刷新将该行加载进 L4 -4. Swiss Table 出现命中后退出等待 -5. 构造 CacheLine 句柄并返回 +1. Swiss Table lookup misses +2. Compute thread enters `Thread.Sleep(0)` spin loop +3. Wait for Predictor-driven prefetch refresh to load the line into L4 +4. Exit wait loop when Swiss Table shows a hit +5. Construct CacheLine handle and return -正常运行时，Predictor 始终超前执行位点 1024 步以上， -miss 仅在冷启动或极端突发时发生。 +Under normal operation, the Predictor always leads the execution point by 1024+ steps, +misses occur only during cold start or extreme bursts. -### 4.4 缓存未命中（可写） +### 4.4 Cache Miss (Writable) -1. Swiss Table 查找未命中 -2. 从 Slab 分配缓存行 -3. 若 Slab 无空闲 → 触发逐出（见 4.6） -4. 缓存行零初始化（`Unsafe.InitBlock(dest, 0, 8192)`） -5. 注册写回回调：`_preprocess[slot]`, `_writeback[slot]`, `_callbackCtx[slot]` -6. 设置逐出标志：`dirty=1, RRPV=0` -7. 写入 `_originKeys[slot]` +1. Swiss Table lookup misses +2. Allocate a cache line from Slab +3. If Slab has no free slots → trigger eviction (see 4.6) +4. Zero-initialize the cache line (`Unsafe.InitBlock(dest, 0, 8192)`) +5. Register writeback callbacks: `_preprocess[slot]`, `_writeback[slot]`, `_callbackCtx[slot]` +6. Set eviction flags: `dirty=1, RRPV=0` +7. Write `_originKeys[slot]` 8. `sfence` -9. 写入 Swiss Table 映射 -10. 构造 CacheLine 句柄并返回 +9. Write Swiss Table mapping +10. Construct CacheLine handle and return -### 4.5 预取刷新 +### 4.5 Prefetch Refresh -由计算线程在 `StepComplete()` 中触发，CAS 抢锁确保单线程执行： +Triggered by compute threads in `StepComplete()`; CAS lock ensures single-thread execution: -1. 同步 refFlags 快照（`_refCounts[i] > 0 → _refFlags[i] = 1`） -2. 从 Predictor 取出待预取条目（每次最多 64 条） -3. 对每个条目： - - Swiss Table 命中 → 标记 RRPV=0（热行保护） - - Swiss Table 未命中 → 执行加载： - a. Slab 分配缓存行 - b. 若 Slab 无空闲 → 触发逐出（见 4.6） - c. 根据 `NeedsRepack` 标志执行 Pack 重排或 memcpy 搬运 - d. 写入逐出元数据（originKey, RRPV=0, dirty=0, 回调=null） +1. Synchronize refFlags snapshot (`_refCounts[i] > 0 → _refFlags[i] = 1`) +2. Retrieve pending prefetch entries from Predictor (up to 64 at a time) +3. For each entry: + - Swiss Table hit → mark RRPV=0 (hot-line protection) + - Swiss Table miss → execute load: + a. Allocate cache line from Slab + b. If Slab has no free slots → trigger eviction (see 4.6) + c. Execute Pack reorder or memcpy based on `NeedsRepack` flag + d. Write eviction metadata (originKey, RRPV=0, dirty=0, callbacks=null) e. `sfence` - f. 写入 Swiss Table 映射 + f. Write Swiss Table mapping -### 4.6 缓存逐出（SIMD 扫描 + 写回） +### 4.6 Cache Eviction (SIMD Scan + Writeback) ``` -逐出扫描伪代码： +Eviction scan pseudocode: - Pass 1: 找 refCount==0, RRPV==3, dirty==0（零成本逐出） + Pass 1: Find refCount==0, RRPV==3, dirty==0 (zero-cost eviction) for i in range(scanCursor, CacheLineCount), step=16: refChunk = SSE2.Load(refFlags + i) refFreeMask = SSE2.MoveMask(SSE2.CompareEqual(refChunk, zero)) evChunk = SSE2.Load(evictionFlags + i) - // RRPV==3 且 dirty==0 → byte & 0x07 == 0x06 + // RRPV==3 and dirty==0 → byte & 0x07 == 0x06 masked = SSE2.And(evChunk, 0x07) coldCleanMask = SSE2.MoveMask(SSE2.CompareEqual(masked, 0x06)) @@ -445,16 +452,16 @@ miss 仅在冷启动或极端突发时发生。 bit = TrailingZeroCount(candidates) idx = i + bit if originKeys[idx] not in Predictor.ActiveKeys: - evict(idx) // 无写回，直接丢弃 + evict(idx) // No writeback, discard directly return candidates &= candidates - 1 - Pass 2: 找 refCount==0, RRPV==3, dirty==1（需写回） - 同上，但匹配 byte & 0x07 == 0x07 - evict 时先执行写回回调 + Pass 2: Find refCount==0, RRPV==3, dirty==1 (needs writeback) + Same as above, but match byte & 0x07 == 0x07 + evict triggers writeback callbacks - Pass 3: 全体 RRPV+1（老化），重试 - 最多 3 轮 + Pass 3: Increment all RRPV+1 (aging), retry + Up to 3 rounds ``` **逐出执行**： @@ -464,25 +471,25 @@ private void Evict(int slot) { byte flags = _evictionFlags[slot]; - // 脏行写回 + // Dirty writeback if ((flags & 0x01) != 0) { nuint origin = _originKeys[slot]; nuint cached = SlotAddressFromIndex(slot); nuint ctx = _callbackCtx[slot]; - // 1. 预处理（可选） + // 1. Pre-processing (optional) var preprocess = (delegate* unmanaged)_preprocess[slot]; if (preprocess != null) preprocess(cached, ctx); - // 2. 写回 + // 2. Writeback var writeback = (delegate* unmanaged)_writeback[slot]; if (writeback != null) writeback(origin, cached, ctx); } - // 标准逐出 + // Standard eviction _map.TryRemove(_originKeys[slot], out _); _slab.Free(SlotAddressFromIndex(slot)); _evictionFlags[slot] = 0; @@ -497,7 +504,7 @@ private void Evict(int slot) ```csharp /// -/// 显式将指定脏缓存行写回。在所有 segment 完成后调用。 +/// Explicitly writes back the specified dirty cache line. Called after all segments complete. /// public void FlushDirty(nuint originKey) { @@ -506,15 +513,15 @@ public void FlushDirty(nuint originKey) int slot = SlotIndex(addr); if ((_evictionFlags[slot] & 0x01) != 0) { - // 执行预处理 + 写回 + // Execute pre-processing + writeback InvokeWriteBack(slot); - _evictionFlags[slot] &= 0xFE; // 清除 dirty 位 + _evictionFlags[slot] &= 0xFE; // Clear dirty bit } } } /// -/// 批量写回所有脏缓存行。在整个矩阵计算结束后调用。 +/// Writes back all dirty cache lines in bulk. Called after entire matrix computation completes. /// public void FlushAllDirty() { @@ -529,39 +536,39 @@ public void FlushAllDirty() } ``` -### 4.8 工作模式切换 +### 4.8 Working Mode Switch ```csharp public enum L4Mode { /// - /// 直通模式：Map() 直接返回原始指针，L4 不介入。 - /// 适用于单 NUMA + 无需重排的场景（如向量加法、标量乘法）。 + /// Pass-through mode: Map() returns the original pointer directly; L4 does not intervene. + /// Suitable for single-NUMA + no-repack scenarios (e.g., vector addition, scalar multiplication). /// StreamTransparent, /// - /// 缓冲区模式：查找/Pack/缓存/逐出全流程。 - /// 适用于需要重排的 FMA 场景 或 多 NUMA 场景。 + /// Buffer mode: full lookup/Pack/cache/eviction pipeline. + /// Suitable for FMA scenarios requiring repacking or multi-NUMA scenarios. /// CacheBuffer } ``` -模式切换在构造时确定，运行时不变。判断逻辑： +Mode is determined at construction time and does not change at runtime. Decision logic: -- 需要 Pack 重排（FMA 的 A 核行）→ CacheBuffer -- 多 NUMA 节点 → CacheBuffer -- 需要 segment 累加 → CacheBuffer -- 单 NUMA + 无需重排 + 无 segment → StreamTransparent +- Requires Pack reordering (FMA core rows of matrix A) → CacheBuffer +- Multiple NUMA nodes → CacheBuffer +- Requires segment accumulation → CacheBuffer +- Single NUMA + no reordering + no segment → StreamTransparent --- -## 5. Segment 累加场景 +## 5. Segment Accumulation Scenario -### 5.1 问题 +### 5.1 Problem -矩阵乘法 `C = A × B` 当 K 维度很大时，需要分段累加： +For matrix multiplication `C = A × B` when the K dimension is large, segmented accumulation is required: ``` segment 0: C_partial = A[kr, 0..KC] × B[0..KC, kc] @@ -569,141 +576,141 @@ segment 1: C_partial += A[kr, KC..2KC] × B[KC..2KC, kc] ... segment N: C_partial += A[kr, N*KC..K] × B[N*KC..K, kc] -最终 C[kr, kc] = Σ 所有 segment 的 partial sum +Final: C[kr, kc] = Σ all segment partial sums ``` -如果直接将每个 segment 的结果 `+=` 到 C 矩阵， -且 C 在远端 NUMA 节点上，则每个 segment 都有跨节点写延迟。 +If each segment's result is `+=` directly to the C matrix, +and C resides on a remote NUMA node, every segment incurs cross-node write latency. -### 5.2 L4 解决方案 +### 5.2 L4 Solution -将 C 的 partial sum 暂存在 L4 的可写缓存行中： +Temporarily store C's partial sums in L4's writable cache lines: ``` segment 0: c = l4.MapWritable(C_key, null, &AccumulateAdd) - // 首次未命中 → 分配零初始化缓存行 + // First miss → allocate zero-initialized cache line FMA(c.Address, a.Address, b.Address) // c += a*b c.Dispose() segment 1: c = l4.MapWritable(C_key, null, &AccumulateAdd) - // 命中！返回已有的 partial sum 缓存行 + // Hit! Returns existing partial sum cache line FMA(c.Address, a.Address, b.Address) // c += a*b c.Dispose() -... 所有 segment 完成后 ... +... after all segments complete ... l4.FlushDirty(C_key) -// 调用 AccumulateAdd(C_origin, cacheline, ctx) -// → 将本地 partial sum 一次性加回远端 C 矩阵 +// Calls AccumulateAdd(C_origin, cacheline, ctx) +// → Adds local partial sum back to remote C matrix in one shot ``` -### 5.3 收益 +### 5.3 Benefits ``` -不用 segment 缓存（OpenBLAS 做法）： - 每个 segment 都跨 NUMA 写 C 矩阵 - 远端写次数 = Zk 次 - 每次写延迟 = ~200-300ns（跨节点） +Without segment buffering (OpenBLAS approach): + Each segment writes C matrix across NUMA + Remote writes per output tile = Zk times + Per-write latency = ~200-300ns (cross-node) -用 L4 segment 缓存： - 所有 segment 的 += 在本地 L4 缓存行完成 - partial sum 大概率留在 L1/L2（8KB 远小于 L1 容量） - 远端写次数 = 1 次（最终 FlushDirty） - 写回用 AVX2 向量化：8192B = 32 次 256-bit store +With L4 segment buffering: + All segment += operations complete locally in L4 cache lines + Partial sums likely remain in L1/L2 (8KB << L1 capacity) + Remote writes per output tile = 1 (final FlushDirty) + Writeback uses AVX2-vectorized stores: 8192B = 32 × 256-bit stores - 远端写减少比 = Zk : 1 + Reduction ratio = Zk : 1 ``` --- -## 6. 独特性与优势 +## 6. Uniqueness and Advantages -### 6.1 确定性预取（窗口 OPT） +### 6.1 Deterministic Prefetch (Windowed OPT) -与硬件缓存替换策略（LRU, RRIP, Hawkeye）的本质区别： +Essential difference from hardware cache replacement policies (LRU, RRIP, Hawkeye): -| 策略 | 信息来源 | 预测方式 | 精度 | -|------------------|-------------|-------------------|-----------| -| LRU | 过去的访问顺序| 最近最少使用 | 中 | -| RRIP | 插入/命中标记| 预测重用距离 | 中高 | -| Hawkeye | OPTgen 回溯 | 学习过去推断未来 | 高 | -| **L4 Predictor** | **未来任务队列** | **直接看到未来** | **完美**（窗口内）| +| Policy | Information Source | Prediction Method | Accuracy | +|---------------------|-------------------------|-------------------------------|---------------------------| +| LRU | Past access order | Least recently used | Moderate | +| RRIP | Insert/hit markers | Predict reuse distance | Moderate-High | +| Hawkeye | OPTgen retrospection | Learn past to infer future | High | +| **L4 Predictor** | **Future task queue** | **Direct future visibility** | **Perfect** (within window) | -L4 的 Predictor 是**确定性的 Oracle**——它不预测，它知道。 -因为矩阵计算的任务序列在调度器分配时就已完全确定。 +L4's Predictor is a **deterministic Oracle** — it does not predict; it knows. +Because matrix computation task sequences are fully determined at scheduler allocation time. -### 6.2 NUMA 自适应 +### 6.2 NUMA Adaptability -同一套代码在不同硬件配置下自动获得 NUMA 收益： +The same code automatically gains NUMA benefits on different hardware configurations: -| 硬件配置 | L4 行为 | -|-----------------|-------------------------------------------| -| 单路 CPU | Pack 缓存 + segment 累加缓冲 | -| 双路 CPU | Pack 缓存 + NUMA 搬运缓存 + segment 累加 | -| 混合架构 CPU | Pack 缓存 + 跨域搬运缓存 + segment 累加 | +| Hardware Configuration | L4 Behavior | +|-------------------------|-------------------------------------------------------------| +| Single-socket CPU | Pack cache + segment accumulation buffer | +| Dual-socket CPU | Pack cache + NUMA migration cache + segment accum. | +| Hybrid-architecture CPU | Pack cache + cross-domain migration cache + segment accum. | -### 6.3 回调驱动的写回策略 +### 6.3 Callback-Driven Writeback Strategy -通过本机函数指针实现写回，避免了： +Using native function pointers for writeback avoids: -- 虚方法调用开销 -- 接口分发开销 -- 委托的 GC 压力 -- 硬编码特定写回逻辑 +- Virtual method call overhead +- Interface dispatch overhead +- Delegate GC pressure +- Hard-coding specific writeback logic -调用者在 `MapWritable` 时注册 preprocess 和 writeback 回调， -L4 在逐出或 Flush 时直接通过函数指针调用， -**零托管开销，零虚分发，与 C 函数调用等价的性能**。 +Callers register `preprocess` and `writeback` callbacks at `MapWritable` time, +L4 invokes them directly through function pointers on eviction or Flush, +**zero managed overhead, zero virtual dispatch, performance equivalent to C function calls**. -### 6.4 引用计数保护 +### 6.4 Reference Count Protection -通过 `CacheLine` 句柄的引用计数，正在被计算线程使用的缓存行 -绝对不会被逐出（包括不会触发写回）。SIMD 逐出扫描通过 `_refFlags` -数组在第一时间跳过 `refCount > 0` 的行，无需检查链表或加锁。 +Through `CacheLine` handle reference counting, cache lines actively in use by compute threads are +absolutely never evicted (including writeback not being triggered). The SIMD eviction scan skips +lines with `refCount > 0` immediately via the `_refFlags` array, without checking linked lists or acquiring locks. -### 6.5 缓存行大小与矩阵 Kernel 对齐 +### 6.5 Cache Line Size Aligned to Matrix Kernel -8192B 的缓存行大小不是任意选取的： +The 8192B cache line size is not arbitrary: -- 1 个 `double MAT_KERNEL`（4×4 double 矩阵）= 128 Bytes -- 1 个缓存行 = 64 个 `MAT_KERNEL` -- 矩阵乘法的微内核每次处理的数据量与缓存行完全对齐 -- 计算核函数以缓存行为单位操作，不存在跨缓存行的情况 +- 1 `double MAT_KERNEL` (4×4 double matrix) = 128 Bytes +- 1 cache line = 64 `MAT_KERNEL` units +- Matrix multiplication micro-kernels operate on data sizes perfectly aligned to cache lines +- Compute kernel functions operate on cache lines as units; no cross-cache-line accesses occur -### 6.6 固定容量的确定性 +### 6.6 Fixed-Capacity Determinism -- Slab 分配器不扩容 → 内存占用上界已知（64 MB） -- SIMD 逐出扫描 → 延迟上界已知（最多 3 轮老化 × 512 次 SIMD 比较） -- Swiss Table 固定 16,384 槽位 → 查找延迟上界已知 -- 写回回调延迟可由调用者控制 → 可预测 -- 整个 L4 的最坏延迟 = 查找 + 逐出(含写回) + 分配 + memcpy，全部可静态计算 +- Slab allocator never expands → memory footprint upper bound is known (64 MB) +- SIMD eviction scan → latency upper bound is known (at most 3 aging rounds × 512 SIMD comparisons) +- Swiss Table fixed 16,384 slots → lookup latency upper bound is known +- Writeback callback latency is controlled by the caller → predictable +- Worst-case L4 latency = lookup + eviction (incl. writeback) + allocation + memcpy, all statically computable -### 6.7 与 StgSharp 现有基础设施的复用 +### 6.7 Reuse of Existing StgSharp Infrastructure -| L4 组件 | 复用的 StgSharp 基础设施 | -|----------------|----------------------------------------| -| 映射表 | `SwissTable`（已实现，经过 benchmark 验证）| -| 映射表 SIMD | `IBucketScanner` + `SIMDID` 架构检测体系 | -| 缓存行分配 | `SlabAllocator.FromBuffer()` | -| 哈希计算 | 硬件 `CRC32C` 指令 | -| 内存分配 | `NativeMemory.AlignedAlloc` | -| 未来多平台支持 | `IBucketScanner` + NRGA 编译管道 | +| L4 Component | Reused StgSharp Infrastructure | +|-----------------------|-------------------------------------------------------| +| Mapping table | `SwissTable` (implemented and benchmark-validated) | +| Mapping table SIMD | `IBucketScanner` + `SIMDID` architecture detection | +| Cache line alloc | `SlabAllocator.FromBuffer()` | +| Hash computation | Hardware `CRC32C` instruction | +| Memory allocation | `NativeMemory.AlignedAlloc` | +| Future multi-platform | `IBucketScanner` + NRGA compilation pipeline | -### 6.8 开创性分析 +### 6.8 Originality Analysis -经过对现有主流数值计算库的调研，L4 缓存器的设计在已知公开实现中 -**没有直接的先例或等价物**。 +After surveying existing mainstream numerical computation libraries, L4's design has +**no direct precedent or equivalent in known public implementations**. -#### 现有库的 NUMA 策略对比 +#### NUMA Strategy Comparison with Existing Libraries -| 库 / 框架 | NUMA 策略 | 软件缓存层 | 映射表 | 逐出策略 | 脏行写回 | -|-------------------|-------------------------------|-----------|-----------|-------------------|------------| -| Intel MKL | 线程绑核 + `numactl` | 无 | 无 | 无 | 无 | -| OpenBLAS | 线程亲和性 + 分块对齐 | 无 | 无 | 无 | 无 | -| BLIS | 多级分块对齐 L1/L2/L3 | 无 | 无 | 无 | 无 | -| Eigen | 无显式 NUMA 处理 | 无 | 无 | 无 | 无 | -| cuBLAS / rocBLAS | GPU 显存（硬件解决） | 无 | 无 | 无 | 无 | -| **StgSharp L4** | **软件管理的线程本地缓存** | **有** | **Swiss Table** | **RRPV+窗口OPT** | **回调驱动** | +| Library / Framework | NUMA Strategy | Software Cache Layer | Mapping Table | Eviction Policy | Dirty Writeback | +|---------------------|-------------------------------------|---------------------|-----------------|-------------------------|--------------------| +| Intel MKL | Thread pinning + `numactl` | None | None | None | None | +| OpenBLAS | Thread affinity + block alignment | None | None | None | None | +| BLIS | Multi-level block alignment L1/L2/L3| None | None | None | None | +| Eigen | No explicit NUMA handling | None | None | None | None | +| cuBLAS / rocBLAS | GPU memory (hardware-solved) | None | None | None | None | +| **StgSharp L4** | **Software-managed thread-local cache** | **Yes** | **Swiss Table** | **RRPV + windowed OPT** | **Callback-driven** | ---