diff --git a/ReadMe.md b/ReadMe.md index 5b6371ba..4d4ca78e 100644 --- a/ReadMe.md +++ b/ReadMe.md @@ -34,12 +34,24 @@ Numerics and linear algebra built on top of the core library: Nitload General Regular Analysis (NGRA) infrastructure for recognizing and processing regular languages: -- **Regex/Automata Core**: Regular expression processing and automata utilities +- **NFA Direction**: Planned generic ε-NFA sequence execution, followed by weighted automata and NGRA-Tensor research - **Parsing Primitives**: Reusable tokenizer and grammar building blocks - **Analysis Pipeline**: Structured processing for language-like inputs NGRA is a foundation for higher-level language tooling. It does not replace `Nitload.Script`; the planned Script implementation depends on NGRA and has not started yet. The source currently under `Nitload.Script` is retained from the earlier EXPRESS implementation and is not part of the active core solution. +The former Text Regex parser, matcher, and source generator are [archived](src/Nitload.RegularAnalysis/Legacy/README.md) +and excluded from compilation. Generic AST and token infrastructure remain active. + +Although Text Regex development is frozen, it produced valuable reusable results: the five-link +AST model, tree normalization and merging techniques, token/analysis infrastructure, and practical +experience with COUNT continuations, group boundaries, and rollback. These results remain available +to inform the new NFA architecture. + +Future string-regex functionality will use .NET `GeneratedRegex`. NGRA will no longer pursue +custom `Span` or SIMD optimizations for string matching; its research focus is generalized +sequence analysis and, later, weighted relation networks. + ### Nitload.Graphics Managed graphics infrastructure built on Common and Mathematics: diff --git a/future/NGRA_Transformation_Plan.md b/future/NGRA_Transformation_Plan.md new file mode 100644 index 00000000..54f332ef --- /dev/null +++ b/future/NGRA_Transformation_Plan.md @@ -0,0 +1,916 @@ +# NGRA 转型计划 + +> 面向本地 Codex / 项目维护者的快速架构说明 +> 项目:Nitload General Regular Analyzer(NGRA) + +## 1. 转型结论 + +NGRA 不再以“高性能字符串正则表达式引擎”为主要目标。 + +原先围绕 `Span`、专用字符串匹配、复杂回溯、Source Generator 等方向继续投入的收益已经明显下降。后续核心工作转向: + +1. 以 **ε-NFA / NFA** 为基础执行模型; +2. 将输入从 `char` 推广为任意可比较、可判定类别的序列元素; +3. 面向 Token Stream、语法前端、半上下文模式分析等场景; +4. 支持自动机归并、分层自动机、状态/断言/前序引用; +5. 在此基础上继续研究 **Weighted Automaton / NGRA-Tensor**。 + +因此,Text Regex 从“主架构”降级为“兼容前端 / 参考实现 / 测试子系统”。 + +--- + +## 2. 当前代码处理原则 + +### 2.1 保留现有 Text Regex 半成品 + +当前 Text Regex Source Generator、COUNT/CONCAT/ALT lowering、生成式 matcher 等代码不删除。 + +原因: + +- 五角星 AST 模型仍然是有效成果; +- AST 搜索、规范化、旋转、合并等代码仍可复用; +- 当前实现记录了 COUNT continuation、group boundary、rollback 等真实工程问题; +- 可作为未来自动机 IR 与旧实现行为的对照; +- 删除会丢失大量探索成果。 + +建议: + +- 将旧生成器移动到 `Legacy` / `Experimental` / `Archive` 目录; +- 通过 `.csproj` 的 `Compile Remove` 或独立非构建项目,使其暂时不参与主编译; +- 不再优先修补完整 `.NET Regex` 兼容性。 + +### 2.2 五角星 AST 继续保留 + +现有节点关系: + +```text + Parent + ↑ +Previous ← Node → Next + ↙ ↘ + Left Right +``` + +继续作为通用语法树 / 结构树基础设施。 + +需要坚持: + +- AST 用于表达语法和结构; +- 自动机 / CFG / Relation Graph 用于表达执行与传播; +- 不再要求 AST 本身承担全部 backtracking / continuation 语义。 + +--- + +## 3. 新主线:ε-NFA + +第一阶段优先实现一个简单、正确、无性能野心的 ε-NFA 模拟器。 + +### 3.1 基础状态模型 + +```text +Current Active States + ↓ +ε-closure + ↓ +consume(input) + ↓ +Next Active States + ↓ +ε-closure +``` + +核心状态: + +```text +State +Transition +StartState +AcceptStates +ActiveStateSet +``` + +Transition 分为: + +```text +ε transition +matched transition +``` + +输入类型不能写死为 `char`。 + +建议: + +```csharp +EpsilonNfa +``` + +或: + +```csharp +EpsilonNfa +``` + +匹配条件可以抽象为: + +```csharp +bool Match(in TInput input) +``` + +或者后续升级为更明确的 Predicate / Category Matcher。 + +### 3.2 第一阶段支持的 Pattern + +第一版只要求: + +```text +literal / predicate +concat +alternation +* ++ +? +{m,n} +ordinary grouping +``` + +暂不追求: + +```text +capture rollback +复杂 greedy / lazy 行为 +lookaround +backreference +完整 .NET Regex compatibility +``` + +其中 `{m,n}` 可以优先在构造期展开为 NFA 结构,而不是运行时维护复杂 COUNT 控制流。 + +--- + +## 4. 为什么以 NFA 为主,而不是 DFA + +DFA 仍然有价值,但不再是必须经过的阶段。 + +核心路线: + +```text +Pattern + ↓ +ε-NFA + ↓ +NFA execution + ↓ +Weighted NFA + ↓ +NGRA-Tensor +``` + +DFA 只作为可选 specialization: + +```text +Boolean +Static +No dynamic relation +No weighted state +``` + +满足这些条件的局部子图,可以由编译器确定化为 DFA fast path。 + +总体原则: + +> NFA 是主语义模型,DFA 是可选优化后端。 + +原因: + +普通 NFA 的活动状态: + +```text +a_i ∈ {0,1} +``` + +加权机可自然推广为: + +```text +a_i ∈ R +``` + +Tensor 版本可以继续推广为: + +```text +a_i ∈ R^d +``` + +多状态同时活跃不再是需要“消除的非确定性”,而会成为模型本身的状态表示。 + +--- + +## 5. NGRA 的目标:广义序列分析 + +NGRA 不再局限于字符串。 + +典型输入: + +```text +char +token +syntax node +semantic node +sensor frame +event +instruction-like object +``` + +Pattern 的作用也从: + +```text +“这个字符串是不是符合 regex” +``` + +扩展为: + +```text +“这个序列在当前状态、历史、类别和上下文条件下, +是否满足某个结构化模式” +``` + +--- + +## 6. 半上下文 Token Analyzer + +NGRA 的近期目标之一,是成为一种介于普通 Regex 与完整 Parser 之间的 Token Stream Analyzer。 + +可用能力包括: + +```text +Token 类别匹配 +Token 属性匹配 +前序状态引用 +局部断言 +Pattern 状态 +符号绑定 +分层子自动机 +局部上下文 +``` + +因此它适合作为: + +```text +Lexer 后端 +Token Regex +语法前端 +编译器预分析器 +DSL Parser Frontend +结构化流分析器 +``` + +“半上下文”的含义不是严格的形式语言分类,而是: + +> Pattern 不只依赖当前 token,也可以依赖有限的已知状态、绑定、前序结果和结构信息,但不要求一开始就构造完整通用 Parser。 + +--- + +## 7. 自动机归并 + +大量 Pattern 应优先在 NFA 层归并,而不是各自独立执行。 + +例如: + +```text + ε → Pattern A + / +GlobalStart ─ε → Pattern B + \ + ε → Pattern C +``` + +Accepting State 可以附带: + +```text +PatternId +MatchKind +Payload +``` + +后续目标: + +- 合并公共路径; +- 消除冗余 ε 边; +- 状态压缩; +- bitset active state; +- pattern priority; +- 多层自动机组合。 + +--- + +## 8. 分层自动机 + +后续重点不是继续增加字符串 Regex 特性,而是建立分层自动机。 + +示意: + +```text +Scalar / Token Layer + ↓ +Local Pattern Layer + ↓ +Phrase / Region Layer + ↓ +Sentence / Structure Layer + ↓ +Semantic / Relation Layer +``` + +低层自动机可以生成高层节点,高层节点继续作为新序列输入。 + +即: + +```text +Input Stream + ↓ +Automaton L0 + ↓ +Derived Nodes + ↓ +Automaton L1 + ↓ +Higher-level Nodes +``` + +这应成为 NGRA 进入编译器前端、自然语言结构分析和复杂流分析的主路径。 + +--- + +## 9. Assertion / Context 的重新定义 + +不再围绕字符串 Regex 的 lookahead / lookbehind 设计核心架构。 + +通用定义应更接近: + +```text +Assert(Context) -> bool +``` + +或者: + +```text +RelationGate(State, Context, History) -> bool +``` + +因此: + +- 字符串前瞻/后顾只是特殊情况; +- 前序引用只是状态引用的一种; +- group/capture 不是核心抽象; +- Context / Binding / Region / History 才是可推广概念。 + +--- + +# NGRA-Tensor 构想 + +## 10. 基本方向 + +NGRA-Tensor 不是简单地“给 NFA 边加 float”。 + +它希望把: + +```text +Boolean multi-state automaton +``` + +推广为: + +```text +Weighted / Tensor Relation Network +``` + +基础演化: + +```text +NFA +a_i ∈ {0,1} + +↓ weighted + +Weighted Automaton +a_i ∈ R + +↓ tensor/state + +NGRA-Tensor +a_i ∈ R^d +``` + +--- + +## 11. Relation 成为一等对象 + +不再采用固定 `Port4` 作为核心语义。 + +推荐模型: + +```text +Source Node + ↓ +[ Relation ] + ↓ +Destination Node +``` + +Relation 包含: + +```text +Source +Destination +Type +Parameters +Weight / Weight Function +Optional State +Eligibility / Gate +``` + +语义上: + +```text +Relation: +决定一条联系是否生效, +以及它如何把 Source 信息贡献给 Destination。 +``` + +Node 负责保存状态。 + +Blend 负责汇合多条 Relation 的贡献。 + +--- + +## 12. Pull,而不是主动广播 + +GPU / 执行层优先采用 destination pull: + +```text +Destination Node + ↓ +请求自己依赖的 Relations + ↓ +读取 Source states + ↓ +Relation Eval + ↓ +Blend +``` + +而不是: + +```text +Source Node + ↓ +主动向大量 Destination 广播 +``` + +优点: + +- 减少 scatter; +- 减少原子写冲突; +- 减少 readiness 管理; +- 更适合 GPU destination-centric execution。 + +--- + +## 13. Relation 的三段语义 + +推荐拆成: + +```text +Eligibility + ↓ +Compatibility + ↓ +Transfer +``` + +即: + +1. **Eligibility**:这条 Relation 当前能不能参与; +2. **Compatibility**:参与后有多重要; +3. **Transfer**:真正传递什么信息。 + +形式上: + +```text +gate_ij = G(state, context, relation) +score_ij = F(query, key, relation) +value_ij = T(source_state, relation) +``` + +最终: + +```text +output_i = Blend(gate_ij * weight_ij * value_ij) +``` + +--- + +## 14. 与 Attention / QKV 的关系 + +QKV 可以作为 Relation 内部算子的第一种 baseline,但不应决定 NGRA-Tensor 的外层拓扑。 + +标准 Attention: + +```text +Q = “我需要什么” +K = “我适不适合被引用” +V = “真正传递的内容” +``` + +数学上: + +```text +score_ij = q_i · k_j +weight_ij = softmax(score_ij) +output_i = Σ weight_ij * v_j +``` + +NGRA-Tensor 的差异: + +- Relation 是显式、长期存在、可带类型的; +- 候选关系通常是稀疏的; +- 不一定对全部 node 做 dense attention; +- Relation 可以有自己的参数、状态、类型和 gate; +- Attention 可以只是某种 Relation Eval backend。 + +--- + +## 15. 稀疏关系,而不是全连接爆破 + +Transformer 的典型关系空间: + +```text +L × L +``` + +NGRA-Tensor 的目标: + +```text +E << N² +``` + +即只计算显式 Relation。 + +但必须承认: + +- 稀疏图带来 gather; +- 不规则访问会伤害 GPU; +- 小算子过多会降低吞吐。 + +因此不能简单逐 Relation 启动 kernel。 + +--- + +## 16. Relation Tile / 脑区式分块 + +后续 GPU backend 应研究: + +```text +Semantic Relation Graph + ↓ +Graph Partition + ↓ +Relation Tiles + ↓ +Block-sparse / Dense subgraph execution +``` + +目标: + +> 块内允许不规则访问,块间尽量转成规则张量传输。 + +示意: + +```text +Dense / semi-dense local region + ↓ +少量跨区 Relation + ↓ +another dense / semi-dense region +``` + +类似“脑区”: + +- 区内关系较密; +- 区间关系较稀; +- 每个区可以编译成中型 tensor operator; +- 跨区使用显式 Relation。 + +这可能成为 NGRA-Tensor 与传统 Transformer 之间的重要折中。 + +--- + +## 17. 语义稀疏 vs 物理稠密 + +前端语义可以是: + +```text +Sparse Relation Graph +``` + +GPU backend 可以通过: + +```text +node reorder +feature reorder +relation reorder +graph clustering +block detection +tile packing +kernel fusion +shared-memory staging +``` + +将其转为: + +```text +Dense blocks ++ +Block-sparse blocks ++ +Irregular sparse tails +``` + +原则: + +> 模型的图结构不等于 GPU 必须逐边解释。 + +--- + +## 18. Node State 可以是高维 Tensor + +NGRA-Tensor 不等于“每个 Node 只有一个 scalar”。 + +Node 可以持有: + +```text +x_i = [f0, f1, ..., fN] +``` + +但 Relation 不一定读取全部 feature。 + +因此可能存在两层稀疏: + +```text +Topology sparsity +Feature sparsity +``` + +Relation 可以只对特定 feature/subspace 感兴趣。 + +例如: + +```text +Relation A: +Q features = [1, 2, 7] +K features = [4, 8] +V features = [16, 17, 18] +``` + +语义上只取这些 feature。 + +物理执行时 GPU compiler 可以为了带宽一次读入更大的连续 block。 + +--- + +## 19. Tensor 观念 + +后续 NGRA-Tensor 设计不要把 Tensor 只理解为“算子”。 + +Tensor 可以同时是: + +```text +状态 +参数 +关系 +mask +中间结果 +数据集合 +算子 +``` + +“算子”只是某个 Tensor 在某次 contraction 中扮演的角色。 + +建议以“语义轴”理解 Tensor,例如: + +```text +Node × Feature +Destination × Source +Relation × Feature +Time × Node × Feature +``` + +优先思考: + +- 哪些轴保留; +- 哪些轴被收缩; +- 哪些轴表示关系; +- 哪些轴表示 feature; +- 哪些轴可以 sparse; +- 哪些轴可以 dense。 + +--- + +## 20. 训练 / 运行时关系发现 + +NGRA-Tensor 长期设想允许结构学习。 + +训练阶段: + +```text +短程强关系 + ↓ +词 / Token 局部关系 + ↓ +词组 / 短句 + ↓ +长句 / 语义 + ↓ +知识网络 / 长程弱关系 +``` + +运行时: + +当现有网络: + +```text +NoActivePath +LowConfidence +HighAmbiguity +Unknown +``` + +可进入较慢的 Relation Discovery: + +```text +Stable Graph + ↓ +Unknown + ↓ +Candidate Relation Search + ↓ +Candidate Graph + ↓ +验证 + ↓ +可能固化进入 Stable Graph +``` + +不建议运行时每一步都修改主图。 + +应区分: + +```text +Fast inference time +Slow structural learning time +``` + +--- + +# 近期开发优先级 + +## Phase 1 — ε-NFA Reference Runtime + +实现: + +```text +EpsilonNfa +StateId +Transition +ε-closure +Consume +Accept +``` + +目标: + +- 语义简单; +- 正确性优先; +- 不做复杂性能优化; +- 不绑定 `char`。 + +## Phase 2 — Pattern → NFA Lowering + +从现有 AST / Pattern IR 构造 NFA: + +```text +literal +predicate +concat +alt +repeat +``` + +## Phase 3 — NFA Merge + +实现: + +```text +multi-pattern merge +PatternId +shared start +accept payload +``` + +然后测试大量 Pattern 同时运行。 + +## Phase 4 — Token Stream + +将输入切换到: + +```text +Token +``` + +支持: + +```text +Token kind +Token value +Token metadata +simple binding +context predicate +``` + +目标:作为编译器前端 / Token Regex 原型。 + +## Phase 5 — Hierarchical Automata + +允许低层自动机输出高层节点,并再次进入更高层自动机。 + +## Phase 6 — Weighted Automaton + +将: + +```text +active : bool +``` + +推广为: + +```text +activation : float +``` + +先不要急于引入复杂 Tensor。 + +## Phase 7 — NGRA-Tensor Prototype + +研究: + +```text +Node state tensor +Typed Relation +Eligibility +Compatibility +Transfer +Blend +Sparse / block-sparse execution +``` + +QKV/Attention 可作为 baseline Relation operator。 + +--- + +# 非目标 + +近期不要优先投入: + +```text +完整 .NET Regex 兼容 +极端 Span 优化 +复杂 Source Generator matcher +完整 capture rollback +所有 lookaround/backreference +为了 Regex 完整性设计核心 IR +``` + +这些功能如确实需要,可以: + +```text +System.Text.RegularExpressions +``` + +作为字符串兼容层。 + +这不是架构退让,而是目标调整。 + +--- + +# 一句话定位 + +NGRA 的新定位: + +> **以 NFA 为基础的广义序列模式与分层自动机框架,并逐步推广到加权 Relation Network 与 NGRA-Tensor。** + +Text Regex: + +> 一个兼容前端 / 参考实现,不再定义 NGRA 的核心架构。 + +NGRA-Tensor: + +> 把 Boolean NFA 的多状态传播推广为显式、稀疏、可加权、可训练、可分块执行的 Relation Network。 diff --git a/src/Nitload.RegularAnalysis/Abstraction/SequenceEmitter.cs b/src/Nitload.RegularAnalysis/Abstraction/SequenceEmitter.cs index fd300036..92d29754 100644 --- a/src/Nitload.RegularAnalysis/Abstraction/SequenceEmitter.cs +++ b/src/Nitload.RegularAnalysis/Abstraction/SequenceEmitter.cs @@ -5,7 +5,6 @@ // SPDX-License-Identifier: MIT // ----------------------------------------------------------------------------- -using Nitload.RegularAnalysis.Text; using System; using System.Collections; using System.Collections.Generic; diff --git a/src/Nitload.RegularAnalysis/GroupIndex.cs b/src/Nitload.RegularAnalysis/Legacy/GroupIndex.cs similarity index 100% rename from src/Nitload.RegularAnalysis/GroupIndex.cs rename to src/Nitload.RegularAnalysis/Legacy/GroupIndex.cs diff --git a/src/Nitload.RegularAnalysis/Legacy/README.md b/src/Nitload.RegularAnalysis/Legacy/README.md new file mode 100644 index 00000000..54102f26 --- /dev/null +++ b/src/Nitload.RegularAnalysis/Legacy/README.md @@ -0,0 +1,51 @@ +# Frozen Text Regex implementation + +Text Regex is archived following the [NGRA transformation plan](../../../future/NGRA_Transformation_Plan.md). +It is no longer NGRA's execution architecture or an active development target. + +## Retained results and future string matching + +Freezing development does not diminish the value of the work already completed. The project +produced reusable AST/token infrastructure, the five-link tree model and its normalization, +rotation, and merging techniques, as well as tested fixed-width COUNT retry logic. The archived +generator also records useful engineering lessons about continuations, group boundaries, label +ownership, and rollback. These are reference material for the new architecture, not a claim that +the old matcher is feature-complete. + +Future string-regex functionality will use .NET `GeneratedRegex`. Further research into custom +`Span` and SIMD optimizations for the Text Regex matcher is discontinued. This boundary +applies to the string-regex implementation, not to SIMD work elsewhere in Nitload. + +## Contents + +- `Text/`: the text tokenizer/parser, regex-specific AST and optimizers, matcher contract, + attribute, and all source-generator files. +- `GroupIndex.cs`: the former group index/name record. +- `SourceGenerate.cs`: the unused source-generation entry stub. +- [Archived debug routines](../../../test/StgSharpDebug/Legacy/TextRegex): the original + AST inspection and generated-source preview programs. + +Source contents and namespaces are preserved. The generic infrastructure in `../Abstraction/` +remains active, including the five-link node interface, generic AST, token abstractions, +compilation stack, and sequence emitter. Text-specific tree transformations remain here +as source material for future Pattern-to-NFA lowering. + +## Build boundary + +The project excludes `Legacy/**/*.cs` from `Compile` and retains the files as `None` items. +They are not copied to build/publish output or packed. The active project no longer references +Roslyn or packages its assembly as a source generator. Debug routines have the same exclusion +in their own project and are not called from `Main`. + +To investigate this implementation again, use a separate experimental project that explicitly +includes the archived files and supplies the former Roslyn dependencies: +`Microsoft.CodeAnalysis.CSharp` 4.13.0 and `Microsoft.CodeAnalysis.Analyzers` 3.11.0. +The debug preview also required Roslyn at runtime and the `StgSharpDebug` friend assembly. +Re-enabling files is not sufficient to make the unfinished matcher complete. + +Historical notes remain in [IR_RETIRED.md](../IR_RETIRED.md) and +[regex_node_transaction.md](../regex_node_transaction.md). They describe earlier experiments, +not the new NFA execution contract. Older `.discard` material is unchanged. + +The next runtime is planned around generic sequence input and Boolean ε-NFA execution; +weighted automata and NGRA-Tensor remain later development/research stages. diff --git a/src/Nitload.RegularAnalysis/SourceGenerate.cs b/src/Nitload.RegularAnalysis/Legacy/SourceGenerate.cs similarity index 100% rename from src/Nitload.RegularAnalysis/SourceGenerate.cs rename to src/Nitload.RegularAnalysis/Legacy/SourceGenerate.cs diff --git a/src/Nitload.RegularAnalysis/Text/RegexAnalyzer.ASTGenerate.cs b/src/Nitload.RegularAnalysis/Legacy/Text/RegexAnalyzer.ASTGenerate.cs similarity index 100% rename from src/Nitload.RegularAnalysis/Text/RegexAnalyzer.ASTGenerate.cs rename to src/Nitload.RegularAnalysis/Legacy/Text/RegexAnalyzer.ASTGenerate.cs diff --git a/src/Nitload.RegularAnalysis/Text/RegexAnalyzer.ASTNodeOptimize.cs b/src/Nitload.RegularAnalysis/Legacy/Text/RegexAnalyzer.ASTNodeOptimize.cs similarity index 100% rename from src/Nitload.RegularAnalysis/Text/RegexAnalyzer.ASTNodeOptimize.cs rename to src/Nitload.RegularAnalysis/Legacy/Text/RegexAnalyzer.ASTNodeOptimize.cs diff --git a/src/Nitload.RegularAnalysis/Text/RegexAnalyzer.ASTOptimize.cs b/src/Nitload.RegularAnalysis/Legacy/Text/RegexAnalyzer.ASTOptimize.cs similarity index 100% rename from src/Nitload.RegularAnalysis/Text/RegexAnalyzer.ASTOptimize.cs rename to src/Nitload.RegularAnalysis/Legacy/Text/RegexAnalyzer.ASTOptimize.cs diff --git a/src/Nitload.RegularAnalysis/Text/RegexAnalyzer.cs b/src/Nitload.RegularAnalysis/Legacy/Text/RegexAnalyzer.cs similarity index 100% rename from src/Nitload.RegularAnalysis/Text/RegexAnalyzer.cs rename to src/Nitload.RegularAnalysis/Legacy/Text/RegexAnalyzer.cs diff --git a/src/Nitload.RegularAnalysis/Text/RegexAnanlyzer.ASTNodeOptimize.cs b/src/Nitload.RegularAnalysis/Legacy/Text/RegexAnanlyzer.ASTNodeOptimize.cs similarity index 100% rename from src/Nitload.RegularAnalysis/Text/RegexAnanlyzer.ASTNodeOptimize.cs rename to src/Nitload.RegularAnalysis/Legacy/Text/RegexAnanlyzer.ASTNodeOptimize.cs diff --git a/src/Nitload.RegularAnalysis/Text/RegexAstNode.AdditionalPayload.cs b/src/Nitload.RegularAnalysis/Legacy/Text/RegexAstNode.AdditionalPayload.cs similarity index 100% rename from src/Nitload.RegularAnalysis/Text/RegexAstNode.AdditionalPayload.cs rename to src/Nitload.RegularAnalysis/Legacy/Text/RegexAstNode.AdditionalPayload.cs diff --git a/src/Nitload.RegularAnalysis/Text/RegexAstNode.cs b/src/Nitload.RegularAnalysis/Legacy/Text/RegexAstNode.cs similarity index 100% rename from src/Nitload.RegularAnalysis/Text/RegexAstNode.cs rename to src/Nitload.RegularAnalysis/Legacy/Text/RegexAstNode.cs diff --git a/src/Nitload.RegularAnalysis/Text/RegexElementLabel.cs b/src/Nitload.RegularAnalysis/Legacy/Text/RegexElementLabel.cs similarity index 100% rename from src/Nitload.RegularAnalysis/Text/RegexElementLabel.cs rename to src/Nitload.RegularAnalysis/Legacy/Text/RegexElementLabel.cs diff --git a/src/Nitload.RegularAnalysis/Text/RegexTokenParser.cs b/src/Nitload.RegularAnalysis/Legacy/Text/RegexTokenParser.cs similarity index 100% rename from src/Nitload.RegularAnalysis/Text/RegexTokenParser.cs rename to src/Nitload.RegularAnalysis/Legacy/Text/RegexTokenParser.cs diff --git a/src/Nitload.RegularAnalysis/Text/RegexTokenReader.cs b/src/Nitload.RegularAnalysis/Legacy/Text/RegexTokenReader.cs similarity index 100% rename from src/Nitload.RegularAnalysis/Text/RegexTokenReader.cs rename to src/Nitload.RegularAnalysis/Legacy/Text/RegexTokenReader.cs diff --git a/src/Nitload.RegularAnalysis/Text/TextRegex.cs b/src/Nitload.RegularAnalysis/Legacy/Text/TextRegex.cs similarity index 100% rename from src/Nitload.RegularAnalysis/Text/TextRegex.cs rename to src/Nitload.RegularAnalysis/Legacy/Text/TextRegex.cs diff --git a/src/Nitload.RegularAnalysis/Text/TextRegexAttribute.cs b/src/Nitload.RegularAnalysis/Legacy/Text/TextRegexAttribute.cs similarity index 100% rename from src/Nitload.RegularAnalysis/Text/TextRegexAttribute.cs rename to src/Nitload.RegularAnalysis/Legacy/Text/TextRegexAttribute.cs diff --git a/src/Nitload.RegularAnalysis/Text/TextRegexSourceGen.Diagnostic.cs b/src/Nitload.RegularAnalysis/Legacy/Text/TextRegexSourceGen.Diagnostic.cs similarity index 100% rename from src/Nitload.RegularAnalysis/Text/TextRegexSourceGen.Diagnostic.cs rename to src/Nitload.RegularAnalysis/Legacy/Text/TextRegexSourceGen.Diagnostic.cs diff --git a/src/Nitload.RegularAnalysis/Text/TextRegexSourceGen.ExitLabel.cs b/src/Nitload.RegularAnalysis/Legacy/Text/TextRegexSourceGen.ExitLabel.cs similarity index 100% rename from src/Nitload.RegularAnalysis/Text/TextRegexSourceGen.ExitLabel.cs rename to src/Nitload.RegularAnalysis/Legacy/Text/TextRegexSourceGen.ExitLabel.cs diff --git a/src/Nitload.RegularAnalysis/Text/TextRegexSourceGen.GenClass.cs b/src/Nitload.RegularAnalysis/Legacy/Text/TextRegexSourceGen.GenClass.cs similarity index 100% rename from src/Nitload.RegularAnalysis/Text/TextRegexSourceGen.GenClass.cs rename to src/Nitload.RegularAnalysis/Legacy/Text/TextRegexSourceGen.GenClass.cs diff --git a/src/Nitload.RegularAnalysis/Text/TextRegexSourceGen.GenContext.cs b/src/Nitload.RegularAnalysis/Legacy/Text/TextRegexSourceGen.GenContext.cs similarity index 100% rename from src/Nitload.RegularAnalysis/Text/TextRegexSourceGen.GenContext.cs rename to src/Nitload.RegularAnalysis/Legacy/Text/TextRegexSourceGen.GenContext.cs diff --git a/src/Nitload.RegularAnalysis/Text/TextRegexSourceGen.GenFile.cs b/src/Nitload.RegularAnalysis/Legacy/Text/TextRegexSourceGen.GenFile.cs similarity index 100% rename from src/Nitload.RegularAnalysis/Text/TextRegexSourceGen.GenFile.cs rename to src/Nitload.RegularAnalysis/Legacy/Text/TextRegexSourceGen.GenFile.cs diff --git a/src/Nitload.RegularAnalysis/Text/TextRegexSourceGen.GenSpecial.cs b/src/Nitload.RegularAnalysis/Legacy/Text/TextRegexSourceGen.GenSpecial.cs similarity index 100% rename from src/Nitload.RegularAnalysis/Text/TextRegexSourceGen.GenSpecial.cs rename to src/Nitload.RegularAnalysis/Legacy/Text/TextRegexSourceGen.GenSpecial.cs diff --git a/src/Nitload.RegularAnalysis/Text/TextRegexSourceGen.Source.cs b/src/Nitload.RegularAnalysis/Legacy/Text/TextRegexSourceGen.Source.cs similarity index 100% rename from src/Nitload.RegularAnalysis/Text/TextRegexSourceGen.Source.cs rename to src/Nitload.RegularAnalysis/Legacy/Text/TextRegexSourceGen.Source.cs diff --git a/src/Nitload.RegularAnalysis/Text/TextRegexSourceGen.cs b/src/Nitload.RegularAnalysis/Legacy/Text/TextRegexSourceGen.cs similarity index 100% rename from src/Nitload.RegularAnalysis/Text/TextRegexSourceGen.cs rename to src/Nitload.RegularAnalysis/Legacy/Text/TextRegexSourceGen.cs diff --git a/src/Nitload.RegularAnalysis/Nitload.RegularAnalysis.csproj b/src/Nitload.RegularAnalysis/Nitload.RegularAnalysis.csproj index c410a7ce..3fca8c47 100644 --- a/src/Nitload.RegularAnalysis/Nitload.RegularAnalysis.csproj +++ b/src/Nitload.RegularAnalysis/Nitload.RegularAnalysis.csproj @@ -1,26 +1,14 @@ - - true - true - true - $(BaseIntermediateOutputPath)Generated - - - - - - - - - - - + + + + diff --git a/src/README.md b/src/README.md index 2960c206..24efcff3 100644 --- a/src/README.md +++ b/src/README.md @@ -10,7 +10,7 @@ This directory contains the production source code, native runtime, supporting a | `Nitload.Mathematics` | Numeric types, vectors, matrices, geometry, and managed access to accelerated math operations. | | `Nitload.Graphics` | OpenGL bindings, rendering infrastructure, shaders, textures, windows, and input integration. | | `Nitload.Native` | C native runtime, GLFW/STBI integration, allocators, platform code, and optimized SIMD/matrix kernels. See its own `README.md` for native build details. | -| `Nitload.RegularAnalysis` | NGRA regular-language analysis, regex processing, intermediate representations, and source-generation infrastructure. | +| `Nitload.RegularAnalysis` | NGRA generic AST/token infrastructure, transitioning toward ε-NFA sequence analysis. Text Regex and its source generator are archived under `Legacy` and excluded from compilation. | | `Nitload.UserInterface` | State-oriented user-interface framework and its incremental source generator. | | `Nitload.TerminalDialogue` | Terminal user-interface components and an interactive terminal application. | | `Nitload.Benchmark` | BenchmarkDotNet workloads for measuring selected runtime and allocation behavior. | diff --git a/test/README.md b/test/README.md index 167dcd3e..7dc11bae 100644 --- a/test/README.md +++ b/test/README.md @@ -16,36 +16,28 @@ Run the active debug harness directly when investigating a subsystem: dotnet run --project test/StgSharpDebug/StgSharpDebug.csproj ``` -## NGRA generation preview +## Archived Text Regex diagnostics -`StgSharpDebug/TestRegexGenerate.cs` contains a standalone debugging routine that calls the -existing NGRA emitter without Roslyn attribute binding. Select it from the debug `Main`: +The Text Regex analyzer and source-generator preview are frozen. Their original source files +are retained under [StgSharpDebug/Legacy/TextRegex](StgSharpDebug/Legacy/TextRegex) and excluded +from compilation, packaging, and build/publish copying. `Main` no longer calls them, and +`--regex-preview` is not an active entry point. -```powershell -dotnet run --project test/StgSharpDebug/StgSharpDebug.csproj -- --regex-preview -``` +The corresponding runtime, text-specific AST/parser, and generator are archived under +[src/Nitload.RegularAnalysis/Legacy](../src/Nitload.RegularAnalysis/Legacy/README.md). +Generic AST and token infrastructure remain active. See the +[NGRA transformation plan](../future/NGRA_Transformation_Plan.md) for the new NFA direction. -This branch returns before `World` or native-memory initialization. Without the switch, the -existing debug flow is unchanged. You can also call `TestRegexGenerate.Run()` directly from -`Main` while debugging; pass a pattern to `Run("abc")` for a single preview. -The switch only selects runtime behavior; building the debug project still builds its normal -project references, including Graphics and Mathematics, and prepares the native library with -CMake. `BuildNativeBeforeManaged` defaults to true; Debug selects `clang-debug`, and other -configurations select `clang-release`. `RequireNativeLibrary` reports a missing DLL at build -time instead of allowing the default runtime path to fail later. +## Native builds -For managed-only preview work without a native toolchain or DLL, add -`-p:BuildNativeBeforeManaged=false -p:RequireNativeLibrary=false` before `--` in the run command. +Building the debug project still builds its normal project references, including Graphics and +Mathematics, and prepares the native library with CMake. `BuildNativeBeforeManaged` defaults +to true; Debug selects `clang-debug`, and other configurations select `clang-release`. +`RequireNativeLibrary` reports a missing DLL at build time instead of allowing execution to +fail later. -Enter raw regular expressions without C# string delimiters. The routine prints only the emitted -code, without status headers or file output. Exceptions propagate directly to the caller/debugger. -Use `:exit` or end-of-input to leave the loop. Empty input is an empty regex. For one-shot mode: - -```powershell -dotnet run --project test/StgSharpDebug/StgSharpDebug.csproj -- --regex-preview 'a{2,3}' -``` - -Set breakpoints in `TestRegexGenerate.cs` or any existing `Generate*` method. The output is a -method-body fragment; this routine does not compile or execute it. +For a managed-only build without a native toolchain or DLL, pass +`-p:BuildNativeBeforeManaged=false -p:RequireNativeLibrary=false`. +This does not make debug routines that use native services runnable without the native library. New repeatable tests should use a test project and avoid machine-specific paths, external mutable data, and dependencies on previous test execution. Keep one-off investigations in `SingleFile` or the manual debug harness until their expected behavior is stable enough for automation. diff --git a/test/StgSharpDebug/TestRegexAnalyse.cs b/test/StgSharpDebug/Legacy/TextRegex/TestRegexAnalyse.cs similarity index 100% rename from test/StgSharpDebug/TestRegexAnalyse.cs rename to test/StgSharpDebug/Legacy/TextRegex/TestRegexAnalyse.cs diff --git a/test/StgSharpDebug/TestRegexGenerate.cs b/test/StgSharpDebug/Legacy/TextRegex/TestRegexGenerate.cs similarity index 100% rename from test/StgSharpDebug/TestRegexGenerate.cs rename to test/StgSharpDebug/Legacy/TextRegex/TestRegexGenerate.cs diff --git a/test/StgSharpDebug/Program.cs b/test/StgSharpDebug/Program.cs index 05d101de..4e99d3d8 100644 --- a/test/StgSharpDebug/Program.cs +++ b/test/StgSharpDebug/Program.cs @@ -37,8 +37,7 @@ namespace Nitload // .UseModule() .InitializeModule(); - TestRegexAnalyse.Run(); - _ = TestRegexGenerate.Run("abc1{1,2}23"); + // Text Regex diagnostics are frozen under Legacy/TextRegex and excluded from compilation. // TestHlsf(); diff --git a/test/StgSharpDebug/StgSharpDebug.csproj b/test/StgSharpDebug/StgSharpDebug.csproj index 70048fe6..badee4b0 100644 --- a/test/StgSharpDebug/StgSharpDebug.csproj +++ b/test/StgSharpDebug/StgSharpDebug.csproj @@ -14,8 +14,10 @@ - - + + + +