diff --git a/future/ngra-regex-sourcegen-goldens.md b/future/ngra-regex-sourcegen-goldens.md index fcb822b0..ac8cfa77 100644 --- a/future/ngra-regex-sourcegen-goldens.md +++ b/future/ngra-regex-sourcegen-goldens.md @@ -1,29 +1,56 @@ -# NGRA Regex 源生成手写样本 +# NGRA Regex 源生成 Checklist -这个文件用于记录源生成器实现前需要手写的正则匹配样本。 +本文件按**一次扫描消耗的前缀 IR 数量**分层，记录源生成器需要实现的形态。 +消耗量从高到低排列，从整个 region 的识别一直降到单条 `MATCH` 指令。 -流程： +## 为什么按 IR 消耗量分层 -1. 选一个表达式。 -2. 手写理想匹配代码。 -3. 生成优化后 IR。 -4. 对照手写代码和 IR 段。 -5. 标注每段 IR 应该生成的代码形状。 -6. 抽取生成器规则。 +早期版本按正则语法分批（基础匹配 / 计数 / ALT / 捕获组……），这描述的是**表达式形态**。 +但 `EmitPrefix` 之后，生成器面对的不再是表达式，而是**带前缀的指令窗口**： -选样原则： +- `a|b` 与 `(?abcd|a(bc)c)` 在语法上分属两批，在前缀 IR 里却共用同一个 + 「region 链 + `Prev_Fail` 切换」骨架； +- `abc` 与 `a[abc]d` 在语法上同属一批，消耗的 IR 条数却不同； +- `WorkingRegion` / `ExeCondition` / `ObserveMethod` / `PopCondition` + 是**横切在每条指令上**的，任何按语法组织的清单都无法表达它们。 -1. 表达式必须是典型正则写法。 -2. 全部样本合起来应覆盖计划支持的大部分 regex 语法。 -3. 样本不以表达式结构优化为目标，避免依赖公共前缀提取、分支树重排等 AST 优化。 -4. `TRY`/`CONDITION` 前缀化改写属于 IR 执行形态整理，不算表达式优化。 +因此清单以「一个 `TryGenerateXxx` 一次吞掉几条 IR」为轴，与 +`RegexAnalyzer.PrefixEmit` 里 `TryEmitXxx` 的调度链保持同构。 + +## 生成器统一约定 + +这些约定是所有层级的前提，先固定下来再写具体层： + +1. **返回值语义**：所有 `TryGenerateXxx` 返回**本次消耗的 IR 条数**，`-1` 表示窗口不匹配。 + 不得返回 pattern 长度、字符数等其它含义的整数。 +2. **调度链顺序**：调度链的 `if / else if` 严格按 L1 → L2 → L3 → L4 排列， + 层号即消耗量的递减顺序，高消耗形态优先尝试，落空后自然降级到单指令。 +3. **窗口约定**：扫描窗口为 `ir_sp[index..]`，长度不足时由各 `TryGenerate` 自行短路返回 `-1`。 + 窥探深度参照 IR 优化器的 8 条窗口。 +4. **前缀不参与消耗计数**：前缀装饰（L5）不单独消耗 IR，它包裹在 L1–L4 产出的代码外层。 +5. **作用域归属**：代码块的作用域由 `Prefix.WorkingRegion` 决定，不由表达式嵌套决定。 +6. **控制流形态**：region 一律生成 `do { ... } while(false)` + `break`， + 不使用 .NET generated regex 的 `goto` + 标签方案。 + +### 正确性核心与纯优化的分界 + +- **L2 + L4 + L5 是正确性核心**：region 作用域、单指令主体、前缀装饰三者齐备， + 任意表达式即可端到端生成出语义正确的代码。 +- **L1 与 L3 是纯优化**：它们只把 L2/L4 会生成的嵌套结构压平成更好的代码形状。 + 未实现时必须干净地返回 `-1`，让窗口自然降级，**不得影响正确性**。 + +这条分界决定了实现顺序：先把 L2/L4/L5 打通拿到可运行基线，再回头做 L1/L3。 ## 记录模板 ```text -Regex: +层级: -覆盖形态: +触发窗口: + +消耗 IR 数: + +覆盖样本: 期望代码形状: @@ -31,90 +58,158 @@ Optimized IR: IR -> Code 规则: +前缀交互: + 备注: ``` -## 第一批：基础线性匹配与转义 +## L0 方法级 -| Regex | 覆盖形态 | 手写重点 | -| --- | --- | --- | -| `abc` | 单个 `MATCH_SEQ` | 最小长度检查、序列匹配、活动指针推进 | -| `a[abc]d` | `MATCH_SEQ` + `MATCH_SET` + `MATCH_SEQ` | set 后端展开前的调用边界 | -| `[abc]` | 单个 `MATCH_SET` | 字符集单字符匹配 | -| `[.]` | set 中的字面点号 | 确认 `[.]` 和 `.` 语义分离 | -| `.` | 任意字符匹配 | `MATCH_SET "."` 的特殊后端语义 | -| `\.` | 转义字面点号 | 确认 `\.` 进入 seq 系 IR | -| `a\*b` | 转义操作符字面量 | tokenizer 已消解转义语义 | +一次消耗**整个 IR 列表 + `RegexInfo`**，只产出骨架，不产出任何匹配逻辑。 -## 第二批：计数 +| # | 触发 | 消耗 | 产出 | 状态 | +| --- | --- | --- | --- | --- | +| L0.1 | `TextRegexSource` 入口 | 全部 | `Match(ReadOnlySpan)` 签名与 `MatchResult` 返回 | [ ] | +| L0.2 | `RegionCount` | 元数据 | 每个 region 的 `start` / `end` / `success` 槽 | [ ] | +| L0.3 | `SingleLineResultCount` | 元数据 | `Observe_Line` 所需的行结果槽 | [ ] | +| L0.4 | `MinPredictLength` | 元数据 | 入口处最小长度断言（可延后） | [ ] | +| L0.5 | — | — | 活动指针、锚点指针、`SourceGenContext` 按需变量声明 | [ ] | -| Regex | 覆盖形态 | 手写重点 | -| --- | --- | --- | -| `a*` | `COUNT_SEQ`，`MIN:0 MAX:-1` | 贪婪计数、允许空匹配 | -| `a+` | `COUNT_SEQ`，`MIN:1 MAX:-1` | 至少一次的计数 | -| `a?` | `COUNT_SEQ`，`MIN:0 MAX:1` | 可选单元 | -| `a{3,5}` | 固定下限 + 可选上限 | 下限展开与剩余计数 | -| `[abc]*` | `COUNT_SET` | 字符集循环 | -| `.{11,13}` | 任意字符计数 | set 语义在后端展开 | +样本：任意，用 `abc` 验证骨架能编译即可。 -## 第三批：ALT 与前缀化 +关注点：region 槽是否必须全量分配、行结果槽能否按 `ObserveSource` 稀疏分配。 -| Regex | 覆盖形态 | 手写重点 | -| --- | --- | --- | -| `a|b` | 最小 ALT | region 缓冲、失败切换、成功短路 | -| `ab|cd|ef` | 多分支 ALT | 多个 case 的失败链 | -| `a|bc|def` | 不同长度 ALT | case 间活动指针恢复 | -| `(ab|[cd]|.)x` | 混合 case ALT | seq、set、任意字符分支后接公共后缀 | +## L1 Region 组合级（纯优化） -## 第四批：复杂计数块 +一次消耗**多个 region + 尾部控制指令**，是窗口消耗量最高的形态。 +这些组合在 `EmitPrefix` 里已经被识别过一次 +（`TryEmitAltChain` / `TryEmitTryGroupTail` / `TryEmitComplexIR`），生成器应与之一一对应。 -| Regex | 覆盖形态 | 手写重点 | -| --- | --- | --- | -| `(ab){2,3}` | `COUNT` complex body | complex body 调用域、计数退出 | -| `(123(4|5)67){3,4}114514` | complex count + ALT + 后缀 | `CommandCount` 重算、region 复用、后续线性匹配 | +除 L1.5–L1.7 外，本层全部是**代码形状优化**：不实现时窗口会降级到 L2 + L5， +生成出语义等价但嵌套更深的代码。 -## 第五批：FIND 与 `.*` +| # | 触发窗口 | 消耗 | 覆盖样本 | 性质 | 状态 | +| --- | --- | --- | --- | --- | --- | +| L1.1 | region 链，后继 region 带 `Prev_Fail` | Σ region | `a\|b`、`ab\|cd\|ef`、`a\|bc\|def` | 压平 | [ ] | +| L1.2 | 同上 + 公共后缀 | Σ region + 后缀 | `(ab\|[cd]\|.)x` | 压平 | [ ] | +| L1.3 | region + `POP FAIL(Prev_Fail)` + `PACK SUCCESS(Prev_Success)` | region + 2 | `(?abc)` | 压平 | [ ] | +| L1.4 | L1.3 且 region 内部是 ALT 链 | 嵌套 | `(?abcd\|a(bc)c)` | 压平 | [ ] | +| L1.5 | `COUNT_COMPLEX` + body | `CommandCount + 1` | `(ab){2,3}` | **必需** | [ ] | +| L1.6 | `COUNT_COMPLEX` + body(含 ALT) + 线性后缀 | 嵌套 + 后缀 | `(123(4\|5)67){3,4}114514` | **必需** | [ ] | +| L1.7 | `FIND_COMPLEX` + body | `CommandCount + 1` | `.*(ab\|cd)` | **必需** | [ ] | -| Regex | 覆盖形态 | 手写重点 | -| --- | --- | --- | -| `.*abc` | 最远/最近搜索前缀 | `FIND_SEQ` 或 `FIND_COMPLEX` 到后续锚点 | -| `abc.*` | 后缀任意吞吐 | 尾部 `.*` 的快速成功策略 | -| `[.]*` | 字面点号计数 | 不应被当成原装 `.*` 优化 | +期望代码形状：L1.1/L1.2 压平为 `if / else if / else` 链；L1.3/L1.4 在 region 成功分支写捕获区间； +L1.5–L1.7 为循环 / 搜索包裹 body，`CommandCount` 必须按前缀化后的实际条数重算。 -## 第六批：字符集语义 +L1.5–L1.7 标记为必需，是因为 `COUNT_COMPLEX` / `FIND_COMPLEX` 的 body 不是独立 region， +L2 接不住它们，没有替代降级路径。 -| Regex | 覆盖形态 | 手写重点 | -| --- | --- | --- | -| `[123abc]` | 离散字符集 | set 后端展开为离散匹配 | -| `[1-9]` | 连续字符集 | set 后端展开为范围匹配 | -| `\w` | 转义字符集 | escape-set 后端语义 | -| `\w+` | 转义字符集计数 | set 计数循环 | +关注点：ALT 各 case 之间活动指针如何恢复、`PACK` 取哪一个成功结果作为捕获区间。 -## 第七批：捕获组 +## L2 Region 级（正确性核心） -| Regex | 覆盖形态 | 手写重点 | -| --- | --- | --- | -| `(abc)` | 普通捕获组 | 默认序号 group | -| `(?abc)` | 简单命名组 | group 起止范围、`PACK` | -| `(?.*)234` | group 内 `.*` + 后续锚点 | `FIND_COMPLEX FURTHEST -1` 与后续 `MATCH_SEQ` | -| `(?.*)a` | group 内 `.*` + 单字符锚点 | 最远搜索回退到后缀匹配 | -| `(?abcd|a(bc)c)` | group 内 ALT | group success 依赖 region | -| `a*.[abc]*(?abcd|a(bc)c)(?.*)234` | 综合样本 | count、set、ALT、group、find、后缀匹配 | +一次消耗 **`CommandCount + 1`**，即一个 `TRY` 归约出的完整 region。 +决定作用域与失败复位策略，是所有嵌套结构的承载体。 -## 当前优先级 +| # | 触发窗口 | 消耗 | 覆盖样本 | 状态 | +| --- | --- | --- | --- | --- | +| L2.1 | `RegexTryIR` + 其 body | `CommandCount + 1` | `(abc)` | [ ] | +| L2.2 | 同上，body 内含嵌套 region | 递归 | `(a(bc))` | [ ] | +| L2.3 | 单指令 region（被折叠的 TRY） | 1 | `(a)` | [ ] | -第一阶段只要求源生成器能稳定处理： +期望代码形状： -1. `MATCH_SEQ` -2. `MATCH_SET` -3. `COUNT_SEQ` -4. `COUNT_SET` -5. 简单 `ALT` -6. `PACK` -7. `FIND_SEQ` -8. `FIND_COMPLEX COUNT:-1` +```csharp +__rN_begin = __cur; +__rN_ok = false; +do +{ + // body，POP 在此 break + __rN_ok = __ok; +} while (false); +if (!__rN_ok) { __cur = __rN_begin; } +__ok = __rN_ok; +``` -低优先级： +失败路径必须把活动指针复位到 region 入口。`POP` 直接编译为 `break`，不引入标签。 + +关注点：region 全部内联展开、嵌套 region 的指针复位链、`break` 只能跳出最近一层 `do`。 + +## L3 多指令定式（纯优化） + +一次消耗 **2–8 条**，滑动窗口内的固定组合。全部属于代码质量优化，非正确性必需。 +每条都必须能在窗口不匹配时干净地降级到 L4。 + +| # | 触发窗口 | 消耗 | 覆盖样本 | 状态 | +| --- | --- | --- | --- | --- | +| L3.1 | `FIND_COMPLEX FURTHEST COUNT:-1` + `PACK` + `MATCH_SEQ` | 3 | `(?.*)234` | [ ] | +| L3.2 | 同上，锚点为单字符 | 3 | `(?.*)a` | [ ] | +| L3.3 | `FIND_SEQ` + `MATCH_SEQ`，合并为一次 `IndexOf` | 2 | `.*abc` | [ ] | +| L3.4 | `MATCH_SEQ` + `COUNT_SEQ`（`a+`、`a{m,n}` 展开形态） | 2 | `a{3,5}`、`a+` | [ ] | +| L3.5 | 连续 `MATCH_SEQ` / `MATCH_SET`，合流为单个 `if` 与表达式 | 2–8 | `a[abc]d` | [ ] | +| L3.6 | 尾部 `COUNT_SET .*`，快速成功 | 1–2 | `abc.*` | [ ] | + +关注点：合并后失败诊断是否还能定位到原指令、合并是否破坏 `Observe_Line` 的行号引用。 + +## L4 单指令级 + +一次消耗 **1 条**。这是调度链的兜底层，必须对每种 `RegexIRCommand` 都有实现。 + +| # | 指令 | 覆盖样本 | 状态 | +| --- | --- | --- | --- | +| L4.1 | `MATCH_SEQ` | `abc`、`\.`、`a\*b` | [ ] | +| L4.2 | `MATCH_SET` 离散 | `[abc]`、`[123abc]`、`[.]` | [ ] | +| L4.3 | `MATCH_SET` 范围 | `[1-9]` | [ ] | +| L4.4 | `MATCH_SET` 转义类 | `\w` | [ ] | +| L4.5 | `MATCH_SET` 任意字符 | `.` | [ ] | +| L4.6 | `COUNT_SEQ` | `a*`、`a?` | [ ] | +| L4.7 | `COUNT_SET` | `[abc]*`、`\w+`、`.{11,13}`、`[.]*` | [ ] | +| L4.8 | `FIND_SEQ` | `.*abc` 降级路径 | [ ] | +| L4.9 | `FIND_SET` | — | [ ] | +| L4.10 | `POP` | ALT 尾 | [ ] | +| L4.11 | `PACK` | `(abc)` 默认序号组 | [ ] | + +关注点：`[.]` 与 `.` 必须走不同后端语义；`[.]*` 不得被当作 `.*` 优化。 + +## L5 前缀装饰（横切，消耗 0） + +前缀不消耗 IR，而是包裹在 L1–L4 产出的代码外层。 +**这一层不实现，L1–L4 的产出就无法拼装成正确的控制流。** + +| # | 前缀字段 | 取值 | 生成形态 | 状态 | +| --- | --- | --- | --- | --- | +| L5.1 | `ExeCondition` | `DEF` | 无条件顺序执行 | [ ] | +| L5.2 | `ExeCondition` | `Prev_Success` | 包裹 `if (前驱成功)` | [ ] | +| L5.3 | `ExeCondition` | `Prev_Fail` | 包裹 `else` / `if (!前驱成功)` | [ ] | +| L5.4 | `ExeCondition` | `Always` | 脱离条件链，始终执行 | [ ] | +| L5.5 | `ObserveMethod` | `NONE` | 读紧邻前驱的隐式结果 | [ ] | +| L5.6 | `ObserveMethod` | `Observe_Line` + `ObserveSource` | 读指定行结果槽 | [ ] | +| L5.7 | `ObserveMethod` | `Observe_Region` + `ObserveSource` | 读指定 region 结果槽 | [ ] | +| L5.8 | `PopCondition` | `Pop_Success` | 写回 region 成功并提前退出 | [ ] | +| L5.9 | `PopCondition` | `Pop_Fail` | 写回 region 失败并提前退出 | [ ] | +| L5.10 | `WorkingRegion` | 任意 | 决定代码落入哪个 region 作用域 | [ ] | + +关注点：`Observe_Line` 只读目标行结果，`Observe_Region` 只读目标 region 结果， +二者都不得退化成「读上一条」。行结果槽仅在被 `Observe_Line` 引用时才分配。 + +## 实现顺序 + +调度链按 L1 → L4 排列，但**实现**先拿正确性核心（L2/L4/L5），再回头填纯优化层（L1/L3）： + +1. **L0** — 生成的空骨架能通过编译，`TextRegexSourceGen.GenFile` 的两处 TODO 闭合。 +2. **L2.1** — `(abc)` 产出正确的 `do{}while(false)` 作用域与复位逻辑。 +3. **L5.1–L5.3、L5.5** — 顺序与成功/失败链能拼装，此时 L2 的 region 才真正可用。 +4. **L4 全部** — 兜底层补齐，任意线性表达式可端到端生成。 +5. **L5.6–L5.9** — 跨行 / 跨 region 观察与提前退出，`a|b` 与 `(?abc)` 自然跑通（嵌套形态）。 + **此时获得可运行基线，任何表达式都能生成正确代码。** +6. **L1.5、L1.7** — 复杂计数与复杂搜索，无降级路径，必须实现。 +7. **L1.1–L1.4** — 把 ALT 链与捕获组的嵌套形态压平。 +8. **L3 全部** — 代码质量优化，可持续追加。 + +第一阶段验收口径（沿用原优先级）：`MATCH_SEQ`、`MATCH_SET`、`COUNT_SEQ`、`COUNT_SET`、 +简单 `ALT`、`PACK`、`FIND_SEQ`、`FIND_COMPLEX COUNT:-1` 全部稳定。 + +低优先级，暂不纳入 checklist： 1. 逐 region 最短长度断言。 2. 复杂字符集完全内联。 @@ -132,3 +227,30 @@ IR -> Code 规则: 5. `Observe_Region` 是否只读取目标 region 结果。 6. `PACK` 使用哪一个成功结果作为捕获区间。 7. 最小长度检查是否值得在当前样本中生成。 +8. 本形态消耗几条 IR，落空时降级到哪一层。 +9. 前缀的执行条件与观察源是否被正确外提，而不是硬编码进指令主体。 + +## 已手写样本索引 + +`test/SingleFile/regex.cs` 中已有的样本与层级对应： + +| 函数 | 表达式 | 对应层级 | +| --- | --- | --- | +| `Match_ABC` | `abc` | L4.1 | +| `Match_A_charset_ABC_C` | `a[abc]d` | L3.5 / L4.1 / L4.2 | +| `Match_charset_ABC` | `[abc]` | L4.2 | +| `Match_Dot` | `.` | L4.5 | +| `Match_A_3_5` | `a{3,5}` | L3.4 | +| `Match_AB_2_3` | `(ab){2,3}` | L1.5 | +| `MatchDiffCase` | `(ab\|[cd]\|.)x` | L1.2 | +| `MatchCountAnyGroup` | `(?.*)x` | L3.1 | + +待补：L2.1 `(abc)`、L1.1 `a\|b`、L1.3 `(?abc)`、L5 各前缀组合的最小样本。 +其中 L5 的样本最关键：需要分别写出 `Prev_Success` / `Prev_Fail` 包裹、 +`Observe_Line` / `Observe_Region` 读取、`Pop_Success` / `Pop_Fail` 写回三组最小代码形状。 + +## 综合验收样本 + +`a*.[abc]*(?abcd|a(bc)c)(?.*)234` + +覆盖 L1.1、L1.3、L1.4、L3.1、L4.6、L4.7，作为全链路收口用例，最后再手写。 diff --git a/src/StgSharp.RegularAnalysis/Text/TextRegexSourceGen.GenClass.MultiIR.cs b/src/StgSharp.RegularAnalysis/Text/TextRegexSourceGen.GenClass.MultiIR.cs index 09e58abb..fd2a9bce 100644 --- a/src/StgSharp.RegularAnalysis/Text/TextRegexSourceGen.GenClass.MultiIR.cs +++ b/src/StgSharp.RegularAnalysis/Text/TextRegexSourceGen.GenClass.MultiIR.cs @@ -8,101 +8,162 @@ using StgSharp.RegularAnalysis.Abstraction; using System; using System.Collections.Generic; -using System.Linq; -using System.Reflection; -using System.Text; -using System.Text.RegularExpressions; -using System.Threading.Tasks; +using System.Threading; namespace StgSharp.RegularAnalysis.Text { internal partial class TextRegexSourceGen { + /* + * ======================================================================================= + * L1, several regions plus a tail + * ======================================================================================= + * + * Highest consumption in the dispatch chain, so these are offered the window first. Each + * shape here was already recognised once during prefix emission, so a generator should + * mirror the matching RegexAnalyzer.PrefixEmit recogniser rather than re-derive the shape. + * + * L1.2 ALT chain with a shared suffix + * window L1.1 followed by linear instructions in the parent region + * sample (ab|[cd]|.)x + * nature flattening, may return -1 + * + * L1.3 group tail + * window region, POP FAIL with Prev_Fail, PACK SUCCESS with Prev_Success + * mirrors TryEmitTryGroupTail + * shape region block, capture written on the success branch + * sample (?abc) + * nature flattening, may return -1 + * + * L1.4 group tail wrapping an ALT chain + * sample (?abcd|a(bc)c) + * nature flattening, may return -1 + * + * L1.5 COUNT_COMPLEX plus body + * window RegexCountComplexIR, then CommandCount entries + * mirrors TryEmitComplexIR + * shape loop wrapping the generated body + * sample (ab){2,3} + * nature REQUIRED, no fallback exists + * + * L1.6 COUNT_COMPLEX whose body contains an ALT chain, plus a linear suffix + * sample (123(4|5)67){3,4}114514 + * nature REQUIRED, no fallback exists + * + * L1.7 FIND_COMPLEX plus body + * window RegexFindComplexIR, then CommandCount entries + * shape search loop wrapping the generated body + * sample .*(ab|cd) + * nature REQUIRED, no fallback exists + * + * L1.5 to L1.7 are required because a complex body is not a region of its own. L2 cannot + * pick it up, so there is no degradation path if these are missing. + * + * CommandCount must be read from the prefixed IR. Prefix emission rebuilds these counts, + * so the value carried by the pre-prefix IR no longer applies. + * + * Open questions to settle while implementing: + * - how the cursor is restored between ALT cases + * - which successful outcome a PACK uses as its capture span + * - whether a region body ever deserves its own generated method instead of inlining + * + * ======================================================================================= + * L2, one region + * ======================================================================================= + * + * Consumes CommandCount + 1, the whole block a TRY lowers to. Together with L4 and L5 this + * is the correctness core: once these three exist, every expression can be generated, just + * with deeper nesting than L1 and L3 would produce. + * + * L2.1 RegexTryIR plus its body sample (abc) + * L2.2 the same, body containing a nested region sample (a(bc)) + * L2.3 a single instruction region folded by prefix emission sample (a) + * + * Target shape, names owned by SourceGenContext: + * + * __rN_begin = __cur; + * __rN_ok = false; + * do + * { + * // body, a POP breaks out here + * __rN_ok = __ok; + * } while (false); + * if (!__rN_ok) { __cur = __rN_begin; } + * __ok = __rN_ok; + * + * Points the implementation has to settle: + * - the failure path restores the cursor to the region entry + * - break only leaves the nearest enclosing do block, so a nested region must not rely + * on it to exit an outer one + * - the whole match body is itself region zero, which is what makes a top level POP + * legal + * + * ======================================================================================= + * L3, fixed multi instruction windows + * ======================================================================================= + * + * Consumes two to eight entries. Pure code quality work, never required for correctness. + * Every one of these must return -1 cleanly when the window does not match, so the entry + * falls through to L4. + * + * L3.1 FIND_COMPLEX FURTHEST COUNT:-1, PACK, MATCH_SEQ 3 (?.*)234 + * L3.2 the same with a single element anchor 3 (?.*)a + * L3.3 FIND_SEQ, MATCH_SEQ, merged into one search 2 .*abc + * L3.4 MATCH_SEQ, COUNT_SEQ, the expanded a+ shape 2 a{3,5}, a+ + * L3.5 a run of MATCH_SEQ and MATCH_SET, merged into one condition 2 to 8 a[abc]d + * L3.6 trailing COUNT_SET over a dot, fast success 1 to 2 abc.* + * + * Two hazards to check for each of these: + * - merging must not break the line indices an Observe_Line prefix refers to + * - a merged window still has to leave __ok describing the whole window + */ - private static int GenerateNamedCountAny( - ReadOnlySpan region_ir, + private static int TryGenerateAltChain( + int outer_level, + ReadOnlySpan ir_span, SourceGenContext sc, - in SequenceEmitter main_emitter, - in List> sub_emitter_list + in SequenceEmitter code, + in List> local_func ) { - if (region_ir[0] is not RegexCountSetIR { IsDotStar:true} || - region_ir[1] is not RegexPackIR pack) { + /* + * L1.1 ALT chain + * window region, then further regions carrying Prev_Fail + * mirrors TryEmitAltChain + * shape if / else if / else + * sample a|b, ab|cd|ef, a|bc|def + * nature flattening, may return -1 + */ + + /* + * [004] 0001, DEF , DEF , NONE, 0000, MATCH_SEQ "abc" + * [005] 0002, DEF , Sccs, NONE, 0000, MATCH_SEQ "d" <- an alt chain begins here + * [006] 0002, Fail, Sccs, LINE, 0005, MATCH_SEQ "c" <- an alt chain ends here + * [007] 0001, Fail, DEF , LINE, 0006, POP FAIL + */ + + const RegexIRPrefixMask simple_alt = RegexIRPrefixMask.Prev_Fail | RegexIRPrefixMask.Pop_Success; + + int cur_level = ir_span[0].Prefix.WorkingRegion; + if (cur_level == outer_level) { return -1; } - if (region_ir[2] is RegexMatchSeqIR) { } - _ = main_emitter.AppendLine($@"do").AppendLine($@"{{"); - _ = main_emitter.AppendLine($@"}}").AppendLine($@"while(true);"); - string group_idx = sc.MakeNewNamedGroup(pack.GroupName); - return 0; - } - - private static void GenerateRegion( - ReadOnlySpan region_ir, - SourceGenContext sc, - in SequenceEmitter main_emitter, - in List> sub_emitter_list - ) - { - int current_region = region_ir[0].Prefix.WorkingRegion; - ReadOnlySpan ir_sp = region_ir; - ReadOnlySpan scan_win; - int index = 0; - do + int i = 1; + RegexIR ir; + for (; i < ir_span.Length; i++) { - int result; - int count = 0; - scan_win = ir_sp[index..]; - if ((result = - TryGenerateFindSeq(scan_win, sc, in main_emitter, - in sub_emitter_list)) > 0) + ir = ir_span[i]; + if ((ir.Prefix.Mask & simple_alt) == simple_alt) { - count += result; - } else - if (TryGenerateMatchSeq(scan_win, main_emitter, sc) > 0) + // simple match + } else if (ir is RegexTryIR _try && (ir.Prefix.Mask & simple_alt) == simple_alt) { - count += 1; - } else if (TryGenerateMatchSet(scan_win, main_emitter, sc) > 0) - { - count++; - } else if (TryGenerateFindSeq(scan_win, main_emitter, sc) > 0) - { - count++; - } else { } - index += count; - } while (index < ir_sp.Length); - } - - private static int TryGenerateFindSeq( - ReadOnlySpan scan_win, - SourceGenContext sc, - in SequenceEmitter main_emitter, - in List> sub_emitter_list - ) - { - if (scan_win[0] is not RegexFindSeqIR find) { - return -1; + // match a try region + ReadOnlySpan body_span = ir_span.Slice(i + 1, i + _try.CommandCount); + i += _try.CommandCount; + } } - const string cur_ptr = "cur_ptr", - begin_ptr = "begin_ptr"; - _ = main_emitter.AppendLine() - .AppendLine(@$"//Find sequence:{FormatStringLiteral(find.Pattern)} at any pos") - .AppendLine(@"do") - .AppendLine(@"{") - .AppendLine(@$"int {cur_ptr}, {begin_ptr} = {current_ptr};") - .AppendLine($@"{ROS_char} rest = "); - - string finder = find.IsFirst ? "FirstIndexOf" : "LastIndexOf"; - - if (scan_win[1] is RegexMatchSeqIR seq) - { - string seq_pat = FormatStringLiteral($"{find.Pattern}{seq.Pattern}"); - _ = main_emitter.AppendLine(@$"//Match sequence:{seq_pat} at current pos") - .AppendLine($@"if( )"); - } - - _ = main_emitter.AppendLine("}").AppendLine(string.Empty); - return 1; + return i; } } diff --git a/src/StgSharp.RegularAnalysis/Text/TextRegexSourceGen.GenClass.Prefix.cs b/src/StgSharp.RegularAnalysis/Text/TextRegexSourceGen.GenClass.Prefix.cs new file mode 100644 index 00000000..cddc2e2f --- /dev/null +++ b/src/StgSharp.RegularAnalysis/Text/TextRegexSourceGen.GenClass.Prefix.cs @@ -0,0 +1,66 @@ +// ----------------------------------------------------------------------------- +// file="TextRegexSourceGen.GenClass.Prefix" +// Project: StgSharp +// Copyright (c) Nitload. +// SPDX-License-Identifier: MIT +// ----------------------------------------------------------------------------- + +namespace StgSharp.RegularAnalysis.Text +{ + internal partial class TextRegexSourceGen + { + + /* + * ======================================================================================= + * L5, prefix decoration + * ======================================================================================= + * + * A prefix consumes no IR of its own. It wraps whatever an L1 to L4 body generator + * produced: + * + * [prefix open ] if () { + * [ body ] ... sets __ok, advances __cur + * [ line slot ] __lK_ok = __ok; + * [ pop ] if (__ok) { __rN_ok = true; break; } + * [prefix close] } + * + * Keeping this outside the body generators is what lets L2 and L4 stay small. They only + * describe what an instruction does, never when it runs or who reads the outcome. Without + * this layer the other layers cannot be assembled into correct control flow at all, so L5 + * is part of the correctness core rather than an optimisation. + * + * Checklist, RegexIRPrefixMask on the left: + * + * L5.1 ExeCondition none run unconditionally + * L5.2 ExeCondition Prev_Success wrap in if (observed) + * L5.3 ExeCondition Prev_Fail wrap in if (!observed) + * L5.4 ExeCondition Always run unconditionally, deliberately detached from + * the surrounding condition chain + * L5.5 ObserveMethod none read the result of the preceding instruction + * L5.6 ObserveMethod Observe_Line read the slot named by ObserveSource + * L5.7 ObserveMethod Observe_Region read the slot named by ObserveSource + * L5.8 PopCondition Pop_Success write the region result true, then leave it + * L5.9 PopCondition Pop_Fail write the region result false, then leave it + * L5.10 WorkingRegion selects which region scope the code lands in + * + * Rules the implementation has to hold: + * + * - Observe_Line and Observe_Region must read the slot ObserveSource names. Neither may + * degrade into reading the previous instruction, which is what an absent observe + * method means and nothing else. + * - Line slots are allocated only for lines some prefix actually observes. That demand + * has to be collected in a pass before body generation, because an observer can be + * emitted before the line it reads, for instance when a nested region observes a line + * of its parent. + * - A pop lowers to break, which leaves the nearest enclosing do while(false) block. + * That block is the region the instruction works in, because every region is emitted + * as one and the whole match body is wrapped as region zero. + * - Open and close must be paired. A body generator that declines the window must be + * detected before the opening brace is emitted, so bodies are generated into a + * detached emitter first. + * + * The open call should report how many braces it opened, so the close call knows how many + * to emit rather than recomputing the condition. + */ + } +} diff --git a/src/StgSharp.RegularAnalysis/Text/TextRegexSourceGen.GenClass.SingleIR.cs b/src/StgSharp.RegularAnalysis/Text/TextRegexSourceGen.GenClass.SingleIR.cs new file mode 100644 index 00000000..bbf1eb1b --- /dev/null +++ b/src/StgSharp.RegularAnalysis/Text/TextRegexSourceGen.GenClass.SingleIR.cs @@ -0,0 +1,69 @@ +// ----------------------------------------------------------------------------- +// file="TextRegexSourceGen.GenClass.SingleIR" +// Project: StgSharp +// Copyright (c) Nitload. +// SPDX-License-Identifier: MIT +// ----------------------------------------------------------------------------- + +namespace StgSharp.RegularAnalysis.Text +{ + internal partial class TextRegexSourceGen + { + + /* + * ======================================================================================= + * L4, one instruction + * ======================================================================================= + * + * The fallback of the dispatch chain. Every RegexIRCommand needs a generator here, because + * this is the only layer guaranteed to be offered every window. L1 and L3 may decline, L4 + * may not. + * + * Each generator consumes exactly one entry and describes only what the instruction does. + * When it runs, which result it reads and whether it leaves the region early are all added + * by L5, so nothing here should inspect RegexIR.Prefix beyond WorkingRegion. + * + * Contract for a generated body, names owned by SourceGenContext: + * + * - assign the outcome to __ok + * - advance __cur only on success + * - never index __cur past the end of the input + * - declare scratch variables inside a block of their own, so a repeated shape does not + * collide with itself + * + * A body must be generated into a detached emitter and only spliced in once it is known to + * be claimed. Emitting a prefix for an instruction that turns out to have no generator + * would open a brace nobody closes. + * + * Checklist: + * + * L4.1 MATCH_SEQ abc, \., a\*b + * L4.2 MATCH_SET, discrete [abc], [123abc], [.] + * L4.3 MATCH_SET, range [1-9] + * L4.4 MATCH_SET, escape class \w + * L4.5 MATCH_SET, any element . + * L4.6 COUNT_SEQ a*, a? + * L4.7 COUNT_SET [abc]*, \w+, .{11,13}, [.]* + * L4.8 FIND_SEQ degradation path of .*abc + * L4.9 FIND_SET - + * L4.10 POP ALT tail + * L4.11 PACK (abc), ordinal group + * + * Semantics worth pinning down before writing these: + * + * - [.] and . are different sets. The former is a literal dot, the latter is any + * element, and [.]* must not be folded into the dot star optimisation. + * - \w has to keep full .NET semantics. TextRegex.IsCharWord is inherited by the + * generated class and already carries them, so the generated code can call it + * directly instead of approximating with an ASCII test. + * - This backend does not backtrack. A lazy repetition therefore consumes exactly its + * lower bound and leaves the rest to the following anchor. Lazy forms needing more + * are expected to have been rewritten into a find by the IR optimizer, so confirm + * that before adding backtracking here. + * - An epsilon sequence and an empty repetition unit both reach these generators. + * Decide whether they are vacuous successes or an optimizer bug worth reporting. + * - PACK decides nothing, so it should leave __ok untouched. Whether it runs at all is + * already expressed by its execution condition. + */ + } +} diff --git a/src/StgSharp.RegularAnalysis/Text/TextRegexSourceGen.GenClass.cs b/src/StgSharp.RegularAnalysis/Text/TextRegexSourceGen.GenClass.cs index 26b82c4a..0bb308d4 100644 --- a/src/StgSharp.RegularAnalysis/Text/TextRegexSourceGen.GenClass.cs +++ b/src/StgSharp.RegularAnalysis/Text/TextRegexSourceGen.GenClass.cs @@ -6,32 +6,104 @@ // ----------------------------------------------------------------------------- using Microsoft.CodeAnalysis.CSharp; -using Microsoft.CodeAnalysis.CSharp.Syntax; using StgSharp.RegularAnalysis.Abstraction; using System; using System.Collections.Generic; using System.Globalization; -using System.Linq; -using System.Reflection; using System.Runtime.CompilerServices; using System.Runtime.InteropServices; -using System.Text; -using System.Text.RegularExpressions; -using System.Threading.Tasks; namespace StgSharp.RegularAnalysis.Text { internal partial class TextRegexSourceGen { + /* + * --------------------------------------------------------------------------------------- + * NGRA text regex source generation backend + * --------------------------------------------------------------------------------------- + * + * The backend turns a fully prefixed IR stream into the body of one Match method. It is + * organised in layers, ordered by how many IR entries a single generator consumes: + * + * L0 whole IR stream method skeleton, slot declarations, entry guards + * L1 several regions ALT chain, group tail, complex count, complex find + * L2 one region the block a TRY lowers to + * L3 two to eight IR fixed windows worth collapsing into better code + * L4 one IR the fallback, one generator per RegexIRCommand + * L5 zero IR prefix decoration wrapped around L1 to L4 output + * + * File layout: + * + * TextRegexSourceGen.GenClass.cs this file, L0 and shared helpers + * TextRegexSourceGen.GenClass.MultiIR.cs L1, L2 and L3 + * TextRegexSourceGen.GenClass.SingleIR.cs L4 + * TextRegexSourceGen.GenClass.Prefix.cs L5 + * TextRegexSourceGen.GenContext.cs variable naming and slot allocation + * + * Contract every generator honours: + * + * 1. A TryGenerateXxx returns how many IR entries it consumed, or -1 when the window + * does not match its shape. It returns an entry count, never a character count or a + * pattern length. + * 2. The dispatch chain offers a window to L1, then L2, then L3, then L4. Higher + * consumption wins, so a shape spanning a whole region plus a tail is never split by + * a smaller generator. + * 3. L2, L4 and L5 together are sufficient for correctness. L1 and L3 only flatten what + * those would already emit, so an unimplemented one must return -1 cleanly rather + * than emit a partial result. + * 4. A prefix consumes no IR. L1 to L4 describe what an instruction does, L5 decides + * when it runs and who reads the outcome. + * 5. Scope follows Prefix.WorkingRegion, not expression nesting. + * + * Control flow shape: + * + * A region lowers to do { ... } while(false), and POP lowers to break. The goto plus + * label scheme used by the .NET generated regex backend is deliberately not adopted. + * + * Generated variable names are owned by SourceGenContext. No generator invents one. + * + * The per shape checklist this backend is rebuilt against lives in + * future/ngra-regex-sourcegen-goldens.md, and the handwritten target code for each shape + * lives in test/SingleFile/regex.cs. + */ private const string current_ptr = "cur_ptr_global"; private const string offset = "offset"; private const string orig_text = "text"; + #region L0 method level + + /// + /// Generates one Match method from a fully prefixed IR stream. + /// + /// + /// Not implemented yet. Returns an empty emitter so the generator pipeline stays runnable + /// while the backend is being rebuilt. + /// internal static SequenceEmitter GenerateSource( TextRegexSource source ) { + // + // L0, method skeleton + // + // TODO L0.1 Emit the Match signature from ROS_char and SourceGenContext.InputSpan. + // TODO L0.2 Emit region anchor and region result slots, count from RegexInfo.RegionCount. + // TODO L0.3 Emit line result slots, only for lines an Observe_Line prefix refers to. + // TODO L0.4 Emit the whole pattern guard from RegexInfo.MinPredictLength. + // TODO L0.5 Emit the cursor, the step result and any scratch declaration the context collected. + // + + /* + * Slot demand is only known once the body exists, so the body has to be generated + * into a detached emitter first and spliced in after the declaration block. + * + * The dispatch chain that walks the IR stream belongs here too. It offers each window + * to L1, L2, L3 then L4, and adds the returned consumption count to the cursor. An + * unclaimed window must produce a hard build break rather than silently emit nothing, + * because code that compiles but matches incorrectly is far harder to diagnose. + */ + SequenceEmitter se = new(); List> sub_match_list = []; SourceGenContext sc = new(); @@ -47,7 +119,8 @@ namespace StgSharp.RegularAnalysis.Text while (true) { - GenerateRegion(region, sc, match_e, sub_match_list); + // GenerateRegion(region, sc, match_e, sub_match_list); + _ = TryGenerateAltChain(0, region, sc, match_e, sub_match_list); if (end >= ir_sp.Length) { break; @@ -72,7 +145,7 @@ namespace StgSharp.RegularAnalysis.Text .AppendLine($@"int {offset} = 0;") .AppendLine(); - _ = sc.GenVarDefine(se); + // _ = sc.GenVarDefine(se); _ = se.Append(match_e); @@ -87,6 +160,10 @@ namespace StgSharp.RegularAnalysis.Text return se; } + #endregion + + #region shared helpers + [MethodImpl(MethodImplOptions.AggressiveInlining)] private static int FindRegionEnd( ReadOnlySpan ir_sp, @@ -111,151 +188,11 @@ namespace StgSharp.RegularAnalysis.Text return SyntaxFactory.Literal(str).Text; } - #region sub generator - - private static int TryGenerateFindSeq( - ReadOnlySpan scan_win, - SequenceEmitter se, - SourceGenContext ng + private static string FormatCharLiteral( + char value ) { - if (scan_win[0] is not RegexFindSeqIR seq) { - return -1; - } - string pattern = seq.Pattern; - int length = pattern.Length; - string index_caller = seq.IsFirst ? "IndexOf" : "LastIndexOf"; - - _ = se.AppendLine() - .AppendLine($@"if(({offset} = [{current_ptr}..].{index_caller}({pattern})) != -1)") - .AppendLine($@"{{") - .AppendLine($@"{current_ptr} += {offset} + {length};") - .AppendLine($@"}}"); - return length; - } - - private static int TryGenerateMatchSeq( - ReadOnlySpan scan_win, - SequenceEmitter se, - SourceGenContext ng - ) - { - if (scan_win[0] is not RegexFindSeqIR seq) { - return -1; - } - string pattern = FormatStringLiteral(seq.Pattern); - int length = pattern.Length; - - _ = se.AppendLine() - .AppendLine($@"if([{current_ptr}..({current_ptr} + {length})] == {pattern})") - .AppendLine($@"{{") - .AppendLine($@"{current_ptr} += {offset} + {length};") - .AppendLine($@"}}"); - return length; - } - - private static int TryGenerateMatchSet( - ReadOnlySpan scan_win, - SequenceEmitter se, - SourceGenContext ng - ) - { - if (scan_win[0] is not RegexMatchSeqIR seq) { - return -1; - } - string pattern = seq.Pattern; - RegexCharSet[] set_pattern = FigureCharSetType(pattern, out bool accept); - - string var_char = ng.CurChar; - _ = se.AppendLine($@"// match charset of {pattern}") - .AppendLine($@"char {var_char} = {orig_text}[{current_ptr}];") - .AppendLine(); - - _ = se.AppendLine("if("); - RegexCharSet p = set_pattern[0]; - string _case = GenerateCase(p); - string _case_begin = accept ? string.Empty : "!("; - _ = se.AppendLine($@"if ({_case_begin}{_case}||"); - - for (int i = 1; i < set_pattern.Length - 1; i++) - { - p = set_pattern[i]; - _case = GenerateCase(p); - if (string.IsNullOrEmpty(_case)) - { - continue; - } - _ = se.AppendLine($@" {_case} ||"); - } - - p = set_pattern[^1]; - _case = GenerateCase(p); - string _case_end = accept ? string.Empty : ")"; - _ = se.AppendLine($@"{_case}{_case_end})") - .AppendLine("{") - .AppendLine($@"{current_ptr} += 1;") - .AppendLine("}"); - - return 1; - - - string GenerateCase( - RegexCharSet c - ) - { - switch (c.Type) - { - // Handle single character - case CharSetType.Single: - string set = c.Value; - if (set.Length == 1) - { - return $@"{var_char} == '{c.Value}'"; - } else if (set.Length < 5) - { - StringBuilder sb = new(); - _ = sb.Append('('); - foreach (char item in set) { - _ = sb.Append(CultureInfo.InvariantCulture, $@"item == '{item}' or "); - } - _ = sb.Append(')'); - return sb.ToString(); - } else - { - return $@"""{c.Value}"".AsSpan().Contains({var_char})"; - } - - // Handle range of characters - case CharSetType.Range: - return $@"({var_char} >= '{c.Value[0]}' && {var_char} <= '{c.Value[1]}')"; - - // Handle set of characters - case CharSetType.Set: - if (c.Accept) - { - return c.Value switch - { - "s" => $@"char.IsWhiteSpace({var_char})", - "w" => $@"char.IsLetterOrDigit({var_char}) || {var_char} == '_'", - "d" => $@"char.IsDigit({var_char})", - _ => throw new InvalidCastException(), - }; - } else - { - return c.Value switch - { - "s" => $@"!char.IsWhiteSpace({var_char})", - "w" => $@"!char.IsLetterOrDigit({var_char}) && {var_char} != '_'", - "d" => $@"!char.IsDigit({var_char})", - _ => throw new InvalidCastException(), - }; - } - case CharSetType.None: - return string.Empty; - default: - return string.Empty; - } - } + return SyntaxFactory.Literal(value).Text; } #endregion diff --git a/src/StgSharp.RegularAnalysis/Text/TextRegexSourceGen.GenContext.cs b/src/StgSharp.RegularAnalysis/Text/TextRegexSourceGen.GenContext.cs index 0b8a89e0..41a7db7a 100644 --- a/src/StgSharp.RegularAnalysis/Text/TextRegexSourceGen.GenContext.cs +++ b/src/StgSharp.RegularAnalysis/Text/TextRegexSourceGen.GenContext.cs @@ -19,86 +19,177 @@ namespace StgSharp.RegularAnalysis.Text #region var naming - private class SourceGenContext + /// + /// Tracks variable naming and declaration demand while generating one match method. + /// + /// + /// Method level state is limited to the active cursor and the result of the most recent + /// instruction. Everything else is either a region slot, a line slot, a capture slot, or + /// a block scoped scratch variable. Scratch names are never reused, so a nested block can + /// always declare its own without colliding with an enclosing one. + /// + private sealed class SourceGenContext { - public bool Need_char_cur { get; set; } + /// + /// Active pointer. Indexes the next unconsumed element of the input. + /// + public const string Cursor = "__cur"; - public bool Need_remain_cur { get; set; } + /// + /// Name of the match method parameter holding the input sequence. + /// + public const string InputSpan = "text"; - public bool Need_find_result { get; set; } + /// + /// Result of the instruction emitted immediately before the current one. + /// + public const string StepResult = "__ok"; - private const string __char_cur = "__char_cur"; - private const string __remain_cur = "__remain_cur"; - private const string __find_result = "__find_result"; + private readonly Dictionary _groupIndex = []; + private readonly SortedSet _lineSlots = []; + private readonly HashSet _observedLines = []; + private readonly SortedSet _regionSlots = []; - private int _min_region_begin_index; + private int _anonymousGroup; + private int _scratchIndex; - public string MakeNewRegionBegin() + /// + /// Records that some prefix observes the result of the instruction at line + /// . + /// + public void MarkLineObserved( + int line + ) { - string name = $"__region_begin_{_min_region_begin_index}"; - _min_region_begin_index++; - return name; + _ = _observedLines.Add(line); } - public string CurChar + /// + /// Indicates whether the instruction at line has to persist + /// its result into a line slot. + /// + public bool IsLineObserved( + int line + ) { - get - { - Need_char_cur = true; - return __char_cur; - } + return _observedLines.Contains(line); } - #region namig group + /// + /// Allocates a fresh block scoped scratch variable name. + /// + /// + /// Short role marker, for example n for a counter or ch for a + /// character. + /// + public string NewScratch( + string tag + ) + { + return $"__{tag}{_scratchIndex++}"; + } - private int _min_named_group; - private Dictionary _name_idx_map = []; + /// + /// Name of the slot holding the cursor value captured when region + /// was entered. + /// + public string RegionAnchor( + int region + ) + { + _ = _regionSlots.Add(region); + return $"__r{region}_begin"; + } - public string MakeNewNamedGroup( + /// + /// Name of the slot holding the match result of region . + /// + public string RegionResult( + int region + ) + { + _ = _regionSlots.Add(region); + return $"__r{region}_ok"; + } + + /// + /// Name of the slot holding the match result of the instruction at line + /// . Only lines actually referenced by an + /// prefix get a slot. + /// + public string LineResult( + int line + ) + { + _ = _lineSlots.Add(line); + return $"__l{line}_ok"; + } + + public string GroupBegin( string name ) { - if (_name_idx_map.TryGetValue(name, out int idx)) { - return $"__named_group_{idx}"; - } - idx = _min_named_group; - _name_idx_map[name] = idx; - _min_named_group++; - return $"__named_group_{idx}"; + return $"__g{ResolveGroup(name)}_begin"; } - #endregion - - public string FindResult - { - get - { - Need_find_result = true; - return __find_result; - } - } - - public string CurRemain - { - get - { - Need_remain_cur = true; - return __remain_cur; - } - } - - public SequenceEmitter GenVarDefine( - SequenceEmitter se + public string GroupEnd( + string name ) { - if (se is null) { + return $"__g{ResolveGroup(name)}_end"; + } + + /// + /// Emits declarations for every slot requested during body generation. + /// + /// + /// Must be called after the method body has been generated, but the produced lines + /// have to be placed before it. + /// + public SequenceEmitter EmitDeclarations( + SequenceEmitter emitter + ) + { + if (emitter is null) { return null!; } - _ = se.AppendLine($@"int {__char_cur}", Need_char_cur) - .AppendLine($@"int {__remain_cur}", Need_remain_cur); - return se; + _ = emitter.AppendLine($@"int {Cursor} = 0;") + .AppendLine($@"bool {StepResult} = true;"); + + foreach (int region in _regionSlots) + { + _ = emitter.AppendLine($@"int {RegionAnchor(region)} = 0;") + .AppendLine($@"bool {RegionResult(region)} = false;"); + } + + foreach (int line in _lineSlots) { + _ = emitter.AppendLine($@"bool {LineResult(line)} = false;"); + } + + foreach (int group in _groupIndex.Values) + { + _ = emitter.AppendLine($@"int __g{group}_begin = -1;") + .AppendLine($@"int __g{group}_end = -1;"); + } + + return emitter; + } + + private int ResolveGroup( + string name + ) + { + string key = string.IsNullOrEmpty(name) ? + $"#{_anonymousGroup++}" : + name; + if (_groupIndex.TryGetValue(key, out int index)) { + return index; + } + index = _groupIndex.Count; + _groupIndex[key] = index; + return index; } } diff --git a/test/SingleFile/regex.cs b/test/SingleFile/regex.cs index 04992e06..f0bdc8b5 100644 --- a/test/SingleFile/regex.cs +++ b/test/SingleFile/regex.cs @@ -164,5 +164,21 @@ static void MatchCountAnyGroup(ReadOnlySpan seq) int ptr = 0; - + int group_start = ptr; + int furthest = -1; + do + { + if((furthest = seq.LastIndexOf("123"))>0) + { + ptr += furthest + 3; + break; + } + } while(true); + // grouping + if(furthest >= 0) + { + ptr = furthest; + int group_end = ptr; + // group "name" matched from group_start to group_end + } } \ No newline at end of file diff --git a/test/StgSharpDebug/TestRegexAnalyse.cs b/test/StgSharpDebug/TestRegexAnalyse.cs index e01bbcc7..927b5cfc 100644 --- a/test/StgSharpDebug/TestRegexAnalyse.cs +++ b/test/StgSharpDebug/TestRegexAnalyse.cs @@ -21,8 +21,8 @@ namespace StgSharp { private static string[] regexSource = [ - @"(?.*)234", - /* + @"12(1234|(012.*34)|9125)34", + /**/ @"http|https|ftp|ftps", @"abc[0-9]x|abc[0-9]y|abd[0-9]x|abd[0-9]y|xyz", @"get_name|get_value|get_type|set_name|set_value|set_type",