youmuliu/stgsharp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

remove old generatoer, use new generator

Browse Source
This commit is contained in:
Youmu Liu
2026-07-28 18:11:59 +08:00
parent ac1e7c68f8
commit 3db2bfd9af
8 changed files with 731 additions and 369 deletions
Download Patch File Download Diff File Expand all files Collapse all files
future/ngra-regex-sourcegen-goldens.md
+202 -80
View File
@@ -1,29 +1,56 @@
# NGRA Regex 源生成手写样本
# NGRA Regex 源生成 Checklist
这个文件用于记录源生成器实现前需要手写的正则匹配样本。
本文件按**一次扫描消耗的前缀 IR 数量**分层，记录源生成器需要实现的形态。
消耗量从高到低排列，从整个 region 的识别一直降到单条 `MATCH` 指令。
流程：
## 为什么按 IR 消耗量分层
1. 选一个表达式。
2. 手写理想匹配代码。
3. 生成优化后 IR。
4. 对照手写代码和 IR 段。
5. 标注每段 IR 应该生成的代码形状。
6. 抽取生成器规则。
早期版本按正则语法分批（基础匹配 / 计数 / ALT / 捕获组……），这描述的是**表达式形态**。
但 `EmitPrefix` 之后，生成器面对的不再是表达式，而是**带前缀的指令窗口**：
选样原则：
- `a|b` 与 `(?<a>abcd|a(bc)c)` 在语法上分属两批，在前缀 IR 里却共用同一个
「region 链 + `Prev_Fail` 切换」骨架；
- `abc` 与 `a[abc]d` 在语法上同属一批，消耗的 IR 条数却不同；
- `WorkingRegion` / `ExeCondition` / `ObserveMethod` / `PopCondition`
是**横切在每条指令上**的，任何按语法组织的清单都无法表达它们。
1. 表达式必须是典型正则写法。
2. 全部样本合起来应覆盖计划支持的大部分 regex 语法。
3. 样本不以表达式结构优化为目标，避免依赖公共前缀提取、分支树重排等 AST 优化。
4. `TRY`/`CONDITION` 前缀化改写属于 IR 执行形态整理，不算表达式优化。
因此清单以「一个 `TryGenerateXxx` 一次吞掉几条 IR」为轴，与
`RegexAnalyzer.PrefixEmit` 里 `TryEmitXxx` 的调度链保持同构。
## 生成器统一约定
这些约定是所有层级的前提，先固定下来再写具体层：
1. **返回值语义**：所有 `TryGenerateXxx` 返回**本次消耗的 IR 条数**，`-1` 表示窗口不匹配。
不得返回 pattern 长度、字符数等其它含义的整数。
2. **调度链顺序**：调度链的 `if / else if` 严格按 L1 → L2 → L3 → L4 排列，
层号即消耗量的递减顺序，高消耗形态优先尝试，落空后自然降级到单指令。
3. **窗口约定**：扫描窗口为 `ir_sp[index..]`，长度不足时由各 `TryGenerate` 自行短路返回 `-1`。
窥探深度参照 IR 优化器的 8 条窗口。
4. **前缀不参与消耗计数**：前缀装饰（L5）不单独消耗 IR，它包裹在 L1–L4 产出的代码外层。
5. **作用域归属**：代码块的作用域由 `Prefix.WorkingRegion` 决定，不由表达式嵌套决定。
6. **控制流形态**：region 一律生成 `do { ... } while(false)` + `break`，
不使用 .NET generated regex 的 `goto` + 标签方案。
### 正确性核心与纯优化的分界
- **L2 + L4 + L5 是正确性核心**：region 作用域、单指令主体、前缀装饰三者齐备，
任意表达式即可端到端生成出语义正确的代码。
- **L1 与 L3 是纯优化**：它们只把 L2/L4 会生成的嵌套结构压平成更好的代码形状。
未实现时必须干净地返回 `-1`，让窗口自然降级，**不得影响正确性**。
这条分界决定了实现顺序：先把 L2/L4/L5 打通拿到可运行基线，再回头做 L1/L3。
## 记录模板
```text
Regex:
层级:
覆盖形态:
触发窗口:
消耗 IR 数:
覆盖样本:
期望代码形状:
@@ -31,90 +58,158 @@ Optimized IR:
IR -> Code 规则:
前缀交互:
备注:
```
## 第一批：基础线性匹配与转义
## L0 方法级
| Regex | 覆盖形态 | 手写重点 |
| --- | --- | --- |
| `abc` | 单个 `MATCH_SEQ` | 最小长度检查、序列匹配、活动指针推进 |
| `a[abc]d` | `MATCH_SEQ` + `MATCH_SET` + `MATCH_SEQ` | set 后端展开前的调用边界 |
| `[abc]` | 单个 `MATCH_SET` | 字符集单字符匹配 |
| `[.]` | set 中的字面点号 | 确认 `[.]` 和 `.` 语义分离 |
| `.` | 任意字符匹配 | `MATCH_SET "."` 的特殊后端语义 |
| `\.` | 转义字面点号 | 确认 `\.` 进入 seq 系 IR |
| `a\*b` | 转义操作符字面量 | tokenizer 已消解转义语义 |
一次消耗**整个 IR 列表 + `RegexInfo`**，只产出骨架，不产出任何匹配逻辑。
## 第二批：计数
| # | 触发 | 消耗 | 产出 | 状态 |
| --- | --- | --- | --- | --- |
| L0.1 | `TextRegexSource` 入口 | 全部 | `Match(ReadOnlySpan<char>)` 签名与 `MatchResult` 返回 | [ ] |
| L0.2 | `RegionCount` | 元数据 | 每个 region 的 `start` / `end` / `success` 槽 | [ ] |
| L0.3 | `SingleLineResultCount` | 元数据 | `Observe_Line` 所需的行结果槽 | [ ] |
| L0.4 | `MinPredictLength` | 元数据 | 入口处最小长度断言（可延后） | [ ] |
| L0.5 | — | — | 活动指针、锚点指针、`SourceGenContext` 按需变量声明 | [ ] |
| Regex | 覆盖形态 | 手写重点 |
| --- | --- | --- |
| `a*` | `COUNT_SEQ`，`MIN:0 MAX:-1` | 贪婪计数、允许空匹配 |
| `a+` | `COUNT_SEQ`，`MIN:1 MAX:-1` | 至少一次的计数 |
| `a?` | `COUNT_SEQ`，`MIN:0 MAX:1` | 可选单元 |
| `a{3,5}` | 固定下限 + 可选上限 | 下限展开与剩余计数 |
| `[abc]*` | `COUNT_SET` | 字符集循环 |
| `.{11,13}` | 任意字符计数 | set 语义在后端展开 |
样本：任意，用 `abc` 验证骨架能编译即可。
## 第三批：ALT 与前缀化
关注点：region 槽是否必须全量分配、行结果槽能否按 `ObserveSource` 稀疏分配。
| Regex | 覆盖形态 | 手写重点 |
| --- | --- | --- |
| `a|b` | 最小 ALT | region 缓冲、失败切换、成功短路 |
| `ab|cd|ef` | 多分支 ALT | 多个 case 的失败链 |
| `a|bc|def` | 不同长度 ALT | case 间活动指针恢复 |
| `(ab|[cd]|.)x` | 混合 case ALT | seq、set、任意字符分支后接公共后缀 |
## L1 Region 组合级（纯优化）
## 第四批：复杂计数块
一次消耗**多个 region + 尾部控制指令**，是窗口消耗量最高的形态。
这些组合在 `EmitPrefix` 里已经被识别过一次
（`TryEmitAltChain` / `TryEmitTryGroupTail` / `TryEmitComplexIR`），生成器应与之一一对应。
| Regex | 覆盖形态 | 手写重点 |
| --- | --- | --- |
| `(ab){2,3}` | `COUNT` complex body | complex body 调用域、计数退出 |
| `(123(4|5)67){3,4}114514` | complex count + ALT + 后缀 | `CommandCount` 重算、region 复用、后续线性匹配 |
除 L1.5–L1.7 外，本层全部是**代码形状优化**：不实现时窗口会降级到 L2 + L5，
生成出语义等价但嵌套更深的代码。
## 第五批：FIND 与 `.*`
| # | 触发窗口 | 消耗 | 覆盖样本 | 性质 | 状态 |
| --- | --- | --- | --- | --- | --- |
| L1.1 | region 链，后继 region 带 `Prev_Fail` | Σ region | `a\|b`、`ab\|cd\|ef`、`a\|bc\|def` | 压平 | [ ] |
| L1.2 | 同上 + 公共后缀 | Σ region + 后缀 | `(ab\|[cd]\|.)x` | 压平 | [ ] |
| L1.3 | region + `POP FAIL(Prev_Fail)` + `PACK SUCCESS(Prev_Success)` | region + 2 | `(?<name>abc)` | 压平 | [ ] |
| L1.4 | L1.3 且 region 内部是 ALT 链 | 嵌套 | `(?<a>abcd\|a(bc)c)` | 压平 | [ ] |
| L1.5 | `COUNT_COMPLEX` + body | `CommandCount + 1` | `(ab){2,3}` | **必需** | [ ] |
| L1.6 | `COUNT_COMPLEX` + body(含 ALT) + 线性后缀 | 嵌套 + 后缀 | `(123(4\|5)67){3,4}114514` | **必需** | [ ] |
| L1.7 | `FIND_COMPLEX` + body | `CommandCount + 1` | `.*(ab\|cd)` | **必需** | [ ] |
| Regex | 覆盖形态 | 手写重点 |
| --- | --- | --- |
| `.*abc` | 最远/最近搜索前缀 | `FIND_SEQ` 或 `FIND_COMPLEX` 到后续锚点 |
| `abc.*` | 后缀任意吞吐 | 尾部 `.*` 的快速成功策略 |
| `[.]*` | 字面点号计数 | 不应被当成原装 `.*` 优化 |
期望代码形状：L1.1/L1.2 压平为 `if / else if / else` 链；L1.3/L1.4 在 region 成功分支写捕获区间；
L1.5–L1.7 为循环 / 搜索包裹 body，`CommandCount` 必须按前缀化后的实际条数重算。
## 第六批：字符集语义
L1.5–L1.7 标记为必需，是因为 `COUNT_COMPLEX` / `FIND_COMPLEX` 的 body 不是独立 region，
L2 接不住它们，没有替代降级路径。
| Regex | 覆盖形态 | 手写重点 |
| --- | --- | --- |
| `[123abc]` | 离散字符集 | set 后端展开为离散匹配 |
| `[1-9]` | 连续字符集 | set 后端展开为范围匹配 |
| `\w` | 转义字符集 | escape-set 后端语义 |
| `\w+` | 转义字符集计数 | set 计数循环 |
关注点：ALT 各 case 之间活动指针如何恢复、`PACK` 取哪一个成功结果作为捕获区间。
## 第七批：捕获组
## L2 Region 级（正确性核心）
| Regex | 覆盖形态 | 手写重点 |
| --- | --- | --- |
| `(abc)` | 普通捕获组 | 默认序号 group |
| `(?<name>abc)` | 简单命名组 | group 起止范围、`PACK` |
| `(?<name>.*)234` | group 内 `.*` + 后续锚点 | `FIND_COMPLEX FURTHEST -1` 与后续 `MATCH_SEQ` |
| `(?<name>.*)a` | group 内 `.*` + 单字符锚点 | 最远搜索回退到后缀匹配 |
| `(?<a>abcd|a(bc)c)` | group 内 ALT | group success 依赖 region |
| `a*.[abc]*(?<a>abcd|a(bc)c)(?<name>.*)234` | 综合样本 | count、set、ALT、group、find、后缀匹配 |
一次消耗 **`CommandCount + 1`**，即一个 `TRY` 归约出的完整 region。
决定作用域与失败复位策略，是所有嵌套结构的承载体。
## 当前优先级
| # | 触发窗口 | 消耗 | 覆盖样本 | 状态 |
| --- | --- | --- | --- | --- |
| L2.1 | `RegexTryIR` + 其 body | `CommandCount + 1` | `(abc)` | [ ] |
| L2.2 | 同上，body 内含嵌套 region | 递归 | `(a(bc))` | [ ] |
| L2.3 | 单指令 region（被折叠的 TRY） | 1 | `(a)` | [ ] |
第一阶段只要求源生成器能稳定处理：
期望代码形状：
1. `MATCH_SEQ`
2. `MATCH_SET`
3. `COUNT_SEQ`
4. `COUNT_SET`
5. 简单 `ALT`
6. `PACK`
7. `FIND_SEQ`
8. `FIND_COMPLEX COUNT:-1`
```csharp
__rN_begin = __cur;
__rN_ok = false;
do
{
// body，POP 在此 break
__rN_ok = __ok;
} while (false);
if (!__rN_ok) { __cur = __rN_begin; }
__ok = __rN_ok;
```
低优先级：
失败路径必须把活动指针复位到 region 入口。`POP` 直接编译为 `break`，不引入标签。
关注点：region 全部内联展开、嵌套 region 的指针复位链、`break` 只能跳出最近一层 `do`。
## L3 多指令定式（纯优化）
一次消耗 **2–8 条**，滑动窗口内的固定组合。全部属于代码质量优化，非正确性必需。
每条都必须能在窗口不匹配时干净地降级到 L4。
| # | 触发窗口 | 消耗 | 覆盖样本 | 状态 |
| --- | --- | --- | --- | --- |
| L3.1 | `FIND_COMPLEX FURTHEST COUNT:-1` + `PACK` + `MATCH_SEQ` | 3 | `(?<name>.*)234` | [ ] |
| L3.2 | 同上，锚点为单字符 | 3 | `(?<name>.*)a` | [ ] |
| L3.3 | `FIND_SEQ` + `MATCH_SEQ`，合并为一次 `IndexOf` | 2 | `.*abc` | [ ] |
| L3.4 | `MATCH_SEQ` + `COUNT_SEQ`（`a+`、`a{m,n}` 展开形态） | 2 | `a{3,5}`、`a+` | [ ] |
| L3.5 | 连续 `MATCH_SEQ` / `MATCH_SET`，合流为单个 `if` 与表达式 | 2–8 | `a[abc]d` | [ ] |
| L3.6 | 尾部 `COUNT_SET .*`，快速成功 | 1–2 | `abc.*` | [ ] |
关注点：合并后失败诊断是否还能定位到原指令、合并是否破坏 `Observe_Line` 的行号引用。
## L4 单指令级
一次消耗 **1 条**。这是调度链的兜底层，必须对每种 `RegexIRCommand` 都有实现。
| # | 指令 | 覆盖样本 | 状态 |
| --- | --- | --- | --- |
| L4.1 | `MATCH_SEQ` | `abc`、`\.`、`a\*b` | [ ] |
| L4.2 | `MATCH_SET` 离散 | `[abc]`、`[123abc]`、`[.]` | [ ] |
| L4.3 | `MATCH_SET` 范围 | `[1-9]` | [ ] |
| L4.4 | `MATCH_SET` 转义类 | `\w` | [ ] |
| L4.5 | `MATCH_SET` 任意字符 | `.` | [ ] |
| L4.6 | `COUNT_SEQ` | `a*`、`a?` | [ ] |
| L4.7 | `COUNT_SET` | `[abc]*`、`\w+`、`.{11,13}`、`[.]*` | [ ] |
| L4.8 | `FIND_SEQ` | `.*abc` 降级路径 | [ ] |
| L4.9 | `FIND_SET` | — | [ ] |
| L4.10 | `POP` | ALT 尾 | [ ] |
| L4.11 | `PACK` | `(abc)` 默认序号组 | [ ] |
关注点：`[.]` 与 `.` 必须走不同后端语义；`[.]*` 不得被当作 `.*` 优化。
## L5 前缀装饰（横切，消耗 0）
前缀不消耗 IR，而是包裹在 L1–L4 产出的代码外层。
**这一层不实现，L1–L4 的产出就无法拼装成正确的控制流。**
| # | 前缀字段 | 取值 | 生成形态 | 状态 |
| --- | --- | --- | --- | --- |
| L5.1 | `ExeCondition` | `DEF` | 无条件顺序执行 | [ ] |
| L5.2 | `ExeCondition` | `Prev_Success` | 包裹 `if (前驱成功)` | [ ] |
| L5.3 | `ExeCondition` | `Prev_Fail` | 包裹 `else` / `if (!前驱成功)` | [ ] |
| L5.4 | `ExeCondition` | `Always` | 脱离条件链，始终执行 | [ ] |
| L5.5 | `ObserveMethod` | `NONE` | 读紧邻前驱的隐式结果 | [ ] |
| L5.6 | `ObserveMethod` | `Observe_Line` + `ObserveSource` | 读指定行结果槽 | [ ] |
| L5.7 | `ObserveMethod` | `Observe_Region` + `ObserveSource` | 读指定 region 结果槽 | [ ] |
| L5.8 | `PopCondition` | `Pop_Success` | 写回 region 成功并提前退出 | [ ] |
| L5.9 | `PopCondition` | `Pop_Fail` | 写回 region 失败并提前退出 | [ ] |
| L5.10 | `WorkingRegion` | 任意 | 决定代码落入哪个 region 作用域 | [ ] |
关注点：`Observe_Line` 只读目标行结果，`Observe_Region` 只读目标 region 结果，
二者都不得退化成「读上一条」。行结果槽仅在被 `Observe_Line` 引用时才分配。
## 实现顺序
调度链按 L1 → L4 排列，但**实现**先拿正确性核心（L2/L4/L5），再回头填纯优化层（L1/L3）：
1. **L0** — 生成的空骨架能通过编译，`TextRegexSourceGen.GenFile` 的两处 TODO 闭合。
2. **L2.1** — `(abc)` 产出正确的 `do{}while(false)` 作用域与复位逻辑。
3. **L5.1–L5.3、L5.5** — 顺序与成功/失败链能拼装，此时 L2 的 region 才真正可用。
4. **L4 全部** — 兜底层补齐，任意线性表达式可端到端生成。
5. **L5.6–L5.9** — 跨行 / 跨 region 观察与提前退出，`a|b` 与 `(?<name>abc)` 自然跑通（嵌套形态）。
**此时获得可运行基线，任何表达式都能生成正确代码。**
6. **L1.5、L1.7** — 复杂计数与复杂搜索，无降级路径，必须实现。
7. **L1.1–L1.4** — 把 ALT 链与捕获组的嵌套形态压平。
8. **L3 全部** — 代码质量优化，可持续追加。
第一阶段验收口径（沿用原优先级）：`MATCH_SEQ`、`MATCH_SET`、`COUNT_SEQ`、`COUNT_SET`、
简单 `ALT`、`PACK`、`FIND_SEQ`、`FIND_COMPLEX COUNT:-1` 全部稳定。
低优先级，暂不纳入 checklist：
1. 逐 region 最短长度断言。
2. 复杂字符集完全内联。
@@ -132,3 +227,30 @@ IR -> Code 规则:
5. `Observe_Region` 是否只读取目标 region 结果。
6. `PACK` 使用哪一个成功结果作为捕获区间。
7. 最小长度检查是否值得在当前样本中生成。
8. 本形态消耗几条 IR，落空时降级到哪一层。
9. 前缀的执行条件与观察源是否被正确外提，而不是硬编码进指令主体。
## 已手写样本索引
`test/SingleFile/regex.cs` 中已有的样本与层级对应：
| 函数 | 表达式 | 对应层级 |
| --- | --- | --- |
| `Match_ABC` | `abc` | L4.1 |
| `Match_A_charset_ABC_C` | `a[abc]d` | L3.5 / L4.1 / L4.2 |
| `Match_charset_ABC` | `[abc]` | L4.2 |
| `Match_Dot` | `.` | L4.5 |
| `Match_A_3_5` | `a{3,5}` | L3.4 |
| `Match_AB_2_3` | `(ab){2,3}` | L1.5 |
| `MatchDiffCase` | `(ab\|[cd]\|.)x` | L1.2 |
| `MatchCountAnyGroup` | `(?<name>.*)x` | L3.1 |
待补：L2.1 `(abc)`、L1.1 `a\|b`、L1.3 `(?<name>abc)`、L5 各前缀组合的最小样本。
其中 L5 的样本最关键：需要分别写出 `Prev_Success` / `Prev_Fail` 包裹、
`Observe_Line` / `Observe_Region` 读取、`Pop_Success` / `Pop_Fail` 写回三组最小代码形状。
## 综合验收样本
`a*.[abc]*(?<a>abcd|a(bc)c)(?<name>.*)234`
覆盖 L1.1、L1.3、L1.4、L3.1、L4.6、L4.7，作为全链路收口用例，最后再手写。
src/StgSharp.RegularAnalysis/Text/TextRegexSourceGen.GenClass.MultiIR.cs
+141 -80
View File
@@ -8,101 +8,162 @@
using StgSharp.RegularAnalysis.Abstraction;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using System.Text;
using System.Text.RegularExpressions;
using System.Threading.Tasks;
using System.Threading;
namespace StgSharp.RegularAnalysis.Text
{
internal partial class TextRegexSourceGen
{
/*
* =======================================================================================
* L1, several regions plus a tail
* =======================================================================================
*
* Highest consumption in the dispatch chain, so these are offered the window first. Each
* shape here was already recognised once during prefix emission, so a generator should
* mirror the matching RegexAnalyzer.PrefixEmit recogniser rather than re-derive the shape.
*
* L1.2 ALT chain with a shared suffix
* window L1.1 followed by linear instructions in the parent region
* sample (ab|[cd]|.)x
* nature flattening, may return -1
*
* L1.3 group tail
* window region, POP FAIL with Prev_Fail, PACK SUCCESS with Prev_Success
* mirrors TryEmitTryGroupTail
* shape region block, capture written on the success branch
* sample (?<name>abc)
* nature flattening, may return -1
*
* L1.4 group tail wrapping an ALT chain
* sample (?<a>abcd|a(bc)c)
* nature flattening, may return -1
*
* L1.5 COUNT_COMPLEX plus body
* window RegexCountComplexIR, then CommandCount entries
* mirrors TryEmitComplexIR
* shape loop wrapping the generated body
* sample (ab){2,3}
* nature REQUIRED, no fallback exists
*
* L1.6 COUNT_COMPLEX whose body contains an ALT chain, plus a linear suffix
* sample (123(4|5)67){3,4}114514
* nature REQUIRED, no fallback exists
*
* L1.7 FIND_COMPLEX plus body
* window RegexFindComplexIR, then CommandCount entries
* shape search loop wrapping the generated body
* sample .*(ab|cd)
* nature REQUIRED, no fallback exists
*
* L1.5 to L1.7 are required because a complex body is not a region of its own. L2 cannot
* pick it up, so there is no degradation path if these are missing.
*
* CommandCount must be read from the prefixed IR. Prefix emission rebuilds these counts,
* so the value carried by the pre-prefix IR no longer applies.
*
* Open questions to settle while implementing:
* - how the cursor is restored between ALT cases
* - which successful outcome a PACK uses as its capture span
* - whether a region body ever deserves its own generated method instead of inlining
*
* =======================================================================================
* L2, one region
* =======================================================================================
*
* Consumes CommandCount + 1, the whole block a TRY lowers to. Together with L4 and L5 this
* is the correctness core: once these three exist, every expression can be generated, just
* with deeper nesting than L1 and L3 would produce.
*
* L2.1 RegexTryIR plus its body sample (abc)
* L2.2 the same, body containing a nested region sample (a(bc))
* L2.3 a single instruction region folded by prefix emission sample (a)
*
* Target shape, names owned by SourceGenContext:
*
* __rN_begin = __cur;
* __rN_ok = false;
* do
* {
* // body, a POP breaks out here
* __rN_ok = __ok;
* } while (false);
* if (!__rN_ok) { __cur = __rN_begin; }
* __ok = __rN_ok;
*
* Points the implementation has to settle:
* - the failure path restores the cursor to the region entry
* - break only leaves the nearest enclosing do block, so a nested region must not rely
* on it to exit an outer one
* - the whole match body is itself region zero, which is what makes a top level POP
* legal
*
* =======================================================================================
* L3, fixed multi instruction windows
* =======================================================================================
*
* Consumes two to eight entries. Pure code quality work, never required for correctness.
* Every one of these must return -1 cleanly when the window does not match, so the entry
* falls through to L4.
*
* L3.1 FIND_COMPLEX FURTHEST COUNT:-1, PACK, MATCH_SEQ 3 (?<name>.*)234
* L3.2 the same with a single element anchor 3 (?<name>.*)a
* L3.3 FIND_SEQ, MATCH_SEQ, merged into one search 2 .*abc
* L3.4 MATCH_SEQ, COUNT_SEQ, the expanded a+ shape 2 a{3,5}, a+
* L3.5 a run of MATCH_SEQ and MATCH_SET, merged into one condition 2 to 8 a[abc]d
* L3.6 trailing COUNT_SET over a dot, fast success 1 to 2 abc.*
*
* Two hazards to check for each of these:
* - merging must not break the line indices an Observe_Line prefix refers to
* - a merged window still has to leave __ok describing the whole window
*/
private static int GenerateNamedCountAny(
ReadOnlySpan<RegexIR> region_ir,
private static int TryGenerateAltChain(
int outer_level,
ReadOnlySpan<RegexIR> ir_span,
SourceGenContext sc,
in SequenceEmitter<string> main_emitter,
in List<SequenceEmitter<string>> sub_emitter_list
in SequenceEmitter<string> code,
in List<SequenceEmitter<string>> local_func
)
{
if (region_ir[0] is not RegexCountSetIR { IsDotStar:true} ||
region_ir[1] is not RegexPackIR pack) {
/*
* L1.1 ALT chain
* window region, then further regions carrying Prev_Fail
* mirrors TryEmitAltChain
* shape if / else if / else
* sample a|b, ab|cd|ef, a|bc|def
* nature flattening, may return -1
*/
/*
* [004] 0001, DEF , DEF , NONE, 0000, MATCH_SEQ "abc"
* [005] 0002, DEF , Sccs, NONE, 0000, MATCH_SEQ "d" <- an alt chain begins here
* [006] 0002, Fail, Sccs, LINE, 0005, MATCH_SEQ "c" <- an alt chain ends here
* [007] 0001, Fail, DEF , LINE, 0006, POP FAIL
*/
const RegexIRPrefixMask simple_alt = RegexIRPrefixMask.Prev_Fail | RegexIRPrefixMask.Pop_Success;
int cur_level = ir_span[0].Prefix.WorkingRegion;
if (cur_level == outer_level) {
return -1;
}
if (region_ir[2] is RegexMatchSeqIR) { }
_ = main_emitter.AppendLine($@"do").AppendLine($@"{{");
_ = main_emitter.AppendLine($@"}}").AppendLine($@"while(true);");
string group_idx = sc.MakeNewNamedGroup(pack.GroupName);
return 0;
}
private static void GenerateRegion(
ReadOnlySpan<RegexIR> region_ir,
SourceGenContext sc,
in SequenceEmitter<string> main_emitter,
in List<SequenceEmitter<string>> sub_emitter_list
)
{
int current_region = region_ir[0].Prefix.WorkingRegion;
ReadOnlySpan<RegexIR> ir_sp = region_ir;
ReadOnlySpan<RegexIR> scan_win;
int index = 0;
do
int i = 1;
RegexIR ir;
for (; i < ir_span.Length; i++)
{
int result;
int count = 0;
scan_win = ir_sp[index..];
if ((result =
TryGenerateFindSeq(scan_win, sc, in main_emitter,
in sub_emitter_list)) > 0)
ir = ir_span[i];
if ((ir.Prefix.Mask & simple_alt) == simple_alt)
{
count += result;
} else
if (TryGenerateMatchSeq(scan_win, main_emitter, sc) > 0)
// simple match
} else if (ir is RegexTryIR _try && (ir.Prefix.Mask & simple_alt) == simple_alt)
{
count += 1;
} else if (TryGenerateMatchSet(scan_win, main_emitter, sc) > 0)
{
count++;
} else if (TryGenerateFindSeq(scan_win, main_emitter, sc) > 0)
{
count++;
} else { }
index += count;
} while (index < ir_sp.Length);
}
private static int TryGenerateFindSeq(
ReadOnlySpan<RegexIR> scan_win,
SourceGenContext sc,
in SequenceEmitter<string> main_emitter,
in List<SequenceEmitter<string>> sub_emitter_list
)
{
if (scan_win[0] is not RegexFindSeqIR find) {
return -1;
// match a try region
ReadOnlySpan<RegexIR> body_span = ir_span.Slice(i + 1, i + _try.CommandCount);
i += _try.CommandCount;
}
}
const string cur_ptr = "cur_ptr",
begin_ptr = "begin_ptr";
_ = main_emitter.AppendLine()
.AppendLine(@$"//Find sequence:{FormatStringLiteral(find.Pattern)} at any pos")
.AppendLine(@"do")
.AppendLine(@"{")
.AppendLine(@$"int {cur_ptr}, {begin_ptr} = {current_ptr};")
.AppendLine($@"{ROS_char} rest = ");
string finder = find.IsFirst ? "FirstIndexOf" : "LastIndexOf";
if (scan_win[1] is RegexMatchSeqIR seq)
{
string seq_pat = FormatStringLiteral($"{find.Pattern}{seq.Pattern}");
_ = main_emitter.AppendLine(@$"//Match sequence:{seq_pat} at current pos")
.AppendLine($@"if( )");
}
_ = main_emitter.AppendLine("}").AppendLine(string.Empty);
return 1;
return i;
}
}
src/StgSharp.RegularAnalysis/Text/TextRegexSourceGen.GenClass.Prefix.cs
+66
View File
@@ -0,0 +1,66 @@
// -----------------------------------------------------------------------------
// file="TextRegexSourceGen.GenClass.Prefix"
// Project: StgSharp
// Copyright (c) Nitload.
// SPDX-License-Identifier: MIT
// -----------------------------------------------------------------------------
namespace StgSharp.RegularAnalysis.Text
{
internal partial class TextRegexSourceGen
{
/*
* =======================================================================================
* L5, prefix decoration
* =======================================================================================
*
* A prefix consumes no IR of its own. It wraps whatever an L1 to L4 body generator
* produced:
*
* [prefix open ] if (<observed result>) {
* [ body ] ... sets __ok, advances __cur
* [ line slot ] __lK_ok = __ok;
* [ pop ] if (__ok) { __rN_ok = true; break; }
* [prefix close] }
*
* Keeping this outside the body generators is what lets L2 and L4 stay small. They only
* describe what an instruction does, never when it runs or who reads the outcome. Without
* this layer the other layers cannot be assembled into correct control flow at all, so L5
* is part of the correctness core rather than an optimisation.
*
* Checklist, RegexIRPrefixMask on the left:
*
* L5.1 ExeCondition none run unconditionally
* L5.2 ExeCondition Prev_Success wrap in if (observed)
* L5.3 ExeCondition Prev_Fail wrap in if (!observed)
* L5.4 ExeCondition Always run unconditionally, deliberately detached from
* the surrounding condition chain
* L5.5 ObserveMethod none read the result of the preceding instruction
* L5.6 ObserveMethod Observe_Line read the slot named by ObserveSource
* L5.7 ObserveMethod Observe_Region read the slot named by ObserveSource
* L5.8 PopCondition Pop_Success write the region result true, then leave it
* L5.9 PopCondition Pop_Fail write the region result false, then leave it
* L5.10 WorkingRegion selects which region scope the code lands in
*
* Rules the implementation has to hold:
*
* - Observe_Line and Observe_Region must read the slot ObserveSource names. Neither may
* degrade into reading the previous instruction, which is what an absent observe
* method means and nothing else.
* - Line slots are allocated only for lines some prefix actually observes. That demand
* has to be collected in a pass before body generation, because an observer can be
* emitted before the line it reads, for instance when a nested region observes a line
* of its parent.
* - A pop lowers to break, which leaves the nearest enclosing do while(false) block.
* That block is the region the instruction works in, because every region is emitted
* as one and the whole match body is wrapped as region zero.
* - Open and close must be paired. A body generator that declines the window must be
* detected before the opening brace is emitted, so bodies are generated into a
* detached emitter first.
*
* The open call should report how many braces it opened, so the close call knows how many
* to emit rather than recomputing the condition.
*/
}
}
src/StgSharp.RegularAnalysis/Text/TextRegexSourceGen.GenClass.SingleIR.cs
+69
View File
@@ -0,0 +1,69 @@
// -----------------------------------------------------------------------------
// file="TextRegexSourceGen.GenClass.SingleIR"
// Project: StgSharp
// Copyright (c) Nitload.
// SPDX-License-Identifier: MIT
// -----------------------------------------------------------------------------
namespace StgSharp.RegularAnalysis.Text
{
internal partial class TextRegexSourceGen
{
/*
* =======================================================================================
* L4, one instruction
* =======================================================================================
*
* The fallback of the dispatch chain. Every RegexIRCommand needs a generator here, because
* this is the only layer guaranteed to be offered every window. L1 and L3 may decline, L4
* may not.
*
* Each generator consumes exactly one entry and describes only what the instruction does.
* When it runs, which result it reads and whether it leaves the region early are all added
* by L5, so nothing here should inspect RegexIR.Prefix beyond WorkingRegion.
*
* Contract for a generated body, names owned by SourceGenContext:
*
* - assign the outcome to __ok
* - advance __cur only on success
* - never index __cur past the end of the input
* - declare scratch variables inside a block of their own, so a repeated shape does not
* collide with itself
*
* A body must be generated into a detached emitter and only spliced in once it is known to
* be claimed. Emitting a prefix for an instruction that turns out to have no generator
* would open a brace nobody closes.
*
* Checklist:
*
* L4.1 MATCH_SEQ abc, \., a\*b
* L4.2 MATCH_SET, discrete [abc], [123abc], [.]
* L4.3 MATCH_SET, range [1-9]
* L4.4 MATCH_SET, escape class \w
* L4.5 MATCH_SET, any element .
* L4.6 COUNT_SEQ a*, a?
* L4.7 COUNT_SET [abc]*, \w+, .{11,13}, [.]*
* L4.8 FIND_SEQ degradation path of .*abc
* L4.9 FIND_SET -
* L4.10 POP ALT tail
* L4.11 PACK (abc), ordinal group
*
* Semantics worth pinning down before writing these:
*
* - [.] and . are different sets. The former is a literal dot, the latter is any
* element, and [.]* must not be folded into the dot star optimisation.
* - \w has to keep full .NET semantics. TextRegex.IsCharWord is inherited by the
* generated class and already carries them, so the generated code can call it
* directly instead of approximating with an ASCII test.
* - This backend does not backtrack. A lazy repetition therefore consumes exactly its
* lower bound and leaves the rest to the following anchor. Lazy forms needing more
* are expected to have been rewritten into a find by the IR optimizer, so confirm
* that before adding backtracking here.
* - An epsilon sequence and an empty repetition unit both reach these generators.
* Decide whether they are vacuous successes or an optimizer bug worth reporting.
* - PACK decides nothing, so it should leave __ok untouched. Whether it runs at all is
* already expressed by its execution condition.
*/
}
}
src/StgSharp.RegularAnalysis/Text/TextRegexSourceGen.GenClass.cs
+88 -151
View File
@@ -6,32 +6,104 @@
// -----------------------------------------------------------------------------
using Microsoft.CodeAnalysis.CSharp;
using Microsoft.CodeAnalysis.CSharp.Syntax;
using StgSharp.RegularAnalysis.Abstraction;
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using System.Reflection;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
using System.Text;
using System.Text.RegularExpressions;
using System.Threading.Tasks;
namespace StgSharp.RegularAnalysis.Text
{
internal partial class TextRegexSourceGen
{
/*
* ---------------------------------------------------------------------------------------
* NGRA text regex source generation backend
* ---------------------------------------------------------------------------------------
*
* The backend turns a fully prefixed IR stream into the body of one Match method. It is
* organised in layers, ordered by how many IR entries a single generator consumes:
*
* L0 whole IR stream method skeleton, slot declarations, entry guards
* L1 several regions ALT chain, group tail, complex count, complex find
* L2 one region the block a TRY lowers to
* L3 two to eight IR fixed windows worth collapsing into better code
* L4 one IR the fallback, one generator per RegexIRCommand
* L5 zero IR prefix decoration wrapped around L1 to L4 output
*
* File layout:
*
* TextRegexSourceGen.GenClass.cs this file, L0 and shared helpers
* TextRegexSourceGen.GenClass.MultiIR.cs L1, L2 and L3
* TextRegexSourceGen.GenClass.SingleIR.cs L4
* TextRegexSourceGen.GenClass.Prefix.cs L5
* TextRegexSourceGen.GenContext.cs variable naming and slot allocation
*
* Contract every generator honours:
*
* 1. A TryGenerateXxx returns how many IR entries it consumed, or -1 when the window
* does not match its shape. It returns an entry count, never a character count or a
* pattern length.
* 2. The dispatch chain offers a window to L1, then L2, then L3, then L4. Higher
* consumption wins, so a shape spanning a whole region plus a tail is never split by
* a smaller generator.
* 3. L2, L4 and L5 together are sufficient for correctness. L1 and L3 only flatten what
* those would already emit, so an unimplemented one must return -1 cleanly rather
* than emit a partial result.
* 4. A prefix consumes no IR. L1 to L4 describe what an instruction does, L5 decides
* when it runs and who reads the outcome.
* 5. Scope follows Prefix.WorkingRegion, not expression nesting.
*
* Control flow shape:
*
* A region lowers to do { ... } while(false), and POP lowers to break. The goto plus
* label scheme used by the .NET generated regex backend is deliberately not adopted.
*
* Generated variable names are owned by SourceGenContext. No generator invents one.
*
* The per shape checklist this backend is rebuilt against lives in
* future/ngra-regex-sourcegen-goldens.md, and the handwritten target code for each shape
* lives in test/SingleFile/regex.cs.
*/
private const string current_ptr = "cur_ptr_global";
private const string offset = "offset";
private const string orig_text = "text";
#region L0 method level
/// <summary>
/// Generates one <c> Match </c> method from a fully prefixed IR stream.
/// </summary>
/// <remarks>
/// Not implemented yet. Returns an empty emitter so the generator pipeline stays runnable
/// while the backend is being rebuilt.
/// </remarks>
internal static SequenceEmitter<string> GenerateSource(
TextRegexSource source
)
{
//
// L0, method skeleton
//
// TODO L0.1 Emit the Match signature from ROS_char and SourceGenContext.InputSpan.
// TODO L0.2 Emit region anchor and region result slots, count from RegexInfo.RegionCount.
// TODO L0.3 Emit line result slots, only for lines an Observe_Line prefix refers to.
// TODO L0.4 Emit the whole pattern guard from RegexInfo.MinPredictLength.
// TODO L0.5 Emit the cursor, the step result and any scratch declaration the context collected.
//
/*
* Slot demand is only known once the body exists, so the body has to be generated
* into a detached emitter first and spliced in after the declaration block.
*
* The dispatch chain that walks the IR stream belongs here too. It offers each window
* to L1, L2, L3 then L4, and adds the returned consumption count to the cursor. An
* unclaimed window must produce a hard build break rather than silently emit nothing,
* because code that compiles but matches incorrectly is far harder to diagnose.
*/
SequenceEmitter<string> se = new();
List<SequenceEmitter<string>> sub_match_list = [];
SourceGenContext sc = new();
@@ -47,7 +119,8 @@ namespace StgSharp.RegularAnalysis.Text
while (true)
{
GenerateRegion(region, sc, match_e, sub_match_list);
// GenerateRegion(region, sc, match_e, sub_match_list);
_ = TryGenerateAltChain(0, region, sc, match_e, sub_match_list);
if (end >= ir_sp.Length)
{
break;
@@ -72,7 +145,7 @@ namespace StgSharp.RegularAnalysis.Text
.AppendLine($@"int {offset} = 0;")
.AppendLine();
_ = sc.GenVarDefine(se);
// _ = sc.GenVarDefine(se);
_ = se.Append(match_e);
@@ -87,6 +160,10 @@ namespace StgSharp.RegularAnalysis.Text
return se;
}
#endregion
#region shared helpers
[MethodImpl(MethodImplOptions.AggressiveInlining)]
private static int FindRegionEnd(
ReadOnlySpan<RegexIR> ir_sp,
@@ -111,151 +188,11 @@ namespace StgSharp.RegularAnalysis.Text
return SyntaxFactory.Literal(str).Text;
}
#region sub generator
private static int TryGenerateFindSeq(
ReadOnlySpan<RegexIR> scan_win,
SequenceEmitter<string> se,
SourceGenContext ng
private static string FormatCharLiteral(
char value
)
{
if (scan_win[0] is not RegexFindSeqIR seq) {
return -1;
}
string pattern = seq.Pattern;
int length = pattern.Length;
string index_caller = seq.IsFirst ? "IndexOf" : "LastIndexOf";
_ = se.AppendLine()
.AppendLine($@"if(({offset} = [{current_ptr}..].{index_caller}({pattern})) != -1)")
.AppendLine($@"{{")
.AppendLine($@"{current_ptr} += {offset} + {length};")
.AppendLine($@"}}");
return length;
}
private static int TryGenerateMatchSeq(
ReadOnlySpan<RegexIR> scan_win,
SequenceEmitter<string> se,
SourceGenContext ng
)
{
if (scan_win[0] is not RegexFindSeqIR seq) {
return -1;
}
string pattern = FormatStringLiteral(seq.Pattern);
int length = pattern.Length;
_ = se.AppendLine()
.AppendLine($@"if([{current_ptr}..({current_ptr} + {length})] == {pattern})")
.AppendLine($@"{{")
.AppendLine($@"{current_ptr} += {offset} + {length};")
.AppendLine($@"}}");
return length;
}
private static int TryGenerateMatchSet(
ReadOnlySpan<RegexIR> scan_win,
SequenceEmitter<string> se,
SourceGenContext ng
)
{
if (scan_win[0] is not RegexMatchSeqIR seq) {
return -1;
}
string pattern = seq.Pattern;
RegexCharSet[] set_pattern = FigureCharSetType(pattern, out bool accept);
string var_char = ng.CurChar;
_ = se.AppendLine($@"// match charset of {pattern}")
.AppendLine($@"char {var_char} = {orig_text}[{current_ptr}];")
.AppendLine();
_ = se.AppendLine("if(");
RegexCharSet p = set_pattern[0];
string _case = GenerateCase(p);
string _case_begin = accept ? string.Empty : "!(";
_ = se.AppendLine($@"if ({_case_begin}{_case}||");
for (int i = 1; i < set_pattern.Length - 1; i++)
{
p = set_pattern[i];
_case = GenerateCase(p);
if (string.IsNullOrEmpty(_case))
{
continue;
}
_ = se.AppendLine($@" {_case} ||");
}
p = set_pattern[^1];
_case = GenerateCase(p);
string _case_end = accept ? string.Empty : ")";
_ = se.AppendLine($@"{_case}{_case_end})")
.AppendLine("{")
.AppendLine($@"{current_ptr} += 1;")
.AppendLine("}");
return 1;
string GenerateCase(
RegexCharSet c
)
{
switch (c.Type)
{
// Handle single character
case CharSetType.Single:
string set = c.Value;
if (set.Length == 1)
{
return $@"{var_char} == '{c.Value}'";
} else if (set.Length < 5)
{
StringBuilder sb = new();
_ = sb.Append('(');
foreach (char item in set) {
_ = sb.Append(CultureInfo.InvariantCulture, $@"item == '{item}' or ");
}
_ = sb.Append(')');
return sb.ToString();
} else
{
return $@"""{c.Value}"".AsSpan().Contains({var_char})";
}
// Handle range of characters
case CharSetType.Range:
return $@"({var_char} >= '{c.Value[0]}' && {var_char} <= '{c.Value[1]}')";
// Handle set of characters
case CharSetType.Set:
if (c.Accept)
{
return c.Value switch
{
"s" => $@"char.IsWhiteSpace({var_char})",
"w" => $@"char.IsLetterOrDigit({var_char}) || {var_char} == '_'",
"d" => $@"char.IsDigit({var_char})",
_ => throw new InvalidCastException(),
};
} else
{
return c.Value switch
{
"s" => $@"!char.IsWhiteSpace({var_char})",
"w" => $@"!char.IsLetterOrDigit({var_char}) && {var_char} != '_'",
"d" => $@"!char.IsDigit({var_char})",
_ => throw new InvalidCastException(),
};
}
case CharSetType.None:
return string.Empty;
default:
return string.Empty;
}
}
return SyntaxFactory.Literal(value).Text;
}
#endregion
src/StgSharp.RegularAnalysis/Text/TextRegexSourceGen.GenContext.cs
+146 -55
View File
@@ -19,86 +19,177 @@ namespace StgSharp.RegularAnalysis.Text
#region var naming
private class SourceGenContext
/// <summary>
/// Tracks variable naming and declaration demand while generating one match method.
/// </summary>
/// <remarks>
/// Method level state is limited to the active cursor and the result of the most recent
/// instruction. Everything else is either a region slot, a line slot, a capture slot, or
/// a block scoped scratch variable. Scratch names are never reused, so a nested block can
/// always declare its own without colliding with an enclosing one.
/// </remarks>
private sealed class SourceGenContext
{
public bool Need_char_cur { get; set; }
/// <summary>
/// Active pointer. Indexes the next unconsumed element of the input.
/// </summary>
public const string Cursor = "__cur";
public bool Need_remain_cur { get; set; }
/// <summary>
/// Name of the match method parameter holding the input sequence.
/// </summary>
public const string InputSpan = "text";
public bool Need_find_result { get; set; }
/// <summary>
/// Result of the instruction emitted immediately before the current one.
/// </summary>
public const string StepResult = "__ok";
private const string __char_cur = "__char_cur";
private const string __remain_cur = "__remain_cur";
private const string __find_result = "__find_result";
private readonly Dictionary<string, int> _groupIndex = [];
private readonly SortedSet<int> _lineSlots = [];
private readonly HashSet<int> _observedLines = [];
private readonly SortedSet<int> _regionSlots = [];
private int _min_region_begin_index;
private int _anonymousGroup;
private int _scratchIndex;
public string MakeNewRegionBegin()
/// <summary>
/// Records that some prefix observes the result of the instruction at line
/// <paramref name="line" />.
/// </summary>
public void MarkLineObserved(
int line
)
{
string name = $"__region_begin_{_min_region_begin_index}";
_min_region_begin_index++;
return name;
_ = _observedLines.Add(line);
}
public string CurChar
/// <summary>
/// Indicates whether the instruction at line <paramref name="line" /> has to persist
/// its result into a line slot.
/// </summary>
public bool IsLineObserved(
int line
)
{
get
{
Need_char_cur = true;
return __char_cur;
}
return _observedLines.Contains(line);
}
#region namig group
/// <summary>
/// Allocates a fresh block scoped scratch variable name.
/// </summary>
/// <param name="tag">
/// Short role marker, for example <c> n </c> for a counter or <c> ch </c> for a
/// character.
/// </param>
public string NewScratch(
string tag
)
{
return $"__{tag}{_scratchIndex++}";
}
private int _min_named_group;
private Dictionary<string, int> _name_idx_map = [];
/// <summary>
/// Name of the slot holding the cursor value captured when region
/// <paramref name="region" /> was entered.
/// </summary>
public string RegionAnchor(
int region
)
{
_ = _regionSlots.Add(region);
return $"__r{region}_begin";
}
public string MakeNewNamedGroup(
/// <summary>
/// Name of the slot holding the match result of region <paramref name="region" />.
/// </summary>
public string RegionResult(
int region
)
{
_ = _regionSlots.Add(region);
return $"__r{region}_ok";
}
/// <summary>
/// Name of the slot holding the match result of the instruction at line
/// <paramref name="line" />. Only lines actually referenced by an
/// <see cref="RegexIRPrefixMask.Observe_Line" /> prefix get a slot.
/// </summary>
public string LineResult(
int line
)
{
_ = _lineSlots.Add(line);
return $"__l{line}_ok";
}
public string GroupBegin(
string name
)
{
if (_name_idx_map.TryGetValue(name, out int idx)) {
return $"__named_group_{idx}";
}
idx = _min_named_group;
_name_idx_map[name] = idx;
_min_named_group++;
return $"__named_group_{idx}";
return $"__g{ResolveGroup(name)}_begin";
}
#endregion
public string FindResult
{
get
{
Need_find_result = true;
return __find_result;
}
}
public string CurRemain
{
get
{
Need_remain_cur = true;
return __remain_cur;
}
}
public SequenceEmitter<string> GenVarDefine(
SequenceEmitter<string> se
public string GroupEnd(
string name
)
{
if (se is null) {
return $"__g{ResolveGroup(name)}_end";
}
/// <summary>
/// Emits declarations for every slot requested during body generation.
/// </summary>
/// <remarks>
/// Must be called after the method body has been generated, but the produced lines
/// have to be placed before it.
/// </remarks>
public SequenceEmitter<string> EmitDeclarations(
SequenceEmitter<string> emitter
)
{
if (emitter is null) {
return null!;
}
_ = se.AppendLine($@"int {__char_cur}", Need_char_cur)
.AppendLine($@"int {__remain_cur}", Need_remain_cur);
return se;
_ = emitter.AppendLine($@"int {Cursor} = 0;")
.AppendLine($@"bool {StepResult} = true;");
foreach (int region in _regionSlots)
{
_ = emitter.AppendLine($@"int {RegionAnchor(region)} = 0;")
.AppendLine($@"bool {RegionResult(region)} = false;");
}
foreach (int line in _lineSlots) {
_ = emitter.AppendLine($@"bool {LineResult(line)} = false;");
}
foreach (int group in _groupIndex.Values)
{
_ = emitter.AppendLine($@"int __g{group}_begin = -1;")
.AppendLine($@"int __g{group}_end = -1;");
}
return emitter;
}
private int ResolveGroup(
string name
)
{
string key = string.IsNullOrEmpty(name) ?
$"#{_anonymousGroup++}" :
name;
if (_groupIndex.TryGetValue(key, out int index)) {
return index;
}
index = _groupIndex.Count;
_groupIndex[key] = index;
return index;
}
}
test/SingleFile/regex.cs
+17 -1
View File
@@ -164,5 +164,21 @@ static void MatchCountAnyGroup(ReadOnlySpan<char> seq)
int ptr = 0;
int group_start = ptr;
int furthest = -1;
do
{
if((furthest = seq.LastIndexOf("123"))>0)
{
ptr += furthest + 3;
break;
}
} while(true);
// grouping
if(furthest >= 0)
{
ptr = furthest;
int group_end = ptr;
// group "name" matched from group_start to group_end
}
}
test/StgSharpDebug/TestRegexAnalyse.cs
+2 -2
View File
@@ -21,8 +21,8 @@ namespace StgSharp
{
private static string[] regexSource = [
@"(?<name>.*)234",
/*
@"12(1234|(012.*34)|9125)34",
/**/
@"http|https|ftp|ftps",
@"abc[0-9]x|abc[0-9]y|abd[0-9]x|abd[0-9]y|xyz",
@"get_name|get_value|get_type|set_name|set_value|set_type",