remove yesterday's generatoer, use new generator

This commit is contained in:
Youmu Liu
2026-07-29 18:10:45 +08:00
parent 3db2bfd9af
commit 97576404cc
4 changed files with 328 additions and 132 deletions
+104 -19
View File
@@ -34,12 +34,14 @@
### 正确性核心与纯优化的分界 ### 正确性核心与纯优化的分界
- **L2 + L4 + L5 是正确性核心**:region 作用域、单指令主体、前缀装饰三者齐备, - **L2 + L4 + L5 + L1 的必需项是正确性核心**:region 作用域、单指令主体、前缀装饰
任意表达式即可端到端生成出语义正确的代码。 三者支撑绝大多数形态;但 `COUNT_COMPLEX` 与 FIND 系(L1.5–L1.9)的 body 不是独立 region,
- **L1 与 L3 是纯优化**:它们只把 L2/L4 会生成的嵌套结构压平成更好的代码形状。 L2 接不住,没有降级路径,因此同样属于核心。
未实现时必须干净地返回 `-1`,让窗口自然降级,**不得影响正确性**。 - **L1 的压平项(L1.1–L1.4)与 L3 全部是纯优化**:它们只把 L2/L4 会生成的嵌套结构
压平成更好的代码形状。未实现时必须干净地返回 `-1`,让窗口自然降级,**不得影响正确性**。
这条分界决定了实现顺序:先把 L2/L4/L5 打通拿到可运行基线,再回头做 L1/L3。 这条分界决定了实现顺序:先把 L2/L4/L5 打通拿到可运行基线,再补 L1 的必需项,
最后才是 L1.1–L1.4 与 L3。
## 记录模板 ## 记录模板
@@ -79,13 +81,13 @@ IR -> Code 规则:
关注点:region 槽是否必须全量分配、行结果槽能否按 `ObserveSource` 稀疏分配。 关注点:region 槽是否必须全量分配、行结果槽能否按 `ObserveSource` 稀疏分配。
## L1 Region 组合级(纯优化) ## L1 Region 组合级(含必需项)
一次消耗**多个 region + 尾部控制指令**,是窗口消耗量最高的形态。 一次消耗**多个 region + 尾部控制指令**,是窗口消耗量最高的形态。
这些组合在 `EmitPrefix` 里已经被识别过一次 这些组合在 `EmitPrefix` 里已经被识别过一次
(`TryEmitAltChain` / `TryEmitTryGroupTail` / `TryEmitComplexIR`),生成器应与之一一对应。 (`TryEmitAltChain` / `TryEmitTryGroupTail` / `TryEmitComplexIR`),生成器应与之一一对应。
除 L1.5–L1.7 外,本层全部是**代码形状优化**:不实现时窗口会降级到 L2 + L5, 除 L1.5–L1.9 外,本层全部是**代码形状优化**:不实现时窗口会降级到 L2 + L5,
生成出语义等价但嵌套更深的代码。 生成出语义等价但嵌套更深的代码。
| # | 触发窗口 | 消耗 | 覆盖样本 | 性质 | 状态 | | # | 触发窗口 | 消耗 | 覆盖样本 | 性质 | 状态 |
@@ -96,15 +98,84 @@ IR -> Code 规则:
| L1.4 | L1.3 且 region 内部是 ALT 链 | 嵌套 | `(?<a>abcd\|a(bc)c)` | 压平 | [ ] | | L1.4 | L1.3 且 region 内部是 ALT 链 | 嵌套 | `(?<a>abcd\|a(bc)c)` | 压平 | [ ] |
| L1.5 | `COUNT_COMPLEX` + body | `CommandCount + 1` | `(ab){2,3}` | **必需** | [ ] | | L1.5 | `COUNT_COMPLEX` + body | `CommandCount + 1` | `(ab){2,3}` | **必需** | [ ] |
| L1.6 | `COUNT_COMPLEX` + body(含 ALT) + 线性后缀 | 嵌套 + 后缀 | `(123(4\|5)67){3,4}114514` | **必需** | [ ] | | L1.6 | `COUNT_COMPLEX` + body(含 ALT) + 线性后缀 | 嵌套 + 后缀 | `(123(4\|5)67){3,4}114514` | **必需** | [ ] |
| L1.7 | `FIND_COMPLEX` + body | `CommandCount + 1` | `.*(ab\|cd)` | **必需** | [ ] | | L1.7 | `FIND_COMPLEX`,`CommandCount ≥ 0` | `CommandCount + 1` | `.*(ab\|cd)` | **必需** | [ ] |
| L1.8 | `FIND_COMPLEX`,`CommandCount = -1` | 1 + region 剩余 | `(?<name>.*)234` | **必需** | [ ] |
| L1.9 | `FIND_SEQ` / `FIND_SET` | 1 + region 剩余 | `.*abc` | **必需** | [ ] |
期望代码形状:L1.1/L1.2 压平为 `if / else if / else` 链;L1.3/L1.4 在 region 成功分支写捕获区间; 期望代码形状:L1.1/L1.2 压平为 `if / else if / else` 链;L1.3/L1.4 在 region 成功分支写捕获区间;
L1.5–L1.7 为循环 / 搜索包裹 body,`CommandCount` 必须按前缀化后的实际条数重算。 L1.5/L1.6 为计数循环包裹 body;L1.7–L1.9 为**候选重试循环**包裹 body(见下)。
`CommandCount` 必须按前缀化后的实际条数重算。
L1.5–L1.7 标记为必需,是因为 `COUNT_COMPLEX` / `FIND_COMPLEX` 的 body 不是独立 region, L1.5–L1.9 标记为必需,是因为它们的 body 都不是独立 region,L2 接不住,没有替代降级路径。
L2 接不住它们,没有替代降级路径。
关注点:ALT 各 case 之间活动指针如何恢复、`PACK` 取哪一个成功结果作为捕获区间。 ### FIND 系的回溯语义
FIND 产出的不是一个位置,是一个**按优先级排序的候选位置序列**:
- `FURTHEST`(贪婪 `.*`):从最远候选起,验证失败往**前**退
- `NEAREST`(惰性 `.*?`):从最近候选起,验证失败往**后**进
body 是候选的**验证器**。验证失败必须取下一个候选重来,候选耗尽才是真失败。
只做一次 `LastIndexOf` 就判定成败是错的——`.*abcX` 里最后一个 `abc` 后面没有 `X` 时,
必须退到倒数第二个 `abc`。
`CommandCount = -1` 的语义已被优化器坐实:`TryReplaceDotStar` 把 `.*` 直接产出
`FindComplex(-1)`,随后 `TyrMergeFindRestTwice` / `TyrMergeFindRestTry` 尽力把它收敛成确定长度,
收敛不掉的保持 -1,表示「body 是 region 剩余全部」。所以后端只需处理两种 body 长度:
确定值,和 -1。
`FIND_SEQ` / `FIND_SET` 没有 `CommandCount` 字段,语义等同 -1。只有当它恰好是 region
最后一条时才退化成单次搜索,此时消耗为 1。
期望代码形状:
```csharp
int __bound = text.Length;
__ok = false;
while (true)
{
int __at = text[__cur..__bound].LastIndexOf("abc");
if (__at < 0) { __ok = false; break; } // 候选耗尽,唯一失败出口
int __save = __cur;
__cur = __cur + __at + 3;
// ── body / region 剩余指令内联展开在这里 ──
if (__ok) { break; } // 验证通过
__cur = __save;
__bound = __save + __at; // 退到前一个候选
}
```
### FIND 与 `POP` 的冲突(必须处理)
`POP` 编译成 `break` 时跳出的是**最近**一层循环,而 FIND 的 `while` 比 region 的
`do{}while(false)` 更近。body 里的 `POP` 会跳出重试循环而不是 region,语义直接错。
不引入 `goto` 的解法是加一个 region 终结标志:
```csharp
do // region N
{
bool __rN_done = false;
...
while (!__rN_done) // find retry
{
// 候选搜索
do // ← 隔离层,POP 的 break 落在这
{
// body,POP: __rN_ok = x; __rN_done = true; break;
} while (false);
if (__rN_done || __ok) { break; }
// 退候选
}
} while (false);
```
关注点:ALT 各 case 之间活动指针如何恢复、`PACK` 取哪一个成功结果作为捕获区间、
嵌套 FIND 会产生嵌套重试循环(复杂度是候选数的乘积,是灾难性回溯的来源)、
重试循环内 `__ok` 跨迭代被复写(`Observe_Line` 指向循环体内某条指令时,
读到的是最后一次迭代的值,当前前缀模型没有覆盖这个情形)。
## L2 Region 级(正确性核心) ## L2 Region 级(正确性核心)
@@ -133,7 +204,11 @@ __ok = __rN_ok;
失败路径必须把活动指针复位到 region 入口。`POP` 直接编译为 `break`,不引入标签。 失败路径必须把活动指针复位到 region 入口。`POP` 直接编译为 `break`,不引入标签。
关注点:region 全部内联展开、嵌套 region 的指针复位链、`break` 只能跳出最近一层 `do`。 **例外**:当 region 内出现 FIND 系指令时,`break` 会被 FIND 的重试循环截获,
必须改用 L1 节里的 `__rN_done` 终结标志方案。
关注点:region 全部内联展开、嵌套 region 的指针复位链、`break` 只能跳出最近一层循环
(`do{}while(false)` 和 FIND 的 `while` 都算)。
## L3 多指令定式(纯优化) ## L3 多指令定式(纯优化)
@@ -144,12 +219,13 @@ __ok = __rN_ok;
| --- | --- | --- | --- | --- | | --- | --- | --- | --- | --- |
| L3.1 | `FIND_COMPLEX FURTHEST COUNT:-1` + `PACK` + `MATCH_SEQ` | 3 | `(?<name>.*)234` | [ ] | | L3.1 | `FIND_COMPLEX FURTHEST COUNT:-1` + `PACK` + `MATCH_SEQ` | 3 | `(?<name>.*)234` | [ ] |
| L3.2 | 同上,锚点为单字符 | 3 | `(?<name>.*)a` | [ ] | | L3.2 | 同上,锚点为单字符 | 3 | `(?<name>.*)a` | [ ] |
| L3.3 | `FIND_SEQ` + `MATCH_SEQ`,合并为一次 `IndexOf` | 2 | `.*abc` | [ ] | | L3.3 | `FIND_SEQ` + `MATCH_SEQ`,把验证器折进搜索模式,消除回溯 | 2 | `.*abc` | [ ] |
| L3.4 | `MATCH_SEQ` + `COUNT_SEQ`(`a+`、`a{m,n}` 展开形态) | 2 | `a{3,5}`、`a+` | [ ] | | L3.4 | `MATCH_SEQ` + `COUNT_SEQ`(`a+`、`a{m,n}` 展开形态) | 2 | `a{3,5}`、`a+` | [ ] |
| L3.5 | 连续 `MATCH_SEQ` / `MATCH_SET`,合流为单个 `if` 与表达式 | 2–8 | `a[abc]d` | [ ] | | L3.5 | 连续 `MATCH_SEQ` / `MATCH_SET`,合流为单个 `if` 与表达式 | 2–8 | `a[abc]d` | [ ] |
| L3.6 | 尾部 `COUNT_SET .*`,快速成功 | 1–2 | `abc.*` | [ ] | | L3.6 | 尾部 `COUNT_SET .*`,快速成功 | 1–2 | `abc.*` | [ ] |
关注点:合并后失败诊断是否还能定位到原指令、合并是否破坏 `Observe_Line` 的行号引用。 关注点:合并后失败诊断是否还能定位到原指令、合并是否破坏 `Observe_Line` 的行号引用。
L3.3 只有在合并后 region 剩余为空时才真正消除回溯,否则 L1.9 的重试循环仍然要保留。
## L4 单指令级 ## L4 单指令级
@@ -164,11 +240,13 @@ __ok = __rN_ok;
| L4.5 | `MATCH_SET` 任意字符 | `.` | [ ] | | L4.5 | `MATCH_SET` 任意字符 | `.` | [ ] |
| L4.6 | `COUNT_SEQ` | `a*`、`a?` | [ ] | | L4.6 | `COUNT_SEQ` | `a*`、`a?` | [ ] |
| L4.7 | `COUNT_SET` | `[abc]*`、`\w+`、`.{11,13}`、`[.]*` | [ ] | | L4.7 | `COUNT_SET` | `[abc]*`、`\w+`、`.{11,13}`、`[.]*` | [ ] |
| L4.8 | `FIND_SEQ` | `.*abc` 降级路径 | [ ] |
| L4.9 | `FIND_SET` | — | [ ] |
| L4.10 | `POP` | ALT 尾 | [ ] | | L4.10 | `POP` | ALT 尾 | [ ] |
| L4.11 | `PACK` | `(abc)` 默认序号组 | [ ] | | L4.11 | `PACK` | `(abc)` 默认序号组 | [ ] |
L4.8 / L4.9 已移出本层。`FIND_SEQ` / `FIND_SET` **不存在单条形态**——它们必须联动 region
剩余部分做候选重试,见 L1.9。只有 FIND 恰好是 region 末条时才退化成一次搜索,
那是 L1.9 的退化分支,不是独立的单指令生成器。编号保留空缺,避免打乱既有引用。
关注点:`[.]` 与 `.` 必须走不同后端语义;`[.]*` 不得被当作 `.*` 优化。 关注点:`[.]` 与 `.` 必须走不同后端语义;`[.]*` 不得被当作 `.*` 优化。
## L5 前缀装饰(横切,消耗 0) ## L5 前缀装饰(横切,消耗 0)
@@ -194,7 +272,8 @@ __ok = __rN_ok;
## 实现顺序 ## 实现顺序
调度链按 L1 → L4 排列,但**实现**先拿正确性核心(L2/L4/L5),再回头填纯优化层(L1/L3): 调度链按 L1 → L4 排列,但**实现**先拿正确性核心(L2/L4/L5),再补 L1 的必需项,
最后才是压平层(L1.1–L1.4 / L3):
1. **L0** — 生成的空骨架能通过编译,`TextRegexSourceGen.GenFile` 的两处 TODO 闭合。 1. **L0** — 生成的空骨架能通过编译,`TextRegexSourceGen.GenFile` 的两处 TODO 闭合。
2. **L2.1** — `(abc)` 产出正确的 `do{}while(false)` 作用域与复位逻辑。 2. **L2.1** — `(abc)` 产出正确的 `do{}while(false)` 作用域与复位逻辑。
@@ -202,12 +281,18 @@ __ok = __rN_ok;
4. **L4 全部** — 兜底层补齐,任意线性表达式可端到端生成。 4. **L4 全部** — 兜底层补齐,任意线性表达式可端到端生成。
5. **L5.6–L5.9** — 跨行 / 跨 region 观察与提前退出,`a|b` 与 `(?<name>abc)` 自然跑通(嵌套形态)。 5. **L5.6–L5.9** — 跨行 / 跨 region 观察与提前退出,`a|b` 与 `(?<name>abc)` 自然跑通(嵌套形态)。
**此时获得可运行基线,任何表达式都能生成正确代码。** **此时获得可运行基线,任何表达式都能生成正确代码。**
6. **L1.5、L1.7** — 复杂计数与复杂搜索,无降级路径,必须实现。 6. **L1.5、L1.7–L1.9** — 复杂计数与 FIND 系,无降级路径,必须实现。
FIND 系要连带处理 `POP` 与重试循环的 `break` 冲突。
7. **L1.1–L1.4** — 把 ALT 链与捕获组的嵌套形态压平。 7. **L1.1–L1.4** — 把 ALT 链与捕获组的嵌套形态压平。
8. **L3 全部** — 代码质量优化,可持续追加。 8. **L3 全部** — 代码质量优化,可持续追加。
第一阶段验收口径(沿用原优先级):`MATCH_SEQ`、`MATCH_SET`、`COUNT_SEQ`、`COUNT_SET`、 第一阶段验收口径(沿用原优先级):`MATCH_SEQ`、`MATCH_SET`、`COUNT_SEQ`、`COUNT_SET`、
简单 `ALT`、`PACK`、`FIND_SEQ`、`FIND_COMPLEX COUNT:-1` 全部稳定。 简单 `ALT`、`PACK`、`FIND_COMPLEX COUNT:-1` 全部稳定。
注:`FIND_SEQ` 已从原口径移除。`TryReplaceDotStar` 产出的是 `FindComplex(-1)`,
IROptimize 里没有 `RegexFindSeqIR` 的生成点,`ngra-regex-ir.md` 里写的
`.*a → FIND_SEQ(a, last)` 尚未落地。在生成点补上之前,L1.9 的 `FIND_SEQ` / `FIND_SET`
分支实际不可达,L3.3 也无从触发。
低优先级,暂不纳入 checklist: 低优先级,暂不纳入 checklist:
@@ -14,109 +14,6 @@ namespace StgSharp.RegularAnalysis.Text
{ {
internal partial class TextRegexSourceGen internal partial class TextRegexSourceGen
{ {
/*
* =======================================================================================
* L1, several regions plus a tail
* =======================================================================================
*
* Highest consumption in the dispatch chain, so these are offered the window first. Each
* shape here was already recognised once during prefix emission, so a generator should
* mirror the matching RegexAnalyzer.PrefixEmit recogniser rather than re-derive the shape.
*
* L1.2 ALT chain with a shared suffix
* window L1.1 followed by linear instructions in the parent region
* sample (ab|[cd]|.)x
* nature flattening, may return -1
*
* L1.3 group tail
* window region, POP FAIL with Prev_Fail, PACK SUCCESS with Prev_Success
* mirrors TryEmitTryGroupTail
* shape region block, capture written on the success branch
* sample (?<name>abc)
* nature flattening, may return -1
*
* L1.4 group tail wrapping an ALT chain
* sample (?<a>abcd|a(bc)c)
* nature flattening, may return -1
*
* L1.5 COUNT_COMPLEX plus body
* window RegexCountComplexIR, then CommandCount entries
* mirrors TryEmitComplexIR
* shape loop wrapping the generated body
* sample (ab){2,3}
* nature REQUIRED, no fallback exists
*
* L1.6 COUNT_COMPLEX whose body contains an ALT chain, plus a linear suffix
* sample (123(4|5)67){3,4}114514
* nature REQUIRED, no fallback exists
*
* L1.7 FIND_COMPLEX plus body
* window RegexFindComplexIR, then CommandCount entries
* shape search loop wrapping the generated body
* sample .*(ab|cd)
* nature REQUIRED, no fallback exists
*
* L1.5 to L1.7 are required because a complex body is not a region of its own. L2 cannot
* pick it up, so there is no degradation path if these are missing.
*
* CommandCount must be read from the prefixed IR. Prefix emission rebuilds these counts,
* so the value carried by the pre-prefix IR no longer applies.
*
* Open questions to settle while implementing:
* - how the cursor is restored between ALT cases
* - which successful outcome a PACK uses as its capture span
* - whether a region body ever deserves its own generated method instead of inlining
*
* =======================================================================================
* L2, one region
* =======================================================================================
*
* Consumes CommandCount + 1, the whole block a TRY lowers to. Together with L4 and L5 this
* is the correctness core: once these three exist, every expression can be generated, just
* with deeper nesting than L1 and L3 would produce.
*
* L2.1 RegexTryIR plus its body sample (abc)
* L2.2 the same, body containing a nested region sample (a(bc))
* L2.3 a single instruction region folded by prefix emission sample (a)
*
* Target shape, names owned by SourceGenContext:
*
* __rN_begin = __cur;
* __rN_ok = false;
* do
* {
* // body, a POP breaks out here
* __rN_ok = __ok;
* } while (false);
* if (!__rN_ok) { __cur = __rN_begin; }
* __ok = __rN_ok;
*
* Points the implementation has to settle:
* - the failure path restores the cursor to the region entry
* - break only leaves the nearest enclosing do block, so a nested region must not rely
* on it to exit an outer one
* - the whole match body is itself region zero, which is what makes a top level POP
* legal
*
* =======================================================================================
* L3, fixed multi instruction windows
* =======================================================================================
*
* Consumes two to eight entries. Pure code quality work, never required for correctness.
* Every one of these must return -1 cleanly when the window does not match, so the entry
* falls through to L4.
*
* L3.1 FIND_COMPLEX FURTHEST COUNT:-1, PACK, MATCH_SEQ 3 (?<name>.*)234
* L3.2 the same with a single element anchor 3 (?<name>.*)a
* L3.3 FIND_SEQ, MATCH_SEQ, merged into one search 2 .*abc
* L3.4 MATCH_SEQ, COUNT_SEQ, the expanded a+ shape 2 a{3,5}, a+
* L3.5 a run of MATCH_SEQ and MATCH_SET, merged into one condition 2 to 8 a[abc]d
* L3.6 trailing COUNT_SET over a dot, fast success 1 to 2 abc.*
*
* Two hazards to check for each of these:
* - merging must not break the line indices an Observe_Line prefix refers to
* - a merged window still has to leave __ok describing the whole window
*/
private static int TryGenerateAltChain( private static int TryGenerateAltChain(
int outer_level, int outer_level,
@@ -165,6 +62,201 @@ namespace StgSharp.RegularAnalysis.Text
} }
return i; return i;
} }
/*
* =======================================================================================
* L1, several regions plus a tail
* =======================================================================================
*
* Highest consumption in the dispatch chain, so these are offered the window first. Each
* shape here was already recognised once during prefix emission, so a generator should
* mirror the matching RegexAnalyzer.PrefixEmit recogniser rather than re-derive the shape.
*
* L1.2 ALT chain with a shared suffix
* window L1.1 followed by linear instructions in the parent region
* sample (ab|[cd]|.)x
* nature flattening, may return -1
*
* L1.3 group tail
* window region, POP FAIL with Prev_Fail, PACK SUCCESS with Prev_Success
* mirrors TryEmitTryGroupTail
* shape region block, capture written on the success branch
* sample (?<name>abc)
* nature flattening, may return -1
*
* L1.4 group tail wrapping an ALT chain
* sample (?<a>abcd|a(bc)c)
* nature flattening, may return -1
*
* L1.5 COUNT_COMPLEX plus body
* window RegexCountComplexIR, then CommandCount entries
* mirrors TryEmitComplexIR
* shape loop wrapping the generated body
* sample (ab){2,3}
* nature REQUIRED, no fallback exists
*
* L1.6 COUNT_COMPLEX whose body contains an ALT chain, plus a linear suffix
* sample (123(4|5)67){3,4}114514
* nature REQUIRED, no fallback exists
*
* L1.7 FIND_COMPLEX with CommandCount >= 0
* window RegexFindComplexIR, then CommandCount entries
* shape candidate retry loop wrapping the body
* sample .*(ab|cd)
* nature REQUIRED, no fallback exists
*
* L1.8 FIND_COMPLEX with CommandCount == -1
* window RegexFindComplexIR, then the rest of the region
* shape candidate retry loop wrapping the rest of the region
* sample (?<name>.*)234
* nature REQUIRED, no fallback exists
*
* L1.9 FIND_SEQ or FIND_SET
* window the instruction, then the rest of the region
* shape candidate retry loop wrapping the rest of the region
* sample .*abc
* nature REQUIRED, no fallback exists
*
* L1.5 to L1.9 are required because none of their bodies is a region of its own. L2
* cannot pick them up, so there is no degradation path if these are missing.
*
* CommandCount must be read from the prefixed IR. Prefix emission rebuilds these counts,
* so the value carried by the pre-prefix IR no longer applies.
*
* ---------------------------------------------------------------------------------------
* Backtracking semantics of the FIND family
* ---------------------------------------------------------------------------------------
*
* A FIND does not yield one position, it yields a priority ordered sequence of candidates.
* FURTHEST starts at the last one and steps backwards on failure, NEAREST starts at the
* first and steps forwards. The body is the verifier for a candidate: when it fails, the
* next candidate has to be tried, and only an exhausted candidate list is a real failure.
* A single LastIndexOf is wrong, because .*abcX has to fall back to the second to last abc
* when the last one is not followed by an X.
*
* CommandCount == -1 means the body is the rest of the region. TryReplaceDotStar emits
* FindComplex(-1) for a bare .*, then TyrMergeFindRestTwice and TyrMergeFindRestTry try to
* collapse it into a definite length. Whatever survives stays at -1. FIND_SEQ and FIND_SET
* carry no CommandCount at all and behave like -1. Only a FIND that happens to be the last
* instruction of its region degrades into a single search, consuming one entry.
*
* Target shape:
*
* int __bound = text.Length;
* __ok = false;
* while (true)
* {
* int __at = text[__cur..__bound].LastIndexOf("abc");
* if (__at < 0) { __ok = false; break; } // candidates exhausted, only failure
* int __save = __cur;
* __cur = __cur + __at + 3;
*
* // body, or the rest of the region, expands here
*
* if (__ok) { break; } // verified
* __cur = __save;
* __bound = __save + __at; // step to the previous candidate
* }
*
* POP conflicts with the retry loop. A POP lowers to break, and the retry while is nearer
* than the region do block, so the break would leave the wrong scope. The fix without goto
* is a region completion flag:
*
* do // region N
* {
* bool __rN_done = false;
* while (!__rN_done) // find retry
* {
* // candidate search
* do // isolation layer, POP breaks out here
* {
* // body, POP sets __rN_ok, sets __rN_done, breaks
* } while (false);
* if (__rN_done || __ok) { break; }
* // step to the next candidate
* }
* } while (false);
*
* Two further hazards:
* - nested FIND produces nested retry loops and the cost multiplies, which is the
* catastrophic backtracking shape, worth a guard or at least a marker
* - __ok is overwritten across retry iterations, so an Observe_Line prefix pointing into
* a retry body reads the value of the last iteration, a case the current prefix model
* does not cover
*
* Open questions to settle while implementing:
* - how the cursor is restored between ALT cases
* - which successful outcome a PACK uses as its capture span
* - whether a region body ever deserves its own generated method instead of inlining
*
* =======================================================================================
* L2, one region
* =======================================================================================
*
* Consumes CommandCount + 1, the whole block a TRY lowers to. Together with L4 and L5 this
* is the correctness core: once these three exist, every expression can be generated, just
* with deeper nesting than L1 and L3 would produce.
*
* L2.1 RegexTryIR plus its body sample (abc)
* L2.2 the same, body containing a nested region sample (a(bc))
* L2.3 a single instruction region folded by prefix emission sample (a)
*
* Target shape, names owned by SourceGenContext:
*
* __rN_begin = __cur;
* __rN_ok = false;
* do
* {
* // body, a POP breaks out here
* __rN_ok = __ok;
* } while (false);
* if (!__rN_ok) { __cur = __rN_begin; }
* __ok = __rN_ok;
*
* Points the implementation has to settle:
* - the failure path restores the cursor to the region entry
* - break only leaves the nearest enclosing loop, and a FIND retry while counts as one,
* so a region containing a FIND needs the __rN_done flag described above instead of a
* plain break
* - a nested region must not rely on break to exit an outer one
* - the whole match body is itself region zero, which is what makes a top level POP
* legal
*
* =======================================================================================
* L3, fixed multi instruction windows
* =======================================================================================
*
* Consumes two to eight entries. Pure code quality work, never required for correctness.
* Every one of these must return -1 cleanly when the window does not match, so the entry
* falls through to L4.
*
* L3.1 FIND_COMPLEX FURTHEST COUNT:-1, PACK, MATCH_SEQ 3 (?<name>.*)234
* L3.2 the same with a single element anchor 3 (?<name>.*)a
* L3.3 FIND_SEQ, MATCH_SEQ, merged into one search 2 .*abc
* L3.4 MATCH_SEQ, COUNT_SEQ, the expanded a+ shape 2 a{3,5}, a+
* L3.5 a run of MATCH_SEQ and MATCH_SET, merged into one condition 2 to 8 a[abc]d
* L3.6 trailing COUNT_SET over a dot, fast success 1 to 2 abc.*
*
* Two hazards to check for each of these:
* - merging must not break the line indices an Observe_Line prefix refers to
* - a merged window still has to leave __ok describing the whole window
*/
private static int TryGenerateTryBlock(
int outer_level,
ReadOnlySpan<RegexIR> ir_span,
SourceGenContext sc,
in SequenceEmitter<string> code,
in List<SequenceEmitter<string>> local_func
)
{
if (ir_span[0]is not RegexTryIR _try) {
return -1;
}
ReadOnlySpan<RegexIR> block = ir_span[1..(1 + _try.CommandCount)];
// TODO generate here
return _try.CommandCount;
}
} }
} }
@@ -44,11 +44,16 @@ namespace StgSharp.RegularAnalysis.Text
* L4.5 MATCH_SET, any element . * L4.5 MATCH_SET, any element .
* L4.6 COUNT_SEQ a*, a? * L4.6 COUNT_SEQ a*, a?
* L4.7 COUNT_SET [abc]*, \w+, .{11,13}, [.]* * L4.7 COUNT_SET [abc]*, \w+, .{11,13}, [.]*
* L4.8 FIND_SEQ degradation path of .*abc
* L4.9 FIND_SET -
* L4.10 POP ALT tail * L4.10 POP ALT tail
* L4.11 PACK (abc), ordinal group * L4.11 PACK (abc), ordinal group
* *
* L4.8 and L4.9 were removed. FIND_SEQ and FIND_SET have no single instruction form,
* because they have to drive a candidate retry loop over the rest of the region. See L1.9
* in TextRegexSourceGen.GenClass.MultiIR.cs. Only a FIND that happens to be the last
* instruction of its region degrades into a single search, and that is a branch of L1.9
* rather than a generator of its own. The numbers are left vacant so existing references
* do not shift.
*
* Semantics worth pinning down before writing these: * Semantics worth pinning down before writing these:
* *
* - [.] and . are different sets. The former is a literal dot, the latter is any * - [.] and . are different sets. The former is a literal dot, the latter is any
@@ -56,10 +61,10 @@ namespace StgSharp.RegularAnalysis.Text
* - \w has to keep full .NET semantics. TextRegex.IsCharWord is inherited by the * - \w has to keep full .NET semantics. TextRegex.IsCharWord is inherited by the
* generated class and already carries them, so the generated code can call it * generated class and already carries them, so the generated code can call it
* directly instead of approximating with an ASCII test. * directly instead of approximating with an ASCII test.
* - This backend does not backtrack. A lazy repetition therefore consumes exactly its * - Repetition in this layer does not backtrack. A lazy repetition therefore consumes
* lower bound and leaves the rest to the following anchor. Lazy forms needing more * exactly its lower bound and leaves the rest to the following anchor. Backtracking
* are expected to have been rewritten into a find by the IR optimizer, so confirm * does exist in this backend, but only inside the FIND retry loop at L1.7 to L1.9,
* that before adding backtracking here. * never here.
* - An epsilon sequence and an empty repetition unit both reach these generators. * - An epsilon sequence and an empty repetition unit both reach these generators.
* Decide whether they are vacuous successes or an optimizer bug worth reporting. * Decide whether they are vacuous successes or an optimizer bug worth reporting.
* - PACK decides nothing, so it should leave __ok untouched. Whether it runs at all is * - PACK decides nothing, so it should leave __ok untouched. Whether it runs at all is
@@ -48,9 +48,10 @@ namespace StgSharp.RegularAnalysis.Text
* 2. The dispatch chain offers a window to L1, then L2, then L3, then L4. Higher * 2. The dispatch chain offers a window to L1, then L2, then L3, then L4. Higher
* consumption wins, so a shape spanning a whole region plus a tail is never split by * consumption wins, so a shape spanning a whole region plus a tail is never split by
* a smaller generator. * a smaller generator.
* 3. L2, L4 and L5 together are sufficient for correctness. L1 and L3 only flatten what * 3. L2, L4, L5 and the required entries of L1 together are sufficient for correctness.
* those would already emit, so an unimplemented one must return -1 cleanly rather * The flattening entries of L1 and all of L3 only improve the shape of what those
* than emit a partial result. * would already emit, so an unimplemented one must return -1 cleanly rather than
* emit a partial result.
* 4. A prefix consumes no IR. L1 to L4 describe what an instruction does, L5 decides * 4. A prefix consumes no IR. L1 to L4 describe what an instruction does, L5 decides
* when it runs and who reads the outcome. * when it runs and who reads the outcome.
* 5. Scope follows Prefix.WorkingRegion, not expression nesting. * 5. Scope follows Prefix.WorkingRegion, not expression nesting.
@@ -60,6 +61,10 @@ namespace StgSharp.RegularAnalysis.Text
* A region lowers to do { ... } while(false), and POP lowers to break. The goto plus * A region lowers to do { ... } while(false), and POP lowers to break. The goto plus
* label scheme used by the .NET generated regex backend is deliberately not adopted. * label scheme used by the .NET generated regex backend is deliberately not adopted.
* *
* A FIND lowers to a candidate retry loop nested inside the region block. That loop is
* nearer than the region do block, so a POP inside it needs the __rN_done flag described
* in TextRegexSourceGen.GenClass.MultiIR.cs rather than a plain break.
*
* Generated variable names are owned by SourceGenContext. No generator invents one. * Generated variable names are owned by SourceGenContext. No generator invents one.
* *
* The per shape checklist this backend is rebuilt against lives in * The per shape checklist this backend is rebuilt against lives in
@@ -90,7 +95,6 @@ namespace StgSharp.RegularAnalysis.Text
// TODO L0.1 Emit the Match signature from ROS_char and SourceGenContext.InputSpan. // TODO L0.1 Emit the Match signature from ROS_char and SourceGenContext.InputSpan.
// TODO L0.2 Emit region anchor and region result slots, count from RegexInfo.RegionCount. // TODO L0.2 Emit region anchor and region result slots, count from RegexInfo.RegionCount.
// TODO L0.3 Emit line result slots, only for lines an Observe_Line prefix refers to. // TODO L0.3 Emit line result slots, only for lines an Observe_Line prefix refers to.
// TODO L0.4 Emit the whole pattern guard from RegexInfo.MinPredictLength.
// TODO L0.5 Emit the cursor, the step result and any scratch declaration the context collected. // TODO L0.5 Emit the cursor, the step result and any scratch declaration the context collected.
// //
@@ -121,6 +125,7 @@ namespace StgSharp.RegularAnalysis.Text
{ {
// GenerateRegion(region, sc, match_e, sub_match_list); // GenerateRegion(region, sc, match_e, sub_match_list);
_ = TryGenerateAltChain(0, region, sc, match_e, sub_match_list); _ = TryGenerateAltChain(0, region, sc, match_e, sub_match_list);
_ = TryGenerateTryBlock(0, region, sc, match_e, sub_match_list);
if (end >= ir_sp.Length) if (end >= ir_sp.Length)
{ {
break; break;
@@ -133,6 +138,15 @@ namespace StgSharp.RegularAnalysis.Text
_ = se.AppendLine($@"public override MatchResult Match({ROS_char} text)")// _ = se.AppendLine($@"public override MatchResult Match({ROS_char} text)")//
.AppendLine(@"{"); .AppendLine(@"{");
int min_predict_length = info.MinPredictLength;
if (min_predict_length > 0) {
_ = se.AppendLine($@"if (text.Length < {min_predict_length})")
.AppendLine(@"{")
.AppendLine(@"return default;")
.AppendLine(@"}")
.AppendLine();
}
int region_count = info.RegionCount; int region_count = info.RegionCount;
for (int i = 0; i < region_count; i++) for (int i = 0; i < region_count; i++)
{ {