remove yesterday's generatoer, use new generator
This commit is contained in:
@@ -34,12 +34,14 @@
|
|||||||
|
|
||||||
### 正确性核心与纯优化的分界
|
### 正确性核心与纯优化的分界
|
||||||
|
|
||||||
- **L2 + L4 + L5 是正确性核心**:region 作用域、单指令主体、前缀装饰三者齐备,
|
- **L2 + L4 + L5 + L1 的必需项是正确性核心**:region 作用域、单指令主体、前缀装饰
|
||||||
任意表达式即可端到端生成出语义正确的代码。
|
三者支撑绝大多数形态;但 `COUNT_COMPLEX` 与 FIND 系(L1.5–L1.9)的 body 不是独立 region,
|
||||||
- **L1 与 L3 是纯优化**:它们只把 L2/L4 会生成的嵌套结构压平成更好的代码形状。
|
L2 接不住,没有降级路径,因此同样属于核心。
|
||||||
未实现时必须干净地返回 `-1`,让窗口自然降级,**不得影响正确性**。
|
- **L1 的压平项(L1.1–L1.4)与 L3 全部是纯优化**:它们只把 L2/L4 会生成的嵌套结构
|
||||||
|
压平成更好的代码形状。未实现时必须干净地返回 `-1`,让窗口自然降级,**不得影响正确性**。
|
||||||
|
|
||||||
这条分界决定了实现顺序:先把 L2/L4/L5 打通拿到可运行基线,再回头做 L1/L3。
|
这条分界决定了实现顺序:先把 L2/L4/L5 打通拿到可运行基线,再补 L1 的必需项,
|
||||||
|
最后才是 L1.1–L1.4 与 L3。
|
||||||
|
|
||||||
## 记录模板
|
## 记录模板
|
||||||
|
|
||||||
@@ -79,13 +81,13 @@ IR -> Code 规则:
|
|||||||
|
|
||||||
关注点:region 槽是否必须全量分配、行结果槽能否按 `ObserveSource` 稀疏分配。
|
关注点:region 槽是否必须全量分配、行结果槽能否按 `ObserveSource` 稀疏分配。
|
||||||
|
|
||||||
## L1 Region 组合级(纯优化)
|
## L1 Region 组合级(含必需项)
|
||||||
|
|
||||||
一次消耗**多个 region + 尾部控制指令**,是窗口消耗量最高的形态。
|
一次消耗**多个 region + 尾部控制指令**,是窗口消耗量最高的形态。
|
||||||
这些组合在 `EmitPrefix` 里已经被识别过一次
|
这些组合在 `EmitPrefix` 里已经被识别过一次
|
||||||
(`TryEmitAltChain` / `TryEmitTryGroupTail` / `TryEmitComplexIR`),生成器应与之一一对应。
|
(`TryEmitAltChain` / `TryEmitTryGroupTail` / `TryEmitComplexIR`),生成器应与之一一对应。
|
||||||
|
|
||||||
除 L1.5–L1.7 外,本层全部是**代码形状优化**:不实现时窗口会降级到 L2 + L5,
|
除 L1.5–L1.9 外,本层全部是**代码形状优化**:不实现时窗口会降级到 L2 + L5,
|
||||||
生成出语义等价但嵌套更深的代码。
|
生成出语义等价但嵌套更深的代码。
|
||||||
|
|
||||||
| # | 触发窗口 | 消耗 | 覆盖样本 | 性质 | 状态 |
|
| # | 触发窗口 | 消耗 | 覆盖样本 | 性质 | 状态 |
|
||||||
@@ -96,15 +98,84 @@ IR -> Code 规则:
|
|||||||
| L1.4 | L1.3 且 region 内部是 ALT 链 | 嵌套 | `(?<a>abcd\|a(bc)c)` | 压平 | [ ] |
|
| L1.4 | L1.3 且 region 内部是 ALT 链 | 嵌套 | `(?<a>abcd\|a(bc)c)` | 压平 | [ ] |
|
||||||
| L1.5 | `COUNT_COMPLEX` + body | `CommandCount + 1` | `(ab){2,3}` | **必需** | [ ] |
|
| L1.5 | `COUNT_COMPLEX` + body | `CommandCount + 1` | `(ab){2,3}` | **必需** | [ ] |
|
||||||
| L1.6 | `COUNT_COMPLEX` + body(含 ALT) + 线性后缀 | 嵌套 + 后缀 | `(123(4\|5)67){3,4}114514` | **必需** | [ ] |
|
| L1.6 | `COUNT_COMPLEX` + body(含 ALT) + 线性后缀 | 嵌套 + 后缀 | `(123(4\|5)67){3,4}114514` | **必需** | [ ] |
|
||||||
| L1.7 | `FIND_COMPLEX` + body | `CommandCount + 1` | `.*(ab\|cd)` | **必需** | [ ] |
|
| L1.7 | `FIND_COMPLEX`,`CommandCount ≥ 0` | `CommandCount + 1` | `.*(ab\|cd)` | **必需** | [ ] |
|
||||||
|
| L1.8 | `FIND_COMPLEX`,`CommandCount = -1` | 1 + region 剩余 | `(?<name>.*)234` | **必需** | [ ] |
|
||||||
|
| L1.9 | `FIND_SEQ` / `FIND_SET` | 1 + region 剩余 | `.*abc` | **必需** | [ ] |
|
||||||
|
|
||||||
期望代码形状:L1.1/L1.2 压平为 `if / else if / else` 链;L1.3/L1.4 在 region 成功分支写捕获区间;
|
期望代码形状:L1.1/L1.2 压平为 `if / else if / else` 链;L1.3/L1.4 在 region 成功分支写捕获区间;
|
||||||
L1.5–L1.7 为循环 / 搜索包裹 body,`CommandCount` 必须按前缀化后的实际条数重算。
|
L1.5/L1.6 为计数循环包裹 body;L1.7–L1.9 为**候选重试循环**包裹 body(见下)。
|
||||||
|
`CommandCount` 必须按前缀化后的实际条数重算。
|
||||||
|
|
||||||
L1.5–L1.7 标记为必需,是因为 `COUNT_COMPLEX` / `FIND_COMPLEX` 的 body 不是独立 region,
|
L1.5–L1.9 标记为必需,是因为它们的 body 都不是独立 region,L2 接不住,没有替代降级路径。
|
||||||
L2 接不住它们,没有替代降级路径。
|
|
||||||
|
|
||||||
关注点:ALT 各 case 之间活动指针如何恢复、`PACK` 取哪一个成功结果作为捕获区间。
|
### FIND 系的回溯语义
|
||||||
|
|
||||||
|
FIND 产出的不是一个位置,是一个**按优先级排序的候选位置序列**:
|
||||||
|
|
||||||
|
- `FURTHEST`(贪婪 `.*`):从最远候选起,验证失败往**前**退
|
||||||
|
- `NEAREST`(惰性 `.*?`):从最近候选起,验证失败往**后**进
|
||||||
|
|
||||||
|
body 是候选的**验证器**。验证失败必须取下一个候选重来,候选耗尽才是真失败。
|
||||||
|
只做一次 `LastIndexOf` 就判定成败是错的——`.*abcX` 里最后一个 `abc` 后面没有 `X` 时,
|
||||||
|
必须退到倒数第二个 `abc`。
|
||||||
|
|
||||||
|
`CommandCount = -1` 的语义已被优化器坐实:`TryReplaceDotStar` 把 `.*` 直接产出
|
||||||
|
`FindComplex(-1)`,随后 `TyrMergeFindRestTwice` / `TyrMergeFindRestTry` 尽力把它收敛成确定长度,
|
||||||
|
收敛不掉的保持 -1,表示「body 是 region 剩余全部」。所以后端只需处理两种 body 长度:
|
||||||
|
确定值,和 -1。
|
||||||
|
|
||||||
|
`FIND_SEQ` / `FIND_SET` 没有 `CommandCount` 字段,语义等同 -1。只有当它恰好是 region
|
||||||
|
最后一条时才退化成单次搜索,此时消耗为 1。
|
||||||
|
|
||||||
|
期望代码形状:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
int __bound = text.Length;
|
||||||
|
__ok = false;
|
||||||
|
while (true)
|
||||||
|
{
|
||||||
|
int __at = text[__cur..__bound].LastIndexOf("abc");
|
||||||
|
if (__at < 0) { __ok = false; break; } // 候选耗尽,唯一失败出口
|
||||||
|
int __save = __cur;
|
||||||
|
__cur = __cur + __at + 3;
|
||||||
|
|
||||||
|
// ── body / region 剩余指令内联展开在这里 ──
|
||||||
|
|
||||||
|
if (__ok) { break; } // 验证通过
|
||||||
|
__cur = __save;
|
||||||
|
__bound = __save + __at; // 退到前一个候选
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### FIND 与 `POP` 的冲突(必须处理)
|
||||||
|
|
||||||
|
`POP` 编译成 `break` 时跳出的是**最近**一层循环,而 FIND 的 `while` 比 region 的
|
||||||
|
`do{}while(false)` 更近。body 里的 `POP` 会跳出重试循环而不是 region,语义直接错。
|
||||||
|
不引入 `goto` 的解法是加一个 region 终结标志:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
do // region N
|
||||||
|
{
|
||||||
|
bool __rN_done = false;
|
||||||
|
...
|
||||||
|
while (!__rN_done) // find retry
|
||||||
|
{
|
||||||
|
// 候选搜索
|
||||||
|
do // ← 隔离层,POP 的 break 落在这
|
||||||
|
{
|
||||||
|
// body,POP: __rN_ok = x; __rN_done = true; break;
|
||||||
|
} while (false);
|
||||||
|
|
||||||
|
if (__rN_done || __ok) { break; }
|
||||||
|
// 退候选
|
||||||
|
}
|
||||||
|
} while (false);
|
||||||
|
```
|
||||||
|
|
||||||
|
关注点:ALT 各 case 之间活动指针如何恢复、`PACK` 取哪一个成功结果作为捕获区间、
|
||||||
|
嵌套 FIND 会产生嵌套重试循环(复杂度是候选数的乘积,是灾难性回溯的来源)、
|
||||||
|
重试循环内 `__ok` 跨迭代被复写(`Observe_Line` 指向循环体内某条指令时,
|
||||||
|
读到的是最后一次迭代的值,当前前缀模型没有覆盖这个情形)。
|
||||||
|
|
||||||
## L2 Region 级(正确性核心)
|
## L2 Region 级(正确性核心)
|
||||||
|
|
||||||
@@ -133,7 +204,11 @@ __ok = __rN_ok;
|
|||||||
|
|
||||||
失败路径必须把活动指针复位到 region 入口。`POP` 直接编译为 `break`,不引入标签。
|
失败路径必须把活动指针复位到 region 入口。`POP` 直接编译为 `break`,不引入标签。
|
||||||
|
|
||||||
关注点:region 全部内联展开、嵌套 region 的指针复位链、`break` 只能跳出最近一层 `do`。
|
**例外**:当 region 内出现 FIND 系指令时,`break` 会被 FIND 的重试循环截获,
|
||||||
|
必须改用 L1 节里的 `__rN_done` 终结标志方案。
|
||||||
|
|
||||||
|
关注点:region 全部内联展开、嵌套 region 的指针复位链、`break` 只能跳出最近一层循环
|
||||||
|
(`do{}while(false)` 和 FIND 的 `while` 都算)。
|
||||||
|
|
||||||
## L3 多指令定式(纯优化)
|
## L3 多指令定式(纯优化)
|
||||||
|
|
||||||
@@ -144,12 +219,13 @@ __ok = __rN_ok;
|
|||||||
| --- | --- | --- | --- | --- |
|
| --- | --- | --- | --- | --- |
|
||||||
| L3.1 | `FIND_COMPLEX FURTHEST COUNT:-1` + `PACK` + `MATCH_SEQ` | 3 | `(?<name>.*)234` | [ ] |
|
| L3.1 | `FIND_COMPLEX FURTHEST COUNT:-1` + `PACK` + `MATCH_SEQ` | 3 | `(?<name>.*)234` | [ ] |
|
||||||
| L3.2 | 同上,锚点为单字符 | 3 | `(?<name>.*)a` | [ ] |
|
| L3.2 | 同上,锚点为单字符 | 3 | `(?<name>.*)a` | [ ] |
|
||||||
| L3.3 | `FIND_SEQ` + `MATCH_SEQ`,合并为一次 `IndexOf` | 2 | `.*abc` | [ ] |
|
| L3.3 | `FIND_SEQ` + `MATCH_SEQ`,把验证器折进搜索模式,消除回溯 | 2 | `.*abc` | [ ] |
|
||||||
| L3.4 | `MATCH_SEQ` + `COUNT_SEQ`(`a+`、`a{m,n}` 展开形态) | 2 | `a{3,5}`、`a+` | [ ] |
|
| L3.4 | `MATCH_SEQ` + `COUNT_SEQ`(`a+`、`a{m,n}` 展开形态) | 2 | `a{3,5}`、`a+` | [ ] |
|
||||||
| L3.5 | 连续 `MATCH_SEQ` / `MATCH_SET`,合流为单个 `if` 与表达式 | 2–8 | `a[abc]d` | [ ] |
|
| L3.5 | 连续 `MATCH_SEQ` / `MATCH_SET`,合流为单个 `if` 与表达式 | 2–8 | `a[abc]d` | [ ] |
|
||||||
| L3.6 | 尾部 `COUNT_SET .*`,快速成功 | 1–2 | `abc.*` | [ ] |
|
| L3.6 | 尾部 `COUNT_SET .*`,快速成功 | 1–2 | `abc.*` | [ ] |
|
||||||
|
|
||||||
关注点:合并后失败诊断是否还能定位到原指令、合并是否破坏 `Observe_Line` 的行号引用。
|
关注点:合并后失败诊断是否还能定位到原指令、合并是否破坏 `Observe_Line` 的行号引用。
|
||||||
|
L3.3 只有在合并后 region 剩余为空时才真正消除回溯,否则 L1.9 的重试循环仍然要保留。
|
||||||
|
|
||||||
## L4 单指令级
|
## L4 单指令级
|
||||||
|
|
||||||
@@ -164,11 +240,13 @@ __ok = __rN_ok;
|
|||||||
| L4.5 | `MATCH_SET` 任意字符 | `.` | [ ] |
|
| L4.5 | `MATCH_SET` 任意字符 | `.` | [ ] |
|
||||||
| L4.6 | `COUNT_SEQ` | `a*`、`a?` | [ ] |
|
| L4.6 | `COUNT_SEQ` | `a*`、`a?` | [ ] |
|
||||||
| L4.7 | `COUNT_SET` | `[abc]*`、`\w+`、`.{11,13}`、`[.]*` | [ ] |
|
| L4.7 | `COUNT_SET` | `[abc]*`、`\w+`、`.{11,13}`、`[.]*` | [ ] |
|
||||||
| L4.8 | `FIND_SEQ` | `.*abc` 降级路径 | [ ] |
|
|
||||||
| L4.9 | `FIND_SET` | — | [ ] |
|
|
||||||
| L4.10 | `POP` | ALT 尾 | [ ] |
|
| L4.10 | `POP` | ALT 尾 | [ ] |
|
||||||
| L4.11 | `PACK` | `(abc)` 默认序号组 | [ ] |
|
| L4.11 | `PACK` | `(abc)` 默认序号组 | [ ] |
|
||||||
|
|
||||||
|
L4.8 / L4.9 已移出本层。`FIND_SEQ` / `FIND_SET` **不存在单条形态**——它们必须联动 region
|
||||||
|
剩余部分做候选重试,见 L1.9。只有 FIND 恰好是 region 末条时才退化成一次搜索,
|
||||||
|
那是 L1.9 的退化分支,不是独立的单指令生成器。编号保留空缺,避免打乱既有引用。
|
||||||
|
|
||||||
关注点:`[.]` 与 `.` 必须走不同后端语义;`[.]*` 不得被当作 `.*` 优化。
|
关注点:`[.]` 与 `.` 必须走不同后端语义;`[.]*` 不得被当作 `.*` 优化。
|
||||||
|
|
||||||
## L5 前缀装饰(横切,消耗 0)
|
## L5 前缀装饰(横切,消耗 0)
|
||||||
@@ -194,7 +272,8 @@ __ok = __rN_ok;
|
|||||||
|
|
||||||
## 实现顺序
|
## 实现顺序
|
||||||
|
|
||||||
调度链按 L1 → L4 排列,但**实现**先拿正确性核心(L2/L4/L5),再回头填纯优化层(L1/L3):
|
调度链按 L1 → L4 排列,但**实现**先拿正确性核心(L2/L4/L5),再补 L1 的必需项,
|
||||||
|
最后才是压平层(L1.1–L1.4 / L3):
|
||||||
|
|
||||||
1. **L0** — 生成的空骨架能通过编译,`TextRegexSourceGen.GenFile` 的两处 TODO 闭合。
|
1. **L0** — 生成的空骨架能通过编译,`TextRegexSourceGen.GenFile` 的两处 TODO 闭合。
|
||||||
2. **L2.1** — `(abc)` 产出正确的 `do{}while(false)` 作用域与复位逻辑。
|
2. **L2.1** — `(abc)` 产出正确的 `do{}while(false)` 作用域与复位逻辑。
|
||||||
@@ -202,12 +281,18 @@ __ok = __rN_ok;
|
|||||||
4. **L4 全部** — 兜底层补齐,任意线性表达式可端到端生成。
|
4. **L4 全部** — 兜底层补齐,任意线性表达式可端到端生成。
|
||||||
5. **L5.6–L5.9** — 跨行 / 跨 region 观察与提前退出,`a|b` 与 `(?<name>abc)` 自然跑通(嵌套形态)。
|
5. **L5.6–L5.9** — 跨行 / 跨 region 观察与提前退出,`a|b` 与 `(?<name>abc)` 自然跑通(嵌套形态)。
|
||||||
**此时获得可运行基线,任何表达式都能生成正确代码。**
|
**此时获得可运行基线,任何表达式都能生成正确代码。**
|
||||||
6. **L1.5、L1.7** — 复杂计数与复杂搜索,无降级路径,必须实现。
|
6. **L1.5、L1.7–L1.9** — 复杂计数与 FIND 系,无降级路径,必须实现。
|
||||||
|
FIND 系要连带处理 `POP` 与重试循环的 `break` 冲突。
|
||||||
7. **L1.1–L1.4** — 把 ALT 链与捕获组的嵌套形态压平。
|
7. **L1.1–L1.4** — 把 ALT 链与捕获组的嵌套形态压平。
|
||||||
8. **L3 全部** — 代码质量优化,可持续追加。
|
8. **L3 全部** — 代码质量优化,可持续追加。
|
||||||
|
|
||||||
第一阶段验收口径(沿用原优先级):`MATCH_SEQ`、`MATCH_SET`、`COUNT_SEQ`、`COUNT_SET`、
|
第一阶段验收口径(沿用原优先级):`MATCH_SEQ`、`MATCH_SET`、`COUNT_SEQ`、`COUNT_SET`、
|
||||||
简单 `ALT`、`PACK`、`FIND_SEQ`、`FIND_COMPLEX COUNT:-1` 全部稳定。
|
简单 `ALT`、`PACK`、`FIND_COMPLEX COUNT:-1` 全部稳定。
|
||||||
|
|
||||||
|
注:`FIND_SEQ` 已从原口径移除。`TryReplaceDotStar` 产出的是 `FindComplex(-1)`,
|
||||||
|
IROptimize 里没有 `RegexFindSeqIR` 的生成点,`ngra-regex-ir.md` 里写的
|
||||||
|
`.*a → FIND_SEQ(a, last)` 尚未落地。在生成点补上之前,L1.9 的 `FIND_SEQ` / `FIND_SET`
|
||||||
|
分支实际不可达,L3.3 也无从触发。
|
||||||
|
|
||||||
低优先级,暂不纳入 checklist:
|
低优先级,暂不纳入 checklist:
|
||||||
|
|
||||||
|
|||||||
@@ -14,109 +14,6 @@ namespace StgSharp.RegularAnalysis.Text
|
|||||||
{
|
{
|
||||||
internal partial class TextRegexSourceGen
|
internal partial class TextRegexSourceGen
|
||||||
{
|
{
|
||||||
/*
|
|
||||||
* =======================================================================================
|
|
||||||
* L1, several regions plus a tail
|
|
||||||
* =======================================================================================
|
|
||||||
*
|
|
||||||
* Highest consumption in the dispatch chain, so these are offered the window first. Each
|
|
||||||
* shape here was already recognised once during prefix emission, so a generator should
|
|
||||||
* mirror the matching RegexAnalyzer.PrefixEmit recogniser rather than re-derive the shape.
|
|
||||||
*
|
|
||||||
* L1.2 ALT chain with a shared suffix
|
|
||||||
* window L1.1 followed by linear instructions in the parent region
|
|
||||||
* sample (ab|[cd]|.)x
|
|
||||||
* nature flattening, may return -1
|
|
||||||
*
|
|
||||||
* L1.3 group tail
|
|
||||||
* window region, POP FAIL with Prev_Fail, PACK SUCCESS with Prev_Success
|
|
||||||
* mirrors TryEmitTryGroupTail
|
|
||||||
* shape region block, capture written on the success branch
|
|
||||||
* sample (?<name>abc)
|
|
||||||
* nature flattening, may return -1
|
|
||||||
*
|
|
||||||
* L1.4 group tail wrapping an ALT chain
|
|
||||||
* sample (?<a>abcd|a(bc)c)
|
|
||||||
* nature flattening, may return -1
|
|
||||||
*
|
|
||||||
* L1.5 COUNT_COMPLEX plus body
|
|
||||||
* window RegexCountComplexIR, then CommandCount entries
|
|
||||||
* mirrors TryEmitComplexIR
|
|
||||||
* shape loop wrapping the generated body
|
|
||||||
* sample (ab){2,3}
|
|
||||||
* nature REQUIRED, no fallback exists
|
|
||||||
*
|
|
||||||
* L1.6 COUNT_COMPLEX whose body contains an ALT chain, plus a linear suffix
|
|
||||||
* sample (123(4|5)67){3,4}114514
|
|
||||||
* nature REQUIRED, no fallback exists
|
|
||||||
*
|
|
||||||
* L1.7 FIND_COMPLEX plus body
|
|
||||||
* window RegexFindComplexIR, then CommandCount entries
|
|
||||||
* shape search loop wrapping the generated body
|
|
||||||
* sample .*(ab|cd)
|
|
||||||
* nature REQUIRED, no fallback exists
|
|
||||||
*
|
|
||||||
* L1.5 to L1.7 are required because a complex body is not a region of its own. L2 cannot
|
|
||||||
* pick it up, so there is no degradation path if these are missing.
|
|
||||||
*
|
|
||||||
* CommandCount must be read from the prefixed IR. Prefix emission rebuilds these counts,
|
|
||||||
* so the value carried by the pre-prefix IR no longer applies.
|
|
||||||
*
|
|
||||||
* Open questions to settle while implementing:
|
|
||||||
* - how the cursor is restored between ALT cases
|
|
||||||
* - which successful outcome a PACK uses as its capture span
|
|
||||||
* - whether a region body ever deserves its own generated method instead of inlining
|
|
||||||
*
|
|
||||||
* =======================================================================================
|
|
||||||
* L2, one region
|
|
||||||
* =======================================================================================
|
|
||||||
*
|
|
||||||
* Consumes CommandCount + 1, the whole block a TRY lowers to. Together with L4 and L5 this
|
|
||||||
* is the correctness core: once these three exist, every expression can be generated, just
|
|
||||||
* with deeper nesting than L1 and L3 would produce.
|
|
||||||
*
|
|
||||||
* L2.1 RegexTryIR plus its body sample (abc)
|
|
||||||
* L2.2 the same, body containing a nested region sample (a(bc))
|
|
||||||
* L2.3 a single instruction region folded by prefix emission sample (a)
|
|
||||||
*
|
|
||||||
* Target shape, names owned by SourceGenContext:
|
|
||||||
*
|
|
||||||
* __rN_begin = __cur;
|
|
||||||
* __rN_ok = false;
|
|
||||||
* do
|
|
||||||
* {
|
|
||||||
* // body, a POP breaks out here
|
|
||||||
* __rN_ok = __ok;
|
|
||||||
* } while (false);
|
|
||||||
* if (!__rN_ok) { __cur = __rN_begin; }
|
|
||||||
* __ok = __rN_ok;
|
|
||||||
*
|
|
||||||
* Points the implementation has to settle:
|
|
||||||
* - the failure path restores the cursor to the region entry
|
|
||||||
* - break only leaves the nearest enclosing do block, so a nested region must not rely
|
|
||||||
* on it to exit an outer one
|
|
||||||
* - the whole match body is itself region zero, which is what makes a top level POP
|
|
||||||
* legal
|
|
||||||
*
|
|
||||||
* =======================================================================================
|
|
||||||
* L3, fixed multi instruction windows
|
|
||||||
* =======================================================================================
|
|
||||||
*
|
|
||||||
* Consumes two to eight entries. Pure code quality work, never required for correctness.
|
|
||||||
* Every one of these must return -1 cleanly when the window does not match, so the entry
|
|
||||||
* falls through to L4.
|
|
||||||
*
|
|
||||||
* L3.1 FIND_COMPLEX FURTHEST COUNT:-1, PACK, MATCH_SEQ 3 (?<name>.*)234
|
|
||||||
* L3.2 the same with a single element anchor 3 (?<name>.*)a
|
|
||||||
* L3.3 FIND_SEQ, MATCH_SEQ, merged into one search 2 .*abc
|
|
||||||
* L3.4 MATCH_SEQ, COUNT_SEQ, the expanded a+ shape 2 a{3,5}, a+
|
|
||||||
* L3.5 a run of MATCH_SEQ and MATCH_SET, merged into one condition 2 to 8 a[abc]d
|
|
||||||
* L3.6 trailing COUNT_SET over a dot, fast success 1 to 2 abc.*
|
|
||||||
*
|
|
||||||
* Two hazards to check for each of these:
|
|
||||||
* - merging must not break the line indices an Observe_Line prefix refers to
|
|
||||||
* - a merged window still has to leave __ok describing the whole window
|
|
||||||
*/
|
|
||||||
|
|
||||||
private static int TryGenerateAltChain(
|
private static int TryGenerateAltChain(
|
||||||
int outer_level,
|
int outer_level,
|
||||||
@@ -165,6 +62,201 @@ namespace StgSharp.RegularAnalysis.Text
|
|||||||
}
|
}
|
||||||
return i;
|
return i;
|
||||||
}
|
}
|
||||||
|
/*
|
||||||
|
* =======================================================================================
|
||||||
|
* L1, several regions plus a tail
|
||||||
|
* =======================================================================================
|
||||||
|
*
|
||||||
|
* Highest consumption in the dispatch chain, so these are offered the window first. Each
|
||||||
|
* shape here was already recognised once during prefix emission, so a generator should
|
||||||
|
* mirror the matching RegexAnalyzer.PrefixEmit recogniser rather than re-derive the shape.
|
||||||
|
*
|
||||||
|
* L1.2 ALT chain with a shared suffix
|
||||||
|
* window L1.1 followed by linear instructions in the parent region
|
||||||
|
* sample (ab|[cd]|.)x
|
||||||
|
* nature flattening, may return -1
|
||||||
|
*
|
||||||
|
* L1.3 group tail
|
||||||
|
* window region, POP FAIL with Prev_Fail, PACK SUCCESS with Prev_Success
|
||||||
|
* mirrors TryEmitTryGroupTail
|
||||||
|
* shape region block, capture written on the success branch
|
||||||
|
* sample (?<name>abc)
|
||||||
|
* nature flattening, may return -1
|
||||||
|
*
|
||||||
|
* L1.4 group tail wrapping an ALT chain
|
||||||
|
* sample (?<a>abcd|a(bc)c)
|
||||||
|
* nature flattening, may return -1
|
||||||
|
*
|
||||||
|
* L1.5 COUNT_COMPLEX plus body
|
||||||
|
* window RegexCountComplexIR, then CommandCount entries
|
||||||
|
* mirrors TryEmitComplexIR
|
||||||
|
* shape loop wrapping the generated body
|
||||||
|
* sample (ab){2,3}
|
||||||
|
* nature REQUIRED, no fallback exists
|
||||||
|
*
|
||||||
|
* L1.6 COUNT_COMPLEX whose body contains an ALT chain, plus a linear suffix
|
||||||
|
* sample (123(4|5)67){3,4}114514
|
||||||
|
* nature REQUIRED, no fallback exists
|
||||||
|
*
|
||||||
|
* L1.7 FIND_COMPLEX with CommandCount >= 0
|
||||||
|
* window RegexFindComplexIR, then CommandCount entries
|
||||||
|
* shape candidate retry loop wrapping the body
|
||||||
|
* sample .*(ab|cd)
|
||||||
|
* nature REQUIRED, no fallback exists
|
||||||
|
*
|
||||||
|
* L1.8 FIND_COMPLEX with CommandCount == -1
|
||||||
|
* window RegexFindComplexIR, then the rest of the region
|
||||||
|
* shape candidate retry loop wrapping the rest of the region
|
||||||
|
* sample (?<name>.*)234
|
||||||
|
* nature REQUIRED, no fallback exists
|
||||||
|
*
|
||||||
|
* L1.9 FIND_SEQ or FIND_SET
|
||||||
|
* window the instruction, then the rest of the region
|
||||||
|
* shape candidate retry loop wrapping the rest of the region
|
||||||
|
* sample .*abc
|
||||||
|
* nature REQUIRED, no fallback exists
|
||||||
|
*
|
||||||
|
* L1.5 to L1.9 are required because none of their bodies is a region of its own. L2
|
||||||
|
* cannot pick them up, so there is no degradation path if these are missing.
|
||||||
|
*
|
||||||
|
* CommandCount must be read from the prefixed IR. Prefix emission rebuilds these counts,
|
||||||
|
* so the value carried by the pre-prefix IR no longer applies.
|
||||||
|
*
|
||||||
|
* ---------------------------------------------------------------------------------------
|
||||||
|
* Backtracking semantics of the FIND family
|
||||||
|
* ---------------------------------------------------------------------------------------
|
||||||
|
*
|
||||||
|
* A FIND does not yield one position, it yields a priority ordered sequence of candidates.
|
||||||
|
* FURTHEST starts at the last one and steps backwards on failure, NEAREST starts at the
|
||||||
|
* first and steps forwards. The body is the verifier for a candidate: when it fails, the
|
||||||
|
* next candidate has to be tried, and only an exhausted candidate list is a real failure.
|
||||||
|
* A single LastIndexOf is wrong, because .*abcX has to fall back to the second to last abc
|
||||||
|
* when the last one is not followed by an X.
|
||||||
|
*
|
||||||
|
* CommandCount == -1 means the body is the rest of the region. TryReplaceDotStar emits
|
||||||
|
* FindComplex(-1) for a bare .*, then TyrMergeFindRestTwice and TyrMergeFindRestTry try to
|
||||||
|
* collapse it into a definite length. Whatever survives stays at -1. FIND_SEQ and FIND_SET
|
||||||
|
* carry no CommandCount at all and behave like -1. Only a FIND that happens to be the last
|
||||||
|
* instruction of its region degrades into a single search, consuming one entry.
|
||||||
|
*
|
||||||
|
* Target shape:
|
||||||
|
*
|
||||||
|
* int __bound = text.Length;
|
||||||
|
* __ok = false;
|
||||||
|
* while (true)
|
||||||
|
* {
|
||||||
|
* int __at = text[__cur..__bound].LastIndexOf("abc");
|
||||||
|
* if (__at < 0) { __ok = false; break; } // candidates exhausted, only failure
|
||||||
|
* int __save = __cur;
|
||||||
|
* __cur = __cur + __at + 3;
|
||||||
|
*
|
||||||
|
* // body, or the rest of the region, expands here
|
||||||
|
*
|
||||||
|
* if (__ok) { break; } // verified
|
||||||
|
* __cur = __save;
|
||||||
|
* __bound = __save + __at; // step to the previous candidate
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* POP conflicts with the retry loop. A POP lowers to break, and the retry while is nearer
|
||||||
|
* than the region do block, so the break would leave the wrong scope. The fix without goto
|
||||||
|
* is a region completion flag:
|
||||||
|
*
|
||||||
|
* do // region N
|
||||||
|
* {
|
||||||
|
* bool __rN_done = false;
|
||||||
|
* while (!__rN_done) // find retry
|
||||||
|
* {
|
||||||
|
* // candidate search
|
||||||
|
* do // isolation layer, POP breaks out here
|
||||||
|
* {
|
||||||
|
* // body, POP sets __rN_ok, sets __rN_done, breaks
|
||||||
|
* } while (false);
|
||||||
|
* if (__rN_done || __ok) { break; }
|
||||||
|
* // step to the next candidate
|
||||||
|
* }
|
||||||
|
* } while (false);
|
||||||
|
*
|
||||||
|
* Two further hazards:
|
||||||
|
* - nested FIND produces nested retry loops and the cost multiplies, which is the
|
||||||
|
* catastrophic backtracking shape, worth a guard or at least a marker
|
||||||
|
* - __ok is overwritten across retry iterations, so an Observe_Line prefix pointing into
|
||||||
|
* a retry body reads the value of the last iteration, a case the current prefix model
|
||||||
|
* does not cover
|
||||||
|
*
|
||||||
|
* Open questions to settle while implementing:
|
||||||
|
* - how the cursor is restored between ALT cases
|
||||||
|
* - which successful outcome a PACK uses as its capture span
|
||||||
|
* - whether a region body ever deserves its own generated method instead of inlining
|
||||||
|
*
|
||||||
|
* =======================================================================================
|
||||||
|
* L2, one region
|
||||||
|
* =======================================================================================
|
||||||
|
*
|
||||||
|
* Consumes CommandCount + 1, the whole block a TRY lowers to. Together with L4 and L5 this
|
||||||
|
* is the correctness core: once these three exist, every expression can be generated, just
|
||||||
|
* with deeper nesting than L1 and L3 would produce.
|
||||||
|
*
|
||||||
|
* L2.1 RegexTryIR plus its body sample (abc)
|
||||||
|
* L2.2 the same, body containing a nested region sample (a(bc))
|
||||||
|
* L2.3 a single instruction region folded by prefix emission sample (a)
|
||||||
|
*
|
||||||
|
* Target shape, names owned by SourceGenContext:
|
||||||
|
*
|
||||||
|
* __rN_begin = __cur;
|
||||||
|
* __rN_ok = false;
|
||||||
|
* do
|
||||||
|
* {
|
||||||
|
* // body, a POP breaks out here
|
||||||
|
* __rN_ok = __ok;
|
||||||
|
* } while (false);
|
||||||
|
* if (!__rN_ok) { __cur = __rN_begin; }
|
||||||
|
* __ok = __rN_ok;
|
||||||
|
*
|
||||||
|
* Points the implementation has to settle:
|
||||||
|
* - the failure path restores the cursor to the region entry
|
||||||
|
* - break only leaves the nearest enclosing loop, and a FIND retry while counts as one,
|
||||||
|
* so a region containing a FIND needs the __rN_done flag described above instead of a
|
||||||
|
* plain break
|
||||||
|
* - a nested region must not rely on break to exit an outer one
|
||||||
|
* - the whole match body is itself region zero, which is what makes a top level POP
|
||||||
|
* legal
|
||||||
|
*
|
||||||
|
* =======================================================================================
|
||||||
|
* L3, fixed multi instruction windows
|
||||||
|
* =======================================================================================
|
||||||
|
*
|
||||||
|
* Consumes two to eight entries. Pure code quality work, never required for correctness.
|
||||||
|
* Every one of these must return -1 cleanly when the window does not match, so the entry
|
||||||
|
* falls through to L4.
|
||||||
|
*
|
||||||
|
* L3.1 FIND_COMPLEX FURTHEST COUNT:-1, PACK, MATCH_SEQ 3 (?<name>.*)234
|
||||||
|
* L3.2 the same with a single element anchor 3 (?<name>.*)a
|
||||||
|
* L3.3 FIND_SEQ, MATCH_SEQ, merged into one search 2 .*abc
|
||||||
|
* L3.4 MATCH_SEQ, COUNT_SEQ, the expanded a+ shape 2 a{3,5}, a+
|
||||||
|
* L3.5 a run of MATCH_SEQ and MATCH_SET, merged into one condition 2 to 8 a[abc]d
|
||||||
|
* L3.6 trailing COUNT_SET over a dot, fast success 1 to 2 abc.*
|
||||||
|
*
|
||||||
|
* Two hazards to check for each of these:
|
||||||
|
* - merging must not break the line indices an Observe_Line prefix refers to
|
||||||
|
* - a merged window still has to leave __ok describing the whole window
|
||||||
|
*/
|
||||||
|
|
||||||
|
private static int TryGenerateTryBlock(
|
||||||
|
int outer_level,
|
||||||
|
ReadOnlySpan<RegexIR> ir_span,
|
||||||
|
SourceGenContext sc,
|
||||||
|
in SequenceEmitter<string> code,
|
||||||
|
in List<SequenceEmitter<string>> local_func
|
||||||
|
)
|
||||||
|
{
|
||||||
|
if (ir_span[0]is not RegexTryIR _try) {
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
ReadOnlySpan<RegexIR> block = ir_span[1..(1 + _try.CommandCount)];
|
||||||
|
|
||||||
|
// TODO generate here
|
||||||
|
return _try.CommandCount;
|
||||||
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -44,11 +44,16 @@ namespace StgSharp.RegularAnalysis.Text
|
|||||||
* L4.5 MATCH_SET, any element .
|
* L4.5 MATCH_SET, any element .
|
||||||
* L4.6 COUNT_SEQ a*, a?
|
* L4.6 COUNT_SEQ a*, a?
|
||||||
* L4.7 COUNT_SET [abc]*, \w+, .{11,13}, [.]*
|
* L4.7 COUNT_SET [abc]*, \w+, .{11,13}, [.]*
|
||||||
* L4.8 FIND_SEQ degradation path of .*abc
|
|
||||||
* L4.9 FIND_SET -
|
|
||||||
* L4.10 POP ALT tail
|
* L4.10 POP ALT tail
|
||||||
* L4.11 PACK (abc), ordinal group
|
* L4.11 PACK (abc), ordinal group
|
||||||
*
|
*
|
||||||
|
* L4.8 and L4.9 were removed. FIND_SEQ and FIND_SET have no single instruction form,
|
||||||
|
* because they have to drive a candidate retry loop over the rest of the region. See L1.9
|
||||||
|
* in TextRegexSourceGen.GenClass.MultiIR.cs. Only a FIND that happens to be the last
|
||||||
|
* instruction of its region degrades into a single search, and that is a branch of L1.9
|
||||||
|
* rather than a generator of its own. The numbers are left vacant so existing references
|
||||||
|
* do not shift.
|
||||||
|
*
|
||||||
* Semantics worth pinning down before writing these:
|
* Semantics worth pinning down before writing these:
|
||||||
*
|
*
|
||||||
* - [.] and . are different sets. The former is a literal dot, the latter is any
|
* - [.] and . are different sets. The former is a literal dot, the latter is any
|
||||||
@@ -56,10 +61,10 @@ namespace StgSharp.RegularAnalysis.Text
|
|||||||
* - \w has to keep full .NET semantics. TextRegex.IsCharWord is inherited by the
|
* - \w has to keep full .NET semantics. TextRegex.IsCharWord is inherited by the
|
||||||
* generated class and already carries them, so the generated code can call it
|
* generated class and already carries them, so the generated code can call it
|
||||||
* directly instead of approximating with an ASCII test.
|
* directly instead of approximating with an ASCII test.
|
||||||
* - This backend does not backtrack. A lazy repetition therefore consumes exactly its
|
* - Repetition in this layer does not backtrack. A lazy repetition therefore consumes
|
||||||
* lower bound and leaves the rest to the following anchor. Lazy forms needing more
|
* exactly its lower bound and leaves the rest to the following anchor. Backtracking
|
||||||
* are expected to have been rewritten into a find by the IR optimizer, so confirm
|
* does exist in this backend, but only inside the FIND retry loop at L1.7 to L1.9,
|
||||||
* that before adding backtracking here.
|
* never here.
|
||||||
* - An epsilon sequence and an empty repetition unit both reach these generators.
|
* - An epsilon sequence and an empty repetition unit both reach these generators.
|
||||||
* Decide whether they are vacuous successes or an optimizer bug worth reporting.
|
* Decide whether they are vacuous successes or an optimizer bug worth reporting.
|
||||||
* - PACK decides nothing, so it should leave __ok untouched. Whether it runs at all is
|
* - PACK decides nothing, so it should leave __ok untouched. Whether it runs at all is
|
||||||
|
|||||||
@@ -48,9 +48,10 @@ namespace StgSharp.RegularAnalysis.Text
|
|||||||
* 2. The dispatch chain offers a window to L1, then L2, then L3, then L4. Higher
|
* 2. The dispatch chain offers a window to L1, then L2, then L3, then L4. Higher
|
||||||
* consumption wins, so a shape spanning a whole region plus a tail is never split by
|
* consumption wins, so a shape spanning a whole region plus a tail is never split by
|
||||||
* a smaller generator.
|
* a smaller generator.
|
||||||
* 3. L2, L4 and L5 together are sufficient for correctness. L1 and L3 only flatten what
|
* 3. L2, L4, L5 and the required entries of L1 together are sufficient for correctness.
|
||||||
* those would already emit, so an unimplemented one must return -1 cleanly rather
|
* The flattening entries of L1 and all of L3 only improve the shape of what those
|
||||||
* than emit a partial result.
|
* would already emit, so an unimplemented one must return -1 cleanly rather than
|
||||||
|
* emit a partial result.
|
||||||
* 4. A prefix consumes no IR. L1 to L4 describe what an instruction does, L5 decides
|
* 4. A prefix consumes no IR. L1 to L4 describe what an instruction does, L5 decides
|
||||||
* when it runs and who reads the outcome.
|
* when it runs and who reads the outcome.
|
||||||
* 5. Scope follows Prefix.WorkingRegion, not expression nesting.
|
* 5. Scope follows Prefix.WorkingRegion, not expression nesting.
|
||||||
@@ -60,6 +61,10 @@ namespace StgSharp.RegularAnalysis.Text
|
|||||||
* A region lowers to do { ... } while(false), and POP lowers to break. The goto plus
|
* A region lowers to do { ... } while(false), and POP lowers to break. The goto plus
|
||||||
* label scheme used by the .NET generated regex backend is deliberately not adopted.
|
* label scheme used by the .NET generated regex backend is deliberately not adopted.
|
||||||
*
|
*
|
||||||
|
* A FIND lowers to a candidate retry loop nested inside the region block. That loop is
|
||||||
|
* nearer than the region do block, so a POP inside it needs the __rN_done flag described
|
||||||
|
* in TextRegexSourceGen.GenClass.MultiIR.cs rather than a plain break.
|
||||||
|
*
|
||||||
* Generated variable names are owned by SourceGenContext. No generator invents one.
|
* Generated variable names are owned by SourceGenContext. No generator invents one.
|
||||||
*
|
*
|
||||||
* The per shape checklist this backend is rebuilt against lives in
|
* The per shape checklist this backend is rebuilt against lives in
|
||||||
@@ -90,7 +95,6 @@ namespace StgSharp.RegularAnalysis.Text
|
|||||||
// TODO L0.1 Emit the Match signature from ROS_char and SourceGenContext.InputSpan.
|
// TODO L0.1 Emit the Match signature from ROS_char and SourceGenContext.InputSpan.
|
||||||
// TODO L0.2 Emit region anchor and region result slots, count from RegexInfo.RegionCount.
|
// TODO L0.2 Emit region anchor and region result slots, count from RegexInfo.RegionCount.
|
||||||
// TODO L0.3 Emit line result slots, only for lines an Observe_Line prefix refers to.
|
// TODO L0.3 Emit line result slots, only for lines an Observe_Line prefix refers to.
|
||||||
// TODO L0.4 Emit the whole pattern guard from RegexInfo.MinPredictLength.
|
|
||||||
// TODO L0.5 Emit the cursor, the step result and any scratch declaration the context collected.
|
// TODO L0.5 Emit the cursor, the step result and any scratch declaration the context collected.
|
||||||
//
|
//
|
||||||
|
|
||||||
@@ -121,6 +125,7 @@ namespace StgSharp.RegularAnalysis.Text
|
|||||||
{
|
{
|
||||||
// GenerateRegion(region, sc, match_e, sub_match_list);
|
// GenerateRegion(region, sc, match_e, sub_match_list);
|
||||||
_ = TryGenerateAltChain(0, region, sc, match_e, sub_match_list);
|
_ = TryGenerateAltChain(0, region, sc, match_e, sub_match_list);
|
||||||
|
_ = TryGenerateTryBlock(0, region, sc, match_e, sub_match_list);
|
||||||
if (end >= ir_sp.Length)
|
if (end >= ir_sp.Length)
|
||||||
{
|
{
|
||||||
break;
|
break;
|
||||||
@@ -133,6 +138,15 @@ namespace StgSharp.RegularAnalysis.Text
|
|||||||
_ = se.AppendLine($@"public override MatchResult Match({ROS_char} text)")//
|
_ = se.AppendLine($@"public override MatchResult Match({ROS_char} text)")//
|
||||||
.AppendLine(@"{");
|
.AppendLine(@"{");
|
||||||
|
|
||||||
|
int min_predict_length = info.MinPredictLength;
|
||||||
|
if (min_predict_length > 0) {
|
||||||
|
_ = se.AppendLine($@"if (text.Length < {min_predict_length})")
|
||||||
|
.AppendLine(@"{")
|
||||||
|
.AppendLine(@"return default;")
|
||||||
|
.AppendLine(@"}")
|
||||||
|
.AppendLine();
|
||||||
|
}
|
||||||
|
|
||||||
int region_count = info.RegionCount;
|
int region_count = info.RegionCount;
|
||||||
for (int i = 0; i < region_count; i++)
|
for (int i = 0; i < region_count; i++)
|
||||||
{
|
{
|
||||||
|
|||||||
Reference in New Issue
Block a user