Skip to content

fix: distinguish await identifiers in synchronous function bodies - #106

Open
exKAZUu wants to merge 7 commits into
mainfrom
fix/exported-class-decorator-ownership
Open

exKAZUu wants to merge 7 commits into
mainfrom
fix/exported-class-decorator-ownership

Conversation

@exKAZUu

@exKAZUu exKAZUu commented Oct 18, 2026 •

Copy link
Copy Markdown
Member

Why

Addresses the ordinary-function portion of #55. function f(){ await\nvalue; } must contain two identifier statements, while the async form contains one await expression. Shared statement rules previously selected the await operator in both bodies.

Requirements

  • required: distinguish await identifiers from async operands using the innermost function context, including nested functions.

  • required: preserve existing default source-goal behavior, public supertype queries, and incremental parsing.

  • chosen: address synchronous versus asynchronous function bodies in this PR; leave source-goal selection, parameters, class fields, and remaining generator/yield distinctions in fix: distinguish script await identifiers from module and async operands #55 so each change remains reviewable.

  • chosen: serialize a bit-packed context stack with a maximum of 8,160 nested function bodies and report a parse error on overflow; the fixed scanner serialization budget makes this bounded design smaller than duplicating the expression and statement grammar by context.

  • chosen: allow 30 seconds for the capacity correctness test because Intel CI completed its full-tree comparisons in 9.136 seconds; retain all depths, repeated async edits, full-tree assertions, and separate performance safeguards.

Customer Summary

Ordinary functions, arrows, methods, generators, and constructor function expressions now treat await as an identifier in their bodies. Nested async functions restore the enclosing interpretation when they end. Trees for affected synchronous bodies change accordingly.

Technical Summary

Grammar markers enter and leave body contexts. The external scanner serializes the stack and selects a distinct await-identifier token in synchronous bodies, retaining the existing await operand decisions elsewhere. Expression-bodied arrows defer leaving their context across operators and suffixes; block arrows retain their existing semicolon decisions. Shared trivia and infix-word scans avoid duplicating boundary logic.

The generated parser uses the canonical corpus/example training profile. The two updated corpus expectations remove incorrect await operators from synchronous bodies.

Testing

  • mise exec rust@1.99.1 -- bun run verify passes lint and type checking; code-gauge findings are described below.
  • mise exec rust@1.99.1 -- bun run build-wasm succeeds.
  • script/tree-sitter test: all 199 corpus cases pass.
  • mise exec rust@1.99.1 -- bun wb test test/unit/awaitContext.test.ts test/unit/constructors.test.ts test/unit/bodyRanges.test.ts test/unit/incremental.test.ts test/unit/performance.test.ts: 50 tests pass.
  • New parser-boundary checks cover eight function forms, public expression/primary-expression captures, nested context restoration, malformed-input repair, repeated incremental async insertion/removal at depths 1, 32, and 8,160, and parse-error behavior at depth 8,161.
  • Follow-up verification: bun wb test on awaitContext, queries, bodyRanges, incremental, and examples passes all 61 tests, including all 1,640 real-world source files. Final shared-trivia cleanup passes the 199-case native corpus and 18 context/query tests.
  • Capacity timeout validation: all four context tests pass locally (985 ms total). A direct profile at depths 1,020 / 4,080 / 8,160 measured approximately 63 / 245 / 584 ms for four incremental parses, four fresh parses, and eight complete tree serializations, with tree strings about 100 / 400 / 800 KB. Intel CI reported 9.136 seconds for the individual capacity test during the full suite; its explicit timeout is 30 seconds. Performance tests and parser limits are unchanged.
  • Full suite remains assigned to PR CI.

Notes

This partially addresses #55 and does not close it. Script/Module ambiguity and the other contexts listed above remain unresolved. The context limit applies to nested function bodies, not ordinary block nesting. Overflow reports an error instead of silently truncating or restoring an incorrect async context.

Code-gauge still reports cognitive complexity 130 and NCSS 136 for the existing main scanner dispatch. The new body transitions and shared boundary scans are extracted; resolving the remaining dispatch error would require restructuring unrelated template, semicolon, export, and resource-binding branches. The remaining scanner file-size and boundary-helper warnings would require splitting a cohesive scanner or moving its decision complexity without simplifying it. Unchanged example files also produce existing code-gauge errors. Generated parser tables retain generator-produced trailing whitespace; handwritten changed files pass git diff --check.

exKAZUu and others added 3 commits October 18, 2026 03:04
BREAKING CHANGE: Decorators before export belong to class_declaration rather than export_statement. The class range includes the intervening export keywords.

Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Track nested async and synchronous bodies across functions, methods and
arrows, preserving context during incremental parsing and boundary scans.
Report an error when the serialized context capacity is exhausted.

Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
@exKAZUu exKAZUu self-assigned this Oct 18, 2026
exKAZUu and others added 4 commits October 18, 2026 23:21
Keep comments outside consuming ternary tokens and handle pending block
arrow continuations before restoring the enclosing function context.

Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Function-context dispatch handles all generated states admitting block
arrow boundaries. Keep the remaining semicolon dispatch focused on its
reachable transitions and remove its duplicate emissions.

Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
The complete incremental/fresh tree comparisons finish in 9.136 seconds
on Intel CI under the full suite. Keep all capacity checks and repeated
edits, with a 30-second timeout for this correctness workload.

Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant