Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(parser)!: use BindingIdentifier for namespace declaration names #6003

Conversation

DonIsaac
Copy link
Collaborator

@DonIsaac DonIsaac commented Sep 23, 2024

Use BindingIdentifier instead of IdentifierName so that AST visitors can access the bound symbol id for the namespace's name. This makes namespace consistent with other named declarations, such as Class, Function, and TSInterfaceDeclaration.

We should consider modifying TSModuleDeclarationName::StringLiteral(...) to also store a symbol_id, since one gets bound in semantic for it as well.

Copy link

graphite-app bot commented Sep 23, 2024

Your org has enabled the Graphite merge queue for merging into main

Add the label “0-merge” to the PR and Graphite will automatically add it to the merge queue when it’s ready to merge. Or use the label “hotfix” to add to the merge queue as a hot fix.

You must have a Graphite account and log in to Graphite in order to use the merge queue. Sign up using this link.

@DonIsaac DonIsaac added the C-enhancement Category - New feature or request label Sep 23, 2024 — with Graphite App
@github-actions github-actions bot added A-parser Area - Parser A-semantic Area - Semantic A-ast Area - AST A-transformer Area - Transformer / Transpiler labels Sep 23, 2024
@DonIsaac DonIsaac marked this pull request as ready for review September 23, 2024 18:04
Copy link
Collaborator Author

DonIsaac commented Sep 23, 2024

Copy link

codspeed-hq bot commented Sep 23, 2024

CodSpeed Performance Report

Merging #6003 will not alter performance

Comparing don/09-23-feat_parser_use_bindingidentifier_for_namespace_declaration_names (01b878e) with main (87f700f)

Summary

✅ 29 untouched benchmarks

@Boshen
Copy link
Member

Boshen commented Sep 24, 2024

Let me check the spec.

@Boshen Boshen self-assigned this Sep 24, 2024
@Boshen Boshen marked this pull request as draft September 24, 2024 02:18
@Boshen
Copy link
Member

Boshen commented Sep 24, 2024

This is incorrect, it's not a binding identifier. Although the "global" fix is correct in the next PR.

@DonIsaac
Copy link
Collaborator Author

How should we make a namespace's symbol is available from the AST?

@Boshen
Copy link
Member

Boshen commented Sep 24, 2024

How should we make a namespace's symbol is available from the AST?

Can you elaborate?

@DonIsaac
Copy link
Collaborator Author

How should we make a namespace's symbol is available from the AST?

Can you elaborate?

BindingIdentifier has a symbol_id property, but IdentifierName does not. In this snippet, a symbol Foo is bound and put into SymbolTable

namespace Foo {}

but since TSModuleDeclarationName is an IdentifierReference, Foo's symbol id is not available in the AST.

@Boshen
Copy link
Member

Boshen commented Sep 24, 2024

How should we make a namespace's symbol is available from the AST?

Can you elaborate?

BindingIdentifier has a symbol_id property, but IdentifierName does not. In this snippet, a symbol Foo is bound and put into SymbolTable

namespace Foo {}

but since TSModuleDeclarationName is an IdentifierReference, Foo's symbol id is not available in the AST.

Let me double check the archived tsc reference...

@DonIsaac DonIsaac force-pushed the don/09-23-feat_parser_use_bindingidentifier_for_namespace_declaration_names branch 2 times, most recently from 46d12f0 to b201de1 Compare September 24, 2024 18:30
@Boshen Boshen closed this Sep 25, 2024
@overlookmotel
Copy link
Collaborator

overlookmotel commented Sep 26, 2024

In my view, regardless of what TS's spec says, in our scheme, anything which has a record in the SymbolTable must have a symbol_id field. Without that field, our transform checker is not able to check the correctness of SymbolTable. I was planning to open a PR doing exactly this when I come to fix the scopes/symbols manipulation in TS transform.

I assume it being IdentifierName is a leftover from before we started resolving TS type references.

I suppose we could add symbol_id field to TSModuleDeclaration instead, but in my view it's much simpler to make the id a BindingIdentifier, as this PR does. Doing it this way also means that visit_identifier will get all `

So am re-opening this and, if you're willing @Boshen, would like to merge it.

@overlookmotel overlookmotel reopened this Sep 26, 2024
@Boshen
Copy link
Member

Boshen commented Sep 26, 2024

I need proof and reference from tsc to confirm this is a BindingIdentifier.

@DonIsaac
Copy link
Collaborator Author

DonIsaac commented Oct 7, 2024

@Boshen from section 10.1 Module Declarations in the TypeScript v1.4 spec:

ModuleDeclaration:
  module IdentifierPath { ModuleBody }
IdentifierPath:
  Identifier
  IdentifierPath . Identifier

The TypeScript spec uses Identifier for all other declarations. For example, here is the grammar for ClassDeclaration in section 8.1 Class Declarations:

ClassDeclaration:
  class Identifier TypeParametersopt ClassHeritage { ClassBody }

Our Class node uses a BindingIdentifier, meaning Identifier in TypeScript is the same as BindingIdentifier in oxc.

@DonIsaac DonIsaac force-pushed the don/09-23-feat_parser_use_bindingidentifier_for_namespace_declaration_names branch from b201de1 to d21f04e Compare October 7, 2024 20:39
@DonIsaac DonIsaac marked this pull request as ready for review October 7, 2024 20:40
@DonIsaac DonIsaac requested a review from Boshen October 7, 2024 20:40
@Boshen
Copy link
Member

Boshen commented Oct 8, 2024

I checked binder.ts

It calls function bindModuleDeclaration(node: ModuleDeclaration) {

and then

    function declareModuleSymbol(node: ModuleDeclaration): ModuleInstanceState {
        const state = getModuleInstanceState(node);
        const instantiated = state !== ModuleInstanceState.NonInstantiated;
        declareSymbolAndAddToSymbolTable(
            node,
            instantiated ? SymbolFlags.ValueModule : SymbolFlags.NamespaceModule,
            instantiated ? SymbolFlags.ValueModuleExcludes : SymbolFlags.NamespaceModuleExcludes,
        );
        return state;
    }

@Boshen Boshen changed the title feat(parser): use BindingIdentifier for namespace declaration names feat(parser)!: use BindingIdentifier for namespace declaration names Oct 8, 2024
@Boshen Boshen force-pushed the don/09-23-feat_parser_use_bindingidentifier_for_namespace_declaration_names branch from d21f04e to 6520a94 Compare October 8, 2024 08:34
@Boshen Boshen added the 0-merge Merge with Graphite Merge Queue label Oct 8, 2024
Copy link

graphite-app bot commented Oct 8, 2024

Merge activity

  • Oct 8, 4:34 AM EDT: The merge label '0-merge' was detected. This PR will be added to the Graphite merge queue once it meets the requirements.
  • Oct 8, 4:34 AM EDT: The merge label '0-merge' was detected. This PR will be added to the Graphite merge queue once it meets the requirements.
  • Oct 8, 4:39 AM EDT: Boshen added this pull request to the Graphite merge queue.
  • Oct 8, 4:47 AM EDT: Boshen merged this pull request with the Graphite merge queue.

…mes (#6003)

Use `BindingIdentifier` instead of `IdentifierName` so that AST visitors can access the bound symbol id for the namespace's name. This makes namespace consistent with other named declarations, such as `Class`, `Function`, and `TSInterfaceDeclaration`.

We should consider modifying `TSModuleDeclarationName::StringLiteral(...)` to also store a `symbol_id`, since one gets bound in semantic for it as well.
@Boshen Boshen force-pushed the don/09-23-feat_parser_use_bindingidentifier_for_namespace_declaration_names branch from 6520a94 to 01b878e Compare October 8, 2024 08:39
@graphite-app graphite-app bot merged commit 01b878e into main Oct 8, 2024
27 checks passed
@graphite-app graphite-app bot deleted the don/09-23-feat_parser_use_bindingidentifier_for_namespace_declaration_names branch October 8, 2024 08:47
@oxc-bot oxc-bot mentioned this pull request Oct 8, 2024
Boshen added a commit that referenced this pull request Oct 8, 2024
## [0.31.0] - 2024-10-08

- 01b878e parser: [**BREAKING**] Use `BindingIdentifier` for `namespace`
declaration names (#6003) (DonIsaac)

- 95ca01c cfg: [**BREAKING**] Make BasicBlock::unreachable private
(#6321) (DonIsaac)

- 020bb80 codegen: [**BREAKING**] Change to `CodegenReturn::code` and
`CodegenReturn::map` (#6310) (Boshen)

- 409dffc traverse: [**BREAKING**] `generate_uid` return a
`BoundIdentifier` (#6294) (overlookmotel)

- 5a73a66 regular_expression: [**BREAKING**] Simplify public APIs
(#6262) (leaysgur)

- 32d972e parser: [**BREAKING**] Treat unambiguous files containing TS
export assignments as modules (#6253) (overlookmotel)

- 4f6bc79 transformer: [**BREAKING**] Remove `source_type` param from
`Transformer::new` (#6251) (overlookmotel)

- afc3ccb napi/transform: [**BREAKING**] Rename
`TransformOptions::react` to `jsx`. (#6211) (Boshen)

- 82ab689 transformer,minifier: [**BREAKING**] Move define and inject
plugin from minifier to transformer (#6199) (Boshen)

### Features

- fa4d505 cfg: Derive more base traits for CFG blocks (#6320) (DonIsaac)
- 14275b1 cfg: Color-code edges in CFG dot diagrams (#6314) (DonIsaac)
- 7566c2d data_structures: Add `as_slice` + `as_mut_slice` methods to
stacks (#6216) (overlookmotel)
- c3c3447 data_structures: Add `oxc_data_structures` crate; add stack
(#6206) (Boshen)
- e304e8c minifier: Minify exponential arithmetic operation. (#6281)
(7086cmd)
- f9ae70c minifier: Minify basic arithmetic calculations. (#6280)
(7086cmd)
- 4008afe minifier: Fold array and object constructors (#6257)
(camchenry)
- 115ccc9 minifier: Bitwise not in exceeded value. (#6235) (7086cmd)
- ee6c850 minifier: Scaffold peephole replace known methods. (#6245)
(7086cmd)
- c32af57 minifier: Fold demical bitwise not for bigint. (#6233)
(7086cmd)
- 23b6464 minifier: Fold true / false comparison. (#6225) (7086cmd)
- 585ccda minifier: Support subtraction assignment. (#6214) (7086cmd)
- cca0034 minifier: Handle positive `NaN` and `Infinity`. (#6207)
(7086cmd)
- dac8f09 minifier: Minify unary plus negation. (#6203) (7086cmd)
- 3b79e1b minifier: Evaluate bigint in fold constant (#6178) (Boshen)
- abd3a9f napi/transform: Perform dce after define plugin (#6312)
(Boshen)
- a0ccc26 napi/transform: Add `lang` option to change source type
(#6309) (Boshen)
- f98e12c napi/transform: Add inject plugin (#6250) (Boshen)
- 291891e napi/transform: Add `define` option (#6212) (Boshen)
- 51a78d5 napi/transform: Rename all mention of React to Jsx; remove
mention of `Binding` (#6198) (Boshen)
- 2f888ed oxc: Add napi transform options (#6268) (Boshen)
- 8729755 oxc,napi/transform: Napi/transform use oxc compiler pipeline
(#6298) (Boshen)
- f6e42b6 sourcemap: Add support for sourcemap debug IDs (#6221) (Tim
Fish)
- 9e62396 syntax_operations: Add crate `oxc_syntax_operations` (#6202)
(Boshen)
- cf20f3a transformer: Exponentiation transform: support private fields
(#6345) (overlookmotel)

### Bug Fixes

- 84b2d07 codegen: Converts line comment to block comment if it is a
`PURE` comment (#6356) (Dunqing)
- e9eeae0 isolated-declarations: False positive for function with a type
asserted parameters (#6181) (Dunqing)
- d953a6b minifier: Correct the reference link (#6283) (dalaoshu)
- 37cbabb minifier: Should not handle the strict operation for bool
comparison. (#6261) (7086cmd)
- e29c067 minifier: Handle exceeded shifts. (#6237) (7086cmd)
- 294da86 napi/transform: Fix index.d.ts (Boshen)
- 9736aa0 oxc_transformer: Define `import.meta` and `import.meta.*`
(#6277) (IWANABETHATGUY)
- 6159560 parser: String `ImportSpecifier`s for type imports (#6352)
(DonIsaac)
- 1380d8b parser: Should regard comments where after `=` as leading
comments of next token (#6355) (Dunqing)
- 2bcd12a transformer: Exponentiation transform: fix reference flags
(#6330) (overlookmotel)
- 28cbfa7 transformer: Exponentiation transform: fix temp var names
(#6329) (overlookmotel)
- 3a4bcc7 transformer: Exponentiation transform: fix temp var names
(#6318) (overlookmotel)
- ccb7bdc transformer: Exponentiation transform: do not replace object
when private property (#6313) (overlookmotel)
- 56d50cf transformer: Exponentiation transform: do not assume `Math` is
not a local var (#6302) (overlookmotel)
- bd81c51 transformer: Exponentiation transform: fix duplicate symbols
(#6300) (overlookmotel)
- 06797b6 transformer: Logical assignment operator transform: fix
reference IDs (#6289) (overlookmotel)
- 4b42047 transformer: Fix memory leak in `ReplaceGlobalDefines` (#6224)
(overlookmotel)
- a28926f transformer: Fix inserting `require` with `front` option
(#6188) (overlookmotel)
- b92fe84 transformer: `NonEmptyStack::push` write value before updating
cursor (#6169) (overlookmotel)

### Performance

- 5db9b30 allocator: Use lower bound of size hint when creating Vecs
from an iterator (#6194) (DonIsaac)
- 788e444 transformer: Parse options from comments only once (#6152)
(overlookmotel)
- da2b2a4 transformer: Look up `SymbolId` for `require` only once
(#6192) (overlookmotel)
- 40bd919 transformer: Faster parsing JSX pragmas from comments (#6151)
(overlookmotel)

### Documentation

- eb1d0b8 transformer: Exponentiation transform: update doc comments
(#6315) (overlookmotel)
- c7636d7 traverse: Remove erroneous doc comment (#6328) (overlookmotel)

### Refactor

- f7d1136 allocator: Remove unnecessary `Vec` impl (#6213)
(overlookmotel)
- 40932f7 cfg: Use IndexVec for storing basic blocks (#6323) (DonIsaac)
- a1e0d30 cfg: Add type alias for Graph (#6322) (DonIsaac)
- 7672793 cfg: Move block data types to separate file (#6319) (DonIsaac)
- cc57541 data_structures: `NonEmptyStack::len` hint that `len` is never
0 (#6220) (overlookmotel)
- 147a5d5 data_structures: Remove `is_empty` methods for non-empty
stacks (#6219) (overlookmotel)
- 61805fd data_structures: Add debug assertion to `SparseStack` (#6218)
(overlookmotel)
- dbfa0bc data_structures: Add `len` method to `StackCommon` trait
(#6215) (overlookmotel)
- ac5a23f minifier: Use ctx.ast.vec instead of Vec::new. (#6331)
(7086cmd)
- 1cee207 minifier: Some boilerplate work for PeepholeFoldConstants
(#6054) (Boshen)
- 5b5daec napi: Use vitest (#6307) (Boshen)
- 58a8615 napi/transform: Remove context (#6306) (Boshen)
- 099ff3a napi/transform: Remove "Binding" from types; fix type error
(#6260) (Boshen)
- 54c1c53 napi/transform: Remove a call on `TransformOptions::clone`
(#6210) (Boshen)
- aa0dbb6 oxc: Add `napi` feature, change napi parser to use `oxc` crate
(#6265) (Boshen)
- 3b53dd4 parser: Provide better error messages for `const` modifiers on
class elements (#6353) (DonIsaac)
- acab777 regular_expression: Misc fixes (#6234) (leaysgur)
- bdd9e92 semantic: Rename vars from `ast_node_id` to `node_id` (#6304)
(overlookmotel)
- d110700 semantic: Dereference IDs as quickly as possible (#6303)
(overlookmotel)
- 03bc041 syntax: Remove some unsafe code creating IDs (#6324)
(overlookmotel)
- bd5fb5a transformer: Exponentiation transform: rename methods (#6344)
(overlookmotel)
- 4aa4e6b transformer: Exponentiation transform: do not wrap in
`SequenceExpression` if not needed (#6343) (overlookmotel)
- a15235a transformer: Exponentiation transform: no cloning (#6338)
(overlookmotel)
- 7d93b25 transformer: Exponentiation transform: split into 2 paths
(#6316) (overlookmotel)
- 15cc8af transformer: Exponentiation transform: break up into functions
(#6301) (overlookmotel)
- 7f5a94b transformer: Use `Option::get_or_insert_with` (#6299)
(overlookmotel)
- 0cea6e9 transformer: Exponentiation transform: reduce identifier
cloning (#6297) (overlookmotel)
- ac7a3ed transformer: Logical assignment transform: reduce identifier
cloning (#6296) (overlookmotel)
- 527f7c8 transformer: Nullish coalescing transform: no cloning
identifier references (#6295) (overlookmotel)
- 7b62966 transformer: Move `BoundIdentifier` into `oxc_traverse` crate
(#6293) (overlookmotel)
- c7fbf68 transformer: Logical assignment operator transform: no cloning
identifier references (#6290) (overlookmotel)
- f0a74ca transformer: Prefer `create_bound_reference_id` to
`create_reference_id` (#6282) (overlookmotel)
- ba3e85b transformer: Fix spelling (#6279) (overlookmotel)
- bc757c8 transformer: Move functionality of common transforms into
stores (#6243) (overlookmotel)
- 1c31932 transformer: Rename var in `VarDeclarations` common transform
(#6242) (overlookmotel)
- 0400ff9 transformer: `VarDeclarations` common transform: check if at
top level with `ctx.parent()` (#6231) (overlookmotel)
- 235cdba transformer: Use AstBuilder instance from TraverseCtx (#6209)
(overlookmotel)
- a7ed29e transformer: Insert `import` statement or `require` depending
on source type (#6191) (overlookmotel)
- 4c63f0e transformer: Rename methods (#6190) (overlookmotel)
- 900cb46 transformer: Convert `ModuleImports` into common transform
(#6186) (overlookmotel)
- 00e2802 transformer: Introduce `TopLevelStatements` common transform
(#6185) (overlookmotel)
- 70d4c56 transformer: Rename `VarDeclarationsStore` methods (#6184)
(overlookmotel)
- 81be545 transformer: Export `var_declarations` module from `common`
module (#6183) (overlookmotel)
- 02fedf5 transformer: Shorten import (#6180) (overlookmotel)
- f2ac584 transformer: Use TraverseCtx's ast in ModuleImports (#6175)
(Dunqing)
- 21b08ba transformer: Shared `VarDeclarations` (#6170) (overlookmotel)
- 0dd9a2e traverse: Add helper methods to `BoundIdentifier` (#6341)
(overlookmotel)
- c0e2fef traverse: Function to get var name from node (#6317)
(overlookmotel)
- adc5381 traverse: `TraverseAncestry` use `NonEmptyStack` (#6217)
(overlookmotel)

### Testing

- 964d71e minifier: Add arithmetic tests for fold constants. (#6269)
(7086cmd)
- fcb4651 minifier: Enable null comparison with bigint. (#6252)
(7086cmd)
- d4f2ee9 transformer: Tidy up transform checker (#6287) (overlookmotel)
- 0f5afd7 transformer: Transform checker output symbol name for
mismatches (#6286) (overlookmotel)

---------

Co-authored-by: Boshen <1430279+Boshen@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0-merge Merge with Graphite Merge Queue A-ast Area - AST A-parser Area - Parser A-semantic Area - Semantic A-transformer Area - Transformer / Transpiler C-enhancement Category - New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants