-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multicharacter tokens not parsed correctly #4
Comments
The problem here is that we parse (or lower) The two arms parse as (roughly) Ok(MacroRules { name: Ident(x), rules: [Rule { matcher: [Punct(Punct { op: '=', spacing: Alone }), Punct(Punct { op: '>', spacing: Alone })], expansion: TokenStream [Ident { sym: println }, Punct { op: '!', spacing: Alone }, Group { delimiter: Parenthesis, stream: TokenStream [Literal { lit: "Space" }] }, Punct { op: ';', spacing: Alone }] }] })
Ok(MacroRules { name: Ident(x), rules: [Rule { matcher: [Punct(Punct { op: '=', spacing: Joint }), Punct(Punct { op: '>', spacing: Alone })], expansion: TokenStream [Ident { sym: println }, Punct { op: '!', spacing: Alone }, Group { delimiter: Parenthesis, stream: TokenStream [Literal { lit: "No space" }] }, Punct { op: ';', spacing: Alone }] }] }) Is case of |
Macro_rules' concept of tokens is confusing and it's not just a matter of looking at whether the proc macro token is Joint or Alone: macro_rules! x {
// These rules are always equivalent.
(=> >) => { println!("Space"); };
(=>>) => { println!("No space"); };
}
fn main() {
x!(=> >); // "Space"
x!(=>>); // "Space"
} macro_rules! x {
// These rules are *not* equivalent.
(= >>) => { println!("Space"); };
(=>>) => { println!("No space"); };
}
fn main() {
x!(=> >); // "No space"
x!(=>>); // "No space"
} They greedily left-to-right form groups of consecutive punctuation according to which multi-character punctuations are recognized by Rust's grammar, and then whitespace between groups is ignored. (This is a limitation that is fixed in the token API of procedural macros.) So for example |
The two branches are currently seen as identical, even before optimizing. This is not correct.
The text was updated successfully, but these errors were encountered: