Adding more flexibility to stateful lexer actions

[spq]

Hi,
I use participle to parse a relatively simple query format.
The format is similar to this: "date:2021-01-01 (content:foo or content:bar) sort:author"
It was easy to get this going with participle, thanks for the project!

I have, however one issue that is not dramatic but I would like to fix:
The sort key can currently appear everywhere in the query, I would like to restrict it to the root level and not next to any operator.

I think that would be relatively easy to solve by using a stateful lexer with an addition to the supported actions:
a) either an Action that Switches the current state
b) or and Action that combines the effect of multiple Actions (I would then Pop and Push)

I think (b) would be preferable as it allows more flexibility (e.g. popping two states).

WDYT?
--spq

[alecthomas]

Hi @spq ,

This kind of problem should be solved in the parser not the lexer.

One thing to note is that embedded structs are included in the parent struct grammar, so you can reuse sub-grammars fairly simply with a bit of care.

eg.

type AST struct {
	Common
	Sort string `|  "sort" ":" @Ident`
}

type Common struct {
	Date          time.Time `  "date" ":" @Date`
	Content       string    `| "content" ":" @Ident`
	Subexpression *Common   `| "(" @@ ")"`
}

Does that make sense?

etc.