Literal Filter Operands and Filter Operator

[kookma]

Official TW docs, for filter state: Literal filter operands cannot contain square brackets but you can work around the issue by using a variable.

Is there any reason for this? Is it possible to address this in TW 5.2.3?

This will give much more flexibility in writing complex, multi steps filters.

[pmario]

It's a parser problem. TWs parser is regexp based. The following info is very simplified.

Let's say the default start-marker is [[ and the end-marker is ]] These 2 elements are used to find the string, that is in between the start and the end marker.

1. The parser "steps" through the source code on a per character basis until if finds the start-marker.
2. Then it uses the end-marker on a per character basis to find the end of a parse rule
3. As it finds the first text string the looks like the end-marker it stops.
4. The string in between the start- and the end-marker is used for further processing and is handled to a new parser, that does the same thing.
5. parsing the code is a big recursive function

As pointed out in step 3. searching for filter end markers stops at the first marker that looks like the end of a string, this causes some limitation in possible operand names.

eg:

• Go to tiddlywiki.com
• Create 2 new tiddlers HelloThere[[ and HelloThere]]
• Open Advanced Search -> Filter tab and enter
• [[HelloThere[[]]
  • It works
• Enter: [[HelloThere]]]] -> syntax error

Since the filter end-marker is ]] and it stops when finding the first one, there are 2 chars ]] left in the filter string. That causes the syntax error.

You may say: Just make the end-marker search greedy ... right?

That would cause a problem with detecting filter runs like: [[tag with spaces]] [[second tag with spaces]]

If the end-marker for filters would be greedy, it would fail to detect 2 tiddlers with spaces here.

As I wrote. That's a very simplified description of how the TW filter-parser works. The main goal in the TW code for parser is: "There should be as few different parsers as possible". ... For consistency, simplicity and maintainability. --

But this also causes some limitations. -- Having limited possibilities at end-markers is one of them

[Jermolene]

Thinking about it, there is one way we could "fix" it.

We could allow parenthesis (, ) as an alias for the inner square brackets in filter operands. Then one could say [(HelloThere]])].

Of course, one wouldn't be able to use parenthesis for titles that contain in )]. I'm not sure it's worth it. It has the feel of a cat and mouse game that we're never going to win. Meanwhile, using macro definitions works well for most cases.

[pmario]

I'm not sure it's worth it. It has the feel of a cat and mouse game that we're never going to win. Meanwhile, using macro definitions works well for most cases.

That's right. There are always some limitations. We could also establish an escape mechanism for \[\] BUT but imo it would only add complexity instead of making the workflow described in the OP simpler.

[kookma]

@pmario
Many Thanks for detailed explanation. I got what is behind the curtain and the origin of this limitation!

@Jermolene
Thank you for your reply. As I got from Mario post: consistency, simplicity and maintainability are TW policy,
So, I think I can live with current limitation, while I can benefit from readable and semantic filter syntax!