General feedback on pagefind

[MaxLap]

Hello, just wanted to give some feedback on this project. I personally find such feedback helpful for my own projects.

First and foremost, the DevX of this is great. Just running a script on the output is super nice and easy to integrate in any static site. Love it.

One thing missing from the --serve is that my site uses a subpath (example.com/docs). Since the hierarchy doesn't have that /docs, --serve didn't appear to be usable for me (no big deal).

404 appear to return a blank page right now. A suggestion to have the body of that 404 be either similar paths, or maybe a list of all paths that are available, to make it easier to figure out things such as the subpath issue i mention above, could even indicate the option to use to set the subpath.

An actual problem I have right now with pagefind is that a lot of the results I see are not expected. This is especially apparent in the edge-cases.

Unexpected 1: If I go on https://pagefind.app/docs/ui/ and search for "super". This lonely result is very weird to me.

There seems to be a fallback somewhere for single letter "words" that match the first letter of the search. Feels odd.

Unexpected 2: At the opposite end of the spectrum. It feels weird that searching for "sa" on https://pagefind.app/docs/ui/ gives those results:

I find it just feels weird as a user, but I have trouble finding a constructive reason / suggestion. It's kind of funny, the sorting of the pages is now based on how many words start with "sa" in the page.

Imagine if google actually did the searches it suggested as you type. You type "How is" and right away, it shows a result for:

• how is kate middleton doing
• how is vinegar made
• how is hiv transmitted
• etc.

Sounds weird but that's kind of what pagefind is doing when in shows a word like "saturation" from a simple "sa". I don't know the solution (but I have a suggestion below), I'm just highlighting something that I feel is unexpected.

**Unexpected 3: ** Searching for "log", as in log file, or maybe math's log function, and seeing "logic" appear high in the list is odd.

I don't know if that would solve everything I mentioned. But I wonder if having an auto-suggestion of words based on the pages would make this feel less unusual. So if I write "sa", it finds words like "same", "saturation", "sample", "safari", and offers those as suggestions for some kind of auto-fill. Then, the search could be more strict. I feel that would be more intuitive.

Anyhow, thanks for the interesting tool, it may be inspiration for how I do the DevX in a future one!

[bglw]

Hi @MaxLap ! 👋

Thanks for the detailed feedback, I'm sure I'll swing back here multiple times in the future to reference things 🙂

Regarding all of the results, it's indeed an area that is somewhat nebulous — both in terms of what Pagefind does and what different parties would like it to do! 😅

What you're seeing in all cases is Pagefind's core doing its best to find some answer to your query. If you search for something long like super, and it isn't found, it'll recursively pare that search term down (supe, sup, su, s) until it finds results.

In a similar vein, as you're seeing with sa, Pagefind effectively does a search for all sa* prefixes. This one is at least smarter, in that the rankings are very weighted toward your search term. So pages with saw will be substantially higher in the rankings than pages with saturation. (The excerpts can sometimes hide this fact, in some cases like this the excerpt might pick a more high-keyword but low-relevance section of the page to highlight).

In both these cases, Pagefind's core doesn't pick a point to give up at, mostly to demur on having to pick that point. Showing results for supe if you search super seems unilaterally good, and likewise logfile as a result for log is (imo) good behavior. Whether that holds for showing the single letter results is unlikely, but also possible in some domains! For now Pagefind tries not to get too smart with it. Perhaps a simple setting like maximumDeviation: 2 will go a long way, to say that results have to ± 2 letters from the search term. Something to ponder.

As for having some auto-suggestion of words, I agree that it would be nice, though is a job I'd push to the UI-layer of Pagefind. (I personally like the search-as-you-type flow). This is why the search core and UI are quite decoupled, it's very possible for another UI layer to be put on top of Pagefind's core that customizes when results are or aren't shown and behaves closer to your example here. I have dormant-but-desired hopes for having some sort of UI library that you can pick from to achieve just this sort of thing as a user 😄 (one day I'll get back to the modular-ui !)

One caveat to that flow as-described is that Pagefind is very bad at knowing what "words" are due to its stemming. With your example:

So if I write "sa", it finds words like "same", "saturation", "sample", "safari", and offers those as suggestions for some kind of auto-fill

Those words are actually indexed as same, satur, sampl, and safari. Pagefind doesn't actually know any examples of a full satur* word until the final result text is loaded, so it wouldn't be able to suggest it while you're typing. At least, not trivially 😅

Anyway, hopefully that helps shed some light on at least why it's doing what it is doing! Thanks again for taking the time to write everything up, I do appreciate it 🙂 Hopefully some of this can make it in as future UI options.