fstalign 1.3.0 resource benchmarking
This page describes some of the new improvements we made in the release of fstalign-1.3.0. Compared to the previous release, the new algorithm (using a Levenshtein pre-processor) should be notably faster (on average 5x-10x)! This comes at the cost of some increased memory usage, but as we will illustrate below, it only becomes a factor on extremely long transcripts (> 20k words).
From this, we can see that across the board, speed is improving with the new release. Short files go from 20-25s processing to 1-2s. The longest files (32k words) went from ~34 minutes to just ~6 minutes!
However, there are still considerations in terms of variance caused by WER. Both the current and past releases experience speed degradation when the observed word error rate is higher.
WER 0.3 --> 0.9 :: Runtime 21s --> 41s
WER 0.3 --> 0.9 :: Runtime 1.6s --> 20s
The analysis here used the generate_wer_test_data.pl script, which script provides an approximate WER, the algorithm could use some fine tuning to be exact.
In general, the RAM consumption between the two releases appears quite similar, with 1.3.0 looking slightly better for short files. However, for our largest test of 32k, 1.2.0 only used 1.6GB of RAM whereas 1.3.0 uses 4.0GB. Therefore, users should exercise care when using fstalign on extremely long files (and consider using --disable-approx-alignment).