Regarding gojq

Tips regarding gojq, the Go implementation of jq

The Go implementation of jq, gojq, follows the jq language specification very closely and offers some features, such as support for YAML and for unbounded-precision integer arithmetic, which jq users might wish to take advantage of. This short article attempts to highlight such features as well as some limitations and drawbacks of gojq that jq users will probably want to be aware of.

The main points of comparison are:

jq-1.6
jqMaster (the "master" version as of around Jan 1, 2020)
gojq-0.12.4
gojq (as of around July 27, 2021)

Most Salient Differences

1. Functional

   • gojq supports infinite-precision integer arithmetic.

     Note, however, that there was a bug in gojq-0.12.4 that caused integer/integer division to revert to floating-point arithmetic; this bug has subsequently been fixed.

   • jqMaster retains the precision of all numeric literals, but gojq only retains the precision of integer literals.

   • gojq always sorts the keys of JSON objects and has no 'keys_unsorted' function.

   • gojq's JSON parser (as opposed to gojq's parser of jq programs) adheres more closely to the JSON standard. In particular, jq allows NaN and Infinite when reading JSON values, and ignores superfluous leading zeros in numbers. gojq on the other hand will interpret 000 as a stream of three zeros:

     $ gojq -n '[inputs]|length' <<< 000
     3

     Similarly, gojq's JSON parser will read 01 as 0 followed by 1, and 00.1 as 0 followed by 0.1.

   • jq's handling of "$-variables" in function definition headers is incorrect, and indeed gojq-0.12.4 implemented the intended semantics; this however introduced an unwanted discrepancy between jq and gojq, and as of this writing, gojq now mimics jq's faulty handling of these "$-variables". (See https://github.com/itchyny/gojq/issues/107)

   • gojq supports YAML, both as input and as output.

   • Some documented jq filters are (intentionally) missing from gojq:

     keys_unsorted
     $__loc__
     input_filename
     input_line_number
     

   • The gojq parser and interpreter can easily be incorporated into a Go project.

   • In jq-1.6 and earlier, the "else" clause is required in an "if" expression, whereas in jqMaster and gojq it is optional.

   • gsub with a left-anchored regex can produce results that are probably unexpected when using jq-1.6, as illustrated by:

     "aaa" | gsub("^a"; "b") #=> "bbb" in jq-1.6
     

     Please see the gojq README for a more comprehensive description of the functional differences.

2. Performance

   Memory management is perhaps gojq's weakest point. For what might be called small or light tasks, gojq is sometimes slightly faster than jq, but generally requires more memory, and often much more memory, to the point of memory exhaustion, which typically results in a somewhat bland failure message. For computationally intensive tasks, gojq is generally slower and often much slower than jq.

3. Bug fixes and support

   Currently, gojq is being maintained more actively than jq, and several important bugs in jq (e.g. related to timezones) have been fixed in gojq. These are summarized in the gojq README: https://github.com/itchyny/gojq/blob/main/README.md#difference-to-jq