implement results API

[jadudm]

Problem

Change management in government is hard. If, in transitioning between systems, we were to invent a new API, it would represent a huge disruption for many of our partners.

For that reason, we want to minimize, or even eliminate, the change costs for our partners. We'll do that by re-implementing the existing API.

How did we discover this problem?

This problem is inherent in the switch between systems. Users expect our API to exist at a given endpoint, take a certain set of parameters, and produce results in a given shape. We cannot/should not violate that.

Job Story(s)

A consumer of our API (e.g. nasa.gov) should see zero interruption to search services when we switch over to Jemison.

What are we planning to do about it?

The goal of the API reimplementation is to faithfully reimplement the Results API.

The goal of this step is to introduce an API service into the stack that would be adequate to the task of being ready for LATO assessment.

That's it.

What are we not planning to do about it?

There is a "clicks" and "typeahead" API; those see low/no use, and we're not making that part of this initial work.

How will we measure success?

we'll be ready for lato assessment when...

Preview Give feedback

1. create api cmd/app #58
   +0
   
2. implement required input parameters #59
   +0
3. unit test coverage for required portions of results API #60
   +0

we'll be ready for initial testing with partners when...

Preview Give feedback

1. implement optional input parameters
2. implement query and results

Security Considerations

Required per CM-4.

This is a read-only API, and is no more or less dangerous than any other HTTP server. (Which is to say...)

We're using standard libraries, and it talks to read-only database files (not a live server). Therefore, we have some confidence that this is a good/secure implementation pathway for API implementation.

Things to consider/address for purposes of security:

Consider/address

Preview Give feedback

1. make sure the content type and media type are correct
2. assess and defend against "billion laughs"/expansion-style attacks
3. limit size of GET URL/parameters/posted data
4. validate inputs for injection flaws (ideally use known good library)
5. consider and explicitly address each of the OWASP top 10 (to start)

Billion laughs: https://en.wikipedia.org/wiki/Billion_laughs_attack
OWASP Top 10 API risks: https://owasp.org/API-Security/editions/2023/en/0x11-t10/

Various validation libraries

Not intended to be exhaustive

• https://github.com/go-playground/validator
• https://github.com/santhosh-tekuri/jsonschema (probably not relevant here)
• https://github.com/faceair/jio
• https://github.com/go-ozzo/ozzo-validation (old/5yrs since update)
• https://github.com/muonsoft/validation
• https://github.com/cohesivestack/valgo

Inspirational articles:

• https://dev.to/kittipat1413/a-guide-to-input-validation-in-go-with-validator-v10-56bp
• https://medium.com/@matryer/patterns-for-decoding-and-validating-input-in-go-data-apis-152291ac7372
• https://husobee.github.io/golang/validation/2016/01/08/input-validation.html

references/resources

• https://search.gov/developer/

Because Fuego is still pre-v1, and because this seems like a nice to have rather than a must have, we recommend using gin as the implementation framework, because it is 1) stable, and 2) very well documented. What we must have is a stable, working API that can be versioned. (Gin's groups are good for this.) What we could have at a later point are docs that are auto-generated. Given that we already have the docs (we're reimplementing an API), I recommend using a battle-tested framework as a starting point.

[jadudm]

Related to #38.

[jadudm]

More related to #69, which is about the typeahead API.