diff --git a/docs/assets/GW3-logo-2.png b/docs/assets/GW3-logo-2.png new file mode 100644 index 0000000..17cf3a7 Binary files /dev/null and b/docs/assets/GW3-logo-2.png differ diff --git a/docs/assets/GW3-logo.png b/docs/assets/GW3-logo.png new file mode 100644 index 0000000..2f06b11 Binary files /dev/null and b/docs/assets/GW3-logo.png differ diff --git a/docs/index.md b/docs/index.md index 7c5039a..34fbb0c 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,7 +1,6 @@

- GeneWeaver Logo - GeneWeaver + GeneWeaver Logo

@@ -38,15 +37,17 @@ [:octicons-arrow-right-24: NCI-60 Workflow](tutorial/nci_60_example_01/) + [:octicons-arrow-right-24: Mapping Gene Identifiers](tutorial/map-gene-ids.ipynb) + [:octicons-arrow-right-24: Authenticating With The Client Library](tutorial/geneweaver_client_login/) - :material-api:{ .lg .middle } __API Standards__ --- - Developed to be consistent with our open API Standards. + Developed to be consistent with The Jackson Laboratory's open source API Standards. - [:octicons-arrow-right-24: Standards Documentation](reference/api-standards/) + [:octicons-arrow-right-24: Standards Documentation](https://thejacksonlaboratory.github.io/api-standards/) - :material-scale-balance:{ .lg .middle } __Open Source, Apache License 2.0__ diff --git a/docs/reference/api-standards/api-monitoring.md b/docs/reference/api-standards/api-monitoring.md deleted file mode 100644 index 02b1468..0000000 --- a/docs/reference/api-standards/api-monitoring.md +++ /dev/null @@ -1,34 +0,0 @@ -# API Monitoring -> These guiding principles will help stability, uptime, and awareness for your applications. - -Actuator endpoints let you monitor and interact with your application. - -## Monitoring -Our endpoints for monitoring and info will exist at - -``` -/api/monitor// -``` - -## Health Endpoint - -**Required** - - -This endpoint provides health information, mainly the status of the application. It will be the -primary target for monitoring services. - -Location: -``` -http://127.0.0.1/api/monitor/server/health -``` -Example Response: -```` -{"name":"Fancy Application","status":"UP","details":"Everything seems okay."} -```` - -| Status Text | Http Status Code | -|--------------|------------------| -| `UP` | 200 | -| `UNKNOWN` | 200 | -| `DOWN` | 503 | diff --git a/docs/reference/api-standards/api-versioning.md b/docs/reference/api-standards/api-versioning.md deleted file mode 100644 index d84292c..0000000 --- a/docs/reference/api-standards/api-versioning.md +++ /dev/null @@ -1,152 +0,0 @@ -# API Versioning and Backwards Compatibility - -!!! success "Overview" - In order to prioritize simplicity, reliability, and maintainability we seek to - **minimize the need for API versioning** _as much as possible_. - -API versioning is a strategy to ensure that changes to the structure, -features, or behavior of an API (Application Programming Interface) do not break -applications that depend on it. APIs allow different software applications to -communicate and interact with each other, so changes to an API can have broad impacts. - -!!! tip - If your service has a single client, **which you control**, then you can avoid - versioning altogether. You also **do not** need to version APIs which have not yet - been released. - -!!! warning - This document talks about versioning API deployments, a process that is separate - from application release versioning, which is described in the - [Software Versions Document](/Operations/Best_Practices/software_versions/). - - In the context of application release versioning, this document describes the - process of incrementing the **major** version number. For example, going from - `v1.0.0` to `v2.0.0`. - - -## Guiding Principles - -1. **Minimal versioning**: We avoid versioning unless absolutely necessary. This - approach reduces complexity, both for the team developing the API and for those who - consume it. We strive, whenever possible, to make changes that are backwards - compatible. - -2. **Independent changes for reproducible science**: To support reproducible science, - it's essential that we can modify APIs without impacting the analysis. This principle - allows us to evolve our systems independently, providing robustness and flexibility. - -When versioning becomes unavoidable due to changes that can break existing clients, we -should follow the strategies provided in -this [guide](https://www.xmatters.com/blog/blog-four-rest-api-versioning-strategies/). - -!!! danger - Versioning is not a substitute for good API design. We should always strive to - design APIs that are flexible and extensible, and that can evolve without breaking - existing integrations. **Incrementing a major version should be a last resort.** - - -## Backwards Compatibility - -Backwards compatibility is as important as versioning. By maintaining backwards -compatibility, we allow API consumers to upgrade at their own pace without breaking -their existing integrations. Strategies to ensure backwards compatibility include: - -1. **Adding new fields**: When adding new data, do so by adding new fields that older - clients will simply ignore. The simplest way to do this is to add new fields that - have a default value. - -2. **Avoid removing fields**: Removing data fields may break existing integrations. It's - safer to deprecate old fields and leave them in place, while encouraging clients to - use new fields. - -3. **Keeping old endpoints**: If creating a new version of an endpoint, consider keeping - the old one for a time to allow clients to transition gradually. - -4. **Communicating changes**: Whenever a change is made that affects the API, it's - important to inform consumers in advance to give them time to adapt. - - -## Service Information Endpoint - -A service information endpoint is a valuable tool in API design, especially in terms of -API versioning and backwards compatibility. The service information endpoint provides a -clear way for client applications to understand the current state of the service, its -capabilities, and its versioning details. This enables clients to interact more -intelligently and efficiently with the service, adjusting their behavior based on the -information returned by this endpoint. - -Here's how a service information endpoint contributes to API versioning and backward -compatibility: - -1. **Communicating Version Information**: - The version of the API is often part of the service information. This makes it clear - to any consumers of the API what version they are interfacing with. Consumers can - then decide how to handle any changes based on the version information. This is where - an application would provide its deployed - [Software Version](/Operations/Best_Practices/software_versions/) number. - -2. **Facilitating Backward Compatibility**: - By communicating what features, workflows, and filesystem protocols are supported by - the service, a client application can adjust its behavior to only use features - supported by the current API version. This allows older clients to still function - correctly, even if new features or changes have been introduced in the latest version - of the API. - -3. **Providing Useful Metadata**: - Additional information such as the name, description, environment, and contact - details helps clients understand more about the service. For instance, it can help - debug issues (e.g., if the environment is "test", the behavior might be different - from a production environment). - -4. **Allowing Intelligent Client Behavior**: - In a more advanced use case, a client application might use the service information - to decide between different APIs or services. - -## When We Need to Version - -Despite our minimal versioning philosophy, there may be situations that require it. -These could include major architectural changes, the addition of new features that -aren't compatible with the current API design, or the nature of the system that's being -changed. - -In these cases, we should follow the versioning strategies outlined below. - -### Versioning Strategies - -When we need to introduce versioning, you might consider adopting one of the following -strategies: - -1. **URI Versioning**: In this strategy, the version information is included in the URI - itself. This is very simple and straightforward, but it can break URI consistency and - may cause confusion. - - 1. **URI Path**: This option involves including the version number as part of - the URI path. This allows clients to cache resources easily, but requires - branching the entire API when incrementing the version. - - 2. **Query Parameters**: This option involves including the version number as a - query parameter. This is straightforward to implement, but can become - difficult to manage across multiple endpoints. It also requires maintaining - all versions of the endpoints in a single application build. - -2. **Request Header Versioning**: This approach keeps the URI clean and includes the - version information in the request header. This preserves URI consistency, but it - requires consumers to add version information to their requests. - -3. **Media Type Versioning (Accept Header)**: This strategy requires the client to - specify the version in the Accept header. It offers clean URIs, but it may increase - complexity for clients. - -4. **Hypermedia As The Engine Of Application State (HATEOAS)**: This is the most complex - strategy, but it provides a lot of flexibility. It allows API consumers to navigate - APIs through hypermedia links. - -Remember, the choice of versioning strategy should be informed by our guiding -principles, the needs of our API consumers, and the nature of the changes that require -versioning. You should pick the strategy that best fits the situation of your -application and it's needs, and prioritize minimizing API versioning as much as -possible. - -!!! tip - For more information on possible versioning strategies, see this - [guide](https://www.xmatters.com/blog/blog-four-rest-api-versioning-strategies/). \ No newline at end of file diff --git a/docs/reference/api-standards/dns-naming.md b/docs/reference/api-standards/dns-naming.md deleted file mode 100644 index d893e6d..0000000 --- a/docs/reference/api-standards/dns-naming.md +++ /dev/null @@ -1,130 +0,0 @@ -# DNS Naming - -DNS (Domain Name System) is a system that translates human-friendly domain names, like -`www.jax.org`, into the numerical IP addresses needed for devices to load Internet -resources. This conversion process is often referred to as DNS name resolution, and it's -carried out by DNS servers. DNS serves as a sort of phonebook for the Internet, allowing -us to use memorable domain names while computers and other devices can handle the -underlying IP addresses they correspond to. - -Consistent DNS naming is important to maintaining API standards, both in terms of -operations and development. Consistent naming conventions enhance ease of use and -understanding of APIs, as intuitive and uniform naming conventions allow developers to -guess the correct endpoint, thereby reducing reliance on extensive documentation lookup. -Furthermore, DNS naming conventions enhances efficiency in both development and -maintenance as the risk of misconfiguration and mistakes drops when there -are standard conventions to follow in handling APIs. Another aspect is interoperability; -standard naming conventions ease integration with other systems and services as they -allow for a common language to be used. Finally, consistency in DNS naming aids in -service discovery, particularly in a microservices architecture in that it streamlines -categorization and comprehension of each service's purpose. - -## Naming Conventions -There is no single standard for DNS naming conventions that applications are _required_ -to follow, however, the following is the default convention that should be followed -unless there is a compelling reason to deviate from it. - -``` --.. -``` - -Where most applications will be hosted on `.jax.org` -``` --.jax.org -``` - -### Production DNS -For production environments, the application will also map a DNS name that **does not** -include the environment name. This is to allow for the use of the shorter DNS name -by most end users. - -``` -.. -``` - -Where most applications will be hosted on `.jax.org` -``` -.jax.org -``` - -### Frontend Clients and Backend APIs -One important consideration is to maintain a list that maps client applications (e.g., -web apps, CLI tools) to the backend services they employ. This also helps manage CORS -settings, if needed. This also helps to make it clear when specific frontend clients are -siblings of specific backend APIs. - -Where possible, it is preferred that the frontend client uses the same DNS name as the -backend API. This allows for intuitive discovery of the backend API by the frontend -client, and by developers, quality assurance, and operations personnel. - -!!! tip "Micro-service Architectures" - In a world of micro-services it's likely that we would move away from a one-to-one - mapping between a UI and an API. In theory services will be for specific purposes, - while clients may become more broad and integrated, pulling from multiple APIs. - - Data or analytic API’s should also be concerned with serving up other programmatic - clients, like scripts or Jupyter notebooks developed by an analyst. - -### Versioning -Versioning should be handled by the application itself, and **not by the DNS name**. -This allows for the application to be updated to a new version without requiring any -changes to the DNS name. - -!!! tip "Versioning Not Required" - Not all APIs will require versioning, and those that do not should not feel - compelled to implement it. Often, versioning can be avoided by careful design of the - API. - - Read more about API Versioning in the API Standards group's - [versioning document](/Development/Best_Practices/API_Standards/api_versioning/). - - -### Environments -The environment name should be one of the following: - -- `dev` - Development environment -- `sqa` - Testing environment -- `stage` - Staging environment -- `prod` - Production environment - -It is highly recommended to deploy each of the above environments, especially if your -application will rely on other applications, or if other applications will rely on your -application. This allows for your application, and those that depend on it, to always -have a matching environment for each deployment environment. - - -#### Staging/Production -The staging and production environments are special environments that should be as close -to identical as possible. The only difference between the two should be the data that -they are using. This is to ensure that any issues that are found in staging can be -reproduced in production. - -Depending on your application deployment model, these two environments can be viewed as -the "Blue" and "Green" in a "Blue/Green" deployment model. - -??? note "Blue/Green Deployments" - Blue/Green deployments are a type of software release management strategy designed to - reduce downtime and risk by running two identical production environments known as - Blue and Green. - - - Blue environment: This is the live production environment that is currently serving - user traffic. - - Green environment: This is the clone of the production environment where you deploy - the new version of the application. - - Initially, the Blue environment is live, serving all user traffic. When a new version - of the software is ready for release, it is deployed to the Green environment. - - Once the new version is tested and ready to go live, the router is switched to direct - all incoming traffic to the Green environment. The Green environment then becomes the - live or active environment. - - The Blue environment, now idle, can be used for preparation for the next release. - - -## Additional Considerations - -1) Applications should be hosted on port 80/443 so that the user does **not** need to -specify a port. -2) It's important to publicize if the application is hosted on an internal-only -IP address as this will affect the ability of users to access the application. diff --git a/docs/reference/api-standards/error-handling.md b/docs/reference/api-standards/error-handling.md deleted file mode 100644 index 49ca1a0..0000000 --- a/docs/reference/api-standards/error-handling.md +++ /dev/null @@ -1,219 +0,0 @@ -# REST API Error Handling Best Practices - -Error handling is crucial for ensuring the reliability and usability of a REST API service. Proper error handling not only helps developers identify and fix issues but also provides clear and informative responses to clients so they may take corrective action. - -## Guidelines -### 1. Use Appropriate HTTP Status Codes - -HTTP status codes are essential for conveying the outcome of API requests and allow a high level grouping of error responses. Use them consistently to indicate the result of each API call, and avoid using obscure codes that API consumers may not be familiar with. Common, widely used error codes provide clear semantics and promote consistency and interoperability. These include: - -- **400 Bad Request**: Malformed request syntax or invalid parameters -- **401 Unauthorized**: Authentication required or invalid credentials -- **403 Forbidden**: The authenticated user does not have permission to perform the operation -- **404 Not Found**: Requested resource does not exist -- **422 Unprocessable Content**: The request is formed correctly but there are semantical errors in the payload -- **500 Internal Server Error**: Generic server-side error - -### 2. Provide Detailed Error Messages - -When an error occurs, include detailed error messages in the response. These messages should provide enough information for consumers to understand the cause of the error and take appropriate actions. However, be cautious not to expose sensitive information or specifics that could be exploited by malicious users. For example, do not include implementation details or detailed stack traces. -**Error messages should clarify the problem and communicate the intended functionality**. For example, if a type check fails on an API to fetch a study record by id, the message "Study ID must be an integer" clearly conveys what is expected. - -### 3. Implement Consistent Error Response Format - -Follow a consistent error response format across all API endpoints, and format error response payloads as JSON. This makes it easier for clients to parse error responses and handle them gracefully. Include fields **status, request_id, error_code, message, timestamp, and trace_id** in your error responses. - -```json -{ - "status": "", HTTP Status Code for the entire request - "request_id": "", Request identifier generated by the API service - "errors": [{ - "error_code": "", Error code (see below) - "message": "", Human readable error details - "timestamp": "", Date/time of the error - "trace_id": "" Pointer to the log trace - }] -} -``` - -### 4. Return an errors array -It is possible for multiple errors to occur within one transaction, so for consistency return an errors collection even if there is only one error. - -### 5. Log Errors for Debugging - -Log errors on the server-side to aid in debugging and monitoring. Include relevant details like error message, error code, request URL, source, user ID (if available), stack trace, and timestamp. Log errors at appropriate severity levels based on the error's impact, for example "Critical", "Error", "Warning", "Info", "Debug". If you would like to learn about BioConnect's logging service, please reference the documentation [here](https://docs.bioconnect.jax.org/core-modules/logging-service/#the-logging-service). - -### 6. Error Codes -Use meaningful error codes in addition to HTTP status codes to provide more specific information. CS is developing a standard list of error codes that may be used across all of our software applications, providing the following benefits: -1. **Clarity and specificity**: HTTP status codes are useful but can be generic and lack context. In addition to providing status codes, error codes allow us to convey more detailed information about the nature of the error. -2. **Consistency**: By aligning on a standard list of error codes, we can ensure that error responses are uniform and predictable, which will make it easier for front end developers to handle errors and provide consistent messaging to users. -3. **Error reporting and troubleshooting**: If errors are properly logged, developers will be able to aggregate log data based on error codes, providing insight into errors that occur most frequently. - - - -The following examples show error codes, in combination with status codes and error messages: -1. invalid parameter - -```json -{ - "status": "400", - "request_id": "37472a48-a34e-4813-b064-f863170f33fc", - "errors": { - "error_code": "parameter_invalid", - "message": "Cannot convert 'abc' to integer", - "timestamp": "2023-07-02T14:07:01", - "trace_id": "fb3a02ac6caa" - } -} -``` - -2. value too large for column in database - -```json -{ - "status": "400", - "request_id": "a83e5e07-06b2-44f9-a18a-8eefdc3f9bf8", - "error": { - "error_code": "value_invalid", - "message": "animal name must be 50 characters or less", - "timestamp": "2023-08-24T01:10:00", - "trace_id": "3ff84c1df586" - - } -} -``` - -Below is the current working list of error codes for CS Rest APIs: - -400-Bad Request - -- parameter_missing -- parameter_invalid -- parameter_length_exceeded -- header_invalid - -401-Unauthorized - -- api_key_required -- api_key_invalid -- credentials_required -- credentials_invalid - -403-Forbidden - -- privileges_insufficient -- request_limit_exceeded - -404-Not found - -- url_invalid - -408-Timeout exceeded - -- timeout_exceeded - -422-Unprocessable content - -- value_missing -- value_invalid -- value_length_exceeded - - -## Use Cases/Examples - -**Missing/Invalid Parameters** -Consider an API that returns a list of animals based on a weight min and max value. A validation check is performed to ensure that both parameters are present and numeric. If the min value type check validation fails, a status code of 400 is returned, along with a "parameter_invalid" error code and the message "min value must be numeric". If the parameter is missing all together, the following response is issued: - -``` - min = request.query_params.get('min_value', None) - if min is None: - return Response( - { - "status": "400", - "request_id": "7abfde24-b2e8-49f9-af5b-5f45bd56cce3", - "errors":[ - { - "error": { - "error_code": "parameter_missing", - "message": "Minimum weight value is required", - "timestamp": "2023-09-24T01:10:00", - "trace_id": "48785f565a27" - } - } - ] - } - ) -``` -This example is a good use case for aggregating and reporting on error data in order to improve user experience. If your error logs are showing high numbers of "parameter_invalid" codes for a certain API, perhaps you should look into front end validation and constraints in order to prevent them. - - -**Insufficient Privileges** -It is important to remember that errors involving privileges fall under the 403 status code rather than 401. In the case of user based actions, a 401 status code essentially means "I don't recognize you" and 403 means "I know who you are but you're not allowed to do this." - -A typical use case involving insufficient privileges is an attempt to update a record when the user does not have read/write access to the data. The API logic should perform an authorization check before an attempt to update the record, and if not authorized, the response should inform the user that their request has been denied due to insufficient privileges. - -``` -return Response( - { - "status": "403", - "request_id": "5bbca962-1f57-4016-bac5-5ae9a28d7d2e", - "errors":[ - { - "error_code": "privileges_insufficient", - "message": "You do not have sufficient privileges to update this record", - "timestamp": "2023-09-24T01:10:00", - "trace_id": "48785f565a27" - } - ] - } -) -``` - -!!! tip "Not all 403 errors are tied to authenticated users" - They could also be used for anonymous actions that can only be performed under certain circumstances (eg time based restrictions.) Another example is a server that only accepts requests from a predefined range of IP addresses. - - -**Database Errors** -Database errors can be due to authentication/authorization issues, lost connection, or data integrity issues such as constraint violations. Database libraries often have some built in error handling that can be leveraged. When possible, you should catch and handle specific exceptions rather than generic "Exception" handlers. -Wrap database operations in try-except blocks to catch and handle exceptions, for example: -``` - try: - study= Study.objects.get(pk=study_id) - return Response({'identifier': study.identifier}) - except Item.DoesNotExist: - return Response( - { - "status": "404", - "request_id": "5bbca962-1f57-4016-bac5-5ae9a28d7d2e", - "errors":[ - { - ... - } - ] - } - ) -``` - -!!! tip "APIs should be idempotent" - There is no way to group requests together in a transaction in REST APIs. To compensate for this, you should endeavor to [make your APIs idempotent](https://restfulapi.net/idempotent-rest-apis/). With the exception of a POST request, sending the same request multiple times should produce the same result. To validate idempotency, implement test cases encompassing multiple invocations of the same request to ensure consistent outcomes irrespective of the number of executions. - -**Timeout** -Timeout errors can occur when there are network or service issues, or if a call to the API involves too much data. A retry mechanism may be used to handle timeout errors, although this would normally be handled in front end client code when a 504 (Gateway timeout) is received. If timeouts are occurring due to high server load, consider load balancing and scaling your application to ameliorate them. Also consider monitoring and alerts to notify you when timeouts become frequent. - -**Errors from 3rd Party Libraries** -When calling methods in a 3rd party library, you can follow best practices for handling specific errors provided you have visibility into the exceptions it may raise. Consider handling these errors with graceful degradation if it's possible for your application to continue to function when they occur, for example if you can't retrieve data use cached data as a fallback. If it's a 3rd party service and it's possible for it to be down, consider implementing a retry mechanism. -An API end point that makes a failed call to a 3rd party service should return a 200 response if the error was isolated to the 3rd party service. - -**Business Logic Failures** -If a request includes data that fails a back end business logic check, return a status code of 422 (Unprocessable Entity.) This indicates that the syntax is correct but the value(s) prevented the call from being successfully processed. Include data in the response indicating the specific failure. For example, and end point that creates an animal record expects a date of birth but the date provided is in the future and fails a business rule check. A "value_invalid" error code and message stating "Date of birth cannot be in the future" allows the client to correct the error. - - - - - - - - - - - diff --git a/docs/reference/api-standards/http-standards.md b/docs/reference/api-standards/http-standards.md deleted file mode 100644 index 72d8b80..0000000 --- a/docs/reference/api-standards/http-standards.md +++ /dev/null @@ -1,84 +0,0 @@ - -Hypertext Transfer Protocol (HTTP) is a stateless application-level protocol used for -communication between web servers and clients, allowing for the transfer of data and -requests for resources over the internet. This document summarizes two important -sources of information for HTTP standards: - -- The Mozilla Developer Network (MDN) HTTP Reference[^1] -- The HTTP Semantics RFC (RFC 9110)[^2] - -## HTTP Methods - -### Common Implementation Methods -The following methods are commonly used for the endpoints we create in our web APIs. -They are useful for creating semantically meaningful RestFUL APIs. - -| Method | Type | Description | -|---------|-----------------------|---------------------------------------------------------------------------------------------------------------------------------| -| GET | Read Only, Idempotent | The GET method requests a representation of the specified resource. Requests using GET should **only retrieve data.** | -| HEAD | Read Only, Idempotent | The HEAD method asks for a response identical to a GET request, but **without the response body**. | -| POST | Non-idempotent | The POST method submits an entity to the specified resource, often **causing a change in state** or side effects on the server. | -| PUT | Idempotent | The PUT method **replaces** all current representations of the target resource with the request payload. | -| DELETE | Idempotent | The DELETE method **deletes** the specified resource. | -| PATCH | Non-idempotent | The PATCH method applies **partial modifications** to a resource. | - -### Common Operational Methods -These methods are most commonly handled by the web-frameworks/web-server and are not -typically used when implementing our own endpoints. - -| Method | Description | -|---------|-------------------------------------------------------------------------------------------| -| CONNECT | The CONNECT method establishes a tunnel to the server identified by the target resource. | -| OPTIONS | The OPTIONS method describes the communication options for the target resource. | -| TRACE | The TRACE method performs a message loop-back test along the path to the target resource. | - - -## HTTP Response Status Codes -We will list only the most relevant status codes here, but in general, we follow -the [mozilla guidelines](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status). The -following descriptions are taken directly from the mozilla documentation. - - - -### 2xx Success -These codes indicate success. The three most common that we use are listed here, and -have only slight differences. - -| Status Code | Description | Long Description | -|-------------|-------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------| -| 200 | OK | The request succeeded. The result meaning of "success" depends on the HTTP method. | -| 201 | Created | The request succeeded, and a new resource was created as a result. This is typically the response sent after POST requests, or some PUT requests. | -| 204 | No Content | There is no content to send for this request, but the headers may be useful. The user agent may update its cached headers for this resource with the new ones. | - -### 4xx Client Error -These codes indicate that the server determined that client did something wrong with the -request. These are the most straightforward to implement from a backend perspective, -and provide the most information to the client about what went wrong. - -| Status Code | Description | Long Description | -|-------------|----------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| 400 | Bad Request | The server cannot or will not process the request due to something that is perceived to be a client error (e.g., malformed request syntax, invalid request message framing, or deceptive request routing). | -| 401 | Unauthorized | Although the HTTP standard specifies "unauthorized", semantically this response means "unauthenticated". That is, the client must authenticate itself to get the requested response. | -| 403 | Forbidden | The client does not have access rights to the content; that is, it is unauthorized, so the server is refusing to give the requested resource. Unlike 401 Unauthorized, the client's identity is known to the server. | -| 404 | Not Found | The server cannot find the requested resource. In the browser, this means the URL is not recognized. In an API, this can also mean that the endpoint is valid but the resource itself does not exist. Servers may also send this response instead of 403 Forbidden to hide the existence of a resource from an unauthorized client. | -| 405 | Method Not Allowed | The request method is known by the server but is not supported by the target resource. For example, an API may not allow calling DELETE to remove a resource. | -| 409 | Conflict | This response is sent when a request conflicts with the current state of the server. | -| 422 | Unprocessable Entity | The request was well-formed but was unable to be followed due to semantic errors. | - -### 5xx Server Error -These codes indicate that the server had a problem processing the request. These are -the most difficult to implement from a backend perspective, and provide the least -information to the client about what went wrong. In general, when using one of these -codes, backend APIs should both log the associated error, and provide as much meaningful -information as possible to the client. - -| Status Code | Description | Long Description | -|-------------|------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| 500 | Internal Server Error | Internal Server Error | -| 501 | Not Implemented | The request method is not supported by the server and cannot be handled. The only methods that servers are required to support (and therefore that must not return this code) are GET and HEAD. | -| 502 | Bad Gateway | This error response means that the server, while working as a gateway to get a response needed to handle the request, got an invalid response. | -| 504 | Gateway Timeout | This error response is given when the server is acting as a gateway and cannot get a response in time. | - - -[^1]: [Mozilla: HTTP Status](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status) -[^2]: [RFC 9110 HTTP Semantics](https://www.rfc-editor.org/rfc/rfc9110) diff --git a/docs/reference/api-standards/index.md b/docs/reference/api-standards/index.md deleted file mode 100644 index 6d7ff36..0000000 --- a/docs/reference/api-standards/index.md +++ /dev/null @@ -1,18 +0,0 @@ ---- -title: API Standards ---- - -# :material-api: API Standards -The API standards group is an open working group that is focused on creating a set of -standards for the development of APIs. - -The group is focused on exploring the following topics: - -* [x] [Standard HTTP Methods & Status Codes](http-standards/) -* [x] [Standard Naming of Domain Names](dns-naming/) -* [x] [Standard Naming of Endpoints](restful-endpoint-naming/) -* [x] [Define Scenarios and Categories of Response Errors](error-handling/) -* [x] [OpenAPI Schemas](openapi-schemas/) (FKA Swagger) -* [x] [Standard for API Versioning and Deprecation](api-versioning/) -* [x] [Standard for API Monitoring and Health](api-monitoring/) -* [ ] Testing of all items listed above. diff --git a/docs/reference/api-standards/openapi-schemas.md b/docs/reference/api-standards/openapi-schemas.md deleted file mode 100644 index 08959a9..0000000 --- a/docs/reference/api-standards/openapi-schemas.md +++ /dev/null @@ -1,95 +0,0 @@ -# OpenAPI Schemas - -!!! success "Overview" - As part of our commitment to creating well-defined, reliable, and easy-to-use APIs, - we adopt OpenAPI[^1] as a standard tool for defining, creating, and documenting our - APIs. OpenAPI helps our teams collaborate more efficiently, enhances the developer - experience, and maintains the quality of our APIs. This document lays out standards - and best practices around the usage and publication of OpenAPI schemas. - -## Why OpenAPI? - -OpenAPI[^1] is an industry-standard method for describing RESTful APIs. By requiring an -OpenAPI document[^2] for each API, we aim to: - -- **Increase Transparency**: With an OpenAPI document, every aspect of an API is clearly -described, from endpoints to response formats. This reduces guesswork and the potential -for misunderstandings. - -- **Streamline Collaboration**: Frontend and backend teams can reference the OpenAPI -document to understand what data is available, how to access it, and what kind of -responses to expect. - -- **Simplify Integration**: Other systems or third-party developers can use the OpenAPI -document to understand how to integrate with our APIs. - -- **Enable API Testing and Monitoring**: OpenAPI documents can be used to generate -testing scripts and monitor APIs for any discrepancies in expected behavior. - -- **Improve Developer Experience**: An OpenAPI document forms the basis for generating -interactive documentation, SDKs, and API explorers, enhancing the overall developer -experience. - -## Key Principles - -1. **Up-to-date Schema**: Each RESTful API must always provide an up-to-date -`openapi.json` file, reflecting the latest version of the API. - -2. **Automatic Generation**: The `openapi.json` file should be automatically generated -from the source code and annotations as part of the build pipeline. - -3. **Public Accessibility**: The `openapi.json` file should be easily accessible, -preferably through a dedicated endpoint (e.g., -`https://api.yourdomain.com/openapi.json`). - -4. **Auto-generated API Documentation**: The OpenAPI schema should be used to -automatically generate and update API navigator pages or interactive API documentation -(like Swagger UI). - -## Detailed Standards - -### OpenAPI File Generation - -- Use tools and libraries that support OpenAPI file generation from code annotations -(e.g., Swagger for Java or SpringFox). - -- The OpenAPI file should be generated as part of the build process. - -- Keep your annotations up-to-date as you make changes to your API. - -### OpenAPI File Hosting - -- The `openapi.json` file should be publicly accessible via a dedicated URL. - -- The file should be placed at a consistent location across all APIs for easy discovery - (e.g., `https://api.yourdomain.com/openapi.json`). - -- The server should deliver the `openapi.json` file with the correct MIME type -`application/json`. - -### Versioning - -- The OpenAPI document should reflect the current version of the API and be updated with -each version change. - -- If multiple versions of the API exist, each version should have its own `openapi.json` -file. - -### Auto-generated API Documentation - -- Use a tool like Swagger UI or ReDoc to automatically generate interactive API -documentation from the OpenAPI schema. - -- The documentation should be publicly accessible and updated automatically whenever the -OpenAPI schema is updated. - -- The documentation should provide interactive features such as the ability to send test -requests. - -!!! info - By adopting OpenAPI and following these standards and best practices, we can improve - the developer experience, enhance the discoverability of our APIs, and ensure our - documentation is always up-to-date. - -[^1]: [OpenAPI Initiative](https://www.openapis.org/) -[^2]: [OpenAPI Specification](https://spec.openapis.org/oas/v3.1.0) diff --git a/docs/reference/api-standards/restful-endpoint-naming.md b/docs/reference/api-standards/restful-endpoint-naming.md deleted file mode 100644 index 24924d5..0000000 --- a/docs/reference/api-standards/restful-endpoint-naming.md +++ /dev/null @@ -1,87 +0,0 @@ -# RESTful API Endpoint Naming Standards - -!!! success "Overview" - The RESTful API Endpoint Naming Standards are a set of guidelines we follow - to maintain **consistency**, **clarity**, and **usability** in our APIs. - -Application Programming Interfaces (APIs) act as the gateways to data and capabilities -of our applications. With the REST architectural style, we can create scalable APIs that -are easy to consume and understand. - -Following a set of established naming conventions helps us create APIs that are -straightforward to use, which in turn accelerates development, minimizes the risk of -errors, and makes our services easier to consume. - -!!! tip - Although these are presented as standards, they should be considered as guidelines. - In specific cases, there might be valid reasons to deviate from these standards. - However, any deviation should be carefully considered and thoroughly discussed - within the team. - -## Guiding Principles - -1. **Resource Identification**: RESTful APIs use nouns (not verbs) to identify resources -or collections of resources. For example, use `/users` not `/getUsers` or `/createUser`. - -2. **Consistency**: Maintain consistent naming conventions across the API. This reduces -ambiguity and increases usability. - -3. **Plural Form Resources**: Resources should be named in plural form, such as `/users` -rather than `/user`. - -4. **Hierarchical Relationships**: Use sub-resources to show relationships between -resources. For example, to get a user's comments, you can use `/users/{id}/comments`. - -5. **Lowercase Letters**: Use lowercase letters for resources and collections. Mixed -case or camelCase can lead to confusion and errors. - -6. **Avoid Underscores (_)**: Underscores can sometimes be interpreted as spaces in -certain contexts and should be avoided. Use hyphens (-) for better readability if -needed[^2]. - -7. **No Trailing Slashes**: Trailing slashes should be avoided. For example, use -`/users` not `/users/`[^1]. - -8. **Non-CRUD Functions**: For routes that don't easily map to CRUD operations (Create, -Read, Update, Delete), consider mapping these to HTTP methods in a sensible way, or -group them under a sub-resource. For example, `/users/{id}/activate`. - -9. **Filters, Sorting, and Pagination**: For large collections, these should be -expressed as query parameters. For example, -`/users?status=active&sort=-registered&page=2`. - -10. **HTTP Status Codes**: Use appropriate HTTP status codes to indicate the status of -the request. For example, '200' for successful GET requests, '201' for successful POST -requests, '400' for bad requests, etc. - - !!! success "HTTP Status Codes" - For more information on the correct status codes to use, see the response - status codes section of the - [HTTP Standards page](/Development/Best_Practices/API_Standards/http_standards/#http-response-status-codes). - -11. **Error Handling**: Always return meaningful error messages and codes, helping the -consumer understand what went wrong and how they might fix it. - -!!! warning - Remember that these are **guidelines**, not hard rules. They serve as a starting - point for the API design, but each API has unique needs and may require certain - exceptions or adaptations. Always prioritize clarity, simplicity, and usability - when designing your API. - -## When Exceptions are Required - -Despite these guiding principles, there may be situations that require exceptions or -deviations. These could include compatibility with legacy systems, specific requirements -of certain clients, or other unique constraints. - -In such cases, exceptions should be carefully considered, thoroughly discussed within -the team, and clearly documented to avoid confusion and ensure everyone understands the -reasons behind the deviation. - -Always consider the potential impact of any exceptions on the overall usability, -clarity, and consistency of the API, and strive to minimize such deviations as much as -possible. - - -[^1]: [RFC 3986 - Uniform Resource Identifier (URI): Generic Syntax](https://www.rfc-editor.org/rfc/rfc3986#section-3.3) -[^2]: [StackOverflow - Hyphens, Underscores, or Camel Case](https://stackoverflow.com/questions/10302179/hyphen-underscore-or-camelcase-as-word-delimiter-in-uris) \ No newline at end of file diff --git a/docs/reference/restful-api.md b/docs/reference/restful-api.md index 46c6eef..ccd9260 100644 --- a/docs/reference/restful-api.md +++ b/docs/reference/restful-api.md @@ -13,3 +13,12 @@ The API is available at [geneweaver-prod.jax.org/aon/api/docs](https://geneweaver-prod.jax.org/aon/api/docs). Read the paper [here](https://pubmed.ncbi.nlm.nih.gov/37891644/). + + +## API Standards + +GeneWeaver follows the API Standards set forth by The Jackson Laboratory. These +standards help to make using the GeneWeaver API easier and consistent. + +You can read more about the standards at the +[API Standards GitHub Repository](https://thejacksonlaboratory.github.io/api-standards/). diff --git a/mkdocs.yml b/mkdocs.yml index aae7440..e298491 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -119,15 +119,6 @@ nav: - Logging In: reference/command-line/logging-in.md - API Commands: reference/command-line/api-commands.md - ReST API: reference/restful-api.md - - API Standards: - - reference/api-standards/index.md - - HTTP Standards: reference/api-standards/http-standards.md - - DNS Naming: reference/api-standards/dns-naming.md - - ReSTful Endpoint Naming: reference/api-standards/restful-endpoint-naming.md - - Response Errors: reference/api-standards/error-handling.md - - OpenAPI Schemas: reference/api-standards/openapi-schemas.md - - API Versioning: reference/api-standards/api-versioning.md - - API Monitoring: reference/api-standards/api-monitoring.md - Available Packages: reference/available-packages.md - Available Tools: reference/available-tools.md # - Scientific Workflows: reference/scientific-workflows.md