Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update documentation #128

Merged
merged 7 commits into from
Jun 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
112 changes: 65 additions & 47 deletions content/_index.en.md
Original file line number Diff line number Diff line change
Expand Up @@ -286,59 +286,63 @@ The default configuration can be found in `config/default.json`. The full refere

```js
{
"services": {
"declarationsPath": "Directory containing services declarations and associated filters"
},
"recorder": {
"versions": {
"storage": {
"<storage-repository>": "Storage repository configuration object; see below"
"@opentermsarchive/engine": {
"trackingSchedule": "Cron expression to define the tracking schedule; see below",
"services": {
"declarationsPath": "Directory containing services declarations and associated filters"
},
"recorder": {
"versions": {
"storage": {
"<storage-repository>": "Storage repository configuration object; see below"
}
},
"snapshots": {
"storage": {
"<storage-repository>": "Storage repository configuration object; see below"
}
}
},
"snapshots": {
"storage": {
"<storage-repository>": "Storage repository configuration object; see below"
"fetcher": {
"waitForElementsTimeout": "Maximum time (in milliseconds) to wait for elements to be present in the page when fetching document in a headless browser"
"navigationTimeout": "Maximum time (in milliseconds) to wait for page to load",
"language": "Language (in ISO 639-1 format) to pass in request headers"
},
"notifier": { // Notify specified mailing lists when new versions are recorded
"sendInBlue": { // SendInBlue API Key is defined in environment variables, see the “Environment variables” section below
"updatesListId": "SendInBlue contacts list ID of persons to notify on terms updates",
"updateTemplateId": "SendInBlue email template ID used for updates notifications"
}
}
},
"fetcher": {
"waitForElementsTimeout": "Maximum time (in milliseconds) to wait for elements to be present in the page when fetching document in a headless browser"
"navigationTimeout": "Maximum time (in milliseconds) to wait for page to load",
"language": "Language (in ISO 639-1 format) to pass in request headers"
},
"notifier": { // Notify specified mailing lists when new versions are recorded
"sendInBlue": { // SendInBlue API Key is defined in environment variables, see the “Environment variables” section below
"updatesListId": "SendInBlue contacts list ID of persons to notify on terms updates",
"updateTemplateId": "SendInBlue email template ID used for updates notifications"
}
},
"logger": { // Logging mechanism to be notified upon error
"smtp": {
"host": "SMTP server hostname",
"username": "User for server authentication" // Password for server authentication is defined in environment variables, see the “Environment variables” section below
},
"sendMailOnError": { // Can be set to `false` if sending email on error is not needed
"to": "The address to send the email to in case of an error",
"from": "The address from which to send the email",
"sendWarnings": "Boolean. Set to true to also send email in case of warning",
}
},
"reporter": { // Reporter mechanism to create GitHub issues when terms content is inaccessible
"githubIssues": {
"repositories": {
"declarations": "GitHub repository where to create issues; expected format: <owner>/<repository>",
"versions": "GitHub repository of versions associated with the declarations; expected format: <owner>/<repository>",
"snapshots": "GitHub repository of snapshots associated with the declarations; expected format: <owner>/<repository>"
"logger": { // Logging mechanism to be notified upon error
"smtp": {
"host": "SMTP server hostname",
"username": "User for server authentication" // Password for server authentication is defined in environment variables, see the “Environment variables” section below
},
"sendMailOnError": { // Can be set to `false` if sending email on error is not needed
"to": "The address to send the email to in case of an error",
"from": "The address from which to send the email",
"sendWarnings": "Boolean. Set to true to also send email in case of warning",
}
},
"reporter": { // Reporter mechanism to create GitHub issues when terms content is inaccessible
"githubIssues": {
"repositories": {
"declarations": "GitHub repository where to create issues; expected format: <owner>/<repository>",
"versions": "GitHub repository of versions associated with the declarations; expected format: <owner>/<repository>",
"snapshots": "GitHub repository of snapshots associated with the declarations; expected format: <owner>/<repository>"
}
}
},
"dataset": { // Release mechanism to create dataset periodically
"title": "Title of the dataset; recommended to be the name of the instance that generated it",
"versionsRepositoryURL": "GitHub repository where the dataset will be published as a release; recommended to be the versions repository for discoverability and tagging purposes",
"publishingSchedule": "Cron expression to define the dataset publishing schedule; see below"
},
"collection-api": { // Collection metadata API
"port": "The port number on which the API will listen for incoming requests",
"basePath": "The base path for the API endpoints"
}
},
"dataset": { // Release mechanism to create dataset periodically
"title": "Title of the dataset; recommended to be the name of the instance that generated it",
"versionsRepositoryURL": "GitHub repository where the dataset will be published as a release; recommended to be the versions repository for discoverability and tagging purposes"
},
"api": { // Collection metadata API
"port": "The port number on which the API will listen for incoming requests",
"basePath": "The base path for the API endpoints"
}
}
```
Expand All @@ -347,6 +351,20 @@ The default configuration is merged with (and overridden by) environment-specifi

For development, in order to have a local configuration that overrides the existing config, it is recommended to create a `config/development.json` file.

#### Schedules
Ndpnt marked this conversation as resolved.
Show resolved Hide resolved

Schedules for tracking and dataset publication are defined using Cron expressions.

A Cron expression is a string comprised of five or six fields separated by spaces, each representing a different unit of time: minute, hour, day of the month, month, and day of the week (and optionally, year). For example, the expression `30 */12 * * *` means "at minute 30 past every 12th hour of every day."

Here are some valid examples of Cron expressions and what they represent:

- `0 0 * * *`: Run at midnight every day.
- `0 */6 * * *`: Run every 6 hours.
- `30 2 * * MON`: Run at 2:30 AM every Monday.
.
Some online tools, such as [crontab.guru](https://crontab.guru), provide a user-friendly interface to create and validate Cron expressions.

#### Storage repositories

Two storage repositories are currently supported: Git and MongoDB. Each one can be used independently for versions and snapshots.
Expand Down
2 changes: 1 addition & 1 deletion content/api/collection.en.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,4 @@ The Collection API exposes JSON data over HTTP. Its [OpenAPI](https://swagger.io

That endpoint exposes both the OpenAPI specification if the requested `Content-Type` is JSON, and a Swagger UI for visual and interactive documentation otherwise.

> For example, the [documentation](http://162.19.74.224/api/v1/docs) of the [Demo collection](https://github.com/OpenTermsArchive/demo-declarations) is publicly available for exploration.
> For example, the [documentation](http://162.19.74.224/collection-api/v1/docs) of the [Demo collection](https://github.com/OpenTermsArchive/demo-declarations) is publicly available for exploration.
44 changes: 24 additions & 20 deletions content/api/federated.en.md → content/api/federation.en.md
Original file line number Diff line number Diff line change
@@ -1,45 +1,49 @@
---
title: Federated
title: Federation
weight: 3
aliases: /api/federated/
---

# Federated Web API [Beta]
# Federation API

Open Terms Archive is a decentralised system that tracks collections of services' terms across multiple servers. Each collection operates its own API, and the federated API unifies search and discovery across collections, fostering collaboration with external applications.
Open Terms Archive is a decentralised system that tracks collections of services' terms across multiple servers. Each collection operates its own API, and the Federation API unifies search and discovery across collections, fostering collaboration with external applications.

The Federated Web API exposes JSON data over HTTP. Its [documentation](http://51.89.136.45/v1/docs/) is provided in a dedicated, interactive interface.
The Federation API exposes JSON data over HTTP. Its [documentation](http://162.19.74.224/federation-api/v1/docs/) is provided in a dedicated, interactive interface.

That endpoint exposes both the [OpenAPI](https://swagger.io/specification/) specification if the requested `Content-Type` is JSON, and a Swagger UI for visual and interactive documentation otherwise.

## Beta

This API is offered as a preview, based on a first use case [defined](https://github.com/OpenTermsArchive/engine/issues/1016) with partner [ToS;DR](https://tosdr.org). Unexpected problems or missing functionality may arise. Please provide feedback through [issues](https://github.com/OpenTermsArchive/federated-api/issues) in the dedicated repository.
This API is offered as a preview, based on a first use case [defined](https://github.com/OpenTermsArchive/engine/issues/1016) with partner [ToS;DR](https://tosdr.org). Unexpected problems or missing functionality may arise. Please provide feedback through [issues](https://github.com/OpenTermsArchive/federation-api/issues) in the dedicated repository.

## Source code

The codebase for the Federated API is available on [`github.com/OpenTermsArchive/federated-api`](https://github.com/OpenTermsArchive/federated-api).
The codebase for the Federation API is available on [`github.com/OpenTermsArchive/federation-api`](https://github.com/OpenTermsArchive/federation-api).

## Configuring

The default configuration can be found in `config/default.json`. The full reference is given below. In the vast majority of cases, the default values should be sufficient and only the email sending data should be changed.

```js
{
"logger": { // Logging mechanism to be notified upon error
"smtp": {
"host": "SMTP server hostname", // Hostname of the SMTP server for sending emails
"username": "User for server authentication" // Password for server authentication is defined in environment variables, see the “Environment variables” section below
},
"sendMailOnError": { // Can be set to `false` to disable sending email on error
"to": "The address to send the email to in case of an error",
"from": "The address from which to send the email",
"sendWarnings": "Boolean. Set to true to also send email in case of warning. Default: false",
"@opentermsarchive/federation-api": {
"logger": { // Logging mechanism to be notified upon error
"smtp": {
"host": "SMTP server hostname", // Hostname of the SMTP server for sending emails
"username": "User for server authentication" // Password for server authentication is defined in environment variables, see the “Environment variables” section below
},
"sendMailOnError": { // Can be set to `false` to disable sending email on error
"to": "The address to send the email to in case of an error",
"from": "The address from which to send the email",
"sendWarnings": "Boolean. Set to true to also send email in case of warning. Default: false",
}
}
"port": "Port number on which the server will listen for incoming connections. Default: 3333",
"basePath": "The base path for the API endpoints",
"collections": [ // Overriding this value creates a risk of splintering the federation, make sure to fully understand what happens when changing this value
"List of collections to federate; see below for how to configure. Default: https://opentermsarchive.org/collections.json"
]
}
"port": "Port number on which the server will listen for incoming connections. Default: 3333",
"collections": [ // Overriding this value creates a risk of splintering the federation, make sure to fully understand what happens when changing this value
"List of collections to federate; see below for how to configure. Default: https://opentermsarchive.org/collections.json"
]
}
```

Expand Down Expand Up @@ -88,4 +92,4 @@ If multiple collections share the same `id`, the latest defined collection in th

## Deploying

Deployment recipes are available in a [dedicated repository](https://github.com/OpenTermsArchive/deployment). Look at the [Federated API section](https://github.com/OpenTermsArchive/deployment#federated-api-application) on the README to know how to deploy the API.
Deployment recipes are available in a [dedicated repository](https://github.com/OpenTermsArchive/deployment).
2 changes: 1 addition & 1 deletion content/collections/federation.en.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ A collection that **joins** the **federation** enjoys the following benefits:
1. Visibility on the Open Terms Archive website lists of collections and datasets.
2. Access to the Open Terms Archive GitHub organisation, administered by the Open Terms Archive core team.
3. Collection logo provided by the Open Terms Archive core team.
4. Referencing in the official [collections list](https://opentermsarchive.org/collections.json), enabling off-the-shelf discovery in the [Federated API]({{< relref "api/federated" >}}).
4. Referencing in the official [collections list](https://opentermsarchive.org/collections.json), enabling off-the-shelf discovery in the [Federation API]({{< relref "api/federation" >}}).
5. Referencing in the official [datasets list](https://opentermsarchive.org/datasets), providing visibility to analysts.
6. Dedicated channel on the Open Terms Archive instant messaging system.
7. API uptime tracking.
Expand Down
Loading