Skip to content

Commit

Permalink
Fix grammar in README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
blcham authored Jul 27, 2024
1 parent 75c8fab commit 038b350
Showing 1 changed file with 26 additions and 26 deletions.
52 changes: 26 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,32 @@
# SPipes

SPipes is tool to manage semantic pipelines defined in RDF inspired by [SPARQLMotion](https://sparqlmotion.org/) language. Each node in a pipeline represents some stateless transformation of data. [SPipes Editor](https://github.com/kbss-cvut/s-pipes-editor) is tool to viewing/editing/executing/debugging SPipes scripts.
SPipes is a tool to manage semantic pipelines defined in RDF inspired by [SPARQLMotion](https://sparqlmotion.org/) language. Each node in a pipeline represents some stateless transformation of data. [SPipes Editor](https://github.com/kbss-cvut/s-pipes-editor) is a tool for viewing/editing/executing/debugging SPipes scripts.

## Basic concepts

All terms defined in this section refers to SPipes terminology:
All terms defined in this section refer to SPipes terminology:

- *Module type* -- A template of transformations.
- *Module* -- A stateless transformation of input data that instantiates a *module type*.
- *Pipeline* -- Directed graph representing data flow of one program call consisting of set of *modules*. The graph must be acyclic and connected, with exactly one sink node. Each node is *module* while edges represent execution dependencies between *modules*. The sink node represents last transformation of the program call, i.e. last transformation of a *module*.
- *Pipeline* -- Directed graph representing data flow of one program call consisting of the set of *modules*. The graph must be acyclic and connected with exactly one sink node. Each node is *module* while edges represent execution dependencies between *modules*. The sink node represents the last transformation of the program call, i.e., the last transformation of a *module*.
- *Script* -- Set of *pipelines*.
- *Function* -- Pointer to a reusable part of a *pipeline* with constraints on input of the execution. It points to a *module* that should be executed. Output of the *module* is output of the function.
- *Function* -- Pointer to a reusable part of a *pipeline* with constraints on input of the execution. It points to a *module* that should be executed. The output of the *module* is the output of the function.
- *Engine* -- Executes transformations of input data according to a *pipeline*.


## SPipes Features

- Execute custom SPARQL functions within pipeline nodes
- Web and CLI interface to execute scripts
- Auditing execution, i.e. logging metadata about execution of modules, their input/output data, etc.
- Auditing execution, i.e., logging metadata about the execution of modules, their input/output data, etc.

## Loading of Pipelines

SPipes loads pipelines by recursive traversal of configured directories, searching for ontology files represented by `.ttl` suffix. *Global scripts* are represented by suffix **.sms.ttl**. A script is identified by ontology iri in which it is defined. Ontology imports (using rdf property **owl:imports**) can be used to modularize scripts into multiple files. Script defines set of pipelines from its ontology import closure.
SPipes loads pipelines by recursive traversal of configured directories, searching for ontology files represented by `.ttl` suffix. *Global scripts* are represented by suffix **.sms.ttl**. A script is identified by ontology IRI in which it is defined. Ontology imports (using rdf property **owl:imports**) can be used to modularize scripts into multiple files. The script defines a set of pipelines from its ontology import closure.

## Examples

SPipes script construction, execution, and execution history tracking is explained
SPipes script construction, execution, and execution history tracking are explained
in [Hello world example](doc/examples/hello-world/hello-world.md).
Script debugging is explained in [skosify example](doc/examples/skosify/skosify.md).
Working with RDF4J repository is explained in [rdf4j example](doc/examples/rdf4j-update/rdf4j-update.md).
Expand All @@ -36,15 +36,15 @@ Constraint validation is described in [constraint validation example](doc/exampl

### SPipes Core

Maven module SPipes Core provides core functionality related to SPipes engine, ontology manager, auditing. It contains configuration file `config-core.properties`, where directories of scripts for loading is configured.
Maven module SPipes Core provides core functionality related to the SPipes engine, ontology manager, and auditing. It contains the configuration file `config-core.properties`, where directories of scripts for loading are configured.

### SPipes Web

Web user interface for SPipes that allows to execute any *function* defined in *global scripts*. The *function* can be called by HTTP GET request

`$WEB_APP_URL/service?_pId=$FUNCTION&$PARAM_NAME_1=$PARAM_VALUE_1&$PARAM_NAME_2=$PARAM_VALUE_2...`, where
* `$WEB_APP_URL` -- url, where SPipes is deployed , e.g. "https://localhost:8080/s-pipes".
* `$FUNCTION` -- a function defined in a global script identified by URL. In case there is no collision the `localName` of the URL can be used. E.g. instead of using URL "http://example.org/my-function" one can use "my-function").
* `$WEB_APP_URL` -- URL, where SPipes is deployed, e.g., "https://localhost:8080/s-pipes".
* `$FUNCTION` -- a function defined in a global script identified by URL. If there is no collision, the `localName` of the URL can be used. E.g., instead of using the URL "http://example.org/my-function," one can use "my-function").
* `$PARAM_NAME_1`, `$PARAM_NAME_2`, ... -- names of parameters e.g. "repositoryName".
* `$PARAM_VALUE_1`, `$PARAM_VALUE_2`, ... -- value of parameters e.g. "myRepository".

Expand All @@ -55,12 +55,12 @@ In addition, there is a [list of reserved parameter names](doc/reserved-paramete

### SPipes CLI

Maven module SPipes CLI provides command-line interface to SPipes engine. In addition to `config-core.properties`, directories configured to load scripts can be overridden by command-line variable SPIPES_ONTOLOGIES_PATH. E.g. in UNIX shell following command can be used:
Maven module SPipes CLI provides a command-line interface to the SPipes engine. In addition to `config-core.properties`, directories configured to load scripts can be overridden by command-line variable SPIPES_ONTOLOGIES_PATH. For E.g. in the UNIX shell, the following command can be used:
export SPIPES_ONTOLOGIES_PATH="/home/someuser/s-pipes-scripts"

### SPipes Modules Registry

Defines dependencies of all specific *module types* that are used in Web and Cli interface at same time.
Defines dependencies of all specific *module types* that are used in Web and Cli interface at the same time.

### SPipes Modules

Expand All @@ -81,14 +81,14 @@ Defines Java model that is used for serialization of metadata about execution of

The following software needs to be installed on the system for development:

- JDK 17, or newer
- JDK 17 or newer
- Maven 3
- Apache Tomcat 9.0

## Dockerization

### Building the Docker Image
The Docker image of SPipes backend can be built using the following command:
The Docker image of the SPipes backend can be built using the following command:

`docker build -t s-pipes-engine .`

Expand All @@ -98,27 +98,27 @@ The following software needs to be installed on the system for development:

`docker run -p 8080:8080 s-pipes-engine:latest`

The endpoint will be available at: http://localhost:8080/s-pipes
The endpoint will be available at http://localhost:8080/s-pipes

#### Configuration of SPipes scripts

By default, scripts are loaded from directory `/scripts` within the filesystem of the s-pipes-engine image.
The directory already contains the necessary definitions of reusable modules, so to extend the scripts,
new scripts must be added to this directory. All subdirectories of `/scripts` are searched recursively.
Good practice is to mount local scripts to e.g. `/scripts/root` directory of the image, e.g.:
By default, scripts are loaded from the directory `/scripts` within the filesystem of the s-pipes-engine image.
The directory already contains the necessary definitions of reusable modules, so
new scripts must be added to this directory to extend the scripts. All subdirectories of `/scripts` are searched recursively.
A good practice is to mount local scripts to e.g., `/scripts/root` directory of the image, e.g.:
`docker run -v ./my-scripts:/scripts/root -p 8080:8080 s-pipes-engine:latest`

Another option to configure scripts is to redefine where SPipes engine searches the scripts using `CONTEXTS_SCRIPTPATHS`:
Another option to configure scripts is to redefine where the SPipes engine searches the scripts using `CONTEXTS_SCRIPTPATHS`:
`docker run -e CONTEXTS_SCRIPTPATHS=/my/special/path -p 8080:8080 s-pipes-engine:latest`

This is particularly useful when one would like to share same path between host filesystem and the docker image as
This is particularly useful when one would like to share the same path between the host filesystem and the docker image as
explained in the following section.

#### Aligning file paths between docker service and host system
#### Aligning file paths between the docker service and host system

For your SPipes script files you can align file paths between Docker services and your host system using mounting.
For your SPipes script files, you can align file paths between Docker services and your host system using mounting.
This allows a directory to be accessible from both Docker services and the host filesystem,
ensuring that file paths remain same. Consequently, you can copy an absolute path to a file from
ensuring that file paths remain the same. Consequently, you can copy an absolute path to a file from
a Docker service and open it on the host filesystem, and vice versa.

For Linux, the typical path is `/home`, while for Windows, it is `/host_mnt/c`.
Expand All @@ -141,9 +141,9 @@ To mount a directory from your host machine to the Docker container, use the fol

## Swagger

Rest API is documented by Swagger. We can open Swagger UI with: `SPIPES_URL/swagger-ui.html`.
Swagger documents rest API. We can open Swagger UI with: `SPIPES_URL/swagger-ui.html`.

## Licences of Reused software components

Beside included software dependencies by Maven, see a [list of reused software components, and their licences](./doc/licences.md).
Besides included software dependencies by Maven, see a [list of reused software components and their licenses](./doc/licenses.md).

0 comments on commit 038b350

Please sign in to comment.