ri-orchestration-twitter

This microservice was created as a result of the OpenReq project funded by the European Union Horizon 2020 Research and Innovation programme under grant agreement No 732463.

Technical description

What does the microservice do

This microservice is responsible for the coordination of all microservices dealing with Twitter data. The main goal of this microservice is to define Twitter accounts that should be continuously observed. In its current state, the microservice first initiates a recurring crawling, classification, and finally storing process of tweets that mention a given account. To achieve this, this miroservice calls the following microservices of OpenReq:

• Crawler: ri-collection-explicit-feedback-twitter

• Classifier: ri-analytics-classification-twitter

• Storage ri-storage-twitter

These microservices must be configured and running in order to use ri-orchestration-twitter.

Which technologies are used

• Go (→ https://github.com/golang/go)

• Gorilla Mux (→ https://github.com/gorilla/mux)

How to install it

• setup and start the microservices: ri-collection-explicit-feedback-twitter , ri-analytics-classification-twitter, ri-storage-twitter

• Define a shared base url of all the three previously mentioned microservices called BASE_URL. BASE_URL will be used as an environment variable in the Docker run command (See following section).

• An SSL certificate must be mounted to the docker image during the run build (see the following example)

• A bearer token must be added as an environment variable called BEARER_TOKEN

Run the following commands to start the microservice:

1. docker build -t ri-orchestration-twitter .

2. docker run -v "<path_to>/ca_chain.crt:/go/src/app/ca_chain.crt" -e "BASE_URL=<BASE_URL_OF_THE_REQUIRED_MICROSERVICES>" -e "BEARER_TOKEN=<token>" -p 9703:9703 ri-orchestration-twitter

How to use it (high-level description)

The API is documented by using Swagger2:

• Rendered Documentation

Notes for developers

This mircoservices orchestrates the communication between several other microservices (crawler, classifier, and storage). The code currently assumes that all these microservices are behind a reverse proxy. Therefore, the class rest_handler.go does not specify any ports. In case you decide to deploy those micoservices without a reverse proxy, we suggest to add the proxies to the endpoints in rest_handler.go.

rest_handler.go

const (
	// analytics layer
	endpointPostClassificationTwitter = "/ri-analytics-classification-twitter/lang/"

	// collection layer
	endpointGetCrawlTweets             = "/ri-collection-explicit-feedback-twitter/mention/%s/lang/%s/fast"
	endpointGetCrawlAllAvailableTweets = "/ri-collection-explicit-feedback-twitter/mention/%s/lang/%s"

	// storage layer
	endpointPostObserveTwitterAccount     = "/ri-storage-twitter/store/observable/"
	endpointGetObservablesTwitterAccounts = "/ri-storage-twitter/observables"
	endpointGetUnclassifiedTweets         = "/ri-storage-twitter/account_name/%s/lang/%s/unclassified"
	endpointPostTweet                     = "/ri-storage-twitter/store/tweet/"
	endpointPostClassifiedTweet           = "/ri-storage-twitter/store/classified/tweet/"
)

Sources

None.

How to contribute

See OpenReq project contribution guidelines

License

Free use of this software is granted under the terms of the EPL version 2 (EPL2.0).