This is a Scala implementation of a system for performing a grid search under common specifications defined for multiple languages.
The objective of this project is to benchmark the language on a real-world distributed system.
- Docker >3 (needs docker swarm)
- For local development:
- Number of replicas:
N_WORKERSconstant is defined in theMakefilefile. - Data config: in
containers/manager/src/main/resources/data.jsonyou can define (this config is built into the container):data: Intervals for each parameter, in format: [start, end, step, precision]agg: Aggregation function to be used: MIN | MAX | AVGmaxItemsPerBatch: Maximum number of items per batch (batches are sub-intervals)
- Manager config: in
containers/manager/src/main/resources/manager.confyou can define (this config is built into the container):- middleware config (rabbitmq addres, credentials, etc)
- metrics config (graphite address)
- data config path
make setupwill make required initializations, equivalent to:make initstarts docker swarmdown_rabbitmqbuild_rabbitmqmake build: builds manager and worker images
make deploy: deploys the manager and worker services locally, alongside with Graphite, Grafana and cAdvisor.make remove: removes all services
There are make scripts to run rabbit, graphana, manager and worker independently. Such as:
run_rabbitmq,run_graphite,run_manager_local,run_worker_local
make manager_logs: shows the logs of the manager servicemake worker_logs: shows the logs of the worker service
make common_publish_localpublishes the common packets to the local maven repository.run_manager_localandrun_worker_localcan be used to run the manager and worker services locally.
make deploy_remotebuilds and deploys the system on the server.
There are make scripts to tunnel the services such as:
tunnel_rabbitmq,tunnel_graphite,tunnel_cadvisor,tunnel_grafana.
This will require ssh access to the server
- Grafana: http://127.0.0.1:8081
- Graphite: http://127.0.0.1:8080
- Logs