test-assignment

👋 About

Welcome to the test task of the Utilizing Runtime Information to Improve Development Process PhD Position!

This task has two parts:

• In the main part, you need to analyze traces and other runtime information of a test project. In the repository you will find a docker-compose configuration containing a service called cat-api and an OpenTelemetry infrastructure to collect its execution traces. The tool should reveal bugs and performance issues in the cat-api service by analyzing execution traces or other runtime information.
• In the bonus part of the task, we ask you to improve this tool using machine learning models.

Requirements:

• Detect performance issues, anomalies, bugs, code or architectural smells using runtime information. As a first step, you should detect the N+1 query problem in the cat-api service.
• Issues found must be linked to source code and shown to the user. For example, the problem is in method X.

Desirable:

• Effectiveness: we will pay attention to the performance of your solution
• Scalability: the solution can be used with different projects, runtimes, and languages.
• Extensibility: detection of new types of issues can be easily added.
• Code quality and tests: the tool is designed with attention to code quality.
• User (developer) experience: it can be used by developers on a daily basis to assist in the software development process.
• Use not only traces, but also logs and other data (even git information).
• Take into account that runtime information may be obtained from different environments (different service installations).

You are welcome to:

• Choose any implementation you want — it can be a CLI tool, standalone tool or IntelliJ plugin, any other options are also valid.
• Modify the test project to add new cases to analyze or change the runtime information processing.

Quick start

To explore the data format, you can analyze the trace data added to the repository.

To work with real data, you can setup the environment:

0. Install docker & docker compose
1. Clone repository or download docker-compose configuration
2. Start docker-compose:

   docker compose up

3. Go to the cat-api HTTP API documentation and call the endpoints. To call endpoint, please press "Try it out" button near the endpoint documentation.
4. You will then be able to find traces in Jaeger UI related to your API calls.

Recorded quickstart: docs/quickstart.gif

Fig.1. C4 container diagram for the test task

Note it doesn't mean that your solution must be dockerized, in the C4 model, a container represents an application or a data store

Development

cat-api

Service to get information about cats.

To build docker image locally, you can run jibDockerBuild command:

./gradlew jibDockerBuild

ktlint is used to check code style, to auto-format source code you can run ktlintFormat command:

./gradlew ktlintFormat

To run tests in the test folder, you can run test command:

./gradlew test

HTTP API

OpenAPI spec is described in the openapi.yaml file.

cat-recommender client

OpenAPI spec for cat-recommender is described in the openapi.yaml file.

Database

Database migrations are available in the db.migration directory. Flyway is used to migrate database schema. If you want to update database schema, please create a new migration (e.g., V3__my_changes.sql). Migration will be applied on the application startup.

cat-api generates cat records on startup.

You can configure connection to the database using your favorite tool, e.g., psql (password: postgres):

 psql -h localhost -p 5432 -d cats -U postgres

Fig. 3. Database schema

cat-recommender-api

Service to generate cat pairs.

HTTP API

OpenAPI spec is described in the openapi.yaml file.

cat-recommender-api HTTP API documentation will be available here.

To call endpoint, please press "Try it out" button near the endpoint documentation.

Build

See README file in cat-recommender directory.

Distributed tracing

Distributed tracing is a method of observing requests as they propagate through distributed cloud environments.

Dataflow is similar to the Jaeger’s SPM demo environment.

1. cat-api is instrumented by OpenTelemetry javaagent.
2. javaagent sends traces and metrics to the otel-collector
3. cat-recommender-api is instrumented by opentelemetry-instrument
4. opentelemetry-instrument sends traces to the otel-collector
5. otel-collector sends data to the Jaeger and Prometheus

Fig. 4. Distributed tracing schema