Add all-in-one Docker Compose and Grafana configuration #997
Conversation
WalkthroughThe recent changes introduce a new Docker Compose configuration for deploying the Yorkie application alongside MongoDB, Prometheus, and Grafana. Documentation enhancements include detailed setup instructions for Grafana dashboards and configuration files for Prometheus and Grafana. New files were created to support Grafana's data sources and general settings, improving the overall deployment and monitoring capabilities of the application. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant Docker
participant Yorkie
participant MongoDB
participant Prometheus
participant Grafana
User->>Docker: Run docker-compose-deploy.yml
Docker->>Yorkie: Start Yorkie service
Docker->>MongoDB: Start MongoDB service
Docker->>Prometheus: Start Prometheus service
Docker->>Grafana: Start Grafana service
Prometheus->>Grafana: Provide metrics data
Grafana->>User: Display metrics visualization
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
window9u
changed the title
window9u
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
Outside diff range, codebase verification and nitpick comments (1)
build/docker/grafana/README.md (1)
7-7: Format the URL as a link.To improve the readability and follow the Markdown best practices, please format the URL as a link.
Apply this diff to format the URL as a link:
-1. Open Grafana in your browser (http://localhost:3000). +1. Open Grafana in your browser ([http://localhost:3000](http://localhost:3000)).Tools
Markdownlint
7-7: null
Bare URL used(MD034, no-bare-urls)
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (7)
- build/docker/README.md (1 hunks)
- build/docker/docker-compose-deploy.yml (1 hunks)
- build/docker/grafana/README.md (1 hunks)
- build/docker/grafana/dashboard.yml (1 hunks)
- build/docker/grafana/datasources.yml (1 hunks)
- build/docker/grafana/grafana.ini (1 hunks)
- build/docker/prometheus.yml (1 hunks)
Files skipped from review due to trivial changes (2)
- build/docker/grafana/grafana.ini
- build/docker/prometheus.yml
Additional context used
yamllint
build/docker/grafana/datasources.yml
[error] 8-8: no new line character at the end of file
(new-line-at-end-of-file)
build/docker/grafana/dashboard.yml
[error] 14-14: no new line character at the end of file
(new-line-at-end-of-file)
build/docker/docker-compose-deploy.yml
[warning] 34-34: wrong indentation: expected 6 but found 8
(indentation)
[error] 49-49: no new line character at the end of file
(new-line-at-end-of-file)
LanguageTool
build/docker/README.md
[uncategorized] ~23-~23: Loose punctuation mark.
Context: ...d Grafana. -docker-compose-deploy.yml: This file is used to deploy Yorkie by o...(UNLIKELY_OPENING_PUNCTUATION)
Markdownlint
build/docker/grafana/README.md
7-7: null
Bare URL used(MD034, no-bare-urls)
Additional comments not posted (8)
build/docker/grafana/datasources.yml (1)
1-8: LGTM! The Prometheus datasource is correctly configured.The YAML configuration correctly defines a Prometheus datasource for Grafana with the following properties:
name: Prometheustype: prometheusaccess: proxyurl: http://prometheus:9090isDefault: trueThis aligns with the PR objective of setting up Grafana with Prometheus.
Tools
yamllint
[error] 8-8: no new line character at the end of file
(new-line-at-end-of-file)
build/docker/README.md (1)
23-24: LGTM!The new bullet point provides a clear explanation of the purpose of the
docker-compose-deploy.ymlfile, which is consistent with the PR summary. This addition enhances the documentation by providing clarity on the deployment process and the components involved.Tools
LanguageTool
[uncategorized] ~23-~23: Loose punctuation mark.
Context: ...d Grafana. -docker-compose-deploy.yml: This file is used to deploy Yorkie by o...(UNLIKELY_OPENING_PUNCTUATION)
build/docker/grafana/README.md (1)
14-28: LGTM!The "Setting a Default Dashboard" section looks good. The instructions are clear and the code block is correctly formatted.
build/docker/docker-compose-deploy.yml (5)
1-3: LGTM!The Docker Compose version and services definition look good.
4-12: LGTM!The Prometheus service configuration looks good.
13-25: LGTM!The Grafana service configuration looks good. Mounting the configuration files for dashboards, data sources, and general settings is a great way to simplify the setup process.
26-31: LGTM!The MongoDB service configuration looks good. Always restarting the service is a good practice to ensure high availability.
32-49: LGTM!The Yorkie service configuration looks good. Building the service using a Dockerfile and specifying the correct command-line arguments ensures that Yorkie is properly configured and connected to the MongoDB service.
Tools
yamllint
[warning] 34-34: wrong indentation: expected 6 but found 8
(indentation)
[error] 49-49: no new line character at the end of file
(new-line-at-end-of-file)
build/docker/grafana/datasources.yml
Outdated
| type: prometheus | ||
| access: proxy | ||
| url: http://prometheus:9090 | ||
| isDefault: true |
There was a problem hiding this comment.
Add a newline at the end of the file.
The yamllint static analysis tool indicates that there is no newline character at the end of the file.
To fix this, append a newline at the end of the file:
datasources:
- name: Prometheus
type: prometheus
access: proxy
url: http://prometheus:9090
isDefault: true
+Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| isDefault: true | |
| isDefault: true | |
Tools
yamllint
[error] 8-8: no new line character at the end of file
(new-line-at-end-of-file)
build/docker/grafana/dashboard.yml
Outdated
| apiVersion: 1 | ||
|
|
||
| providers: | ||
| - name: 'dashboards' | ||
| orgId: 1 | ||
| folder: '' | ||
| folderUid: '' | ||
| type: file | ||
| disableDeletion: true | ||
| updateIntervalSeconds: 10 | ||
| allowUiUpdates: true | ||
| options: | ||
| path: /var/lib/grafana | ||
| foldersFromFilesStructure: true |
There was a problem hiding this comment.
LGTM! The Grafana dashboard configuration looks good.
The configuration follows the Grafana dashboard provisioning syntax and the options are set appropriately.
Add a new line at the end of the file.
The yamllint static analysis tool has flagged an error for missing new line at the end of the file.
To fix the error, add a new line at the end of the file:
| apiVersion: 1 | |
| providers: | |
| - name: 'dashboards' | |
| orgId: 1 | |
| folder: '' | |
| folderUid: '' | |
| type: file | |
| disableDeletion: true | |
| updateIntervalSeconds: 10 | |
| allowUiUpdates: true | |
| options: | |
| path: /var/lib/grafana | |
| foldersFromFilesStructure: true | |
| foldersFromFilesStructure: true | |
| + |
Tools
yamllint
[error] 14-14: no new line character at the end of file
(new-line-at-end-of-file)
| - '27017:27017' | ||
| yorkie: | ||
| build: | ||
| context: ../../ |
There was a problem hiding this comment.
Fix indentation and add a new line at the end of the file.
The yamllint static analysis tool reported the following issues:
- Wrong indentation on line 34: expected 6 but found 8.
- No new line character at the end of the file.
Please apply the following fixes:
- Adjust the indentation on line 34 to match the expected indentation level:
- context: ../../
+ context: ../../- Add a new line character at the end of the file.
Also applies to: 49-49
Tools
yamllint
[warning] 34-34: wrong indentation: expected 6 but found 8
(indentation)
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (3)
- build/docker/docker-compose-deploy.yml (1 hunks)
- build/docker/grafana/dashboard.yml (1 hunks)
- build/docker/grafana/datasources.yml (1 hunks)
Files skipped from review due to trivial changes (2)
- build/docker/grafana/dashboard.yml
- build/docker/grafana/datasources.yml
Additional context used
yamllint
build/docker/docker-compose-deploy.yml
[error] 49-49: no new line character at the end of file
(new-line-at-end-of-file)
Additional comments not posted (4)
build/docker/docker-compose-deploy.yml (4)
4-12: LGTM!The code changes are approved.
13-25: LGTM!The code changes are approved.
26-31: LGTM!The code changes are approved.
32-49: LGTM!The code changes are approved.
Tools
yamllint
[error] 49-49: no new line character at the end of file
(new-line-at-end-of-file)
hackerwins
force-pushed
the
main
branch
2 times, most recently
from
8eb4284
to
b77c25a
Compare
|
@window9u Could you rebase your commit to current upstream? There were some changes in previous commit. |
this file also launches yorkie with docker-compose-full.yml
window9u
force-pushed
the
docker-compose
branch
from
24e55e0
to
d7908d5
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (7)
- build/docker/README.md (1 hunks)
- build/docker/docker-compose-deploy.yml (1 hunks)
- build/docker/grafana/README.md (1 hunks)
- build/docker/grafana/dashboard.yml (1 hunks)
- build/docker/grafana/datasources.yml (1 hunks)
- build/docker/grafana/grafana.ini (1 hunks)
- build/docker/prometheus.yml (1 hunks)
Files skipped from review due to trivial changes (4)
- build/docker/grafana/dashboard.yml
- build/docker/grafana/datasources.yml
- build/docker/grafana/grafana.ini
- build/docker/prometheus.yml
Additional context used
LanguageTool
build/docker/README.md
[uncategorized] ~23-~23: Loose punctuation mark.
Context: ...d Grafana. -docker-compose-deploy.yml: This file is used to deploy Yorkie by o...(UNLIKELY_OPENING_PUNCTUATION)
Markdownlint
build/docker/grafana/README.md
7-7: null
Bare URL used(MD034, no-bare-urls)
yamllint
build/docker/docker-compose-deploy.yml
[error] 49-49: no new line character at the end of file
(new-line-at-end-of-file)
Additional comments not posted (1)
build/docker/docker-compose-deploy.yml (1)
1-49: Comprehensive Review of docker-compose-deploy.ymlThis Docker Compose file is well-structured and integrates multiple services effectively. Here are some specific observations and suggestions:
Version Specification:
- The Docker Compose file version is set to '3.3', which is appropriate for the features used in this file.
Service Definitions:
- Prometheus:
- Correctly configured with volume mappings and command overrides.
- Grafana:
- Properly set up with necessary volume mappings for dashboards and data sources.
- Dependency on Prometheus is correctly defined.
- MongoDB:
- Standard MongoDB service setup with restart policy.
- Yorkie:
- Build context and Dockerfile are correctly specified.
- Command and port configurations are appropriate.
- Dependency on MongoDB is correctly defined.
Networking and Ports:
- Ports are appropriately exposed for Prometheus, Grafana, MongoDB, and Yorkie, facilitating easy access to these services.
Static Analysis Feedback:
- Indentation Issue: Previously noted indentation issue on line 34 has been addressed in this review.
- New Line at End of File: The file should end with a newline character to comply with POSIX standards and facilitate better diffs in version control.
Suggestions:
- Consider adding a newline at the end of the file to resolve the yamllint error and improve file compatibility.
- Ensure that all configurations, especially those related to networking and volume mappings, are tested thoroughly to prevent runtime issues.
Overall, this Docker Compose file is well-prepared for deploying the integrated environment with Yorkie, MongoDB, Grafana, and Prometheus. It is recommended to proceed with testing in a controlled environment to ensure all components interact as expected.
Tools
yamllint
[error] 49-49: no new line character at the end of file
(new-line-at-end-of-file)
build/docker/README.md
Outdated
| - `docker-compose-deploy.yml`: This file is used to deploy Yorkie by one command. | ||
| It runs Yorkie, MongoDB, and monitoring tools. |
There was a problem hiding this comment.
Clarify the deployment process in the new section.
The addition of the docker-compose-deploy.yml file is well-documented. However, to enhance clarity and usability:
- It might be helpful to include more details about what "deploy Yorkie by one command" entails. Specifically, what configurations or setups are handled automatically by this compose file?
- Consider adding a brief example or command snippet that illustrates how to use this new compose file, similar to the examples provided for the other compose files.
Additionally, address the punctuation issue flagged by the static analysis tool:
- The dash before "
docker-compose-deploy.yml" could be replaced with a colon for better readability and to match the style of documentation.
Would you like me to draft a more detailed explanation or example usage for this section?
Tools
LanguageTool
[uncategorized] ~23-~23: Loose punctuation mark.
Context: ...d Grafana. -docker-compose-deploy.yml: This file is used to deploy Yorkie by o...(UNLIKELY_OPENING_PUNCTUATION)
build/docker/grafana/README.md
Outdated
| If you want to set a default dashboard, follow these steps: | ||
|
|
||
| 1. Open the [Yorkie dashboard](https://grafana.com/grafana/dashboards/18560). | ||
| 2. Download the dashboard JSON file. | ||
| 3. Paste the contents into `build/docker/grafana/yorkie_dashboard.json`. | ||
| 4. Mount the docker volume to the Grafana container by adding the following line to your `docker-compose.yml` file: | ||
|
|
||
| ```yaml | ||
| volumes: | ||
| - ./grafana/your_dashboard_name.json:/var/lib/grafana/your_dashboard_name.json:ro | ||
| ``` | ||
|
|
||
| You can import other dashboards by following the same steps. |
There was a problem hiding this comment.
Enhance instructions for setting a default dashboard.
The steps for setting a default dashboard are well-detailed. However, to avoid confusion, consider specifying the exact location of the docker-compose.yml file and clarifying the example volume path. This will help users correctly implement the changes without ambiguity.
Suggested improvements:
-4. Mount the docker volume to the Grafana container by adding the following line to your `docker-compose.yml` file:
+4. Mount the docker volume to the Grafana container by adding the following line to your `docker-compose.yml` file located at `build/docker/`:
```yaml
volumes:
- - ./grafana/your_dashboard_name.json:/var/lib/grafana/your_dashboard_name.json:ro
+ - ./grafana/yorkie_dashboard.json:/var/lib/grafana/yorkie_dashboard.json:ro
<!-- This is an auto-generated comment by CodeRabbit -->
build/docker/grafana/README.md
Outdated
| To import dashboards into Grafana, follow these steps: | ||
|
|
||
| 1. Open Grafana in your browser (http://localhost:3000). | ||
| 2. Log in with the default credentials (admin/admin). | ||
| 3. Click "Dashboards" in the left sidebar. | ||
| 4. Click "New" and then "Import". | ||
| 5. Use the Yorkie dashboard ID `18560` to import the dashboard. | ||
| 6. Click "Load" and "Import" and the dashboard will be made. | ||
|
|
There was a problem hiding this comment.
Clarify and enhance dashboard import instructions.
The instructions for importing dashboards are clear and concise. However, to comply with Markdown best practices, consider linking the URL text instead of using a bare URL. This enhancement improves readability and user experience.
Suggested change for the bare URL:
-1. Open Grafana in your browser (http://localhost:3000).
+1. Open Grafana in your browser ([http://localhost:3000](http://localhost:3000)).Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| To import dashboards into Grafana, follow these steps: | |
| 1. Open Grafana in your browser (http://localhost:3000). | |
| 2. Log in with the default credentials (admin/admin). | |
| 3. Click "Dashboards" in the left sidebar. | |
| 4. Click "New" and then "Import". | |
| 5. Use the Yorkie dashboard ID `18560` to import the dashboard. | |
| 6. Click "Load" and "Import" and the dashboard will be made. | |
| To import dashboards into Grafana, follow these steps: | |
| 1. Open Grafana in your browser ([http://localhost:3000](http://localhost:3000)). | |
| 2. Log in with the default credentials (admin/admin). | |
| 3. Click "Dashboards" in the left sidebar. | |
| 4. Click "New" and then "Import". | |
| 5. Use the Yorkie dashboard ID `18560` to import the dashboard. | |
| 6. Click "Load" and "Import" and the dashboard will be made. | |
Tools
Markdownlint
7-7: null
Bare URL used(MD034, no-bare-urls)
There was a problem hiding this comment.
Thank you for your contribution.
- About docker-compose file
I thought docker-compose-full.yaml file was used in our CI process or other places, but it was not. Therefore, as you have suggested, let's integrate into one single file.
- About folder structure
Since we have created new grafana folder, how about integrating prometheus.yml file into that folder and rename it to monitoring folder?
Also, this approach was used in our devops repository.
- Grafana dashboard import
If there is no way to do so, I think your instruction is good enough for users to follow.
FYI: In our current chart, we can import Grafana dashboard dynamically with dashboard id.
yorkie/build/charts/yorkie-monitoring/values.yaml
Lines 95 to 102 in b77c25a
- remove dashboard.yml volume in grafana - fix directory name
There was a problem hiding this comment.
Actionable comments posted: 1
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (6)
- build/docker/README.md (1 hunks)
- build/docker/docker-compose-full.yml (2 hunks)
- build/docker/monitoring/README.md (1 hunks)
- build/docker/monitoring/datasources.yml (1 hunks)
- build/docker/monitoring/grafana.ini (1 hunks)
- build/docker/monitoring/prometheus.yml (1 hunks)
Files skipped from review due to trivial changes (3)
- build/docker/monitoring/README.md
- build/docker/monitoring/datasources.yml
- build/docker/monitoring/prometheus.yml
Additional context used
LanguageTool
build/docker/README.md
[uncategorized] ~21-~21: Loose punctuation mark.
Context: ...uns MongoDB. -docker-compose-full.yml: This file builds Yorkie and launches it...(UNLIKELY_OPENING_PUNCTUATION)
Additional comments not posted (10)
build/docker/monitoring/grafana.ini (6)
1-3: Configuration for Grafana paths is set correctly.The provisioning path is correctly set to
/etc/grafana/provisioning, which is standard for Grafana setups.
4-6: Server configuration is standard.The HTTP port is set to 3000, which is the default for Grafana.
11-13: User sign-up configuration is enabled.Allowing user sign-up is set to true. Ensure this setting aligns with your security policies.
14-16: Anonymous access is disabled.This is a good security practice, ensuring that users must authenticate.
17-19: Dashboard versioning is configured.Keeping 20 versions of each dashboard can help with managing changes and rollbacks.
20-21: Unified alerting feature is enabled.This is a newer feature in Grafana that enhances alerting capabilities. Ensure that this setting is intended and understood by the team.
build/docker/README.md (1)
21-22: Updated description fordocker-compose-full.ymlis clear and informative.The updated text clarifies the role of
docker-compose-full.ymlin building Yorkie and launching it alongside MongoDB and monitoring tools. This enhancement improves the documentation's accuracy and usability.Tools
LanguageTool
[uncategorized] ~21-~21: Loose punctuation mark.
Context: ...uns MongoDB. -docker-compose-full.yml: This file builds Yorkie and launches it...(UNLIKELY_OPENING_PUNCTUATION)
build/docker/docker-compose-full.yml (3)
8-10: Prometheus configuration is updated correctly.The command and volume path for the Prometheus configuration are correctly set, ensuring that Prometheus uses the correct configuration file.
Also applies to: 12-12
18-22: Grafana configuration enhancements are appropriate.The command and volume settings for Grafana have been updated to specify the new configuration and data source files. This setup ensures that Grafana is configured correctly upon startup.
31-48: New Yorkie service configuration is comprehensive.The addition of the Yorkie service with detailed build and command specifications enhances the Docker Compose setup. The dependency on MongoDB is correctly configured, and the port mappings are appropriate for external access.
|
Thank you for taking the time to review my PR. I appreciate your feedback and have implemented your recommendations! |
What this PR does / why we need it:
This PR introduces a new
docker-compose-deploy.ymlfile that allows running Yorkie, MongoDB, Grafana, and Prometheus together in a single command. While we already have adocker-compose-full.ymlfor running MongoDB, Grafana, and Prometheus, this new file includes the Yorkie server as well. This addition enables easier testing of a complete deployment setup.Additionally, this PR addresses an issue discovered on Linux (Ubuntu) systems where Prometheus couldn't recognize the
host.docker.internal:8081andlocalhost:8080targets for Yorkie when usingdocker-compose-full.ymland running the Yorkie server separately. I found that using the Docker bridge IP (typically 172.17.0.1) resolves this issue. A comment explaining this has been added to theprometheus.ymlfile.However, it's important to note that this solution doesn't work in WSL2. For WSL2 users, it's recommended to use the new
docker-compose-deploy.ymlfile instead.Lastly, I've added Grafana setup instructions to the docker-compose file and included steps for configuring the dashboard in Grafana's README.md.
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
File naming suggestion:
docker-compose-full.ymltodocker-compose-observer.ymldocker-compose-deploy.ymltodocker-compose-full.ymlAlthough the new file is currently named
docker-compose-deploy.yml, it doesn't seem entirely suitable for actual deployment purposes. The suggested renaming better reflects the contents and purposes of these files.Grafana dashboard import:
Initially, I attempted to include dashboard import in the Grafana configuration but found that Grafana doesn't support this feature directly. While it's possible to add JSON files for automatic dashboard setup, I decided against including them in the source code to keep it clean. Instead, I've added instructions for setting up the default dashboard in Grafana's README.md.
Does this PR introduce a user-facing change?:
Additional documentation:
Checklist:
Summary by CodeRabbit
New Features
Documentation