Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Makefile target to set up all aligned components locally #1348

Merged
merged 3 commits into from
Nov 11, 2024

Conversation

IAvecilla
Copy link
Collaborator

Makefile target to set up all aligned components locally

Description

Since there's a lot of components now that are part of Aligned, this makefile target aims to be able to set up all of them just using one command. This requires to have tmux installed since every different part will be hosted in a different tmux window under the aligned session.

How to test

Just run make setup_local_aligned_all and do a tmux attach. You should see a few sessions, each one with the name of the component that is running. After a minute or so you should see all the components running including the explorer and the telemetry.

Type of change

Please delete options that are not relevant.

  • New feature
  • Bug fix
  • Optimization
  • Refactor

Checklist

  • “Hotfix” to testnet, everything else to staging
  • Linked to Github Issue
  • This change depends on code or research by an external entity
    • Acknowledgements were updated to give credit
  • Unit tests added
  • This change requires new documentation.
    • Documentation has been added/updated.
  • This change is an Optimization
    • Benchmarks added/run
  • Has a known issue
  • If your PR changes the Operator compatibility (Ex: Upgrade prover versions)
    • This PR adds compatibility for operator for both versions and do not change batcher/docs/examples
    • This PR updates batcher and docs/examples to the newer version. This requires the operator are already updated to be compatible

@IAvecilla IAvecilla self-assigned this Nov 1, 2024
Copy link
Collaborator

@Oppen Oppen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are missing instructions.

Operator:

clang: error: no such file or directory: '/Users/pelito/Code/aligned_layer/setup-aligned-local-target/operator/merkle_tree/lib/libmerkle_tree.dylib'

This is caused, most likely, due to a missing make build_all_ffi, or a dependency triggering that target.

Telemetry:

Running telemetry...
WARN[0000] /Users/pelito/Code/aligned_layer/setup-aligned-local-target/telemetry-docker-compose.yaml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion 
[+] Running 2/0
 ✔ Network setup-aligned-local-target_default          Created                                                                                                                                                                             0.0s 
 ✔ Volume "setup-aligned-local-target_cassandra-data"  Created                                                                                                                                                                             0.0s 
 ⠋ Container otel-collector                            Creating                                                                                                                                                                            0.0s 
 ⠋ Container cassandra                                 Creating                                                                                                                                                                            0.0s 
Error response from daemon: Conflict. The container name "/otel-collector" is already in use by container "e4f02f043038d70bf1febc21e97ad3ad66cb860a81b4f7f8d12647bda05905d4". You have to remove (or rename) that container to be able to reuse that name.
make: *** [open_telemetry_start] Error 1

This is caused due to lack of cleanup of previous runs. We should see how to handle those cases gracefully. A similar error that I often find is that localstack leaves remnants of other runs.

Explorer:


15:02:26.762 [info] == Migrated 20240830171448 in 0.0s

15:02:26.776 [info] == Running 20241010133259 Explorer.Repo.Migrations.AddIsValidField.change/0 forward

15:02:26.776 [info] alter table batches

15:02:26.783 [info] == Migrated 20241010133259 in 0.0s
/bin/bash: pnpm: command not found

Anvil:

pelito@pelito-mbp setup-aligned-local-target % make anvil_start_with_block_time
Starting Anvil...
anvil --load-state contracts/scripts/anvil/state/alignedlayer-deployed-anvil-state.json --block-time 7
error: invalid value 'contracts/scripts/anvil/state/alignedlayer-deployed-anvil-state.json' for '--load-state <PATH>': failed to parse json file: "contracts/scripts/anvil/state/alignedlayer-deployed-anvil-state.json": data did not match any variant of untagged enum SerializableTransactionType at line 1 column 496756

For more information, try '--help'.
make: *** [anvil_start_with_block_time] Error 2

@IAvecilla
Copy link
Collaborator Author

There are missing instructions.

Operator:

clang: error: no such file or directory: '/Users/pelito/Code/aligned_layer/setup-aligned-local-target/operator/merkle_tree/lib/libmerkle_tree.dylib'

This is caused, most likely, due to a missing make build_all_ffi, or a dependency triggering that target.

Yes, exactly. I suggest running this locally before using the new target, just as we currently do when setting up the components one by one. Including this in the target command might introduce unnecessary overhead that isn't always needed.

Telemetry:

Running telemetry...
WARN[0000] /Users/pelito/Code/aligned_layer/setup-aligned-local-target/telemetry-docker-compose.yaml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion 
[+] Running 2/0
 ✔ Network setup-aligned-local-target_default          Created                                                                                                                                                                             0.0s 
 ✔ Volume "setup-aligned-local-target_cassandra-data"  Created                                                                                                                                                                             0.0s 
 ⠋ Container otel-collector                            Creating                                                                                                                                                                            0.0s 
 ⠋ Container cassandra                                 Creating                                                                                                                                                                            0.0s 
Error response from daemon: Conflict. The container name "/otel-collector" is already in use by container "e4f02f043038d70bf1febc21e97ad3ad66cb860a81b4f7f8d12647bda05905d4". You have to remove (or rename) that container to be able to reuse that name.
make: *** [open_telemetry_start] Error 1

This is caused due to lack of cleanup of previous runs. We should see how to handle those cases gracefully. A similar error that I often find is that localstack leaves remnants of other runs.

Right. For now, just to get it working, I added a docker down command before running all the telemetry setup. Regarding the problem of previous data being present after a restart on some components, we should add a way to delete the docker volumes since this will take down all the telemetry stuff but keep the old traces too.

Explorer:


15:02:26.762 [info] == Migrated 20240830171448 in 0.0s

15:02:26.776 [info] == Running 20241010133259 Explorer.Repo.Migrations.AddIsValidField.change/0 forward

15:02:26.776 [info] alter table batches

15:02:26.783 [info] == Migrated 20241010133259 in 0.0s
/bin/bash: pnpm: command not found

I think that you're missing the pnpm dependency, you should install it previously to run all the components like the docs says.

Anvil:

pelito@pelito-mbp setup-aligned-local-target % make anvil_start_with_block_time
Starting Anvil...
anvil --load-state contracts/scripts/anvil/state/alignedlayer-deployed-anvil-state.json --block-time 7
error: invalid value 'contracts/scripts/anvil/state/alignedlayer-deployed-anvil-state.json' for '--load-state <PATH>': failed to parse json file: "contracts/scripts/anvil/state/alignedlayer-deployed-anvil-state.json": data did not match any variant of untagged enum SerializableTransactionType at line 1 column 496756

For more information, try '--help'.
make: *** [anvil_start_with_block_time] Error 2

This was an issue fixed in the staging branch, I pulled those changes and now seems to work fine.

@IAvecilla IAvecilla requested a review from Oppen November 8, 2024 19:13
@Oppen
Copy link
Collaborator

Oppen commented Nov 11, 2024

I believe with all its caveats it can still be a productivity boost and it can be improved later, so I changed my mind on requesting changes.

@JuArce JuArce added this pull request to the merge queue Nov 11, 2024
Merged via the queue into staging with commit d5c06dc Nov 11, 2024
1 check passed
@JuArce JuArce deleted the setup-aligned-local-target branch November 11, 2024 16:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants