Add regional dr environment without a hub #907
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nirs
requested review from
hatfieldbrian,
netzzer and
tjanssen3
and removed request for
netzzer
|
Example run: |
|
Example run with the hubless env: |
|
Example run with standard regional-dr environment and basic-test. |
hatfieldbrian
approved these changes
Jun 15, 2023
There was a problem hiding this comment.
I wonder if the common bits with test/envs/regional-dr.yaml can be shared to avoid duplication
There was a problem hiding this comment.
With the current code no, there is way to include common configuration or "kustomize" existing env. Do you have any idea how to add such feature?
|
Need to test that kube object protection works, can be done with modified shio-demo.sh. |
nirs
force-pushed
the
hubless
branch
2 times, most recently
from
10e2974
to
37d7511
Compare
|
Changes since @hatfieldbrian review: |
|
Rebased on #903 after it was rebased. |
|
Rebased again on #903 after recent changes. |
nirs
force-pushed
the
hubless
branch
2 times, most recently
from
1269228
to
3bdd3ed
Compare
tjanssen3
approved these changes
Jul 8, 2023
Running this should be easy - you follow the instructions in Setting up ramen test env, and then you run: At this point ramen is deployed and configured on both clusters, so you should be able to create a VRG and add resources that should be backed up by kube object protection, or maybe a recipe. Hopefully you should be able to use https://github.com/RamenDR/ramen/tree/main/hack/recipe_e2e with this environment by disabling all the setup code. It would be best if we can convert the recipe_e2e test to something like basic-test so any developer will be able to do: And verifying that changes do not break kube object protection or recipe support. At some point this will run automatically for every pr in the CI server. But let rebase this pr first - it has some conflicts after recent pr were merged. |
|
Converted to draft because more work is needed to add submariner without a hub. When this pr was posted, the volsync addon was only installing the CRDs, but now it actually install the volsync operator and its self test require submariner. |
|
@tjanssen3 I rebased on main:
This should be ready for testing as described in #907 (comment) |
|
@nirs thanks for the instructions! I was able to get the drenv
I encountered an error while attempting to install the velero component, however. Are there any additional setup steps required? Test system is running ubuntu 18.04.2 with python v3.6.9. (ramen) tjanssen@eyewall4:~/go/src/github.com/nirs/ramen$ (cd test && drenv start envs/regional-dr-hubless.yaml)
2023-07-10 11:21:33,122 INFO [rdr-hubless] Starting environment
2023-07-10 11:21:33,286 INFO [dr1] Starting minikube cluster
2023-07-10 11:21:34,289 INFO [dr2] Starting minikube cluster
2023-07-10 11:24:00,633 INFO [dr1] Cluster started in 147.35 seconds
2023-07-10 11:24:00,634 INFO [dr1/0] Running addons/cert-manager/start
2023-07-10 11:24:00,635 INFO [dr1/1] Running addons/csi-addons/start
2023-07-10 11:24:28,702 INFO [dr2] Cluster started in 174.41 seconds
2023-07-10 11:24:28,704 INFO [dr2/0] Running addons/cert-manager/start
2023-07-10 11:24:28,705 INFO [dr2/1] Running addons/csi-addons/start
2023-07-10 11:24:37,703 INFO [dr1/0] addons/cert-manager/start completed in 37.07 seconds
2023-07-10 11:24:37,703 INFO [dr1/0] Running addons/rook-operator/start
2023-07-10 11:24:45,819 INFO [dr1/1] addons/csi-addons/start completed in 45.18 seconds
2023-07-10 11:24:45,819 INFO [dr1/1] Running addons/olm/start
2023-07-10 11:24:58,427 INFO [dr2/0] addons/cert-manager/start completed in 29.72 seconds
2023-07-10 11:24:58,428 INFO [dr2/0] Running addons/rook-operator/start
2023-07-10 11:25:14,411 INFO [dr2/1] addons/csi-addons/start completed in 45.71 seconds
2023-07-10 11:25:14,412 INFO [dr2/1] Running addons/olm/start
2023-07-10 11:25:23,838 INFO [dr1/0] addons/rook-operator/start completed in 46.14 seconds
2023-07-10 11:25:23,839 INFO [dr1/0] Running addons/rook-cluster/start
2023-07-10 11:26:09,831 INFO [dr2/0] addons/rook-operator/start completed in 71.40 seconds
2023-07-10 11:26:09,831 INFO [dr2/0] Running addons/rook-cluster/start
2023-07-10 11:27:05,661 INFO [dr1/1] addons/olm/start completed in 139.84 seconds
2023-07-10 11:27:05,662 INFO [dr1/1] Running addons/minio/start
2023-07-10 11:28:36,425 INFO [dr2/1] addons/olm/start completed in 202.01 seconds
2023-07-10 11:28:36,425 INFO [dr2/1] Running addons/minio/start
2023-07-10 11:29:06,126 INFO [dr1/1] addons/minio/start completed in 120.46 seconds
2023-07-10 11:29:06,127 INFO [dr1/1] Running addons/velero/start
2023-07-10 11:29:06,454 ERROR [dr1/1] Cluster failed
Traceback (most recent call last):
File "/home/tjanssen/go/src/github.com/nirs/ramen/test/drenv/__main__.py", line 118, in execute
f.result()
File "/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result
return self.__get_result()
File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/tjanssen/go/src/github.com/nirs/ramen/test/drenv/__main__.py", line 274, in run_worker
run_addon(addon, worker["name"], hooks=hooks, allow_failure=allow_failure)
File "/home/tjanssen/go/src/github.com/nirs/ramen/test/drenv/__main__.py", line 290, in run_addon
run_hook(hook, addon["args"], name, allow_failure=allow_failure)
File "/home/tjanssen/go/src/github.com/nirs/ramen/test/drenv/__main__.py", line 297, in run_hook
run(hook, *args, name=name)
File "/home/tjanssen/go/src/github.com/nirs/ramen/test/drenv/__main__.py", line 312, in run
for line in commands.watch(*cmd):
File "/home/tjanssen/go/src/github.com/nirs/ramen/test/drenv/commands.py", line 127, in watch
raise Error(args, p.returncode, error)
drenv.commands.Error: Command failed:
command: ('addons/velero/start', 'dr1')
exitcode: 1
error:
Traceback (most recent call last):
File "addons/velero/start", line 38, in <module>
deploy(cluster)
File "addons/velero/start", line 26, in deploy
"--wait",
File "/home/tjanssen/go/src/github.com/nirs/ramen/test/drenv/commands.py", line 101, in watch
env=env,
File "/usr/lib/python3.6/subprocess.py", line 729, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'velero': 'velero'
2023-07-10 11:29:24,435 INFO [dr1/0] addons/rook-cluster/start completed in 240.60 seconds
2023-07-10 11:29:24,436 INFO [dr1/0] Running addons/rook-pool/start
2023-07-10 11:29:37,709 INFO [dr2/1] addons/minio/start completed in 61.28 seconds
2023-07-10 11:29:37,710 INFO [dr2/1] Running addons/velero/start
2023-07-10 11:29:38,045 ERROR [dr2/1] Cluster failed
Traceback (most recent call last):
File "/home/tjanssen/go/src/github.com/nirs/ramen/test/drenv/__main__.py", line 118, in execute
f.result()
File "/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result
return self.__get_result()
File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/tjanssen/go/src/github.com/nirs/ramen/test/drenv/__main__.py", line 274, in run_worker
run_addon(addon, worker["name"], hooks=hooks, allow_failure=allow_failure)
File "/home/tjanssen/go/src/github.com/nirs/ramen/test/drenv/__main__.py", line 290, in run_addon
run_hook(hook, addon["args"], name, allow_failure=allow_failure)
File "/home/tjanssen/go/src/github.com/nirs/ramen/test/drenv/__main__.py", line 297, in run_hook
run(hook, *args, name=name)
File "/home/tjanssen/go/src/github.com/nirs/ramen/test/drenv/__main__.py", line 312, in run
for line in commands.watch(*cmd):
File "/home/tjanssen/go/src/github.com/nirs/ramen/test/drenv/commands.py", line 127, in watch
raise Error(args, p.returncode, error)
drenv.commands.Error: Command failed:
command: ('addons/velero/start', 'dr2')
exitcode: 1
error:
Traceback (most recent call last):
File "addons/velero/start", line 38, in <module>
deploy(cluster)
File "addons/velero/start", line 26, in deploy
"--wait",
File "/home/tjanssen/go/src/github.com/nirs/ramen/test/drenv/commands.py", line 101, in watch
env=env,
File "/usr/lib/python3.6/subprocess.py", line 729, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'velero': 'velero'
2023-07-10 11:29:46,191 INFO [dr2/0] addons/rook-cluster/start completed in 216.36 seconds
2023-07-10 11:29:46,191 INFO [dr2/0] Running addons/rook-pool/start
2023-07-10 11:29:52,260 INFO [dr1/0] addons/rook-pool/start completed in 27.82 seconds
2023-07-10 11:29:52,260 INFO [dr1/0] Running addons/rook-toolbox/start
2023-07-10 11:29:56,236 INFO [dr1/0] addons/rook-toolbox/start completed in 3.98 seconds
2023-07-10 11:30:16,789 INFO [dr2/0] addons/rook-pool/start completed in 30.60 seconds
2023-07-10 11:30:16,790 INFO [dr2/0] Running addons/rook-toolbox/start
2023-07-10 11:30:19,915 INFO [dr2/0] addons/rook-toolbox/start completed in 3.13 seconds
(ramen) tjanssen@eyewall4:~/go/src/github.com/nirs/ramen$ git log | head -n 1
commit 5fc835d6e7719d8269f2c35bcabdf2e40514952c |
I think you are missing the developer quick start guide: This is not your fault, this document is not linked yet from the main README or CONTRIBUTING page. If you follow the steps there you should have a fully working environment and be able to run the basic-test (which should be documented there).
We document "You need minikube version supporting the --extra-disks option. The tool was tested with minikube v1.26.1." here: I agree we need to check versions of the tools and fail early. PRs improving this are welcome :-)
Looks like you did not install the I hope we can automate tools installations at some point. This is low priority since this is a one time task for new contributors or when you rebuild your environment. |
|
Reporting back after some testing. Initially, starting (ramen) tjanssen@eyewall4:~/go/src/github.com/nirs/ramen$ (cd test && drenv start envs/regional-dr-hubless.yaml)
2023-07-11 10:36:31,422 INFO [rdr-hubless] Starting environment
2023-07-11 10:36:32,491 INFO [dr1] Starting minikube cluster
2023-07-11 10:36:32,905 INFO [dr2] Starting minikube cluster
2023-07-11 10:37:32,364 INFO [dr2] Cluster started in 59.46 seconds
2023-07-11 10:37:32,364 INFO [dr2] Waiting until all deployments are available
2023-07-11 10:37:34,170 INFO [dr1] Cluster started in 61.68 seconds
2023-07-11 10:37:34,170 INFO [dr1] Waiting until all deployments are available
2023-07-11 10:38:09,237 INFO [dr2] Deployments are available in 36.87 seconds
2023-07-11 10:38:09,239 INFO [dr2/0] Running addons/cert-manager/start
2023-07-11 10:38:09,239 INFO [dr2/1] Running addons/csi-addons/start
2023-07-11 10:38:14,549 INFO [dr2/1] addons/csi-addons/start completed in 5.31 seconds
2023-07-11 10:38:14,557 INFO [dr2/1] Running addons/olm/start
2023-07-11 10:38:15,930 INFO [dr2/0] addons/cert-manager/start completed in 6.69 seconds
2023-07-11 10:38:15,933 INFO [dr2/0] Running addons/rook-operator/start
2023-07-11 10:38:17,096 INFO [dr1] Deployments are available in 42.93 seconds
2023-07-11 10:38:17,097 INFO [dr1/0] Running addons/cert-manager/start
2023-07-11 10:38:17,097 INFO [dr1/1] Running addons/csi-addons/start
2023-07-11 10:38:20,576 INFO [dr1/1] addons/csi-addons/start completed in 3.48 seconds
2023-07-11 10:38:20,576 INFO [dr1/1] Running addons/olm/start
2023-07-11 10:38:21,343 INFO [dr1/0] addons/cert-manager/start completed in 4.25 seconds
2023-07-11 10:38:21,343 INFO [dr1/0] Running addons/rook-operator/start
2023-07-11 10:38:21,605 INFO [dr2/1] addons/olm/start completed in 7.05 seconds
2023-07-11 10:38:21,605 INFO [dr2/1] Running addons/minio/start
2023-07-11 10:38:21,852 INFO [dr2/0] addons/rook-operator/start completed in 5.92 seconds
2023-07-11 10:38:21,853 INFO [dr2/0] Running addons/rook-cluster/start
2023-07-11 10:38:22,407 INFO [dr2/1] addons/minio/start completed in 0.80 seconds
2023-07-11 10:38:22,409 INFO [dr2/1] Running addons/velero/start
2023-07-11 10:38:25,254 INFO [dr1/1] addons/olm/start completed in 4.68 seconds
2023-07-11 10:38:25,254 INFO [dr1/1] Running addons/minio/start
2023-07-11 10:38:25,616 INFO [dr1/0] addons/rook-operator/start completed in 4.27 seconds
2023-07-11 10:38:25,616 INFO [dr1/0] Running addons/rook-cluster/start
2023-07-11 10:38:26,317 INFO [dr1/1] addons/minio/start completed in 1.06 seconds
2023-07-11 10:38:26,318 INFO [dr1/1] Running addons/velero/start
2023-07-11 10:38:29,000 INFO [dr1/0] addons/rook-cluster/start completed in 3.38 seconds
2023-07-11 10:38:29,000 INFO [dr1/0] Running addons/rook-pool/start
2023-07-11 10:38:29,784 INFO [dr2/0] addons/rook-cluster/start completed in 7.93 seconds
2023-07-11 10:38:29,784 INFO [dr2/0] Running addons/rook-pool/start
2023-07-11 10:38:31,698 INFO [dr1/0] addons/rook-pool/start completed in 2.70 seconds
2023-07-11 10:38:31,702 INFO [dr1/0] Running addons/rook-toolbox/start
2023-07-11 10:38:32,265 INFO [dr2/0] addons/rook-pool/start completed in 2.48 seconds
2023-07-11 10:38:32,265 INFO [dr2/0] Running addons/rook-toolbox/start
2023-07-11 10:38:33,072 INFO [dr2/0] addons/rook-toolbox/start completed in 0.81 seconds
2023-07-11 10:38:33,751 INFO [dr1/0] addons/rook-toolbox/start completed in 2.05 seconds
2023-07-11 10:38:58,743 INFO [dr2/1] addons/velero/start completed in 36.33 seconds
2023-07-11 10:38:58,743 INFO [dr2/1] Running addons/velero/test
2023-07-11 10:39:00,326 INFO [dr1/1] addons/velero/start completed in 34.01 seconds
2023-07-11 10:39:00,326 INFO [dr1/1] Running addons/velero/test
2023-07-11 10:39:45,024 INFO [dr2/1] addons/velero/test completed in 46.28 seconds
2023-07-11 10:39:48,036 INFO [dr1/1] addons/velero/test completed in 47.71 seconds
2023-07-11 10:39:48,038 INFO [rdr-hubless/0] Running addons/rbd-mirror/start
2023-07-11 10:39:48,038 INFO [rdr-hubless/1] Running addons/submariner/start
2023-07-11 10:40:42,655 INFO [rdr-hubless/0] addons/rbd-mirror/start completed in 54.62 seconds
2023-07-11 10:40:42,656 INFO [rdr-hubless/0] Running addons/rbd-mirror/test
2023-07-11 10:41:15,166 INFO [rdr-hubless/0] addons/rbd-mirror/test completed in 32.51 seconds
2023-07-11 10:41:46,635 INFO [rdr-hubless/1] addons/submariner/start completed in 118.60 seconds
2023-07-11 10:41:46,635 INFO [rdr-hubless/1] Running addons/submariner/test
2023-07-11 10:42:40,185 INFO [rdr-hubless/1] addons/submariner/test completed in 53.55 seconds
2023-07-11 10:42:40,185 INFO [rdr-hubless/1] Running addons/volsync/start
2023-07-11 10:43:25,170 INFO [rdr-hubless/1] addons/volsync/start completed in 44.98 seconds
2023-07-11 10:43:25,170 INFO [rdr-hubless/1] Running addons/volsync/test
2023-07-11 10:43:46,807 ERROR [rdr-hubless/1] Cluster failed
Traceback (most recent call last):
File "/home/tjanssen/go/src/github.com/nirs/ramen/test/drenv/__main__.py", line 118, in execute
f.result()
File "/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result
return self.__get_result()
File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/tjanssen/go/src/github.com/nirs/ramen/test/drenv/__main__.py", line 274, in run_worker
run_addon(addon, worker["name"], hooks=hooks, allow_failure=allow_failure)
File "/home/tjanssen/go/src/github.com/nirs/ramen/test/drenv/__main__.py", line 290, in run_addon
run_hook(hook, addon["args"], name, allow_failure=allow_failure)
File "/home/tjanssen/go/src/github.com/nirs/ramen/test/drenv/__main__.py", line 297, in run_hook
run(hook, *args, name=name)
File "/home/tjanssen/go/src/github.com/nirs/ramen/test/drenv/__main__.py", line 312, in run
for line in commands.watch(*cmd):
File "/home/tjanssen/go/src/github.com/nirs/ramen/test/drenv/commands.py", line 127, in watch
raise Error(args, p.returncode, error)
drenv.commands.Error: Command failed:
command: ('addons/volsync/test', 'dr1', 'dr2')
exitcode: 1
error:
Traceback (most recent call last):
File "addons/volsync/test", line 168, in <module>
run_replication(cluster1)
File "addons/volsync/test", line 114, in run_replication
context=cluster,
File "/home/tjanssen/go/src/github.com/nirs/ramen/test/drenv/kubectl.py", line 90, in wait
_watch("wait", *args, context=context, log=log)
File "/home/tjanssen/go/src/github.com/nirs/ramen/test/drenv/kubectl.py", line 106, in _watch
for line in commands.watch(*cmd, input=input):
File "/home/tjanssen/go/src/github.com/nirs/ramen/test/drenv/commands.py", line 127, in watch
raise Error(args, p.returncode, error)
drenv.commands.Error: Command failed:
command: ('kubectl', 'wait', '--context', 'dr1', 'replicationsource/busybox-src', '--for=jsonpath={.status.lastManualSync}=replication-1', '--namespace=busybox', '--timeout=120s')
exitcode: 1
error:
error: status is not found
(ramen) tjanssen@eyewall4:~/go/src/github.com/nirs/ramen$ git status
On branch hubless
Your branch is up to date with 'origin/hubless'.
nothing to commit, working tree clean
(ramen) tjanssen@eyewall4:~/go/src/github.com/nirs/ramen$ (cd test && drenv start envs/regional-dr-hubless.yaml)
2023-07-11 10:47:11,776 INFO [rdr-hubless] Starting environment
2023-07-11 10:47:12,348 INFO [dr1] Starting minikube cluster
2023-07-11 10:47:13,295 INFO [dr2] Starting minikube cluster
2023-07-11 10:48:10,432 INFO [dr2] Cluster started in 57.14 seconds
2023-07-11 10:48:10,432 INFO [dr2] Waiting until all deployments are available
2023-07-11 10:48:36,275 INFO [dr1] Cluster started in 83.93 seconds
2023-07-11 10:48:36,275 INFO [dr1] Waiting until all deployments are available
2023-07-11 10:49:05,110 INFO [dr2] Deployments are available in 54.68 seconds
2023-07-11 10:49:05,111 INFO [dr2/0] Running addons/cert-manager/start
2023-07-11 10:49:05,111 INFO [dr2/1] Running addons/csi-addons/start
2023-07-11 10:49:08,141 INFO [dr2/1] addons/csi-addons/start completed in 3.03 seconds
2023-07-11 10:49:08,142 INFO [dr2/1] Running addons/olm/start
2023-07-11 10:49:08,709 INFO [dr2/0] addons/cert-manager/start completed in 3.60 seconds
2023-07-11 10:49:08,709 INFO [dr2/0] Running addons/rook-operator/start
2023-07-11 10:49:12,729 INFO [dr2/1] addons/olm/start completed in 4.59 seconds
2023-07-11 10:49:12,729 INFO [dr2/1] Running addons/minio/start
2023-07-11 10:49:13,376 INFO [dr2/1] addons/minio/start completed in 0.65 seconds
2023-07-11 10:49:13,376 INFO [dr2/1] Running addons/velero/start
2023-07-11 10:49:13,642 INFO [dr2/0] addons/rook-operator/start completed in 4.93 seconds
2023-07-11 10:49:13,643 INFO [dr2/0] Running addons/rook-cluster/start
2023-07-11 10:49:17,936 INFO [dr1] Deployments are available in 41.66 seconds
2023-07-11 10:49:17,937 INFO [dr1/0] Running addons/cert-manager/start
2023-07-11 10:49:17,937 INFO [dr1/1] Running addons/csi-addons/start
2023-07-11 10:49:19,410 INFO [dr2/1] addons/velero/start completed in 6.03 seconds
2023-07-11 10:49:19,411 INFO [dr2/1] Running addons/velero/test
2023-07-11 10:49:21,085 INFO [dr1/1] addons/csi-addons/start completed in 3.15 seconds
2023-07-11 10:49:21,085 INFO [dr1/1] Running addons/olm/start
2023-07-11 10:49:21,356 INFO [dr1/0] addons/cert-manager/start completed in 3.42 seconds
2023-07-11 10:49:21,357 INFO [dr1/0] Running addons/rook-operator/start
2023-07-11 10:49:25,602 INFO [dr1/1] addons/olm/start completed in 4.52 seconds
2023-07-11 10:49:25,602 INFO [dr1/1] Running addons/minio/start
2023-07-11 10:49:26,434 INFO [dr1/1] addons/minio/start completed in 0.83 seconds
2023-07-11 10:49:26,435 INFO [dr1/1] Running addons/velero/start
2023-07-11 10:49:26,530 INFO [dr1/0] addons/rook-operator/start completed in 5.17 seconds
2023-07-11 10:49:26,530 INFO [dr1/0] Running addons/rook-cluster/start
2023-07-11 10:49:29,762 INFO [dr2/0] addons/rook-cluster/start completed in 16.12 seconds
2023-07-11 10:49:29,762 INFO [dr2/0] Running addons/rook-pool/start
2023-07-11 10:49:30,582 INFO [dr2/0] addons/rook-pool/start completed in 0.82 seconds
2023-07-11 10:49:30,583 INFO [dr2/0] Running addons/rook-toolbox/start
2023-07-11 10:49:31,533 INFO [dr2/0] addons/rook-toolbox/start completed in 0.95 seconds
2023-07-11 10:49:32,504 INFO [dr1/1] addons/velero/start completed in 6.07 seconds
2023-07-11 10:49:32,504 INFO [dr1/1] Running addons/velero/test
2023-07-11 10:49:32,585 INFO [dr1/0] addons/rook-cluster/start completed in 6.06 seconds
2023-07-11 10:49:32,586 INFO [dr1/0] Running addons/rook-pool/start
2023-07-11 10:49:33,665 INFO [dr1/0] addons/rook-pool/start completed in 1.08 seconds
2023-07-11 10:49:33,666 INFO [dr1/0] Running addons/rook-toolbox/start
2023-07-11 10:49:34,338 INFO [dr1/0] addons/rook-toolbox/start completed in 0.67 seconds
2023-07-11 10:49:55,070 INFO [dr2/1] addons/velero/test completed in 35.66 seconds
2023-07-11 10:50:06,008 INFO [dr1/1] addons/velero/test completed in 33.50 seconds
2023-07-11 10:50:06,010 INFO [rdr-hubless/0] Running addons/rbd-mirror/start
2023-07-11 10:50:06,010 INFO [rdr-hubless/1] Running addons/submariner/start
2023-07-11 10:50:38,574 INFO [rdr-hubless/0] addons/rbd-mirror/start completed in 32.56 seconds
2023-07-11 10:50:38,574 INFO [rdr-hubless/0] Running addons/rbd-mirror/test
2023-07-11 10:51:16,969 INFO [rdr-hubless/1] addons/submariner/start completed in 70.96 seconds
2023-07-11 10:51:16,970 INFO [rdr-hubless/1] Running addons/submariner/test
2023-07-11 10:51:23,555 INFO [rdr-hubless/0] addons/rbd-mirror/test completed in 44.98 seconds
2023-07-11 10:52:22,935 INFO [rdr-hubless/1] addons/submariner/test completed in 65.97 seconds
2023-07-11 10:52:22,935 INFO [rdr-hubless/1] Running addons/volsync/start
2023-07-11 10:52:28,773 INFO [rdr-hubless/1] addons/volsync/start completed in 5.84 seconds
2023-07-11 10:52:28,774 INFO [rdr-hubless/1] Running addons/volsync/test
2023-07-11 10:53:18,507 INFO [rdr-hubless/1] addons/volsync/test completed in 49.73 seconds
2023-07-11 10:53:18,508 INFO [rdr-hubless] Environment started in 366.73 secondsI tried this twice, and both times I got an error, then ran it again and the error didn't occur. The deploy and config steps completed without issue: (ramen) tjanssen@eyewall4:~/go/src/github.com/nirs/ramen$ ramenctl deploy test/envs/regional-dr-hubless.yaml
2023-07-11 10:55:09,145 INFO [ramenctl] Starting deploy
2023-07-11 10:55:09,156 INFO [ramenctl] Saving image 'quay.io/ramendr/ramen-operator:latest'
2023-07-11 10:55:19,868 INFO [ramenctl] Loading image in cluster 'dr1'
2023-07-11 10:55:19,869 INFO [ramenctl] Loading image in cluster 'dr2'
2023-07-11 10:55:38,345 INFO [ramenctl] Deploying ramen operator in cluster 'dr1'
2023-07-11 10:56:03,596 INFO [ramenctl] Deploying ramen operator in cluster 'dr2'
2023-07-11 10:56:13,283 INFO [ramenctl] Finished deploy in 64.14 seconds
(ramen) tjanssen@eyewall4:~/go/src/github.com/nirs/ramen$ ramenctl config test/envs/regional-dr-hubless.yaml
2023-07-11 10:56:36,811 INFO [ramenctl] Starting config
2023-07-11 10:56:37,120 INFO [ramenctl] Creating s3 secret in cluster 'dr1'
2023-07-11 10:56:39,101 INFO [ramenctl] Updating ramen config map in cluster 'dr1'
2023-07-11 10:56:39,513 INFO [ramenctl] Creating s3 secret in cluster 'dr2'
2023-07-11 10:56:41,394 INFO [ramenctl] Updating ramen config map in cluster 'dr2'
2023-07-11 10:56:41,752 INFO [ramenctl] Finished config in 4.94 secondsThe only thing missing in the veleroNamespaceSecretKeyRef:
key: cloud
name: cloud-credentialsI had some issues related to using Podman and the containerd runtime on my system, but after trying again with Docker and and the docker runtime, I believe the issues were fixed. I did go through an extra step to ensure that the most recent image is loaded onto |
Thanks for testing this!
OK, this is a bug in the addon - we have to wait until
It makes sense, when you run again the status is available so the test succeeds.
I guess you mean this: I'll add it in the next version. Can we create the secret from the velero secret? Creating it from file means that we have
We don't use podman in the regional-dr env.Do you mean using podman to build the
Ramen image? why do you need to check this?
Why is this needed? The goal is to make drenv work out of the box without changing anything, so you can always use the recent version from git. Do we need to change something? add some option?
Great!, can you share what was tested and how? can we make it easy for other developers to run the same tests? |
Fixed by #983
Fixed in #984, please review. I will rebase this pr on both fixes later for testing new version. |
nirs
force-pushed
the
hubless
branch
2 times, most recently
from
2c2a936
to
06f82a6
Compare
nirs
force-pushed
the
hubless
branch
3 times, most recently
from
7879a31
to
a62e37a
Compare
nirs
force-pushed
the
hubless
branch
3 times, most recently
from
a335367
to
4314a13
Compare
To support an environment without a hub we need different flows when a hub is available and not. Refactor the code to make it easy to support this case. Signed-off-by: Nir Soffer <nsoffer@redhat.com>
If environment does not have a hub: - `ramenctl deploy` skips hub deployment - `ramenctl undeploy` skips the hub deployment - `ramenctl config` installs the s3 secret and ramen-dr-cluster-operator-config map on the dr clusters. - `ramenctl unconfig` delete the s3 secrets from the dr clusters. Signed-off-by: Nir Soffer <nsoffer@redhat.com>
For testing kube object projection and recipes in an environment without
a hub. This is not really supported but we know about some users using
this topology, and having a test environment makes it easier to support
them and make sure we don't break them by mistake.
This environment is very similar to regional-dr.yaml, with these
changes:
- The hub cluster was removed
- ocm components were removed, since there is no hub
- submariner is deployed at the end instead of of running on the hub at
the start of the deployment.
Because we run submariner at the end, and it takes about 140 seconds to
deploy and run the self test, starting this environment is slower than
the full regional-dr environment.
An alternative is to remove submariner and volsync from this
environment, and disable volsync in ramen config when a hub is not
available.
Example run:
$ drenv start envs/regional-dr-hubless.yaml
2023-07-10 00:24:52,538 INFO [rdr-hubless] Starting environment
2023-07-10 00:24:52,583 INFO [dr1] Starting minikube cluster
2023-07-10 00:24:53,617 INFO [dr2] Starting minikube cluster
2023-07-10 00:27:17,476 INFO [dr1] Cluster started in 145.15 seconds
2023-07-10 00:27:17,477 INFO [dr1/0] Running addons/cert-manager/start
2023-07-10 00:27:17,477 INFO [dr1/1] Running addons/csi-addons/start
2023-07-10 00:27:21,254 INFO [dr2] Cluster started in 147.89 seconds
2023-07-10 00:27:21,255 INFO [dr2/0] Running addons/cert-manager/start
2023-07-10 00:27:21,255 INFO [dr2/1] Running addons/csi-addons/start
2023-07-10 00:27:40,867 INFO [dr1/0] addons/cert-manager/start completed in 23.39 seconds
2023-07-10 00:27:40,867 INFO [dr1/0] Running addons/rook-operator/start
2023-07-10 00:27:44,417 INFO [dr2/0] addons/cert-manager/start completed in 23.16 seconds
2023-07-10 00:27:44,417 INFO [dr2/0] Running addons/rook-operator/start
2023-07-10 00:27:50,084 INFO [dr1/1] addons/csi-addons/start completed in 32.61 seconds
2023-07-10 00:27:50,084 INFO [dr1/1] Running addons/olm/start
2023-07-10 00:27:53,573 INFO [dr2/1] addons/csi-addons/start completed in 32.32 seconds
2023-07-10 00:27:53,573 INFO [dr2/1] Running addons/olm/start
2023-07-10 00:28:11,292 INFO [dr1/0] addons/rook-operator/start completed in 30.42 seconds
2023-07-10 00:28:11,292 INFO [dr1/0] Running addons/rook-cluster/start
2023-07-10 00:28:16,015 INFO [dr2/0] addons/rook-operator/start completed in 31.60 seconds
2023-07-10 00:28:16,016 INFO [dr2/0] Running addons/rook-cluster/start
2023-07-10 00:29:17,888 INFO [dr1/1] addons/olm/start completed in 87.80 seconds
2023-07-10 00:29:17,888 INFO [dr1/1] Running addons/minio/start
2023-07-10 00:29:21,088 INFO [dr2/1] addons/olm/start completed in 87.51 seconds
2023-07-10 00:29:21,088 INFO [dr2/1] Running addons/minio/start
2023-07-10 00:30:13,180 INFO [dr2/0] addons/rook-cluster/start completed in 117.16 seconds
2023-07-10 00:30:13,181 INFO [dr2/0] Running addons/rook-pool/start
2023-07-10 00:30:14,771 INFO [dr1/1] addons/minio/start completed in 56.88 seconds
2023-07-10 00:30:14,771 INFO [dr1/1] Running addons/velero/start
2023-07-10 00:30:31,209 INFO [dr1/0] addons/rook-cluster/start completed in 139.92 seconds
2023-07-10 00:30:31,209 INFO [dr1/0] Running addons/rook-pool/start
2023-07-10 00:30:42,475 INFO [dr2/0] addons/rook-pool/start completed in 29.29 seconds
2023-07-10 00:30:42,475 INFO [dr2/0] Running addons/rook-toolbox/start
2023-07-10 00:30:49,389 INFO [dr1/1] addons/velero/start completed in 34.62 seconds
2023-07-10 00:30:49,389 INFO [dr1/1] Running addons/velero/test
2023-07-10 00:30:59,842 INFO [dr2/1] addons/minio/start completed in 98.75 seconds
2023-07-10 00:30:59,843 INFO [dr2/1] Running addons/velero/start
2023-07-10 00:31:00,068 INFO [dr1/0] addons/rook-pool/start completed in 28.86 seconds
2023-07-10 00:31:00,068 INFO [dr1/0] Running addons/rook-toolbox/start
2023-07-10 00:31:03,056 INFO [dr1/0] addons/rook-toolbox/start completed in 2.99 seconds
2023-07-10 00:31:04,891 INFO [dr2/0] addons/rook-toolbox/start completed in 22.42 seconds
2023-07-10 00:31:24,670 INFO [dr1/1] addons/velero/test completed in 35.28 seconds
2023-07-10 00:31:32,393 INFO [dr2/1] addons/velero/start completed in 32.55 seconds
2023-07-10 00:31:32,393 INFO [dr2/1] Running addons/velero/test
2023-07-10 00:32:07,067 INFO [dr2/1] addons/velero/test completed in 34.67 seconds
2023-07-10 00:32:07,067 INFO [rdr-hubless/0] Running addons/rbd-mirror/start
2023-07-10 00:32:07,068 INFO [rdr-hubless/1] Running addons/submariner/start
2023-07-10 00:33:03,816 INFO [rdr-hubless/0] addons/rbd-mirror/start completed in 56.75 seconds
2023-07-10 00:33:03,816 INFO [rdr-hubless/0] Running addons/rbd-mirror/test
2023-07-10 00:33:20,852 INFO [rdr-hubless/0] addons/rbd-mirror/test completed in 17.04 seconds
2023-07-10 00:33:33,223 INFO [rdr-hubless/1] addons/submariner/start completed in 86.16 seconds
2023-07-10 00:33:33,223 INFO [rdr-hubless/1] Running addons/submariner/test
2023-07-10 00:34:28,427 INFO [rdr-hubless/1] addons/submariner/test completed in 55.20 seconds
2023-07-10 00:34:28,427 INFO [rdr-hubless/1] Running addons/volsync/start
2023-07-10 00:35:01,109 INFO [rdr-hubless/1] addons/volsync/start completed in 32.68 seconds
2023-07-10 00:35:01,109 INFO [rdr-hubless/1] Running addons/volsync/test
2023-07-10 00:36:40,248 INFO [rdr-hubless/1] addons/volsync/test completed in 99.14 seconds
2023-07-10 00:36:40,249 INFO [rdr-hubless] Environment started in 707.96 seconds
Part-of: RamenDR#902
Signed-off-by: Nir Soffer <nsoffer@redhat.com>
tjanssen3
approved these changes
Jul 26, 2023
There was a problem hiding this comment.
I ran through this on 26bd058 with the following steps, and it produced no error:
# create drenv virtual environment
make venv
# activate virtual environment
source venv
# set up regional dr environment: create dr1, dr2
(cd test && drenv start envs/regional-dr-hubless.yaml)
ramenctl deploy test/envs/regional-dr-hubless.yaml
ramenctl config test/envs/regional-dr-hubless.yamlThen I ran some local tests that aren't currently available in a PR, but follow the recipe_e2e flow of running bash scripts setup.sh, then protect.sh, then failover.sh, but this also requires configuring the environment for the dr1/dr2 setup, which I won't detail here. Updating these scripts (or making the local version public) is probably a larger conversation that should happen later, but that should not block this PR. For now, these changes look good to me.
|
@tjanssen3 thanks for testing! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For testing kube object projection in a setup without a hub.
Part-of: #902