From 7b7794f2c452847444f45daeaf166c2aaa8c9970 Mon Sep 17 00:00:00 2001 From: jeanluciano Date: Tue, 26 Mar 2024 15:19:20 -0500 Subject: [PATCH 01/10] first draft --- docs/ecs_guide.md | 188 +++++++++++++++++++++++---------------- docs/img/Workpool_UI.png | Bin 0 -> 28533 bytes 2 files changed, 113 insertions(+), 75 deletions(-) create mode 100644 docs/img/Workpool_UI.png diff --git a/docs/ecs_guide.md b/docs/ecs_guide.md index d188a89a..ba3c4360 100644 --- a/docs/ecs_guide.md +++ b/docs/ecs_guide.md @@ -14,8 +14,22 @@ ECS (Elastic Container Service) tasks are a good option for executing Prefect 2 Prefect enables remote flow execution via [workers](https://docs.prefect.io/concepts/work-pools/#worker-overview) and [work pools](https://docs.prefect.io/concepts/work-pools/#work-pool-overview). To learn more about these concepts please see our [deployment tutorial](https://docs.prefect.io/tutorial/deployments/). For details on how workers and work pools are implemented for ECS, see the diagram below: + #### Architecture Diagram + ```mermaid +%%{ + init: { + 'theme': 'base', + 'themeVariables': { + 'primaryColor': '#2D6DF6', + 'primaryTextColor': '#fff', + 'lineColor': '#FE5A14', + 'secondaryColor': '#E04BF0', + 'tertiaryColor': '#fff' + } + } +}%% graph TB subgraph ecs_cluster[ECS cluster] @@ -26,10 +40,11 @@ graph TB fr_task_definition[Flow run task definition] - subgraph ecs_task["ECS task execution
(Flow run infrastructure)"] - style ecs_task text-align:center - - flow_run((Flow run)) + subgraph ecs_task["ECS task execution"] + style ecs_task text-align:center,display:flex + + + flow_run((Flow run)) end fr_task_definition -->|defines| ecs_task @@ -41,17 +56,19 @@ graph TB end end - subgraph github["GitHub"] + subgraph github["ECR"] flow_code{{"Flow code"}} end flow_code --> |pulls| ecs_task prefect_worker -->|polls| workqueue prefect_workpool -->|configures| fr_task_definition ``` + ### ECS in Prefect Terms + !!! tip "ECS tasks != Prefect tasks" - An ECS task is **not** the same thing as a [Prefect task](https://docs.prefect.io/latest/concepts/tasks/#tasks-overview). - + An ECS task is **not** the same thing as a [Prefect task](https://docs.prefect.io/latest/concepts/tasks/#tasks-overview). + ECS tasks are groupings of containers that run within an ECS Cluster. An ECS task's behavior is determined by its task definition. An [*ECS task definition*](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definitions.html) is the blueprint for the ECS task. It describes which Docker containers to run and what you want to have happen inside these containers. @@ -76,47 +93,43 @@ You can use either EC2 or Fargate as the capacity provider. Fargate simplifies i !!! tip If you prefer infrastructure as code check out this [Terraform module](https://github.com/PrefectHQ/prefect-recipes/tree/main/devops/infrastructure-as-code/aws/tf-prefect2-ecs-worker) to provision an ECS cluster with a worker. + ### Prerequisites Before you begin, make sure you have: - An AWS account with permissions to create ECS services and IAM roles. - The AWS CLI installed on your local machine. You can [download it from the AWS website](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html). -- An [ECS Cluster](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html) to host both the worker and the flow runs it submits. Follow [this guide](https://docs.aws.amazon.com/AmazonECS/latest/userguide/create_cluster.html) to create an ECS cluster or simply use the default cluster. -- A [VPC](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html) configured for your ECS tasks. A VPC is a good idea if using EC2 and required if using Fargate. +- An [ECS Cluster](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html) to host both the worker and the flow runs it submits. This guide use the default cluster. To create your own follow [this guide](https://docs.aws.amazon.com/AmazonECS/latest/userguide/create_cluster.html). +- A [VPC](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html) configured for your ECS tasks. This guide uses the default VPC. ### Step 1: Set Up an ECS work pool Before setting up the worker, create a simple [work pool](https://docs.prefect.io/latest/concepts/work-pools/#work-pool-configuration) of type ECS for the worker to pull work from. -Create a work pool from the Prefect UI or CLI: +Create a work pool from the CLI: ```bash prefect work-pool create --type ecs my-ecs-pool ``` -Configure the VPC and ECS cluster for your work pool via the UI: -![VPC](img/VPC_UI.png) +Or from Prefect UI: +![WorkPool](img/Workpool_UI.png) -Configuring custom fields is easiest from the UI. +!!! + Because this guide uses fargate as the capacity provider and the default VPC and ECS cluster, no further configuration is needed. -![ECSCluster](img/ECSCluster_UI.png) -!!! Warning - You need to have a VPC specified for your work pool if you are using AWS Fargate. - -![Launch](img/LaunchType_UI.png) Next, set up a Prefect ECS worker that will discover and pull work from this work pool. -### Step 2: Start a Prefect worker in your ECS cluster. - +### Step 2: Start a Prefect worker in your ECS cluster -To create an [IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-custom.html#roles-creatingrole-custom-trust-policy-console) for the ECS task using the AWS CLI, follow these steps: +First start by creating the [IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-custom.html#roles-creatingrole-custom-trust-policy-console) required in order for your worker and flows to run. The sample flow in this guide is very simple and therefore you will only be creating one role, `taskExecutionRole`. To create a IAM role for the ECS task using the AWS CLI, follow these steps: 1. **Create a trust policy** - The trust policy will specify that ECS can assume the role. + The trust policy will specify that the ECS service containing the prefect worker will be able to assume the role required for calling other AWS services. Save this policy to a file, such as `ecs-trust-policy.json`: @@ -136,33 +149,34 @@ To create an [IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_role } ``` -2. **Create the IAM role** +2. **Create the IAM roles** - Use the `aws iam create-role` command to create the role: + Use the `aws iam create-role` command to create the roles that you will be using. The `ecsTaskExecutionRole` will be used by the worker to talk to make AWS API calls on your behalf when starting ecs tasks. ```bash - aws iam create-role \ + aws iam create-role \ --role-name ecsTaskExecutionRole \ --assume-role-policy-document file://ecs-trust-policy.json ``` + !!! tip + Depending on the requirements of your task, it is advised to create a [second role for your tasks](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html). This will contain permissions required while running the task. Because our sample flow doesn't interact with AWS you don't have to. 3. **Attach the policy to the role** - - Amazon has a managed policy named `AmazonECSTaskExecutionRolePolicy` that grants the permissions necessary for ECS tasks. Attach this policy to your role: + For this guide the ECS worker will require the permissions to pull images from ECR and publish logs to CloudWatch. Amazon has a managed policy named `AmazonECSTaskExecutionRolePolicy` that grants the permissions necessary for ECS tasks. [See here](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_execution_IAM_role.html) for other common execution role permissions. Attach this policy to your task execution role: ```bash aws iam attach-role-policy \ - --role-name ecsTaskExecutionRole \ + --role-name ecsTaskExecutionRole \ --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy ``` Remember to replace the `--role-name` and `--policy-arn` with the actual role name and policy Amazon Resource Name (ARN) you want to use. - Now, you have a role named `ecsTaskExecutionRole` that you can assign to your ECS tasks. This role has the necessary permissions to pull container images and publish logs to CloudWatch. +### Step 3: Creating ECS worker service -4. **Launch an ECS Service to host the worker** +1. **Launch an ECS Service to host the worker** Next, create an ECS task definition that specifies the Docker image for the Prefect worker, the resources it requires, and the command it should run. In this example, the command to start the worker is `prefect worker start --pool my-ecs-pool`. @@ -177,8 +191,8 @@ To create an [IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_role ], "cpu": "512", "memory": "1024", - "executionRoleArn": "", - "taskRoleArn": "", + "executionRoleArn": "", + "taskRoleArn": "", "containerDefinitions": [ { "name": "prefect-worker", @@ -194,11 +208,11 @@ To create an [IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_role "environment": [ { "name": "PREFECT_API_URL", - "value": "https://api.prefect.cloud/api/accounts//workspaces/" + "value": "" }, { "name": "PREFECT_API_KEY", - "value": "" + "value": "" } ] } @@ -206,31 +220,28 @@ To create an [IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_role } ``` - - Use `prefect config view` to view the `PREFECT_API_URL` for your current Prefect profile. Use this to replace both `` and ``. + - Use `prefect config view` to view the `PREFECT_API_URL` for your current Prefect profile. Use this to replace ``. - For the `PREFECT_API_KEY`, individuals on the organization tier can create a [service account](https://docs.prefect.io/latest/cloud/users/service-accounts/) for the worker. If on a personal tier, you can pass a user’s API key. - - Replace both instances of `` with the ARN of the IAM role you created in Step 2. + - Replace both instances of `` with the ARN of the IAM role you created in Step 2. - Notice that the CPU and Memory allocations are relatively small. The worker's main responsibility is to submit work through API calls to AWS, _not_ to execute your Prefect flow code. !!! tip - To avoid hardcoding your API key into the task definition JSON see [how to add environment variables to the container definition](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/secrets-envvar-secrets-manager.html#secrets-envvar-secrets-manager-update-container-definition). The API key must be stored as plain text, not the key-value pair dictionary that it is formatted in by default. + To avoid hardcoding your API key into the task definition JSON see [how to add sensitive data using AWS secrets manager to the container definition](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/specifying-sensitive-data-tutorial.html#specifying-sensitive-data-tutorial-create-taskdef). + +2. **Register the task definition:** -6. **Register the task definition:** - Before creating a service, you first need to register a task definition. You can do that using the `register-task-definition` command in the AWS CLI. Here is an example: ```bash - aws ecs register-task-definition --cli-input-json file://task-definition.json ``` - - Replace `task-definition.json` with the name of your JSON file. -7. **Create an ECS service to host your worker:** +3. **Create an ECS service to host your worker:** Finally, create a service that will manage your Prefect worker: @@ -239,22 +250,23 @@ To create an [IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_role ```bash aws ecs create-service \ --service-name prefect-worker-service \ - --cluster \ + --cluster \ --task-definition \ --launch-type FARGATE \ --desired-count 1 \ - --network-configuration "awsvpcConfiguration={subnets=[],securityGroups=[]}" + --network-configuration "awsvpcConfiguration={subnets=[],securityGroups=[],assignPublicIp='ENABLED'}" ``` - - Replace `` with the name of your ECS cluster. - - Replace `` with the path to the JSON file you created in Step 2, `` with a comma-separated list of your VPC subnet IDs. Ensure that these subnets are aligned with the vpc specified on the work pool in step 1. - - Replace `` with a comma-separated list of your VPC security group IDs. + - Replace `` with the name of your ECS cluster. - Replace `` with the ARN of the task definition you just registered. + - Replace `` with a comma-separated list of your VPC subnet IDs. Ensure that these subnets are aligned with the vpc specified on the work pool in step 1. You can view subnet ids with the following command: + `aws ec2 describe-subnets --filter Name=` + - Replace `` with a comma-separated list of your VPC security group IDs. !!! tip "Sanity check" - The work pool page in the Prefect UI allows you to check the health of your workers - make sure your new worker is live! + The work pool page in the Prefect UI allows you to check the health of your workers - make sure your new worker is live! Although keep in mind it can take ECS a few minuter for a service to go online. -### Step 4: Pick up a flow run with your new worker! +### Step 4: Pick up a flow run with your new worker 1. Write a simple test flow in a repo of your choice: @@ -272,15 +284,64 @@ To create an [IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_role my_flow() ``` +2. This guide uses ECR to store you code data. Create a repo with the following command: + + ```bash + aws ecr create-repository \ + --repository-name \ + --region + ``` + + To have prefect build your image when deploying your flow create `prefect.yaml` with the following + + ```yaml + name: ecs-worker-guide + prefect-version: 2.14.20 + + # build section allows you to manage and build docker images + build: + - prefect_docker.deployments.steps.build_docker_image: + id: build_image + requires: prefect-docker>=0.3.1 + image_name: + tag: latest + dockerfile: auto + + # push section allows you to manage if and how this project is uploaded to remote locations + push: + - prefect_docker.deployments.steps.push_docker_image: + requires: prefect-docker>=0.3.1 + image_name: '{{ build_image.image_name }}' + tag: '{{ build_image.tag }}' + + # the deployments section allows you to provide configuration for deploying flows + deployments: + - name: my_ecs_deployment + version: + tags: [] + description: + entrypoint: flow.py:my_flow + parameters: {} + work_pool: + name: ecs-dev-pool + work_queue_name: + job_variables: + image: '{{ build_image.image }}' + schedules: [] + pull: + - prefect.deployments.steps.set_working_directory: + directory: /opt/prefect/ecs-worker-guide + + ``` + 2. [Deploy](https://docs.prefect.io/2.11.0/tutorial/deployments/#create-a-deployment) the flow to the server, specifying the ECS work pool when prompted. ```bash - prefect deploy my_flow.py:my_flow + prefect deploy my_flow.py:my_ecs_deployment ``` 3. Find the deployment in the UI and click the **Quick Run** button! - ## Optional Next Steps 1. Now that you are confident your ECS worker is healthy, you can experiment with different work pool configurations. @@ -289,26 +350,3 @@ To create an [IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_role - Would an EC2 `Launch Type` speed up your flow run execution? These infrastructure configuration values can be set on your ECS work pool or they can be overridden on the deployment level through [job_variables](https://docs.prefect.io/concepts/infrastructure/#kubernetesjob-overrides-and-customizations) if desired. - - -2. Consider adding a [build action](https://docs.prefect.io/concepts/deployments-ux/#the-build-action) to your Prefect Project [`prefect.yaml`](https://docs.prefect.io/concepts/deployments-ux/#the-prefect-yaml-file) if you want to automatically build a Docker image and push it to an image registry `prefect deploy` is run. - -Here is an example build action for ECR: - ```yaml - build: - - prefect.deployments.steps.run_shell_script: - id: get-commit-hash - script: git rev-parse --short HEAD - stream_output: false - - prefect.deployments.steps.run_shell_script: - id: ecr-auth-step - script: aws ecr get-login-password --region | docker login --username - AWS --password-stdin <>.dkr.ecr..amazonaws.com - stream_output: false - - prefect_docker.deployments.steps.build_docker_image: - requires: prefect-docker>=0.3.0 - image_name: .dkr.ecr.us-east-2.amazonaws.com/ - tag: '{{ get-commit-hash.stdout }}' - dockerfile: auto - push: true - ``` diff --git a/docs/img/Workpool_UI.png b/docs/img/Workpool_UI.png new file mode 100644 index 0000000000000000000000000000000000000000..755dbe97048ffbf048654a1875006781382f5ed6 GIT binary patch literal 28533 zcmbSzbzD?y*S3MANOu~*&GXAt?`eN+W%Wp@c)+l@3Zo#f9Ycd3w{R1sMD=+Zf{!h(4Ea7xx2GdUmwkIpii<8r8Tf*Z$zQ_Fc>p3%nN?v@Td)4Ip$>3&EQ8&6kxp*?u!g{m z+CFS!ebw;BxKBPGC5#Y2j;br?M()&al~g(eY4W`gvTr&wpK_@Xth1Tqu3by_n!1(< zKsR#^^;-TU%XYog6+Ot(a3}Oe2jki9Fq6PD{7{oV7dy3f+Y{wh@<(lW;@S|+!q>UD zNt>oGEe@8uzxxw!o^Fm6R@O(kFt7RF!>Pb7eg5xW(BEMQe_j`xFnd&2{HD5qO2oZP z7vhj*Jym6O{zHiMMz8mv^fAMWx)Z}Df@xwt^=&%>N~;5D)@!g5eJb!&2^F8s=M;W> zgY7dC*4Goq-j4#eM)H*(i6pLtk+60e^h)Sel<#r8sREgir`?zu0+1+LnBKk*>TFHF zD~e`z77TC1Vu+vhLC(Ska;Rm4oWZlMN!D`>bz9@bDbu#Kvoi#;)C*$Qmxq>Iuu?;D zXnJ)WnW1-iknq*fh`#oS*J`pwz1z-|ZT7^;_T&hS_g+g4Dyg~`=xN1qgEjeFDC&L6@IutJ*ZG`Gb`k5u zPgV|!P5Yd0wAUNTeoldOPmKd{UfBy=pG@1Q+!MTvx@c!O zZ^hxPQdbtgsuDh5&)E@vb;e#hvuNPH5CrLUQTV?~0q96(JpRvbr^dM}ejHAn6cz%+ zaO9lV;Jps;N>9Q`io5>-MWk~V4cG-{;AMLhBMh`f4~dv%FQMQuhXmtNF%B%%L`=Id zQt;cQ>cl#xC@CQUO>hwPJSsa(UL5kcIsUP7^f`;YZ**sp)Vp({#Wq(5GYsdSVKa{F z#=jchN#rz~TX++pi9B732J^lf1>vXD0b>E0C+&aA){rLT%H07T{ib@47@h{6OQxqV zTmJcp)V#FuWa^0x(BfGR=(cgb~uCO)>PKhgF?4vAUr<%)b8% z?`(M&tu?6uUrwt~@2c9*@A>JJR9Zk)#?i1st-&beG$Gy-@4Nu@JcV-lhw00E+Mgy@ zbRON%92Frf;_5?vCM&)#ykjb4ouGUVimn!e$ zmOgVCNgh1BBE4abK+ zZX{1pAa%q766MZkGodp7W^U!;3JE6@$fNr&vKIcloVlrJOLVKKGGZK;f97olQ<{s~ zD^H00p=Z`Nk<WW0~qL zK?OTxhkO=8nb@$|N?Xn;$(Ty|2E{IlZ2haRQ5VqCX4I5os=$zc)9GTkW0I+Bk#R>D zq8eu4+(qpi0Xc`7?_RTPj)Yv3NcAu-V{|S0uXmct<7Xb3b8k!hsvNt ziwvI=#p6&G*QfNR4!~bHay1&0J6S7VJW3+5C*cgyGZ{i#WUkneK&fv|;S<`>LEFR^qxf-fSzEP=8}# z{IMJRyd2a7Sgw2>&&kL$!|U@TVPIzck{mQGZ>l0Z{rA<@<3)~_rE0l7_J)!x|5`PH zuk&EVOl@K~oL80Pzx1o{k&1a?PbBx_8|pxRM_S=|m_5N=O_!U}s7rR6xk*>c2gHB# z-2Znm`rO5frD&7c2nuoA*qMVB)|vShc%4~qqP-Ib;>Xbj&n(1|Xh|toq{j339dff! zbY}GXhud#Kw=rzS zz|fmGXw>HG(0o}@Cbd#0z%A^uj%T;f+Cj>T6hL~iWR=i)Z3}{7`q$1Toe|0;HiZIW zo<{=w&Z|Ymk2ss9839$D7du;)zkYCnF+-%>@K)#J__6(`eD>tky#?|5bW!n%SUFOw zHUH9M)1F=gNejCD1bMyN_t zbWPjqKK<7CY|BoiatL+l<&Hm(k=tUZEcEV!^`mLCMR1IhY85UU67XXj0N3mb!HfMX z(eApn^v%5|EV?x|p=@qO=^46P&@(kO{x zAEc;x$uGd%Io@w~Fw?JhsjJ0rv`QqN$xY$4-aa@u=o5_$t)Ru+Ndd@T6jc@Y7_*_WoAw|mmW)V@h zW@KOIP$Cx>>{g_lQA+c=06!1B>+6O~8dvAMR)>zeS@$Y|<&C!awfJ>IX^N;0H?qSQU^i3J25qp4OEXW)#h_yzX0*yj9dU z3vJ$-^Ff+a&Twe&Rc_5*SXUB1{gq`W>@qQu05}he0Xj&;t_NbL7(wT;oZRj~4!DXo zu-e_{_H~GjH#IhP$0AP4miEl2KQ^sLzm?A!Ljq`OF(3OmMEadB*Ot>Z@TTIr4{BZ797Z84 z*WsK^PTD-I8lYXtR0o9t`vPub_T0V<->I(6m=iL7ORxD4Op#Q_BBnpVO+L?Dt2%6^ zl%A7pdZpMFoO@pUgeZOHn(S%Zso{N57U8(~fJ(sou%EuS%MMyCr#DEpdH4Y%y$k}e z?>?OK0+-d03aqp?Jzqw3EqAN?G#@0+^3UW0=6v6_D)b)Camb^+9#;E0Jh9V0PWa~) zw>gh|sf-W)FQ$FBiVaY-*F6c$>Ya)yd~IVT`n@66$*HMV!B@h&-lkUsJ9G@?zE?Cm zGwRBX%UgaCI6IML@y+J_U&b?}x3KKV&t=Xfhj9rJbKP@tzFvK*yh-tjb3Iy5ufI)BaIKgEvxp4*YSBfl67}l3U}p+QtMtH zrjg`%R>Y83)Y|PU^&^k>NtlBa2*LBm16svzI=sky-d&wWi$@*$_&gStnPHP#@cp`* zl~$&KUZRS4h*!0uEVi+8kt<3#nD1f>((AH-L?b8jO|s#g!kcqmbk2Y4(y1)ojT*n}18)rqpGmQA zvPR+zKGOTDTm9}6BtuQq*G(^@Z9v^D4c)A~_ zw>2(2ne_u7huh$Vh#>^S@6;&XYx6r9d*A!p0aumYO*zW`67>RibJls9tVTM=$@1%A zFXoPz|5`>>oG@*uZAl&tiu_b%!FWI&3!#K{Wzq))K<*)Trr|r#T?0PQ6`+u_Ub{Xv z`4pIdDugo!KzHXlLpChCDA)pB+eI2w-uj|EvV<@9?lwxQ459NsL@g^98L8(Fl!b7w zX(L}z4tbyKinrs97Eb5hZ=gu!wN!!Q4G5i2E)MyYoddmPfk&RLW%!j-EK39g*A1j0 z9ffyBi=M|WANpN;jj|gwi~9g!|w#q03o_-4x#(Nyk>`v=@ z+G0RZX!UA~XRyH-dnL;^HbAIux2daGj6L=z9s1BHPZ)+|;}C80wQnGuLvf9Xw_%M< z5bkp;vi5$o`$Xh8WE{^rlD+94UW4uq)mSlI(V>6lY}l)J6xjG`ViVJUKIU>3$@)0% z;r;n9{Pv5Jbnom&A+&^k6)qbU%8S2@LTF4QDy)PU~a*b=)Od)yq-5 za}9>$B7OQhS8m;3RCNaw$>@rxhtU?4P?kp_oyh0?^I@8zqsd}`kkmSzr3onbr*-M^ zUPSL{XyUjVT;yb;(zjyW3s!C-oI?G`_5lCMjB_Lyrd$yyB;T}_J$Sc7!Z9cP z^ZIjv!LNiwxgr+keV;d`JLBA<#JOa$R?Z}8o(f`b31q=vBJK`XXIWA!5dON3q-}Jh zxu}IdIP|*1<}}Kjs5ym)&Edm>#MogHNs&BLAA4%0I=Tt#{R=J4P?D^k@(;4Cqvlt zH-^Tvo!VB}gy@viCNXVnTzBi{o7!KbCnMjwVgXx~{#=9a1!m&Fl3`w1DodAE`qpvp z@^J8_c2XS1rM5-6ufErgnfaN(-j6HA?lDJ4Dk}6aV7J73a+Q$Ea=^q zKVf)DzpLIFS)&SZ`gjmfW#1fP+xnvJ1n##hMaVyUgrSxDs)^yL%|x|v2ln(W^S0n) zL?>0Lk~TRZnRL^9y&{DqwO-r2b!p=ig#{yKNi!;}{e79^QgB|X%a+?m=8ehKxF;0E z^boJ71kWk&YrmLs$I^~~4&tp>DSX8lNO4z4D`Z(Q$BPopXHZPnKDcYP_fKbn}h_qUQnPbT^s@FdfmKA)tO?8F$PKC_$o z5md3;?8gLNOS|S_*xRmD?w+76e{c5qzMGZTHpP2uH}a39!9Gr&z@E&(BHbfaC+)je zU?_$N-FlLLs_!`#7U%s2#m>lHxbXYkqLm;ohnCK^Ah{xqVJO(JlFKP0IG22zXKzTh z(e_8=F%Aw9cAH~Yah2Q;D*J{Ww(uia1#HxDDU3-(rv~mTTq@9zV%a<6+(qjp)oOh= z=mQoHBRP+p-t^CS#_`Q>F@6ni*Dt>FKdBQR&;ZYrD$r*(WPNqK9=aY@3eE6}fw!bE zeepe|^&1ST+|Q^g{pGh%{h=b9I~a_QB&U7S5_hj79P6_UgC^ChCfdTsW;y!z?3v@4 zMIP^=Z;TaYC9>-2=zI;WT9e9$r0@-^ka?b@(tL3#|q0_@n2pnTR8O7GqUvQA$P?=jvpyp zXb&JUi!yzbw6sO43kyW?22niE+jc;G1^FdcDQy;mM6Z>6T^wh((>+R;~r#1JZFo_n$F892xpTI&wsnM@gL# z36&^8H4>jy+Kh>pt45;0;F*2HN=es^3j5Ybk~~Z**?UHHnA{cvdRrK$O3nCa(wnjJ z2TVpF!;)C$elDXp+lI+e8YFd(&=sk?h6);T+TPyY@oJpd`&5>Rad9{by0+LV9lfB6 zALxsT6~_4rpUCz4Y%t*Da?)$pP0Wy2axWs|`;^K|kanu-WqHhEv6NSfWY}J+)Wawa zGKu~6^Dq6&GA}vn@UQ^MI(TJH!9@ii0Zj>(g zHN{%z?ax5#_yxxtlm$o;FX;)wrX^Ba{qH+}hMwt%I;>8KcQr}pW9jHkT61PLY?3s7`$B-&e%v=?wu-+v$)EIVq~n_3 z3#?wl}^=hZ>gvma?#OgXT?Wf~825pUsj$0BN>b;`kEwDGsq6zGdR@`dz z-;e5~K#D>haes)j@BjXG3x=5@?~66)6HJ0Hk4}UOG0Kil2OFSPB?IM{lBbjktd|hI z^MnQy0_wRWqD^(%FZAx$JGz=bhgX>4h?kFavNonPnX}*TtBX30m-y@;89U8|RARyM zk$k3vj=0QMbVzNyecJ4@@kTr}v03)^zLAkkHq1D3IX#<(n>pYuMc-@gS>@#6KE5=3 zj+A@^m0s@d$lg--iTt(=532OJRq#7 zU~J4o$%pW&8pyT%45Ge%s@_7VlpK0DeL0KnFV>Rw&zxDlw10&@${i+C!f_^J{}TSYBhB(?b6RT z`Fr`+<@_-ExPkg0Vl3(STW=Dlvdug}BXuu*nC?UoMi82j;?A??PYR4y6J;Mn^&IPP zNPEi+*v6{wSK;#&8?kfWRUNm=&de@C=ilO*?t4@3fQwCb7WU?=S}qXz=1-AlM}%qL z^Y1m)JN>S1@dUjq)#5Q5y6AmERbOqKuKFS_o@kQrmS-NO?;}i8l!?iURDe+9$E&XT zsADo5=h1gIpE@tGc&-$TGePJrQ&fCkA7(6Llwcn7nDuDbTMf{~2cDZ(ppnTwnS~~^ zKVQa!#i8Go(;EtSdltq*ydaPP88V49eL*;3L{Ow#8>RE1`SO5D7o+D^^0hC2vuK!~Hz2vDVY05*4 zUb(79?)SD6W$8g|HmOg$ZJa)OIXX{-VCsauXT5-<+``A0x-f7lDU~cPO&_5pa~Y{E z&UM|P8kgym{#9i~R5@V0f#(lpA~KUm{xKz&mc6&&^FFp~nf~f772=m$YoI*p>o~S< zAq|4;T$T}xACEiEuvAVkU{RluZTndLOV<5KUvpisf=H}=yvI3xL&!H)wiVFJs#2RE zF}fP<^~rVYHw9%=2Vilr#mE2#;can3TV$*#ErS`kmDP*!#A%`YhD`5##|FsX@J z|1A*zmG6;G=M$QhHdYn|5wn6N1rJcvVU;{0viWl5&GcpDv(|HiCX2&9Ik%LxdeatG zY*vpZf88PNeKwmhmTSTGH&!P-AGv{`ng5uzi{knDF%_YzMHmkO*8yOP`;eYbAWi=S zpKjf`m*0VtY~*GooouT9P2(;WI^}&m%(f}kabxa}Ju(jRZ7T`-X{z%Rf<&+~msa{u z5+bkERQeoNOkbxppohO8|2kjab6*sDw{oz_7;oqst7SozN+$V4*C_(hsz9{phOEQ~~nX#qA{lteWh>6!B zm(tM$4?5nb)#$86686)hny-C8sxo}*Ez)Fow>E>om>7j5j!zffb1^^nDHqf__~M8Y zvx>(NTjbTQ7~5|H8f-=#zt?e4*YC_s@wC_rmA(UWC1#&e8FkI5d4;l49+hj7epg}N zc4im)eQ`|yDx=<1IUqF2DAshEU3!mG!Pg2_#41g|KsPE%=o(&ow;5%`Lx$cPXi}E^ zeMhP^()W7KzV&EK>xx}*io+_zH8?1ehtjWpH>FLiy_ZrIVKIKJR;xhzb^0@l9+_CO z##_P=uZ-he$cos0JI;d3_FB`0TUpJ(dpa@;#(5yLMc_AkNSba^1$!L5y{XNrDBiAA z|MG1QqnpAlg2bbVPvl!XNNU@Y1g-iMVZM^DHtC&FtDv6A8RhR&G6m1rO$1Thc?bx` z29C-)u&Fa+5I{1Lz>>ZSC5Do6l7YTqzFB5wB1lXS2g_^L$@vVcvp3|RAG>M zTt7#E=j_Wsv9N;7;dWNE=jHy|PBKkT?;&Y$Y4hC3CQl$%+`3UkjGJUeuxS`iJIz&} zOQ|~gN#OecslNiinWp%lst$f3hn65xhPX)G?5&YI{%>^_5*}a%K+Ejd1LcsRx?}rY z;7Z;%+R_vX`#rR5CL1#YDtg`OWy{$j!LaJUkjEPajve<9PJJqj+ zNk+ma-h$n7>lBE{DQMf~7?pWu&)laukD-w{6I%$(*%l6?pnAVwwnp+CbL)bm@ko!A zv)Q?u=`Nf3`lJR(`@I8Fe(uv3clGWBErWH8@KSfPOKF@t;LV3cVBwRAHrq6EDxpv4 zGumDq?`e$G=2M-O_C40Eu*F}~u|+^SELgR}9R%C_p>DRE(1_y?@eB!E>z&mUwQKa+ z@UohqnjemCvF z>1}_U0R`=QaU)Fd?TP;Pd)eUw72Ia;gu5O0Qf>>L@-6b@S<&+~F7{g62GSf94Q9auUL*4oK7I}lt1A=32db^W{cT?Io$BPTA zXBA+M=|^=L2A&(Zx=X!J;{_JkI9ep5jbG;vbZQ-ZBDFT%C(XR#I+LSX5Omo5-lg+8 zEPIfxgnMWp`Di}D?=s(aAAK=7*wk}%PsH3}2MF!-FSd+{C_%bfm$Bn8_T&tj`}4w& zrWbT1XN+`6iHsGV?A?FciY)bO9B@{qvncQmtPg|2JUFu6^MpPk_zS86QJ@`M-g3(k z|7E^RQ7AoIU)`U;runQd=5>Ka%gi11AUSYq(c+z;J1JHUi5%$~Ybv+O8qlY22t^Zk$kO4frGR zZbhv_LfxZoACnsN{jO~J_AFxG!veD~rIteTG89vX_UgPju}|F9%2ug^D$D$PpCVte zuzKk=R8fdvXQynp2ZamEc+PYwaJ6Ht%@2qsuyOUe0I`!u1uMqR-HNM^y zYrt7NNZ&|)0bOnC9*sWK`*0WAjpf5bR|>LV@AF<95pYE)4@a>EQfS|dMOM#Si1Duo z;oda<%PXmFZt?A3p<9I3-v-1kmaDYNQBKZ*A3PlE^Y`~6yx!X>r7xA>zFvbkQ};dt z^)h7KSK!V~Vxe0Qz&rUCBtngsks3{v8FDx}y$Wh~bSPX9>lg;*bQaCuyeGXMN zq;CMFuP6XWGq*@mllmsm>W}vIlyo`kzuYo6;yp23?R6GQ)D1BAxx^hcZ=8c)oHy^M zCIcX=%!Mi>vNw?COnRUh_dM2phnfkYkmv63D&B_0L}0hIg-D5zj%NirQ5@z z(eby!|4Hfry#$&eL3_?{1N>F#!F;*TZbnkD_;2pkdraadKr`s^u5OaIKxV)=C01Z- zI{urz^&AbG5NJkBi0^-4OK}6^B(D=xAaHZW?qLDV`1zvw=KWq#d~izc0k}4lwRW>Y zwZ2#93KFh20q*B$VF}NZxy|$@E6nuI4pu6anCt_RAit;$s^29L&j|?pqmX}c$M#YP zpWiOgF3;`-{ys;?mkg+R@>Hnbx;c;VkGBlE*NZ1#7A=N*RqCZww&Zi&>_juk?W5O6 zFa^TG(~3N;o5U0PJG#OjY>r^s9a`ga#f|S0#BbDe8`CtWxs#=-mFdP*(cj4p0w#Z0 zEsiDRrp&Mb(p0YPqH5a>p~RNnj+L3ZBnPJcGw@8fnr zn5e>~G5(7d|YH=CgiwB4k9J@e)$Sv>?qx@z=m&%+zdz?TQw&PGZ@e1q8m zV*f4eiC;7Xv8CSv^aI4SeY2~2z7qTBaqdF){!$kd06h|dvx#*m0FKmd)A=TAEfCFq z6s}J2rxIMcp&jtK%PIK+(%Bty1Vtl&EJXQK&TXDdS#$PYeU*!w0q8c?@az3( zKgg`xG`ng60K<13?@Yt00rctW0)DMTogwI)2VD$f-A2MuC;Y$ZiuFJktWL|oqvLpE zqzji;WMZeR?SWmi=!pa1JNSMOAo_H9z5;AAompQOy%e))28`+oqUpje90@QB94>a7 zn}3sgOxj3SS~#0dTlJl~8Aw;YspjYbOf7y^PvZ4xvGbn*CtwhlPHcwPX5trsY&9VO zz7eld3W>*4y@t0fcK`@OHvA0ei_^|jHEXa&MR-No4hwX`8E33mm!IFfkK8&S$Vhd=V^vIrPt{*0ih9GY`RXL_j)3V;d|{bD0Hf#0C1XGUo}9v0iPX#8$;y1MgKj2$ zfci(YmTl;Jxi2d(?8)ZMsm=&LyCqNMVfialLO|0r0~op73xLNqdkiz&EonM`7hWqD z$7lyoI>Ev_RdIzXS*_-|9qp|^+)FDn0;GKDcQR=)8$##1{-)NhQOD|Q{pWfK1HY?d zDW807Dp7+*PuJS55~vUa2K3zXUAV982D=TEB`Shj=c&)ROIKjT-`j-0m!`~RqL@PG z?$Wk945!jdxkj(n9smK?0snj$GT|}f6_8n$b%cI~y4Y>g;WRY`&0O9xdCY{d?3~x%Upu*JCG{)Ae=7_8vvl3elxEw0Ni9&U0Kwu84J7+ zL^jnf0b%s+6wnV;Pw_VIQLeJ-ChOAIZSwX2F5aD#dfDiC9Lre_cZ4xu^fBIteeE$c zn%3mlFsDlNUS$5zfyxYcu(-w;tEe{_IQhhw=1+SB`fYE*?`_$WF~8PYE~*@OWqOBf zl^%oEJ=3Q{(E|{v6VU=R=eP4eKDecdgTB2~`5f>l!YRbH@=f(SyCi)#$Lq^AaT<;+ z?H)B|e$r3Z>cIY_7SJ!!)95kYguWF1!7G4?$(=LSiZJO94gc8QWr?u@ z&Dq02@!cqKlzU|i(e8+{xNTB@y6Cht6Nx7hu~R0w3ors-=K(wQQoGhbtk5SvsqB6N zm`7fnQif9x^Xr;8C82Iya;_h(ipks^t%2bnei)jW5D^sMs_D3w7~Oetp3<<0{y@2h z;P?r>D6ufx!X`(CRt5^n@8_9&KIAnXN^%`giL$+D# z2kt9m^R?@5qHf*dQuh!0sJPN^Tt;pC4seHbHv$Ax>u@K8mfx~Ro(6zH<%cXQjMe7v zT>UvI6OYiAHyfn6?31Mc$Sy(U&0l2P=j4!PK;RI7ywxh8^|FaOh{|l7pKq1!%w&bo zZkCZ(ejjv&>947b<|(F_uW37nmzk3+2=pK(Ei!CL6=%MMn<#Rm;AHIudC$l8nJWOa zhfLiD_wtM8>kIMTL65aG3sNCs<(`k#9H0=$uk1|Odt`=a9ppj0@AjigE*QiH7V_>V z0le8!;uA|fZ#-=F9(=@zs)Sr?LTc3}tkgbVCBB!hL)%QB6 z_{oGxRP``GB2ERo1ZVN<%gPmY_359v`qEn%$ZZDs#Lr`2ma5D|h*X-GWq)Rv4PR%` zI7hgl06PQ$n9<`(jv4r3RfjRVx$;R{XTSS^L*i8YQ{x_Fdc=6-o?F+i=Q`wMMsfO-5(Pfik z0Po5gd*@fy3z{$*yT0!S6vWvY*6!tuYOjfThcRDqA_hVIk7XjtE;GHa*2J&n1`-1( zu*I$217a9o@tI+@=9yf|Z1e@|LpTsb;Y_a`8}5R{3CG=>ML-FVS8aCU!b9c8r=?2V>t;dsoE=MlMMJe?u zLAOy=+5Pg+d2BrU9to;t+7>gq7Q(1p5iQ1)C)s!Z^kh1IJLctCvj+q@+YceF6*{8; zUY+z)%4Nr1MX&s0Fm8U?*9r+W?U9da)HBNEdqMsA+87t@M9Knu{n)Qf*$hC`Ht!N% zJykGm_mhxV__FTD_Ri7&Q7F|WKu0pQ`vEGG_%10YMIbqeT{5iApo%I>TnrBPvgROk z<&~NI+Msa`b=Z#`Fend2|p*3Fc${Tl|>9#jyTp~Hkdlt{Mda_XUEG9tue6)-b3UN{<2uH|b% zyH&DM#~ms!+x4d(9v1RmFD*P|i!_?VENnJ_V7w-H_u*rRQ}!)-?Jkukjv3Mv68_u( zr5_{Z(R(#&wiXO(hxb9DC<(bA#U%1%Mmo;n!;byDu+MiQvOm3=IA#7y*7{YAT=k>Q z1*N_j}|%j=+nhc zj%7`)6GvI}d8kFSha&~Ee3D>XnfSF>d3+Fzyqwl$Ee%|Ax`@hbmnLa~a4nGAeFZgw zjDF_~6axugOyf1*V(5C_Z?Puwk9TLktzae@7b^0Zd|7s?>K329XJBYR)ZrJu@J*6@ zQOfAWr)*k)0w6viG~y6-oDFdB%Q7G#AuQUu zSn7i?zB870j1O^r{@g6sh<$)p7xF+UF>uCLb7siffM|Tfa8-r&;@sz+PJ3-a)g%1z zav4LCcE(o{Dt-9z0c8W+pjwa}W&k)o*@0SXXs2d+U|>H4H`jr|+-E6*f@fRczAC*x z+CBQqxzxdVQiVEqeSBu)7UQ8rys$J5*0>dt*hs9(C2TvrO?g%X?AYeDfliwnye`V2~W{ApG8ZPDu>Lc zXE5)~U%ltQ-P$f!mBwTU2lzw6Qdc1-ZKi~_{@2>D@Go*sv`7D;0{_;e%sn(;Fe$u4 zKCzy5*LPXwN?@S)?ATRor6&nMu6!U{4ayk&cCDt!#Vvd2n`!&ClBOd)tpO-cw)V-{ zxh@jb)i~=YOS>two6+1!xXsxx)uxz+iW9yriqmU57p_ll6mz{g$K#f#wJRR0`M${^ zg4BRjIkvUJ)Ph;d9fI|vJ#cYKvBR(fyvMIAIFhOpF%iuz#=(Tt)$vAkW{Rw>nPMye zO`@o{L#n)`H7qvb5k9e*8&y>OicBdk++#R!o6uV}FA0LK;jU^BXHib%G$z&->sH{|V4=rPq7C$>?|kCMYv%FVJ+=D4OOZ=~W$p9gtm5ozpZ1w*g|O@9@?t3NFYSl5N z`-~~3bN2%){6Hh>1OqW}Ei&Ik_jGD(GAmJcfh3Q!nRfjgkIgbmK}to+aKen#Xvd;V zNIK^MM=FQ`*iTlgZ*x(jX{~&dg0(Z2?shDOkq9=BVoBf1CA&vnXg%R5NNLCicQlll zcm`szks>Z~2L0G>r*g}9u%I%f!yj=c8p^ex@<_GMU0MY{KIv_;w4Av~>GBa~uiZcq znRz7F0_ZV3S>Sn#A_F8aM(mL4QD}VsLKg71? zyIy>EZsgWl23b9Wd{W){@usI)bx~bT(|BVNbghM{fuT0;0xo{NiE=-LH%Pk|K@V%F zW^Z4(x%4(pAJioo?5_5w>=w(L%`)vatn)Lo>~=<^tSgCM;~|Ow-iDUPmP3Akk^^*i zs^U65J+Yth{8)4J~85B@elg<=l5x$uPGDeemszV_HK6B-4PR;MEJ;D z?U7(~^;3NTyD%FPhSrsWBQ@VCO`#Rxo{!JCXTJ^YZ8X6EB3>M%!YqknPGQXwLptjy z_Q7_At(Hx8f~~$e%Zr*dE8S7s3Qd8t^K8>t>RR9b(`A`VZ4$WSQeMiP;qN0Qb0jq- zS}`msIFIZ;daseMk-sE#lgAv-!ivkQzo7N)ylR2+!}m)u^mcvk63IEE$^uTnnhmW& zCY{O;z^f?u17}`q4J7G8qkeJ&x7`Ht@$9?n^a*?P`zKx2?m>{8!uevG+PNbP{(eK} zL)&X7GW{ziXCH$aZ|O&NJ~uO@#X7=t>zo*xN355swsutfv0hBD8=YNcEBk?4 zNciib#kQ98)03voXVwfnM~aUV=jcgTH2}e@skD7lZ8>C&JbwSM#4c^Bq%9rl0o&f0 zSubCjO|3hey}Z}#zU6pV3p#%Cx^pQ?X!PpvM0Y02)y;v&0=10r*^>hg>m)&&;P%`Sr(;Yx9o%c-{%%sBXwe7oW-%a+} z6KV|j9Gq#@P>gd3*H#_4%z-)iVfIeP3;90V$Se%HGC4Qai^!rkHI|zLqE}M2yK_5N zUZxg;@phqqgsA>O91@vS*uprW{PxX0k*V@+zuVxC<9z&X?9w_s*o$m!1C>_BI)mBl zb$B;g)IyF!zkY^wu6Y7n1Qo7*w*a5L2x46Vo6XgkISV&++1XO>-a#3izJ%2uE0u(S zhh)kq;m7chnbT198MWr8q>hz>F9EljAXX4k3 zl)Oc21!pkWHtc&q$})Ut`kRDL?Teiq_AA@k;)Rm~yI!IOf3Ho)SwfFiV^OOE7jTDb z?UcEGLu8^B1@FG~`_jfuNv#3Odvz)1DYSPm%IV;h}QnT0LVV)sy(qd#JQ#o`w|e*Nb^5_n7Y^7WF`A zFh!Ur#pzzMs6~1Tw{5WQI)D2cs9Q1fV1Ej)+wo?eAKU_B@Lz*{L9pX!lZ7Z=~}iT?NW$D`dzCWae3x|X~y z4%01r`>C;zpX{YxTXn}n59=f-u33+8*Y#qeYoXp`uDbqQwj+oAnL<+SK)(K%Zi4mB z3ZKo$H;B5{eW3}u5;oMKiP=HNFWI_CG#P_e8>QA+v$iMJ3!o}KX06F$Yxh9i#@(fozM0zw1h#7|4GnOtbtT+=ws->_fE;Pg*4R;SB*O!wuCZTt zC!`#GxxOyw)#aWnE*+XGhL?xy8s=tAt272ps}$EwtJKvP66p<{-&3B9Ldt z=1!bBE~K#zIZRY=H6x97_2|uma?kZsba=jqOtiDDwOZ5;SgEWD;Us&_Wy zN8Cw^XTSO{7z5e|g%n*9=%Rdj5{Lf6>Y(9rBAZUSyRv=mD-xT>tm8+dNnXyL26N~a zz0oDxMRO%H5GMV(;ckOk_9Z@zqLqX>p@D=_1mE&mWLb}~u4bjgFme#K^Oykzgy5r>`a7HgwxZMaq@6|{l3S1-ty48psELX5*~ z(!}Kd%X0@}cs>+M?)MAAnp^HcM3U&7!dWXVtC*|i#HZK%uQ%FRTS0ARh4shPLUm&_ zsK!tyUGx1XwYOL5UOlaM%#J*CU);TSm6%X&l7?fL{hI`>p<;Mmxvt?gX00!Dw$UkNS~%RfFVX)`N8mqF58Uw9^@G9?ro+IkM2qE7HrlTNR$ zAZ}j_w`L-b2q51Xli&L^8UYYZ97tGBPYkh5#|p*}$1tz@8e6cV*LJxzGe~vyQ3NIL zc(Fm_)D94#FHy_AY(!t}OWyK5w~DeJ%KmQX*;;aVCYhpL=ajJD-B#FvbA@xI5J@?v z2Sly6C$gpGf9J%`a5l3!2_v+-cE^5knNcN+`+Z0eyJCWJoDBkrXpSpAL$Q!S?XO(s=Z|4F~2nRxsR01!*~Z_Jf|pu5Bq5F*?R9xxi%^J zxwvkQ)-*l)va11i_84aa(w+REZBpR{syXMrrO`C}p^0u!t%fxK7#AI0LTRBHk|I?> zf#HifBZCK9XkU|QHGl6(nNi|&5et~&Z}&&d}Tjaoih`x%;*YIWbV2RaL(L%lS<2!9DrwGxZcHh z;0iJ?)~(&UtD`fUgHJJ!M|;^wBeePL#ZC)qNY-%T*5S9DdPe`I!vR@TmgWH7Wr+fH zxUa;9*77{wiA|CZX0;+-CZsT;yqOj#J zFk%)gUg5ByoWPPW+d-B=Gp!ViN4En|1MM;2%oE+K(yJ?Qv-y>2BsLd~lbY|ty*~{2 zC~n?BQZ*35H*^LeRDno-logW)W;iYI*no^=V_=&Hk0n>Yr52bt-a-~L?Sy931cYY;uAtZwgbEMkZ0{lzjf=u9bsyHCIYhtKV z9Pmc&T773^p-rqt!2zNcK4)MhWE!`pyZKzt&>-GQIqi#Ia|V!MVvsfIQJFJP9X+>= zX7cE>F!)0?a#{UI7hu;l0XUFtoH9x8?y^^BCc|7YbaXK@$zk080xi;Qj1RVr=h)`^ojw`B?ypJQ_ z(BE{ER15dPWcb`=9>k_)WQ{1^9Rs$g@MkRs<$z;|jBA=uCI&$rRsVJ)1oA%hJdJedB9J@k5+n7d^H302)Ecp z1DA=A0~ei7E9$9o$_X8zZK@otKi{nzN%qF)XK49rgF^Y2i7|Lc+0?4LIPwwE94!W) zMqcb%x$sojIN`EJ$cd!fkP^dW=&NXbmZ#Z?v$BE;M$DSjf{c z@ERfricfi#Y&(sS;s95LBq5k7L-J#!rZ3kAfoCgl`6MT*Pwq}#pny!bds?rAOc>U=QS-GT zn#*N<;QBqOV(P1kW3nqXOEI%07mToT@RtTiM2i2nt%j!n+izu003d_^3tN^FM z6(_SJN>Wf_Y#YIg%yZ+j8=n zFgjF&Ko#|*hvx8R!~xQDcXGOaS#J1|X91hE?5e|=wH_u-cPCrdVrb3XUKzB-0-$z; zZ^M6#OI=Ftq~7_ji&Dq=U+hb?H!Z}7h|(B~Rapr{D{l@UQAf}4Hq}-S(g&Ow!mMMP zHnLP5$=wi8m4lW`X-_~nA7!gS9GR1!K)M>CfRZdClCONY2TEmuDi3HuOS9VgiUf0C^$kE>-H=3ywDDSXdl-l!vm;F zA8XpsN5ui?>;?Ey9b$fcONXNj=`lo63DUCjmIYMtd1ERcZ}WS(@hPfJ)JswgvGMeC zuF5rT!5QPgi8Sq$2!dh#J>a_u(AatHPEQ$s1g!rKA7LhSjtz!P-~8z7IGh3cf<+O% zU}<6w3`9r^#Ap8o(?ReZs&%X%$OEjGgsA0?yJ=!aHjI26IZ%&ZAATk8!q0=d12WW& zp5|F~oWy6pIB0}Azw!F@!D$mDKJPi7wehA$VoK#B0*dV^UUOpEtD$VL*tryT5lKTc zUS-+zQlfOY8iC|s`r?~*vF||O4)Al-+!FoHykp{^;{bw*(-V@EcJ)ALz6xYcR68v4 zRvo$bmYK%>Ph;t`R$vqm`Ay-~0?-ExbI$Z9zQE3=+c2C5QmDx3q6CxF?}0^K6aNXO z*A76!cs_F^G$J;3V_oibq&Vk1SKV+}b8(t1cujr;dS(pd`wam@#7_VcwPr*OM9L?3 zwuXc>pX*!0h3~bqMR)>Q_S5ZFn+_Ul49y~=rDOVIfiU9WqRh2IF#dxR`eDmuVXUhCDXfpjbL4qvN2 z!n(1O2@B&jUM{s2=zeMX^31QS)qbV>*};!WjH}iy_L*9H=6vwb!SVJ$llL)+i@<@? z0UU3|Iv8X>~aARx}6ov@B>f! z)AQCPf3WYY0zJ7pk)-c@@fW480lF!yi^cny{q#4OH_DRSN#U*RlB%5zw+DWR{aT&V zuyn=mt-Nm;)FsDkQ3t7^ZP_ADjw)e+;x|K5`?ah4@3;RnRi0Zyd+S`(nlc9c>u!5^9jz$6MI@3305CC#&XYXdE2(^O++7^tf4%ydRAb9# zfHveP!Q9bpOwkrT9s}LfbQ2efjo%WgUQbY$70<6wX86L) z#ylgQym*1Zq{h4H+61Vx1lV5tjyn=2i)Z(j%sO$23J>Ro?q~yKCc07f4s7j6Iluf( z1FvJPYn0*%n@MVCxia@1`8)8W=)3?th?>B%giwK;FMuGR@06tU9xd=+@LD7lu`4F+ zN=>5r!H7357NPnyG|SAV%+le&_MB*1PwL_7bb7hgF0m3#;A&+#f!i3y$cnYhrH62D z*#<)RYpLUW`^eJ;9dYjkXxB0QVcDJf_0CR%q`gEKjqdPrVw-~Sg4tx3X##fK_0_-7 z4o9sn>Pj_h_BdeOcc`eMZP}{PEf3OFV_VzLa*%MNiH<<=ZEoy4v zRi73j@Q~zvs1nRstqSCuI;oXtT{}_HMU4zZ?Z^A3tyapSxag89M#LD#gN}b61fmos zF)&sLhqu|n{6*)teAVDs4(f@(s;ixA|NiHGp{yXp2l!)3&b*Cu3lqtHhQG2(U*J?o zD+sl|xaq*me?3WBuFZLSPP28YH3JXe^%z8AM1VQKlftMfa1=zMno~at>%F;bH9*D7 zUb6`P%)dm?IV5>(dH<3!!K~ljxKT=BMhnlA6WGsaO%yrZ$Uc1V{oO+dmqu-d z_#?A|eb-vMvN7ZmH$^Y4q5AgocpHX4s8|{vt=gm3aU;$sT$+#LEJsZ!GX+^6TNi>J z?;3GjL5s#7Q;K@=Y6fEzX66cWI3znEA@(^|A2q-1X4|bIG}WXb^tr-ouCQ!q?P{E> z%MspcX6MJmIwjM`mFP;NBP~n-{+D__ub%ZB(jnzBRDw>L7)vvA=^;INW5iZMWZcaV zbcf#bsKtF8+>X!iLwp>_WIc+S76tf}Npl)^*+Pbr)9)V2s(I*qH!}5$I!G3aVM(Bb zJ6?l?{7v@&lkd_mBxy3x%;{v-18=tV)gqt9j`AtxHy{o}w_JKms>67T0DU(Zs zf5p3BJ)eYJDb|xdNS?tRr2KR7>#$MaoY$6ka^gTeLayWgWMD@dpcWAwY{xtddUjVev_nunQ{hg%sfe!D88W;R;8 z(OE+i$(`mpVIf;id)lrJ@Dd++5)kMVInJnf(*on##iIrMMQ3yX9zvvD(Pu@p=EIO0 zvtv|uXv|0|n0HzhLBQ9v=bt^?^&yy~9pgDi#&($8$=?JGm{hOv&;2XizRmp08Ze#^ z!oX{5iQfN6rN$e`+G>VyRLxpQQA*te4z3>eG8isN$^ea(52(r*3>{NRRP*UTr#*#O z#M?ra5~IelXN}OgAx$GAgb4M0xmlPhYk7Eav62=?H8F;4(A{cjnPW8*3}8XoMUfIB zi+IY7NS?Cj#6)Hf>uEhtMn^}A#?}*OSlH zL03GG4)*e@IKeGVeOsnZz|}Tht}NMo2wip7X%@%C9EhJv7x~orxjnd!Y5)RgI*u!t z>&)c})xn7}-^LB9tup_hmU~zZ(~!W)2Qa5c81Xg~Zsi0Ns=paw3?aZ^TDRc*Sn2i( z+)F0H*#gEA2USn)@%(}{LH=d>o-EWK%yD9~@)pBZNvNgvfGM)3gB^#PcZ5T?R{A78n<{DQWsC-Tak1ip7uR{W3HWr z#u90kUmGX@C9V8{vNGl+c@fO`0!bD+Gv^YFUBjf%8l8)!`7+@wy#EE6Uh|i&-fb_R zhlz)-fV>Zqx>Xj6O>8$q7Lt?2N|hzLZjV$vn*cN?-P50pEv{y{T}Y_OGwc3i?)dWD z%f~Kyu3ru=w!7})(+Dmw77Ujy4d_kjtPbuODOfvbg z+-{kj(}&H1>0cv4p7nMQV=Rs6koNHXXd$Y?Ydf4(iIvv_{7})JndtS1mKVAA$R=AG;2Vmd8!BR9X{!6zY9jasM9G#EoHXY z!0kBs#D5E*VFHCZKG0%V;N@M#hjk?1_bn5WE3h;+^aE76pE})cRAd>B#a2YoOMz5j z1etM}>Q9I;;@rP-cmb@z`d@xexYFt|sX_x%Z0X0hS)fv7xXJzm)-MbU0DP+17Ye=+ z+aWymP%rK+4pB)Ytv4&4-)uY?md%IZFH;y1Ppg`jTl-vDBwy|+kq4caygxPZq=FRD z@|*~5e?xOn?=y`iA67eFH&5U;sIz0>U^S;>Po@-Iyu-w$h8s1$z;!*S)d`%V$|)F) zxFOg~mZ@R{nu0w|8i7^=A~>}2G^(uM#%0Cjj@+^Gh|Dc4F80JDwg{Wi3H5s| zEExB)GgWF$w7)=$i)IBr(+k-&H#iWxG{z=DwyIw~#oGNtbOJ6ovnJ`x^O#oAt5id) zS|GO0lirIDYVKy!75v>%faijaVVy#U9xBHZfZ=NX(_A)&h5^U^U$eEmG-QLBTjz6H z(2R-EZOAR*0l?`Jrm*&Wg|8P6a=Ho=jpDnu9*x^;a#5lI6(KE!x1-6O@64}F@(GFe z2ZV=7vvEg5LREr*LSJ60!X$crtL(kFiuRo5K$jrp$q4n4?l@<(-wTt)uv`8p(S7<3 zKl2%r`3zgd@6W~Q_1}AsX&zLos#bExp6K0N`upX$*v62UEc?0N<-SWXzsP752 zI~r+Iw}jGf$rUjU`BNs-c#OQ!y2;bJY8!@9zAJL=ti~(9=-?ntkOTj ziAO3cKI4o-JsNrhxmZ`e;|o-b56a5Phvu^K^dD3C%__7C&1%Yw%@g_9tOC(o z+!@7FGAE&B{}c9ICT(N8UNw>eu}+cPlGn^@$-cPur~NN+oQLHMim6hKcSK5vTVN=n zi0)4>cR3q)!ugklfDs`8m}$ehE+SLI>R#R-M_Tj-$Wfg#j9SN<%${NYV9`tZN>m3B zcJ^cr-V;il)z&W&o|1o4(5WlrHM!t|I5KrBA-X9rj(}+YnRfReFyJ7!yRvCRhE9Tj zJo`pm9Htz^hLkO}pnLUO@X8R#V(5#bAaX146@43;Cst=S%zn-`k7p_0R9qxBcoa zKYnKaQxj3wB}*|g0nrmOR>}`w&;AvrCdhbZ4REg_q#StL?wV@{+(a(j8>^3ourFQu zbgoTj#KDWM`QHWxt5boh2f^l$8DkkalqI}dz8Kk~kc*BD18)h8r1CUX9h#w#YO;1h z{6p(*_UpUk$*S@T1KiX(ER7kS@DeN+->ZwI53~EBykt#pjoa-M$~s&wD!RxItQb(o37akwj;g)5`S{X!9 zb=h_gZ&;~!q^5l0lvewnOAmxA)a^h2(~_KX$X_@hl%{g~l|M1w)>OfzuvASFOsV7+ zKe{jr#i0^-x;PlQX;)ofwel0NKkflC4SYvZ<5+!$&x$&f>_QTQG^|wLl|+BB;V3p* zr#C91y=Tl}tow^WodwEJLXp-9$gD2i1J}{_lLK~*d-FA<04lWSakjQ$|Bargl zEYj?h6Tw_H8GbHdUX}vcTcTti8wsujRmQY_is&FA>~N~<55P1Z2uaW!2w&1u?VbLr z9oQR1!FfxInph1WiaBc`#8Xf7J^_2-eHr+Iu7JX-{;8C%10}}M z`F?5VZxNcQ|0-YFbpD;)O&5Tw&hogA;^KvfoDT`{&1YFoAA@5jxKSSn?{%Wd)$i`R z;q~WbN0!eDX&HhL^Phg%CqF!x)>n8RJ^}%SD}R6MgG25^tN<1mDYu^*ug5`i<$CHZ zSRFT0w_)cyE{PxMY=4Q2qN`>Nf#~8*t=Mu1X)f*S*Mh6CDob_)KM4RKM?$aF?8cX? z`$KOkBY_vj29WMw{>)JT2fAn@+$Y{{G-^r^>A=; zv$Y(7uwY;K8~Vi4aF!xZ)7y-*k3KI=rIv)0iWv%?;}+(YQinpC`TkI{9!SC%dq&ly zL678$0bn=9-agvF$+Rwyo1r|}hP_bEwP}rD4X}E0(Wj=TQ6$+uqnScAo2FDs*#ho*8nj`hvNMSh`T`n}`K9dAm=AR^v+5`4d#qky>DS=BcB|X_g1#05x zVxYHjFLUn$u&^hDu&l^fRnWvMe;G$qLxkrLns|Juv*OYKBx{x?7M{edop1=Bu zi_MQrChOSu`|#@~P11-%S9!X?qmE4xJ9z;)v51q;OXDf8L98WR8v)+|p*Gv_>`ge& z_f?K)gA&S|gbj$f%q$#KE;l!6C~|GqtAwU1o{YZyM400vRA==~ufrQ0E@;r|c(`>g zCVv_oTcBB_1>IYmV*9l>^$2k7BfZqZWrbz5TGg^I+-8WK3B)RuG19!{a}!0ad3?WDMcT|#{1(gxP#bfJjAqXxdBb6eGVj#2ahGRdSngN`Topa4*7L!nsj2_;BCb>9k{GHD==f-*cOFf zbi5FQ6HN;s8ZuMbbIC8%)CyUOu*O=u7&9abCBZ7Q3BCMOjpCQjLji8}mptFG`gx7A z<}n*wF1$Ws5elm>W_dMhe95B=+wlbwg3TLMEB|03O@E9sl9j~fpRWcfslI;QCeC7m|E{i%_gF@TUipQ2p; zct)t^TP5L^bCk2XkH?Y1vB$BiY@n03-KrUv-2Z&R_jp8t;ksVJ*s-E^yGNm>5$39conj#SGzXL#6{4c{%>!Tag0a#UHzYKzPT zQ*(F6Q^ELcX}l&M^q#Mj<^ggQV-3IJ1ooogBf3{PT5}pYw%)TH{B;>+H+VM?J*>RN zJfD%)w02Z|T;pGcA5rmmaVUwAcb43)O>seg9Yf4S@=pYg6e8sWrwo<3O6bzb2Y8Qb zdXmpyIXd^;qRo|nJvFlU<@*aq6pWb;j;P|;zjO#Am}`H6;)yrqF!%GdYCkhy-vARN zIu`;A?Rt8B+0`!+yyRs}O&gz_$lMEG{<{7=%mKio;tx=kj4JVaTPAdNNpk3fSxdcG z#~V3e_G;XWcohIlm4CPC zYWN)=8&<}o?6X%}UK%p@S79DGCK>%+lPHeJ%t6o~J5rAJLcXQv12UnNpdivQ@ zzH4_0g**{w*CnPsJ2Cp$2m3(gm&=(6Pk)@xDoGbc&1Oi<_D-v_sw`YZYiAQELoPfr z+X#OHh7i&L+T?2O!jYsWi#UM|FL^KBpSK?6)@By$Pkd=R$3II~iz;%*bY^lC$uHOc z;*=DWBC?`z-0R!d?c_!mCbligK^QNc*e%HUP zu`qj}qYb8?brq8t-INI44~+df<~jc_S3TsRI{ylvYw)M{sUT`hHBb zdv>f#xs|TZSX($cay=YKZcySQ{8aVX*Eaq+WkY!%`Rh(wF%4Rv4Vo!Nq?FKEHvtMb zvGwMYge6*;yG07nWlvSP;3T^#Kg_ZO$zTTRz?!qTgH%kO5M(aFn{9?)WUFEAC$)yL znjl2`lfIcU`j-hPi)rO^i4N$)|C})7W#2T) z@pbJ?1Z2*CB?_>=4y3I2vBep8S#kQ6U71C%bE=70;k1cw&Pm2(-XFF9$$?h4q~+OE zd3lw;Z&^a(H8v&l=(*u$^ZYa(ywZh+Z2xK|G5k61X(ZPhZ*~2;&~@Wokw>|7eXC0a z^MTBLVS}_~U~POZ|BUe4cwq~-VYO8sI~~05D>rl)9kt-1w3V+6DPt~C zU`)~n*gIS^!>C>74S`AI$xmI+lO2r1_@ex>%n6$qC-}(1*L(@v!%ToQu8T$g>WDXh z39Whf%P~^6imaln*!GqyO@g*c&PYeqDbar#wE12ureO#eYZ6Z0!zO36FYFlT#x3YL zr<)w5DxjAQ5id#t6idGz+^VfI^~g&)WP~*W9LM>r+d)1VnZze2eQdTli?k}D(yx1& z#D|UN?N82)ZbY4E_%d`?$Ls>Dw<*k0o$-=}eezV)sDGuj6{^yBx2-!0!`dW$Yyx5q(~l9U)Vj(L1awc-?kGIN7YnDGi46R6-w#=A_TYu6hb$F~C#JmL0rE)7qS~4JKWQdQqD;Z# z(F<9zYfeW=yi0(A0|77|5P)lM0)0aKALoZm%;AZ`(F_%}wcN(f?EiWd5q$Gck10_| zk}_lIkB=(cF(=&r;9oGJ^aGd+HqtSdf4qIj8$}>3`|Z2NlUM(HbYSUmzEa^Sut6M} z#l6w042o!^6$cj4*Wgtn;!GrD6f`U%M%m!MH!9F*7k`MSGX4ABzaI|45o>*ti2a{K r{yw<$y8XB%ONZ)zyz>9y?eH3@VwR>WG>-pnS*N0?@v=hR?9=}OrQ1@M literal 0 HcmV?d00001 From 51deb33ea1cb7fa1bf683558a33d5b4716cdb406 Mon Sep 17 00:00:00 2001 From: jeanluciano Date: Wed, 27 Mar 2024 09:44:59 -0500 Subject: [PATCH 02/10] second draft --- docs/ecs_guide.md | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/docs/ecs_guide.md b/docs/ecs_guide.md index ba3c4360..a4e73499 100644 --- a/docs/ecs_guide.md +++ b/docs/ecs_guide.md @@ -15,7 +15,7 @@ Prefect enables remote flow execution via [workers](https://docs.prefect.io/conc For details on how workers and work pools are implemented for ECS, see the diagram below: -#### Architecture Diagram +#### Architecture diagram ```mermaid %%{ @@ -64,7 +64,7 @@ graph TB prefect_workpool -->|configures| fr_task_definition ``` -### ECS in Prefect Terms +### ECS in Prefect terms !!! tip "ECS tasks != Prefect tasks" An ECS task is **not** the same thing as a [Prefect task](https://docs.prefect.io/latest/concepts/tasks/#tasks-overview). @@ -103,9 +103,9 @@ Before you begin, make sure you have: - An [ECS Cluster](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html) to host both the worker and the flow runs it submits. This guide use the default cluster. To create your own follow [this guide](https://docs.aws.amazon.com/AmazonECS/latest/userguide/create_cluster.html). - A [VPC](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html) configured for your ECS tasks. This guide uses the default VPC. -### Step 1: Set Up an ECS work pool +### Step 1: Set up an ECS work pool -Before setting up the worker, create a simple [work pool](https://docs.prefect.io/latest/concepts/work-pools/#work-pool-configuration) of type ECS for the worker to pull work from. +Before setting up the worker, create a simple [work pool](https://docs.prefect.io/latest/concepts/work-pools/#work-pool-configuration) of type ECS for the worker to pull work from. If doing so from CLI be sure to [auth into prefect cloud](https://docs.prefect.io/latest/cloud/cloud-quickstart/#log-into-prefect-cloud-from-a-terminal). Create a work pool from the CLI: @@ -125,11 +125,11 @@ Next, set up a Prefect ECS worker that will discover and pull work from this wor ### Step 2: Start a Prefect worker in your ECS cluster -First start by creating the [IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-custom.html#roles-creatingrole-custom-trust-policy-console) required in order for your worker and flows to run. The sample flow in this guide is very simple and therefore you will only be creating one role, `taskExecutionRole`. To create a IAM role for the ECS task using the AWS CLI, follow these steps: +First start by creating the [IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-custom.html#roles-creatingrole-custom-trust-policy-console) required in order for your worker and flows to run. The sample flow in this guide doesn't interact with many other AWS services, so you will only be creating one role, `taskExecutionRole`. To create a IAM role for the ECS task using the AWS CLI, follow these steps: 1. **Create a trust policy** - The trust policy will specify that the ECS service containing the prefect worker will be able to assume the role required for calling other AWS services. + The trust policy will specify that the ECS service containing the Prefect worker will be able to assume the role required for calling other AWS services. Save this policy to a file, such as `ecs-trust-policy.json`: @@ -161,7 +161,7 @@ First start by creating the [IAM role](https://docs.aws.amazon.com/IAM/latest/Us ``` !!! tip - Depending on the requirements of your task, it is advised to create a [second role for your tasks](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html). This will contain permissions required while running the task. Because our sample flow doesn't interact with AWS you don't have to. + Depending on the requirements of your task, it is advised to create a [second role for your tasks](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html). This will contain permissions required while running the task. For example if, your workflow loaded data into an S3 bucket, you would need Because our sample flow doesn't interact with AWS, you don't have to. 3. **Attach the policy to the role** For this guide the ECS worker will require the permissions to pull images from ECR and publish logs to CloudWatch. Amazon has a managed policy named `AmazonECSTaskExecutionRolePolicy` that grants the permissions necessary for ECS tasks. [See here](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_execution_IAM_role.html) for other common execution role permissions. Attach this policy to your task execution role: @@ -224,7 +224,8 @@ First start by creating the [IAM role](https://docs.aws.amazon.com/IAM/latest/Us - For the `PREFECT_API_KEY`, individuals on the organization tier can create a [service account](https://docs.prefect.io/latest/cloud/users/service-accounts/) for the worker. If on a personal tier, you can pass a user’s API key. - - Replace both instances of `` with the ARN of the IAM role you created in Step 2. + - Replace both instances of `` with the ARN of the IAM role you created in Step 2. You can grab this by running: + ```aws iam get-role --role-name taskExecutionRole --query 'Role.[RoleName, Arn]' --output text``` - Notice that the CPU and Memory allocations are relatively small. The worker's main responsibility is to submit work through API calls to AWS, _not_ to execute your Prefect flow code. @@ -264,7 +265,7 @@ First start by creating the [IAM role](https://docs.aws.amazon.com/IAM/latest/Us - Replace `` with a comma-separated list of your VPC security group IDs. !!! tip "Sanity check" - The work pool page in the Prefect UI allows you to check the health of your workers - make sure your new worker is live! Although keep in mind it can take ECS a few minuter for a service to go online. + The work pool page in the Prefect UI allows you to check the health of your workers - make sure your new worker is live! Note that it can take a few minutes for an ECS service to come online. ### Step 4: Pick up a flow run with your new worker @@ -292,11 +293,12 @@ First start by creating the [IAM role](https://docs.aws.amazon.com/IAM/latest/Us --region ``` - To have prefect build your image when deploying your flow create `prefect.yaml` with the following + To have Prefect build your image when deploying your flow create a `prefect.yaml` file with the following specification: ```yaml name: ecs-worker-guide - prefect-version: 2.14.20 + # this is pre-populated by running prefect init + prefect-version: 2.14.20 # build section allows you to manage and build docker images build: @@ -334,7 +336,7 @@ First start by creating the [IAM role](https://docs.aws.amazon.com/IAM/latest/Us ``` -2. [Deploy](https://docs.prefect.io/2.11.0/tutorial/deployments/#create-a-deployment) the flow to the server, specifying the ECS work pool when prompted. +2. [Deploy](https://docs.prefect.io/tutorial/deployments/#create-a-deployment) the flow to the server, specifying the ECS work pool when prompted. ```bash prefect deploy my_flow.py:my_ecs_deployment From a5df78922e7dc0cb2e4002693c14fa7069368e4f Mon Sep 17 00:00:00 2001 From: jeanluciano Date: Thu, 28 Mar 2024 13:51:07 -0500 Subject: [PATCH 03/10] missed formatting --- docs/ecs_guide.md | 42 ++++++++++++++++++++++-------------------- 1 file changed, 22 insertions(+), 20 deletions(-) diff --git a/docs/ecs_guide.md b/docs/ecs_guide.md index 1e61e8e6..6df248e5 100644 --- a/docs/ecs_guide.md +++ b/docs/ecs_guide.md @@ -155,9 +155,11 @@ Use the `aws iam create-role` command to create the roles that you will be using --assume-role-policy-document file://ecs-trust-policy.json ``` - !!! tip +!!! tip Depending on the requirements of your task, it is advised to create a [second role for your tasks](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html). This will contain permissions required while running the task. For example if, your workflow loaded data into an S3 bucket, you would need Because our sample flow doesn't interact with AWS, you don't have to. + ### 3. Attach the policy to the role + For this guide the ECS worker will require the permissions to pull images from ECR and publish logs to CloudWatch. Amazon has a managed policy named `AmazonECSTaskExecutionRolePolicy` that grants the permissions necessary for ECS tasks. [See here](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_execution_IAM_role.html) for other common execution role permissions. Attach this policy to your task execution role: ```bash @@ -169,9 +171,9 @@ For this guide the ECS worker will require the permissions to pull images from E Remember to replace the `--role-name` and `--policy-arn` with the actual role name and policy Amazon Resource Name (ARN) you want to use. -### Step 3: Creating ECS worker service +## Step 3: Creating ECS worker service -1. **Launch an ECS Service to host the worker** +### 1. Launch an ECS Service to host the worker Next, create an ECS task definition that specifies the Docker image for the Prefect worker, the resources it requires, and the command it should run. In this example, the command to start the worker is `prefect worker start --pool my-ecs-pool`. @@ -227,7 +229,7 @@ Next, create an ECS task definition that specifies the Docker image for the Pref !!! tip To avoid hardcoding your API key into the task definition JSON see [how to add sensitive data using AWS secrets manager to the container definition](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/specifying-sensitive-data-tutorial.html#specifying-sensitive-data-tutorial-create-taskdef). -2. **Register the task definition:** +### 2. Register the task definition Before creating a service, you first need to register a task definition. You can do that using the `register-task-definition` command in the AWS CLI. Here is an example: @@ -237,7 +239,7 @@ Before creating a service, you first need to register a task definition. You can Replace `task-definition.json` with the name of your JSON file. -3. **Create an ECS service to host your worker:** +### 3. Create an ECS service to host your worker Finally, create a service that will manage your Prefect worker: @@ -255,7 +257,7 @@ aws ecs create-service \ - Replace `` with the name of your ECS cluster. - Replace `` with the ARN of the task definition you just registered. -- Replace `` with a comma-separated list of your VPC subnet IDs. Ensure that these subnets are aligned with the vpc specified on the work pool in step 1. You can view subnet ids with the following command: +- Replace `` with a comma-separated list of your VPC subnet IDs. Ensure that these subnets are aligned with the vpc specified on the work pool in step 1. You can view subnet ids with the following command: `aws ec2 describe-subnets --filter Name=` - Replace `` with a comma-separated list of your VPC security group IDs. @@ -264,23 +266,23 @@ aws ecs create-service \ ## Step 4: Pick up a flow run with your new worker -1. Write a simple test flow in a repo of your choice: +### 1. Write a simple test flow in a repo of your choice - `my_flow.py` +`my_flow.py` - ```python - from prefect import flow, get_run_logger +```python +from prefect import flow, get_run_logger - @flow - def my_flow(): - logger = get_run_logger() - logger.info("Hello from ECS!!") +@flow +def my_flow(): + logger = get_run_logger() + logger.info("Hello from ECS!!") - if __name__ == "__main__": - my_flow() - ``` +if __name__ == "__main__": + my_flow() +``` -2. This guide uses ECR to store you code data. Create a repo with the following command: +### 2. This guide uses ECR to store you code data. Create a repo with the following command ```bash aws ecr create-repository \ @@ -331,13 +333,13 @@ pull: ``` -2. [Deploy](https://docs.prefect.io/tutorial/deployments/#create-a-deployment) the flow to the server, specifying the ECS work pool when prompted. +### 2. [Deploy](https://docs.prefect.io/tutorial/deployments/#create-a-deployment) the flow to the server, specifying the ECS work pool when prompted ```bash prefect deploy my_flow.py:my_ecs_deployment ``` -3. Find the deployment in the UI and click the **Quick Run** button! +### 3. Find the deployment in the UI and click the **Quick Run** button! ## Optional Next Steps From 456ad139169c31a310648660961ecf9b086e003e Mon Sep 17 00:00:00 2001 From: jeanluciano Date: Thu, 28 Mar 2024 14:17:05 -0500 Subject: [PATCH 04/10] code block fix --- docs/ecs_guide.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/ecs_guide.md b/docs/ecs_guide.md index 6df248e5..84f4c277 100644 --- a/docs/ecs_guide.md +++ b/docs/ecs_guide.md @@ -2,7 +2,7 @@ ## Why use ECS for flow run execution? -ECS (Elastic Container Service) tasks are a good option for executing Prefect 2 flow runs for several reasons: +ECS (Elastic Container Service) tasks are a good option for executing Prefect flow runs for several reasons: 1. **Scalability**: ECS scales your infrastructure in response to demand, effectively managing Prefect flow runs. ECS automatically administers container distribution across multiple instances based on demand. 2. **Flexibility**: ECS lets you choose between AWS Fargate and Amazon EC2 for container operation. Fargate abstracts the underlying infrastructure, while EC2 has faster job start times and offers additional control over instance management and configuration. @@ -87,15 +87,13 @@ You can use either EC2 or Fargate as the capacity provider. Fargate simplifies i
-# AWS CLI Guide + !!! tip If you prefer infrastructure as code check out this [Terraform module](https://github.com/PrefectHQ/prefect-recipes/tree/main/devops/infrastructure-as-code/aws/tf-prefect2-ecs-worker) to provision an ECS cluster with a worker. ## Prerequisites -Before you begin, make sure you have: - - An AWS account with permissions to create ECS services and IAM roles. - The AWS CLI installed on your local machine. You can [download it from the AWS website](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html). - An [ECS Cluster](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html) to host both the worker and the flow runs it submits. This guide use the default cluster. To create your own follow [this guide](https://docs.aws.amazon.com/AmazonECS/latest/userguide/create_cluster.html). @@ -222,7 +220,9 @@ Next, create an ECS task definition that specifies the Docker image for the Pref - For the `PREFECT_API_KEY`, individuals on the organization tier can create a [service account](https://docs.prefect.io/latest/cloud/users/service-accounts/) for the worker. If on a personal tier, you can pass a user’s API key. - Replace both instances of `` with the ARN of the IAM role you created in Step 2. You can grab this by running: - ```aws iam get-role --role-name taskExecutionRole --query 'Role.[RoleName, Arn]' --output text``` +``` +aws iam get-role --role-name taskExecutionRole --query 'Role.[RoleName, Arn]' --output text +``` - Notice that the CPU and Memory allocations are relatively small. The worker's main responsibility is to submit work through API calls to AWS, _not_ to execute your Prefect flow code. @@ -234,7 +234,7 @@ Next, create an ECS task definition that specifies the Docker image for the Pref Before creating a service, you first need to register a task definition. You can do that using the `register-task-definition` command in the AWS CLI. Here is an example: ```bash - aws ecs register-task-definition --cli-input-json file://task-definition.json +aws ecs register-task-definition --cli-input-json file://task-definition.json ``` Replace `task-definition.json` with the name of your JSON file. From 86a7ce9865f3e0d9cb7719c3aa554d22256865c9 Mon Sep 17 00:00:00 2001 From: jeanluciano Date: Mon, 1 Apr 2024 09:13:04 -0500 Subject: [PATCH 05/10] review edits --- docs/ecs_guide.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/ecs_guide.md b/docs/ecs_guide.md index 84f4c277..ff1f4f54 100644 --- a/docs/ecs_guide.md +++ b/docs/ecs_guide.md @@ -145,7 +145,7 @@ Save this policy to a file, such as `ecs-trust-policy.json`: ### 2. Create the IAM roles -Use the `aws iam create-role` command to create the roles that you will be using. The `ecsTaskExecutionRole` will be used by the worker to talk to make AWS API calls on your behalf when starting ecs tasks. +Use the `aws iam create-role` command to create the roles that you will be using. The `ecsTaskExecutionRole` will be used by the worker to start ECS tasks. ```bash aws iam create-role \ @@ -154,11 +154,11 @@ Use the `aws iam create-role` command to create the roles that you will be using ``` !!! tip - Depending on the requirements of your task, it is advised to create a [second role for your tasks](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html). This will contain permissions required while running the task. For example if, your workflow loaded data into an S3 bucket, you would need Because our sample flow doesn't interact with AWS, you don't have to. + Depending on the requirements of your flows, it is advised to create a [second role for your ECS tasks](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html). This role will contain the permissions required by the ECS tasks in which your flows will run. For example, if your workflow loads data into an S3 bucket, you would need a role with additional permissions specific to accessing S3. ### 3. Attach the policy to the role -For this guide the ECS worker will require the permissions to pull images from ECR and publish logs to CloudWatch. Amazon has a managed policy named `AmazonECSTaskExecutionRolePolicy` that grants the permissions necessary for ECS tasks. [See here](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_execution_IAM_role.html) for other common execution role permissions. Attach this policy to your task execution role: +For this guide the ECS worker will require permissions to pull images from ECR and publish logs to CloudWatch. Amazon has a managed policy named `AmazonECSTaskExecutionRolePolicy` that grants the permissions necessary for starting ECS tasks. [See here](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_execution_IAM_role.html) for other common execution role permissions. Attach this policy to your task execution role: ```bash @@ -169,7 +169,7 @@ For this guide the ECS worker will require the permissions to pull images from E Remember to replace the `--role-name` and `--policy-arn` with the actual role name and policy Amazon Resource Name (ARN) you want to use. -## Step 3: Creating ECS worker service +## Step 3: Creating an ECS worker service ### 1. Launch an ECS Service to host the worker @@ -217,7 +217,7 @@ Next, create an ECS task definition that specifies the Docker image for the Pref - Use `prefect config view` to view the `PREFECT_API_URL` for your current Prefect profile. Use this to replace ``. -- For the `PREFECT_API_KEY`, individuals on the organization tier can create a [service account](https://docs.prefect.io/latest/cloud/users/service-accounts/) for the worker. If on a personal tier, you can pass a user’s API key. +- For the `PREFECT_API_KEY`, individuals on the pro tier can create a [service account](https://docs.prefect.io/latest/cloud/users/service-accounts/) for the worker. If on a personal tier, you can pass a user’s API key. - Replace both instances of `` with the ARN of the IAM role you created in Step 2. You can grab this by running: ``` From d07f3ae8435b78b5c64480f1d3185ffa387cb748 Mon Sep 17 00:00:00 2001 From: jeanluciano Date: Mon, 1 Apr 2024 13:45:47 -0500 Subject: [PATCH 06/10] styling prefect.yaml --- docs/ecs_guide.md | 45 ++++++++++++++++++++++++--------------------- 1 file changed, 24 insertions(+), 21 deletions(-) diff --git a/docs/ecs_guide.md b/docs/ecs_guide.md index ff1f4f54..1149e2af 100644 --- a/docs/ecs_guide.md +++ b/docs/ecs_guide.md @@ -96,12 +96,12 @@ You can use either EC2 or Fargate as the capacity provider. Fargate simplifies i - An AWS account with permissions to create ECS services and IAM roles. - The AWS CLI installed on your local machine. You can [download it from the AWS website](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html). -- An [ECS Cluster](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html) to host both the worker and the flow runs it submits. This guide use the default cluster. To create your own follow [this guide](https://docs.aws.amazon.com/AmazonECS/latest/userguide/create_cluster.html). +- An [ECS Cluster](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html) to host both the worker and the flow runs it submits. This guide uses the default cluster. To create your own follow [this guide](https://docs.aws.amazon.com/AmazonECS/latest/userguide/create_cluster.html). - A [VPC](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html) configured for your ECS tasks. This guide uses the default VPC. ## Step 1: Set up an ECS work pool -Before setting up the worker, create a simple [work pool](https://docs.prefect.io/latest/concepts/work-pools/#work-pool-configuration) of type ECS for the worker to pull work from. If doing so from CLI be sure to [auth into prefect cloud](https://docs.prefect.io/latest/cloud/cloud-quickstart/#log-into-prefect-cloud-from-a-terminal). +Before setting up the worker, create a [work pool](https://docs.prefect.io/latest/concepts/work-pools/#work-pool-configuration) of type ECS for the worker to pull work from. If doing so the from CLI be sure to [auth into prefect cloud](https://docs.prefect.io/latest/cloud/cloud-quickstart/#log-into-prefect-cloud-from-a-terminal). Create a work pool from the CLI: @@ -109,17 +109,17 @@ Create a work pool from the CLI: prefect work-pool create --type ecs my-ecs-pool ``` -Or from Prefect UI: +Or from the Prefect UI: ![WorkPool](img/Workpool_UI.png) !!! - Because this guide uses fargate as the capacity provider and the default VPC and ECS cluster, no further configuration is needed. + Because this guide uses Fargate as the capacity provider and the default VPC and ECS cluster, no further configuration is needed. Next, set up a Prefect ECS worker that will discover and pull work from this work pool. ## Step 2: Start a Prefect worker in your ECS cluster -First start by creating the [IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-custom.html#roles-creatingrole-custom-trust-policy-console) required in order for your worker and flows to run. The sample flow in this guide doesn't interact with many other AWS services, so you will only be creating one role, `taskExecutionRole`. To create a IAM role for the ECS task using the AWS CLI, follow these steps: +First start by creating the [IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-custom.html#roles-creatingrole-custom-trust-policy-console) required in order for your worker and flows to run. The sample flow in this guide doesn't interact with many other AWS services, so you will only be creating one role, `taskExecutionRole`. To create an IAM role for the ECS task using the AWS CLI, follow these steps: ### 1. Create a trust policy @@ -161,7 +161,6 @@ Use the `aws iam create-role` command to create the roles that you will be using For this guide the ECS worker will require permissions to pull images from ECR and publish logs to CloudWatch. Amazon has a managed policy named `AmazonECSTaskExecutionRolePolicy` that grants the permissions necessary for starting ECS tasks. [See here](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_execution_IAM_role.html) for other common execution role permissions. Attach this policy to your task execution role: ```bash - aws iam attach-role-policy \ --role-name ecsTaskExecutionRole \ --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy @@ -284,12 +283,16 @@ if __name__ == "__main__": ### 2. This guide uses ECR to store you code data. Create a repo with the following command +Use the following AWS CLI command to create an ECR repository. The name you choose for your repository will be reused in the next step when defining your Prefect deployment. + ```bash aws ecr create-repository \ --repository-name \ --region ``` +### 3. Create a `prefect.yaml` + To have Prefect build your image when deploying your flow create a `prefect.yaml` file with the following specification: ```yaml @@ -316,30 +319,30 @@ push: # the deployments section allows you to provide configuration for deploying flows deployments: - name: my_ecs_deployment -version: -tags: [] -description: -entrypoint: flow.py:my_flow -parameters: {} -work_pool: - name: ecs-dev-pool - work_queue_name: - job_variables: - image: '{{ build_image.image }}' -schedules: [] + version: + tags: [] + description: + entrypoint: flow.py:my_flow + parameters: {} + work_pool: + name: ecs-dev-pool + work_queue_name: + job_variables: + image: '{{ build_image.image }}' + schedules: [] pull: -- prefect.deployments.steps.set_working_directory: - directory: /opt/prefect/ecs-worker-guide + - prefect.deployments.steps.set_working_directory: + directory: /opt/prefect/ecs-worker-guide ``` -### 2. [Deploy](https://docs.prefect.io/tutorial/deployments/#create-a-deployment) the flow to the server, specifying the ECS work pool when prompted +### 4. [Deploy](https://docs.prefect.io/tutorial/deployments/#create-a-deployment) the flow to the server, specifying the ECS work pool when prompted ```bash prefect deploy my_flow.py:my_ecs_deployment ``` -### 3. Find the deployment in the UI and click the **Quick Run** button! +### 5. Find the deployment in the UI and click the **Quick Run** button! ## Optional Next Steps From 1bd0aee72b7a7104261d89b5ce4b1e0b6186e2fd Mon Sep 17 00:00:00 2001 From: jeanluciano Date: Wed, 3 Apr 2024 09:28:54 -0500 Subject: [PATCH 07/10] mentions selfmanaged --- docs/ecs_guide.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/docs/ecs_guide.md b/docs/ecs_guide.md index 1149e2af..8e0a3500 100644 --- a/docs/ecs_guide.md +++ b/docs/ecs_guide.md @@ -9,7 +9,7 @@ ECS (Elastic Container Service) tasks are a good option for executing Prefect fl 3. **AWS Integration**: Easily connect with other AWS services, such as AWS IAM and CloudWatch. 4. **Containerization**: ECS supports Docker containers and offers managed execution. Containerization encourages reproducible deployments. -## ECS Flow run execution +## ECS flow run execution Prefect enables remote flow execution via [workers](https://docs.prefect.io/concepts/work-pools/#worker-overview) and [work pools](https://docs.prefect.io/concepts/work-pools/#work-pool-overview). To learn more about these concepts please see our [deployment tutorial](https://docs.prefect.io/tutorial/deployments/). @@ -83,7 +83,7 @@ Once the flow run completes, the ECS containers of the cluster are spun down to If you specify a task definition [ARN (Amazon Resource Name)](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference-arns.html) in the work pool, the worker will use that ARN when spinning up the ECS Task, rather than creating a task definition from the fields supplied in the work pool configuration. -You can use either EC2 or Fargate as the capacity provider. Fargate simplifies initiation, but lengthens infrastructure setup time for each flow run. Using EC2 for the ECS cluster can reduce setup time. In this example, we will show how to use Fargate. +You can use either EC2 or Fargate as the capacity provider. Fargate simplifies initiation, but lengthens infrastructure setup time for each flow run. Using EC2 for the ECS cluster can reduce setup time. In this example, we will show how to use Fargate.
@@ -98,6 +98,7 @@ You can use either EC2 or Fargate as the capacity provider. Fargate simplifies i - The AWS CLI installed on your local machine. You can [download it from the AWS website](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html). - An [ECS Cluster](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html) to host both the worker and the flow runs it submits. This guide uses the default cluster. To create your own follow [this guide](https://docs.aws.amazon.com/AmazonECS/latest/userguide/create_cluster.html). - A [VPC](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html) configured for your ECS tasks. This guide uses the default VPC. +- Prefect Cloud account or Prefect self-managed instance. ## Step 1: Set up an ECS work pool @@ -124,7 +125,7 @@ First start by creating the [IAM role](https://docs.aws.amazon.com/IAM/latest/Us ### 1. Create a trust policy The trust policy will specify that the ECS service containing the Prefect worker will be able to assume the role required for calling other AWS services. - +os Save this policy to a file, such as `ecs-trust-policy.json`: ```json @@ -216,7 +217,7 @@ Next, create an ECS task definition that specifies the Docker image for the Pref - Use `prefect config view` to view the `PREFECT_API_URL` for your current Prefect profile. Use this to replace ``. -- For the `PREFECT_API_KEY`, individuals on the pro tier can create a [service account](https://docs.prefect.io/latest/cloud/users/service-accounts/) for the worker. If on a personal tier, you can pass a user’s API key. +- For the `PREFECT_API_KEY`, if you are on a paid plan you can create a [service account](https://docs.prefect.io/latest/cloud/users/service-accounts/) for the worker. If your are on a free plan, you can pass a user’s API key. - Replace both instances of `` with the ARN of the IAM role you created in Step 2. You can grab this by running: ``` @@ -291,7 +292,7 @@ aws ecr create-repository \ --region ``` -### 3. Create a `prefect.yaml` +### 3. Create a `prefect.yaml` file To have Prefect build your image when deploying your flow create a `prefect.yaml` file with the following specification: @@ -336,7 +337,7 @@ pull: ``` -### 4. [Deploy](https://docs.prefect.io/tutorial/deployments/#create-a-deployment) the flow to the server, specifying the ECS work pool when prompted +### 4. [Deploy](https://docs.prefect.io/tutorial/deployments/#create-a-deployment) the flow to the Prefect Cloud or your self-managed server instance, specifying the ECS work pool when prompted ```bash prefect deploy my_flow.py:my_ecs_deployment From 55d31102420524a88e55853ad4a8fd7542bb5808 Mon Sep 17 00:00:00 2001 From: jeanluciano Date: Wed, 3 Apr 2024 16:08:23 -0500 Subject: [PATCH 08/10] grammar fix --- docs/ecs_guide.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/docs/ecs_guide.md b/docs/ecs_guide.md index 8e0a3500..ca360c26 100644 --- a/docs/ecs_guide.md +++ b/docs/ecs_guide.md @@ -83,12 +83,11 @@ Once the flow run completes, the ECS containers of the cluster are spun down to If you specify a task definition [ARN (Amazon Resource Name)](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference-arns.html) in the work pool, the worker will use that ARN when spinning up the ECS Task, rather than creating a task definition from the fields supplied in the work pool configuration. -You can use either EC2 or Fargate as the capacity provider. Fargate simplifies initiation, but lengthens infrastructure setup time for each flow run. Using EC2 for the ECS cluster can reduce setup time. In this example, we will show how to use Fargate. +You can use either EC2 or Fargate as the capacity provider. Fargate simplifies initiation, but lengthens infrastructure setup time for each flow run. Using EC2 for the ECS cluster can reduce setup time. In this example, we will show how to use Fargate.
- !!! tip If you prefer infrastructure as code check out this [Terraform module](https://github.com/PrefectHQ/prefect-recipes/tree/main/devops/infrastructure-as-code/aws/tf-prefect2-ecs-worker) to provision an ECS cluster with a worker. @@ -102,7 +101,7 @@ You can use either EC2 or Fargate as the capacity provider. Fargate simplifies i ## Step 1: Set up an ECS work pool -Before setting up the worker, create a [work pool](https://docs.prefect.io/latest/concepts/work-pools/#work-pool-configuration) of type ECS for the worker to pull work from. If doing so the from CLI be sure to [auth into prefect cloud](https://docs.prefect.io/latest/cloud/cloud-quickstart/#log-into-prefect-cloud-from-a-terminal). +Before setting up the worker, create a [work pool](https://docs.prefect.io/latest/concepts/work-pools/#work-pool-configuration) of type ECS for the worker to pull work from. If doing so from the CLI, be sure to [authenticate with Prefect Cloud](https://docs.prefect.io/latest/cloud/cloud-quickstart/#log-into-prefect-cloud-from-a-terminal). Create a work pool from the CLI: @@ -125,7 +124,7 @@ First start by creating the [IAM role](https://docs.aws.amazon.com/IAM/latest/Us ### 1. Create a trust policy The trust policy will specify that the ECS service containing the Prefect worker will be able to assume the role required for calling other AWS services. -os + Save this policy to a file, such as `ecs-trust-policy.json`: ```json @@ -155,7 +154,7 @@ Use the `aws iam create-role` command to create the roles that you will be using ``` !!! tip - Depending on the requirements of your flows, it is advised to create a [second role for your ECS tasks](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html). This role will contain the permissions required by the ECS tasks in which your flows will run. For example, if your workflow loads data into an S3 bucket, you would need a role with additional permissions specific to accessing S3. + Depending on the requirements of your flows, it is advised to create a [second role for your ECS tasks](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html). This role will contain the permissions required by the ECS tasks in which your flows will run. For example, if your workflow loads data into an S3 bucket, you would need a role with additional permissions to access S3. ### 3. Attach the policy to the role From 5306e0a55e039b7c1190969dd6d588102827e34c Mon Sep 17 00:00:00 2001 From: jeanluciano Date: Thu, 4 Apr 2024 10:30:16 -0500 Subject: [PATCH 09/10] indent fixes --- docs/ecs_guide.md | 106 +++++++++++++++++++++++----------------------- 1 file changed, 54 insertions(+), 52 deletions(-) diff --git a/docs/ecs_guide.md b/docs/ecs_guide.md index ca360c26..f8e04952 100644 --- a/docs/ecs_guide.md +++ b/docs/ecs_guide.md @@ -128,24 +128,23 @@ The trust policy will specify that the ECS service containing the Prefect worker Save this policy to a file, such as `ecs-trust-policy.json`: ```json - - { - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Allow", - "Principal": { - "Service": "ecs-tasks.amazonaws.com" - }, - "Action": "sts:AssumeRole" - } - ] - } +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "Service": "ecs-tasks.amazonaws.com" + }, + "Action": "sts:AssumeRole" + } + ] +} ``` ### 2. Create the IAM roles -Use the `aws iam create-role` command to create the roles that you will be using. The `ecsTaskExecutionRole` will be used by the worker to start ECS tasks. +Use the `aws iam create-role` command to create the roles that you will be using. For this guide, the `ecsTaskExecutionRole` will be used by the worker to start ECS tasks, and will also be the role assigned to the ECS tasks running your Prefect flows. ```bash aws iam create-role \ @@ -177,41 +176,41 @@ Next, create an ECS task definition that specifies the Docker image for the Pref **Create a JSON file with the following contents:** ```json - { - "family": "prefect-worker-task", - "networkMode": "awsvpc", - "requiresCompatibilities": [ - "FARGATE" - ], - "cpu": "512", - "memory": "1024", - "executionRoleArn": "", - "taskRoleArn": "", - "containerDefinitions": [ - { - "name": "prefect-worker", - "image": "prefecthq/prefect:2-latest", - "cpu": 512, - "memory": 1024, - "essential": true, - "command": [ - "/bin/sh", - "-c", - "pip install prefect-aws && prefect worker start --pool my-ecs-pool --type ecs" - ], - "environment": [ - { - "name": "PREFECT_API_URL", - "value": "" - }, - { - "name": "PREFECT_API_KEY", - "value": "" - } - ] - } - ] - } +{ + "family": "prefect-worker-task", + "networkMode": "awsvpc", + "requiresCompatibilities": [ + "FARGATE" + ], + "cpu": "512", + "memory": "1024", + "executionRoleArn": "", + "taskRoleArn": "", + "containerDefinitions": [ + { + "name": "prefect-worker", + "image": "prefecthq/prefect:2-latest", + "cpu": 512, + "memory": 1024, + "essential": true, + "command": [ + "/bin/sh", + "-c", + "pip install prefect-aws && prefect worker start --pool my-ecs-pool --type ecs" + ], + "environment": [ + { + "name": "PREFECT_API_URL", + "value": "prefect-api-url>" + }, + { + "name": "PREFECT_API_KEY", + "value": "" + } + ] + } + ] +} ``` - Use `prefect config view` to view the `PREFECT_API_URL` for your current Prefect profile. Use this to replace ``. @@ -251,7 +250,7 @@ aws ecs create-service \ --task-definition \ --launch-type FARGATE \ --desired-count 1 \ - --network-configuration "awsvpcConfiguration={subnets=[],securityGroups=[],assignPublicIp='ENABLED'}" + --network-configuration "awsvpcConfiguration={subnets=[],securityGroups=[],assignPublicIp='ENABLED'}" ``` - Replace `` with the name of your ECS cluster. @@ -262,10 +261,13 @@ aws ecs create-service \ !!! tip "Sanity check" The work pool page in the Prefect UI allows you to check the health of your workers - make sure your new worker is live! Note that it can take a few minutes for an ECS service to come online. + If your worker does not come online and you are using the command from this guide, you may not be using the default VPC. For connectivity issues, check your VPC's configuration and refer to the [ECS outbound networking guide](https://docs.aws.amazon.com/AmazonECS/latest/bestpracticesguide/networking-outbound.html). ## Step 4: Pick up a flow run with your new worker -### 1. Write a simple test flow in a repo of your choice +This guide uses ECR to store a Docker image containing your flow code. To do this, we will write a flow, then deploy it using build and push steps that copy flow code into a Docker image and push that image to an ECR repository. + +### 1. Write a simple test flow `my_flow.py` @@ -281,7 +283,7 @@ if __name__ == "__main__": my_flow() ``` -### 2. This guide uses ECR to store you code data. Create a repo with the following command +### 2. Create an ECR repository Use the following AWS CLI command to create an ECR repository. The name you choose for your repository will be reused in the next step when defining your Prefect deployment. From 6273b72a7cac85c9d40eb4a45b7d6425cc3e0e6c Mon Sep 17 00:00:00 2001 From: jeanluciano Date: Fri, 5 Apr 2024 11:26:47 -0500 Subject: [PATCH 10/10] --- docs/ecs_guide.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/ecs_guide.md b/docs/ecs_guide.md index f8e04952..7e5380ac 100644 --- a/docs/ecs_guide.md +++ b/docs/ecs_guide.md @@ -346,7 +346,7 @@ prefect deploy my_flow.py:my_ecs_deployment ### 5. Find the deployment in the UI and click the **Quick Run** button! -## Optional Next Steps +## Optional next steps 1. Now that you are confident your ECS worker is healthy, you can experiment with different work pool configurations.