ECS Works Pools Should Support Specifying Security Groups & Subnets for Flow Runs

[Jrodseth]

Currently, ECS Workers run tasks without specifying a security group or subnet(s) which results in tasks running in the VPC's default security group/a random subnet. This 1.) forces the default security group to by modified if task runs require additional security group rules and 2.) can result in deployment to a private subnet without a NAT gateway.

For example: when using a custom ECS task definition which mounts an EFS volume, the default security group must be modified to allow traffic to and from the filesystem on port 2049.

Long term, it would be great if ECS workpools and ECS push workpools supported setting security group arn(s)/subnet (arns) for prefect tasks.

In the short term, it may be sufficient to reuse whichever security group/subnet is attached to the ECS worker. I believe this could be achieved easily by modifying _load_vpc_network_config to fetch the ECS worker's security group and set it in this line:

prefect-aws/prefect_aws/workers/ecs_worker.py

Line 1283 in 416d55e

"securityGroups": [],

[HughZurname]

To add a little more to Jrodseth's excellent investigation above (thank you!), if you use the networking config supplied in /prefect-recipes/devops/infrastructure-as-code/aws/aws-services/network the default security group has no egress or ingress and the worker task ends up failing. So without a good example to replace this, you are likely to have anyone following your recipes and using ecs workers to fall into this gap.