Usage

module "glue-job" {
  source  = "pquintero/glue-job/aws"
  version = "0.0.2"
  # insert the 12 required variables here
}

Requirements

Name	Version
terraform	>= 0.15.0
aws	>= 3.68.0

Providers

Name	Version
aws	>= 3.68.0

Modules

No modules.

Resources

Name	Type
aws_glue_job.glue_job	resource

Inputs

Name	Description	Type	Default	Required
enable_glue_connection	Enable glue connection usage	bool	false	no
enable_glue_job	Enable glue job usage	bool	false	no
environment	Environment for service	string	"STAGE"	no
glue_connection_catalog_id	(Optional) The ID of the Data Catalog in which to create the connection. If none is supplied, the AWS account ID is used by default.	any	null	no
glue_connection_connection_properties	(Required) A map of key-value pairs used as parameters for this connection.	map	{}	no
glue_connection_connection_type	(Optional) The type of the connection. Supported are: JDBC, MONGODB. Defaults to JDBC.	string	"JDBC"	no
glue_connection_description	(Optional) Description of the connection.	any	null	no
glue_connection_match_criteria	(Optional) A list of criteria that can be used in selecting this connection.	any	null	no
glue_connection_name	The name of the connection.	string	""	no
glue_connection_physical_connection_requirements	(Optional) A map of physical connection requirements, such as VPC and SecurityGroup.	list	[]	no
glue_job_additional_connections	(Optional) The list of connections used for the job.	list	[]	no
glue_job_command	(Required) The command of the job.	list	[]	no
glue_job_connections	(Optional) The list of connections used for this job.	any	null	no
glue_job_default_arguments	(Optional) The map of default arguments for this job. You can specify arguments here that your own job-execution script consumes, as well as arguments that AWS Glue itself consumes. For information about how to specify and consume your own Job arguments, see the Calling AWS Glue APIs in Python topic in the developer guide. For information about the key-value pairs that AWS Glue consumes to set up your job, see the Special Parameters Used by AWS Glue topic in the developer guide.	map	{ "--enable-glue-datacatalog": true, "--enable-metrics": true, "--extra-jars": "s3://bancoripleype-dev-artifactory/libs/httpclient-4.5.13.jar,s3://bancoripleype-dev-artifactory/libs/hudi-spark-bundle_2.11-0.8.0.jar,s3://bancoripleype-dev-artifactory/libs/spark-avro_2.11-2.4.3.jar", "--job-bookmark-option": "job-bookmark-disable", "--job-language": "python" }	no
glue_job_description	(Optional) Description of the job.	any	null	no
glue_job_execution_property	(Optional) Execution property of the job.	list	[]	no
glue_job_max_capacity	(Optional) The maximum number of AWS Glue data processing units (DPUs) that can be allocated when this job runs. Required when pythonshell is set, accept either 0.0625 or 1.0.	any	null	no
glue_job_max_retries	(Optional) The maximum number of times to retry this job if it fails.	any	null	no
glue_job_name	The name you assign to this job. It must be unique in your account.	string	""	no
glue_job_notification_property	(Optional) Notification property of the job.	list	[]	no
glue_job_number_of_workers	(Optional) The number of workers of a defined workerType that are allocated when a job runs.	any	null	no
glue_job_role_arn	The ARN of the IAM role associated with this job.	any	null	no
glue_job_security_configuration	(Optional) The name of the Security Configuration to be associated with the job.	any	null	no
glue_job_timeout	(Optional) The job timeout in minutes. The default is 2880 minutes (48 hours).	number	2880	no
glue_job_worker_type	(Optional) The type of predefined worker that is allocated when a job runs. Accepts a value of Standard, G.1X, or G.2X.	any	null	no
glue_version	(Optional) The version of glue to use, for example '1.0'. For information about available versions, see the AWS Glue Release Notes.	any	null	no

Outputs

Name	Description
glue_job_arn	Amazon Resource Name (ARN) of Glue Job
glue_job_id	Glue job name

Test

gem install bundler
bundle install --jobs 4 --retry 3
bundle exec kitchen test

Doc

pre-commit run -a

Autor

Module managed by pquintero