This job type allows you to copy S3 objects from one bucket to another.
This is useful for staging purposes, especially for running other Athena job types using this location.
“Run Time” is the Time in UTC time zone when you want the job to run.
“Number of Runs per day” is the number of times per day you want the job to run. This could be anywhere from 1 to 1440 times per day (every minute is currently the maximum)
Fill in the “Email address” to be receive job run notifications by email. This only works if you’ve signed up for SendGrid and completed the details in the SendGrid section.
“Pipeline Name” groups jobs together to run consecutively and dependently. This means subsequent job steps eg: Step 2 will only run once Step 1 has completed successfully. To change the Pipeline that the job belongs to, simply change the Pipeline Name field.
“Region” is the AWS region where you want the query to run.
“Job Name” is the unique name of the job. It can’t exist even in a different pipeline.
“From Bucket”
Name of the source bucket in S3 containing the objects you want to copy (exlude the s3:// prefix)
“To Bucket”
Name of the destination bucket to copy the S3 objects to (exlude the s3:// prefix)
“Prefix”
(Optional) The prefix of the files you want to copy in S3 eg: path/to/my/text/
If you specify a prefix, this is then created in the destination bucket.
“Sub Folder Names”
(Optional) The pipe ie: | separated list of sub folders to copy eg: myfolder1|myfolder2
This enables you to filter which objects you want to copy. Otherwise the bucket’s entire contents will be copied.