AWS Redshift Copy Command from S3 filter out partitions

Question

I am trying to copy parquet files from S3 partitions to Redshift, Is there a way to filter out partitions under a folder other than looping through the partitions and doing one by one and filtering out the not needed partition.

If I need to copy the whole partition S3/01-01-2021 by filtering out Mon partition alone, is there a way. S3/01-01-2021/Mon S3/01-01-2021/Tue S3/01-01-2021/Wed S3/01-01-2021/Thu S3/01-01-2021/Fri

Answer 1

You can make a COPY manifest file and put only the files you want in it. Does that meet your need?

https://docs.aws.amazon.com/redshift/latest/dg/loading-data-files-using-manifest.html