Use copy query to load data, so I can load a specific pattern/prefix, but I want to load data after a certain timestamp, such as greater than’ 2014-07-09 10:00:00′. What should I do?
>In the file Process the file before loading into S3 (and upload only the data with a timestamp greater than $SOME_TIMESTAMP)
>Use the COPY command to load the file into an intermediate table (it can be a temporary table-as long as you stay in the same session) and run:
insert into YOUR_ORIGINAL_TABLE (select * from YOUR_TEMP_TABLE where timestamp> WHATEVER_YOU_NEED)
So I am trying to start from S3 bucket Load the data into my Redshift database. I have a table’Example’ with the field’timestamp’ in the format’YY-MM-DD HH:MM:SS’.
Use copy query to load data, so I can load a specific pattern/prefix, but I want to load data after a certain timestamp, such as greater than ‘2014-07-09 10:00:00’. How can I handle it?
You have two options:
>Process the file before loading it into S3 (and upload only the timestamp Data greater than $SOME_TIMESTAMP)
>Use the COPY command to load the file into the intermediate table (it can be a temporary table-as long as you stay in the same session) and run:
< pre>insert into YOUR_ORIGINAL_TABLE (select * from YOUR_TEMP_TABLE where timestamp> WHATEVER_YOU_NEED)