Q: Does Amazon Redshift support multi-zone deployment?
Currently, Amazon Redshift only supports single availability zone deployment. By loading data from the same set of Amazon S3 input files into two Amazon Redshift data warehouse clusters in each AZ, you can run data warehouse clusters in multiple AZs. With Redshift Spectrum, you can run multiple clusters across Availability Zones and access data in Amazon S3 without loading it into the cluster. In addition, you can also restore the data warehouse cluster from the data warehouse cluster snapshot to another availability zone.
Q: How does Amazon Redshift back up data? How to restore the cluster from backup?
When loading data, Amazon Redshift copies all data in the data warehouse cluster and continuously backs it up to S3. Amazon Redshift always tries to maintain at least three copies of data (the original data on the compute node, the replica data, and the backup data on Amazon S3). Redshift can also asynchronously copy your snapshots to S3 in another region for disaster recovery.
By default, Amazon Redshift enables automated backups of data warehouse clusters with a one-day retention period. You can configure it to be as long as 35 days.
Free backup storage is limited by the total storage size on the nodes in the data warehouse cluster, and is only applicable to activated data warehouse clusters. For example, if you have a total data warehouse storage size of 8TB, then we will provide up to 8TB of backup storage at no additional charge.
Q: How can I scale the size and performance of an Amazon Redshift data warehouse cluster?
How do you want to improve query performance or deal with excessive use of CPU, memory or I/O, then you can increase the data warehouse cluster through AWS Management Console or ModifyCluster API The number of nodes. When you modify your data warehouse cluster, the requested changes are applied immediately. You can obtain free metrics for computing usage, storage usage, and read/write traffic of Amazon Redshift data warehouse clusters through the AWS Management Console or Amazon CloudWatch API. You can also add more user-defined indicators through the custom indicator function of Amazon Cloudwatch.
With Redshift Spectrum, you can run multiple Amazon Redshift clusters that access the same data in Amazon S3. You can use different clusters for different use cases. For example, you can use one cluster for standard reports and another cluster for data science queries. Your marketing team can use its own cluster that is different from the operations team. According to the type and number of nodes in the local cluster and the number of files that need to be processed to run the query, Redshift Spectrum will automatically allocate the tasks of the query to multiple Redshift Spectrum worker threads outside the shared resource pool to read and process data from Amazon S3 , And then return the results to the Amazon Redshift cluster for any remaining processing.
Q: Are Amazon Redshift and Redshift Spectrum compatible with my preferred business intelligence software package and ETL tools?
Amazon Redshift uses industry standard SQL and can be accessed using standard JDBC and ODBC drivers. You can download Amazon Redshift custom JDBC and ODBC drivers from the “Connect Client” tab of the Redshift console. Our integrations with mainstream BI and ETL vendors have been verified, and many of them offer free trials to help you get started with loading and analyzing data. You can also enter the AWS Marketplace to deploy and configure solutions that can be used with Amazon Redshift within minutes.
Redshift Spectrum supports all Amazon Redshift client tools. Client tools can continue to use ODBC or JDBC connections to connect to Amazon Redshift cluster endpoints without any changes.
The query syntax and query functions used to access the tables in Redshift Spectrum are exactly the same as those used to access the tables in the local storage of the Redshift cluster. You can refer to the external table using the schema name defined in the CREATE EXTERNAL SCHEMA command used to register the external table.
Q: What data formats and compression formats does Redshift Spectrum support?
Redshift Spectrum currently supports many open source data formats, including Avro, CSV, Grok, Ion, JSON, ORC, Parquet, RCFile, RegexSerDe, SequenceFile, TextFile and TSV.
Redshift Spectrum currently supports Gzip and Snappy compression.
Q: How to load data into Amazon Redshift data warehouse?
You can load data into Amazon Redshift from a range of data sources, including Amazon S3, Amazon DynamoDB, Amazon EMR, AWS Glue, AWS Data Pipeline and/or Amazon Any SSH-enabled host on EC2 or locally. Amazon Redshift attempts to load data into each computing node in parallel to maximize the data ingestion speed of the data warehouse cluster.
Yes, users can connect to Amazon Redshift using ODBC or JDBC and issue the ‘insert’ SQL name to insert data. Please note that this is slower than using S3 or DynamoDB, because those methods load data to each compute node in parallel, while SQL insert statements load data through a single leader node.
Q: How do I load data from existing Amazon RDS, Amazon EMR, Amazon DynamoDB, and Amazon EC2 data sources to Amazon Redshift?
You can use the COPY command to load data directly from Amazon EMR, Amazon DynamoDB, or any SSH-enabled host into Amazon Redshift in parallel. In addition, you can use Redshift Spectrum to load data from Amazon S3 into the cluster using a simple INSERT INTO command. In this way, you can load data in various formats, such as Parquet and RC, into the cluster. Please note that if you use this method, Redshift Spectrum will accumulate billing for the amount of data scanned from Amazon S3.
In addition, many ETL companies have also certified Amazon Redshift in order to use it with their own tools, and many of them also provide free trials to help you start loading data. AWS Data Pipeline provides high-performance, reliable, and fault-tolerant solutions that can load data from various AWS data sources. You can use AWS Data Pipeline to specify the data source and ideal data transformation, and then execute a pre-written import script to load your data into Amazon Redshift. In addition, AWS Glue is a fully managed Extract, Transform, and Load (ETL) service that allows you to easily prepare and load data for analysis. You can create and run AWS Glue ETL tasks with just a few clicks in the AWS Management Console.
Q: Can I directly access Amazon Redshift compute nodes?
No. Your Amazon Redshift compute node is in a private network space and can only be accessed from the leader node of the data warehouse cluster. This provides another layer of protection for your data security.
