雲端服務 Amazon Web Service (2)
基本介紹
教學目標
學會如何根據大數據處理階段，選擇最適合的 Amazon 雲端服務。
重點概念
Big Data Portfolio of Services
在大數據處理的階段根據下表，就能在篩選出最適合的 Amazon 雲端服務。
| 處理階段 | 雲端服務 | |
|---|---|---|
| 收集 | AWS Direct Connect、AWS Import/Export、Amazon Kinesis | |
| 儲存 | Amazon S3、Amazon DynamoDB、Amazon Glacier | |
| 處理 & 分析 | Amazon Redshift 、Amazon EMR、Amazon EC2 |
(參考資源: AWS re:Invent 2014 | (BDT303) Construct ETL Pipeline w/ AWS Data Pipeline, Amazon EMR & Redshift )
Batch Processing
Amazon EMR 雲端服務搭配 Apache Mahout 開源專案進行資料分析處理。
- Building a Recommender with Apache Mahout on Amazon Elastic MapReduce (EMR)
http://blogs.aws.amazon.com/bigdata/post/Tx1TDK3HHBD4EZL/Building-a-Recommender-with-Apache-Mahout-on-Amazon-Elastic-MapReduce-EMR
Amazon EMR 雲端服務搭配 Apache Spark 開源專案進行資料分析處理。
- Run Spark and Spark SQL on Amazon Elastic MapReduce
https://aws.amazon.com/articles/Elastic-MapReduce/4926593393724923
Real-time Processing
Amazon Kinesis 雲端服務搭配 Amazon Redshift 雲端服務進行資料分析處理。
- Amazon Kinesis/Redshift編～アクセスログをkinesisで加工してTableauで表示してみよう① [全4回]
http://recipe.kc-cloud.jp/archives/6289
Amazon Kinesis 雲端服務搭配 Storm 開源專案進行資料分析處理。
- Marry Amazon Kinesis and Storm Using the New Kinesis Storm Spout
http://aws.amazon.com/blogs/aws/marry-kinesis-and-storm-using-the-new-kinesis-storm-spout/
相關資源
- AWS Direct Connect
http://aws.amazon.com/directconnect/ - AWS Import/Export
http://aws.amazon.com/importexport/ - Amazon S3
http://aws.amazon.com/s3/ - Amazon DynamoDB
http://aws.amazon.com/dynamodb/ - Amazon Redshift
http://aws.amazon.com/redshift/ - Amazon EMR
http://aws.amazon.com/elasticmapreduce/ - Apache Mahout
http://mahout.apache.org/ - AWS re:Invent 2014 | (BDT303) Construct ETL Pipeline w/ AWS Data Pipeline, Amazon EMR & Redshift
https://www.youtube.com/watch?v=oOIgMSv2rug