AWS is one of the platform that you can run your web server or application quickly and easily
This is a course learning note on AWS Fundamental:Going Cloud Native, provide by AWS through Coursera. There are more settings regarding security and accessibility by setting Roles, Groups, Users, permission policies, or issues on balancing your incoming traffic by utilizing Load Balancer, but we are not going to talk about that in this post.
VPC, an Amazon Virtual Private Cloud, in which we put all our application and services in, so that we can have full control on how internet traffic goes in and out. …
Step by Step setting up Apache Spark cluster based on Hadoop
This is a step by step notes to setting up Apache Spark cluster for team projects, help received from Mr. Steven Chiu, Instructor of Institute for Information Industry, Taiwan.
Spark is probably one of the most popular Big data processing platforms that being used nowadays. In order to manage vast amount of data that we collect for our team projects, we decided to utilize Hadoop and Spark platform for furthur data processing.
This note demonstrates setting up Spark cluster on Linux ubuntu system.
First of all, we need to have all needed installation packages prepared for working environment. In our class, for example, we have prepared jdk(1.8.0), Scala(2.12.11), Anaconda(2020.02-Linux-X86_64) for python 3.7, hadoop(2.10.0) and Spark(2.4.5), stored in ~/Downloads directory for demonstration purpose. …