Sometimes you want control over the EC2 Instance placement strategy
That strategy can be defined using placement groups
When you create a placement group, you specify one of the following strategies for the group:
1) Cluster : clusters instances into a low-latency group in a single AZ
동일한 AZ, 동일한 Rack 으로 저지연, 네트워크성능 뛰어남
- Pros : Great network (10 Gbps bandwidth between instances)
- Cons : If the rack fails, all instances fails at the same time
- Use case :
Big Data job that needs to complete fast
Application that needs extremely low latency and high network throughput
2) Spread : spreads instances across underlyng hardware (max 7 instances per group per AZ) eg. critical applications
AZ여러군데 퍼뜨리는 방법으로, 모든 instance의 동시 처리실패 가능성을 낮춤, AZ 하나에 7개 인스턴스 제한
- Pros :
Can span across AZ
Reduced risk is simultaneous failure
EC2 Instances are on different physical hardware
- Cons :
Limited to 7 instances per AZ per placement group
- Use case :
Application that needs to maximize high availability
Critical Applications where each instance must be isolated from failure from each other
3) Partition : spreads instances across many different partitions (which rely on different sets of racks) within an AZ. Scales to 100s of EC2 instance per group (Hadoop, Cassandra, Kafka)
파티션 여러개에 인스턴스를 퍼뜨리는 방식으로, AZ 당 최대 7개의 파티션 제한, 최대 100개의 인스턴스 제약
- Up to 7 partitions per AZ
- Up to 100s of EC2 instances
- The instances in a partition do not share racks with the instances in the other partitions
- A partition failure can affect many EC2 but won't affect other partitions
- EC2 instances get access to the partition information as metadata
- Use cases :
HDFS, HBase, Kafka, Cassandra
[ # Placement Group 설정 ]
1) Placement Group 생성 :
create Placement Group > Strategy 선택 (Clustered/Spread/Partition) > 생성
2) Placement Group 설정 :
launch Instance > choose AMI > Choose Instance Type > Configure Instance 단계에서 Placement group 선택
* Partition 의 경우 auto distribution/혹은 파티션 지정 가능
* Clustered Placement group 은 free tier Instance Type 에서 선택 불가
These images can be customised at runtime using EC2 User data
- AMIs can be built for Linux or Windows machines
# Why would you use a custom AMI?
- Using a custom built AMI can provide the following advantages:
- Pre-installed packages needed
- Faster boot time (no need for EC2 user data at boot time)
- Machine comes configured with monitoring/enterprise software
- Security concerns - control over the machines in the network
- Control of maintenance and update of AMIs over time
- Active Directory Integration out of the box
- Installing your app ahead of time (for faster deployes when auto-scaling)
- Using someone else's AMI that is optimised for running an app, DB, etc..
# Using Public AMIs
- You can leverage AMIs from other people
- You can also pay for other people's AMI by the hour
-- These people have optimised the software
-- The machine is easy to run and configure
-- You basically rent "expertise" from the AMI creator
- AMI can be found and published on the Amazon Marketplace
* Warning : Do not use an AMI you don't trust, Some AMIs might come with malware or may not be secure
[ # AMI Storage ]
- Your AMI take space and they live in Amazone S3
- Amazon S3 is a durable, cheap and resilient storage where most of your backups will live (but you won't see them in the S3 console)
- By default, your AMIs are private, and locked for your account/region
- An AMI created for a region can only be seen in that region
- AMI is region locked and the same ID cannot be used across regions
- You can also make your AMIs public and share them with our AWS accounts or sell them on the AMI Marketplace
[ # AMI pricing ]
- AMIs live in Amazon S3, so you get charged for the actual space in takes in Amazon S3
- Amazon S3 pricing in US-EAST-I:
First 50TB/month: $0.023 per GB
Next 450TB/month:$0.022 per GB
- Overall it is quite inexpensive to store private AMIs
- Make sure to remove the AMIs you don't use
[ Cross Acount AMI Copy ]
AMI 는 공유가 가능.
타 계정의 AMI 를 복사하고자 할 경우 AMI 소유자가 권한을 부여해야 가능.
- You can share an AMI with another AWS account.
- Sharing an AMI doesn't affect the ownership of the AMI
- If you copy an AMI that has been shared with your account, you are the owner of the target AMI in your account
- To copy an AMI that was shared with you from another account, the owner of the source AMI must grant you read permissions for the storage that backs the AMI, either the associated EBS snapshot (for an Amazon EBS-backed AMI) or an associated S3 bucket (for an instance store-backed AMI).
* Limits
Windows AMI 와 같은 billingProduct AMI는 다른 계정으로부터 카피할 수 없음.
billingProduct AMI 로 EC2 인스턴스를 런칭한 후 해당 인스턴스로 AMI를 생성하여 복사하는 방식으로 복사가 가능.
1) You can't copy an encrypted AMI that was shared with you from another account. Instead, if the underlying snapshot and encryption key were shared with you, you can copy the snapshot while re-encrypting it with a key of your own. You own the copied snapshot, and can register it as a new AMI.
2) You can't copy an AMI with an associated billingProduct code that was shared with you from another account. This includes Windows AMIs and AMIs from the AWS Marketplace. To copy a shared AMI with a billingProduct code, launch an EC2 instance in your account using the shared AMI and then create an AMI from the instance.
# 내 AMI 를 기타 유저(AWS account)에게 공유하기
AMI 우클릭 > Modify Image Permissions > AWS account number 입력 및 Add Permission 클릭
* Add "create volume" permissions to the following associated snapshots when creating permissions
- 위 옵션을 체크할 경우 AMI에 대한 직접 copy 를 허용, 체크하지 않을 경우 직접 copy 를 불허.
- billingProduct AMI 를 카피하는 방식과 동일하게 본 AMI로 EC2 를 런칭한 후, AMI를 생성하는 방식으로 copy 가능
EC2의 inbound/outbound 방화벽 정책으로 SG 를 한개 설정하여 여러개의 EC2 인스턴스에 동일하게 적용시킬 수 있다. (region 제약)
- Security Groups are the fundamental of network security in AWS
- They control how traffic is allowed into or out of our EC2 Machines
- acting as a "firewall" on EC2 instances
- They regulate :
1) Access to Ports
2) Authorised IP ranges (CIDR) - ipv4 and ipv6
3) Control of inbound/outbound network
- can be attached to multiple EC2 instances
- locked down to a region /VPC combination
* if your application is not accessible (time out), then it's a security group issue (방화벽 문제)
* if your application gives a "connection refused" error, then it's an application error or it's not launched
- All inbound traffic is blocked by default
- All outbound traffic is authorised by default
[ # EC2 에 Apache 설치 ]
1) EC2 에 접속
2) > sudo su
root 로 switch user
3) > yum update -y
force update machine
4) > yum install -y httpd.x86_64
아파치 설치
5) > systemctl start httpd.service
서비스 시작
6) > enable httpd.service
enabled across reboots
7) curl localhost:80
테스트
8) ec2publicip:80
외부에서 접속해보기 -> connection time out 발생
9) Security group 의 inbound 설정에 http 80 포트 추가
10) 재시도 성공 확인
security groups can communicate straight through to other instances
[ 3. Elastic IPs ]
고정된 공인 아이피, Elastic IP 설정시 EC2 인스턴스를 재기동해도 공인아이피가 바뀌는 현상이 나타나지 않는다
Elastic IP를 사용하기 보단, DNS (Route 53) 을 사용 하는게 구조적으로 낫다
- Elastic IP is a public IPv4 IP you own as long as you don't delete it
* if restart EC2 instance, it can change its public IP
- if you need to have a fixed public IP for your instance, you need an Elastic IP
- with an Elastic IP address, you can mask the failure of an instance or software by rapidly remapping the address to another instance in your account.
- you can only have 5 Elastic IP in your account (can increase if you ask AWS)
* try to avoid using Elastic IP :
1) They often reflect poor architectural decisions
2) Instead, use a random public IP and register a DNS name to it
[ 3. EC2 User Data ]
인스턴스 런칭시 실행되는 초기 스크립트로써 업데이트수행/프로그램설치 등을 EC2 런칭과 동시에 수행시키고자 할 때 사용한다 (AMI 를 사용하여 대체할 수 있다)
- It is possible to bootstrap our instances using an EC2 User data script
- bootstrapping means launching commands when a machine starts
- That script is only run once at the instance first start (인스턴스 런칭과 동시에 스크립트 실행)
- EC2 user data is used to automate boot tasks such as:
1) Installing updates
2) Downloading common files from the internet
* The EC2 User Data Script runs with the root user
* where to put/change user data
생성시 : configure instance details step 의 advanced details 에서 설정
생성후 : instance 우클릭 > instance settings > change user data
[ 4. EC2 Instance Launch Types ]
EC2 인스턴스는 아래와 같이 5가지 런치타입이 존재한다.
애플리케이션의 목적에 따라 런치타입을 바꾸어 비용절감을 할 수 있다.
1. On Demand Instances : short workload, predictable pricing
2. Reserved : (Minimum 1year)
- Reserved Instances: long workloads
- Convertible Reserved Instances: long workloads with flexible instances
- Scheduled Reserved Instances: eg. every Thursday between 3 and 6 pm
3. Spot Instances : short workloads, for cheap, can lose instances (less reliable)
4. Dedicated Instances : no other customers will share your hardware
5. Dedicated Hosts : book an entire physical server, control instance placement
1. EC2 On Demand
필요시에만 사용, 비쌈
- Pay for what you use (billing per second, after the first minute)
- Has the highest cost but no upfront payment
- No long term commitment
- Recommended for short-term and un-interrupted workloads, where you can't predict how the application will behave.
2. EC2 Reserved Instances
일정 기간에 대한 선불제, 비교적 저렴
- Up to 75% discount compared to On-demand
- Pay upfront for what you use with long term commitment
- Reservation period can be 1 or 3 years
- Reserve a specific instance type
- Recommended for steady state useage applications(think database)
* Convertible Reserved Instance
- can change the EC2 instance type
- Up to 54% discount
* Scheduled Reserved Instances
- launch within time window you reserve
- When you require a fraction of day/week/month
3. EC2 Spot Instances
여유자원을 싸게 사용하는 방식으로 사용자가 최대 입찰 가격을 정해놓고 사용, 가격이 최대가격이상이 되면 중지
- Can get a discount of up to 90% compared to On-demand
- Instances that you can "lose" at any point of time if your max prices is less then the current spot price
- The Most cost-efficient instances in AWS
- Useful for workloads that are resilient(회복력있는/탄력적인) to failure
eg. Batch jobs, Data analysis, Image processing
- Not great for critical jobs or databases
- Great combo : Reserved Instances for baseline + On-Demand & Spot for peeks
4. EC2 Dedicated Hosts
사용자 전용의 물리적 서버, 딮한 설정 가능
- Physical dedicated EC2 server for your use
- Full control of EC2 Instance placement
- Visibility into the underlying sockets/physical cores of the hardware
- Allocated for your account for a 3 year period reservation
- More expensive
- Useful for software that have complicated licensing model
Or for companies that have strong regulatory or compliance needs
5. EC2 Dedicated Instances
계정에 귀속된 인스턴스
- Instances running on hardware that's dedicated to you
- May share hardware with other instances in same account
- No control over instance placement (can move hardware after Stop/Start)