DEVELOPyo

[ AWS Monitoring : CloudWatch Alarms ]

Alarm은 metric 값에 대한 알림을 주기위해 사용

- Alarms are used to trigger notifications for any metric

- Various options (sampling, %, max, min, etc..)

- Alarms States :

  1) OK

  2) INSUFFICIENT_DATA

  3) ALARM

- Period :

  -- Length of time in seconds to evaludate the metric

  -- High resolution custom metrics : 10 sec, 30 sec, or multiples of 60 sec

[ CloudWatch Alarm Targets ]

CloudWatch 가 Alarm 이 울리면(metric 값에 의해) EC2 를 Stop, Terminate, Reboot, or Recover 할 수 있음

Auto Scaling 실행

SNS 로 알림

- Stop, Terminate, Reboot, or Recover an EC2 Instance 

- Trigger Auto Scaling Action

- Send notification to SNS (from which you can do pretty much anything)

[ EC2 Instance Recovery ]

- Status Check :

  Instance status = check the EC2 VM

  System status = check the underlying hardware

Recovery : Same Private, Public, Elastic IP, metadata, placement group

[ CloudWatch Alarm : good to know ]

CloudWatch Logs Metrics Filter 에 의해 알람이 생성되며 알람은 SNS 를 통해 adm에게 알림

- Alarms can be created based on CloudWatch Logs Metrics Filters

- To test alarms and notifications, set the alarm state to Alarm using CLI

aws cloudwatch set-alarm-state --alarm-name "myalarm" --state-value ALARM --state-reason "testing purposes"

[ CloudWatch Events ]

AWS 서비스 관제기능

EC2 인스턴스 기동 등의 AWS 서비스로부터 이벤트 인터셉트. 스케쥴링이나 크론으로 설정 가능

* EC2 인스턴스 실행시 관제 메시지 발생시키거나 하는 등의 기능

Event Pattern : Intercept events from AWS services (Sources)

- Example sources: EC2 Instance Start, CodeBuild Failure, S3, Trusted Advisor

- Can intercept any API call with CloudTrail integration

Schedule or Cron

A JSON payload is created from the event and passed to a target

- Compute : Lambda, Batch, ECS task

- Integration : SQS, SNS, Kinesis Data Streams, Kinesis Data Firehose

- Orchestration : Step Functions, CodePipeline, CloudBuild

- Maintenance : SSM, EC2 Actions

[ AWS Monitoring : CloudWatch Logs ]

- Applications can send logs to CloudWatch using the SDK

- CloudWatch can collect log from :

  1) Elastic Beanstalk : collection of logs from application

  2) ECS : collection from containers

  3) AWS Lambda : collection from function logs

  4) VPC Flow Logs : VPC specific logs

  5) API Gateway

  6) CloudTrail based on filter

  7) CloudWatch log agents : for example on EC2 machines

  8) Route53 : Log DNS queries

- CloudWatch Logs can go to :

  1) Batch exporter to S3 for archival

  2) Stream to ElasticSearch cluster for further analytics

[ AWS CloudWatch Logs ]

- Logs storage architecture :

  -- Log groups : arbitrary(임의의) name, usually representing an application

  -- Log stream : instances within application/log files/containers

- Can define log expiration policies (never expire, 30 days, etc..)

- Using the AWS CLI we can trail CloudWatch logs

- To send logs to CloudWatch, make sure IAM permissions are correct!

- Security : encryption of logs using KMS at the Group Level

[ CloudWatch Logs Metric Filter & Insights ]

- CloudWatch Logs can use filter expressions

  -- For example, find a specific IP inside of a log

  -- Metric filters can be used to trigger alarms

※ CloudWatch Logs Insights (new - Nov 2018) can be used to query logs and add queries to CloudWatch Dashboards

[ CloudWatch Logs for EC2 ]

- By default, no logs from your EC2 machine will go to CloudWatch

- You need to run a CloudWatch agent on EC2 to push the log files you want

- Make sure IAM permissions are correct

- The CloudWatch log agent can be setup on-premises too

※ On-premise : 자사가 보유한 서버에 서비스 구축

※ Off-premise : AWS와 같은 원격 클라우드 등에 서비스 구축

[ CloudWatch Log Agent & Unified Agent ]

- For virtual servers (EC2 instances, on-premise servers..)

1. CloudWatch Logs Agent

   - Old version of the agent

   - Can only send to CloudWatch Logs

-2. CloudWatch Unified Agent

   - Collect additional system-level metrics such as RAM, processes, etc...

   - Collect logs to send to CloudWatch Logs

   - Centralized configuration using SSM Parameter Store

[ CloudWatch Unified Agent - Metrics ] 

- Collected directly on your Linux server / EC2 instance

1) CPU (active, guest, idle, system, user, steal)

2) Disk metrics (free, used, total), Disk IO (writes, reads, bytes, iops)

3) RAM (free, inactive, used, total, cached)

4) Netstat (number of TCP and UDP connections, net packets, bytes)

5) Processes (total, dead, bloqued, idle, running, sleep)

6) Swap Sapce (free, used, used &)

※ Reminder : out-of-the box metrics for EC2 - disk, CPU, network (high level)

[ AWS Monitoring : CloudWatch ]

[ CloudWatch Metrics ]

- CloudWatch provides metrics for every services in AWS

- Metric is a variable to monitor (CPUUtilization, NetworkIn..)

- Metrics belong to namespaces

- Demension is an attribute of a metric (instance id, environment, etc...)

- Up to 10 dimensions per metric

- Metrics have timestamps

- Can create CloudWatch dashboards of metrics

[ EC2 Detailed monitoring ] 

- EC2 instance metrics have metrics "every 5 minutes"

- With detailed monitoring (for a cost), you get data "every 1 minute"

- Use detailed monitoring if you want to scale faster for your ASG

- The AWS Free Tier allows us to have 10 detailed monitoring metrics

※ Note : EC2 Memory usage is by default no pushed (must be pushed from inside the instance as a custom metric)

[ CloudWatch Custom Metrics ]

2주 과거, 2시간 미래의 매츠릭 데이터 포인트 사용 가능 (EC2 instance 시간이 정확하게 맞춰져있어야 함)

PutMetricData API 호출을 통해 커스텀 매트릭을 CloudWatch에  보낼 수 있음

- Possibility to define and send your own custom metrics to CloudWatch

- Example : memory(RAM) usage, disk space, number of logged in users

- Use API call PutMetricData

- Ability to use dimensions (attributes) to segment metrics

  -- Instance.id

  -- Environment.name

- Metric resolution (StorageResolution API parameter - two possible value) :

  -- Standard : 1 minute (60 seconds)

  -- High Resolution : 1/5/10/30 second(s) - Higher cost

※ Important : Accepts metric data points two weeks in the past and two hours in the future (make usre to configure your EC2 instance time correctly)

[ CloudWatch Dashboards ]

여러개의 AWS Account 및 regions 의 그래프를 대시보드에 사용 가능

- Great way to setup custom dashboards for quick access to key metrics and alarms

- Dashboards are global

- Dashboards can include graphs from different AWS accounts and regions ***

- You can change the time zone & time range of the dashboards

- You can setup automatic refresh (10s, 1m, 2m, 5m, 15m)

- Dashboards can be shared with people who don't have an AWS account (public, email address, 3rd party SSO provider through Amazon Cognito)

- Pricing :

  --  3 dashboards (up to 50 metrics) for free

  -- $3 per dashboard per month afterwards