AWS MSK+Docker+Prometheus+Grafana(테스트)

aws 문서
https://catalog.us-east-1.prod.workshops.aws/workshops/c2b72b6f-666b-4596-b8bc-bafa5dcca741/en-US/openmonitoring/installwithdocker

도커 설치
sudo yum install docker -y

도커 서비스 실행
sudo service docker start

sudo 명령어 없이 commands 실행
sudo usermod -aG docker ec2-user

디렉토리 생성
/home/ec2-user/prometheus 

yml.json 파일 생성 

-------------------------------------------------------------------------------------------------------------------
■■■prometheus.yml■■■
-------------------------------------------------------------------------------------------------------------------
# file: prometheus.yml
# my global config
global:
  scrape_interval:     10s

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
# The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
- job_name: 'prometheus'
  static_configs:
  # 9090 is the prometheus server port
  - targets: ['localhost:9090']
- job_name: 'broker'
  file_sd_configs:
  - files:
    - 'targets.json'
-------------------------------------------------------------------------------------------------------------------
■■■targets.json■■■
-------------------------------------------------------------------------------------------------------------------
[
  {
    "labels": {
      "job": "jmx"
    },
    "targets": [
      "b-1.cocomskcluster01.lg9j5t.c4.kafka.ap-northeast-2.amazonaws.com:11001",
      "b-2.cocomskcluster01.lg9j5t.c4.kafka.ap-northeast-2.amazonaws.com:11001"
    ]
  },
  {
    "labels": {
      "job": "node"
    },
    "targets": [
      "b-1.cocomskcluster01.lg9j5t.c4.kafka.ap-northeast-2.amazonaws.com:11002",
      "b-2.cocomskcluster01.lg9j5t.c4.kafka.ap-northeast-2.amazonaws.com:11002"
    ]
  }
]
-------------------------------------------------------------------------------------------------------------------

도커 런 
docker run -d -p 9090:9090 --name=prometheus -v /home/ec2-user/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml -v /home/ec2-user/prometheus/targets.json:/etc/prometheus/targets.json prom/prometheus --config.file=/etc/prometheus/prometheus.yml

EC2 ip:9090으로 접속 테스

-------------------------------------------------------------------------------------------------------------------
■■■Grafana■■■
 
도커 런
docker run -d -p 3000:3000 --name=grafana -e "GF_INSTALL_PLUGINS=grafana-clock-panel" grafana/grafana

접속(admin/admin)
ip:3000

prometheus 연동
Configuration - Add data source - prometheus 선택 - URL = ip:9090 // cookies Server - Save&test (오류 시 보안그룹에 EC2 IP 넣기)

대시 보드
https://grafana.com/grafana/dashboards (https://grafana.com/grafana/dashboards/?search=msk msk 대시보드)
Dashboards - import - 해당 코드 넣기

차후에는 커스텀

-------------------------------------------------------------------------------------------------------------------
■■■telegram 연동■■■

■Token 발급■
Alerting - Contact points - Telegram (토큰 먼저 생성)

Telegram bot token 생성
검색창에 BotFather 검색
V표 선택

/newbot

username 은 텔레그렘 시스템 안에서 중복 안되게 뒤에 bot 으로 끝나게끔
token 값 받아서 grafana에 입력

■ChatID 발급■
웹브라우저에 https://api.telegram.org/bot{토큰값}/getUpdates
(bot11111AAAA...) 약간 시간 걸림

쭉 값이 나오는데 음수 값이 ChatID 

grafana에 입력 하기 

알람 받을 방은 아까 만든 bot 이름을 채팅방에 초대 하면 그쪽으로 알람이 옴