AWS CloudWatch でオンプレのサーバーを監視する最小限の設定

はじめに

本記事ではオンプレサーバをaws のcloudwatch で監視する最小限の設定を説明します

構成と要件

collectdはインストールを前提

cloudwatch agent をオンプレのubuntu サーバーにインストール

オンプレサーバー側でTCP 443 をオープン

collectd のインストール

apt update -y && sudo apt install collectd -y

上記のコマンドで必ずcollectd はインストールしておいてください

CloudWatchエージェントのインストール

wget で入手して、dpkgコマンドでインストールします

wget https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb

sudo dpkg -i -E ./amazon-cloudwatch-agent.deb
root@david-minipc:~# wget https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb
--2024-06-02 18:44:54--  https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb
s3.amazonaws.com (s3.amazonaws.com) をDNSに問いあわせています... 3.5.12.145, 52.217.198.144, 52.217.174.208, ...
s3.amazonaws.com (s3.amazonaws.com)|3.5.12.145|:443 に接続しています... 接続しました。
HTTP による接続要求を送信しました、応答を待っています... 200 OK
長さ: 106072364 (101M) [application/octet-stream]
‘amazon-cloudwatch-agent.deb’ に保存中

amazon-cloudwatch-agent.deb          100%[===================================================================>] 101.16M  8.72MB/s    in 22s     

2024-06-02 18:45:17 (4.62 MB/s) - ‘amazon-cloudwatch-agent.deb’ へ保存完了 [106072364/106072364]

root@david-minipc:~# sudo dpkg -i -E ./amazon-cloudwatch-agent.deb
以前に未選択のパッケージ amazon-cloudwatch-agent を選択しています。
(データベースを読み込んでいます ... 現在 243339 個のファイルとディレクトリがインストールされています。)
./amazon-cloudwatch-agent.deb を展開する準備をしています ...
create group cwagent, result: 0
create user cwagent, result: 0
amazon-cloudwatch-agent (1.300040.0b650-1) を展開しています...
amazon-cloudwatch-agent (1.300040.0b650-1) を設定しています ...

設定ウィザードを起動

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard

上記のコマンドで、設定ウィザードを起動します.下記の通りに設定すれば問題ないはずです

root@david-minipc:~# /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard
================================================================
= Welcome to the Amazon CloudWatch Agent Configuration Manager =
=                                                              =
= CloudWatch Agent allows you to collect metrics and logs from =
= your host and send them to CloudWatch. Additional CloudWatch =
= charges may apply.                                           =
================================================================
On which OS are you planning to use the agent?
1. linux
2. windows
3. darwin
default choice: [1]:

Trying to fetch the default region based on ec2 metadata...
I! imds retry client will retry 1 timesD! should retry true for imds error : RequestError: send request failed
caused by: Put "http://169.254.169.254/latest/api/token": context deadline exceeded (Client.Timeout exceeded while awaiting headers)D! should retry true for imds error : RequestError: send request failed
caused by: Put "http://169.254.169.254/latest/api/token": context deadline exceeded (Client.Timeout exceeded while awaiting headers)2024/06/02 21:50:07 D! could not get region from imds v2 thus enable fallback
W! could not get region from ec2 metadata... EC2MetadataRequestError: failed to get EC2 instance identity document
caused by: RequestError: send request failed
caused by: Get "http://169.254.169.254/latest/dynamic/instance-identity/document": context deadline exceeded (Client.Timeout exceeded while awaiting headers)Are you using EC2 or On-Premises hosts?
1. EC2
2. On-Premises
default choice: [2]:

Please make sure the credentials and region set correctly on your hosts.
Refer to http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html
Which user are you planning to run the agent?
1. cwagent
2. root
3. others
default choice: [1]:
2
Do you want to turn on StatsD daemon?
1. yes
2. no
default choice: [1]:
2
Do you want to monitor metrics from CollectD? WARNING: CollectD must be installed or the Agent will fail to start
1. yes
2. no
default choice: [1]:

Do you want to monitor any host metrics? e.g. CPU, memory, etc.
1. yes
2. no
default choice: [1]:

Do you want to monitor cpu metrics per core?
1. yes
2. no
default choice: [1]:

Would you like to collect your metrics at high resolution (sub-minute resolution)? This enables sub-minute resolution for all metrics, but you can customize for specific metrics in the output json file.
1. 1s
2. 10s
3. 30s
4. 60s
default choice: [4]:

Which default metrics config do you want?
1. Basic
2. Standard
3. Advanced
4. None
default choice: [1]:

Current config as follows:
{
        "agent": {
                "metrics_collection_interval": 60,
                "run_as_user": "root"
        },
        "metrics": {
                "metrics_collected": {
                        "collectd": {
                                "metrics_aggregation_interval": 60
                        },
                        "cpu": {
                                "measurement": [
                                        "cpu_usage_idle"
                                ],
                                "metrics_collection_interval": 60,
                                "resources": [
                                        "*"
                                ],
                                "totalcpu": true
                        },
                        "disk": {
                                "measurement": [
                                        "used_percent"
                                ],
                                "metrics_collection_interval": 60,
                                "resources": [
                                        "*"
                                ]
                        },
                        "diskio": {
                                "measurement": [
                                        "write_bytes",
                                        "read_bytes",
                                        "writes",
                                        "reads"
                                ],
                                "metrics_collection_interval": 60,
                                "resources": [
                                        "*"
                                ]
                        },
                        "mem": {
                                "measurement": [
                                        "mem_used_percent"
                                ],
                                "metrics_collection_interval": 60
                        },
                        "net": {
                                "measurement": [
                                        "bytes_sent",
                                        "bytes_recv",
                                        "packets_sent",
                                        "packets_recv"
                                ],
                                "metrics_collection_interval": 60,
                                "resources": [
                                        "*"
                                ]
                        },
                        "swap": {
                                "measurement": [
                                        "swap_used_percent"
                                ],
                                "metrics_collection_interval": 60
                        }
                }
        }
}
Are you satisfied with the above config? Note: it can be manually customized after the wizard completes to add additional items.
1. yes
2. no
default choice: [1]:

Do you have any existing CloudWatch Log Agent (http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AgentReference.html) configuration file to import for migration?
1. yes
2. no
default choice: [2]:
2
Do you want to monitor any log files?
1. yes
2. no
default choice: [1]:

Log file path:
/var/log/messages
Log group name:
default choice: [messages]

Log group class:
1. STANDARD
2. INFREQUENT_ACCESS
default choice: [1]:
1
Log stream name:
default choice: [{hostname}]

Log Group Retention in days
1. -1
2. 1
3. 3
4. 5
5. 7
6. 14
7. 30
8. 60
9. 90
10. 120
11. 150
12. 180
13. 365
14. 400
15. 545
16. 731
17. 1096
18. 1827
19. 2192
20. 2557
21. 2922
22. 3288
23. 3653
default choice: [1]:

Do you want to specify any additional log files to monitor?
1. yes
2. no
default choice: [1]:
2
Do you want the CloudWatch agent to also retrieve X-ray traces?
1. yes
2. no
default choice: [1]:
2
Existing config JSON identified and copied to:  /opt/aws/amazon-cloudwatch-agent/etc/backup-configs
Saved config file to /opt/aws/amazon-cloudwatch-agent/bin/config.json successfully.
Current config as follows:
{
        "agent": {
                "metrics_collection_interval": 60,
                "run_as_user": "root"
        },
        "logs": {
                "logs_collected": {
                        "files": {
                                "collect_list": [
                                        {
                                                "file_path": "/var/log/messages",
                                                "log_group_class": "STANDARD",
                                                "log_group_name": "messages",
                                                "log_stream_name": "{hostname}",
                                                "retention_in_days": -1
                                        }
                                ]
                        }
                }
        },
        "metrics": {
                "metrics_collected": {
                        "collectd": {
                                "metrics_aggregation_interval": 60
                        },
                        "cpu": {
                                "measurement": [
                                        "cpu_usage_idle"
                                ],
                                "metrics_collection_interval": 60,
                                "resources": [
                                        "*"
                                ],
                                "totalcpu": true
                        },
                        "disk": {
                                "measurement": [
                                        "used_percent"
                                ],
                                "metrics_collection_interval": 60,
                                "resources": [
                                        "*"
                                ]
                        },
                        "diskio": {
                                "measurement": [
                                        "write_bytes",
                                        "read_bytes",
                                        "writes",
                                        "reads"
                                ],
                                "metrics_collection_interval": 60,
                                "resources": [
                                        "*"
                                ]
                        },
                        "mem": {
                                "measurement": [
                                        "mem_used_percent"
                                ],
                                "metrics_collection_interval": 60
                        },
                        "net": {
                                "measurement": [
                                        "bytes_sent",
                                        "bytes_recv",
                                        "packets_sent",
                                        "packets_recv"
                                ],
                                "metrics_collection_interval": 60,
                                "resources": [
                                        "*"
                                ]
                        },
                        "swap": {
                                "measurement": [
                                        "swap_used_percent"
                                ],
                                "metrics_collection_interval": 60
                        }
                }
        }
}
Please check the above content of the config.
The config file is also located at /opt/aws/amazon-cloudwatch-agent/bin/config.json.
Edit it manually if needed.
Do you want to store the config in the SSM parameter store?
1. yes
2. no
default choice: [1]:

What parameter store name do you want to use to store your config? (Use 'AmazonCloudWatch-' prefix if you use our managed AWS policy)
default choice: [AmazonCloudWatch-linux]

Which region do you want to store the config in the parameter store?
default choice: [ap-northeast-1]

Which AWS credential should be used to send json config to parameter store?
1. AKIAYS2NWVGTD2UQYO4M(From SDK)
2. AKIAYS2NWVGTD2UQYO4M(From Profile: AmazonCloudWatchAgent)
3. Other
default choice: [1]:
2
Successfully put config to parameter store AmazonCloudWatch-linux.
Program exits now.
root@david-minipc:~# 
root@david-minipc:~# aws ssm get-parameter --name AmazonCloudWatch-linux
{
    "Parameter": {
        "Name": "AmazonCloudWatch-linux",
        "Type": "String",
        "Value": "{\n\t\"agent\": {\n\t\t\"metrics_collection_interval\": 60,\n\t\t\"run_as_user\": \"root\"\n\t},\n\t\"logs\": {\n\t\t\"logs_collected\": {\n\t\t\t\"files\": {\n\t\t\t\t\"collect_list\": [\n\t\t\t\t\t{\n\t\t\t\t\t\t\"file_path\": \"/var/log/messages\",\n\t\t\t\t\t\t\"log_group_class\": \"STANDARD\",\n\t\t\t\t\t\t\"log_group_name\": \"messages\",\n\t\t\t\t\t\t\"log_stream_name\": \"{hostname}\",\n\t\t\t\t\t\t\"retention_in_days\": -1\n\t\t\t\t\t}\n\t\t\t\t]\n\t\t\t}\n\t\t}\n\t},\n\t\"metrics\": {\n\t\t\"metrics_collected\": {\n\t\t\t\"collectd\": {\n\t\t\t\t\"metrics_aggregation_interval\": 60\n\t\t\t},\n\t\t\t\"cpu\": {\n\t\t\t\t\"measurement\": [\n\t\t\t\t\t\"cpu_usage_idle\"\n\t\t\t\t],\n\t\t\t\t\"metrics_collection_interval\": 60,\n\t\t\t\t\"resources\": [\n\t\t\t\t\t\"*\"\n\t\t\t\t],\n\t\t\t\t\"totalcpu\": true\n\t\t\t},\n\t\t\t\"disk\": {\n\t\t\t\t\"measurement\": [\n\t\t\t\t\t\"used_percent\"\n\t\t\t\t],\n\t\t\t\t\"metrics_collection_interval\": 60,\n\t\t\t\t\"resources\": [\n\t\t\t\t\t\"*\"\n\t\t\t\t]\n\t\t\t},\n\t\t\t\"diskio\": {\n\t\t\t\t\"measurement\": [\n\t\t\t\t\t\"write_bytes\",\n\t\t\t\t\t\"read_bytes\",\n\t\t\t\t\t\"writes\",\n\t\t\t\t\t\"reads\"\n\t\t\t\t],\n\t\t\t\t\"metrics_collection_interval\": 60,\n\t\t\t\t\"resources\": [\n\t\t\t\t\t\"*\"\n\t\t\t\t]\n\t\t\t},\n\t\t\t\"mem\": {\n\t\t\t\t\"measurement\": [\n\t\t\t\t\t\"mem_used_percent\"\n\t\t\t\t],\n\t\t\t\t\"metrics_collection_interval\": 60\n\t\t\t},\n\t\t\t\"net\": {\n\t\t\t\t\"measurement\": [\n\t\t\t\t\t\"bytes_sent\",\n\t\t\t\t\t\"bytes_recv\",\n\t\t\t\t\t\"packets_sent\",\n\t\t\t\t\t\"packets_recv\"\n\t\t\t\t],\n\t\t\t\t\"metrics_collection_interval\": 60,\n\t\t\t\t\"resources\": [\n\t\t\t\t\t\"*\"\n\t\t\t\t]\n\t\t\t},\n\t\t\t\"swap\": {\n\t\t\t\t\"measurement\": [\n\t\t\t\t\t\"swap_used_percent\"\n\t\t\t\t],\n\t\t\t\t\"metrics_collection_interval\": 60\n\t\t\t}\n\t\t}\n\t}\n}",
        "Version": 3,
        "LastModifiedDate": 1717332810.58,
        "ARN": "arn:aws:ssm:ap-northeast-1:590184098214:parameter/AmazonCloudWatch-linux",
        "DataType": "text"
    }
}
root@david-minipc:~# 
root@david-minipc:~# 
root@david-minipc:~# /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m onPremise -c ssm:AmazonCloudWatch-linux -s
[agent]
  collection_jitter = "0s"
  debug = false
  flush_interval = "1s"
  flush_jitter = "0s"
  hostname = ""
  interval = "60s"
  logfile = "/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log"
  logtarget = "lumberjack"
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  omit_hostname = false
  precision = ""
  quiet = false
  round_interval = false

[inputs]

  [[inputs.cpu]]
    fieldpass = ["usage_idle"]
    interval = "60s"
    percpu = true
    totalcpu = true

  [[inputs.disk]]
    fieldpass = ["used_percent"]
    interval = "60s"
    tagexclude = ["mode"]

  [[inputs.diskio]]
    fieldpass = ["write_bytes", "read_bytes", "writes", "reads"]
    interval = "60s"

  [[inputs.logfile]]
    destination = "cloudwatchlogs"
    file_state_folder = "/opt/aws/amazon-cloudwatch-agent/logs/state"

    [[inputs.logfile.file_config]]
      file_path = "/var/log/messages"
      from_beginning = true
      log_group_class = "STANDARD"
      log_group_name = "messages"
      log_stream_name = "david-minipc"
      pipe = false
      retention_in_days = -1

/opt/aws/amazon-cloudwatch-agent/bin/config-downloader --output-file /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json --download-source ssm:AmazonCloudWatch-linux --mode onPrem --config /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml
****** processing amazon-cloudwatch-agent ******
Got Home directory: /root I! Set home dir Linux: /root I! SDKRegionWithCredsMap region: ap-northeast-1 Region: ap-northeast-1 credsConfig: map[] Successfully fetched the config and saved in /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/ssm_AmazonCloudWatch-linux.tmp
Start configuration validation...
2024/06/02 21:54:11 Reading json config file path: /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/ssm_AmazonCloudWatch-linux.tmp ...
2024/06/02 21:54:11 I! Valid Json input schema.
2024/06/02 21:54:11 Under path : /logs/ | Info : Got hostname david-minipc as log_stream_name
2024/06/02 21:54:11 D! delta processor required because metrics with diskio or net are set
2024/06/02 21:54:11 Configuration validation first phase succeeded
I! Detecting run_as_user...
Got Home directory: /root
I! Set home dir Linux: /root
I! SDKRegionWithCredsMap region:  ap-northeast-1
Got Home directory: /root

AWS側の設定

手順1: IAMロールの作成

  1. AWS Management Consoleにログイン:
  1. IAMサービスに移動:
    • コンソールの上部にある検索バーに「IAM」と入力し、IAMサービスに移動します

ポリシーの作成:

  • 左側のナビゲーションペインで「ポリシー」を選択し、「ポリシーの作成」をクリックします。

アクセスの許可を指定します。

ポリシー名の設定

ポリシーの作成をクリックします

ユーザーの作成

ポリシーに設定する ユーザを設定します

ユーザーの作成ボタンを押します

作成したポリシーをチェックして  「次へ」のボタンを押します

ユーザーの作成 ボタンを押します

下記のようにユーザーが設定されました

アクセスキーとシークレットを取得

IAMからユーザを選択して、先ほどのユーザーをクリックします

サードパーティサービスをクリックします

上記のレコメンデーションを理解し、、、、のチェックボックスにチェックして 「次へ」のボタンをクリックします

説明タグを入力したら「アクセスキーを作成」ボタンを押します

作成されたキーとシークレットアクセスキーは大切に誰にも知られないように補完しましょう

aws configure

aws cliをインストールしていない場合には、オンプレのサーバに下記のコマンドでインストールしましょう

sudo apt-get update
sudo apt-get install awscli
root@david-minipc:~# aws configure
AWS Access Key ID [None]: AKIAYdsfsdfdsD2UQYO4M
AWS Secret Access Key [None]: HN/09Fsdfsdfsk3XEvl5M+
Default region name [None]: 
Default output format [None]: 

上記のようにaws configure でIAMユーザに結びついているキーとシークレットを設定できます。

サービスの立ち上げ

下記のコマンドでサービスを立ち上げます

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m onPremise -c ssm:AmazonCloudWatch-linux -s
root@david-minipc:~# sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m onPremise -c ssm:AmazonCloudWatch-linux -s
****** processing amazon-cloudwatch-agent ******
Got Home directory: /root I! Set home dir Linux: /root I! SDKRegionWithCredsMap region: ap-northeast-1 Region: ap-northeast-1 credsConfig: map[] Successfully fetched the config and saved in /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/ssm_AmazonCloudWatch-linux.tmp
Start configuration validation...
2024/06/02 22:04:48 Reading json config file path: /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/ssm_AmazonCloudWatch-linux.tmp ...
2024/06/02 22:04:48 I! Valid Json input schema.
2024/06/02 22:04:48 Under path : /logs/ | Info : Got hostname david-minipc as log_stream_name
2024/06/02 22:04:48 D! delta processor required because metrics with diskio or net are set
2024/06/02 22:04:48 Configuration validation first phase succeeded
I! Detecting run_as_user...
Got Home directory: /root
Got Home directory: /root
I! Set home dir Linux: /root
I! SDKRegionWithCredsMap region:  ap-northeast-1
/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent -schematest -config /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml
Configuration validation second phase succeeded
Configuration validation succeeded
systemctl restart amazon-cloudwatch-agent

systemd で登録されていれば、systemctl でスタートできます

systemd でenable にしておいて、status で確認すれば下記のようになっているはずです

root@david-minipc:~# systemctl status  amazon-cloudwatch-agent
● amazon-cloudwatch-agent.service - Amazon CloudWatch Agent
     Loaded: loaded (/etc/systemd/system/amazon-cloudwatch-agent.service; enabled; vendor preset: enabled)
     Active: active (running) since Sun 2024-06-02 22:04:50 JST; 30min ago
   Main PID: 230967 (amazon-cloudwat)
      Tasks: 10 (limit: 13945)
     Memory: 30.5M
        CPU: 13.936s
     CGroup: /system.slice/amazon-cloudwatch-agent.service
             └─230967 /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent -config /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml -envconfig /opt/aws/amaz>

 6月 02 22:05:20 david-minipc start-amazon-cloudwatch-agent[231014]: I! Detecting run_as_user...
 6月 02 22:05:20 david-minipc start-amazon-cloudwatch-agent[231014]: Got Home directory: /root
 6月 02 22:05:20 david-minipc start-amazon-cloudwatch-agent[231014]: Got Home directory: /root
 6月 02 22:05:20 david-minipc start-amazon-cloudwatch-agent[231014]: I! Set home dir Linux: /root
 6月 02 22:05:20 david-minipc start-amazon-cloudwatch-agent[231014]: I! SDKRegionWithCredsMap region:  ap-northeast-1
 6月 02 22:05:20 david-minipc start-amazon-cloudwatch-agent[231014]: 2024/06/02 22:05:20 Under path : /logs/ | Info : Got hostname david-minipc as log_stream_name
 6月 02 22:05:20 david-minipc start-amazon-cloudwatch-agent[231014]: 2024/06/02 22:05:20 D! delta processor required because metrics with diskio or net are set
 6月 02 22:05:20 david-minipc start-amazon-cloudwatch-agent[231014]: 2024/06/02 22:05:20 Configuration validation first phase succeeded
 6月 02 22:05:20 david-minipc start-amazon-cloudwatch-agent[230967]: /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json does not exist or cannot read. Skipping it.
 6月 02 22:05:20 david-minipc start-amazon-cloudwatch-agent[230967]: I! Detecting run_as_user...

Cloudwatch における監視

問題なければホスト(オンプレ)からログを監視できるはずです。

いろいろなメトリックを取得することができます

cloudwatch のほうに移っていただくと、いろいろメトリックを設定できます。

メトリクス>>すべてのメトリクス から

設定が正しくされていれば、CWAgent から host ,name を選択することで、ご自身のオンプレのサーバからいろいろなメトリックスがを選択できるはずです。下記の例では 私のオンプレサーバであるdavid-minipcのいろいろなメトリックを選択できるようになっています

いろいろ、いじってみました、Cloudwatchのダッシュボードを利用すると、月額3USD請求されますのでご注意ください。

日割りの請求金額はコストで確認できます。

下記にあるように $3.00 par Dashboard par month とあるので、ご注意ください

最後に

いかがでしたでしょうか?これでだいたいオンプレサーバにcloudwatch agent をインストールして数値を取得するところまではお判りいただけたと思います。ただ、ダッシュボード一つにつき3USDはちょっと高いなというのが私の感覚です(12か月の無料枠でもカバーできません)

本記事がお役に立てれば幸いです

「AWS CloudWatch でオンプレのサーバーを監視する最小限の設定」への1件の返信

コメントを残す

メールアドレスが公開されることはありません。 が付いている欄は必須項目です