Prometheus配置监控ip、端口连通，get、post接口连通和状态码

##方法：blackbox_exporter 部署在一台主机上集中配置

第一部分
1.1下载blackbox_exporter安装包
wget https://github.com/prometheus/blackbox_exporter/releases/download/v0.19.0/blackbox_exporter-0.19.0.linux-amd64.tar.gz

tar xf blackbox_exporter-0.19.0.linux-amd64.tar.gz
cd blackbox_exporter-0.19.0.linux-amd64/

1.2，启动配置
cat > /usr/lib/systemd/system/blackbox_exporter.service << EOF

[Unit]
Description=blackbox_exporter
After=network.target

[Service]
User=root
Type=simple
ExecStart=/apps/mon/blackbox_exporter-0.19.0.linux-amd64/blackbox_exporter --config.file=/apps/mon/blackbox_exporter-0.19.0.linux-amd64/blackbox.yml
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

1.3 加载，启动
systemctl daemon-reload
systemctl start blackbox_exporter.service
systemctl enable blackbox_exporter.service
systemctl status blackbox_exporter.service

1.4 blackbox_exporter配置文件
blackbox.yml
位置：/…/blackbox_exporter/blackbox.yml（取决于blackbox_exporter的安装位置）

modules:
http_2xx:
prober: http
http_post_2xx:
prober: http
http:
method: POST
tcp_connect:
prober: tcp
pop3s_banner:
prober: tcp
tcp:
query_response:
- expect: “^+OK”
tls: true
tls_config:
insecure_skip_verify: false
grpc:
prober: grpc
grpc:
tls: true
preferred_ip_protocol: “ip4”
grpc_plain:
prober: grpc
grpc:
tls: false
service: “service1”
ssh_banner:
prober: tcp
tcp:
query_response:
- expect: “^SSH-2.0-”
- send: “SSH-2.0-blackbox-ssh-check”
irc_banner:
prober: tcp
tcp:
query_response:
- send: “NICK prober”
- send: “USER prober prober prober :prober”
- expect: “PING ?[^ ]+)”
send: “PONG ${1}”
- expect: “^:[ ]+ 001”
icmp:
prober: icmp

上面是自带的，下面是自定义的

http_post_2xx_with_prometheus_post_check_token:
prober: http
http:
method: POST
headers:
Content-Type: application/json #添加头部
body: ‘{“token”:“prometheus_post_check_token”}’ #发送的相关数据

第二部分
prometheus配置
prometheus.yml
位置：/etc/prometheus/prometheus.yml

全局配置

global:

默认拉取频率

scrape_interval: 15s

拉取超时时间

scrape_timeout: 10s

评估规则频率

evaluation_interval: 15s

规则文件配置

rule_files: [‘/etc/prometheus/rules/*.yml’]

告警配置

alerting:
alertmanagers:

follow_redirects: true
scheme: http
timeout: 10s
api_version: v2
static_configs:
- targets: []

拉取配置，添加监控项

scrape_configs:

监控prometheus

job_name: prometheus
metrics_path: /metrics
static_configs:
- targets:
  - localhost:9090

监控ip是否能ping通，docker启动的blackbox-exporter不建议用此监控，可能会有报错

job_name: icmp_ping
metrics_path: /probe
params:
module: [icmp]
file_sd_configs:
- files: [‘/apps/mon/prometheus-2.30.0.linux-amd64/dis/icmp_ping.yml’]
  refresh_interval: 10s
  relabel_configs:
- source_labels: [address]
  regex: (.*)(:80)?
  target_label: __param_target
  replacement: ${1}
- source_labels: [__param_target]
  target_label: instance
- source_labels: [__param_target]
  regex: (.*)
  target_label: ping
  replacement: ${1}
- source_labels: []
  regex: .*
  target_label: address
  replacement: localhost:9115

#icmp_ping.yml
位置：/etc/prometheus/conf.d/icmp_ping/icmp_ping.yml

targets: [‘192.168.7.254’, ‘192.168.10.200’]
labels:
group: ‘ping监控’

监控端口是否能连通

job_name: tcp_port
metrics_path: /probe
params:
module: [tcp_connect]
file_sd_configs:
- files: [‘/apps/mon/prometheus-2.30.0.linux-amd64/dis/tcp_port.yml’]
  refresh_interval: 10s
  relabel_configs:
- source_labels: [address]
  target_label: __param_target
- source_labels: [__param_target]
  target_label: instance
- target_label: address
  replacement: localhost:9115

#tcp_port.yml
位置：/etc/prometheus/conf.d/tcp_port/tcp_port.yml

targets: [‘10.120.150.12:80’, ‘10.120.150.13:80’]
labels:
group: ‘Nginx-端口监控’

监控get请求

job_name: http_get
metrics_path: /probe
params:
module: [http_2xx]
file_sd_configs:
- files: [‘/etc/prometheus/conf.d/http_get/*.yml’]
  refresh_interval: 10s
  relabel_configs:
- source_labels: [address]
  target_label: __param_target
- source_labels: [__param_target]
  target_label: instance
- target_label: address
  replacement: 192.168.7.254:9115

#http_get.yml
位置：/etc/prometheus/conf.d/http_get/http_get.yml

targets:
- http://192.168.7.254:8082/api/heartbeat/check/
  labels:
  name: ‘get测试’
targets:
- http://192.168.7.254:8083/api/heartbeat/check_test/
  labels:
  name: ‘get测试2’

监控post请求

job_name: http_post
metrics_path: /probe
params:
module: [http_post_2xx]
file_sd_configs:
- files: [‘/etc/prometheus/conf.d/http_post/*.yml’]
  refresh_interval: 10s
  relabel_configs:
- source_labels: [address]
  target_label: __param_target
- source_labels: [__param_target]
  target_label: instance
- target_label: address
  replacement: 192.168.7.254:9115

#监控post请求

job_name: http_post_with_token
metrics_path: /probe
params:
module: [http_post_2xx_with_prometheus_post_check_token]
file_sd_configs:
- files: [‘/etc/prometheus/conf.d/http_post_with_token/*.yml’]
  refresh_interval: 10s
  relabel_configs:
- source_labels: [address]
  target_label: __param_target
- source_labels: [__param_target]
  target_label: instance
- target_label: address
  replacement: 192.168.7.254:9115

#http_post.yml
位置：/etc/prometheus/conf.d/http_post/http_post.yml

targets:
- http://192.168.7.254:8082/api/heartbeat/post_check/
  labels:
  name: ‘post测试’

#http_post_with_token.yml
位置：/etc/prometheus/conf.d/http_post_with_token/http_post_with_token.yml

targets:
- http://192.168.7.254:8082/api/heartbeat/post_check/
  labels:
  name: ‘post带body测试’

#规则
rules.yml
位置：/etc/prometheus/rules/rules.yml

groups:

name: probe_http_status_code
rules:
- alert: probe_http_status_code
  expr: probe_http_status_code != 200
  for: 1m
  labels:
  severity: critical
  annotations:
  summary: “{{ $labels.instance }} 状态码异常”
  description: “请尽快检测”
  groups:
name: probe_success
rules:
- alert: probe_success
  expr: probe_success == 0
  for: 1m
  labels:
  severity: critical
  annotations:
  summary: “接口/主机/端口 {{ $labels.instance }} 无法联通”
  description: “请尽快检测”

#模板ID 9965号模板，数据源选择Prometheus 模板下载地址
https://grafana.com/grafana/dashboards/9965文章来源地址https://uudwc.com/A/GOdA