Portainer and Prometheus

Portainer lets you manage your docker containers via web UI.

In this article we will put up two sets of containers: portainer and prometheus.

Here is our file structure after we have created all files from this article:

- /home/user/docker
  docker-compose.yml
  - data
    - prometheus
      - config
        prometheus.yml
        rules.yml
    - alertmanager
      - config
        alertmanager.yml

Files for Prometheus

Our Prometheus will select metrics from Jira and send alerts to a Slack channel. You can watch this video for more details.

prometheus.yml

global:
  scrape_interval:     15s
  evaluation_interval: 15s
rule_files:
  - "/etc/prometheus/rules.yml"
alerting:
 alertmanagers:
   - static_configs:
     - targets:
       - alertmanager:9093
scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']
  - job_name: 'jira'
    scheme: https
    metrics_path: '/jira/plugins/servlet/prometheus/metrics'
    static_configs:
      - targets: ['localhost:8085']

rules.yml

groups:
  - name: AllInstances
    rules:
    - alert: InstanceDown
      expr: up == 0
      for: 1m
      annotations:
        title: 'Instance {{ $labels.instance }} down'
        description: '{{ $labels.instance }} of job {{ $labels.job }} has been down for more than 1 minute.'
      labels:
        severity: 'critical'

    - alert: TooManyUsers
      expr: jira_all_active_users_gauge > 1
      annotations:
        title: 'Too many users for {{ $labels.instance }}'
        description: '{{ $labels.instance }} of job {{ $labels.job }} has more than 1 user.'
      labels:
        severity: 'meduim'

alertmanager.yml

global:
  resolve_timeout: 30s
  slack_api_url: 'your slack api url'

route:
  receiver: 'slack-notifications'

receivers:
- name: 'slack-notifications'
  slack_configs:
  - channel: '#alerts'
    send_resolved: true
    icon_url: https://avatars3.githubusercontent.com/u/3380462
    title: |-
      [{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}] {{ .CommonLabels.alertname }} for {{ .Comm$
      {{- if gt (len .CommonLabels) (len .GroupLabels) -}}
        {{" "}}(
        {{- with .CommonLabels.Remove .GroupLabels.Names }}
          {{- range $index, $label := .SortedPairs -}}
            {{ if $index }}, {{ end }}
            {{- $label.Name }}="{{ $label.Value -}}"
          {{- end }}
        {{- end -}}
        )
      {{- end }}
    text: >-
      {{ range .Alerts -}}
      *Alert:* {{ .Annotations.title }}{{ if .Labels.severity }} - `{{ .Labels.severity }}`{{ end }}

      *Description:* {{ .Annotations.description }}

      *Details:*
        {{ range .Labels.SortedPairs }} • *{{ .Name }}:* `{{ .Value }}`
        {{ end }}

Run portainer

We will run portainer in a container. We create a docker-compose.yml file in the /home/user/docker folder:

version: '3'
services:
  portainer:
    image: portainer/portainer
    container_name: portainer
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./portainer:/data
    ports:
      - 8080:9000
    expose:
      - 9000

Let’s run this docker-compose.yml file with:

docker-compose up -d

Now our portainer runs on port 8080.

Create prometheus stack in portainer

Now we can login to portainer with localhost:8080.

We will be asked to set the password for the admin user:

Set the password and push the Create user button:

Choose Local:

Push the Connect button:

Click on the available docker:

Click on the Stacks section:

Click on the Add stack button and add the following code:

version: '2'
services:
  prometheus:
    image: prom/prometheus:latest
    container_name: monitoring_prometheus
    restart: unless-stopped
    volumes:
      - /home/user/docker/data/prometheus/config:/etc/prometheus/
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
    expose:
      - 9090
    ports:
      - 9090:9090
   
  alertmanager:
    image: prom/alertmanager:latest
    volumes:
      - /home/user/docker/data/alertmanager/config:/etc/alertmanager/
    ports:
      - 9093:9093
    expose:
      - 9093