Upstream Docker Swarm Support

[roidelapluie]

Dear Community,

I am working on integrating Docker Swarm discovery upstream in Prometheus.

The upstream work is at prometheus#7420

I did not start that work based on this fork ; as the SD mechanisms have changed since this fork was written. However, I think that the implementation I am proposing is more complete and all the usecases proposed by this fork should be available.

Your feedback is valuable to us.

[umang5195]

@roidelapluie I tired using Docker Swarm Service Discovery in prometheus, but facing problems using it. I followed the docker swarm support documentation. Created a daemon.json file and mounted /var/run/docker.sock in prometheus container. Container is giving permission denied error as prometheus is running as nobody and doesn't have access to mounted /var/run/docker.sock. Below is my prometheus.yml.
Prometheus Version : v2.20.1

 prometheus:
    image: prom/prometheus
    networks:
      - monitor
    ports:
      - "9090:9090"
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.path=/prometheus'
      - '--storage.tsdb.retention=${PROMETHEUS_RETENTION:-24h}'
    volumes:
      - prometheus:/prometheus
      - /home/efs/devops/dsm:/etc/prometheus:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
    deploy:
      mode: replicated
      replicas: 1
      resources:
        limits:
          memory: 1024M
        reservations:
          memory: 128M

Prometheus.yml

scrape_configs:
  - job_name: 'docker'
    dockerswarm_sd_configs:
    - host: unix:///var/run/docker.sock
      role: nodes

Error:
monitor_prometheus.1.uwda893b1nws@monitoring.dev.ai | level=error ts=2020-08-06T07:21:19.106Z caller=refresh.go:98 component="discovery manager scrape" discovery=dockerswarm msg="Unable to refresh target groups" err="error while listing swarm nodes: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get \"http://%2Fvar%2Frun%2Fdocker.sock/v1.24/nodes\": dial unix /var/run/docker.sock: connect: permission denied"