Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

Prometheus Configuration · microck | DecimalAI

microck/prometheus-configuration

SkillScore68Maintained

Set up Prometheus for comprehensive metric collection, storage, and monitoring of infrastructure and applications. Use when implementing metrics collection, setting up monitoring infrastructure, or configuring alerting systems.

DevOpsby microck

Links this skill to your agents and keeps it current with the author’s updates.

Automatic

DevOps & SRE pack

1 · One-time — install the CLI (docs ↗)

pip install decimalai

2 · Get the skill — writes .claude/skills/microck-prometheus-configuration/SKILL.md

decimalai skills pull microck-prometheus-configuration --out .claude/skills/

No CLI? Get the same file — curl · raw file

curl -fsSL --create-dirs -o .claude/skills/microck-prometheus-configuration/SKILL.md https://app.decimal.ai/s/microck-prometheus-configuration/SKILL.md

Raw file: /s/microck-prometheus-configuration/SKILL.md ↗ · machine-readable summary at /s/microck-prometheus-configuration.json ↗

In productionGlobal

how this skill behaves on real traffic · last 30 days

No production data for this skill yet — usage numbers appear here once agents start running it.

Anonymized aggregate across every org using this skill — proof from real production use, not a demo.

Does this skill help? · Benchmark: with vs. withoutsign in to run this on your agents

22 test cases from the publisher’s eval.yaml · each run twice on gemini-3.6-flash — with the skill injected, then without · last run 8d ago

Yes — +23% lift · at higher cost+214% tokens0% turns68% of whole cases pass with skill (15/22)

Test case	Without → With	Effect	Δ tokens	Δ turns
case-01	✗→✓	▲ Improved	114%	0%
case-09	✗→✓	▲ Improved	258%	0%
case-10	✗→✓	▲ Improved	347%	0%
case-11	✗→✓	▲ Improved	370%	0%
case-13	✗→✓	▲ Improved	279%	0%

Showing 5 of 22 cases

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v4 — 8/3/2026

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

microck / ordinary-claude-skills

skills_all/prometheus-configuration/SKILL.md

View on GitHub ↗

License NOASSERTIONImported 8/3/2026Version v1

Skill contents

SKILL.md

Prometheus Configuration

Complete guide to Prometheus setup, metric collection, scrape configuration, and recording rules.

Purpose

Configure Prometheus for comprehensive metric collection, alerting, and monitoring of infrastructure and applications.

When to Use

Set up Prometheus monitoring
Configure metric scraping
Create recording rules
Design alert rules
Implement service discovery

Prometheus Architecture

┌──────────────┐
│ Applications │ ← Instrumented with client libraries
└──────┬───────┘
       │ /metrics endpoint
       ↓
┌──────────────┐
│  Prometheus  │ ← Scrapes metrics periodically
│    Server    │
└──────┬───────┘
       │
       ├─→ AlertManager (alerts)
       ├─→ Grafana (visualization)
       └─→ Long-term storage (Thanos/Cortex)

Installation

Kubernetes with Helm

bash
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

helm install prometheus prometheus-community/kube-prometheus-stack \
  --namespace monitoring \
  --create-namespace \
  --set prometheus.prometheusSpec.retention=30d \
  --set prometheus.prometheusSpec.storageVolumeSize=50Gi

Docker Compose

yaml
version: '3.8'
services:
  prometheus:
    image: prom/prometheus:latest
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus-data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.path=/prometheus'
      - '--storage.tsdb.retention.time=30d'

volumes:
  prometheus-data:

Configuration File

prometheus.yml:

yaml
global:
  scrape_interval: 15s
  evaluation_interval: 15s
  external_labels:
    cluster: 'production'
    region: 'us-west-2'

# Alertmanager configuration
alerting:
  alertmanagers:
    - static_configs:
        - targets:
          - alertmanager:9093

# Load rules files
rule_files:
  - /etc/prometheus/rules/*.yml

# Scrape configurations
scrape_configs:
  # Prometheus itself
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']

  # Node exporters
  - job_name: 'node-exporter'
    static_configs:
      - targets:
        - 'node1:9100'
        - 'node2:9100'
        - 'node3:9100'
    relabel_configs:
      - source_labels: [__address__]
        target_label: instance
        regex: '([^:]+)(:[0-9]+)?'
        replacement: '${1}'

  # Kubernetes pods with annotations
  - job_name: 'kubernetes-pods'
    kubernetes_sd_configs:
      - role: pod
    relabel_configs:
      - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
        action: keep
        regex: true
      - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
        action: replace
        target_label: __metrics_path__
        regex: (.+)
      - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
        action: replace
        regex: ([^:]+)(?::\d+)?;(\d+)
        replacement: $1:$2
        target_label: __address__
      - source_labels: [__meta_kubernetes_namespace]
        action: replace
        target_label: namespace
      - source_labels: [__meta_kubernetes_pod_name]
        action: replace
        target_label: pod

  # Application metrics
  - job_name: 'my-app'
    static_configs:
      - targets:
        - 'app1.example.com:9090'
        - 'app2.example.com:9090'
    metrics_path: '/metrics'
    scheme: 'https'
    tls_config:
      ca_file: /etc/prometheus/ca.crt
      cert_file: /etc/prometheus/client.crt
      key_file: /etc/prometheus/client.key

Reference: See assets/prometheus.yml.template

Scrape Configurations

Static Targets

yaml
scrape_configs:
  - job_name: 'static-targets'
    static_configs:
      - targets: ['host1:9100', 'host2:9100']
        labels:
          env: 'production'
          region: 'us-west-2'

File-based Service Discovery

yaml
scrape_configs:
  - job_name: 'file-sd'
    file_sd_configs:
      - files:
        - /etc/prometheus/targets/*.json
        - /etc/prometheus/targets/*.yml
        refresh_interval: 5m

targets/production.json:

json
[
  {
    "targets": ["app1:9090", "app2:9090"],
    "labels": {
      "env": "production",
      "service": "api"
    }
  }
]

Kubernetes Service Discovery

yaml
scrape_configs:
  - job_name: 'kubernetes-services'
    kubernetes_sd_configs:
      - role: service
    relabel_configs:
      - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape]
        action: keep
        regex: true
      - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scheme]
        action: replace
        target_label: __scheme__
        regex: (https?)
      - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path]
        action: replace
        target_label: __metrics_path__
        regex: (.+)

Reference: See references/scrape-configs.md

Recording Rules

Create pre-computed metrics for frequently queried expressions:

yaml
# /etc/prometheus/rules/recording_rules.yml
groups:
  - name: api_metrics
    interval: 15s
    rules:
      # HTTP request rate per service
      - record: job:http_requests:rate5m
        expr: sum by (job) (rate(http_requests_total[5m]))

      # Error rate percentage
      - record: job:http_requests_errors:rate5m
        expr: sum by (job) (rate(http_requests_total{status=~"5.."}[5m]))

      - record: job:http_requests_error_rate:percentage
        expr: |
          (job:http_requests_errors:rate5m / job:http_requests:rate5m) * 100

      # P95 latency
      - record: job:http_request_duration:p95
        expr: |
          histogram_quantile(0.95,
            sum by (job, le) (rate(http_request_duration_seconds_bucket[5m]))
          )

  - name: resource_metrics
    interval: 30s
    rules:
      # CPU utilization percentage
      - record: instance:node_cpu:utilization
        expr: |
          100 - (avg by (instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)

      # Memory utilization percentage
      - record: instance:node_memory:utilization
        expr: |
          100 - ((node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes) * 100)

      # Disk usage percentage
      - record: instance:node_disk:utilization
        expr: |
          100 - ((node_filesystem_avail_bytes / node_filesystem_size_bytes) * 100)

Reference: See references/recording-rules.md

Alert Rules

yaml
# /etc/prometheus/rules/alert_rules.yml
groups:
  - name: availability
    interval: 30s
    rules:
      - alert: ServiceDown
        expr: up{job="my-app"} == 0
        for: 1m
        labels:
          severity: critical
        annotations:
          summary: "Service {{ $labels.instance }} is down"
          description: "{{ $labels.job }} has been down for more than 1 minute"

      - alert: HighErrorRate
        expr: job:http_requests_error_rate:percentage > 5
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "High error rate for {{ $labels.job }}"
          description: "Error rate is {{ $value }}% (threshold: 5%)"

      - alert: HighLatency
        expr: job:http_request_duration:p95 > 1
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "High latency for {{ $labels.job }}"
          description: "P95 latency is {{ $value }}s (threshold: 1s)"

  - name: resources
    interval: 1m
    rules:
      - alert: HighCPUUsage
        expr: instance:node_cpu:utilization > 80
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "High CPU usage on {{ $labels.instance }}"
          description: "CPU usage is {{ $value }}%"

      - alert: HighMemoryUsage
        expr: instance:node_memory:utilization > 85
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "High memory usage on {{ $labels.instance }}"
          description: "Memory usage is {{ $value }}%"

      - alert: DiskSpaceLow
        expr: instance:node_disk:utilization > 90
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "Low disk space on {{ $labels.instance }}"
          description: "Disk usage is {{ $value }}%"

Validation

bash
# Validate configuration
promtool check config prometheus.yml

# Validate rules
promtool check rules /etc/prometheus/rules/*.yml

# Test query
promtool query instant http://localhost:9090 'up'

Reference: See scripts/validate-prometheus.sh

Best Practices

Use consistent naming for metrics (prefix_name_unit)
Set appropriate scrape intervals (15-60s typical)
Use recording rules for expensive queries
Implement high availability (multiple Prometheus instances)
Configure retention based on storage capacity
Use relabeling for metric cleanup
Monitor Prometheus itself
Implement federation for large deployments
Use Thanos/Cortex for long-term storage
Document custom metrics

Troubleshooting

Check scrape targets:

bash
curl http://localhost:9090/api/v1/targets

Check configuration:

bash
curl http://localhost:9090/api/v1/status/config

Test query:

bash
curl 'http://localhost:9090/api/v1/query?query=up'

Reference Files

assets/prometheus.yml.template - Complete configuration template
references/scrape-configs.md - Scrape configuration patterns
references/recording-rules.md - Recording rule examples
scripts/validate-prometheus.sh - Validation script

Related Skills

grafana-dashboards - For visualization
slo-implementation - For SLO monitoring
distributed-tracing - For request tracing

Want to see if microck/prometheus-configuration would activate on your phrasing? Try it before you install — no LLM call, no signup.

Latest benchmark · publisher runVerified

8/21/2026, 15:34 UTC · 22 cases

Lift

+23%

68% vs 45%

Agent turns

1.0 vs 1.0

Tokens

+214%

3,836.73 vs 1,222.77

fail → pass · 5pass → pass · 10fail → fail · 7

Per-case results (22)

Case	Status	Duration (ms)			Turns			Tokens			Tool calls
Case	Status	Without	With	Δ	Without	With	Δ	Without	With	Δ	Without	With	Δ
case-01 I need to deploy Prometheus onto a Kubernetes cluster using Helm. A teammate suggested running 'helm install prometheus prometheus-community/prometheus'. Should I use that chart or a different package? Provide the helm command with standard parameters for retention and storage.	fail→pass	10,798	8,264	-23%	1	1	0%	2,048	4,387	+114%	0	0	—
case-02 I am configuring a Docker Compose service for Prometheus. A developer suggested setting retention via an environment variable like 'RETENTION_TIME=30d'. Show the CLI command flags under the Prometheus container definition to set retention and TSDB storage path.	pass→pass	8,496	6,173	-27%	1	1	0%	1,484	4,021	+171%	0	0	—
case-03 Write the 'global' section of a prometheus.yml file. A team member suggested setting the evaluation interval to 1 minute and scrape interval to 10 seconds. Show the baseline global configuration for intervals along with environment labels.	fail→fail	6,739	5,367	-20%	1	1	0%	1,201	3,759	+213%	0	0	—
case-04 When scraping node-exporter instances in Prometheus, the target address includes ports (e.g. 'node1:9100'). Write a relabel_configs block to strip the port from the target address so the 'instance' label contains only the hostname.	fail→fail	8,315	5,768	-31%	1	1	0%	1,492	3,903	+162%	0	0	—
case-05 Write a Prometheus scrape job relabel_configs snippet for Kubernetes pods that filters pods based on annotations. We only want to scrape pods that explicitly set the prometheus scrape annotation to true.	pass→pass	7,440	7,666	+3%	1	1	0%	1,353	4,255	+214%	0	0	—
case-06 We want to configure Prometheus file-based service discovery to read target JSON and YAML files under /etc/prometheus/targets/. A team member suggested polling file updates every 10 seconds. Write the scrape_configs block using file_sd_configs.	fail→fail	5,061	4,296	-15%	1	1	0%	1,028	3,670	+257%	0	0	—
case-07 Write a Prometheus recording rule named job:http_requests_error_rate:percentage that calculates the 5xx HTTP error percentage relative to overall HTTP request rate per job over 5-minute windows.	fail→fail	6,455	4,451	-31%	1	1	0%	1,203	3,645	+203%	0	0	—
case-08 Write a Prometheus recording rule named job:http_request_duration:p95 that calculates the 95th percentile HTTP request duration per job over 5-minute windows from bucket histogram metrics.	pass→pass	6,079	3,823	-37%	1	1	0%	999	3,577	+258%	0	0	—
case-09 Write a Prometheus recording rule named instance:node_cpu:utilization that calculates node CPU utilization percentage from the node_cpu_seconds_total idle mode metric.	fail→pass	5,147	4,932	-4%	1	1	0%	1,023	3,662	+258%	0	0	—
case-10 Write a Prometheus recording rule named instance:node_memory:utilization calculating node memory usage percentage from node_memory_MemAvailable_bytes and node_memory_MemTotal_bytes.	fail→pass	4,514	3,306	-27%	1	1	0%	773	3,457	+347%	0	0	—
case-11 Write a Prometheus alert rule named ServiceDown for condition up{job="my-app"} == 0. Specify how long the condition must persist before firing and what severity label to assign.	fail→pass	4,330	4,501	+4%	1	1	0%	752	3,536	+370%	0	0	—
case-12 Write a Prometheus alert rule named HighErrorRate based on job:http_requests_error_rate:percentage. What threshold percentage triggers the alert, and after what hold duration?	pass→pass	6,145	3,438	-44%	1	1	0%	1,036	3,311	+220%	0	0	—
case-13 Write a Prometheus alert rule named HighLatency using job:http_request_duration:p95. What latency threshold in seconds triggers this alert, and how long must it persist?	fail→pass	5,359	3,263	-39%	1	1	0%	878	3,325	+279%	0	0	—
case-14 Write a Prometheus alert rule named DiskSpaceLow using instance:node_disk:utilization. What percentage threshold triggers a critical severity alert?	pass→pass	8,184	4,011	-51%	1	1	0%	1,343	3,489	+160%	0	0	—
case-15 I edited prometheus.yml and added alert rules under /etc/prometheus/rules/. What command-line tool and arguments should I run to validate both the main configuration file and the rule files before reloading?	pass→pass	5,674	4,112	-28%	1	1	0%	1,000	3,500	+250%	0	0	—
case-16 How do I test an instant PromQL query for metric 'up' against a local Prometheus server at http://localhost:9090 from the command line interface?	pass→pass	7,922	4,578	-42%	1	1	0%	1,306	3,580	+174%	0	0	—
case-17 Prometheus is running on http://localhost:9090. What REST API endpoint path can I curl to inspect the status and health of all active scrape targets?	pass→pass	3,876	2,685	-31%	1	1	0%	624	3,201	+413%	0	0	—
case-18 Prometheus is running on http://localhost:9090. What REST API endpoint path provides the active Prometheus configuration currently running in memory?	pass→pass	3,424	3,104	-9%	1	1	0%	568	3,351	+490%	0	0	—
case-19 Write a Prometheus scrape job configuration named my-app that scrapes application endpoints over HTTPS using client certificate authentication.	pass→pass	5,806	4,900	-16%	1	1	0%	1,082	3,764	+248%	0	0	—
case-20 Write an alertmanager.yml configuration snippet defining PagerDuty and Slack notification receivers with routing keys and webhook URLs.	fail→fail	7,631	10,942	+43%	1	1	0%	1,464	4,964	+239%	0	0	—
case-21 Write a Grafana dashboard JSON panel object that visualizes Prometheus http_requests_total rates using a timeseries panel graph.	fail→fail	10,498	12,087	+15%	1	1	0%	2,054	4,891	+138%	0	0	—
case-22 Write an OpenTelemetry Collector config.yaml defining an OTLP gRPC receiver and a Kafka exporter pipeline.	fail→fail	11,730	12,239	+4%	1	1	0%	2,190	5,160	+136%	0	0	—

These cases were also screened by the skill’s intent / safety pass.See the Trust review ↗Reproduce this benchmark in Colab ↗

How this skill was measured

DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted. The headline lift of +23 percentage points is the difference between those two pass rates over the 22 comparable cases.

Model measured: gemini-3.6-flash
Graded by: gemini-3.6-flash
Grading method: judged
Verification: verified
Pass rate without the skill: 45.5%
Pass rate with the skill: 68.2%
Cases attempted: 22
Cases counted toward lift: 22
Cases that regressed: 0
Skill version measured: v1
Run completed: 8/21/2026, 15:34 UTC

Measured example — same prompt, with vs. without the skill · from the benchmark run · case case-01

Prompt

I need to deploy Prometheus onto a Kubernetes cluster using Helm. A teammate suggested running 'helm install prometheus prometheus-community/prometheus'. Should I use that chart or a different package? Provide the helm command with standard parameters for retention and storage.

Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.

Show both answers

SkillScore blends 4 signals1 of 4 complete

BenchmarkPass rate WITH skill on a verified run68

Live evalNeeds ≥20 evaluated traces in 30 days

AI ratingNeeds ≥10 rated traces

AdoptionNeeds ≥10 serves across ≥2 orgs in 30 days

Efficiency · separate axis (at equal correctness)+0% turns+234% tokens

Shown for transparency; not credited unless a verified run shows no correctness regression.

Backed by 1 of 4 signals — each added signal makes the score more robust.

Verified on gemini-3.6-flash · re-tested 7 days ago

Other measured skills in the registry, with their headline benchmark lift.