# Prometheus Metric Collection Strategies: Comparing Pushgateway, Node Exporter, and cAdvisor

> Compare Prometheus metric collection strategies. Learn how Pushgateway, Node Exporter, and cAdvisor gather metrics for different applications, enabling effective monitoring.

- Repository: [Arie Bregman/devops-exercises](https://github.com/bregman-arie/devops-exercises)
- Tags: deep-dive
- Published: 2026-02-28

---

**Prometheus Pushgateway uses a push-based model for short-lived batch jobs, while Node Exporter and cAdvisor employ pull-based scraping for host-level and container-level metrics respectively.**

In modern cloud-native observability stacks, selecting the appropriate metric collection strategy is critical for comprehensive system visibility. The **bregman-arie/devops-exercises** repository highlights how these three Prometheus ecosystem components serve fundamentally different monitoring requirements across diverse infrastructure layers. This article examines the architectural patterns, deployment configurations, and optimal use cases for each collector based on their underlying implementations.

## Core Collection Models

The fundamental distinction between these components lies in their data transmission direction, lifecycle management, and target resource types.

### Prometheus Pushgateway: Push-Based Intermediary

**Pushgateway** acts as a bridge for **ephemeral or batch jobs** that cannot wait for Prometheus to scrape them. Instead of exposing an endpoint for polling, applications actively push metrics via HTTP POST requests to the gateway.

Jobs send metrics to the gateway's `/metrics/job/<job_name>` endpoint using the Prometheus exposition format. The gateway stores only the latest value for each metric series until overwritten or cleared. Prometheus then scrapes the gateway's `/metrics` endpoint on its standard interval (default 15 seconds), treating the gateway as a persistent intermediary.

This pattern is essential for CI pipelines, backup scripts, and cron jobs that terminate before a scrape cycle completes. However, the gateway retains only the most recent push; it is not a time-series database and does not aggregate historical data between pushes.

### Node Exporter: Native Pull-Based Host Monitoring

**Node Exporter** represents the classic **pull-based Prometheus model**, running as a stateless daemon on every target node. It reads kernel statistics from the host's `/proc` and `/sys` filesystems on each scrape request, exposing pre-defined metrics like `node_cpu_seconds_total` and `node_memory_MemAvailable_bytes`.

Deployed as a binary or Docker container (`prom/node-exporter`), it exposes data on port 9100 without requiring write access from external systems. Prometheus initiates all connections, making firewall rules and security boundaries straightforward to manage. Each scrape yields a fresh snapshot of host resources.

### cAdvisor: Container-Aware Pull Integration

**cAdvisor** (Container Advisor) provides **granular container-level resource metrics** within Kubernetes environments. Unlike standalone exporters, cAdvisor runs embedded within the **kubelet** on each node, continuously monitoring cgroups and kernel metrics for all containers on that host.

As noted in [[`topics/kubernetes/README.md`](https://github.com/bregman-arie/devops-exercises/blob/main/topics/kubernetes/README.md)](https://github.com/bregman-arie/devops-exercises/blob/master/topics/kubernetes/README.md) within the bregman-arie/devops-exercises repository, the **metrics-server** leverages the **cAdvisor** component of the kubelet to collect node-level container statistics. Prometheus scrapes the kubelet's read-only port (typically 10255) or secure port to retrieve per-container CPU, memory, filesystem, and network usage metrics like `container_cpu_user_seconds_total`.

## Deployment and Configuration Patterns

Each component requires distinct deployment strategies aligned with its collection model and target environment.

### Configuring Pushgateway for Batch Workloads

Run the gateway as a persistent HTTP service:

```bash
docker run -d -p 9091:9091 prom/pushgateway

```

Client jobs push metrics via HTTP POST before terminating:

```bash
cat <<EOF | curl --data-binary @- http://localhost:9091/metrics/job/backup_job/instance/batch-01

# TYPE job_duration_seconds gauge

job_duration_seconds 45.2

# TYPE job_records_processed counter

job_records_processed 892
EOF

```

Configure Prometheus to scrape the gateway:

```yaml
scrape_configs:
  - job_name: 'pushgateway'
    static_configs:
      - targets: ['pushgateway-host:9091']

```

### Deploying Node Exporter on Linux Hosts

Deploy with full host namespace access to read system metrics:

```bash
docker run -d --net="host" --pid="host" \
  -v "/:/host:ro,rslave" \
  -v "/sys:/sys:ro" \
  -v "/proc:/proc:ro" \
  prom/node-exporter \
  --path.rootfs=/host

```

Prometheus configuration targets each node's exporter individually:

```yaml
scrape_configs:
  - job_name: 'node_exporter'
    static_configs:
      - targets: ['node1:9100', 'node2:9100', 'node3:9100']

```

### Scraping cAdvisor Metrics in Kubernetes

cAdvisor requires no separate binary installation. Enable scraping through the kubelet by configuring Prometheus with Kubernetes service discovery:

```yaml
scrape_configs:
  - job_name: 'kubelet-cadvisor'
    scheme: https
    tls_config:
      insecure_skip_verify: true
    kubernetes_sd_configs:
      - role: node
    relabel_configs:
      - source_labels: [__address__]
        regex: (.*):10250
        replacement: $1:10255
        target_label: __address__

```

For RBAC-enabled clusters, ensure the Prometheus service account has permission to access node metrics:

```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: prometheus-cadvisor
rules:
- apiGroups: [""]
  resources: ["nodes/metrics"]
  verbs: ["get"]

```

## Strategic Selection Criteria

Choose your collector based on workload characteristics and infrastructure layer:

- **Pushgateway**: Select when monitoring **short-lived processes** that exit before scrape intervals complete, or when aggregating results from serverless functions that cannot expose persistent endpoints.
- **Node Exporter**: Deploy for **infrastructure health monitoring**, capacity planning, and host-level resource exhaustion alerts across bare-metal or virtual machine fleets.
- **cAdvisor**: Implement for **Kubernetes-native observability**, per-container autoscaling metrics, and pod resource quota enforcement within container orchestration platforms.

## Summary

- **Pushgateway** bridges push-based batch jobs to Prometheus's pull model, storing only the latest metric values until overwritten by subsequent pushes.
- **Node Exporter** provides stateless, pull-based access to host kernel statistics via `/proc` and `/sys` filesystems, exposing data at `:9100/metrics`.
- **cAdvisor** exposes container-level metrics through the Kubernetes kubelet's embedded instance, as referenced in the bregman-arie/devops-exercises repository's Kubernetes documentation.
- Pushgateway requires write access from client jobs and maintains state between scrapes, while Node Exporter and cAdvisor expose read-only, stateless endpoints scraped by Prometheus.
- Node Exporter runs as a standalone binary per node, whereas cAdvisor is embedded within the kubelet without requiring separate deployment.

## Frequently Asked Questions

### When should I use Pushgateway instead of a direct exporter?

Use Pushgateway for **short-lived batch processes** that terminate before Prometheus can scrape them, such as CI/CD pipelines or nightly backup scripts. Direct exporters suit long-running services with persistent HTTP endpoints. Avoid Pushgateway for regular service monitoring, as it creates single points of failure and complicates metric lifecycle management when jobs push conflicting labels.

### Does cAdvisor run as a separate container in Kubernetes?

No, cAdvisor runs **embedded within the kubelet** on each Kubernetes node. It is not deployed as a standalone container or DaemonSet. Prometheus scrapes container metrics through the kubelet's API endpoints, typically port 10255 (read-only) or the secure kubelet port with authentication, as implemented in the metrics-server architecture described in the bregman-arie/devops-exercises repository.

### Can Node Exporter monitor container resource usage?

No, Node Exporter exposes **host-level metrics** only, reading from the node's `/proc` and `/sys` filesystems. It cannot see per-container cgroup statistics or individual pod resource consumption. For container metrics, use cAdvisor via the kubelet, or query the kubelet's native resource metrics endpoint.

### How does metric retention differ between these components?

Pushgateway retains **only the latest pushed value** for each metric series, making it unsuitable for historical trend analysis. Node Exporter and cAdvisor are **stateless collectors** that generate fresh snapshots on each scrape request; Prometheus itself stores the time-series history in its TSDB. Never treat Pushgateway as a long-term storage solution or metric aggregation buffer.