Local Agent Mode

HealOps supports two integration paths. Choose based on whether the credentials live in the cloud or on the machine where your monitoring tools run.

How the agent knows where to look

This is the most important thing to understand before setting up the local agent. HealOps does not scan arbitrary log files on disk. Instead, it works in two modes depending on how alerts reach it:

Mode 1 — Webhook-triggered investigation (recommended)

Your monitoring tool (Datadog, Grafana, PagerDuty, Alertmanager) detects a problem and pushes an alert to the local agent via webhook. The alert payload tells HealOps what broke — which service, which host, which alert rule fired, and when. HealOps then uses your configured integrations to query the relevant data sources using those details:

Grafana Loki → fetch logs for the affected service around the alert time
Datadog → fetch metrics and APM traces for the affected service
Kubernetes → check pod status and recent events
Sentry → look for correlated error events
AWS CloudWatch → check infrastructure metrics

The LLM reasons over that evidence to produce the RCA. Your raw logs never leave your machine — only the RCA summary is sent to the cloud.

Datadog detects high error rate on checkout-service
    │
    │  POST /webhook/datadog  {service: "checkout", alert: "error_rate > 5%", timestamp: ...}
    ▼
healops start (your machine)
    │
    │  run_investigation() ─── asks Grafana: "show me logs for checkout-service last 30min"
    │                     ─── asks Datadog: "show me metrics for checkout-service"
    │                     ─── asks Kubernetes: "are checkout pods healthy?"
    ▼
AgentState { root_cause: "OOM in checkout-service pod due to memory leak in v2.3.1", ... }
    │
    │  POST /api/investigations/ingest (HEALOPS_INGEST_TOKEN)
    ▼
app.healops.ai dashboard

Setup: configure healops integrations setup <service> for each tool, then run healops start and point your monitoring tool’s webhook at http://<your-machine>:8787/webhook/<source>.

Mode 2 — Direct log file watching (`healops hermes watch`)

For applications that write to a local log file (Python services, batch jobs, pipelines), HealOps can tail the file directly and detect incidents without needing a monitoring tool:

healops hermes watch --log-path /var/log/myapp/errors.log --investigate

HealOps reads the log file on your machine, classifies entries using an AI model, and when it detects a HIGH or CRITICAL incident it runs an investigation and can send alerts to Telegram. You tell HealOps exactly which file to watch via --log-path. It only reads that specific file.

	Mode 1: Webhook	Mode 2: Hermes log watch
Alert source	Monitoring tool pushes	HealOps reads file directly
Log access	Via Grafana/Datadog/etc. APIs	Direct file read (stays local)
Setup	`healops start` + configure webhook	`healops hermes watch --log-path ...`
Best for	Cloud-native, Kubernetes, SaaS	Batch jobs, pipelines, local apps

Two integration paths

Integration type	Where credentials live	How incidents reach the dashboard
OAuth / API key	HealOps cloud (encrypted per-org)	Cloud calls the API directly during RCA
CLI / local	Your machine or infra	`healops start` receives webhooks, investigates locally, pushes RCA to cloud

OAuth/API key integrations (Slack, GitHub, Datadog webhook, PagerDuty) are connected via app.healops.ai/integrations — paste the credentials and they’re stored encrypted in AWS Secrets Manager. CLI-based integrations (Grafana with local credentials, on-prem Kubernetes, private databases) require HealOps to run on a machine that can reach those services. The local agent model below covers this path.

Local agent setup

Step 1 — Install HealOps

Homebrew
curl
pip

brew tap healops-ai/tap
brew install healops-ai/tap/healops

curl -fsSL https://install.healops.ai | bash

pip install healops-cli

Step 2 — Configure LLM + integrations

healops onboard

The onboarding wizard walks you through selecting an LLM provider (Anthropic, OpenAI, Ollama, etc.) and connecting observability integrations — Grafana, Datadog, Sentry, AWS, and more. You can also set up individual integrations at any time:

healops integrations setup grafana
healops integrations setup datadog
healops integrations verify grafana    # confirm the connection works

Step 3 — Get your ingest token

The ingest token lets the local agent push investigation results to your app.healops.ai dashboard.

Sign in to app.healops.ai
Go to Settings → API Keys
Create a new key with the Ingest scope
Add it to your environment:

export HEALOPS_INGEST_TOKEN=ho_ingest_xxxx

Or add it to your .env file:

HEALOPS_INGEST_TOKEN=ho_ingest_xxxx
HEALOPS_API_URL=https://api.healops.ai   # default, can omit

Step 4 — Start the local agent

healops start

The agent starts a local HTTP server on localhost:8787 and prints a summary:

╭──────────────── healops start ────────────────╮
│  Webhook endpoint:  http://127.0.0.1:8787/webhook
│  Ingest token:      configured
│  Ingest URL:        https://api.healops.ai
│
│  Configure your monitoring tool to POST alerts
│  to the webhook URL above.
╰────────────────────────────────────────────────╯

Options:

healops start --port 9000               # custom port
healops start --host 0.0.0.0            # accept external connections
healops start --port 9000 --host 0.0.0.0

Environment variable overrides: HEALOPS_AGENT_PORT, HEALOPS_AGENT_HOST.

Step 5 — Point your monitoring tool at the agent

Configure a webhook notification in your monitoring tool:

Tool	URL
Datadog	`http://<your-machine>:8787/webhook/datadog`
Grafana	`http://<your-machine>:8787/webhook/grafana`
PagerDuty	`http://<your-machine>:8787/webhook/pagerduty`
Alertmanager	`http://<your-machine>:8787/webhook/alertmanager`
Generic	`http://<your-machine>:8787/webhook`

When an alert fires, HealOps:

Receives the webhook payload
Runs a full RCA investigation locally using your configured integrations
Pushes the result to your app.healops.ai dashboard via HEALOPS_INGEST_TOKEN

How it works

Datadog / Grafana / PagerDuty
    │  webhook POST
    ▼
healops start (your machine / infra)
    │  run_investigation() — uses local credentials
    ▼
AgentState { root_cause, evidence, recommendations }
    │  POST /api/investigations/ingest
    ▼
app.healops.ai dashboard

The investigation runs entirely on your machine using your local integration credentials. Only the final RCA result (root cause, recommendations, evidence summary) is sent to the cloud — your raw metrics, logs, and trace data never leave your infrastructure.

Run as a background service

systemd (Linux)

# /etc/systemd/system/healops.service
[Unit]
Description=HealOps Local Agent
After=network.target

[Service]
Type=simple
User=your-user
EnvironmentFile=/etc/healops/healops.env
ExecStart=/usr/local/bin/healops start --host 0.0.0.0 --port 8787
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

sudo systemctl enable --now healops

launchd (macOS)

<!-- ~/Library/LaunchAgents/ai.healops.agent.plist -->
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key><string>ai.healops.agent</string>
  <key>ProgramArguments</key>
  <array>
    <string>/usr/local/bin/healops</string>
    <string>start</string>
  </array>
  <key>EnvironmentVariables</key>
  <dict>
    <key>HEALOPS_INGEST_TOKEN</key><string>ho_ingest_xxxx</string>
  </dict>
  <key>RunAtLoad</key><true/>
  <key>KeepAlive</key><true/>
</dict>
</plist>

launchctl load ~/Library/LaunchAgents/ai.healops.agent.plist

Docker

FROM python:3.12-slim
RUN pip install healops-cli
ENV HEALOPS_INGEST_TOKEN=""
ENV HEALOPS_AGENT_HOST="0.0.0.0"
ENV HEALOPS_AGENT_PORT="8787"
EXPOSE 8787
CMD ["healops", "start"]

Troubleshooting

healops start exits immediately with “uvicorn is required” Install the standard extras:

pip install 'healops-cli[standard]'

Alerts arrive but no investigation runs

Check that LLM_PROVIDER and the matching API key are set: healops health
Verify integrations are configured: healops integrations verify <service>
Run a test investigation manually: healops investigate -i tests/e2e/kubernetes/fixtures/datadog_k8s_alert.json

Results don’t appear in the dashboard

Confirm HEALOPS_INGEST_TOKEN is set and valid
Check the agent output for ↑ pushed to dashboard or an error message
Test with curl -X POST http://localhost:8787/webhook -H 'Content-Type: application/json' -d '{"title":"test alert"}'

Overview

Local

Enterprise

How the agent knows where to look

Mode 1 — Webhook-triggered investigation (recommended)

Mode 2 — Direct log file watching (`healops hermes watch`)

Two integration paths

Local agent setup

Step 1 — Install HealOps

Step 2 — Configure LLM + integrations

Step 3 — Get your ingest token

Step 4 — Start the local agent

Step 5 — Point your monitoring tool at the agent

How it works

Run as a background service

systemd (Linux)

launchd (macOS)

Docker

Troubleshooting

​How the agent knows where to look

​Mode 1 — Webhook-triggered investigation (recommended)

​Mode 2 — Direct log file watching (healops hermes watch)

​Two integration paths

​Local agent setup

​Step 1 — Install HealOps

​Step 2 — Configure LLM + integrations

​Step 3 — Get your ingest token

​Step 4 — Start the local agent

​Step 5 — Point your monitoring tool at the agent

​How it works

​Run as a background service

​systemd (Linux)

​launchd (macOS)

​Docker

​Troubleshooting

How the agent knows where to look

Mode 1 — Webhook-triggered investigation (recommended)

Mode 2 — Direct log file watching (`healops hermes watch`)

Two integration paths

Local agent setup

Step 1 — Install HealOps

Step 2 — Configure LLM + integrations

Step 3 — Get your ingest token

Step 4 — Start the local agent

Step 5 — Point your monitoring tool at the agent

How it works

Run as a background service

systemd (Linux)

launchd (macOS)

Docker

Troubleshooting