> ## Documentation Index > Fetch the complete documentation index at: https://docs.healops.ai/llms.txt > Use this file to discover all available pages before exploring further. # Local Agent Mode > Run HealOps as a local monitoring service on your machine or in your infrastructure HealOps supports two integration paths. Choose based on whether the credentials live in the cloud or on the machine where your monitoring tools run. ## How the agent knows where to look This is the most important thing to understand before setting up the local agent. **HealOps does not scan arbitrary log files on disk.** Instead, it works in two modes depending on how alerts reach it: ### Mode 1 — Webhook-triggered investigation (recommended) Your monitoring tool (Datadog, Grafana, PagerDuty, Alertmanager) detects a problem and **pushes an alert** to the local agent via webhook. The alert payload tells HealOps *what* broke — which service, which host, which alert rule fired, and when. HealOps then uses your **configured integrations** to query the relevant data sources using those details: * Grafana Loki → fetch logs for the affected service around the alert time * Datadog → fetch metrics and APM traces for the affected service * Kubernetes → check pod status and recent events * Sentry → look for correlated error events * AWS CloudWatch → check infrastructure metrics The LLM reasons over that evidence to produce the RCA. **Your raw logs never leave your machine** — only the RCA summary is sent to the cloud. ``` Datadog detects high error rate on checkout-service │ │ POST /webhook/datadog {service: "checkout", alert: "error_rate > 5%", timestamp: ...} ▼ healops start (your machine) │ │ run_investigation() ─── asks Grafana: "show me logs for checkout-service last 30min" │ ─── asks Datadog: "show me metrics for checkout-service" │ ─── asks Kubernetes: "are checkout pods healthy?" ▼ AgentState { root_cause: "OOM in checkout-service pod due to memory leak in v2.3.1", ... } │ │ POST /api/investigations/ingest (HEALOPS_INGEST_TOKEN) ▼ app.healops.ai dashboard ``` **Setup:** configure `healops integrations setup ` for each tool, then run `healops start` and point your monitoring tool's webhook at `http://:8787/webhook/`. ### Mode 2 — Direct log file watching (`healops hermes watch`) For applications that write to a **local log file** (Python services, batch jobs, pipelines), HealOps can tail the file directly and detect incidents without needing a monitoring tool: ```bash theme={null} healops hermes watch --log-path /var/log/myapp/errors.log --investigate ``` HealOps reads the log file on your machine, classifies entries using an AI model, and when it detects a HIGH or CRITICAL incident it runs an investigation and can send alerts to Telegram. You tell HealOps exactly which file to watch via `--log-path`. It only reads that specific file. | | Mode 1: Webhook | Mode 2: Hermes log watch | | ------------ | ----------------------------------- | ------------------------------------- | | Alert source | Monitoring tool pushes | HealOps reads file directly | | Log access | Via Grafana/Datadog/etc. APIs | Direct file read (stays local) | | Setup | `healops start` + configure webhook | `healops hermes watch --log-path ...` | | Best for | Cloud-native, Kubernetes, SaaS | Batch jobs, pipelines, local apps | ## Two integration paths | Integration type | Where credentials live | How incidents reach the dashboard | | ------------------- | --------------------------------- | ---------------------------------------------------------------------------- | | **OAuth / API key** | HealOps cloud (encrypted per-org) | Cloud calls the API directly during RCA | | **CLI / local** | Your machine or infra | `healops start` receives webhooks, investigates locally, pushes RCA to cloud | **OAuth/API key integrations** (Slack, GitHub, Datadog webhook, PagerDuty) are connected via `app.healops.ai/integrations` — paste the credentials and they're stored encrypted in AWS Secrets Manager. **CLI-based integrations** (Grafana with local credentials, on-prem Kubernetes, private databases) require HealOps to run on a machine that can reach those services. The local agent model below covers this path. *** ## Local agent setup ### Step 1 — Install HealOps ```bash theme={null} brew tap healops-ai/tap brew install healops-ai/tap/healops ``` ```bash theme={null} curl -fsSL https://install.healops.ai | bash ``` ```bash theme={null} pip install healops-cli ``` ### Step 2 — Configure LLM + integrations ```bash theme={null} healops onboard ``` The onboarding wizard walks you through selecting an LLM provider (Anthropic, OpenAI, Ollama, etc.) and connecting observability integrations — Grafana, Datadog, Sentry, AWS, and more. You can also set up individual integrations at any time: ```bash theme={null} healops integrations setup grafana healops integrations setup datadog healops integrations verify grafana # confirm the connection works ```

Step 3 — Get your ingest token

The ingest token lets the local agent push investigation results to your `app.healops.ai` dashboard. 1. Sign in to [app.healops.ai](https://app.healops.ai) 2. Go to **Settings → API Keys** 3. Create a new key with the **Ingest** scope 4. Add it to your environment: ```bash theme={null} export HEALOPS_INGEST_TOKEN=ho_ingest_xxxx ``` Or add it to your `.env` file: ```bash theme={null} HEALOPS_INGEST_TOKEN=ho_ingest_xxxx HEALOPS_API_URL=https://api.healops.ai # default, can omit ``` ### Step 4 — Start the local agent ```bash theme={null} healops start ``` The agent starts a local HTTP server on `localhost:8787` and prints a summary: ``` ╭──────────────── healops start ────────────────╮ │ Webhook endpoint: http://127.0.0.1:8787/webhook │ Ingest token: configured │ Ingest URL: https://api.healops.ai │ │ Configure your monitoring tool to POST alerts │ to the webhook URL above. ╰────────────────────────────────────────────────╯ ``` **Options:** ```bash theme={null} healops start --port 9000 # custom port healops start --host 0.0.0.0 # accept external connections healops start --port 9000 --host 0.0.0.0 ``` Environment variable overrides: `HEALOPS_AGENT_PORT`, `HEALOPS_AGENT_HOST`. ### Step 5 — Point your monitoring tool at the agent Configure a webhook notification in your monitoring tool: | Tool | URL | | ------------ | ------------------------------------------------- | | Datadog | `http://:8787/webhook/datadog` | | Grafana | `http://:8787/webhook/grafana` | | PagerDuty | `http://:8787/webhook/pagerduty` | | Alertmanager | `http://:8787/webhook/alertmanager` | | Generic | `http://:8787/webhook` | When an alert fires, HealOps: 1. Receives the webhook payload 2. Runs a full RCA investigation locally using your configured integrations 3. Pushes the result to your `app.healops.ai` dashboard via `HEALOPS_INGEST_TOKEN` *** ## How it works ``` Datadog / Grafana / PagerDuty │ webhook POST ▼ healops start (your machine / infra) │ run_investigation() — uses local credentials ▼ AgentState { root_cause, evidence, recommendations } │ POST /api/investigations/ingest ▼ app.healops.ai dashboard ``` The investigation runs **entirely on your machine** using your local integration credentials. Only the final RCA result (root cause, recommendations, evidence summary) is sent to the cloud — your raw metrics, logs, and trace data never leave your infrastructure. *** ## Run as a background service ### systemd (Linux) ```ini theme={null} # /etc/systemd/system/healops.service [Unit] Description=HealOps Local Agent After=network.target [Service] Type=simple User=your-user EnvironmentFile=/etc/healops/healops.env ExecStart=/usr/local/bin/healops start --host 0.0.0.0 --port 8787 Restart=on-failure RestartSec=5 [Install] WantedBy=multi-user.target ``` ```bash theme={null} sudo systemctl enable --now healops ``` ### launchd (macOS) ```xml theme={null} Labelai.healops.agent ProgramArguments /usr/local/bin/healops start EnvironmentVariables HEALOPS_INGEST_TOKENho_ingest_xxxx RunAtLoad KeepAlive ``` ```bash theme={null} launchctl load ~/Library/LaunchAgents/ai.healops.agent.plist ``` ### Docker ```dockerfile theme={null} FROM python:3.12-slim RUN pip install healops-cli ENV HEALOPS_INGEST_TOKEN="" ENV HEALOPS_AGENT_HOST="0.0.0.0" ENV HEALOPS_AGENT_PORT="8787" EXPOSE 8787 CMD ["healops", "start"] ``` *** ## Troubleshooting **`healops start` exits immediately with "uvicorn is required"** Install the standard extras: ```bash theme={null} pip install 'healops-cli[standard]' ``` **Alerts arrive but no investigation runs** * Check that `LLM_PROVIDER` and the matching API key are set: `healops health` * Verify integrations are configured: `healops integrations verify ` * Run a test investigation manually: `healops investigate -i tests/e2e/kubernetes/fixtures/datadog_k8s_alert.json` **Results don't appear in the dashboard** * Confirm `HEALOPS_INGEST_TOKEN` is set and valid * Check the agent output for `↑ pushed to dashboard` or an error message * Test with `curl -X POST http://localhost:8787/webhook -H 'Content-Type: application/json' -d '{"title":"test alert"}'`