Skip to main content

Overview

HealOps can mask sensitive infrastructure identifiers (pod names, cluster names, hostnames, account IDs, service names, IP addresses, emails) before sending text to external LLMs, and restore the originals in any user-facing output (Slack report, problem MD, ingest). This lets teams use external models while keeping raw identifiers private to the investigation runtime. Masking is off by default. Enable it per investigation via environment variables — no code changes required.

How it works

  1. When masking is enabled, node_investigate replaces sensitive identifiers in the evidence dict with stable placeholders like <POD_0>, <NAMESPACE_0>, <CLUSTER_1>. The placeholder→original map is stored in the investigation state under masking_map.
  2. The diagnosis LLM receives masked evidence, so raw identifiers never hit the model.
  3. After the LLM returns its root-cause analysis, diagnose_root_cause unmasks the output so downstream state and display code see real identifiers.
  4. publish_findings runs a final unmask pass on the Slack message and blocks before delivery, as defence in depth.
The same identifier always maps to the same placeholder within a single investigation, so the LLM’s reasoning about <POD_0> remains coherent.

Environment variables

Policies are read fresh from the environment at the start of each investigation — changes take effect on the next run without a restart.

Built-in identifier kinds

Round-trip guarantee

For the built-in detectors and extra regex patterns, mask → unmask round-trips the original payload byte-for-byte. See tests/masking/test_integration_with_k8s_fixture.py for a worked example against a realistic Datadog k8s alert.

Relationship to guardrails

The masking layer is complementary to the one-way GuardrailEngine. Guardrails handle hard-block rules (credit cards, API keys) and replace matches with [REDACTED] irreversibly. Masking handles infrastructure identifiers reversibly so they can be restored for user-facing output. Both can be active together: guardrails apply first at the LLM client layer, then masking at the node layer.

Example

During the investigation the LLM sees masked evidence; the final Slack report shows the original pod, namespace, and cluster names.