Skip to main content

Forward decision logs with the logs forwarder

Forward the decision logs of a policy decision point (PDP) to Elasticsearch, or print them to the stdout of a Fluent Bit container, with the Permit logs forwarder. This page is for operators who deploy the PDP to Kubernetes with the Permit Helm chart and want decision logs in their own logging platform.

The logs forwarder is part of the PDP Helm chart. For a PDP that you run with docker run, print decisions to the container output instead. See Decision logs in the PDP configuration.

Prerequisites​

  • A PDP deployed with the Permit Helm chart. See Deploy the PDP with Helm.
  • For Elasticsearch: the host, port, index, and a user name and password that can write to the index.

How the logs forwarder works​

When pdp.logs_forwarder.enabled is true, the Helm chart makes these changes to the PDP pod:

  1. The PDP container writes its logs to the /tmp/pdp.log file on a volume that both containers share.
  2. The chart adds a sidecar container that runs Fluent Bit, an open source log processor and forwarder.
  3. Fluent Bit reads /tmp/pdp.log, keeps only the decision log entries, and sends each decision as JSON to the output you choose: stdout or elasticsearch.

The PDP rotates /tmp/pdp.log by size and deletes rotated files after a retention period. The rotation and retention values control both.

Turn on the logs forwarder​

Set these values in your values file for the PDP chart, or pass them with --set. For all chart values, see the chart's values.yaml.

ValueDefaultDescription
pdp.logs_forwarder.enabledfalseSet to true to add the Fluent Bit sidecar.
pdp.logs_forwarder.typestdoutThe output: stdout or elasticsearch.
pdp.logs_forwarder.elasticsearch.hostNoneThe Elasticsearch host. Required when type is elasticsearch.
pdp.logs_forwarder.elasticsearch.port443The Elasticsearch port. Fluent Bit connects over TLS.
pdp.logs_forwarder.elasticsearch.indexNoneThe Elasticsearch index to write decision logs to.
pdp.logs_forwarder.elasticsearch.cloud_authNoneThe Elasticsearch credentials in the {user}:{password} form.
pdp.logs_forwarder.rotation250 MBThe file size at which the PDP rotates /tmp/pdp.log.
pdp.logs_forwarder.retention3 daysHow long the PDP keeps rotated log files.
pdp.logs_forwarder.debug_modefalseSet to true to print Fluent Bit's own debug logs.

To send decision logs to Elasticsearch:

  1. Set pdp.logs_forwarder.enabled to true.
  2. Set pdp.logs_forwarder.type to elasticsearch.
  3. Set host, port, index, and cloud_auth under pdp.logs_forwarder.elasticsearch.
  4. Run helm upgrade for the PDP release with the new values.

To print decision logs to stdout, set pdp.logs_forwarder.enabled to true and keep type as stdout, then run helm upgrade.

Protect the Elasticsearch credentials

cloud_auth holds the Elasticsearch password in plain text. Anyone who can run helm get values on the release can read it. Keep the values file out of source control.

Confirm that decision logs are forwarded​

  1. Run a permit.check() call against the PDP.
  2. Check the output you chose:
    • For stdout: read the logs of the fluentbit container in the PDP pod, for example with kubectl logs <pdp-pod-name> -c fluentbit. The output has a JSON record with a decision_id for the check.
    • For elasticsearch: search the index you set. The index has a document with a decision_id for the check.

If no decision appears, set pdp.logs_forwarder.debug_mode to true, upgrade the release, and read the fluentbit container logs for connection errors.

Next steps​

Contents

Edit this page

Need help?