Logging: Comparison of Frameworks

Logging is an often overlooked part of software development, and something I have been neglecting only up till recently. In this post, I compare several logging frameworks.

What is a Logging Framework?

A logging framework (or stack) is a collection of software components which aggregate, analyze and display information collected from sources.

In simple terms, the stages are as follows:

graph TD
  Sources --> Aggregators
  Aggregators --> Analyzers
  Analyzers --> Visualizers

Note: Analyzers may also store the raw log data.

For each stage, there are several choices of software you can choose from. We can add this to the diagram:

graph TD

  subgraph Sources
    Syslog
    Systemd-journald
    Kubernetes
    Docker
    Kafka[Apache Kafka]
    Apps[Application Logs]
  end

  Sources --> Aggregators

  subgraph Aggregators
    Promtail
    FluentD
    FluentBit
        subgraph ES["Elastic Stack (ES)"]
        direction TB
        Filebeat --> Logstash
        end
    Vector
  end

  Aggregators --> Analyzers

  subgraph Analyzers
    Loki
    Elasticsearch
  end

  Analyzers --> Visualizers

  subgraph Visualizers
    Grafana
    Kibana["Kibana (ES)"]
  end

The 3 most popular logging stacks (combinations) are:

PLG:

• Promtail
• Loki
• Grafana

EFK/ELK

• FluentD/Logstash
• Elasticsearch
• Kibana

Aggregator Comparison TODO

Analyzer Comparison TODO

Visualizer Comparison TODO

Comments