User Guide

Rainbow Gum: JDK 21+ SLF4J logging implementation

Fast, modular, GraalVM native friendly and easy to use.

Getting Started

First step is to add Rainbow Gum as a dependency.

Second if you are not familiar with logging in the Java ecosystem please refer to the excellent Logging facade SLF4J documentation particularly the logger API and what logging Levels are. Logging levels are and how dotted logger names inherit levels is covered in the JDKs logging facade System.Logger.Level as well as Logback effective level which Rainbow Gum follows.

The third step is to configure. Most configuration is just configuring logger names to levels, output, and formatting. We will cover the two major ways to do this in the following two sub sections.

Using System properties

Assuming we have installed RainbowGum as a dependency a simple configuration example using System Properties is below:

  java \
  -Dlogging.appender.console.encoder=pattern \
  -Dlogging.encoder.console.pattern="[%thread] %-5level %logger{15} - %msg%n" \
  -Dlogging.level.com.myapp=DEBUG \
  myapp.jar

What happens with the above is Rainbow Gum will load its default Rainbow Gum via the service loader and use the properties to configure its builders. Unfortunately System properties on the command line can be rather tedious but luckily you can plugin in your own properties system easily.

Using RainbowGum Builders

For organizations that have many projects it is recommended to use the programmatic builder approach and make a standardized jar (module) shared between all your projects so that logging configuration is consistent. Assuming we have installed RainbowGum as a dependency a simple configuration example using Java directly and the ServiceLoader is below.

public class GettingStartedExample implements RainbowGumProvider {

	@Override
	public Optional<RainbowGum> provide(LogConfig config) {

		return RainbowGum.builder(config) //
			.route(r -> {
				r.level(Level.DEBUG, "com.myapp");
				r.appender("console", a -> {
					a.encoder(new PatternEncoderBuilder("console")
						// We use the pattern encoder which follows logback pattern
						// syntax.
						.pattern("[%thread] %-5level %logger{15} - %msg%n")
						// We use properties to override the above pattern if set.
						.fromProperties(config.properties())
						.build());
				});
			}) //
			.optional();
	}

}

RainbowGumServiceProvider has additional documentation on how to register a service loader aware jar.

Description

Rainbow Gum is a JDK 21+ opinionated SLF4J implementation that aims to be easier to use while leveraging newer JDK technology. Rainbow Gum unlike Logback or Log4J (2) does not offer as much flexibility but is simpler and has less overhead. The readme in the Rainbow Gum project discusses more extensively on the opinionated design and philosophy.

Project Information

Source Control

https://github.com/jstachio/rainbowgum

Team

Adam Gent (agentgt) - lead

Issues

https://github.com/jstachio/rainbowgum/issues

Community

https://github.com/jstachio/rainbowgum/discussions

User Guide

This document

Javadoc

This document (modules listing at bottom)

The project follows semantic versioning.

Requirements

Java 21 or greater
A build system that supports running the Java compiler annotation processor

The only module needed during runtime is java.base

Limitations

Currently Rainbow Gum does not provide support for:

External config - by design you will need to bring your own configuration lib or just use env/system properties.
Rolling of log files. Will be added in the future.
SLF4J Marker support - libraries and application rarely use it compared to MDC.

If any of these limitations (or others not listed) are a show stopper please let us know by filing an issue. Particularly Markers. If you are using Markers we would like to hear from you.

How it works

Most users will use Rainbow Gum through a logging facade such as SLF4J and the only interaction with Rainbow Gum is configuration of output and formatting. Rainbow Gum provides three ways to configure out of the box:

Programmatic Java configuration using builders.
Simple String key/value properties often derived from System properties / env variables
Dependency driven configuration where including a jar as dependency changes behavior automatically

In practice many will use a mixture of all three styles. Rainbow Gum uses the Service Loader to load "default" configurations. Other than logger levels most configuration is simply choosing which jars to be included at runtime and providing some simple properties.

Architecture

Rainbow Gum's design has many superficial similarities to Logback, Log4j2, and Reload4j in that there are Appenders and Encoders as well as how log levels are inherited but most of Rainbow Gum's design is very different!

The key difference in Rainbow Gum is that once configured and initialized the logging system is locked in and cannot be changed. Certain parts of the system can be changed at runtime but require opt-in which includes changing levels of logger names.

Another major difference is that Rainbow Gum is highly modularized, immutable and componentized. In other frameworks OOP inheritance is heavily abused and riddled with state management. Furthermore there are components that have too much responsibility which is the case with Appenders in Logback or "Managers" in Log4j2. Furthermore Rainbow Gum uses zero reflection other than the Service Loader and follows the builder pattern extensively to separate configuration from immutable runtime components. Almost all components have a builder to programmatically configure and the builders can take flat string key values as configuration. The other logging frameworks use an enormous amount of reflection which slows initialization time. (While the ServiceLoader is technically reflection it is GraalVM native friendly and is the preferred way for pluggable components in modern JDKs.)

The results of the above choices make Rainbow Gum far lighter than logback, log4j2, and reload4j particularly in initialization time as well as security surface area.

The key components expressed in the flow of log events is as follows:

In the next sections we will cover Config, Publishers, Appenders, Encoders, and Outputs. For most level resolving, formatters (which are simplified encoders), and Outputs are the main points of interest in customizing and configuring.

Config

A defining characteristic of Rainbow Gum is that it does not have a special configuration format like Logback, Log4j2, and Reload4j (all three use XML as the default). The expectation is that for simple configuration simple properties interface analogous to Function<String,String> is good enough. For more complicated configuration programmatic configuration using the builders and service loader should be used.

For implementers of plugins LogConfig allows registering of services and plugins (it also contains the global LogProperties ) however most users will not need to know about it. For those coming from Logback LogConfig is analogous to what Logback calls "Context".

Rainbow Gum out of the box uses System properties for LogProperties but an optional module io.jstach.rainbowgum.avaje will allow using Avaje Config as the properties provider. Other configuration systems will be added in the future but an important requirement is that these systems do not do any logging or if they do allow it to be turned off, intercepted or blocked. IMPORTANT: Rainbow Gum core does not do any interpolation of property values! That is the responsibility of the LogProperties backing implementation and the values retrieved should already be interpolated.

NOTE: Throughout the documentation Properties and URI syntax is used to express configurable properties but it is a not a requirement and your configuration system may have a different format like YAML or JSON.

A listing of property patterns that are used out of the box is discussed in LogProperties. A general pattern is:

logging.{componentType}.{name}.{subComponent}=URI
logging.{subComponent}.{name}.{propertyName}=someStringThatIsConverted

An interpolated key (note key not value) example might be:

logging.appender.example.encoder=gelf
logging.encoder.example.prettyPrint=true

The astute might notice that "gelf" does not look like a URI. It is but URIs missing schemes like the above get normalized so that the path is used as the scheme. Thus gelf is actually gelf:/// Because the first property is a URI one can do:

logging.appender.example.encoder=gelf:///?prettyPrint=true

URIs play an important part for component provision as the URI scheme is used for plugin lookup as well as single line configuration. This follows 12 factor recommendation of resources and configuration.

Config Change during runtime

While most of the logging components in Rainbow Gum are immutable and cannot be changed during runtime there are some exceptions including custom components.

A supporting configuration system (a LogProperties provider) can notify that there has been a change by using the Change Publisher. This is how level resolving changes can be published.

Config change is by default off even on supporting configuration systems and can only be enabled by setting "logging.global.change" to true.

Router

A Router has a LevelResolver and a Publisher. A Level Resolver takes a logger name and produces a level. The logging facade implementation than decides based on the level whether or not to construct an event. It then passes the event to the router which in turn uses the publisher to schedule the event. For those familiar with other logging frameworks Rainbow Gum has no actual concept of named "Loggers" but a route from a router is the closest analog.

When configuring routers you are configuring Level Resolvers which are logger names to levels, which publisher to use and which appenders should be associated with the publisher. We call this configuration a "route". As discussed later on a publisher has appenders and appenders have outputs thus an important consequnce of this is if you want outputs to have different level thresholds (e.g. a debug.log and an error.log) you will need multiple routers.

Example configuring router with properties using "logging.route.{name}." with name set to "example".

logging.routes=example
logging.route.example.appenders=appender1,appender2
logging.route.example.publisher=async
logging.route.example.level.com.mycompany=DEBUG

If "logging.routes" is not set the route name is assumed to be "default". Notice that routes can contain their own level configuration and thus a route is a way to group appenders with the same level resolving.

Level Resolvers

As mentioned previously routers have a level resolver. A level resolver simply resolves the level (an enum) from a logger name (String). Because of LevelResolver functional interface we can chain them with fallbacks. An important fallback level resolver that almost all routers use is the global level resolver. These levels can be configured with properties where logger names prefixed with "logging.level" as the key and the level as the value. Below is an example:

logging.level.com.mycompany.stuff=DEBUG
logging.level.com.mycompany=INFO
logging.level=ERROR

Although not required most level resolvers including the global follow an inheritance or prefix model where com.mycompany.stuff.foo will resolve to DEBUG and com.mycompany.bar will resolve to INFO and anything not prefixed with com.mycompany will resolve to ERROR.

An important consideration is that Rainbow Gum by default assumes Level Resolvers are cached or static and will never change. This is a critical feature of Rainbow Gum as it allows facade loggers like in SLF4J to be lower overhead than almost all other logging frameworks! That being said RainbowGum allows levels to be changed if the backing configuration framework supports reloading and the logger name is allowed to be changed.

Below is an example of configuring to allow "changing" loggers

logging.global.change=true          # this is required to turn on level changing.
logging.change.com.mycompany=level  # only logger names starting with com.mycompany can have their levels changed.

Caller Information

Somewhat related to level resolvers and config change is how to enable caller info. LogEvent.Caller has stack trace like information on where the logging call was made. It is incredibly slow so like changing loggers it must be enabled and you can enable it with logger name inheritance so not all loggers will be slow. Below is an example of configuring to allow "changing" loggers with caller info:

logging.global.change=true               # this is required to turn on level changing.
logging.change.com.mycompany=caller      # only logger names starting with com.mycompany will have caller info

Caller info can be disabled dynamically with supporting configuration systems just like level changing. Note that "logging.change" is a comma separated list of LogConfig.ChangePublisher.ChangeType however caller info will automatically allow the logger to change levels as well.

Publishers

A LogPublisher schedules delivery of a LogEvent to a group of Appenders. Publishers come in two types:

synchronous - the default
asynchronous

An appender should only belong to one publisher. In other logging frameworks like Logback and Reload4J this responsibility is handled by an Appender but Rainbow Gum separates this responsibility out.

An example configuring a route named "example" to use the default async publisher:

logging.route.example.publisher=async
logging.publisher.example.bufferSize=1024

Using an async publisher is a complicated topic with many caveats one of them being what to do if the buffer is full. Rainbow Gum's default async publisher is a simple blocking queue implementation that has a single consumer thread and will block producing threads if the queue is full. The consumer thread simple iterates over the appenders pushing to each one. This loosely follows the single writer principle. In the future particularly with virtual threads more sophisticated async publishers might be offered that might fan-out and other exotic strategies.

NOTE: If you want some appenders to be async and others sync you just create multiple routes.

Appenders

An appender contains an Encoder and an Output. Unlike other logging frameworks it is rare to have custom Appenders as most of the work is done by the encoder and output. That is an appender can be mostly thought of as a tuple of encoder and output.

Encoders

Encoders encode an event into binary. Because most encoding is text based RainbowGum has the concept of LogFormatter which can be turned into an Encoder and is covered in the next section. However there are scenarios where generating bytes directly is desirable.

Rainbow Gum JSON encoder module io.jstach.rainbowgum.json has efficient JSON encoders. They are encoders instead of formatters (which are covered next) because writting JSON as bytes is more efficient.

Configuring an appenders encoder with properties looks something like:

logging.appenders=myappender
logging.appender.myappender.encoder=gelf
logging.encoder.myappender.prettyPrint=true

The above will use the GELF JSON encoder and configure it to pretty print.

Encoders should have a builder as well to configure programmatically and the builders javadoc has the string properties that can configure it.

Here are some of the supported encoders (not a complete listing):

Formatters

Formatting of log events as textual data is so common that Rainbow Gum has special support for that with LogFormatter. While formatters can at high-level be considered an encoder for technical API reasons they are not. However any formatter can be converted to an encoder with LogEncoder.of(io.jstach.rainbowgum.LogFormatter).

Pattern Formatter

For those coming from Logback or Log4j2 Rainbow Gum has an optional module for Logback style based pattern formatters. This allows describing an output format using String.format percent style syntax. Rainbow Gum implements most of the builtin Logback pattern keywords with far less overhead.

Like Logback you can even add your own pattern keywords which is covered in io.jstach.rainbowgum.pattern module.

The pattern module is pulled in by default with the aggregate rainbow gum dependency.

NOTE: If you plan on using the color pattern keywords it is generally a good idea (for now) to use the io.jstach.rainbowgum.jansi module which is covered next. Regardless ANSI escape can be globally disabled by setting "logging.global.ansi.disable" to true which will disable Jansi and the default pattern formatter from omitting ANSI escape sequences.

JAnsi support

Rainbow Gum also has support for proper cross platform ANSI output with JAnsi through an optional module. Unlike other frameworks where this is detected with reflection Rainbow Gum uses the Service Loader which is GraalVM native friendly.

Why is JAnsi desirable?

Support stripping ANSI characters on piping out which is desirable if you only use the console for output which is often the case for kubernetes and other microservices.
Windows color output support.

NOTE: Future versions of the JDK may require command line arguments for using jars packaged with native libraries. Given that Rainbow Gum favors security we strongly agree with the draft JEP and are exploring other options for the future.

Jansi can be disabled with "logging.jansi.disable" or "logging.global.ansi.disable" boolean properties.

A major reason to disable Jansi is because it may accidentally strip ANSI escape characters for terminals that do support ANSI but Jansi cannot determine they do. The most notable case of this problem are IDE terminals/consoles particularly IntelliJ's.

Outputs

LogOutput do the actual work of outputting an event. Configuring an appenders output with properties looks something like:

  logging.appenders=myappender
  logging.appender.myappender.output=stdout

One caveat with file support output via URI is if you need to output to a file in the current working directory the URI should be prefixed with ./.

  logging.appenders=myappender
  # logging.appender.myappender.output=app.log this is wrong.
  logging.appender.myappender.output=./app.log

The reason is that Rainbow Gum will think app.log is a URI scheme if it is not prefixed. Of course fully qualified file URI are supported as well like: file:///./app.log. For Spring Boot compatibility "logging.file.name" is also supported if custom routers and appenders are not configured.

Here are some of the supported outputs (not a complete listing):

stdout and stderr
FileOutputBuilder with URI scheme of "file"
RabbitMQOutputBuilder with URI scheme of "amqp"

Filtering

Like logback Rainbow Gum provides two types of filtering through functional composition:

router based filtering - similar to logbacks normal filters.
SLF4J based filtering - similar to logback Turbo Filters

Event based Filtering

Unfortunately at this time router based filtering is not configurable through properties and can only be added using Rainbow Gum builder and composition has to be done manually.

An example using router based filtering:

RainbowGum.builder(config) //
	.route(rb -> {
		rb.factory(RouterFactory.of(e -> {
			/*
			 * We only log DEBUG level events.
			 */
			return switch (e.level()) {
				case DEBUG -> e;
				default -> null;
			};
		}));
		/*
		 * If we do not set the level correctly the router will never get the
		 * event regardless of the logic of the above filtering function.
		 */
		rb.level(Level.DEBUG);
	});

The above creates a custom LogRouter.Router with a function. Event based filtering is still based on the invariants of level resolvers not changing (unless they are configured to be dynamic) so not all events are delivered if the level resolved does not not allow it! A work around is to adjust the levels of the router so that it does get all events desired and is why filtering is associated with routing. Unfortunately this can be inefficient as the event always has to be created so if performance matters consider using SLF4J based filtering.

SLF4J based filtering

SLF4J based filtering is not simple filtering but rather decorating of the SLF4J loggers somewhat akin to Servlet filtering. Rainbow Gum hands off its generated SLF4J logger to you and you can decorate or even disregard it and provide your own implementation. This is extremely powerful and efficient but requires more work. There is not really an analog in Logback or Log4j but as usual with great power comes great responsibility!

These filters are usually loaded with the ServiceLoader so manually using the Rainbow Gum builder is not required like router based filtering.

Installation

For most simply including the dependency io.jstach.rainbowgum:rainbowgum is enough. That dependency will transitively pull in the most commonly used modules. For a more minimal setup (total application size in terms of classes) io.jstach.rainbowgum:rainbowgum-core and io.jstach.rainbowgum:rainbowgum-slf4j (if using SLF4J) should be used.

Maven

<properties>
    <rainbowgum.version>0.5.0</rainbowgum.version>
</properties>
...
<dependencies>
    <dependency>
        <groupId>io.jstach.rainbowgum</groupId>
        <artifactId>rainbowgum</artifactId>
        <version>${rainbowgum.version}</version>
        <scope>runtime</scope>
    </dependency>
</dependencies>

If you plan on configuring Rainbow Gum programmatically you will need to make a module and create a service loader registration. In that case you will want the dependency like:

<properties>
    <rainbowgum.version>0.5.0</rainbowgum.version>
</properties>
...
<dependencies>
    <dependency>
        <groupId>io.jstach.rainbowgum</groupId>
        <artifactId>rainbowgum-core</artifactId>
        <version>${rainbowgum.version}</version>
        <scope>compile</scope>
    </dependency>
</dependencies>

Gradle


dependencies {
    
    runtimeOnly 'io.jstach.rainbowgum:rainbowgum:0.5.0'
}

If you plan on configuring Rainbow Gum programmatically you will need to make a module and create a service loader registration. In that case you will want the dependency like:


dependencies {
    
    implementation 'io.jstach.rainbowgum:rainbowgum-core:0.5.0'
}

Extensions and Integrations

SLF4J 2.0

Rainbow Gum SLF4J module io.jstach.rainbowgum.slf4j supports:

However there is currently no Marker support.

Rainbow Gum supports key value pairs in LoggingEventBuilder by overlaying on top of the current MDC at the time the event is constructed and put into the events key values. The value parameter in LoggingEventBuilder.addKeyValue(String,Object) are converted to String immediately as only String values are supported at this time.

Rainbow Gum SLF4J implementation is unique in that it has two special implementation of loggers:

Level Logger - logger based on level threshold and can never change!
Changing Logger - level and other configuration can change.

Other logging implementations like Logback by default use something analogous to changing loggers which require a constant check if the level threshold has changed. Level loggers do not need to do that check. Unless changing loggers is turned on by default Level Loggers are used which are close to zero cost for discarding events.

java.lang.System.Logger and java.util.logging

Rainbow Gum io.jstach.rainbowgum.jdk module provides special integration and adapters for the builtin JDK logging facilities. The impetus for this is these logging facilities can be used early in the JDK boot processes well before logging has fully initialized. The default rainbow gum dependency has this integration as a transitive dependency.

The integration will make sure that neither the System.Logger or java.util.logging initialize Rainbow Gum too early by queueing the events. When a Rainbow Gum initializes and set as global the events will be replayed. If the events level are equal to System.Logger.Level.ERROR and a normal Rainbow Gum has not been bound the messages will be printed to System.err. The idea is something catastrophic has happened that will probably cause Rainbow Gum to never load and thus never replay the events and you will not be able to figure out what happened otherwise.

SLF4J does provide an adapter/bridge for the System.Logger (org.slf4j:slf4j-jdk-platform-logging) but its use may cause Rainbow Gum to initialize too early. However that maybe desirable if:

You are sure that Rainbow Gum can initialize early
Your application uses System.Logger (the SLF4J adapter will initialize Rainbow Gum on System.Logger usage if using io.jstach.rainbowgum.slf4j)

In the future Rainbow Gum may include a System.Logger provider module that does eager initialization.

NOTE: While the JDK System.Logger is good for low level libraries that rarely log it's API (and Rainbow Gum implementation) is not designed for performance. For applications and frameworks that do a lot of logging the SLF4J facade is the preferred choice.

Spring Boot

Spring Boot allows configuration through simple properties for logging that tries to be logging framework agnostic. How this works is Spring Boot configures the logging framework from its own set of properties. There is a shocking amount of code Spring Boot requires to do this for Logback and Log4J2 because both of those frameworks were not designed well for programmatic configuration.

Rainbow Gum at the moment does not require special integration because it largely follows the Spring Boot property patterns. For example configuring levels to logger name with properties in Spring Boot is exactly the same as it is in Rainbow Gum:

logging.level.com.mypackage=INFO

At some point Spring Boot will likely diverge enough on its property keys that special integration will be needed. Rainbow Gum will have a solution for that in the future.

Extension development guide

To a develop a custom plugin it is probably easiest to look at the other rainbow gum modules. An important concept with extensions are the registries which allow you to register a provider with a URI scheme.

LogConfig contains all the registries and a special generic registry: ServiceRegistry. The service registry is allows other plugins to share components. To access these registries at configuration time as a plugin is to implement a RainbowGumServiceProvider.Configurator.

Next one should construct builders that are properties friendly to create the components so that programmatic configuration users can use your extension. Rainbow Gum provides an annotation processor that helps create builders io.jstach.rainbowgum.annotation.

  <properties>
      <rainbowgum.version>0.5.0</rainbowgum.version>
  </properties>
  ...
  <dependencies>
      <dependency>
          <groupId>io.jstach.rainbowgum</groupId>
          <artifactId>rainbowgum-core</artifactId>
          <version>${rainbowgum.version}</version>
          <scope>compile</scope>
      </dependency>
      <!-- This is the annotation processor -->
      <dependency>
          <groupId>io.jstach.rainbowgum</groupId>
          <artifactId>rainbowgum-apt</artifactId>
          <version>${rainbowgum.version}</version>
          <!-- the following maven config is important as the annotation processor should never be a transitive dep -->
          <scope>provided</scope>
          <optional>true</optional>
      </dependency>
  </dependencies>

Once the annotation processor is enabled one can create a properties aware builder with a single static factory method annotated with LogConfigurable.

FAQ

There is a typo in this documentation how can I fix it?

If you would like to make corrections please file an issue or even better fork, edit, PR this file:

doc/overview.html

Where is the Javadoc?

Shockingly this document is the Javadoc! To be precise it is the aggregate javadoc overview.html. The modules javadocs should be at the bottom of this document and the search bar at the top can be used to find documented classes.

Modules

Module

Description

io.jstach.rainbowgum

Core module for RainbowGum which provides low level components for logging as well as a builder for creating custom RainbowGums.

io.jstach.rainbowgum.annotation

Rainbowgum Annotations used for code generation.

io.jstach.rainbowgum.avaje

Uses Avaje Config for LogProperties.

io.jstach.rainbowgum.disruptor

EXPERIMENTAL Disruptor async publisher.

io.jstach.rainbowgum.jansi