rainbowgum API 0.6.0
User Guide
Rainbow Gum: JDK 21+ SLF4J logging implementationFast, modular, GraalVM native friendly and easy to use.
Contents
Getting Started
First step is to add Rainbow Gum as a dependency.
Second if you are not familiar with logging in the Java ecosystem please refer to
the excellent Logging facade SLF4J documentation
particularly the logger API
and what logging Levels are. Logging levels are and how dotted logger names inherit levels is covered in
the JDKs logging facade System.Logger.Level
as well as
Logback effective level
which Rainbow Gum follows.
Using System properties
Assuming we have installed RainbowGum as a dependency a simple configuration example using System Properties is below: java \
-Dlogging.appender.console.encoder=pattern \
-Dlogging.encoder.console.pattern="[%thread] %-5level %logger{15} - %msg%n" \
-Dlogging.level.com.myapp=DEBUG \
myapp.jar
Using RainbowGum Builders
For organizations that have many projects it is recommended to use the programmatic builder approach and make a standardized jar (module) shared between all your projects so that logging configuration is consistent. Assuming we have installed RainbowGum as a dependency a simple configuration example using Java directly and theServiceLoader
is below.
public class GettingStartedExample implements RainbowGumProvider {
@Override
public Optional<RainbowGum> provide(LogConfig config) {
return RainbowGum.builder(config) //
.route(r -> {
r.level(Level.DEBUG, "com.myapp");
r.appender("console", a -> {
a.encoder(new PatternEncoderBuilder("console")
// We use the pattern encoder which follows logback pattern
// syntax.
.pattern("[%thread] %-5level %logger{15} - %msg%n")
// We use properties to override the above pattern if set.
.fromProperties(config.properties())
.build());
});
}) //
.optional();
}
}
RainbowGumServiceProvider
has additional documentation on how to register
a service loader aware jar.
Description
Rainbow Gum is a JDK 21+ opinionated SLF4J implementation that aims to be easier to use while leveraging newer JDK technology. Rainbow Gum unlike Logback or Log4J (2) does not offer as much flexibility but is simpler and has less overhead. The readme in the Rainbow Gum project discusses more extensively on the opinionated design and philosophy.Project Information
- Source Control
- https://github.com/jstachio/rainbowgum
- Team
-
- Adam Gent (agentgt) - lead
- Issues
- https://github.com/jstachio/rainbowgum/issues
- Community
- https://github.com/jstachio/rainbowgum/discussions
- User Guide
- This document
- Javadoc
- This document (modules listing at bottom)
Requirements
- Java 21 or greater
- A build system that supports running the Java compiler annotation processor
java.base
Limitations
Currently Rainbow Gum does not provide support for:- External config - by design you will need to bring your own configuration lib or just use env/system properties.
- Rolling of log files without external tool - there is support for rolling without losing events using external tool such as logrotate.
- SLF4J Marker support - libraries and application rarely use it compared to MDC.
How it works
Most users will use Rainbow Gum through a logging facade such as SLF4J and the only interaction with Rainbow Gum is configuration of output and formatting. Rainbow Gum provides three ways to configure out of the box:- Programmatic Java configuration using builders.
- Simple String key/value properties often derived from System properties / env variables
- Dependency driven configuration where including a jar as dependency changes behavior automatically
Architecture
Rainbow Gum's design has many superficial similarities to Logback, Log4j2, and Reload4j in that there are Appenders and Encoders as well as how log levels are inherited but most of Rainbow Gum's design is very different!The key difference in Rainbow Gum is that once configured and initialized the logging system is locked in and cannot be changed. Certain parts of the system can be changed at runtime but require opt-in which includes changing levels of logger names.
Another major difference is that Rainbow Gum is highly modularized, immutable and componentized. In other frameworks
OOP inheritance is heavily abused and riddled with state management. Furthermore there are components that
have too much responsibility which is the case with Appenders in Logback or "Managers" in Log4j2.
Furthermore Rainbow Gum uses zero reflection other than the Service Loader and follows the builder pattern
extensively to separate configuration from immutable runtime components. Almost all components have
a builder to programmatically configure and the builders can take flat string key values as configuration.
The other logging frameworks use an enormous amount of reflection which slows initialization time.
(While the ServiceLoader
is technically reflection it is GraalVM native friendly and
is the preferred way for pluggable components in modern JDKs.)
The results of the above choices make Rainbow Gum far lighter than logback, log4j2, and reload4j particularly in initialization time as well as security surface area.
The key components expressed in the flow of log events is as follows:- Logging Facade (SLF4J)
RainbowGum
LogRouter
LogPublisher
LogAppender
LogEncoder
LogOutput
Config
A defining characteristic of Rainbow Gum is that it does not have a special configuration format like Logback, Log4j2, and Reload4j (all three use XML as the default). The expectation is that for simple configuration simple properties interface analogous toFunction<String,String>
is good enough. For more complicated configuration
programmatic configuration using the builders and service loader should be used.
For implementers of plugins LogConfig
allows registering of services
and plugins (it also contains the global LogProperties
) however
most users will not need to know about it. For those coming from Logback
LogConfig
is analogous to what Logback calls "Context".
Rainbow Gum out of the box uses System properties for LogProperties
but an optional module io.jstach.rainbowgum.avaje
will allow using Avaje Config
as the properties provider. Other configuration systems will be added in the future but an
important requirement is that these systems do not do any logging or if they do allow it to be
turned off, intercepted or blocked.
IMPORTANT: Rainbow Gum core does not do any interpolation of property values!
That is the responsibility of the LogProperties
backing implementation and the values retrieved should already be interpolated.
NOTE:
Throughout the documentation Properties
and URI
syntax is used to express
configurable properties but it is a not a requirement and
your configuration system may have a different format like YAML or JSON.
A listing of property patterns that are used out of the box is discussed in LogProperties
.
A general pattern is:
logging.{componentType}.{name}.{subComponent}=URI
logging.{subComponent}.{name}.{propertyName}=someStringThatIsConverted
logging.appender.example.encoder=gelf
logging.encoder.example.prettyPrint=true
gelf
is actually gelf:///
Because the first property is a URI one can do:
logging.appender.example.encoder=gelf:///?prettyPrint=true
URI scheme
is used for plugin lookup as well as single line configuration. This follows
12 factor recommendation of resources and configuration.
Config Change during runtime
While most of the logging components in Rainbow Gum are immutable and cannot be changed during runtime there are some exceptions including custom components.A supporting configuration system (a LogProperties provider) can notify that there has been a change by using the Change Publisher. This is how level resolving changes can be published.
Config change is by default off even on supporting configuration systems and can only be enabled by setting "logging.global.change" to true.
Router
A Router has aLevelResolver
and a Publisher.
A Level Resolver takes a logger name and produces a level
.
The logging facade implementation than decides based on the level whether or not to construct an event.
It then passes the event to the router which in turn uses the publisher to schedule the event.
For those familiar with other logging frameworks
Rainbow Gum has no actual concept of named "Loggers" but a route from a router is the closest analog.
When configuring routers you are configuring
Level Resolvers
which are logger names to levels,
which publisher to use and which appenders should be associated with the publisher.
We call this configuration a "route". As discussed later on
a publisher has appenders and appenders have outputs thus
an important consequnce of this is if you want outputs to have different level thresholds
(e.g. a debug.log and an error.log) you will need multiple routers.
example
".
logging.routes=example
logging.route.example.appenders=appender1,appender2
logging.route.example.publisher=async
logging.route.example.level.com.mycompany=DEBUG
Level Resolvers
As mentioned previously routers have a level resolver. A level resolver simply resolves the level (an enum) from a logger name (String). Because ofLevelResolver
functional interface we can chain them
with fallbacks. An important fallback level resolver that almost all routers use
is the global level resolver. These levels can be configured with
properties
where logger names prefixed with
"logging.level" as the key and the level as the value.
Below is an example:
logging.level.com.mycompany.stuff=DEBUG
logging.level.com.mycompany=INFO
logging.level=ERROR
com.mycompany.stuff.foo
will resolve to DEBUG
and com.mycompany.bar
will resolve to INFO
and anything not prefixed with com.mycompany
will resolve to ERROR
.
An important consideration is that Rainbow Gum by default assumes Level Resolvers are cached or static and will never change. This is a critical feature of Rainbow Gum as it allows facade loggers like in SLF4J to be lower overhead than almost all other logging frameworks! That being said RainbowGum allows levels to be changed if the backing configuration framework supports reloading and the logger name is allowed to be changed.
Below is an example of configuring to allow "changing" loggerslogging.global.change=true # this is required to turn on level changing.
logging.change.com.mycompany=level # only logger names starting with com.mycompany can have their levels changed.
Caller Information
Somewhat related to level resolvers and config change is how to enable caller info.LogEvent.Caller
has stack trace like information on where the logging call was made.
It is incredibly slow so like changing loggers it must be enabled and you can enable it with logger name inheritance
so not all loggers will be slow.
Below is an example of configuring to allow "changing" loggers with caller info:
logging.global.change=true # this is required to turn on level changing.
logging.change.com.mycompany=caller # only logger names starting with com.mycompany will have caller info
LogConfig.ChangePublisher.ChangeType
however caller info will automatically allow
the logger to change levels as well.
Publishers
ALogPublisher
schedules delivery of a LogEvent
to a group of
Appenders. Publishers come in two types:
- synchronous - the default
- asynchronous
An example configuring a route named "example" to use the default async publisher:
logging.route.example.publisher=async
logging.publisher.example.bufferSize=1024
NOTE: If you want some appenders to be async and others sync you just create multiple routes.
Appenders
An appender contains an Encoder and anOutput
.
Unlike other logging frameworks it is rare to have custom Appenders as most of the work is done by the encoder and output.
That is an appender can be mostly thought of as a tuple of encoder and output.
Encoders
Encoders encode an event into binary. Because most encoding is text based RainbowGum has the concept ofLogFormatter
which can be turned into an Encoder and is covered
in the next section. However there are scenarios where generating bytes directly is desirable.
Rainbow Gum JSON encoder module io.jstach.rainbowgum.json
has efficient JSON encoders.
They are encoders instead of formatters (which are covered next) because writting JSON as bytes is more efficient.
logging.appenders=myappender
logging.appender.myappender.encoder=gelf
logging.encoder.myappender.prettyPrint=true
Encoders should have a builder as well to configure programmatically and the builders javadoc has the string properties that can configure it.
Here are some of the supported encoders (not a complete listing):Formatters
Formatting of log events as textual data is so common that Rainbow Gum has special support for that withLogFormatter
. While formatters can at high-level be considered
an encoder for technical API reasons they are not. However any formatter can be converted to an
encoder with LogEncoder.of(io.jstach.rainbowgum.LogFormatter)
.
Pattern Formatter
For those coming from Logback or Log4j2 Rainbow Gum has an optional module for Logback style based pattern formatters. This allows describing an output format usingString.format
percent style syntax. Rainbow Gum implements most of the builtin Logback
pattern keywords with far less overhead.
Like Logback you can even add your own pattern keywords which is covered in io.jstach.rainbowgum.pattern
module.
NOTE: If you plan on using the color pattern keywords it is generally a good idea (for now) to use
the io.jstach.rainbowgum.jansi
module which is covered next. Regardless ANSI escape can be
globally disabled by setting "logging.global.ansi.disable" to
true
which will disable Jansi and the default pattern formatter from omitting ANSI escape sequences.
JAnsi support
Rainbow Gum also has support for proper cross platform ANSI output with JAnsi through an optional module. Unlike other frameworks where this is detected with reflection Rainbow Gum uses the Service Loader which is GraalVM native friendly.
Why is JAnsi desirable?- Support stripping ANSI characters on piping out which is desirable if you only use the console for output which is often the case for kubernetes and other microservices.
- Windows color output support.
NOTE: Future versions of the JDK may require command line arguments for using jars packaged with native libraries. Given that Rainbow Gum favors security we strongly agree with the draft JEP and are exploring other options for the future.
Jansi can be disabled with "logging.jansi.disable" or "logging.global.ansi.disable" boolean properties.A major reason to disable Jansi is because it may accidentally strip ANSI escape characters for terminals that do support ANSI but Jansi cannot determine they do. The most notable case of this problem are IDE terminals/consoles particularly IntelliJ's.
Outputs
LogOutput
do the actual work of outputting an event.
Configuring an appenders output with properties
looks something like:
logging.appenders=myappender
logging.appender.myappender.output=stdout
./
.
logging.appenders=myappender
# logging.appender.myappender.output=app.log this is wrong.
logging.appender.myappender.output=./app.log
app.log
is a URI scheme if it is not prefixed.
Of course fully qualified file URI are supported as well like: file:///./app.log
.
For Spring Boot compatibility "logging.file.name" is also supported
if custom routers and appenders are not configured.
Here are some of the supported outputs (not a complete listing):
- stdout and stderr
FileOutputBuilder
with URI scheme of "file"RabbitMQOutputBuilder
with URI scheme of "amqp"
Rolling Files
Rainbow Gum currently does not support rolling of files on its own but does provide a mechanism to safely allow external programs such as logrotate to do the rolling. How this typically works:- External program moves the current log file. Rainbow Gum will continue to log to the same file but the file name is effectively changed.
- Signals to the Java process with Rainbow Gum to reopen the files. This signal is usually done via HTTP or TCP socket since Java does not support Unix signals.
- The external program then typically compresses, delete or send the log files elsewhere.
(The other method of rotating files without doing a move and reopen is called "copy truncate" but has the potential of losing events. This method avoids that problem. )
Unfortunately Rainbow Gum does not offer a way to receive the external signal as this can vary greatly across applications. The following is an example of reopening outputs using HTTP for signaling./*
* This method could be bound to an internal HTTP route say /log/rotate such that curl
* http://localhost:8080/log/rotate will block and wait till all the applicable
* outputs have reopened.
*/
@RequestMapping("/log/rotate")
@ResponseBody
public String someInternalHttpRequestHandler() {
var gum = RainbowGum.getOrNull();
if (gum != null) {
List<LogResponse> response = gum.config() //
.outputRegistry() //
.reopen(); // Here is where we siginal to reopen outputs that support
// reopening.
return response.toString();
}
return "";
}
http://localhost:8080/log/rotate
and we have configured our logging like
logging.appenders=myappender
logging.appender.myappender.output=/var/log/app.log
/var/log/app.log
{
rotate 4
weekly
missingok
notifempty
compress
delaycompress
# we only need one call to reopen all files
sharedscripts
# in most cases it is best for rainbowgum recreate the file which wil happen after the postrotate
nocreate
postrotate
/usr/bin/curl -s -o /dev/null -w "%{http_code}" http://localhost:8080/log/rotate | {
read status
if [ "$status" -ne 200 ]; then
logger "logrotate: Error hitting endpoint, received HTTP status $status"
fi
}
endscript
}
Filtering
Like logback Rainbow Gum provides two types of filtering through functional composition:- router based filtering - similar to logbacks normal filters.
- SLF4J based filtering - similar to logback Turbo Filters
Event based Filtering
Unfortunately at this time router based filtering is not configurable through properties and can only be added using Rainbow Gum builder and composition has to be done manually.An example using router based filtering:
RainbowGum.builder(config) //
.route(rb -> {
rb.factory(RouterFactory.of(e -> {
/*
* We only log DEBUG level events.
*/
return switch (e.level()) {
case DEBUG -> e;
default -> null;
};
}));
/*
* If we do not set the level correctly the router will never get the
* event regardless of the logic of the above filtering function.
*/
rb.level(Level.DEBUG);
});
LogRouter.Router
with a function.
Event based filtering is still based on the invariants
of level resolvers not changing (unless they are configured to be dynamic) so
not all events are delivered if the level resolved does not not allow it!
A work around is to adjust the levels of the router so that it does get
all events desired and is why filtering is associated with routing.
Unfortunately this can be inefficient as the event always has to be created
so if performance matters consider using SLF4J based filtering.
SLF4J based filtering
SLF4J based filtering is not simple filtering but rather decorating of the SLF4J loggers somewhat akin to Servlet filtering. Rainbow Gum hands off its generated SLF4J logger to you and you can decorate or even disregard it and provide your own implementation. This is extremely powerful and efficient but requires more work. There is not really an analog in Logback or Log4j but as usual with great power comes great responsibility!These filters are usually loaded with the ServiceLoader so manually using the Rainbow Gum builder is not required like router based filtering.
Installation
For most simply including the dependencyio.jstach.rainbowgum:rainbowgum
is enough. That dependency will transitively pull in the most commonly used modules.
For a more minimal setup (total application size in terms of classes)
io.jstach.rainbowgum:rainbowgum-core
and io.jstach.rainbowgum:rainbowgum-slf4j
(if using SLF4J)
should be used.
Maven
<properties>
<rainbowgum.version>0.6.0</rainbowgum.version>
</properties>
...
<dependencies>
<dependency>
<groupId>io.jstach.rainbowgum</groupId>
<artifactId>rainbowgum</artifactId>
<version>${rainbowgum.version}</version>
<scope>runtime</scope>
</dependency>
</dependencies>
<properties>
<rainbowgum.version>0.6.0</rainbowgum.version>
</properties>
...
<dependencies>
<dependency>
<groupId>io.jstach.rainbowgum</groupId>
<artifactId>rainbowgum-core</artifactId>
<version>${rainbowgum.version}</version>
<scope>compile</scope>
</dependency>
</dependencies>
Gradle
dependencies {
runtimeOnly 'io.jstach.rainbowgum:rainbowgum:0.6.0'
}
If you plan on configuring Rainbow Gum programmatically you will need to make a module and create a
service loader registration. In that case you will want the dependency like:
dependencies {
implementation 'io.jstach.rainbowgum:rainbowgum-core:0.6.0'
}
Extensions and Integrations
SLF4J 2.0
Rainbow Gum SLF4J moduleio.jstach.rainbowgum.slf4j
supports:
However there is currently no Marker
support.
Rainbow Gum supports key value pairs
in LoggingEventBuilder
by overlaying on top
of the current MDC at the time the event is constructed and put into the
events key values
.
The value parameter in
LoggingEventBuilder.addKeyValue(String,Object)
are
converted to String
immediately as only String values are
supported at this time.
Rainbow Gum SLF4J implementation is unique in that it has two special implementation of loggers:
- Level Logger - logger based on level threshold and can never change!
- Changing Logger - level and other configuration can change.
java.lang.System.Logger and java.util.logging
Rainbow Gumio.jstach.rainbowgum.jdk
module provides special integration and adapters for the builtin JDK logging facilities.
The impetus for this is these logging facilities can be used early in the JDK boot processes well before
logging has fully initialized. The default rainbow gum dependency has this integration as a transitive dependency.
The integration will make sure that neither the System.Logger or java.util.logging initialize
Rainbow Gum too early by queueing the events. When a Rainbow Gum initializes and
set as global the events will be replayed.
If the events level are equal to System.Logger.Level.ERROR
and a normal Rainbow Gum has not been bound
the messages will be printed to System.err
.
The idea is something catastrophic has happened that will probably cause Rainbow Gum to never load and thus never replay the events
and you will not be able to figure out what happened otherwise.
SLF4J does provide an adapter/bridge for the System.Logger (org.slf4j:slf4j-jdk-platform-logging) but its use may cause Rainbow Gum to initialize too early. However that maybe desirable if:
- You are sure that Rainbow Gum can initialize early
- Your application uses System.Logger
(the SLF4J adapter will initialize Rainbow Gum on System.Logger usage if using
io.jstach.rainbowgum.slf4j
)
NOTE: While the JDK System.Logger is good for low level libraries that rarely log it's API (and Rainbow Gum implementation) is not designed for performance. For applications and frameworks that do a lot of logging the SLF4J facade is the preferred choice.
Spring Boot
Spring Boot allows configuration through simple properties for logging that tries to be logging framework agnostic. How this works is Spring Boot configures the logging framework from its own set of properties. There is a shocking amount of code Spring Boot requires to do this for Logback and Log4J2 because both of those frameworks were not designed well for programmatic configuration.
Rainbow Gum at the moment does not require special integration because it largely follows the Spring Boot property patterns. For example configuring levels to logger name with properties in Spring Boot is exactly the same as it is in Rainbow Gum:
logging.level.com.mypackage=INFO
At some point Spring Boot will likely diverge enough on its property keys that special integration will be needed. Rainbow Gum will have a solution for that in the future.
Extension development guide
To a develop a custom plugin it is probably easiest to look at the other rainbow gum modules. An important concept with extensions are the registries which allow you to register a provider with a URI scheme.LogConfig
contains all the registries
and a special generic registry: ServiceRegistry
.
The service registry is allows other plugins to share components.
To access these registries at configuration time as a plugin is to implement
a RainbowGumServiceProvider.Configurator
.
Next one should construct builders that are properties friendly to create the components
so that programmatic configuration users can use your extension. Rainbow Gum provides
an annotation processor that helps create builders
io.jstach.rainbowgum.annotation
.
<properties>
<rainbowgum.version>0.6.0</rainbowgum.version>
</properties>
...
<dependencies>
<dependency>
<groupId>io.jstach.rainbowgum</groupId>
<artifactId>rainbowgum-core</artifactId>
<version>${rainbowgum.version}</version>
<scope>compile</scope>
</dependency>
<!-- This is the annotation processor -->
<dependency>
<groupId>io.jstach.rainbowgum</groupId>
<artifactId>rainbowgum-apt</artifactId>
<version>${rainbowgum.version}</version>
<!-- the following maven config is important as the annotation processor should never be a transitive dep -->
<scope>provided</scope>
<optional>true</optional>
</dependency>
</dependencies>
LogConfigurable
.
FAQ
There is a typo in this documentation how can I fix it?
If you would like to make corrections please file an issue or even better fork, edit, PR this file:doc/overview.html
Where is the Javadoc?
Shockingly this document is the Javadoc! To be precise it is the aggregate javadocoverview.html
.
The modules javadocs should be at the bottom of this document and the search bar at the top can be
used to find documented classes.