Rainbow Gum 0.10.0
User Guide
Rainbow Gum: JDK 21+ SLF4J logging implementationFast, modular, GraalVM native friendly and easy to use.
Contents
Getting Started
First step is to add Rainbow Gum as a dependency.
Second if you are not familiar with logging in the Java ecosystem please refer to
the excellent Logging facade SLF4J documentation
particularly the logger API.
What logging levels are and how dotted logger names inherit levels is covered in
the JDKs logging facade System.Logger.Level as well as
Logback effective level
which Rainbow Gum follows.
Using Spring Boot
If using Spring Boot see Rainbow Spring Boot Integration section.Using System properties
Assuming we have installed RainbowGum as a dependency a simple configuration example using System Properties is below: java \
-Dlogging.appender.console.encoder=pattern \
-Dlogging.encoder.console.pattern="[%thread] %-5level %logger{15} - %msg%n" \
-Dlogging.level.com.myapp=DEBUG \
myapp.jar
Using RainbowGum Builders
For technical organizations that have many projects it is recommended to use the programmatic builder approach and make a standardized jar (module) shared between all your projects so that logging initialization and configuration is consistent. Assuming we have installed RainbowGum as a dependency a simple configuration example using Java directly and theServiceLoader is below.
public class GettingStartedExample implements RainbowGumProvider {
@Override
public Optional<RainbowGum> provide(LogConfig config) {
return RainbowGum.builder(config) //
.route(r -> {
r.level(Level.DEBUG, "com.myapp");
r.appender("console", a -> {
a.encoder(new PatternEncoderBuilder("console")
// We use the pattern encoder which follows logback pattern
// syntax.
.pattern("[%thread] %-5level %logger{15} - %msg%n")
// We use properties to override the above pattern if set.
.fromProperties(config.properties())
.build());
});
}) //
.optional();
}
}
RainbowGumServiceProvider has additional documentation on how to register
a service loader aware jar.
Description
Rainbow Gum is a JDK 21+ opinionated SLF4J implementation that aims to be easier to use while leveraging newer JDK technology. Rainbow Gum unlike Logback or Log4J (2) does not offer as much flexibility but is simpler and has less overhead. The readme in the Rainbow Gum project discusses more extensively on the opinionated design and philosophy.Project Information
- Source Control
- https://github.com/jstachio/rainbowgum
- Team
-
- Adam Gent (agentgt) - lead
- Issues
- https://github.com/jstachio/rainbowgum/issues
- Community
- https://github.com/jstachio/rainbowgum/discussions
- User Guide
- This document
- Javadoc
- This document (modules listing at bottom)
Requirements
- Java 21 or greater
- A build system that supports running the Java compiler annotation processor
java.base
Limitations
Currently Rainbow Gum does not provide support for:- Default external config file (e.g. log4j2.xml) - by design but do not worry because Rainbow Gum provides lots of extensions to use your applications configuration system. Out of the box system properties are supported.
- Rolling of log files without external tool - there is support for rolling without losing events using external tool such as logrotate.
- SLF4J Marker support - libraries and application rarely use it compared to MDC.
- It is actually fairly expensive to load resources from the module/classpath on initialization. If your configuration system is already loading a resource it should be used instead.
- In GraalVM native and in some cases modular environment loading module/classpath resources is more complicated. We do not want an OOB experience where it works in normal HotSpot but not in GraalVM native.
- It increases the security surface and Rainbow Gum aims to have security and integrity by default. Implicit loading of resources we feel is against this and if done should be a chosen opt-in which you can have with various extensions.
How it works
Most users will use Rainbow Gum through a logging facade such as SLF4J and the only interaction with Rainbow Gum is configuration of output and formatting. Rainbow Gum provides three ways to configure out of the box:- Simple String key/value properties often derived from System properties / env variables
- Programmatic Java configuration using builders.
- Dependency driven configuration where including a jar as dependency changes behavior automatically
Architecture
Rainbow Gum's design has many superficial similarities to Logback, Log4j2, and Reload4j in that there are Appenders and Encoders as well as how log levels are inherited but most of Rainbow Gum's design is very different!The key difference in Rainbow Gum is that once configured and initialized the logging system is locked in and cannot be changed. Certain parts of the system can be changed at runtime but require opt-in which includes changing levels of logger names.
Another major difference is that Rainbow Gum is highly modularized, immutable and componentized. In other frameworks
OOP inheritance is heavily abused and riddled with state management. Furthermore there are components that
have too much responsibility which is the case with Appenders in Logback or "Managers" in Log4j2.
Furthermore Rainbow Gum uses zero reflection other than the Service Loader and follows the builder pattern
extensively to separate configuration from immutable runtime components. Almost all components have
a builder to programmatically configure and the builders can take flat string key values as configuration.
The other logging frameworks use an enormous amount of reflection which slows initialization time.
(While the ServiceLoader is technically reflection it is GraalVM native friendly and
is the preferred way for pluggable components in modern JDKs.)
The results of the above choices make Rainbow Gum far lighter than logback, log4j2, and reload4j particularly in initialization time as well as security surface area.
The key components expressed in the flow of log events is as follows:- Logging Facade (SLF4J)
RainbowGumLogRouterLogPublisherLogAppenderLogEncoderLogOutput
Roughly the hiearchy (through composition) of components is:
- Routes have a single publisher a level resolver and a list of appenders.
- Publishers are given the list of appenders from the route on start.
- Appenders have a single encoder and output.
Config
A defining characteristic of Rainbow Gum is that it does not have a special configuration format like Logback, Log4j2, and Reload4j (all three use XML as the default). The expectation is that for simple configuration simple properties interface analogous toFunction<String,String> is good enough. For more complicated configuration
programmatic configuration using the builders and service loader should be used.
For implementers of plugins LogConfig allows registering of services
and plugins (it also contains the global LogProperties ) however
most users will not need to know about it. For those coming from Logback
LogConfig is analogous to what Logback calls "Context".
Properties based
Rainbow Gum out of the box uses System properties forLogProperties
but an optional module io.jstach.rainbowgum.avaje will allow using Avaje Config
as the properties provider. Other configuration systems will be added in the future but an
important requirement is that these systems do not do any logging or if they do allow it to be
turned off, intercepted or blocked.
IMPORTANT: Rainbow Gum core does not do any interpolation of property values!
That is the responsibility of the LogProperties
backing implementation and the values retrieved should already be interpolated.
NOTE:
Throughout the documentation Properties and URI syntax is used to express
configurable properties but it is a not a requirement and
your configuration system may have a different format like YAML or JSON.
A listing of property patterns that are used out of the box is discussed in LogProperties.
A general pattern is:
logging.{componentType}.{name}.{subComponent}=URI
logging.{subComponent}.{name}.{propertyName}=someStringThatIsConverted
logging.appender.example.encoder=gelf
logging.encoder.example.prettyPrint=true
gelf is actually gelf:///
Because the first property is a URI one can do:
logging.appender.example.encoder=gelf:///?prettyPrint=true
URI scheme
is used for plugin lookup as well as single line configuration. This follows
12 factor recommendation of resources and configuration.
Plural keys for enabling/disabling
Because Rainbow Gum's property config model is a lookup of key to value and not a Map like collection special plural keys are used to indicate what is activated. These keys canonical end in"s" and are comma separated.
logging.appenders=appender1,appender2 # appenders that are enabled.
logging.appender.appender1.encoder=...
logging.appender.appender2.output=...
logging.appender.appender3.output=....
appender3 will not be used
and will not produce a configuration error if incorrect. This pattern
allows large groups of configuration to be turned on or off similar to
profiles in other systems.
Implicit default configuration
Rainbow Gum will happily start with zero properties set because most of the properties described above have a built-in default when missing. Nothing here is hidden magic though - the defaults are just what get used when a property lookup falls through to code instead of a configured value, and that same "out of the box" setup can be reproduced explicitly with:logging.level=INFO
logging.routes=default
logging.route.default.publisher=sync
# logging.appenders is shorthand for logging.route.default.appenders - it only
# works as a fallback for the "default" route, not any other route name.
logging.appenders=console
logging.appender.console.output=stdout
logging.appender.console.encoder=ttll
-
logging.appendersdefaults to justconsole- notconsole,file. A file appender is only ever added automatically, and only when "logging.file.name" (logging.file.name) is itself set - in which case the implicit default becomeslogging.appenders=file,console(file first, console still included, not replaced). Settinglogging.appendersyourself to any value, including justfile, disables this auto-add behavior entirely since an explicit value always wins. -
stdoutandttllabove areURIschemes (see the plain properties example earlier in this section) resolved through the sameLogOutputRegistry/LogEncoderRegistrylookup any explicitoutput/encodervalue goes through -ttllis Rainbow Gum's built-in Logback-esque "Time Thread Level Logger" line formatter, not a Spring Boot-ism. -
logging.route.default.publisher=syncmeans every appender on the default route runs synchronously on the calling thread by default (no background queue/thread) - seeLogPublisherRegistryfor theasyncalternative and its own defaults/properties.
Builder based
Rainbow Gum can be configured programmatically through builders. The builders normally will ignore whatever properties (key values) are set externally but most builders can be configured by properties throughLogBuilder.fromProperties(io.jstach.rainbowgum.LogProperties)
and thus a combination of properties and builders can be used.
Config Change during runtime
While most of the logging components in Rainbow Gum are immutable and cannot be changed during runtime there are some exceptions including custom components.A supporting configuration system (a LogProperties provider) can notify that there has been a change by using the Change Publisher. This is how level resolving changes can be published.
Config change is by default off even on supporting configuration systems and can only be enabled by setting "logging.global.change" to true.
Router
A Router has aLevelResolver, appenders and a Publisher.
A Level Resolver takes a logger name and produces a level.
The logging facade implementation then decides based on the level whether or not to construct an event.
It then passes the event to the router which in turn uses the publisher to schedule the event.
For those familiar with other logging frameworks
Rainbow Gum has no actual concept of named "Loggers" but a route from a router is the closest analog.
When configuring routers you are configuring
Level Resolvers which are logger names to levels,
which publisher to use and which appenders should be associated with the publisher.
We call this configuration a "route". As discussed later on
a publisher has appenders and appenders have outputs thus
an important consequnce of this is if you want outputs to have different level thresholds
(e.g. a debug.log and an error.log) you will need multiple routes.
example".
logging.routes=example
logging.route.example.appenders=appender1,appender2
logging.route.example.publisher=async
logging.route.example.level.com.mycompany=DEBUG
Level Resolvers
As mentioned previously routers have a level resolver. A level resolver simply resolves the level (an enum) from a logger name (String). Because ofLevelResolver functional interface we can chain them
with fallbacks. An important fallback level resolver that almost all routers use
is the global level resolver. These levels can be configured with
properties where logger names prefixed with
"logging.level" as the key and the level as the value.
Below is an example:
logging.level.com.mycompany.stuff=DEBUG # Descedant of "com.mycompany"
logging.level.com.mycompany=INFO # Ancestor to "com.mycompany.stuff"
logging.level=ERROR
Although not required most level resolvers including the global follow an inheritance or prefix model where
com.mycompany.stuff.foo will resolve to DEBUG
and com.mycompany.bar will resolve to INFO
and anything not prefixed with com.mycompany will resolve to ERROR.
An analog is to think of the "." as directory path separators like in filesystems
and resolution happens by going up the directories.
This behavior is stated more formally in
Logback's Effective Level
which is what Rainbow Gum follows as well.
Changing Levels at Runtime
Rainbow Gum by default assumes Level Resolvers are cached or static and will never change at runtime. This is a critical feature of Rainbow Gum as it allows facade loggers like in SLF4J to be lower overhead than almost all other logging frameworks! That being said RainbowGum allows levels to be changed at runtime if the backing configuration framework supports reloading and the logger name is allowed to be changed.
Below is an example of configuring to allow "changing" loggerslogging.global.change=true # this is required to turn on level changing.
logging.change.com.mycompany=level # only logger names starting with com.mycompany can have their levels changed.
LogConfig.ChangePublisher
is used and depending on configuration backend may require a manual publish call.
Route Level Priority
A router's level resolver is actually a chain: the route's own level configuration is checked first, and only if the route has no opinion for a particular logger name does resolution fall through to the global level resolver. This means a route's level configuration always takes priority over the global configuration for any logger name the route has configured regardless of whether the global resolver has a deeper/more specific prefix match for that same name.
logging.level.com.mycompany.orders=DEBUG # global: specific override
logging.route.errors.level=ERROR # route "errors": broad override
Without this priority rule a naive merge of the two resolvers into one combined prefix walk would let the
more specific global entry (com.mycompany.orders=DEBUG) win, silently defeating the route's
intent to restrict itself to ERROR and above. Because the route's resolver is consulted
first and independently of the global one, the "errors" route above correctly resolves
com.mycompany.orders to ERROR no matter what the global resolver says about
that name. A route only falls back to the global resolver for logger names it has not configured an
opinion about itself.
Level Groups
Rainbow Gum out of the box supports Spring Boot like Log Groups . However unlike Spring Boot groups are not enabled unless "logging.groups" contains the group. (this is largely because Spring Boots configuration model is like a Map and Rainbow Gums a Function.)Furthermore group levels but not group definitons can be assigned on the router itself. Using Spring Boots example:
logging.groups=tomcat # This is required otherwise the tomcat group will not be resolved
logging.group.tomcat=org.apache.catalina,org.apache.coyote,org.apache.tomcat
logging.level.tomcat=trace
logging.route.myroute.level.tomcat=info
org.apache.catalina, org.apache.coyote, and org.apache.tomcat
and their descedants
will resolve to INFO for "myroute" but TRACE for other routes.
Note that just like logger names groups are case sensitive.
Additivity
Those familiar with Logback and Log4j2/1 "additivity" may wonder how that is achieved. Additivity is associated with a logger name in those frameworks and means that the logger level settings will apply to other appenders associated with logger (name) and its ancestors. If false the settings only apply to the associated appenders at that logger and descendants.In rainbowgum the configuration including logger name to level is done on the route and rainbowgum allows multiple appenders on a route and multiple routes.
For example let us assume we want logger namecom.mycompany (and descendants)
at
DEBUG or greater to go to a file called mycompany.log and
WARN for console com.mycompany (and everything else).
In Logback and others this is achieved by setting the appenders
on com.mycompany and additivity=false but in rainbowgum this is done
by configuring separate routes:
logging.level=INFO
logging.routes=console,mycompany
logging.route.mycompany.appenders=mycompany
logging.route.mycompany.level.com.mycompany=DEBUG
logging.route.console.level=WARN
logging.route.console.appenders=console
logging.appender.mycompany.output=file:///./mycompany.log
logging.appender.console.output=stdout
WARN or greater (ERROR)
to stdout (terminal/console output).
The route my "mycompany" will log DEBUG and greater but less than or equal
of INFO for
com.mycompany (and descedants like com.mycompany.something)
to the file mycompany.log.
IF we only want mycompany.log to have com.mycompany and descendants logger name entries
with DEBUG or greater than we turn off all other logger names.
To do this we add to the above configuration:
logging.route.mycompany.level=OFF
Caller Information
Somewhat related to level resolvers and config change is how to enable caller info.LogEvent.Caller has stack trace like information on where the logging call was made.
It is incredibly slow so like changing loggers it must be enabled and you can enable it with logger name inheritance
so not all loggers will be slow.
Below is an example of configuring to allow "changing" loggers with caller info:
logging.global.change=true # this is required to turn on level changing.
logging.change.com.mycompany=caller # only logger names starting with com.mycompany will have caller info
LogConfig.ChangePublisher.ChangeType however caller info will automatically allow
the logger to change levels as well.
Publishers
ALogPublisher schedules delivery of a LogEvent to a group of
Appenders. Publishers come in two types:
- synchronous - the default
- asynchronous
An example configuring a route named "example" to use the default async publisher:
logging.route.example.publisher=async
logging.publisher.example.bufferSize=1024
NOTE: If you want some appenders to be async and others sync you just create multiple routes.
Rainbow Gum has an experimental async publisher that uses the LMAX Disruptor ringbuffer:io.jstach.rainbowgum.disruptor.
If the jar is found on start it will replace the default async publisher.
Appenders
An appender contains an Encoder and anOutput.
Unlike other logging frameworks there are no custom Appenders as most of the work is done by the encoder and output.
That is an appender can be mostly thought of as a tuple of encoder and output as well as it supervises both.
Appenders are associated to the previously mentioned route. Like previously mentioned when a route is not given a name it is automatically named "default". Thus to register appenders to the default route one can use:
logging.appenders=myappender
logging.appender.myappender.flags=reuse_buffer # an example config of appender
logging.route.example.appenders=myappender
logging.appender.myappender.flags=reuse_buffer # an example config of appender
Appender Configuration
Appenders can be configured either programmatically through thebuilder
or through properties.
- Output
- "logging.appender.{name}.output" = URI
- Encoder
- "logging.appender.{name}.encoder" = URI
- Flags
- "logging.appender.{name}.flags" = List of
LogAppender.AppenderFlag
LogAppender.AppenderFlag.DISABLE_IMMEDIATE_FLUSH which
will tell the output to not flush after each event. Flushing after each event is enabled by default to follow
12 Factors requirements of events written unbuffered synchronously;
this flag should be used to opt out of that for performance reasons.
Appender Reentry Protection
A problem that can occur in any logging framework is if an output, appender, or encoder does logging which if synchronous will usually cause a stackoverflow and if asynchronous an infinite stream of events. Let us create a hypothetical example where we have an Output that uses a message queue client. The message queue client uses SLF4J for logging. If an event is sent that causes the message queue client to log an event and that event is not discarded and the output client then logs the event it created you get reentry in a synchronous publisher. With asynchronous it is far worse as there will be less immediate evidence that something is broken and instead a never ending supply of log entries.
Some logging frameworks like logback provide reentry protection
usually through threadlocals but one must understand this only works if logging is synchronous (synchronous publisher), hurts performance,
and is just a misleading bandaid over a potential serious problem if the events are being dropped (which logback does by default).
That is why Rainbow Gum does not provide reentry protection unless requested!
However the flags LogAppender.AppenderFlag.REENTRY_DROP and LogAppender.AppenderFlag.REENTRY_LOG
can be used to provide similar and arguably better protection than logback.
Overall the real solution is to fix the output so that it filters events that would cause such behavior or ideally not do any logging.
Encoders
Encoders encode an event into binary. Because most encoding is text based RainbowGum has the concept ofLogFormatter which can be turned into an Encoder and is covered
in the next section. However there are scenarios where generating bytes directly is desirable.
Rainbow Gum JSON encoder module io.jstach.rainbowgum.json has efficient JSON encoders.
They are encoders instead of formatters (which are covered next) because writting JSON as bytes is more efficient.
logging.appenders=myappender
logging.appender.myappender.encoder=gelf
logging.encoder.myappender.prettyPrint=true
Encoders should have a builder as well to configure programmatically and the builders javadoc has the string properties that can configure it.
A typical practice is to use a pattern encoder that produces human readable logs for development and a JSON encoder for production switched out based on configuration.JSON Encoders
Rainbow Gum provides several JSON encoders in therainbowgum-json module. Unless prettyPrint is enabled the JSON encoders
included follow the JSON Lines standard as well as of course the JSON standard
RFC-8259. However note that there is no standard on time format in JSON so the formatting of log event
time can vary from JSON encoders depending on the formats specification. The module also can be used to create ones own custom JSON encoders.
The following sections cover the various encoders.
GELF JSON Encoder
GelfEncoderBuilder -
The GELF JSON encoder follows the
GELF JSON specification
with translations of levels that are roughly similar to syslog. Notice that although the GELF specification does not
require JSON Lines this encoder will keep each event on its own line unless
prettyPrint is enabled.
logging.appenders=myappender
logging.appender.myappender.encoder=gelf
logging.encoder.myappender.host=somehost
ECS JSON Encoder
EcsEncoderBuilder -
Elastic Common Schema (ECS)
JSON, URI scheme "ecs". Two shapes are supported:
by default field names are the flattened, dotted ECS names (e.g. "log.level") as the reference
ecs-logging-java implementation writes them; setting
logging.encoder.{name}.structured=true instead nests fields as JSON objects (e.g.
"log":{"level":...}) matching
Spring Boot's ECS structured
logging format. MDC / key values have no reserved ECS field in either shape so, matching both reference
implementations, they are written as top-level fields using their own key name:
logging.appenders=myappender
logging.appender.myappender.encoder=ecs
logging.encoder.myappender.serviceName=myapp
logging.encoder.myappender.structured=true
Logstash JSON Encoder
LogstashEncoderBuilder - format produced by
logstash-logback-encoder's
LogstashEncoder, URI scheme
"logstash". Key values are flattened
top-level fields (matching the reference implementation) and @timestamp is rendered with
an offset using a configurable zone
(defaults to the system default zone):
logging.appenders=myappender
logging.appender.myappender.encoder=logstash
logging.encoder.myappender.zoneId=UTC
Logback JSON Encoder
LogbackJsonEncoderBuilder - resembles
Logback's own opinionated JSON encoder,
URI scheme "logback". Nests key values
under a mdc object and, for throwables, a throwable object with a
stepArray of stack frames (recursing into cause):
logging.appenders=myappender
logging.appender.myappender.encoder=logback
Pattern Encoder
Pattern Encoder is a mostly compatible to logback format encoder that renders logs as text based on a pattern. See Pattern Formatter section andPatternEncoderBuilder
Formatters
Formatting of log events as textual data is so common that Rainbow Gum has special support for that withLogFormatter. While formatters can at high-level be considered
an encoder for technical API reasons they are not. However any formatter can be converted to an
encoder with LogEncoder.of(io.jstach.rainbowgum.LogFormatter).
Pattern Formatter
For those coming from Logback or Log4j2 Rainbow Gum has an optional module for Logback style based pattern formatters. This allows describing an output format usingString.format percent style syntax, e.g.
"%-5level %logger - %msg%n". Rainbow Gum implements most of the builtin Logback pattern keywords with
far less overhead. The builtin keywords are listed in
PatternRegistry.KeywordKey and
PatternRegistry.ColorKey.
The pattern module is pulled in by default with the aggregate rainbow gum dependency and is
already the default encoder for console output.
NOTE: The color pattern keywords work out of the box with no extra module.
Rainbow Gum detects whether the process is attached to an ANSI capable terminal using only
System.console() and well known environment variables (NO_COLOR, TERM) - see
AnsiSupport - and simply does not emit ANSI escape sequences when it is not.
The separate JAnsi module covered below is no longer generally needed; it is now largely a
legacy option for a handful of edge cases. ANSI escape can also be globally disabled by setting
"logging.global.ansi.disable" to true, which disables
both the default pattern formatter's ANSI output and JAnsi if that module is in use.
Selecting and Configuring the Pattern Encoder
The pattern encoder's URI scheme is "pattern". Because it is already the default for console output, setting a pattern on an appender is often all that is needed:logging.appenders=myappender
logging.appender.myappender.encoder=pattern
logging.encoder.myappender.pattern=[%thread] %-5level %logger{15} - %msg%n
true.
Configuring PatternConfig with Properties
PatternConfig carries the platform specific settings that keywords
need: time zone (for %d/%date), line separator (for %n), whether ANSI is disabled, and
the start time used by %r/%relative. It is resolved per encoder name through
"logging.pattern.config.{name}.", so different appenders can
each have their own:
logging.appender.myappender.encoder=pattern
logging.pattern.config.myappender.zoneId=UTC
logging.pattern.config.myappender.lineSeparator=\n
logging.pattern.config.myappender.ansiDisabled=true
%property{key} is different: it looks up ad-hoc key values that are not tied to a particular
encoder, through "logging.pattern.property.":
logging.pattern.property.app=myapp
%property{app} anywhere in a pattern outputs myapp.
Custom Keywords and Programmatic Configuration
Like Logback you can add your own pattern keywords, andPatternConfig
can instead be set programmatically as a single default shared by every encoder that has no more specific
property configuration. Both - with full code examples - are covered in the
io.jstach.rainbowgum.pattern module javadoc.
Automatic Exception Rendering
Notice that none of the pattern examples above (e.g.[%thread] %-5level %logger{15} - %msg%n)
mention an exception keyword at all. As with Logback, this is intentional and safe: if a pattern does not
otherwise use %ex/%xEx (or their aliases), Rainbow Gum automatically appends a default
exception formatter to the end of the compiled pattern, so a logged throwable is never silently dropped just
because the pattern forgot to ask for it.
To opt out and suppress exception rendering entirely, add %nopex (or %nopexception)
anywhere in the pattern - like the other throwable keywords, its mere presence is enough to disable the
automatic append, and it renders nothing itself:
logging.encoder.myappender.pattern=%msg%nopex
JAnsi support (largely legacy)
Rainbow Gum still has an optional JAnsi module for cross platform ANSI
output, but it is essentially deprecated and should not be needed by most users. It predates
AnsiSupport, the JDK-native ANSI detection the pattern
module now uses by default, and its original reasons for existing have mostly gone away:
- Windows. JAnsi's original purpose was translating ANSI escapes into Win32 console calls for the legacy Windows console, which did not understand ANSI at all. Modern Windows terminals (Windows 10+, Windows Terminal) support ANSI/VT sequences natively, so this is largely a non-issue today.
-
Stripping ANSI on piping/redirection. JAnsi works by wrapping
System.out/System.errand stripping escape sequences at write time when it decides the destination is not a real terminal - which requires JAnsi to correctly detect that in the first place.AnsiSupportinstead decides once, usingSystem.console(), whether to emit ANSI escapes at all, which is both simpler and does not depend on a terminal-detection library getting it right. This matters because terminal detection is a genuinely hard problem in practice: for example JLine's terminal detection - used by a number of other tools for the same purpose - is known to not correctly detect piped/redirected output in some cases, emitting escape codes into a pipe it should have recognized as non-interactive.
NOTE:
Future versions of the JDK may require command line arguments
for using jars packaged with native libraries, which JAnsi is. Given that Rainbow Gum favors security we
strongly agree with the draft JEP - this is one more reason AnsiSupport's pure-JDK approach (no
native code, no reflection, no separate terminal library) is now the preferred path.
A further reason to avoid Jansi is that it may accidentally strip ANSI escape characters for terminals that do support ANSI but Jansi cannot determine they do. The most notable case of this problem are IDE terminals/consoles particularly IntelliJ's.
The module is kept around for the remaining edge cases where its Win32 console translation is still relevant (e.g. genuinely legacy Windows environments), but new users should not need to add it.
Outputs
LogOutput do the actual work of outputting an event.
Configuring an appenders output with properties
looks something like:
logging.appenders=myappender
logging.appender.myappender.output=stdout
./.
logging.appenders=myappender
# logging.appender.myappender.output=app.log this is wrong.
logging.appender.myappender.output=./app.log
app.log is a URI scheme if it is not prefixed.
Of course fully qualified file URI are supported as well like: file:///./app.log.
For Spring Boot compatibility "logging.file.name" is also supported
if custom routers and appenders are not configured. Unlike the generic output property above,
"logging.file.name" is not subject to the ./
caveat - since it is documented to always be a file path (never a URI scheme reference) Rainbow Gum resolves
it as one directly, so both of the following work as expected without needing a ./ prefix:
logging.file.name=app.log
logging.file.name=logs/app.log
file:///./app.log is still honored as-is.
Here are some of the supported outputs (not a complete listing):
- stdout and stderr
FileOutputBuilderwith URI scheme of "file"
Rolling Files
Rainbow Gum currently does not support rolling of files on its own but does provide a mechanism to safely allow external programs such as logrotate to do the rolling. How this typically works:- External program moves the current log file. Rainbow Gum will continue to log to the same file but the file name is effectively changed.
- Signals to the Java process with Rainbow Gum to reopen the files. This signal is usually done via HTTP or TCP socket since Java does not support Unix signals.
- The external program then typically compresses, delete or send the log files elsewhere.
(The other method of rotating files without doing a move and reopen is called "copy truncate" but has the potential of losing events. This method avoids that problem. )
Unfortunately Rainbow Gum does not offer a way to receive the external signal as this can vary greatly across applications. The following is an example of reopening outputs using HTTP for signaling./*
* This method could be bound to an internal HTTP route say /log/rotate such that curl
* http://localhost:8080/log/rotate will block and wait till all the applicable
* outputs have reopened.
*/
@RequestMapping("/log/rotate")
@ResponseBody
public String someInternalHttpRequestHandler() {
var gum = RainbowGum.getOrNull();
if (gum != null) {
List<LogResponse> response = gum.config() //
.outputRegistry() //
.reopen(); // Here is where we siginal to reopen outputs that support
// reopening.
return response.toString();
}
return "";
}
http://localhost:8080/log/rotate and we have configured our logging like
logging.appenders=myappender
logging.appender.myappender.output=/var/log/app.log
/var/log/app.log
{
rotate 4
weekly
missingok
notifempty
compress
delaycompress
# we only need one call to reopen all files
sharedscripts
# in most cases it is best for rainbowgum recreate the file which wil happen after the postrotate
nocreate
postrotate
/usr/bin/curl -s -o /dev/null -w "%{http_code}" http://localhost:8080/log/rotate | {
read status
if [ "$status" -ne 200 ]; then
logger "logrotate: Error hitting endpoint, received HTTP status $status"
fi
}
endscript
}
Filtering
Like logback Rainbow Gum provides two types of filtering through functional composition:- router based filtering - similar to logbacks normal filters.
- SLF4J based filtering - similar to logback Turbo Filters
Event based Filtering
Unfortunately at this time router based filtering is not configurable through properties and can only be added using Rainbow Gum builder and composition has to be done manually.An example using router based filtering:
RainbowGum.builder(config) //
.route(rb -> {
rb.factory(RouterFactory.of(e -> {
/*
* We only log DEBUG level events.
*/
return switch (e.level()) {
case DEBUG -> e;
default -> null;
};
}));
/*
* If we do not set the level correctly the router will never get the
* event regardless of the logic of the above filtering function.
*/
rb.level(Level.DEBUG);
});
LogRouter.Router
with a function.
Event based filtering is still based on the invariants
of level resolvers not changing (unless they are configured to be dynamic) so
not all events are delivered if the level resolved does not not allow it!
A work around is to adjust the levels of the router so that it does get
all events desired and is why filtering is associated with routing.
Unfortunately this can be inefficient as the event always has to be created
so if performance matters consider using SLF4J based filtering.
SLF4J based filtering
SLF4J based filtering is not simple filtering but rather decorating of the SLF4J loggers somewhat akin to Servlet filtering. Rainbow Gum hands off its generated SLF4J logger to you and you can decorate or even disregard it and provide your own implementation. This is extremely powerful and efficient but requires more work. There is not really an analog in Logback or Log4j but as usual with great power comes great responsibility!These filters are usually loaded with the ServiceLoader so manually using the Rainbow Gum builder is not required like router based filtering.
Do not implement the decorated Logger by extending
AbstractLogger directly - getting SLF4J caller-info depth right
means depending on that class's internal dispatch shape, which is an unversioned
implementation detail and easy to get subtly wrong. Extend
AbstractFilteringLogger
instead, which owns and tests its own depth accounting. A decorator that samples DEBUG
level events and surfaces any Marker as a key value (there is no
built-in Marker support otherwise):
public class DecoratorExample extends LoggerDecoratorService {
@Override
public String name() {
return "sampling";
}
@Override
public Logger decorate(RainbowGum rainbowGum, DepthAwareLogger previousLogger, int depth) {
return new SamplingLogger(previousLogger);
}
static class SamplingLogger extends AbstractFilteringLogger {
private final AtomicInteger debugCount = new AtomicInteger();
SamplingLogger(DepthAwareLogger delegate) {
super(delegate);
}
@Override
protected boolean isEnabled(Level level, @Nullable Marker marker) {
// Cheap pre-check: keep only every 10th DEBUG event, everything else
// passes through unchanged. This runs before any LoggingEventBuilder
// is built.
if (level != Level.DEBUG) {
return true;
}
return debugCount.incrementAndGet() % 10 == 0;
}
@Override
protected boolean decorate(LoggingEventBuilder builder, @Nullable Marker marker) {
if (marker != null) {
// Rainbow Gum core has no built-in Marker support (it is not stored
// on the event), so surface it as a key value instead.
builder.addKeyValue("marker", marker.toString());
}
return true;
}
}
}
Example Configuration
Everything above introduced routes, level resolvers, publishers, appenders, encoders, and outputs one at a time. This section puts them together into one non-trivial configuration, first as properties and then the equivalent using builders, so the full shape of a realistic setup is visible in one place rather than spread across many small snippets.
Assume a hypothetical service com.mycompany.orders that wants:
- A human readable, synchronous console appender for local development.
- A structured (JSON) file appender for log aggregation, using an async publisher so encoding and file IO do not block request threads.
- A separate
errorsonly file, independent of the two routes above, following the multiple routes pattern discussed earlier. - Noisy third party packages (
org.apache.http,io.netty,com.zaxxer.hikari) turned down toWARNeverywhere via a level group.
Properties
logging.level=INFO
logging.level.com.mycompany.orders=DEBUG
logging.groups=noisy
logging.group.noisy=org.apache.http,io.netty,com.zaxxer.hikari
logging.level.noisy=WARN
logging.routes=console,structured,errors
logging.route.console.appenders=console
logging.route.console.level=INFO
logging.route.structured.appenders=jsonfile
logging.route.structured.publisher=async
logging.route.structured.level=DEBUG
logging.route.errors.appenders=errorfile
logging.route.errors.level=ERROR
logging.route.errors.level.com.mycompany.orders=ERROR
logging.publisher.structured.bufferSize=2048
logging.appender.console.output=stdout
logging.appender.console.encoder=pattern
logging.appender.jsonfile.output=./logs/orders.json.log
logging.appender.jsonfile.encoder=gelf
logging.appender.jsonfile.flags=reuse_buffer
logging.appender.errorfile.output=./logs/orders-error.log?bufferSize=10000
logging.appender.errorfile.encoder=pattern
logging.encoder.console.pattern=[%thread] %-5level %logger{36} - %msg%n
logging.encoder.jsonfile.host=orders-service
logging.encoder.jsonfile.prettyPrint=false
logging.encoder.errorfile.pattern=%d{ISO8601} [%thread] %-5level %logger{36} - %msg%n%ex
Encoder configuration is keyed by appender name, but lives under a completely
different property prefix than the appender itself. The console appender's
own settings are under logging.appender.console.*, but its encoder's settings are under
logging.encoder.console.* — a sibling prefix, not a child of the appender's.
The same is true for jsonfile and errorfile. Conceptually the encoder
belongs to the appender; in the property key space it is just another top level component that
happens to share the appender's name.
A route level threshold does not act as a hard ceiling — it merges with the global
level resolver by prefix specificity, not by route priority. Without the line
logging.route.errors.level.com.mycompany.orders=ERROR, the errors route
would still emit DEBUG and INFO events for
com.mycompany.orders despite logging.route.errors.level=ERROR, because
the global logging.level.com.mycompany.orders=DEBUG is a more specific
prefix match than the route's bare (root) ERROR setting, and specificity wins
regardless of which resolver (route or global) the match came from. To make a route strictly
more restrictive than the global baseline for a given logger name prefix, that same
prefix has to be overridden again at the route level — setting a coarser route level alone is
not enough once a more specific global override exists for names under it.
A property value that is itself a URI can carry its own query parameters, as an
alternative to separate logging.<component>.<name>.<param>
properties. logging.appender.errorfile.output above is set to
./logs/orders-error.log?bufferSize=10000 rather than a bare path: the part before
? is still resolved as the file path exactly as with a bare path, and everything
after ? is parsed the same way a query string would be and made available as
properties scoped to that output — here, bufferSize, equivalent to also having
written logging.output.errorfile.bufferSize=10000 on its own line. This works for a
relative, scheme-less path just as well as it does for an explicit file:// URI; which
style to use is purely a matter of preference for how compact versus explicit the properties file
should read.
Builder
The same configuration (minus the JSON encoder, see the note below) using builders:RainbowGum.builder(config) //
.route("console", r -> {
r.level(Level.INFO);
r.level(Level.WARNING, "org.apache.http");
r.level(Level.WARNING, "io.netty");
r.level(Level.WARNING, "com.zaxxer.hikari");
r.appender("console", a -> {
a.output(LogOutput.ofStandardOut());
a.encoder(new PatternEncoderBuilder("console").pattern("[%thread] %-5level %logger{36} - %msg%n")
.fromProperties(config.properties())
.build());
});
}) //
.route("structured", r -> {
r.level(Level.DEBUG, "com.mycompany.orders");
r.publisher(PublisherFactory.async().bufferSize(2048).build());
r.appender("detailfile", a -> {
/*
* A URI-scheme based encoder like the JSON module's "gelf" is
* resolved through the service loader, which is why it is comfortable
* to reference from properties (see the properties example above)
* without the aggregate rainbowgum module needing to require the JSON
* module at compile time. Referencing an optional module's encoder
* class directly the way this builder is written would require adding
* that module as an explicit dependency.
*/
a.output(new FileOutputBuilder("detailfile").fileName("./logs/orders-detail.log").build());
a.encoder(new PatternEncoderBuilder("detailfile")
.pattern("%d{ISO8601} [%thread] %-5level %logger{50} - %msg%n")
.fromProperties(config.properties())
.build());
a.flags(EnumSet.of(AppenderFlag.REUSE_BUFFER));
});
}) //
.route("errors", r -> {
r.level(Level.ERROR);
r.appender("errorfile", a -> {
// Appenders flush after every event by default (see
// AppenderFlag#DISABLE_IMMEDIATE_FLUSH), so no flag is needed
// here to get synchronous, unbuffered error writes.
a.output(new FileOutputBuilder("errorfile").fileName("./logs/orders-error.log").build());
a.encoder(new PatternEncoderBuilder("errorfile")
.pattern("%d{ISO8601} [%thread] %-5level %logger{36} - %msg%n%ex")
.fromProperties(config.properties())
.build());
});
});
Notice the structured route above uses a plain text pattern encoder instead of the
JSON module's gelf encoder used in the properties version. A URI scheme based encoder
like gelf is resolved through the service loader at runtime, which is precisely why it
is comfortable to reference from properties without the aggregate rainbowgum module
needing to requires the JSON module at compile time. Referencing
GelfEncoder directly from a builder the way the rest of
this example is written would require adding rainbowgum-json as an explicit compile
dependency of whatever module the builder code lives in — illustrating a real trade-off between
the two configuration styles rather than only a stylistic one.
Installation
For most simply including the dependencyio.jstach.rainbowgum:rainbowgum
is enough. That dependency will transitively pull in the most commonly used modules.
For a more minimal setup (total application size in terms of classes)
io.jstach.rainbowgum:rainbowgum-core
and io.jstach.rainbowgum:rainbowgum-slf4j
(if using SLF4J)
should be used.
Maven
<properties>
<rainbowgum.version>0.10.0</rainbowgum.version>
</properties>
...
<dependencies>
<dependency>
<groupId>io.jstach.rainbowgum</groupId>
<artifactId>rainbowgum</artifactId>
<version>${rainbowgum.version}</version>
<scope>runtime</scope>
</dependency>
</dependencies>
<properties>
<rainbowgum.version>0.10.0</rainbowgum.version>
</properties>
...
<dependencies>
<dependency>
<groupId>io.jstach.rainbowgum</groupId>
<artifactId>rainbowgum-core</artifactId>
<version>${rainbowgum.version}</version>
<scope>compile</scope>
</dependency>
</dependencies>
Gradle
dependencies {
runtimeOnly 'io.jstach.rainbowgum:rainbowgum:0.10.0'
}
If you plan on configuring Rainbow Gum programmatically you will need to make a module and create a
service loader registration. In that case you will want the dependency like:
dependencies {
implementation 'io.jstach.rainbowgum:rainbowgum-core:0.10.0'
}
Extensions and Integrations
SLF4J 2.0
Rainbow Gum SLF4J moduleio.jstach.rainbowgum.slf4j supports:
There is no built-in Marker support (it is not stored on the event
or used for routing/filtering by default), but a
decorator can
observe markers and act on them, e.g. surface one as a key value or filter on it.
Rainbow Gum supports key value pairs
in LoggingEventBuilder by overlaying on top
of the current MDC at the time the event is constructed and put into the
events key values.
The value parameter in
LoggingEventBuilder.addKeyValue(String,Object) are
converted to String immediately as only String values are
supported at this time.
Rainbow Gum SLF4J implementation is unique in that it has two special implementation of loggers:
- Level Logger - logger based on level threshold and can never change!
- Changing Logger - level and other configuration can change.
java.lang.System.Logger and java.util.logging
Rainbow Gumio.jstach.rainbowgum.jdk module provides special integration and adapters for the builtin JDK logging facilities.
The impetus for this is these logging facilities can be used early in the JDK boot processes well before
logging has fully initialized. The default rainbow gum dependency has this integration as a transitive dependency.
The integration will make sure that neither the System.Logger or java.util.logging initialize
Rainbow Gum too early by queueing the events if some other facade is detected like SLF4J. When a Rainbow Gum initializes and
set as global the events will be replayed.
If the events level are equal to System.Logger.Level.ERROR and a normal Rainbow Gum has not been bound
the messages will be printed to System.err.
The idea is something catastrophic has happened that will probably cause Rainbow Gum to never load and thus never replay the events
and you will not be able to figure out what happened otherwise.
The exception of the above is if no other facade is detected
the module will initialize Rainbow Gum. The idea is that your application only uses the JDK logging facilities
(the Rainbow Gum SLF4J implementation is not found in the module/classpath). Thus only having the dependencies
rainbowgum-core, and rainbowgum-jdk will work as normal.
SLF4J does provide an adapter/bridge for the System.Logger (org.slf4j:slf4j-jdk-platform-logging) but its use may cause Rainbow Gum to initialize too early. However that maybe desirable if:
- You are sure that Rainbow Gum can initialize early
- Your application uses System.Logger
(the SLF4J adapter will initialize Rainbow Gum on System.Logger usage if using
io.jstach.rainbowgum.slf4j)
The following System properties allow greater control of the queueing and initialization. Because of how early initialization can happen this configuration can only be done with System properties.
- "logging.global.queue.level" = level
- "logging.global.queue.error" = level
- "logging.systemlogger.initialize" = init option
NOTE: While the JDK System.Logger is good for low level libraries that rarely log it's API (and Rainbow Gum implementation) is not designed for performance. For applications and frameworks that do a lot of logging the SLF4J facade is the preferred choice.
Because java.util.logging (JUL) and System.Logger can be used very early on currently MDC (set by SLF4J) and caller info diagnostics is not provided at this time. Caller info maybe provided later as it is agnostic of SLF4J and if you would like that please file an issue. Support for changeable logging levels is provided for the System Logger provided the loggers are created after Rainbow Gum initializes.
Spring Boot
Rainbow Gum provides Spring Boot 3 and 4 support that is maintained by the Rainbow Gum project, as two separate modules/artifacts - one per Spring Boot major version. Rainbow Gum cannot compete with Spring Boot's own bundled Logback/Log4j2 integration on "we maintain this forever regardless of what Spring Boot does next", so rather than bet on one major version staying source-compatible with the next (they usually are not), each supported major gets its own artifact:io.jstach.rainbowgum.spring.boot4/rainbowgum-spring-boot4-starter- for Spring Boot 4 (recommended for new projects).io.jstach.rainbowgum.spring.boot3/rainbowgum-spring-boot3-starter- for Spring Boot 3.
io.jstach.rainbowgum.spring.boot3/boot4) are not
exported and never imported by consumers - Spring Boot discovers them itself through META-INF/spring.factories
and the Java service loader - so those packages are duplicated per major version rather than shared, and are not
necessarily identical in behavior; each module's own javadoc has the authoritative full supported/not-supported
property table for which Spring Boot logging properties (logging.*, spring.output.ansi.enabled,
etc.) it supports, kept next to the code that implements it rather than duplicated here where it could drift out of
sync -
io.jstach.rainbowgum.spring.boot4 and io.jstach.rainbowgum.spring.boot3. The one package you might
actually import,
io.jstach.rainbowgum.spring.boot.spi, lives in its own genuinely shared module (rainbowgum-spring-spi,
pulled in transitively by both starters) because it only depends on org.springframework.core.env.Environment,
which is expected to remain stable across Spring Boot major versions - so if you have implemented
SpringRainbowGumServiceProvider yourself, that import never needs to change when you bump major versions.
Spring Boot allows configuration through simple properties for logging that tries to be logging framework agnostic. How this works is Spring Boot configures the logging framework from its own set of properties. There is a shocking amount of code Spring Boot requires to do this for Logback and Log4J2 because both of those frameworks were not designed well for programmatic configuration.
Rainbow Gum key value configuration is purposely very similar and largely compatible with
Spring Boot's however Spring Boot still requires special integration if you would like
to use its configuration system (for example application.properties).
This is because Spring Boot uses/needs logging for initialization of its configuration system.
<dependencies>
<dependency>
<groupId>io.jstach.rainbowgum</groupId>
<artifactId>rainbowgum-spring-boot4-starter</artifactId>
<version>0.10.0</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<!--
This is the important part as logback will be used otherwise
regardless if rainbow gum spring boot integration is used.
-->
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
rainbowgum-spring-boot4-starter with
rainbowgum-spring-boot3-starter above - everything else is identical.
Note that if one chooses to not use one of the rainbowgum-spring-boot*-starter artifacts
or the underlying rainbowgum-spring-boot3/rainbowgum-spring-boot4 module directly,
Spring will re-initialize java.util.logging and take over it so again the integration is recommended.
JBoss Logging
JBoss Logging is sadly another logging facade. It is supported by forcing JBoss to use its builtin SLF4J bridge which should use Rainbow Gum's SLF4J implementation. When Rainbow Gum initializes its SLF4J implementation the following will happen to encourage JBoss Logging to use SLF4J.if (System.getProperty("org.jboss.logging.provider") == null) {
System.setProperty("org.jboss.logging.provider", "slf4j");
}
Commons Logging (jcl)
Commons Logging support can be had by using:- jcl-over-slf4j - if not using Spring, or using Spring Framework 6 / Spring Boot 3 (or earlier) and one does not want
spring-jcl. - spring-jcl - if using Spring Framework 6 / Spring Boot 3 (or earlier).
spring-jcl. That being said
if one does not want Spring "sniffing" around by reflection and not using Spring Boot jcl-over-slf4j is preferred
but requires excluding spring-jcl.
Spring Framework 7 / Spring Boot 4 changed this: spring-jcl was removed and Spring went back to depending
directly on the real Apache Commons Logging (1.3+), which as of
that version added its own built-in SLF4J discovery. So with Rainbow Gum's Spring Boot 4 integration, commons-logging
calls are routed to SLF4J (and therefore Rainbow Gum) automatically by commons-logging itself - no jcl-over-slf4j
substitution or spring-jcl/commons-logging exclusion dance needed.
For anything other than Spring Framework 7 / Spring Boot 4, the original commons-logging should generally not be used and should be excluded from dependencies that need it.
Extension development guide
To a develop a custom plugin it is probably easiest to look at the other rainbow gum modules. An important concept with extensions are the registries which allow you to register a provider with a URI scheme.LogConfig contains all the registries
and a special generic registry: ServiceRegistry.
The service registry is allows other plugins to share components.
To access these registries at configuration time as a plugin is to implement
a RainbowGumServiceProvider.Configurator.
Next one should construct builders that are properties friendly to create the components
so that programmatic configuration users can use your extension. Rainbow Gum provides
an annotation processor that helps create builders
io.jstach.rainbowgum.annotation.
<properties>
<rainbowgum.version>0.10.0</rainbowgum.version>
</properties>
...
<dependencies>
<dependency>
<groupId>io.jstach.rainbowgum</groupId>
<artifactId>rainbowgum-core</artifactId>
<version>${rainbowgum.version}</version>
<scope>compile</scope>
</dependency>
<!-- This is the annotation processor -->
<dependency>
<groupId>io.jstach.rainbowgum</groupId>
<artifactId>rainbowgum-apt</artifactId>
<version>${rainbowgum.version}</version>
<!-- the following maven config is important as the annotation processor should never be a transitive dep -->
<scope>provided</scope>
<optional>true</optional>
</dependency>
</dependencies>
LogConfigurable.
FAQ
There is a typo in this documentation how can I fix it?
If you would like to make corrections please file an issue or even better fork, edit, PR this file:doc/overview.html
Where is the Javadoc?
Shockingly this document is the Javadoc! To be precise it is the aggregate javadocoverview.html.
The modules javadocs should be at the bottom of this document and the search bar at the top can be
used to find documented classes.
java.util.logging) integration.io.jstach.rainbowgum.spring.boot3 and
io.jstach.rainbowgum.spring.boot4.