Spring Boot Centralized Logging with Graylog
In this article, I’ll explain how to centralize the logs of your Spring Boot application with Graylog.
Join the DZone community and get the full member experience.
Join For FreeLogs are an essential part of any system, they give you detailed information about your application, what your system is doing, and what caused the error if something went wrong.
Almost all systems generate logs in one form or another, these logs are written to files on local drives. Debugging the error in a production application through hundreds of log files and on hundreds of servers can be very time consuming and complicated. One approach to this problem is to create a centralized log management system that can collect and aggregate different types of logs in one location.
What is Graylog?
Graylog is a powerful platform that allows for easy log management of both structured and unstructured data along with debugging applications. It is based on Elasticsearch, MongoDB, and Scala. Graylog has a main server, which receives data from its clients installed on different servers, and a web interface, which visualizes the data and allows to work with logs aggregated by the main server.
Setup Graylog
Many installation methods are recommended in official documents. Here we are using docker compose to easily setup a graylog instance with data persistent support.
XML
xxxxxxxxxx
1
58
1
version: '2'
2
services:
3
# MongoDB: https://hub.docker.com/_/mongo/
4
mongodb:
5
image: mongo:3
6
volumes:
7
- mongo_data:/data/db
8
# Elasticsearch: https://www.elastic.co/guide/en/elasticsearch/reference/6.x/docker.html
9
elasticsearch:
10
image: docker.elastic.co/elasticsearch/elasticsearch-oss:6.8.2
11
volumes:
12
- es_data:/usr/share/elasticsearch/data
13
environment:
14
- http.host=0.0.0.0
15
- transport.host=localhost
16
- network.host=0.0.0.0
17
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
18
ulimits:
19
memlock:
20
soft: -1
21
hard: -1
22
mem_limit: 1g
23
# Graylog: https://hub.docker.com/r/graylog/graylog/
24
graylog:
25
image: graylog/graylog:3.1
26
volumes:
27
- graylog_journal:/usr/share/graylog/data/journal
28
environment:
29
# CHANGE ME (must be at least 16 characters)!
30
- GRAYLOG_PASSWORD_SECRET=somepasswordpepper
31
# Password: admin
32
- GRAYLOG_ROOT_PASSWORD_SHA2=8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
33
- GRAYLOG_HTTP_EXTERNAL_URI=http://127.0.0.1:9000/
34
links:
35
- mongodb:mongo
36
- elasticsearch
37
depends_on:
38
- mongodb
39
- elasticsearch
40
ports:
41
# Graylog web interface and REST API
42
- 9000:9000
43
# Syslog TCP
44
- 1514:1514
45
# Syslog UDP
46
- 1514:1514/udp
47
# GELF TCP
48
- 12201:12201
49
# GELF UDP
50
- 12201:12201/udp
51
# Volumes for persisting data, see https://docs.docker.com/engine/admin/volumes/volumes/
52
volumes:
53
mongo_data:
54
driver: local
55
es_data:
56
driver: local
57
graylog_journal:
58
driver: local
Start the graylog instance with docker-compose up -d
Sending Application Logs (Spring Boot Application)
After creating the Spring Boot application, add a logback-gelf dependency in the pom file.
XML
xxxxxxxxxx
1
1
<dependency>
2
<groupId>de.siegmar</groupId>
3
<artifactId>logback-gelf</artifactId>
4
<version>2.0.0</version>
5
</dependency>
We used Logback-gelf which is used here to generate logs in Graylog. There is a detailed introduction to using logback-gelf on here.
- logback.xml file
XML
xxxxxxxxxx
1
52
1
<configuration>
2
3
4
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
5
<encoder>
6
<pattern>%green(%date) %highlight(%-5level) %yellow([%-4relative]) %magenta([%thread]) %cyan(%logger{10}) %gray([%file:%line]) %blue(: %msg%n)</pattern>
7
<charset>UTF-8</charset>
8
</encoder>
9
</appender>
10
11
<appender name="GELF" class="de.siegmar.logbackgelf.GelfUdpAppender">
12
<graylogHost>127.0.0.1</graylogHost>
13
<graylogPort>12201</graylogPort>
14
<maxChunkSize>508</maxChunkSize>
15
<useCompression>true</useCompression>
16
<encoder class="de.siegmar.logbackgelf.GelfEncoder">
17
<originHost>127.0.0.1</originHost>
18
<includeRawMessage>false</includeRawMessage>
19
<includeMarker>true</includeMarker>
20
<includeMdcData>true</includeMdcData>
21
<includeCallerData>false</includeCallerData>
22
<includeRootCauseData>false</includeRootCauseData>
23
<includeLevelName>false</includeLevelName>
24
<shortPatternLayout class="ch.qos.logback.classic.PatternLayout">
25
<pattern>%m%nopex</pattern>
26
</shortPatternLayout>
27
<fullPatternLayout class="ch.qos.logback.classic.PatternLayout">
28
<pattern>%m%n</pattern>
29
</fullPatternLayout>
30
<staticField>app_name:backend</staticField>
31
<staticField>os_arch:${os.arch}</staticField>
32
<staticField>os_name:${os.name}</staticField>
33
<staticField>os_version:${os.version}</staticField>
34
</encoder>
35
</appender>
36
37
<root level="INFO">
38
<appender-ref ref="STDOUT" />
39
</root>
40
41
<logger name="com.logging.springboot2graylog">
42
<appender-ref ref="MAIN_LOG_FILE" />
43
<appender-ref ref="GELF" />
44
</logger>
45
46
<logger name="com.logging.springboot2graylog.interceptor.RestControllerInterceptor" additivity="false">
47
<level value="DEBUG"/>
48
<appender-ref ref="GELF" />
49
</logger>
50
51
52
</configuration>
Receiving Logs (Graylog Server)
Access to the Graylog server in the browser http://ip:9000.
To send the data to Graylog, you must, therefore, configure an entry. This will tell Graylog to accept the log messages.
Go to the Web interface -> System -> Entries and select "GELF UDP" and click on "Launch a new entry".
Run your Spring Boot application. you can see the captured logs in the Graylog interface.
You can find the complete source code in my Github page.
Topics:
spring boot 2.2,
java,
docker,
graylog
Opinions expressed by DZone contributors are their own.
Comments