DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones AWS Cloud
by AWS Developer Relations
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones
AWS Cloud
by AWS Developer Relations
  1. DZone
  2. Data Engineering
  3. Data
  4. Upgrade Guide To Spring Data Elasticsearch 5.0

Upgrade Guide To Spring Data Elasticsearch 5.0

Learn about the latest Spring Data Elasticsearch 5.0.1 with Elasticsearch 8.5.3, starting with the proper configuration of the Elasticsearch Docker image.

Arnošt Havelka user avatar by
Arnošt Havelka
CORE ·
Jan. 26, 23 · Tutorial
Like (2)
Save
Tweet
Share
6.63K Views

Join the DZone community and get the full member experience.

Join For Free

Recently, I wrote the article, "Pagination With Spring Data Elasticsearch 4.4," but we already have new Spring Data Elasticsearch 5.0 and Spring Boot 3.0 releases. There are several changes and issues we should consider when doing the upgrade.

The goal of this article is to highlight changes when upgrading the sat-elk project with these technologies:

  1. Spring Data Elasticsearch 5.0.1
  2. Spring Boot 3.0.2
  3. Elasticsearch 8.5.3

Key Changes

  • Elasticsearch 8.5 has enabled SSL by default.
  • Spring Data Elasticsearch 5.0 uses a new Elasticsearch client library.
  • Spring Boot 3.0 moved from javax to jakarta package (dependencies).
  • SearchHits behaves a little bit differently.

In This Article, You Will Learn

  • How to set up Elasticsearch 8.5.3
  • How to configure Spring Data Elasticsearch 5.0 in a project
  • How to deal with upgrading to the latest technologies

Elasticsearch

Our goal is to have an application to manage data via the Spring Data Elasticsearch in Elasticsearch. You can find more details in my initial article "Introduction to Spring Data Elasticsearch 4.1." Here, you can find only the basic steps with the highlighted differences or changes.

Let's check the changes first.

Changes

The last article used Spring Data Elasticsearch in version 4.4, but the latest version (at the time of writing this article) is version 5.0.1. You can find all the details here.

We should keep in mind the compatibility matrix that contains the compatible versions of the main technologies -> Spring framework, Spring Boot, Spring Data Release Train, Spring Data Elasticsearch, and of course, Elasticsearch itself.

Our versions are driven by Spring Boot 3.0.2 and Spring Data Release Train 2022.0.1.

Docker Setup

The first significant change in the used  elasticsearch Docker image is the switch to HTTPS by default instead of the previous HTTP. The differences in Elasticsearch configuration (located in /usr/share/elasticsearch/config/elasticsearch.yml file) in these versions are:

Elasticsearch 7.17.8:

YAML
 
cluster.name: "docker-cluster"
network.host: 0.0.0.0


Elasticsearch 8.5.3:

YAML
 
cluster.name: "docker-cluster"
network.host: 0.0.0.0

# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
#----------------------- END SECURITY AUTO CONFIGURATION -------------------------


The previous tip (see the section"Disable XPack in Elasticsearch" in my previous article) doesn't work now, probably because it was a kind of workaround. Therefore, we should disable X-Pack security properly.

Let's follow these steps to set up Elasticsearch correctly.

Custom Network

Shell
 
docker network create sat-elk-net


Elasticsearch

Shell
 
docker run -d --name sat-elasticsearch --net sat-elk-net -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" -e "xpack.security.enabled=false" elasticsearch:8.5.3


Note: See the -e "xpack.security.enabled=false" extra parameter for disabling HTTPS.

ElasticHQ

Shell
 
docker run -d --name sat-elastichq --net sat-elk-net -p 5000:5000 elastichq/elasticsearch-hq


Spring Data Elasticsearch

All upgrade notes/hints for version 5.0.x can be found here.

Maven Dependency

We use the spring-boot-starter-data-elasticsearch dependency in our Maven project (pom.xml) as demonstrated below. We can find the latest available version in the Maven Central repository.

XML
 
<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
	<version>3.0.2</version>
</dependency>


Additionally, our code also depends on the Spring MVC (for exposing REST endpoints) and the jackson.dataformat CSV module (for reading CSV files). We should add them to our project as: 

XML
 
<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-web</artifactId>
	<version>3.0.2</version>
</dependency>
<dependency>
	<groupId>com.fasterxml.jackson.dataformat</groupId>
	<artifactId>jackson-dataformat-csv</artifactId>
	<version>2.14.1</version>
</dependency>


Elasticsearch client

A second key change (after the Elasticsearch Docker image configuration) represents a new Elasticsearch client used by Spring Data Elasticsearch. So far we used configuration spring.elasticsearch.uris in application.properties file, but it doesn't work now.

We can still re-use it with a small adjustment as:

YAML
 
spring:
  elasticsearch:
    rest:
      uris: oxygen-arnost.ifs.dev.fra.aws.dbgcloud.io:9200


Note: Only the leading protocol was removed.

For the new Elasticsearch client, we need to add a ElasticsearchClientConfig class where we use this configuration property as:

Java
 
@Configuration
public class ElasticsearchClientConfig extends ElasticsearchConfiguration {

	@Value("${spring.elasticsearch.rest.uris}")
	String connetionUrl;

	@Override
	public ClientConfiguration clientConfiguration() {
		return ClientConfiguration.builder()
				.connectedTo(connetionUrl)
				.build();
	}

}


We also need to add a ElasticsearchClientConfigTest test for this class in order to satisfy Sonar.

Java
 
@SpringBootTest
class ElasticsearchClientConfigTest {

	@MockBean
	CityRepository cityRepository;

	@Autowired
	ElasticsearchClientConfig elasticsearchClientConfig;

	@Test
	void clientConfiguration() {
		assertThat(elasticsearchClientConfig.connetionUrl).contains("oxygen-arnost");
	}

}


SearchHitsImpl Changes

There were two minor issues related to our usage of SearchHits interface in the previous article.

SearchHitsImpl Changed Interface

Since version 5.0, the constructor of the SearchHitsImpl class was extended with a new pointInTimeId argument. It has no impact on our implementation, as it was used only in tests that were removed after all.

The change is not documented well, but the definition can be found in SearchHits class:

When doing a search with a point in time, the response contains a new point in time id value.

Not Working Jackson Conversion for Aggregations

Since Spring Data Elasticsearch 5.0.1, the SearchHit instance contains aggregations attribute filled (it was null in the previous version). 

See the result from Spring Data Elasticsearch 4.4.6:

Result from Spring Data Elasticsearch 4.4.6

And also from Spring Data Elasticsearch 5.0.1.

Result from Spring Data Elasticsearch 5.0.1

The search action itself is working correctly. The issue happens when we try to send the SearchHits result directly to the client as JSON. Spring doesn't contain any converter for the AggregationsContainer class and produces this error:

Plain Text
 
2023-01-05T08:28:44.707+01:00 ERROR 6656 --- [nio-8080-exec-2] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed: org.springframework.http.converter.HttpMessageConversionException: Type definition error: [simple type, class org.springframework.data.elasticsearch.client.elc.ElasticsearchAggregations]] with root cause

com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class org.springframework.data.elasticsearch.client.elc.ElasticsearchAggregations and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: org.springframework.data.elasticsearch.core.SearchHitSupport$SearchPageImpl["searchHits"]->org.springframework.data.elasticsearch.core.SearchHitsImpl["aggregations"])
	at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77) ~[jackson-databind-2.14.1.jar:2.14.1]
	at com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1306) ~[jackson-databind-2.14.1.jar:2.14.1]
	at com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:408) ~[jackson-databind-2.14.1.jar:2.14.1]
	at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.failForEmpty(UnknownSerializer.java:53) ~[jackson-databind-2.14.1.jar:2.14.1]
	at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.serialize(UnknownSerializer.java:30) ~[jackson-databind-2.14.1.jar:2.14.1]
	...
	at java.base/java.lang.Thread.run(Thread.java:833) ~[na:na]


Therefore, we can simplify our search feature with pagination as:

  • Remove both previous searchPage and searchHits methods in CityController and CityService.
  • Modify our search method in the CityService as:
Java
 
@SuppressWarnings("unchecked")
public Page<City> search(String name, String country, String subcountry, Pageable pageable) {
	return (Page<City>) unwrapSearchHits(searchPageFor(searchHits(name, country, subcountry, pageable), pageable));
}


This way we achieve dynamic search with pagination without the need to tackle the SearchHits related issues.

The new dynamic search implementation can be verified on http://localhost:8080/api/cities?name=be&country=Czech&subcountry=bohemia&size=5&sort=name,asc with this output: 

JSON
 
{
  "content": [
    {
      "id": "ePHmOIUBcEaiCL6qmck4",
      "name": "Benešov",
      "country": "Czech Republic",
      "subcountry": "Central Bohemia",
      "geonameid": 3079508
    },
    ...
  ],
  "pageable": {
    "sort": {
      "empty": false,
      "sorted": true,
      "unsorted": false
    },
    "offset": 0,
    "pageSize": 5,
    "pageNumber": 0,
    "paged": true,
    "unpaged": false
  },
  "last": true,
  "totalElements": 3,
  "totalPages": 1,
  "size": 5,
  "number": 0,
  "sort": {
    "empty": false,
    "sorted": true,
    "unsorted": false
  },
  "first": true,
  "numberOfElements": 3,
  "empty": false
}


Conclusion

This article has covered the upgrade to the latest Spring Data Elasticsearch 5.0.1 with Elasticsearch 8.5.3 (at the time of the article). We started with the proper configuration of the Elasticsearch Docker image. Next, we demonstrated the changed configuration due to the new Elasticsearch client library. In the end, we summarized all the needed fixes in our application in order to make all our features work again as before.

The complete source code demonstrated above is available in my GitHub repository. All the changes (related to this upgrade) are visible in PR #58.

Elasticsearch Spring Data Data (computing)

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Practical Example of Using CSS Layer
  • GitLab vs Jenkins: Which Is the Best CI/CD Tool?
  • How To Build an Effective CI/CD Pipeline
  • 3 Main Pillars in ReactJS

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: