DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones AWS Cloud
by AWS Developer Relations
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones
AWS Cloud
by AWS Developer Relations
  1. DZone
  2. Coding
  3. Java
  4. Maven Central Failover Mechanism Improves: Temporary IP change on Monday

Maven Central Failover Mechanism Improves: Temporary IP change on Monday

Brian Fox user avatar by
Brian Fox
·
May. 10, 11 · Interview
Like (1)
Save
Tweet
Share
4.75K Views

Join the DZone community and get the full member experience.

Join For Free

Spoiler Alert! This post contains information about a change to Maven Central’s IP addresses. If your network has firewall rules in place that need specific IPs, be sure to read this post.

We’re working hard and investing continued effort into making sure that Central is as available as possible. As Maven Central supports a world of developers, even a few minutes of downtime is completely unacceptable to us. In line with our previous efforts to make Maven Central as bullet-proof and available as possible, we are planning to make the US repository even more fault tolerant using a tool called Pacemaker. Once we’ve had time to evaluate the impact of the changes described in this post we will deploy similar measure for our European Union and Asia/Pacific mirrors.

As a follow up to our previous enhancements to central, we are planning to make the Maven Central US repository even more fault tolerant.

The US repository runs on two virtual machines (VMs) in a VMWare cluster with 4 physical nodes configured to use the High Availability support in VMWare. Despite having multiple levels of fault-tolerance, recovery from a misconfiguration or other catastrophic failure still requires a DNS update to a standby IP to restore Maven Central. This DNS change requires time: time to make the change and then an often unpredictable time for DNS changes to propagate over the entire Internet.

This is unacceptable to us. Millions of developers depend on Maven Central, we’ve invested in redundant virtual machines running on redundant physical hardware. If there is an unforeseen event, the problem should be addressed in a few seconds.

To achieve immediate failover in the event of failure we will be using a tool called Pacemaker to manage Maven Central’s floating IP cluster. Pacemaker monitors the repository IP address, Nginx process status, and sample content from Maven Central. If Pacemaker identifies a failure in any one of these components it will immediately failover to the backup machine. In my testing, this takes about 3-5 seconds to occur.

In a previous post I discussed the systems we have in place and how the IPs are configured:

We are aware that some users have firewall rules that are locked to the external service IP. Because of this, we strive to maintain a consistent IP for each system, however the primary mechanism for accessing the repository is by DNS for most users. At times, our failover escalation or maintenance procedures may require us to redirect the DNS for one system to another. For this reason, if you have firewall rules in place that need specific IPs, please allow this list so that you won’t be affected by any temporary transitions:

  • 207.223.240.88 : US primary
  • 207.223.240.92 : US staging / standby
  • 89.167.251.252: UK Primary
  • 89.167.251.253: UK standby

Since we declared .88 to be the primary IP in the US, I intend to use that as the new clustered ip. The migration plan is as follows:

  1. At approximately 9PM CST Monday, May 9th, we will update the DNS entries for repo1/repo2.maven.org to point at 207.223.240.92 (the standby server).
  2. Tuesday morning, the 207.223.240.88 ip will be added to the cluster. We’ll test the ip and failover using this address.
  3. Once the testing is successful, we will revert the DNS entry back to 207.223.240.88.

Both DNS shifts should be completely transparent since both servers will be actively capable of serving artifacts, so regardless of which IP resolves for you during that transition time, the repo will respond. The Expiration TTL on the DNS data is currently 5 minutes, so normally everyone would follow over during that time, but we are going to allow 12 hours (just in case).

We will be rolling out this same setup to our UK servers in a few weeks.

I can’t thank Contegix enough for their ongoing support. They conceived of and implemented this virtual IP solution. They are playing a big role in making Maven Central more reliable.

 

From http://www.sonatype.com/people/2011/05/maven-central-failover-mechanism-improves-temporary-ip-change-on-monday/

Apache Maven

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Unlock the Power of Terragrunt’s Hierarchy
  • Shift-Left: A Developer's Pipe(line) Dream?
  • 5 Steps for Getting Started in Deep Learning
  • What’s New in the Latest Version of Angular V15?

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: