Private Remote Maven Repository With Artipie

Artipie is an open-source binary artefact management tool under an MIT license. It supports the following types of repositories: Docker, Maven, NPM, PyPI, Anaconda, RPM, Gem, Go, and so on. If you are eager to learn about Artipie in detail, you can read the DZone article An Easy Way to Get Your Own Binary Repository.

Let’s focus on the Maven repository. Maven is a tool that can be used for building and managing any Java-based project. Maven has two types of repositories:

• The local repository is a directory on the computer where Maven runs. Maven caches downloads from the remote maven repositories in ‘{user.home}/.m2’ folder. The local repository contains temporary build artefacts.
• Remote repositories are repositories accessed by using a variety of protocols. These repositories can be public, for example, repo.maven.apache.org or private, they are accessed only inside an organization, sharing private artefacts between development teams. 

Local and remote maven repositories have the same structure and are built, extended and supported according to the same “maven repository rules”. Thus, it’s possible to expose anyone’s local maven repository as a remote maven repository.

We will set up a private remote maven repository. 

First, we have to get the Artipie jar file, which you can find on this page. In the next step, we should define an Artipie main configuration that is represented by a ‘yml’ file. So, let’s create a folder for Artipie config ‘{user.home}/artipie/config’ with the ‘my-artipie.yml’ file, which has the following content:

meta:
 storage:
   type: fs
   path: {user.home}/artipie/repo

Where: 

• Field ‘type’ describes which type of storage Artipie will use to get the configuration of repositories. In our case, it’s ‘fs’— the file system storage.
• Field ‘path’ is the absolute path to the directory in the file system where repository config files will be stored.

Now, we are ready to define a maven repository config that is also represented by a ‘yml’ file:

repo:
 type: maven
 storage:
   type: fs
   path: {user.home}/artipie/data

Where:

• Field ‘type’ describes a repository type; in our case, it’s ‘maven’.
• Field ‘storage’ defines the parameters of the file storage where artefacts are placed.

We call this ‘my-maven.yml’ file and place it in the folder ‘{user.home}/artipie/repo’. As you will see later, Artipie uses the filename as the maven repository name.

Now everything is done, and we are ready to launch Artipie. Execute the following command in a terminal:

java -jar {path-to-jar}/artipie-latest-jar-with-dependencies.jar \ 
	--config-file={user.home}/artipie/config/my-artipie.yml \
	--port=8085

Where:

• ‘--config-file’ parameter points to the Artipie main configuration file.
• ‘--port’ parameter defines the port to start the service.

That is all. Now, we have our maven repository!

Let’s try to deploy an artefact to our Artipie’s maven repository. We use the ‘hamcrest-2.2.jar’ file of the well-known Hamcrest library. Having this file downloaded on the local computer, I just copy it from the local maven repository ‘{user.home}/.m2/repository/org/hamcrest/hamcrest/2.2/hamcrest-2.2.jar’ to a temp folder, for example, ‘{user.home}/temp’. Next, to deploy a 3rd party JAR we will use the ‘deploy:deploy-file’ goal under maven-deploy-plugin. The deployment command can be the following:

mvn deploy:deploy-file -DgroupId=org.hamcrest \
    -DartifactId=hamcrest \
	-Dversion=2.2 \
	-Dfile={user.home}/temp/hamcrest-2.2.jar \
	-Durl=http://localhost:8085/my-maven/

You can learn more about the ‘deploy:deploy-file’ goal here.

The console output should be:

Uploading to remote-repository: http://localhost:8085/my-maven/org/hamcrest/hamcrest/2.2/hamcrest-2.2.jar
Uploaded to remote-repository: http://localhost:8085/my-maven/org/hamcrest/hamcrest/2.2/hamcrest-2.2.jar (123 kB at 851 kB/s)
Uploading to remote-repository: http://localhost:8085/my-maven/org/hamcrest/hamcrest/2.2/hamcrest-2.2.pom
Uploaded to remote-repository: http://localhost:8085/my-maven/org/hamcrest/hamcrest/2.2/hamcrest-2.2.pom (392 B at 21 kB/s)
Downloading from remote-repository: http://localhost:8085/my-maven/org/hamcrest/hamcrest/maven-metadata.xml
Uploading to remote-repository: http://localhost:8085/my-maven/org/hamcrest/hamcrest/maven-metadata.xml
Uploaded to remote-repository: http://localhost:8085/my-maven/org/hamcrest/hamcrest/maven-metadata.xml (296 B at 2.1 kB/s)

Here we can see that the ‘hamcrest-2.2.jar’ artefact was successfully deployed to ‘http://localhost:8085/my-maven/’ repository. 

Artipie stores artefacts using the same folder structure as it does maven, so now the folder  ‘{user.home}/artipie/data’ looks like in the following image:

If you already have artefacts cached in a local maven repository and do not want to load them again, just copy the folder  ‘{user.home}/.m2/repository’ to a directory where you are going to store your repository artefacts. In the example, we could copy the local maven repository content to the ‘{user.home}/artipie/data’ folder and these artefacts would be accessible by Artipie’s repository URL. This feature allows us to quickly and easily deploy Artipie’s maven repository based on the existing maven-compatible storage.

Frequently, we want to use private and public maven repositories simultaneously. With Artipie, this requirement can be reached using the ‘maven-proxy’ repository type. The maven-proxy repository redirects the requests to the remote repositories. Let’s turn our existing maven repository into the maven-proxy repository. We have to change the ‘my-maven.yml’ file in the following way:

repo:
  type: maven-proxy
  remotes:
    - url: https://repo.maven.apache.org/maven2
    - url: {your_private_remote_maven_repository}

Where:

• The field ‘type’ is the ‘maven-proxy’ which means using the Artipie maven repository in proxy mode. 
• The field ‘remotes’ defines the list of remote maven repositories which Artipie will request for needed artefacts.

Note that you do not have to shut down Artipie to change the repository configuration. All repository configuration changes will be applied on the fly.

Artipie will try to get the artefact from the list of remote repositories one by one until the artefact is found. To check to download, we will use the maven dependency plugin and any artefact, for example, the same ‘org.hamcrest:hamcrest’ artefact, but 2.1 version. The maven dependency plugin first tries to get an artefact from the local repository and only then from remote repositories. If you already have the folder  ‘{user.home}/.m2/repository/org/hamcrest/hamcrest/2.1’, just delete it to force the maven plugin to request the artefact from the remote repository. Then you may add the repository’s description to ‘{user.home}/.m2/settings.xml’ or create a new maven settings file. Not to affect other projects, we will go with the second option — create maven settings ‘my-maven-settings.xml’ in the folder ‘{user.home}/artipie/’ and define the profile describing our remote repository:

<?xml version="1.0"?>
<settings>
    <profiles>
        <profile>
            <id>artipie</id>
            <repositories>
                <repository>
                    <id>my-maven</id>
                    <url>http://localhost:8085/my-maven/</url>
                </repository>
            </repositories>
        </profile>
    </profiles>
    <activeProfiles>
        <activeProfile>artipie</activeProfile>
    </activeProfiles>
  </settings>

Now, we have to execute the Maven command:

mvn -s {user.home}/artipie/my-maven-settings.xml \
	dependency:get -Dartifact=org.hamcrest:hamcrest:2.1:jar

You can see the following piece of console output:

Downloading from my-maven: http://localhost:8085/my-maven/org/hamcrest/hamcrest/2.1/hamcrest-2.1.pom
Downloaded from my-maven: http://localhost:8085/my-maven/org/hamcrest/hamcrest/2.1/hamcrest-2.1.pom (1.1 kB at 2.0 kB/s)
Downloading from my-maven: http://localhost:8085/my-maven/org/hamcrest/hamcrest/2.1/hamcrest-2.1.jar
Downloaded from my-maven: http://localhost:8085/my-maven/org/hamcrest/hamcrest/2.1/hamcrest-2.1.jar (123 kB at 676 kB/s)

Artipie found the ‘org.hamcrest:hamcrest:2.1:jar’ artefact in the ‘https://repo.maven.apache.org/maven2’ repository and downloaded it. If someone else requests this artefact again, Artipie will try to get it from one of the repositories listed in the configuration file. However, to reduce internet traffic, it is useful to cache the downloaded artefacts in the maven-proxy storage. Let’s add the cache option to the ‘my-maven.yml’ file:

repo:
  type: maven-proxy
  remotes:
    - url: https://repo.maven.apache.org/maven2
      cache:
        storage:
          type: fs
          path: {user.home}/artipie/data
    - url: {your_private_remote_maven_repository}

Where:

• The ‘storage’ field defines the storage where artefacts are cached.

Before trying to download the ‘hamcrest-2.1.jar’ file one more time, do not forget to delete the ‘{user.home}/.m2/repository/org/hamcrest/hamcrest/2.1’ folder. The console output should be the same as above, but now the folder  ‘{user.home}/artipie/data’ has the following structure:

For the next requests for the ‘org.hamcrest:hamcrest:2.1’ artefact, Artipie will get it in file storage rather than download one from external repositories. This behaviour is precisely what we wanted.

Artipie is an open-source binary artefact management tool that allows you to create any repository from a bunch of supported repositories simply and fast. In this article, we don’t touch on some Artipie features, such as security, the ability to host data on a custom storage, etc. But you may find this information in Artipie’s wiki documentation. To encourage our efforts in growing Artipie, please give us a star on GitHub.