Intro to Jenkins Pipelines and Publishing Over SSH

In many projects, things that seem very small come to be decisive factors to continue on the current path or find a better one. Starting from simple text editors to tools used for a long period of time, we all have different flavors for each tool in hand. Merging these ideas sometimes comes to be a to-do, and while this happens for any kind of work done in a group, there are also some other factors that shape the path to it.

This time, we came across an issue which let us think about how to proceed. Our project is being developed as an integration of many standalone microservices. This led us to use different resource files for our remote development and production environments. After considering different options, we finally decided to deploy these files through SSH (from our build server to where the application server is). Since we are using Jenkins for CI/CD, we had to use an ssh plugin.

In this post, I would like to address how we used Publish over SSH to achieve what we needed.

I assume that you are already familiar with how to use SSH and configure it on Jenkins.

The plugin has the following features:

• SCP - Send files over SSH (SFTP)
• Execute commands on a remote server (can be disabled for a server configuration, or for the whole plugin)
• Use username and password (keyboard-interactive) or public key authentication
• Passwords/passphrases are encrypted in the configuration files and in the UI
• SSH SFTP/ SSH Exec can be used as a build step during the build process (Freestyle and Matrix projects)
• SSH before a (maven) project build, or to run after a build whether the build was successful or not (see Build wrappers below)
• The plugin is "promotion aware" (send files directly from the artifacts directory of the build that is being promoted) see Promotions
• Optionally override the authentication credentials for each server in the job configuration (or provide them if they have not been provided for that server in the global configuration)
• Optionally retry if the transfer of files fails (useful for flakey connections)
• Enable the command/ script to be executed in a pseudo TTY(since some commands are not allowed to run without TTY sessions)

If you would like to proceed with the GUI version of this plugin, you can see how to configure it here.

Our first step will be to define what stages to use:

node {
  stage('Getting ready...') {
    git branch: "dev", credentialsId: 'your_id_on_jenkins', url: 'https://github.com/amenski/jenkins_pipeline.git'
  }

Building stage:

stage('Build'){
  script{
    try {
      if(isUnix()){
          sh 'mvn clean package'
      }else{
       bat 'mvn clean package'
      }
    } catch(err) {
    throw err
    }
   }
}

Let's assume that till this point everything has gone right.

stage('SSH transfer') {
 script {
  sshPublisher(
   continueOnError: false, failOnError: true,
   publishers: [
    sshPublisherDesc(
     configName: "${env.SSH_CONFIG_NAME}",
     verbose: true,
     transfers: [
      sshTransfer(
       sourceFiles: "${path_to_file}/${file_name}, ${path_to_file}/${file_name}",
       removePrefix: "${path_to_file}",
       remoteDirectory: "${remote_dir_path}",
       execCommand: "run commands after copy?"
      )
     ])
   ])
 }
}

In this block, we have the sshPublisher keyword, which holds different values corresponding to their respective keywords. By default, the plugin makes the build "UNSTABLE" if there are errors. You can change this behavior by adding FailOnError: true, which tells it to fail the job if there are any problems. The publishers array holds details about where to send what and some additional tuning parameters, including whether there is a command we need to issue after the files have been published to the server.

In the event that it is important to have more transfers or commands to issue, the sshTransferblock can be repeatedly used as:

transfers:[
  sshTransfer(
    execCommand: "Run commands before copy?"
   ),
  sshTransfer(
    sourceFiles:"${path_to_file}/${file_name}, ${path_to_file}/${file_name}",
    removePrefix: "${path_to_file}",
    remoteDirectory: "${remote_dir_path}",
    execCommand: "run commands after copy?"
  )
])

Deploy:

stage('Deploy'){
  if(currentBuild.currentResult == "SUCCESS" || currentBuild.currentResult == "UNSTABLE"){
    if (isUnix()) {
      sh 'echo "Build Succeded."'
      }else{
        bat 'echo "Build Succeded."'
      }
     }
    }

I personally recommend that you use placeholders in every configuration related fields. It is much easier for future changes. To accomplish this, you can work with environment variables or parameters like in the above case and access them with "${env.VAR_NAME}" or "${params.VAR_NAME}".