Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Automate It: Be Lazy (Part 2)

DZone's Guide to

Automate It: Be Lazy (Part 2)

Let's keep learning tips to automate your JIRA workflows and save you time using a Netflix command line client.

· DevOps Zone ·
Free Resource

Do you need to strengthen the security of the mobile apps you build? Discover more than 50 secure mobile development coding practices to make your apps more secure.

I will continue from my previous post about automation.

The following parts are left from my process:

The Process

  1. Create an issue for myself (if already existing, move to step 2).
  2. Create a branch within the appropriate project based on the issue.
  3. Assign the issue to myself.
  4. Migrate the issue into the state of IN PROGRESS
  5. Done.
  6. Done.
  7. Done.
  8. Done.
  9. Close the appropriate JIRA issue with a reference to the commit in a comment.

If I work on updates of dependencies or parents in our builds, it is required to make a JIRA ticket for each of those updates (OK, it's not really required, but it is useful). It makes sense to test each of those updates separately in Jenkins. In the end, it is important information for the end users, so they can see what has been changed.

Automation Step 4

The first step is to create a ticket with some information like a summary, a description and giving Affected versions and Fix Version/s useful values. This is only needed if we do not have a ticket for the work yet.

So far, I was going into my browser opened the appropriate project and pressed the Create button and typed in the appropriate values. I already mastered the copy-and-paste parts from the browser URL to the command line. After some time, I thought about this and came to the conclusion that there must be a more convenient way.

I tried to use JIRA's REST API with some JavaScript which I used earlier to create some release notes, but it was too complex to handle via JavaScript. After some research, I found a command line client which looks promising.

I started to play around with the command line client. There are some things to mention about it:

  1. Single binary artifact; simply download it and put on the path. Name it as you like (I decided to name it jira-cli)
  2. Templating mechanism to create custom commands
  3. Simple and powerful default commands
  4. Easy configuration via yaml file

First, you need to login into the JIRA instance, which can be handled by the following:

$ jira-cli login --endpoint=https://issues.apache.org/jira

You will be prompted for your password. The user which is used to log in is derived by the user you are working with. For me, it was wrong, so I needed to add a parameter:

$ jira-cli login --endpoint=https://issues.apache.org/jira --user=khmarbaise

Now, I can view a ticket in the command line very easily:

$ jira-cli view MEJB-118
issue: MEJB-118
created: 339 days ago
status: Closed
summary: Remove "J2EE" from plugin description and site
project: MEJB
issuetype: Task
assignee: khmarbaise
reporter: afloom
fixVersions: 3.0.1
priority: Trivial
votes: 0
description: |
  The term "J2EE" was repalced by "Java EE" by Sun/oracle logn time ago and we shouldn't use "J2EE". It's currently used at least in the plugin's description (pom.xml) and on the intro page of the plugin's site.
  Replace "J2EE Enterprise Javabean (EJB)" with "Java Enterprise JavaBean (EJB)".

comments:
  - | # khmarbaise, 338 days ago
    Done in [r1805332|https://svn.apache.org/r1805332]
  - | # hudson, 338 days ago
    SUCCESS: Integrated in Jenkins build maven-plugins #9069 (See [https://builds.apache.org/job/maven-plugins/9069/])
    [MEJB-118] Remove "J2EE" from plugin description and site (khmarbaise: [http://svn.apache.org/viewvc/?view=rev&rev=1805332])
    * (edit) maven-ejb-plugin/pom.xml
    * (edit) maven-ejb-plugin/src/site/apt/index.apt.vm

To take a short look at a particular issue, it is much faster than going via the browser.

After a more detailed reading of the documentation, I found that I could put the information about the endpoint and the user to be used for JIRA into a configuration file (within the home directory) .jira.d/config.yml, which looks like this:

endpoint: https://issues.apache.org/jira/
user: khmarbaise
login: khmarbaise

With this, the login command can be reduced to jira-cli login. Easy, right? The jira-cli will save session information in the .jira.d/ directory, which is stored in the cookies.js file. For my convenience, I have created a Git repository in .jira.d and added cookies.js to the .gitignores file to prevent me from commiting the cookies.js file accidentally.

Now I have the ability to create JIRA tickets via command line. So far, so good. But how do I know for which project I need to create a ticket?

But, wait a second. The pom.xml contains the information project.issueManagement.url, which is the URL for the JIRA tracker. I needed to find a way to extract it from the pom.xml.

Luckily, the maven-help-plugin is the way to go here, and it looks like this (for brevity, I have added \; usually this is a single line):

ISSUE_URL=$(mvn org.apache.maven.plugins:maven-help-plugin:3.1.0:evaluate \
  -Dexpression=project.issueManagement.url \
  -q \
  -DforceStdout)

This will extract the URL for JIRA out of the pom file. You get something like this:

https://issues.apache.org/jira/browse/MDEPLOY

Unfortunately, this is not the project you need for jira-cli. What I need is the MDEPLOY. This can easily achieved:

PROJECT=$(basename $ISSUE_URL)

This is the information needed to create a JIRA ticket. Hm…really? No. What about the version for Fixed versions/s and Affected versions ?

This can be done like so:

PLAIN_VERSION=$(mvn build-helper:parse-version \
  -Dx=\${parsedVersion.majorVersion}.\${parsedVersion.minorVersion}.\${parsedVersion.incrementalVersion} \
  org.apache.maven.plugins:maven-help-plugin:3.1.0:evaluate \
  -Dexpression=x \
  -q \
  -DforceStdout)

This will extract the version out of the pom file without the trailing -SNAPSHOT. It assumes that the version number has three digits.

So now comes a favorite feature of jira-cli. You can create your own commands very easy. You simply define things like this in your config.yml file:

custom-commands:
  - name: mine
    help: display issues assigned to me
    script: |-
       list --template table --query "resolution = unresolved and assignee=currentuser() ORDER BY priority asc, created"

Now you can simply call jira-cli like this: jira-cli mine and you will see the list of issues which are assigned to yourself. See more details in the documentation of jira-cli. Based on the above, I have created a custom command:

custom-commands:
  - name: createdependencyupgrade
    help: creates an Dependency Upgrade ticket.
    options:
      - name: affected
        help: The affected version
      - name: fix
        help: The fix versions.
      - name: project
        default: $JIRA_PROJECT
      - name: description
        default: ""
      - name: priority
        default: "Minor"
    args:
      - name: summary
        required: true
    script: |-
       create --project= --issuetype="Dependency upgrade" -o summary="" -o description="" -o fixVersions="" -o versions="" -o priority="" --noedit

As the result of this, I can create a JIRA ticket via this:

jira-cli createdependencyupgrade --project MWAR --fix="3.0.2" --affected="3.0.2" "Summary Text"

These have been the prerequisite to finally create a JIRA ticket in a very convenient way. The result of previous things and integrating some other parts is now my createdependencyupgradeissue.sh script, which I can simply call like this:

$ createdependencyupgradeissue.sh "Upgrade maven-plugins parent to version 32"

The output of that script looks like this at the moment:

$ createdependencyupgradeissue.sh "Upgrade maven-plugins parent to version 32"
Checking if we are on a branch...done.
Check that we are on master...done.
Extracting issue url from pom.xml file...done.
Extracting version from pom.xml file...done.
Creating JIRA issue...done.
OK MWAR-419 https://issues.apache.org/jira/browse/MWAR-419
M    pom.xml
Switched to a new branch 'MWAR-419'

As you might already have realized, I have added things to create a branch in Git based on the JIRA issue name and switched to that branch.

The fourth step of automation has been accomplished, which means more improvement, but not enough.

Automation Step 5

If I wanted to assign a ticket to myself, so far, I had to go to the ticket page and click on the assign to me link which prerequisites to know the issue number and type in the correct URL etc.

So based on my assumptions that the branch name is the JIRA issue number the only thing I need to do, is to extract the number from the branch. This can simply be achieved by:

BRANCH=$(git symbolic-ref --short HEAD)

So next the thing is to assign the ticket to myself. This can be done with jira-cli easily like this:

$ jira-cli take $BRANCH

I have to add some sanity checks in case of errors and the result is the takeissue.sh script which I can use within the branch like this:

$ takeissue.sh

The fifth step of automation is accomplished which means becoming comfortable but I think, we could do more.

Automation Step 6

This step is more or less identical to the previous one cause I would like to change the state of an issue into IN PROGRESS. This can easily be achieved by using jira-cli like this:

$ jira-cli start MDEP-234

Based on the ideas before, this can be baked into a script startissue.sh which results in such a simple call

$ startissue.sh

The sixth step of automation accomplished which means more improvement is possible.

Automation Step 7

I can create an issue in JIRA including the creation of the branch in Git, so what about the commit here? Usually, I update files or add some new ones, which is done by git add ., and afterwards, I can commit the state.

Based on the issue, the commit message should look like this:

[XXX-123] - Summary 
Optional description

In the majority of cases, I need only the first line. In other words, this is a copy-and-paste from the issue I created before. Let us make that easier. As you know, I created a script, commitonissue.sh, which does exactly that.

It identifies the branch based on the name of the branch in Git and gets the information about the summary via jira-cli from JIRA, which results in such a line:

SUMMARY=$(jira-cli view $BRANCH | grep "^summary: " | cut -d " " -f2-)

Finally, I need to do the real commit, which can be handled like this:

git commit -a -m"[$BRANCH] - $SUMMARY"

The seventh step of automation is accomplished, which means we are on the home stretch, but have not yet crossed the line.

Automation Step 8

This step is nothing new, it is just improving something that already exists. The gitmergeclean script from the first post needs to be tweaked. It should also close the JIRA ticket after the branch has been merged to the master with an appropriate comment on it.

I have added the following lines to the script to handle this:

#
# Get the latest commit HASH
#
COMMITHASH=$(git rev-parse HEAD)
#
# Get the GIT URL from pom file:
# TODO: Can we do some sanity checks? Yes: scm:git:..  if not FAIL!
echo -n "Get the git url from pom file..."
GITURL=$(mvn help:evaluate -Dexpression=project.scm.connection -q -DforceStdout | cut -d":" -f3-)
echo " URL: $GITURL"
GITPROJECT=$(basename $GITURL)
GITBASE=$(dirname $GITURL)
#
echo "Closing JIRA issue $BRANCH"
jira-cli close -m"Done in [$COMMITHASH|$GITBASE?p=$GITPROJECT;a=commitdiff;h=$COMMITHASH]" --resolution=Done $BRANCH
## Error handling?
echo "Closing finished."

The BRANCH information is already extracted at the beginning of the script. The final lines of the script are to call jira-cli to close the issue with an appropriate comment and set the resolution to a useful value. In our case, it is simply Done. The GITURL is extacted to create a useful comment which contains a link to the real git repository showing the diff of the commit that has been made. You can take a look at the MCLEAN-87 issue, which shows a link to the git repository.

A full output of the new gitmergeandclean script looks like this:

~/ws-git-maven/plugins/maven-rar-plugin (MRAR-79)$ gitmergeandclean.sh
Switched to branch 'master'
Your branch is up to date with 'origin/master'.
Updating 1ffdfba..055bbda
Fast-forward
 pom.xml | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
Total 0 (delta 0), reused 0 (delta 0)
remote: Sending notification emails to: ['"commits@maven.apache.org" <commits@maven.apache.org>']
remote: To git@github:apache/maven-rar-plugin.git
remote:    1ffdfba..055bbda  055bbda0907661aa6515cc8e79ba66cee43aec12 -> master
remote: Syncing refs/heads/master...
To https://gitbox.apache.org/repos/asf/maven-rar-plugin.git
   1ffdfba..055bbda  master -> master
remote: Sending notification emails to: ['"commits@maven.apache.org" <commits@maven.apache.org>']
remote: To git@github:apache/maven-rar-plugin.git
remote:  - [deleted]         MRAR-79
remote: Syncing refs/heads/MRAR-79 (FORCED)...
To https://gitbox.apache.org/repos/asf/maven-rar-plugin.git
 - [deleted]         MRAR-79
Deleted branch MRAR-79 (was 055bbda).
Get the git url from pom file... URL: https://gitbox.apache.org/repos/asf/maven-rar-plugin.git
Closing JIRA issue MRAR-79
OK MRAR-79 https://issues.apache.org/jira/browse/MRAR-79
Closing finished.

Conclusion

Finally, for a dependency upgrade, the process looks like this:

  1. Change the appropriate information in pom.xml file
  2. createdependencyupgradeissue.sh "Upgrade maven-plugins parent to version 32"
  3. commitonissue.sh
  4. gitpushwithlease.sh
  5. takeissue.sh
  6. startissue.sh. After this, I have to wait for the result from Jenkins. If the build is successful, I can merge back to the master. Otherwise, I need to find the cause of the problem. In this case, the result was successful, so I can do it like this:
  7. gitmergeandclean.sh

The output in total looks like this:

~/ws-git-maven/plugins/maven-linkcheck-plugin (master *)$ createdependencyupgradeissue.sh "Upgrade maven-plugins parent to version 32"
Checking if we are on a branch...done.
Check that we are on master...done.
Extracting issue url from pom.xml file...done.
Extracting version from pom.xml file...done.
Creating JIRA issue...done.
OK MLINKCHECK-32 https://issues.apache.org/jira/browse/MLINKCHECK-32
M    pom.xml
Switched to a new branch 'MLINKCHECK-32'
~/ws-git-maven/plugins/maven-linkcheck-plugin (MLINKCHECK-32 *)$ commitonissue.sh
[MLINKCHECK-32 cd34b97] [MLINKCHECK-32] - Upgrade maven-plugins parent to version 32
 1 file changed, 1 insertion(+), 1 deletion(-)
~/ws-git-maven/plugins/maven-linkcheck-plugin (MLINKCHECK-32)$ gitpushwithlease.sh
Counting objects: 3, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 1.53 KiB | 1.53 MiB/s, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: Sending notification emails to: ['"commits@maven.apache.org" <commits@maven.apache.org>']
remote: To git@github:apache/maven-linkcheck-plugin.git
remote:  * [new branch]      cd34b97ddb661d5881bf7fc01e3291a9b0f67041 -> MLINKCHECK-32
remote: Syncing refs/heads/MLINKCHECK-32...
To https://gitbox.apache.org/repos/asf/maven-linkcheck-plugin.git
 * [new branch]      MLINKCHECK-32 -> MLINKCHECK-32
~/ws-git-maven/plugins/maven-linkcheck-plugin (MLINKCHECK-32)$ takeissue.sh
~/ws-git-maven/plugins/maven-linkcheck-plugin (MLINKCHECK-32)$ startissue.sh

Here is the time where I have to wait for the results from Jenkins. In this case, the result was successful, so I can continue like this:

~/ws-git-maven/plugins/maven-linkcheck-plugin (MLINKCHECK-32)$ gitmergeandclean.sh
Switched to branch 'master'
Your branch is up to date with 'origin/master'.
Updating ff5dfab..cd34b97
Fast-forward
 pom.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Total 0 (delta 0), reused 0 (delta 0)
remote: Sending notification emails to: ['"commits@maven.apache.org" <commits@maven.apache.org>']
remote: To git@github:apache/maven-linkcheck-plugin.git
remote:    ff5dfab..cd34b97  cd34b97ddb661d5881bf7fc01e3291a9b0f67041 -> master
remote: Syncing refs/heads/master...
To https://gitbox.apache.org/repos/asf/maven-linkcheck-plugin.git
   ff5dfab..cd34b97  master -> master
remote: Sending notification emails to: ['"commits@maven.apache.org" <commits@maven.apache.org>']
remote: To git@github:apache/maven-linkcheck-plugin.git
remote:  - [deleted]         MLINKCHECK-32
remote: Syncing refs/heads/MLINKCHECK-32 (FORCED)...
To https://gitbox.apache.org/repos/asf/maven-linkcheck-plugin.git
 - [deleted]         MLINKCHECK-32
Deleted branch MLINKCHECK-32 (was cd34b97).
Get the git url from pom file... URL: https://gitbox.apache.org/repos/asf/maven-linkcheck-plugin.git
Closing JIRA issue MLINKCHECK-32
OK MLINKCHECK-32 https://issues.apache.org/jira/browse/MLINKCHECK-32
Closing finished.

If you look at my previous post, you might be astonished that the number of process steps has not been reduced dramatically. Those scripts are replacing a large number of manual steps (git command line typing, Jira in the browser, clicking, typing, etc.) which are now simply handled by scripts, which means the steps are always the same and I am sure I will not forget steps. Furthermore, many scripts can be used in other situations, as well, which makes my life easier — for example, in handling pull requests, etc.

This was a much longer post. I hope you enjoyed reading it.

Check out tips for blazing the way from agile to DevSecOps with security built into your mobile app toolchain.

Topics:
devops ,jira ,open source ,tutorial ,automation ,git

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}