Personal Mini Toolchains
Learn how to quickly build your own toolchain and give yourself extra time.
by
Join the DZone community and get the full member experience.
Join For FreeIt’s all about repetition. If you found yourself doing the same thing again and again for many times, then chances are, you want to simplify that and eventually automate it. Even the small task makes sense to automate if the amount of repetition is high enough. Let me give you an example.
Let’s imagine you write a script to synchronize files in between the projects. It took you 8 hours to create that script and each run saves 10 minutes of your time. This means you need to run that script 48 times to “pay off” your initial investment. Anything more than that is your “profit”.
Saving 10 minutes here and there might not seem like a big deal. But if you know how to identify and effectively implement more small tasks like this, then it will scale up very quickly and create you a lot of time to do anything you like. For example, to make comic t-shirts (;
There are many ways of automating everything. This time, I will give you the real example of creating a Personal Mini Toolchain in Java. In general, a toolchain is a program that connects together several tools together to perform a complex task. I call it Personal Mini Toolchain because it exists to save your personal time and you can start in a very minimalistic way (and grow it up from there). The great news is, that once you understand the principles, then you can apply them in any area, industry, and using any technology you like.
Step 1 - Figure Out
The most important step is to figure out what is worth and realistic for toolchain to handle. Only then it makes sense to worry about it. Worth is about return on the investment. Realistic means that the set of actions needs to be simple enough and flow from start to end without much of decision logic. Let’s imagine you put the actions into a graph like it is in the image below. You have a better chance to automate the one which is on the left side rather than the one on the right side.
Let’s look at the concrete example. As a hobby, I am creating my own Tyracorn game engine (youtube playlist is here). I started the whole project as a standard Java desktop application using the Maven build system. At a certain point, I decided to run it also on Android. So I opened up the Android developer portal and started to study. I learned how to create an Android project, how to make an OpenGL context, and eventually made it work.
Now I had 2 projects, so whenever I made changes in the main project and wanted to test them on my phone, then I had to do the following.
- Open Android studio.
- Copy and paste the source files.
- Build project.
- Install project on my phone.
After I did this like 50 times, I got bored and started to figure a way how a computer can make this for me. It is important that I did that so many times. It validated the sense of automation. And manual exercise gave me enough insights to do the job.
Now, what is the lesson of this? Start your automation by doing everything manually again and again. This will confirm you that this is a bit worth investing in the effort, and give you the chance to simplify and deeply understand what are you doing. And remember, you don’t need to cover everything right from the beginning. You can start even with a single little portion, it will grow naturally over time.
Step 2 - Create
Now, let’s focus on my concrete example. Because of the manual experience, I figured out that my mini toolchain will be helpful if it does these.
- Generate the Android project. Then I can open the Android Studio and look at it.
- Build and install the project on my device. All this without the need to touch the Android Studio.
- Have the ability to add more platforms and tasks later on.
I started by creating a new maven project configured to produce an executable fat jar. The code takes just the first two arguments from the command line, uses reflection to look up the class and method by these, and calls it passing the rest of the arguments as a list. It looks like this.
/**
* Main entry to the application.
*
* @param args arguments
*/
public static void main(String[] args) {
List as = Arrays.asList(args);
if (as.size() < 2) {
printHelp();
System.exit(1);
}
runCommand(as.get(0), as.get(1), as.subList(2, as.size()));
System.out.println("");
System.out.println("-----------------");
System.out.println("Job Done!");
System.out.println("-----------------");
}
/**
* Runs the command.
*
* @param command command
* @param method method
* @param args arguments
*/
public static void runCommand(String command, String method, List args) {
try {
String className = "com.tyracorn.toolchain." + StringUtils.capitalize(command) + "Command";
Class commandClass = Class.forName(className);
Method m = commandClass.getMethod(method, List.class);
m.invoke(null, args);
} catch (ClassNotFoundException | NoSuchMethodException | SecurityException | IllegalAccessException |
IllegalArgumentException | InvocationTargetException e) {
throw new RuntimeException(e);
}
}
Then I started to study the Android documentation to see if I can build and install the project on my phone just from the command line. Turned out that Google did a great job by preparing Gradle tasks to do all the above. And, as a bonus, I can also automatically sign the application by my secret keys before uploading to the play store.
I used a plain Android project as a base. Then replaced certain values in build.gradle by placeholders (e.g. applicationId is replaced by '$tyracorn.application.id') which will be filled in later on by toolchain. Then packed the whole directory into a zip file and place it into a toolchain project resource directory. In addition, in the project to be converted, I created a specific directory and placed configuration file and additional resources (e.g. icons in various formats) in there. Then the workflow is following.
- Clean up target directory
- Unzip the template
- Copy over the source code and assets
- Copy over the additional Android specific resources
- Override placeholders from the template by the actual values
- Build project
- Install on the phone
If you are interested, then here is the source code.
xxxxxxxxxx
/**
* Generates the android project.
*
* @param args arguments
*/
public static void generate(List args) {
try {
// prepare directories
System.out.println("preparing directories");
File projDir = new File(args.get(args.size() - 1)).getCanonicalFile();
Guard.beTrue(projDir.isDirectory(), "%s is not a directory", projDir.getAbsolutePath());
File targetDir = new File(projDir, "target");
if (!targetDir.isDirectory()) {
targetDir.mkdirs();
Guard.beTrue(targetDir.isDirectory(), "unable to create a target directory");
}
File androidProjectDir = new File(targetDir, "android-project");
if (androidProjectDir.isDirectory()) {
FileUtils.deleteDirectory(androidProjectDir);
}
Guard.beFalse(androidProjectDir.exists(), "android-project directory cannot exists in this point");
// prepare properties
Map properties = new HashMap<>();
for (Object key : System.getProperties().keySet()) {
properties.put((String) key, System.getProperty((String) key));
}
for (int i = 0; i < args.size() - 1; ++i) {
String arg = args.get(i);
if (arg.equals("-c")) {
Guard.beTrue(i < args.size() - 2, "config directory must be specified after -c argument");
File cdir = new File(args.get(i + 1));
Guard.beTrue(cdir.isDirectory(), "config directory must be an existing directory: %s", cdir);
System.out.println("applying config: " + cdir.getAbsolutePath());
Map signingProps = Dut.copyMap(Props.load(new File(cdir, "signing.properties")));
String storeFilePath = new File(cdir, signingProps.get("tyracorn.signing.storeFile")).getAbsolutePath().replaceAll("\\\\", "/");
signingProps.put("tyracorn.signing.storeFile", storeFilePath);
properties.putAll(signingProps);
i = i + 1;
}
}
// loading config files
Config config = Config.load(new File(projDir, "src/main/platforms/android/config.json"));
Pom pom = Pom.load(new File(projDir, "pom.xml"));
// unpack template
System.out.println("unpacking template");
unzipResource("android-project.zip", targetDir);
// copy files from main project
System.out.println("copying source files from the main project");
File srcSrcDir = new File(projDir, "src/main/java");
File srcTargetDir = new File(targetDir, "android-project/app/src/main/java");
FileUtils.copyDirectory(srcSrcDir, srcTargetDir);
List excludeClasses = config.getStringList("excludedClasses");
for (String ec : excludeClasses) {
File cfile = new File(srcTargetDir, ec.replaceAll("\\.", "/") + ".java");
if (cfile.isFile()) {
Guard.beTrue(cfile.delete(), "unable to delete file %s", cfile);
}
}
System.out.println("copying asset files from the main project");
File assetsSrcDir = new File(projDir, "src/main/assets");
File assetsTargetDir = new File(targetDir, "android-project/app/src/main/assets/external");
FileUtils.copyDirectory(assetsSrcDir, assetsTargetDir);
System.out.println("copying app dir from the android configuration");
File srcAppDir = new File(projDir, "src/main/platforms/android/app");
if (srcAppDir.isDirectory()) {
File targetAppDir = new File(targetDir, "android-project/app");
FileUtils.copyDirectory(srcAppDir, targetAppDir);
}
// adjust template
System.out.println("merging templates");
String gid = pom.getGroupId();
String aid = pom.getArtifactId();
if (aid.contains("-")) {
String[] parts = aid.split("\\-");
aid = parts[parts.length - 1];
}
String appId = gid + "." + aid;
String appVersion = pom.getVersion().replace("-SNAPSHOT", "");
Guard.beTrue(StringUtils.isNumeric(appVersion), "only numberic verion is supported, please look to the pom file: %s", appVersion);
String signingStoreFile = properties.getOrDefault("tyracorn.signing.storeFile", "tyracorn-dev.jks");
String signingStorePassword = properties.getOrDefault("tyracorn.signing.storePassword", "Password1");
String signingKeyAlias = properties.getOrDefault("tyracorn.signing.keyAlias", "tyracorn-dev");
String signingKeyPassword = properties.getOrDefault("tyracorn.signing.keyPassword", "Password1");
Map vars = Dut.map(
"tyracorn.application.id", appId,
"tyracorn.application.version", appVersion,
"tyracorn.signing.storeFile", signingStoreFile,
"tyracorn.signing.storePassword", signingStorePassword,
"tyracorn.signing.keyAlias", signingKeyAlias,
"tyracorn.signing.keyPassword", signingKeyPassword,
"loadingScreen", config.getString("loadingScreen"),
"startScreen", config.getString("startScreen"));
System.out.println("merging build properties");
File appGradleBuild = new File(targetDir, "android-project/app/build.gradle");
Templates.merge(appGradleBuild, vars);
System.out.println("merging launch screens");
File mainActivity = new File(targetDir, "android-project/app/src/main/java/com/tyracorn/android/MainActivity.java");
Templates.merge(mainActivity, vars);
} catch (IOException e) {
throw new RuntimeException(e);
}
}
/**
* Builds the android project.
*
* @param args arguments
*/
public static void build(List args) {
generate(args);
try {
File projDir = new File(args.get(args.size() - 1)).getCanonicalFile();
File targetDir = new File(projDir, "target");
File androidProjectDir = new File(targetDir, "android-project");
String gradlePath = androidProjectDir.getCanonicalPath() + File.separator + "gradlew.bat";
System.out.println("building the project");
String buildRes = Cmds.executeSimple(androidProjectDir, gradlePath, "build");
System.out.println(buildRes);
} catch (IOException e) {
throw new RuntimeException(e);
}
}
/**
* Installs the project to the device.
*
* @param args arguments
*/
public static void install(List args) {
build(args);
try {
File projDir = new File(args.get(args.size() - 1)).getCanonicalFile();
File targetDir = new File(projDir, "target");
File androidProjectDir = new File(targetDir, "android-project");
String gradlePath = androidProjectDir.getCanonicalPath() + File.separator + "gradlew.bat";
System.out.println("installing the project to the device");
String installRes = Cmds.executeSimple(androidProjectDir, gradlePath, "installRelease");
System.out.println(installRes);
} catch (IOException e) {
throw new RuntimeException(e);
}
}
There is also a part which allows me to specify a directory with production signing keys. This allows me to use the same tool when making package for the play store while keeping the secrets out of the application versioning control system. I added this later on.
Now, I know this is far away from perfect, but it does a job. It saves me time. So I decided it is good enough for now and moved to the next thing. Knowing when to stop is an important part of the programming job. Not everything needs to be super generic and perfect. The main point is that it serves to you.
Step 3 - Use and Maintain
Having the compiled jar file, usage is very simple. Open the command line and call java -jar jar-file-path.jar android install path\to\project\dir. Then the toolchain generates the Android project, builds it, and installs it on the connected phone. Alternatively, it’s possible to replace install by build or generate to stop earlier in the process. In addition, adding -c path\to\config\dir after the install applies external configuration (e.g. sign apk with the production keys).
Maintenance is very simple. Unless Google decides to change the build process (which is not happening that often), then it’s only about using the toolchain and adding new capabilities when they become useful. And that’s very easy to do because the code is not trying to be generic, configurable, and scalable to millions of users. This is the type of code, which is focused only on you.
If you have read it up to here, then you can download my Tyracorn Showpark application, which was fully built by this toolchain. It is a showcase application for my game engine. I know the graphic isn’t nice at the time of writing this article, but the system works well. Eventually, I will hire a freelancer, using the system as I described in the Mastering Freelancers guide, to make it pretty.
Summary
Now you have seen how to quickly build a toolchain. The most important thing is to figure out the piece to automate and simplify that as much as possible. Then start by creating a simple toolchain for that one purpose. Don’t try to cover everything. That will come to you over time. And finally, use it a lot to create yourself more time for whatever you love.
Published at DZone with permission of Radek Hecl, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.
Trending
-
Revolutionizing Algorithmic Trading: The Power of Reinforcement Learning
-
Constructing Real-Time Analytics: Fundamental Components and Architectural Framework — Part 2
-
Event-Driven Architecture Using Serverless Technologies
-
Introduction to Domain-Driven Design
Comments