Join the DZone community and get the full member experience.
Join For FreeMany build tools use a classic task-based approach for organizing the CI/CD pipeline. Among these tools, there are both old (Ant, NAnt) and modern ones (Gradle, Cake, Nuke).
Let's start with the definition of the CI/CD pipeline.
CI/CD Pipeline
By CI/CD pipeline, we will mean a sequence of stages (targets, tasks) describing and forming the process of software build, test, and deployment. Typically it has the form of a graph. Directed acyclic graph structure to be more precise. Look:
It's a simple variation of the CI/CD pipeline. Each blue box is a stage/target/task. As you might have guessed, arrows mean dependencies.
Since I'm better familiar with Cake than with other build tools, I would use it for further explanation. A structural template for the above pipeline may look like this:
C#
x
1
Task("Clean-Up-Workspace")
2
.Does(() =>
3
{
4
...
5
});
6
7
Task("Restore-Dependencies")
8
.IsDependentOn("Clean-Up-Workspace")
9
.Does(() =>
10
{
11
...
12
});
13
14
Task("Build")
15
.IsDependentOn("Restore-Dependencies")
16
.Does(() =>
17
{
18
...
19
});
20
21
Task("Unit-Tests-On-Windows")
22
.IsDependentOn("Build")
23
.Does(() =>
24
{
25
...
26
});
27
28
Task("Unit-Tests-On-Linux")
29
.IsDependentOn("Build")
30
.Does(() =>
31
{
32
...
33
});
34
35
Task("Package")
36
.IsDependentOn("Unit-Tests-On-Windows")
37
.IsDependentOn("Unit-Tests-On-Linux")
38
.Does(() =>
39
{
40
...
41
});
For now, it looks quite clear and all dependencies are evident.
Implementation
We have a skeleton for the future build procedure and now it's time to get some muscles. Almost all implementations I've seen use a bunch of global variables in order to share some information between the tasks. For instance, I've taken the default Cake build example and adapted it a bit:
C#
x
1
#tool nuget:?package=NUnit.ConsoleRunner&version=3.4.0
2
3
var target = Argument("target", "Default");
4
var configuration = Argument("configuration", "Release");
5
var packageVersion = Argument("version", "1.0.0");
6
7
var buildDir = Directory("./src/Example/bin") + Directory(configuration);
8
var packageNuspec = "./src/Example.nuspec";
9
var packageOutputDir = "./artifacts";
10
var solutionPath = "./src/Example.sln";
11
var testsFailFast = true;
12
var testsPath = $"./src/**/bin/{configuration}/*.Tests.dll";
13
14
Task("Clean-Up-Workspace")
15
.Does(() =>
16
{
17
CleanDirectory(buildDir);
18
});
19
20
Task("Restore-Dependencies")
21
.IsDependentOn("Clean-Up-Workspace")
22
.Does(() =>
23
{
24
NuGetRestore(solutionPath);
25
});
26
27
Task("Build")
28
.IsDependentOn("Restore-Dependencies")
29
.Does(() =>
30
{
31
if (IsRunningOnWindows())
32
{
33
MSBuild(solutionPath, settings =>
34
settings.SetConfiguration(configuration));
35
}
36
else
37
{
38
XBuild(solutionPath, settings =>
39
settings.SetConfiguration(configuration));
40
}
41
});
42
43
Task("Unit-Tests-On-Windows")
44
.IsDependentOn("Build")
45
.Does(() =>
46
{
47
NUnit3(testsPath, new NUnit3Settings {
48
NoResults = true,
49
StopOnError = testsFailFast
50
});
51
});
52
53
Task("Unit-Tests-On-Linux")
54
.IsDependentOn("Build")
55
.Does(() =>
56
{
57
Information("Not implemented yet.");
58
});
59
60
Task("Package")
61
.IsDependentOn("Unit-Tests-On-Windows")
62
.IsDependentOn("Unit-Tests-On-Linux")
63
.Does(() =>
64
{
65
NuGetPack(packageNuspec, new NuGetPackSettings {
66
Properties = new Dictionary<string, string> {
67
{ "Configuration", configuration }
68
},
69
OutputDirectory = packageOutputDir,
70
ArgumentCustomization = args => args.Append($"-Version {packageVersion}")
71
});
72
});
Let me show what disadvantages of this procedure I see:
- Non-obvious on which parameters each task depends. All tasks are "parameterless". I've quoted the word because, in fact, they take parameters, but in an implicit way of global variables, or even worse - hardcoded values.
- From the first drawback follows the second - all dependencies are called without any parameters and all return values of such tasks are ignored as well.
For me, as a guy with some development background, such an approach to the build procedure implementation looks outdated. The example I've given is a toy, in a real-world scenario we will have dozens of global variables and tasks. It will be extremely unclear and tough to understand why this particular task doesn't work, where used variables are defined, where initialized, and what are their actual values at the moment of the task execution.
Let's correct the original pipeline schema to note that all tasks depend on global variables scope.
How Can We Correct the Situation?
Let's consider the flow of parameters passing from task to task.
Our example is pretty simple and therefore we have only one shared parameter "solutionPath" which is used in both "Restore Dependencies" and "Build" tasks. In the current implementation, this parameter is global.
I assume that a better solution should look like this (it's only pseudocode and will not work):
C#
xxxxxxxxxx
1
82
1
#tool nuget:?package=NUnit.ConsoleRunner&version=3.4.0
2
3
var target = Argument("target", "Default");
4
var configuration = Argument("configuration", "Release");
5
var packageVersion = Argument("version", "1.0.0");
6
7
var buildDir = Directory("./src/Example/bin") + Directory(configuration);
8
var packageNuspec = "./src/Example.nuspec";
9
var packageOutputDir = "./artifacts";
10
var solutionPath = "./src/Example.sln";
11
var testsFailFast = true;
12
var testsPath = $"./src/**/bin/{configuration}/*.Tests.dll";
13
14
Task("Clean-Up-Workspace")
15
.Takes(_buildDir: buildDir)
16
.Does(() =>
17
{
18
CleanDirectory(_buildDir);
19
});
20
21
Task("Restore-Dependencies")
22
.Takes(_solutionPath: solutionPath)
23
.IsDependentOn("Clean-Up-Workspace")
24
.Does(() =>
25
{
26
NuGetRestore(_solutionPath);
27
});
28
29
Task("Build")
30
.Takes(_configuration: configuration)
31
.Takes(_solutionPath: solutionPath)
32
.IsDependentOnWithArguments("Restore-Dependencies", _solutionPath)
33
.Does(() =>
34
{
35
if (IsRunningOnWindows())
36
{
37
MSBuild(_solutionPath, settings =>
38
settings.SetConfiguration(_configuration));
39
}
40
else
41
{
42
XBuild(_solutionPath, settings =>
43
settings.SetConfiguration(_configuration));
44
}
45
});
46
47
Task("Unit-Tests-On-Windows")
48
.Takes(_testsFailFast: testsFailFast)
49
.Takes(_testsPath: testsPath)
50
.IsDependentOn("Build")
51
.Does(() =>
52
{
53
NUnit3(_testsPath, new NUnit3Settings {
54
NoResults = true,
55
StopOnError = _testsFailFast
56
});
57
});
58
59
Task("Unit-Tests-On-Linux")
60
.IsDependentOn("Build")
61
.Does(() =>
62
{
63
Information("Not implemented yet.");
64
});
65
66
Task("Package")
67
.Takes(_configuration: configuration)
68
.Takes(_packageNuspec: packageNuspec)
69
.Takes(_packageOutputDir: packageOutputDir)
70
.Takes(_packageVersion: packageVersion)
71
.IsDependentOn("Unit-Tests-On-Windows")
72
.IsDependentOn("Unit-Tests-On-Linux")
73
.Does(() =>
74
{
75
NuGetPack(_packageNuspec, new NuGetPackSettings {
76
Properties = new Dictionary<string, string> {
77
{ "Configuration", _configuration }
78
},
79
OutputDirectory = _packageOutputDir,
80
ArgumentCustomization = args => args.Append($"-Version {_packageVersion}")
81
});
82
});
In the new approach, I've introduced Takes(...) and IsDependentOnWithArguments(...) methods that are used to inject parameters into task context and pass parameters to dependency respectively.
Benefits:
- clear input parameters for the tasks
- theoretical ability to cover tasks with unit tests
Conclusion
Despite the fact that examples were given in Cake, all said is applicable for other task-based build tools.
In the article, I've shown some limitations imposed by the task-based approach and presented a theoretical way of how to bypass them. Perhaps, you already asked similar questions and have a better solution, then please share this with me if possible.
Topics:
devops,
ci/cd,
ci/cd pipeline,
build tools
Opinions expressed by DZone contributors are their own.
Comments