In a continuous integration environment, it is very important that a build can complete as quick as possible to provide feedback to developers as early as possible after they submit a build verification request.
You can execute CPU intensive build steps in parallel, and run them on different agent nodes. The results from different nodes can be collected back to server for further processing. Let's assume a product comprises of component1 and component2, and both of them take a long time to build. You can build component1 and component2 in parallel as follows:
The idea here is that step build component1 and build component2 are designed to run on the least loaded agent nodes, while all other steps are running on the same agent node.
There are still a few problems. The nodes running step build component1 and build component2 are not guaranteed to execute on the same node that executes the step checkout. How can we make sure that the necessary files are available when component1 and component2 starts to build? The same issue exists when running step package, how can we make sure that build results of component1 and component2 are available for packaging if the build steps and package step do not run on the same agent?
This is solved by specifying input files and output files for steps. Let's assume that the agent node to run master step is agent1, and the checkout step retrieves files into two folders into workspace that resides on agent1: component1-src, and component2-src. Folder component1-src contains all files used to build component1, while folder component2-src contains all files used to build component2. Build results of comonent1 are saved to folder component1-binary, and build results of component2 are saved to folder component2-binary. Now we can specify following file requirements: