Speed up Build Process

Scenario

In a continuous integration environment, it is very important that a build can complete in a short time to provide feedback to developer as soon as possible after they submit a build verification request.

Solution

You can parallelizing CPU intensive build steps, and run them on different agent nodes. The results from different node can be collected back for further processing. Assuming a product comprises of component1 and component2, and both of them take a long time to build. You can parallelize build of component1 and component2 like below:

The node match condition property for these steps can be specified like below:

• master: node.isAgent()
  this tells master step to run only on agent node (in order to reduce server load)
• checkout: node==parent.node
  this tells step checkout to run on the same agent as the master step
• build components: node==parent.node
  this tells step build components to run on the same agent as the master step
• build component1: node.isAgent()
  this tells step build component1 to run on agent node selected by load balancing algorithem
• build component2: node.isAgent()
  this tells step build component2 to run on agent node selected by load balancing algorithem
• *package: node==parent.node
  this tells step package to run on the same agent as the master step
• publish: node==parent.node
  this tells step publish to run on the same agent as the master step

The idea here is that step build component1 and build component2 is designed to run on least loaded agent nodes, while all other steps are running in the same agent node.

There is still a problem: The nodes running step build component1 and build component2 are not guaranteed to be the same node running step checkout. How can we make sure that necessary files are available when build component1 and component2? This is solved by specifying file requirements for step build component1 and build component2. Let's assume that the agent node choosed to run master step is agent1, and the checkout step retrieves files into two folders into workspace resides on agent1: component1, and component2. Folder component1 contains all files used to build component1, while folder component2 contains all files used to build component2. We can specify file requirements of step build component1 and build component2 like below:
file requirement of step build component1

file requirement of step build component2