h3. Scenario
In a continuous integration environment, it is very important that a build can complete as quick as possible to provide feedback to developers as early as possible after they submit a build verification request.
h3. Solution
You can execute CPU intensive build steps in parallel, and run them on different agent nodes. The results from different nodes can be collected back to server for further processing. Let's assume a product comprises of component1 and component2, and both of them take a long time to build. You can build component1 and component2 in parallel as follows:
!parallel-build.png!
The _node match condition_ property for these steps can be specified as follows:
* *master:* node.isAgent()
this tells the master step to run only on agent node (in order to [reduce server load|Reduce Server Load])
* *checkout:* node==current.parent.node
this tells the step _checkout_ to run on the same agent as the master step
* *build components:* node==current.parent.node
this tells the step _build components_ to run on the same agent as the master step
* *build component1*: node.isAgent()
this tells the step _build component1_ to run on agent node selected by load balancing algorithm
* *build component2*: node.isAgent()
this tells the step _build component2_ to run on agent node selected by load balancing algorithm
* *package*: node==current.parent.node
this tells the step _package_ to run on the same agent as the master step
* *publish:* node==current.parent.node
this tells the step _publish_ to run on the same agent as the master step
The idea here is that step _build component1_ and _build component2_ are designed to run on the least loaded agent nodes, while all other steps are running on the same agent node.
There are still a few problems. The nodes running step _build component1_ and _build component2_ are not guaranteed to execute on the same node that executes the step _checkout_. How can we make sure that the necessary files are available when component1 and component2 starts to build? The same issue exists when running step _package_, how can we make sure that build results of component1 and component2 are available for packaging if the build steps and package step do not run on the same agent?
This is solved by specifying input files and output files for steps. Let's assume that the agent node to run master step is _agent1_, and the _checkout_ step retrieves files into two folders into workspace that resides on _agent1_: _component1-src_, and _component2-src_. Folder _component1-src_ contains all files used to build component1, while folder _component2-src_ contains all files used to build component2. Build results of comonent1 are saved to folder _component1-binary_, and build results of component2 are saved to folder _component2-binary_. Now we can specify following file requirements:
* *for step* *{_}build component1{_}*
!filetransfer1.png!
Specified input files will be transfered from workspace of the node running parent step to workspace of the node running current step. In this case, all source files of component1 will be transfered from _agent1_ to the node running step _build component1_. If the same nodes are used for parent and child step, no file transfer will be done.
Specified out files will be transfered from workspace of the node running current step to workspace of the node running parent step. In this case, all generated binary files by step _build component1_ will be put back into workspace of _agent1_. In case of same nodes are used for parent and child step, no file transfer will be done.
* *for step* *{_}build component2{_}*
!filetransfer2.png!
Refer to input files and out files explanation of step _build component1_ above.
In a continuous integration environment, it is very important that a build can complete as quick as possible to provide feedback to developers as early as possible after they submit a build verification request.
h3. Solution
You can execute CPU intensive build steps in parallel, and run them on different agent nodes. The results from different nodes can be collected back to server for further processing. Let's assume a product comprises of component1 and component2, and both of them take a long time to build. You can build component1 and component2 in parallel as follows:
!parallel-build.png!
The _node match condition_ property for these steps can be specified as follows:
* *master:* node.isAgent()
this tells the master step to run only on agent node (in order to [reduce server load|Reduce Server Load])
* *checkout:* node==current.parent.node
this tells the step _checkout_ to run on the same agent as the master step
* *build components:* node==current.parent.node
this tells the step _build components_ to run on the same agent as the master step
* *build component1*: node.isAgent()
this tells the step _build component1_ to run on agent node selected by load balancing algorithm
* *build component2*: node.isAgent()
this tells the step _build component2_ to run on agent node selected by load balancing algorithm
* *package*: node==current.parent.node
this tells the step _package_ to run on the same agent as the master step
* *publish:* node==current.parent.node
this tells the step _publish_ to run on the same agent as the master step
The idea here is that step _build component1_ and _build component2_ are designed to run on the least loaded agent nodes, while all other steps are running on the same agent node.
There are still a few problems. The nodes running step _build component1_ and _build component2_ are not guaranteed to execute on the same node that executes the step _checkout_. How can we make sure that the necessary files are available when component1 and component2 starts to build? The same issue exists when running step _package_, how can we make sure that build results of component1 and component2 are available for packaging if the build steps and package step do not run on the same agent?
This is solved by specifying input files and output files for steps. Let's assume that the agent node to run master step is _agent1_, and the _checkout_ step retrieves files into two folders into workspace that resides on _agent1_: _component1-src_, and _component2-src_. Folder _component1-src_ contains all files used to build component1, while folder _component2-src_ contains all files used to build component2. Build results of comonent1 are saved to folder _component1-binary_, and build results of component2 are saved to folder _component2-binary_. Now we can specify following file requirements:
* *for step* *{_}build component1{_}*
!filetransfer1.png!
Specified input files will be transfered from workspace of the node running parent step to workspace of the node running current step. In this case, all source files of component1 will be transfered from _agent1_ to the node running step _build component1_. If the same nodes are used for parent and child step, no file transfer will be done.
Specified out files will be transfered from workspace of the node running current step to workspace of the node running parent step. In this case, all generated binary files by step _build component1_ will be put back into workspace of _agent1_. In case of same nodes are used for parent and child step, no file transfer will be done.
* *for step* *{_}build component2{_}*
!filetransfer2.png!
Refer to input files and out files explanation of step _build component1_ above.