Report Aggregation

Version 17 by Robin Shen
on Jun 25, 2011 07:23.

<< Changes from 16 to 17

compared with

Current by Robin Shen
on Jun 25, 2011 14:08.

Key

These lines were removed. This word was removed.

These lines were added. This word was added.

View page history

There are 11 changes. View first change.

-h3. Introduce
-Report aggregation can be used to collect build metrics from low level configurations and aggregate them into high level configurations to provide an instant overview of desired tree. A configuration can define its own aggregation rules or inherit aggregation rules from parent configurations. Taking below configuration hierarchy for example, you may define a JUnit aggregation rule in ROOT configuration to aggregate JUnit metrics from all descendant configurations:
+h1. What is report aggregation
+Report aggregation is designed to provide quick insight of build metrics of descendant configurations from the parent configuration. For example you may define a JUnit aggregation in root configuration to display the overall test number, success rate and test duration for all configurations in the system. The statistics page of root configuration will also include aggregation statistics to display aggregated metrics over time. Metrics will be aggregated when build runs, so you will not see aggregated metrics immediately after aggregation definition.
-{code}
-  + ROOT
-  |
-  +---+ QuickBuild
-  |
-  +------+ V3.x
-  |
-  +---------+ DEV
-  |
-  +---------+ INTEGRATION
-  |
-  +------+ V4.x
-  |
-{code}
+h1. Define aggregations
+Aggregations can be defined in configuration setting page like below:
+!aggregation-definition.png!
-After each build in the system, the JUnit metrics will be sent to ROOT configuration for aggregation, so eventually you will see the JUnit summary of the whole system in overview page of the ROOT configuration. The JUnit summary tells you how many tests in the system, the overall success rate and duration, and the statistics overtime.
+The report set field tells QuickBuild which report sets to aggregate. If left empty, QuickBuild will aggregate all available report sets.
-after finishing building configuration _ROOT/QuickBuild/V3.X/DEV_, the tests related metrics will be collected and aggregated to _DEV_ itself, then go up to configuration _V3.X_, then continue to _QuickBuild_, and finally to _ROOT_. When go to the configuration overview tab, a _Recent JUnit Tests_ panel is displayed and it contains the tests information of the build:
+Aggregations defined in high level configurations will be inherited by low level configurations. This makes aggregation at different levels much easier. Taking below hierarchy for example:
+!aggregation-configuration-hierarchy.png!
-!aggregation1.png!
+We can define three JUnit aggregations at root configuration to aggregate JUnit metrics from DEV, QA and RELEASE configurations respectively. After building of all configurations, overview page of the root configuration will display three aggregation result shown as below:
-after finishing building configuration _ROOT/QuickBuild/V3.X/INTEGRATION_, the same process will be taken place, and a new row is added to the table like:
+!root-junit-aggregation.png!
-!aggregation2.png!
+The _Release JUnit Summary_ table displays JUnit summaries from all release configurations in the system, and the same applies for _QA JUnit Summary_ and _Dev JUnit Summary_.
-In the table above, each row shows the metrics of the latest build of a configuration, that is, when you run configuration _ROOT/QuickBuild/V3.X/DEV_ again, the entry for it will be replaced to the new build _1.0.1_.
+If switch to overview page of _Project A_, it also has three JUnit summaries. However instead of summarizing the RELEASE/QA/DEV configurations of the whole system, these summaries are aggregated from RELEASE/QA/DEV configurations under the _Project A_ configuration. The reason is that _Project A_ inherits aggregations from root configuration. Taking Release aggregation for instance, it declares to aggregate metrics from all child release configurations. In root configuration, this means to aggregate from all release configurations in the system, but in _Project A_, this means to aggregate from all release configurations under _Project A_.
-and in statistics page, a tab 'JUnit Aggregated' displays the trends of the aggregated metrics and you can group by day, by week and by month.
+Additionally, statistics page of relevant configurations will display the aggregated metrics over time, grouped by day, week, or month.
-h3. How to define aggregation
-To define aggregation is very straightforward. Go to configuration setting page and select Aggregations tab, then add an aggregation you want, you need input the aggregation name and aggregation condition. The report set field is used to tell QuickBuild you only want to aggregate some specific report sets, if leave empty, QuickBuild aggregates all available report sets.
-You may define the aggregations as more as you want by specifying the aggregation condition and report set, for example, you may define an aggregation which only aggregates all the metrics from all DEV configurations, and define another to aggregate from all INTEGRATION configurations.
-Just like all other configuration settings, the aggregation setting can be inherited/overridden from its parent.
-You may also add the aggregation gadgets to the dashboard, so you can know the status of a configuration at first glance.
+You may also add the aggregation gadgets to the user dashboard to know the aggregated build metrics at first glance.