How to aggregate test report for mutli-project build with different plugins?

Aggregating Test Reports for Multi-Project Builds with Different Plugins

In a multi-project build, you often have different modules using different testing frameworks and plugins. Aggregating test reports from these disparate sources can be challenging. This article will explore various approaches to effectively consolidate test reports from multiple projects with varying plugins.

Approach 1: Using a Central Reporting Tool

1.1 Choosing a Tool

Several tools can centralize test reports from various sources. Here are some popular options:

• Jenkins: A widely used continuous integration/continuous delivery (CI/CD) server with powerful reporting capabilities.
• SonarQube: Primarily a code quality platform, but also integrates with test reports for comprehensive analysis.
• Allure Framework: A versatile reporting tool compatible with various testing frameworks.

1.2 Configuration

Configure the chosen tool to integrate with your projects’ test reporting plugins. For example, in Jenkins:

pipeline {
  agent any
  stages {
    stage('Build and Test') {
      steps {
        // Build each project
        ...
        // Publish test results
        allureReport(pattern: '**/target/allure-results/**', reportDir: 'allure-report')
      }
    }
  }
}

1.3 Visualization and Analysis

Centralized reporting tools provide insightful dashboards and detailed analysis. You can view:

• Overall test execution summary (pass/fail rates, time taken)
• Detailed reports for individual tests, projects, or build runs
• Trend analysis to identify patterns and regressions

Approach 2: Custom Report Generation

2.1 Scripting

Write scripts to gather test results from different projects and generate a consolidated report. This approach provides flexibility but requires coding expertise.

#!/bin/bash

# Get test reports from each project
project1_report=$(find ./project1/target -name 'junit-report.xml')
project2_report=$(find ./project2/target -name 'testng-results.xml')

# Concatenate reports into a single file
cat "$project1_report" "$project2_report" > aggregated_report.xml

# Generate a human-readable report
# (using a custom script or tool)

2.2 Report Aggregation Libraries

Use dedicated libraries to simplify the process of collecting and combining test results. Examples include:

• ReportNG (Java): Enables aggregation of TestNG reports.
• JUnit (Java): Offers features for combining JUnit test results.
• PyTest (Python): Facilitates report aggregation for PyTest tests.

Comparison of Approaches

Conclusion

Choosing the best approach depends on factors like project complexity, existing infrastructure, and team expertise. Centralized reporting tools offer a simpler and quicker path for basic reporting. Custom report generation provides greater control and flexibility but requires more effort. Evaluate your needs and consider both approaches to achieve optimal test report aggregation for your multi-project build.