Skip to content

Latest commit

 

History

History
341 lines (253 loc) · 11.3 KB

README.md

File metadata and controls

341 lines (253 loc) · 11.3 KB
Raw

Gradle plugin that visualises tests execution schedule

build status Codecov Gradle Plugin Portal This project is using Percy.io for visual regression testing.

Usage

build.gradle.kts:

plugins {
    id("io.github.platan.tests-execution-chart") version "0.6.1"
}

build.gradle:

plugins {
    id "io.github.platan.tests-execution-chart" version "0.6.1"
}
./gradlew test createTestsExecutionReport --rerun-tasks

#...

Tests execution schedule report saved to /my-project/build/reports/tests-execution/mermaid/test.txt file.
Tests execution schedule report saved to /my-project/build/reports/tests-execution/json/test.json file.
Tests execution schedule report saved to /my-project/build/reports/tests-execution/html/test.html file.

Example result (HTML report):

Reports

createTestsExecutionReport task creates reports in HTML, JSON and Mermaid formats.

HTML

Located in build/reports/tests-execution/html/*.html.

JSON

Located in build/reports/tests-execution/json/*.json.

  • startTime - in milliseconds since the epoch
  • endTime - in milliseconds since the epoch
{
  "results": [
    {
      "className": "Test1Spec",
      "durationMs": 120,
      "endTime": 1667936777974,
      "startTime": 1667936777854,
      "resultType": "SUCCESS",
      "testName": "test 2"
    },
    {
      "className": "Test1Spec",
      "durationMs": 213,
      "endTime": 1667936778067,
      "startTime": 1667936777854,
      "resultType": "SUCCESS",
      "testName": "test 1"
    }
  ]
}

Mermaid

Located in build/reports/tests-execution/mermaid/*.txt.

gantt
dateFormat YYYY-MM-DDTHH:mm:ss.SSSZZ
axisFormat %H:%M:%S.%L
section Test1Spec
test 2 - 120 ms :active, 2022-11-08T20:46:17.854+0100, 2022-11-08T20:46:17.974+0100
test 1 - 213 ms :active, 2022-11-08T20:46:17.854+0100, 2022-11-08T20:46:18.067+0100

rendered: (info):

gantt
    dateFormat YYYY-MM-DDTHH:mm:ss.SSSZZ
    axisFormat %H:%M:%S.%L
    section Test1Spec
        test 2 - 120 ms: active, 2022-11-08T20:46:17.854+0100, 2022-11-08T20:46:17.974+0100
        test 1 - 213 ms: active, 2022-11-08T20:46:17.854+0100, 2022-11-08T20:46:18.067+0100
Loading

Configuration

Options:

Key Type Description Default
formats.html.enabled boolean Generate report in html format true
formats.html.script.embed boolean If true mermaid source will be downloaded and used locally in html false
formats.html.script.src url Url to mermaid which should be used to generate html report https://cdn.jsdelivr.net/npm/mermaid@11.3.0/dist/mermaid.min.js
formats.html.script.config.maxTextSize int Limit on the size of text used to generate diagrams 50000
formats.json.enabled boolean Generate report in json format true
formats.mermaid.enabled boolean Generate report in mermaid text format true
components.suites.enabled boolean Add suites/classes to report false
shiftTimestampsToStartOfDay boolean Adjust the earliest timestamp to the start of the day false
marks.totalTimeOfAllTests.enabled boolean Enable mark showing total time of all tests false
marks.totalTimeOfAllTests.name string Label used for mark total time of all tests

build.gradle.kts:

configure<io.github.platan.tests_execution_chart.CreateTestsExecutionReportExtension> {
    formats {
        html {
            enabled.set(true)
            script {
                embed.set(false)
                config {
                    maxTextSize.set(110000)
                }
                src.set("https://cdn.jsdelivr.net/npm/mermaid@11.3.0/dist/mermaid.min.js")
            }
        }
        json {
            enabled.set(true)
        }
        mermaid {
            enabled.set(true)
        }
    }
    components {
        suites {
            enabled.set(false)
        }
    }
    shiftTimestampsToStartOfDay.set(true)
    marks {
        totalTimeOfAllTests {
            enabled.set(true)
            name.set("total time of all tests")
        }
    }
}

build.gradle:

createTestsExecutionReport {
    formats {
        html {
            enabled = true
            script {
                embed = false
                config {
                    maxTextSize = 110000
                }
                src = "https://cdn.jsdelivr.net/npm/mermaid@11.3.0/dist/mermaid.min.js"
            }
        }
        json {
            enabled = true
        }
        mermaid {
            enabled = true
        }
    }
    components {
        suites {
            enabled = false
        }
    }
    shiftTimestampsToStartOfDay = true
    marks {
        totalTimeOfAllTests {
            enabled = true
            name = 'total time of all tests'
        }
    }
}

Option components.suites.enabled

When enabled (set to true) it allows to add suites/classed to report.

Troubleshooting

I see Maximum text size in diagram exceeded in an HTML report instead of a chart.

Mermaid has limit of maximum allowed size of the text diagram. Default value of this size is 50000. This can be overridden using formats.html.script.config.maxTextSize option.

Development

You can publish plugin locally:

cd tests-execution-chart
./gradlew publish

Artifacts are published to ../local-plugin-repository directory.

Configure your project:

settings.gradle (set proper path instead of /path-to):

pluginManagement {
    repositories {
        maven {
            url '/path-to/local-plugin-repository'
        }
        gradlePluginPortal()
    }
}

build.gradle (replace X.X.X-SNAPSHOT with the value returned by ./gradlew cV -q -Prelease.quiet):

plugins {
    id 'io.github.platan.tests-execution-chart' version 'X.X.X-SNAPSHOT'
}

Release

  1. ./gradlew createRelease -Prelease.versionIncrementer=incrementPatch/incrementMinor/incrementMinor - creates a new tag and updates examples in README (but does not commit changes in README)
  2. git push origin release-[X.X.X]
  3. Prepare ~/.gradle/gradle.properties (gradle.publish.key, gradle.publish.secret) and run ./gradlew clean :tests-execution-chart-commons:publishToSonatype closeSonatypeStagingRepository
  4. Verify tests-execution-chart-commons module in staging repository (https://s01.oss.sonatype.org)
  5. Run ./gradlew findSonatypeStagingRepository releaseSonatypeStagingRepository
  6. Prepare ~/.gradle/gradle.properties (sonatypeUsername, sonatypePassword, signing.keyId, signing.password, signing.secretKeyRingFile) and run ./gradlew clean build publishPlugins
  7. Edit Changelog in README.md and git commit -m "Document release [X.X.X]"
  8. Update io.github.platan.tests-execution-chart plugin version in this project

Modules

tests-execution-chart-commons

tests-execution-chart-commons module contains common code which is used by the plugin. This library can be also used to build other tools.

Motivation

JUnit and Spock support parallel execution. Both frameworks allow configuring execution mode for classes/specifications and methods/features. JUnit/Spock documentation illustrates how this configuration affects the execution schedule of tests.

Unfortunately, I did not find existing tool which allows to visualise tests executions schedule for particular project. This Gradle plugin tries to address this need.

Gradle can generate reports in JUnit XML format. But such reports cannot be used to generate charts, because they do not have start timestamps for test methods, only for test classes (with seconds precision).

Changelog

Unreleased

0.6.1 (16 October 2024)

Changed

  • Mermaid library updated to version 11.3.0

0.6.0 (17 October 2023)

Added

  • Add option to include suites/classes in report

0.5.0 (15 October 2023)

Changed

  • Mermaid library updated to version 10.5.0

0.4.0 (1 June 2023)

Added

  • A new option marks.totalTimeOfAllTests allows to add mark showing total time of all tests

Changed

  • Do not remove # character from names of tasks/sections

0.3.1 (29 March 2023)

Added

  • A new option shiftTimestampsToStartOfDay allows to adjust the earliest timestamp to the start of the day

0.3.0 (12 March 2023)

Changed

  • Use dateFormat compatible with Mermaid 9.4.2
  • Link to Mermaid 9.4.3 instead of the latest version

0.2.1 (07 March 2023)

  • (fix) Render chart in HTML report when the test name contains a backtick character

0.2.0 (02 February 2023)

  • (feature) Allow to set maxTextSize config option in Mermaid script in HTML report

0.1.0 (09 November 2022)

  • Add option to create a report that visualises the tests execution schedule

License

This project is licensed under the MIT license.