Tyrrrz/ActionsTestLogger

Actions Test Logger is a custom logger for dotnet test that integrates with Actions. When using this logger, failed tests are listed in job annotations and highlighted in code diffs. Additionally, this logger also generates a job summary that contains detailed information about the executed test run.

By using this project or its source code, for any purpose and in any shape or form, you grant your implicit agreement to all the following statements:

• You condemn Russia and its military aggression against Ukraine
• You recognize that Russia is an occupant that unlawfully invaded a sovereign state
• You support Ukraine's territorial integrity, including its claims over temporarily occupied territories of Crimea and Donbas
• You reject false narratives perpetuated by Russian state propaganda

To learn more about the war and how you can help, click here. Glory to Ukraine! 🇺🇦

• 📦 NuGet: dotnet add package ActionsTestLogger

To use Actions Test Logger, install it in your test project and modify your Actions workflow by adding --logger Actions to dotnet test:

name: main
on: [push, pull_request]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Install .NET
        uses: actions/setup-dotnet@v4

      - name: Build & test
        run: dotnet test --configuration Release --logger Actions

By default, the logger will only report failed tests in the job summary and annotations. If you want the summary to include detailed information about passed and skipped tests as well, update the workflow as follows:

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      # ...

      - name: Build & test
        run: >
          dotnet test
          --configuration Release
          --logger "Actions;summary.includePassedTests=true;summary.includeSkippedTests=true"

Warning: The new testing platform (i.e. Microsoft.Testing.Platform) is not yet supported. This is because VSTest and MTP are using different extensibility models, and this project existed before MTP existed. To use Actions Test Logger, make sure to use the classic testing experience (vstest) instead.

Important: Ensure that your test project references the latest version of Microsoft.NET.Test.Sdk. Older versions of this package may not be compatible with the logger.

Important: If you are using .NET SDK v2.2 or lower, you need to set the <CopyLocalLockFileAssemblies> property to true in your test project.

Actions Test Logger can leverage source information to link reported test results to the locations in the source code where the corresponding tests are defined. By default, dotnet test does not collect source information, so the logger relies on stack traces to extract it manually. This approach only works for failed tests, and even then may not always be fully accurate.

To instruct the runner to collect source information, add the RunConfiguration.CollectSourceInformation=true argument to the command as shown below:

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      # ...

      - name: Build & test
        # Note that the space after the last double dash (--) is intentional
        run: >
          dotnet test
          --configuration Release
          --logger Actions
          --
          RunConfiguration.CollectSourceInformation=true

Note: This option can also be enabled by setting the corresponding property in a .runsettings file instead.

Warning: Source information collection may not work properly with legacy .NET Framework.

When running dotnet test, you can customize the logger's behavior by passing additional options:

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      # ...

      - name: Build & test
        run: >
          dotnet test
          --configuration Release
          --logger "Actions;annotations.titleFormat=@test;annotations.messageFormat=@error"

Use the annotations.titleFormat option to specify the annotation title format used for reporting test failures.

The following replacement tokens are available:

• @test — replaced with the display name of the test
• @traits.TRAIT_NAME — replaced with the value of the trait named TRAIT_NAME
• @error — replaced with the error message
• @trace — replaced with the stack trace
• @framework — replaced with the target framework

Default: @test.

Examples:

• @test → MyTests.Test1
• [@traits.Category] @test → [UI Tests] MyTests.Test1
• @test (@framework) → MyTests.Test1 (.NETCoreApp,Version=v6.0)

Use the annotations.messageFormat option to specify the annotation message format used for reporting test failures. Supports the same replacement tokens as annotations.titleFormat.

Default: @error.

Examples:

• @error → AssertionException: Expected 'true' but found 'false'
• @error\n@trace → AssertionException: Expected 'true' but found 'false', followed by stacktrace on the next line

Use the summary.includePassedTests option to specify whether passed tests should be included in the summary. If you want to link passed tests to their corresponding source definitions, make sure to also enable source information collection.

Default: false.

Warning: If your test suite is really large, enabling this option may cause the summary to exceed the maximum allowed size.

Use the summary.includeSkippedTests option to specify whether skipped tests should be included in the summary. If you want to link skipped tests to their corresponding source definitions, make sure to also enable source information collection.

Default: false.

Warning: If your test suite is really large, enabling this option may cause the summary to exceed the maximum allowed size.

Use the summary.includeNotFoundTests option to specify whether empty test assemblies should be included in the summary.

Using test filters might result in some test assemblies not yielding any matching tests. This might be done on purpose in which case reporting these may not be helpful.

Default: true.