Skip to content

Add DataProvider execution support to Allure TestNG setup section #1236

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
skuznetsov-al merged 7 commits into from
Feb 4, 2026
Merged

Add DataProvider execution support to Allure TestNG setup section #1236

skuznetsov-al merged 7 commits into from
Feb 4, 2026
+579 −11

Conversation

@baflQA
Copy link
Contributor

@baflQA baflQA commented Jan 21, 2026 •
edited
Loading

Context

TestNG DataProviders are a vital part of the test lifecycle, but their execution was previously invisible in Allure reports. This made it difficult to debug issues occurring during data generation or to see attachments (like API logs or database queries) added within a DataProvider method.

This PR implements IDataProviderListener in the AllureTestNg adapter to capture DataProvider executions as fixtures in the "Set up" section of the Allure report.

Key features:

  • Visibility: DataProvider execution is now shown as a fixture in the report.
  • Isolation: Each DataProvider execution is associated with a dedicated container for the specific test method, preventing setup fixtures from leaking to other tests in the same class.
  • Attachments: Attachments added within a DataProvider are now correctly displayed under the DataProvider's setup step.
  • Failure Handling: DataProvider failures are captured and reported with BROKEN status and include full stack traces.
  • Retry Support: Compatibility with TestNG DataProvider retries is ensured; each retry attempt is recorded as a distinct setup fixture within the test container.

Checklist

baflQA added 4 commits January 21, 2026 21:22
@baflQA
feat: Add DataProvider support to Allure report setup section
41acc10 
This commit implements IDataProviderListener in AllureTestNg to report DataProvider executions as fixtures in the Allure report.
- Implemented beforeDataProviderExecution, afterDataProviderExecution, and onDataProviderFailure.
- DataProvider executions now appear in the 'Set up' section.
- Attachments added within DataProviders are now visible.
- Failed DataProviders are correctly reported with BROKEN status.
- Added tests to verify correct reporting of successful and failed DataProviders.
@baflQA
fix: Isolate DataProvider fixtures to specific test methods
b5eee09 
Previously, DataProvider fixtures were attached to the Class container, causing them to appear in the setup of all tests within that class.
This commit creates a dedicated container for each DataProvider execution and links it only to the specific test method requesting the data.
This ensures that DataProvider execution details (and failures) are correctly associated only with the relevant tests.
@baflQA
fix: Reuse DataProvider container for retries and handle suppressed f…
02dbd0a 
…ailures

- Changed dataProviderContainerUuidStorage to use computeIfAbsent to reuse the existing container for a test method if it already exists (e.g. during retry).
- Added currentExecutable.remove() in beforeDataProviderExecution to force generation of a new UUID for each execution attempt, ensuring retries are captured as distinct fixtures even if TestNG suppresses failure events or we missed cleanup.
- Added FlakyDataProvider sample and test case to verify retry reporting.
@baflQA
test: Update container count assertions in AllureTestNgTest
54416e6 
Updated container count expectations in multipleSuites and parallelMethods tests to account for the new DataProvider execution containers introduced by the fix.
@github-actions github-actions bot added the theme:testng TestNG related issue label Jan 21, 2026
@baflQA
Copy link
Contributor Author

baflQA commented Jan 25, 2026

Hey @baev - be So kind to review this Pr when You have some time 🙏

@baev
Copy link
Member

baev commented Jan 27, 2026

Great job! Could you please add tests to check the following:

  1. The container is properly linked to the test result (by checking container.children.contains(tr.uuid)).
  2. When there are multiple tests, all tests are linked to the container as well.
  3. When the data provider is located in the parent class, it is linked to results from both the base and child classes.
  4. When running a suite of two classes, each with a data provider, results should be linked to the correct (and only the correct) containers.
  5. Check behavior with parallel=true and data-provider-thread-count.

@CLAassistant
Copy link

CLAassistant commented Jan 27, 2026 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

baflQA added 2 commits January 27, 2026 21:58
@baflQA
test: add more tests for DataProvider support
6c23f0a
@baflQA
refactor: simplify assertions and improve readability in AllureTestNg…
e8068ef 
…Test

Updated assertions to use more concise methods and removed unnecessary code. This enhances code clarity and maintainability.
@baflQA baflQA force-pushed the feature/dataprovider-setup branch from 256b59b to e8068ef Compare January 27, 2026 21:04
@baflQA
Copy link
Contributor Author

baflQA commented Jan 27, 2026

Great job! Could you please add tests to check the following:

  1. The container is properly linked to the test result (by checking container.children.contains(tr.uuid)).
  2. When there are multiple tests, all tests are linked to the container as well.
  3. When the data provider is located in the parent class, it is linked to results from both the base and child classes.
  4. When running a suite of two classes, each with a data provider, results should be linked to the correct (and only the correct) containers.
  5. Check behavior with parallel=true and data-provider-thread-count.

I hope that this is done.

@baev baev added the type:improvement Change that improves some user experience but can't be considered as a new feature label Feb 3, 2026
baev
baev approved these changes Feb 3, 2026
View reviewed changes
Copy link
Member

@baev baev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@baev
Copy link
Member

baev commented Feb 3, 2026

@baflQA great job 👍

@baflQA
Copy link
Contributor Author

baflQA commented Feb 3, 2026

Thanks. What are the next steps? I am looking forward to the next release including this feature 🤞

@baev
Copy link
Member

baev commented Feb 4, 2026

@skuznetsov-al will review the code and merge it if everything is on point, shouldn't take long

@skuznetsov-al skuznetsov-al merged commit 5c1f6ee into allure-framework:main Feb 4, 2026
4 of 5 checks passed
@baflQA baflQA deleted the feature/dataprovider-setup branch February 4, 2026 11:16
@baflQA
Copy link
Contributor Author

baflQA commented Feb 9, 2026

Hey there! When can i expect the next allure-Java release?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@baev baev baev approved these changes

Assignees

@skuznetsov-al skuznetsov-al

Labels

theme:testng TestNG related issue type:improvement Change that improves some user experience but can't be considered as a new feature

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@baflQA @baev @CLAassistant @skuznetsov-al