Test Plans
Before Xray 2.0, there was no specific Test Plan issue, though you could implement your “test plan” in two ways:
- just by creating Test Executions and consider them as your “test plan” for a given version
- or, by creating Test Sets; you could define one or more Test Sets for a given version using the Fix Version field
Either way, probably, you will have many test cycles or as we call – “Test Executions”. Each Test Execution has its own tests, depending on the requirements you want to validate, the available resources, the target revision, etc. Each one may be complementary, overlapping, be a subset/superset of another one or have exactly the same tests.
It’s quite normal that you end up having something similar to the following picture.
So, how can you have an overall status of your tests?
Introducing the “Test Plan”
The Test Plan purpose is to address effective test planning, namely by tracking relevant tests (i.e. the ones in the Test Plan) and their results.
With the new Test Plan issue, you have a consolidated overview of tests. You are also able to assign one, or more, Test Plans to a given release and track progress of those tests within each Test Plan.
Your Test Plan starts by specifying the planned begin and end dates, along with the Tests that will be part of it. Normally, you would also indicate the Fix Version, so that executions you create in the Test Plan context will inherit it.
As soon as you create a Test Plan, you are able to add more Tests, even from existing Test Sets. A progress bar shows the “Overall Execution Status”, grouping and couting tests by status.
Executing tests for a Test Plan is straightforward; you can easily create a Test Execution for all the tests in the Test Plan, or just for the ones that are still failing, or that are not yet ran, or for any other test run statuses you wish to.
If you already have Test Executions, you can make them part of an existing Test Plan by setting the “Test Plan” custom field in the Test Execution. As simple as that.
In the Test Plan screen you may see, and filter by, the tests that make your Test Plan.
For each Test, you may see its latest status and all related Test Executions.
As soon as you start executing tests in Test Executions that are associated with the Test Plan, you will see the results being reflected directly on the overall execution status. Notice the “Test Plan Status” custom field, which you may include in your screens or in your JIRA’s issues search results table, as a column.
Besides the traditional way of calculating requirement coverage for a given version, the requirement coverage may now be calculated taking into account tests from a given Test Plan, and related Test Executions. Therefore, you are able to analyze requirements based on a wider scope (i.e. the “version”) or on a more constrained way, just for the Test Plan.
In sum…
Until now you had to organize your tests in Test Sets. And that makes all sense. But one thing is organizing the tests, by grouping them, and another thing is planning tests for a given release. Test Executions provided the basics for test planning but lacked a higher level and consolidated view of the status of a group of tests.
Xray’s “Test Plan” allows effective test planning and facilitates schedule test execution and tracking testing progress, for the tests that are most relevant to you.
Finally, before you ask…
- A “Test Plan” seems like a group of Test Executions or a group of Tests…
Yes, a bit of both in fact. A Test Plan contains a list of Tests. For calculating the status of each test, it takes into account test runs from Test Executions explicitly linked to the Test Plan, through the field “Test Plan”.
- Can I clone “Test Plans”?
Yes. The list of related Tests is copied; Tests are not cloned.
- Can I have multiple Test Plans for a version?
Yes, if you wish to track different groups of Tests (e.g. regression tests vs tests for new requirements).
- Do I have to use “Test Plans”?
You don’t but soon you will find great advantages if you start using them.
I need to create test-execution from test-plan with all test using rest api.
Hi Rakesh,
You can find all the information here: https://confluence.xpand-it.com/display/public/XRAY/Tests+-+REST#Tests-REST-CreatingandEditingTests-JiraRESTAPI
And an example here:
https://developer.atlassian.com/server/jira/platform/jira-rest-api-examples/#creating-an-issue-examples
Best regards,
Xray Team
Hello,
Is there a way to export test execution results and connect it to a desired test plan as well?
I know how to post test executions but I can’t link it with a test plan.
Hi Piotr,
You can import your Test Plans with the following steps here.
If you need any further assistance don’t hesitate to contact our support team here.
Best regards,
Xray Team
“As soon as you start executing tests in Test Executions that are associated with the Test Plan, you will see the results being reflected directly on the overall execution status”.
==> Is there any case that the results of test executions don’t reflected on the overall execution status of test plan?
I have faced an issue that the results of test executions don’t update to overall status of test plan. But when I try with the new JIRA/XRAY server, it’s working fine.
Hi,
That behaviour was a known problem that is already fixed. So if you have an old version of Xray prior to 3.2 to rectify this problem, you should first make a Recalculation of Requirement Status and then perform a re-index.
Contact us through our Support Portal if you need more assistance. https://jira.xpand-it.com/servicedesk/customer/portal/2
Best regards,
Xray Team
I would like to set up test plan , test cases and test case templates in X-ray,could you please tell me how to do that. thanks
Hi Gouri,
There is no straight answer to your question as setting up test plans and test cases can mean a couple of different things. You can take a look at these documentation pages to better familiarize yourself with Xray terms and configurations:
https://confluence.xpand-it.com/display/public/XRAY/Terms+and+Concepts
https://confluence.xpand-it.com/display/public/XRAY/Custom+Fields+and+Screen+Configuration
For more detailed support, you can open up a support ticket and our support team can better clarify your inquiries. 🙂
Best regards,
Team Xray
Is there any way to build daily test execution dashboard ? Dound test run finish date does not appear on JIRA to write query! I am looking for test executed per day by team, how can i get that.
Hi Anuj,
There are two gadgets that allow you to use a time frame to show results. There is no daily value, however, you can filter the results by week.
Find out more info about these gadgets here: https://confluence.xpand-it.com/display/public/XRAY/Test+Runs+List+Gadget
https://confluence.xpand-it.com/display/public/XRAY/Test+Runs+Summary+Gadget
Best regards,
Team Xray
Hello,
is there abny possibility in JIRA backend for XRAY to configure the limit of attached test plans to one Test? Or other way round how many tests are allowed to attach to one test?
Wir are getting the error in the field for “test plan associated with a test” of a test:
“This field is not avilable due to its high number of entries (165).Please use the issue view screen toadd or remove Test plans.”
And it prevents to save the test case.
thank you
Hi Lena,
Xray currently has an option on Global Settings, where the user can define the max number of Tests inside a Test Plan.
Regarding the message that you are receiving, it is due to the limit that the field Test Plan associated a Test has whenever creating an issue. However, to overcome this issue, you can create the Test Plan and navigate to it, then will be able to add how many desired tests to the Test Plan. Resuming, this is only a limitation in the Create Issue screen.
If you need further support – feel free to reach out via Service Desk.
Best regards,
Team Xray
Thank you for your replay, I opened SD ticket for this!
BR,
Lena
Is there anyway to clear the overall execution status at the top of the test plan in order to show the status based upon an environment? We use the same test plan as the software is promoted through the different environments before it is moved to production,. However, the overall execution status always shows the latest test execution of the tests which are usually a combination of executions in different environments.
Hi Wendy,
Currently, it is not possible to clear the overall execution status at the top of the test plan based on the environment since the calculation of the overall execution status is based on the latest status.
I hope this helps answer your question.
Best regards,
Team Xray