Xray V3.0: Three or Tree?

Our team has been working hard to achieve its next major milestone: the release Xray version 3.0!

With v3.0, we aim to address some highly requested features, including:

• Ranking of Tests within the Test Plan
• Testing Evolution/Progress, from the Tests’ point-of-view
• Hierarchical Tests organization

The last one, and the most complex was implemented as two different but complementary features: the Test Repository and the Test Plan Board. They address organization at different levels: project and Test Plan. This is something you will only find in very few enterprise-level tools. Now, you can also take advantage of it in Jira, thanks to Xray.

Being able to support different browsers and having an awesome UI with a great UX have been our goals from the start. Thus, we iterated and tested it with real users. The result is an interface that is easy to use, with shortcuts and drag-and-drops so you can be more efficient. Less pain, more gain J

Ranking of Tests within the Test Plan

This has been requested for some time and it makes all sense. Ranking of Tests at the Test Plan level, similar to what we have for Test Executions, allows users to define the “priority” (i.e., the ranking) on how these should be addressed during the execution phase.

Whenever users create planned Test Executions from the Test Plan, the ranking of the Tests will be inherited from the Test Plan.

Ranking can be achieved by dragging-and-dropping the Tests within the respective section in the Test Plan issue screen.

Test Repository

The Test Repository allows the hierarchical organization of Tests at the project level by enabling users to organize Tests in folders.

This is a different approach from the traditional way of organizing Tests in flat lists by using Test Sets, which was the only available method before Xray v3.0.

Users coming from legacy Test Management tools may find this approach more natural and intuitive. Other organizations may prefer the Test Set approach instead. Thus, a general setting will enable users to globally opt-in or opt-out of this hierarchical way of organizing Tests.

Some things you can do from some selected Tests or from some folder:

• Create a Test Plan or add to an existing one
• Create a Test Execution or add to an existing one
• Create a Test Set or add to an existing one

In the Test Repository screen, you can see some meta-folders that provide you quick ways to see or filter some relevant Tests that you can use afterwards:

• All: all Test issues within the current project
• Orphans: all non-organized Test issues, in the context of the current project

To facilitate your organization – and contrary to the possibility of copying files into different folders in a computer – for a given Test, you’re only allowed to put it in one folder. Also, folders with similar names cannot be created under the same parent folder.

Test Plan Board

While the Test Repository deals with the organization of Tests within the scope of a given project during the specification phase, the Test Plan Board deals with the organization at the planning level (and consequently, at execution).

For a given Test Plan that is tracking some group of Tests and their executions, you may wish to organize the Tests by component, by sub-system or by any other criteria.

The (Test Plan) Board gives you the ability to organize the Tests that are part of your Test Plan in multiple folders and sub-folders. Implicitly, you can also define the order of the Tests and the relevance of those grouped Tests (i.e., folders) by ranking them within the tree presented in the Board.

By providing you the ability to define groups and sub-groups (i.e., folders and sub-folders) for the Test Plan’s Tests, you’re also able to track them independently and see their current statuses right away, even for the same Test Environment.

Doing operations, such as creating Test Executions, for a subset of Tests organized in this hierarchy is straightforward.

Note: The Board has its own ranking, independent from the ranking made in the Test Plan issue screen. If you want to use the Board ranking to schedule the Test Executions, you need to do it from the Board.

Tests Evolution Gadget

For some time, people requested us the ability to track the progress of a group of Tests through time. Xray provides a similar feature for requirement coverage with the Historical Daily Coverage report. However, from a tester’s point-of-view, it’s also important to track the evolution of the Tests planned for a specific release or sprint by considering the executions made.

The Tests Evolution tool provides just that: to specify the list of Tests for which you want to see the evolution of their calculated status, based on the runs of the Test Executions you provide. Thus, on a daily basis, you can see the trend of their calculated execution status.

Now what?

Stay tuned, follow our blog and our Twitter account.
We will schedule a webinar soon to explain and demonstrate the Xray v3.0 features in more detail.
We know that you’ll have many questions concerning hierarchical Tests organization, but it’s just too much information to fit in this blog post J

In our extensive documentation, you’ll also find deeper insights, tips and recommendations.

Don’t worry. We’ll be publishing more posts focused on these features. In the meantime, happy testing!