Testing should start early in the project lifecycle. Ideally, you have a tool that allows teams to capture and organize issues, assign work and follow team activity. That way, you spend less time managing and more time developing the software. Fortunately, this tool already exists: Jira!
You don’t need to be a tech ninja to work with Jira. In fact, it is accessible to any person on your team.
If you decide to use Jira as your project management tool, add Xray to it.
In this article we are going to explain why Jira users love Xray for Test Management.
Xray’s core characteristic is that it leverages Jira’s native features. Its Test, Pre-Condition, Test Set, (Sub) Test Execution and Test Plan are all Jira issue types. This means that testing is managed in the same way as any other issue type you may have (e.g., Bug, Story).
Let’s look at some characteristics of Jira issues that you can take advantage of for Xray issues:
1. Agile / Scrum Boards
2. Search + JQL
3. Integration with other apps
4. Reports / Listings
5. REST API
6. CSV Import
7. Comments
8. Linking
9. Work Logs
10. Assignment & Accountability
1. Agile / Scrum Boards
Xray issues can be included in Scrum boards along with information from QA. Thus, everyone involved in the project can have a clear view of the real status of the issues being handled in the scope of some sprint, not just from a development perspective, but also from a quality perspective.
More info here.
2. Search + JQL
Easily search and interact with Xray issues using JQL, custom fields and specific JQL functions (more than 20!).
By using JQL and dedicated JQL functions, you can leverage your existing Jira knowledge to search what you need, based on the fields relevant to you.
More info here.
3. Integration with other apps
Since a Jira issue is a core Jira concept/artifact, it is used as the basis for many add-ons. Since Xray also uses Jira issues, then the integration with many existing add-ons is available out-of-the-box.
Some examples are Xporter, Structure, R4J and others.
4. Reports / Listings
Like any Jira issue, Xray issues can easily be included in listings. Xray also provides custom fields for each issue type in order to enhance relevant information, in the context of the entity that is being shown (e.g., “requirement status” to indicate the coverage status of a requirement, or the “Test Execution Status” to indicate the progress of a Test Execution).
The “Filter Results” gadget can also list Tests, Test Sets, Tests Plan or requirements and include, for example, information about their status.
5. REST API
Use Jira’s REST API out-of-the-box to query, create or update Xray issues.
A simple example of creation of test cases is found here.
6. CSV Import
Xray entities can be imported as CSV, the same way that you import Jira issues, in general.
Xray issue types have some specific custom fields that give you the ability to do additional operations during the import process.
More info here.
7. Comments
Collaborate by leaving comments in Xray issues for other users to see.
Reference and involve people in the design of test cases or in the definition of a test plan. Make contributions easier, no matter in which testing phase you’re currently working on.
8. Linking
Relate Xray issues to other issues in your Jira instance (e.g., Story, Bug, Task, etc.) — the Jira way.
This facilitates clear navigation between related information, so you can easily access it whenever you need it.
9. Work Logs
You have the ability to add work logs in Test Execution issues in order to record the time it took to run the related tests.
Work logs can also be added to other Xray issue types, as way to report time spent on them.
More info here.
10. Assignment & Accountability
Xray issues can be assigned and watchers can be added to them.
Ensure issues get done by assigning them to users, whether you’re dealing with the specification of a test case or the execution of a bunch of tests within a given Test Execution.
In summary…
Software testing is one of the most important phases in the software development life cycle. The quality of your end product is only as good as your testing. That is why you need a powerful tool that is available in the same place all your team members are already working in – inside Jira.
Why choose an external tool for testing when you can use Xray and have everyone get the maximum benefit from Jira and Jira issue types? What are you waiting for? Try Xray now and see what thousands of Jira users have been raving about.
Hello 🙂
Why is there no way to add another column/field to manual test step result table?
Action | Data | Expected Result | Attachments
How is the workflow supposed to be if there is an Actual Result that is not like the Expected result?
Feedback from my team is that it is dearly needed.
I’m can’t imagine that a way to add more fields to that view was “forgotten”. So we’re currently trying to figure out how the intended workflow is.
Can you give us a hint?
Hi Jan,
Thank you for your comment!
Regarding that we have these two improvements in backlog XRAY-794 and XRAY-1319. Please Vote on it, so we’re able to keep track of your interest and Watch it in order to be notified on any changes.
Best regards,
Xray Team
Hello , I would like to automate the test execution . I am using pycharm to run my tests with ‘Behave’. Can you guide how to create a script to automate my test execution. Thank you in advance.
Hi Vinay,
Please check the following links from our documentation that should help understanding how the automation of Tests works in Xray:
Best regards,
Xray Team
We are evaluating the test management tools, since Dev teams are using Jira we want to use X-ray, is there a way to integrate the testNG test results to X-ray, any pointers would highly be appreciated
Hi Maqsood,
You can check all the information about the integration with TestNG here:https://confluence.xpand-it.com/display/XRAY/Integration+with+TestNG-based+test+runners
Best regards,
Xray Team
Is there a way to integrate it with testNG automated tests? Any documentation would highly be appreciated
Hi Maqsood,
You can check all the information about the integration with TestNG here:https://confluence.xpand-it.com/display/XRAY/Integration+with+TestNG-based+test+runners
Best regards,
Xray Team
Hello,
I would like to know if we can integrate with Cypress automated tests as our testers are keen to use Cypress for automation.
Hi Sravani,
We have a tutorial for testing using Cypress that might help your team: https://confluence.xpand-it.com/display/public/XRAY/Testing+using+Cypress+in+JavaScript
Best regards,
Xray Team
Hi there,
Is there a way to integrate X-ray with future and to mark test results based on test script execution
Hi Priya,
Please see here our documentation about Continuous Integration Tools
– https://confluence.xpand-it.com/display/public/XRAY/Continuous+Integration+Tools
– https://confluence.xpand-it.com/display/public/XRAY/Tutorial%3A+using+PowerShell+scripts+for+Continuous+Integration
Best regards,
Xray Team
We are using PySys as our automation framework running in Jenkins can we update our test execution results using the REST API?
Hi Robert,
Yes, you are able to update the Test Execution results. To do this, you need to import the Execution results to the existing Test Execution. In our documentation, you can find more details regarding the Integration with Jenkins here.
If you need additional assistance, reach out to our support who will be happy to help you.
Best regards,
Xray Team
Hi Xray Team
Can you please guide me, how can we integrate DevTest with Xray?
Hi Amanpreet,
We currently don’t have this integration. Here you can find the official integrations we have available at the moment: https://confluence.xpand-it.com/pages/releaseview.action?pageId=21761014
Although we do not have a specific integration with DevTest, we have an extensive REST API that allows submitting results to Xray in the most varied formats, you can always try this option. We support multiple automation frameworks too. The links below could be of your interest:
https://confluence.xpand-it.com/display/public/XRAY/Integrating+with+Testing+Frameworks
https://confluence.xpand-it.com/display/public/XRAY/TTT%3A+Automation
Hope this is useful!
Best regards,
Xray Team
Hi there,
We started using Xray recently, and we have concern over changes logging to history.
Whenever we make changes to test steps in Tests, JIRA does not log any history for that, as it does for any other change made for that JIRA issue.
Is there a way to track changes made in test steps?
Hi Milos,
Currently, the Test Steps changes are registered on the Issues “History,” and you should be able to see them. If you are experiencing any problems, please contact our Service Desk portal so we can better help you!
Best regards,
Team Xray
Hi,
I currently have all my feature tests in Pycharm and I am just wondering is there a way I can do a bulk export of all those feature tests from pycharm to XRay in JIRA.
Right now I am doing it one by one and its very cumbersome.
Please Advise.
Thanks.
Hi Raunaq,
Execution results can be imported to Jira through JSON/XML representation formats specified in Import Execution Results: https://confluence.xpand-it.com/display/public/XRAY/Import+Execution+Results
For each import file format, Xray provides a specific REST endpoint, you can check at the bellow documentation:
Import Execution Results REST – https://confluence.xpand-it.com/display/public/XRAY/Import+Execution+Results+-+REST
Best regards,
Team Xray
I have used quite a few add-ons integrating Jira with the respective Jira test management tools, like QARA, Zephyr, etc. Will definitely try out Xray as well.
Great choice! Let us know if you need any help. 🙂
Best regards,
Team Xray