Why Requirements Coverage is critical for Software Development

Requirements are the core of every software development process. Requirements help in defining the process and the quality criteria the software should match. Here’s where Requirements Coverage comes into the picture.

When working with a large number of requirements, it is likely that engineers might miss checking a few requirements. This can lead to scope creep, budget and time overrun.

Requirements Coverage is a process that allows controlling how many requirements have test cases associated with them, what type of test has been designed and how many requirements are still left before we execute our test cases.

In this article, we will discuss the importance of Requirements Coverage and how it benefits software development and testing.

Continue reading Why Requirements Coverage is critical for Software Development

Latest Update About RMsis version 2.2.2

We have recently released a new version of RMsis.

RMsis 2.2.2 is an incremental update on 2.2.0. 

Changes:  

  • Bugfix for Authentication error due to SameSite cookie issue
  • Provide RMsis APIs to manage attachments

Let’s explore these changes in more detail

Continue reading Latest Update About RMsis version 2.2.2

Performance and scale testing of your App on Jira DC

If you have apps listed on Atlassian Marketplace then you might have come across this regulatory requirement from Atlassian that your applications must perform as per a set criteria. This is good for customers because if an app is in the marketplace then they can be sure that it meets the performance and scalability benchmarks. They can safely rely on Atlassian’s judgement. And it is good for the app providers also because it boosts their confidence in their own app. Clearing the benchmark is not a small feat. Test data is huge. 1 million issues, 2 million attachments, 6 million comments. Check out the full set on the developer website.

We have two apps on the marketplace – (1) RMsis – Requirement Management for Jira (2) LXP – Links Explorer Traceability & Hierarchy. Both of them are available for Jira Data Center and earstwhile Jira Server edition. LXP is available on the cloud as well. I have gone through the performance and scale test for one app. Hence I am writing down my experience here. It will definitely be useful when I go for another round for another app.

This article is not to scare you about the bechmarking test but to encourage you to go for it. Atlassian has done a great job of documenting the steps. This post will serve as an add-on to get you started. I’ll describe the concepts involved.

A well known fact is that the Jira Data Centre or Jira DC is designed to serve a lot more concurrent users under enterprise load than you would normally do with server edition. Basically, you can add high performing nodes as your usage goes beyond a limit. You can start with single node and ramp-up the hardware as the situation demands. Atlassian has to make sure that the apps also perform and scale well along with Jira. Hence the tests.

To list your apps on the Atlassian Marketplace, you are required to test the apps on 1-node, 2-node and 4-node Jira DC environment. And submit the test results to Atlassian. The math goes like this – 4 machines (max) to install Jira DC, one machine to host database, and one for executing the test-cases. This last one could be your laptop if your whole setup is local i.e. within the premise. In total, you will require 6 machines.

Next three sections are dedicated to the high-level steps to complete this exercise. For detailed instructions inside each step, I recommend the documentation published by Atlassian. I’ll give references wherever necessary. Next sections will cover:

  1. Setting up the hardware
  2. Setup Jira DC and your app
  3. Develop and execute the tests

Setup the hardware

If you are working in a large enterprise with own cluster then probably you will have to go through some administrative tickets to provision the hardware required for the tests. Other option is to quickly launch virtual machines on Amazon AWS cloud and delete when you are done. In this article, I’ll talk about the latter.

Atlassian and AWS have jointly worked on some open-source quick-start tools that can be readily used for our purpose. You can either create the infrastructure on AWS manually or you may use the quickstart tools to automatically create the required infrastructure. Remember that AWS infrastructure will incur some cost.

Although the quick-start help document is big, it is a must read if you really want to understand what is happening under the hood. You will appreciate the power of automation after going through it. It has some tips to customize the infrastructure as well; especially to reduce some cost. But in my experience, the whole exercise took less than a single working day if you stretch a bit, so you will not save much even if you customize it. I would recommend that you use the quick-start tool as-it-is. And save time instead of cost. In case you follow my recommendation then you just have to clone the quickstart-atlassian-jira git-repository. And copy it to a dedicated AWS S3 Bucket. You may push it in any other bucket but it is better to keep it isolated for security reasons as well. On a command-line, you will do just this (I am assuming you have aws-cli and git installed):

$ git clone --recurse-submodules \
  https://github.com/aws-quickstart/quickstart-atlassian-jira.git

$ aws s3 mb s3://<BUCKET_NAME> --region <AWS_REGION>

$ aws s3 cp quickstart-atlassian-jira \
  s3://<BUCKET_NAME>/quickstart-atlassian-jira \
  --recursive --acl public-read

With public access to the above directory inside AWS S3, you can now launch a complete AWS CloudFormation without actually thinking about the network design, hardware setup, database setup, Jira Software installation. The steps are simple:

  1. Go to AWS CloudFormation
  2. Click Create stack
  3. Provide Amazon S3 URL of the quickstart-jira-dc-with-vpc.template.yaml file in the above bucket. (https://bucket_name.s3.amazonaws.com/quickstart-atlassian-jira/templates/quickstart-jira-dc-with-vpc.template.yaml)
  4. Follow the wizard

I have collapsed multiple things in the fourth step. Basically, in the wizard, you will have to select the type of AWS virtual machines that you want to use for Jira DC and PostgreSQL installation and setup database passwords. Atlassian advises you to create a development setup first; with smaller AWS instances. t3-medium is sufficient for development. I think you should consider that advice. On a development setup, when you pass 95% test-cases then go for the actual tests with the full-fledged enterprise class data-set provided by Atlassian.

Setup Jira Software Data Center and your App

One of the advantages of using the quick-start setup is that you don’t have to install Jira DC yourself. The declarative way in which it is done is really nice and it scales just like that. You can start with just 1-node and based on your load graph of the day or may be a week, you can schedule adding and reducing nodes. The setup needs to be configured only once. And the setup is also easy. But in this article we are talking about the scale and performance testing. So all this scaling will happen in a controlled way. We’ll increase nodes one-by-one and save the test results for each scenario.

After your first instance is up. Just copy the load-balancer url and open it in your browser. It will take you to the initial Jira setup screen where you specify the site-title, administrator email-id etc. Once this is done, your single-node setup is ready to use.

Jira DC setup for development and actual enterprise testing are not different. The only difference is in the infrastructure. You don’t need too many nodes at the time of development and you don’t need bigger machines. A t3-medium sized single node is sufficient.

When testing on the large data-set, you are required to test on multiple nodes as well. To add more nodes to the above setup:

  1. Goto AWS CloudFormation console.
  2. Select the stack and choose update-stack from actions.
  3. Click next and change the minimum and maximum number of cluster nodes. Done.

You are done. In few minutes, your new node will be up. And for the tests to work you will need to synchronise Jira search index on all the nodes. Data Center App Performance Toolkit has a shell-script to synchronise the search index. You are required to do this every time you add a new node.

Let us now move on to see what are the scenarios that Atlassian wants us to test.

Develop and execute test-cases

We are required to submit test results for five scenarios:

  1. 1-node, without app
  2. 1-node, with app
  3. 1-node, with app, with app-specific tests included
  4. 2-node, with app, with app-specific tests included
  5. 4-node, with app, with app-specific tests included

Once your 1-node setup is up and running, import the large data-set provided by Atlassian. This is a huge data-set, as I have mentioned earlier. If the system is performing well with this data-set then the probability is rare that it will face any performance related issues in the customer’s deployment.

Atlassian has prepared a test suite which has large number of tests that simulate 21000 user (200 concurrent), 54500 actions per hour. Test duration for single run is ~45 minutes. You are required to add a few test cases for your app as well. If your app just adds some UI elements then adding selenium tests is all that you need. But if the app adds some APIs then add a few Jmeter calls as well for each API. Code examples are given in the Data Center App Performance Toolkit.

As suggested by Atlassian, do not jump directly to the final testing. First launch a development environment on AWS, with smaller instances, and then build your app-specific test cases on a separate branch in the git repository. Simplest way is to fork the Atlassian’s repository and clone from there. Then create a separate branch for adding app-specific tests. If you have more than one app in the Atlassian Marketplace then create a branch for each app. Push all your changes to your remote repository.

In the development scenario, you should not import the full data-set. Instead create 2 test projects with five-five issues in each. That is sufficient for development. Also you can run the tests directly from your own laptop. You need to set some variables in dc-app-performance-toolkit/app/jira.yml to let it know the URL of your Jira DC installation. Once the single-node Jira DC development setup is ready in AWS, just run the performance toolkit.

$ bzt jira.yml

The results are saved in dc-app-performance-toolkit/app/results/jira/YY-MM-DD-hh-mm-ss directory. Check results_summary.log. Do not proceed with the full data-set tests until you have all the actions with 95+% success rate.

Once you are done with test developments, launch the full infrastructure with 1-node. Then increase nodes as specified in the full document. It is recommended that you fire the test suite from a virtual machine on AWS, instead of your laptop/workstation. Otherwise things like power or network outage may affect the execution and you will have to start over again. Each run takes a long time ~45 minutes.

Once you single-node setup is ready, you are required to upload the full data-set. This has three steps:

  1. populate_db.sh (~1 hour)
  2. upload_attachments.sh (~2 hours)
  3. Re-index from Jira Administration screen (~30 minutes)

On the AWS execution environment (the VM that we fired for running tests), to save you from lots of installations, Atlassian has created a docker image. You can directly fire the tests from that docker image. So the only installation that you need is docker. In short:

  1. Launch an Ubuntu 18.03 LTS VM
  2. Install docker
  3. Checkout your dc test repository
  4. Instead of installing and running bzt, just launch the docker container
$ docker run --shm-size=4g -v "$PWD:/dc-app-performance-toolkit" \ atlassian/dcapt jira.yml

Again the results are save in dc-app-performance-toolkit/app/results/jira/YY-MM-DD-hh-mm-ss. You will need to do this five times, once for each scenario.

Conclusion

It took me 10 hours to complete the full exercise; with some breaks, of course. But the whole thing was worth it. It gives me confidence that our apps will perform really well in the production environment of our customers. Nothing is more satisfying than a happy customer.

In case you need any help in the above setup then you may join the Slack Community and ask there as well. The community is really active and there is dedicated staff from Atlassian to help you out.

Enjoy testing!

What to Expect from RMsis 2.2.0

We have recently added a new release for RMsis.

RMsis 2.2.0 is an incremental update on 2.1.0. New features include

  • Latest Version filter
  • APIs to automatically create test run and fill test results

Let’s explore these changes in more detail.

Continue reading What to Expect from RMsis 2.2.0

How RMsis can simplify your Requirements Management activities?

Wondered why is requirements management so important to software and product development teams? The purpose of requirements management is to ensure product development goals are successfully met.

Requirements Management is a simple process of documenting, analyzing and agreeing on requirements so that engineering teams always have a current and approved set of requirements.

Requirements management provides a way to avoid errors by keeping track of changes in requirements and fostering communication with stakeholders throughout the engineering lifecycle.

It is reported that as many as 71% of software projects fail due to poor requirements management, making it the single biggest reason for project failure – higher than bad technology, missed deadlines, or change management fiascoes.

In this article, we will discuss how critical Requirements management is and how RMsis can simplify the process.


Continue reading How RMsis can simplify your Requirements Management activities?

7 Best Practices that must be followed by Project Managers

With increasing complexities and changing dynamics of the marketplace, customer’s need is evolving.

And you, as a Project Manager, need agility to adapt customers’ rapidly changing requirements while helping your development team to meet their task.

In addition to responding to customer’s feedback and demands, Project Managers have to cater to modern project development needs like- iterative roadmap, a large number of requirements documents, and a geographically distributed team.

As these challenges mount, the pressure is on you to evolve your organization’s development process and deliver the best quality products.

Clearly, continuing to communicate via unstructured methods like emails, chats, and general spreadsheets, will not help you in delivering successful projects.

Continue reading 7 Best Practices that must be followed by Project Managers

Tips to Enhance your Requirements Writing skills

Writing Requirements requires both skills and practice. A better requirements document can save your organization a fortune with clear communication between the developer and product stakeholders. This, in turn, reflects across the organization, including greater transparency, lesser rework, and improves productivity.

While every organization has different requirements and methodologies, the fundamentals for writing requirements remain the same. 

In this article, we will share a few tips that can help you enhance your requirements writing skills.

Continue reading Tips to Enhance your Requirements Writing skills

What are the most critical Challenges in Requirements Management?

Every Project Manager or Business Analyst understands the efficacy of managing requirements to deliver a successful project. Whether its a Project Management or Software Development, the foundation of success or failure depends on requirements management.

The vitality of requirements crosses project methodologies, whether agile or traditional. It also crosses industries, from financial, manufacturing, construction, energy, telecom, or others.

Analysts report that as many as 71% of software projects that fail do so because of poor requirements management, making it the single biggest reason for project failure – bigger than bad technology, missed deadlines, or change management fiascoes.

Continue reading What are the most critical Challenges in Requirements Management?

Requirements Management – Why it is essential for Project success


The pressure of creating smarter products and systems is increasing the challenge for engineering enterprises.  With the rapid market changes in all industries, complexities to manage system requirements are on the rise.

Today, enterprises have to withstand the pressures of high-speed evolution of technologies, the emergence of new technologies, and associated process change. Simultaneously, harmonising with a broader range of stakeholder’s interests in the development.

It is essential to use robust requirements management tools to overcome such challenges. The requirements management function is the first step in a trail of traceability from requirements through user stories, design, code generation, testing, and deployment.

In this article, we will discuss the importance of Requirements Management for any successful project.

Continue reading Requirements Management – Why it is essential for Project success

Requirements Version Management: Why you need it in Project Development

Almost every product evolves through a series of releases or iterations that correct defects, add more features, or update previously implemented features. It’s quite possible that for a product there can be a number of versions running in parallel.

Therefore you need to determine the best ways to store and manage these requirements. And avoid confusion about which requirements were used in each version while branching. An in-depth version control system helps to achieve this. 

Version control or version management allows the management of changes in requirements over time. It allows a complete retrace of all versions in each project, including the author of those versions.

Continue reading Requirements Version Management: Why you need it in Project Development