Skip to main content

Issue Priority and Severity

Article

Issue Priority and Severity

Article by Peter Clark | Comments: (16) | Fri, 12/30/2005 - 3:02pm
Summary:

There are several topics that can trigger near religious fervor in software developers--languages, indentation, and comments come immediately to mind. One of Peter Clark's personal favorites is the relationship of issue priority to issue severity in defect tracking systems. Just what the heck do all those levels mean, anyway? In this week's column, Peter describes a solution that his company devised to clearly define the characteristics of severity and priority and help them better understand how the two work together.

Not all requirements are created equal. Some requirements are more central to the purpose of an application than others. For example, our company produces airport baggage handling systems. While it is important that the reports show the right numbers, it is absolutely central to the system that bags are delivered to the correct flight. An issue that prevents that, or that causes a potential safety hazard will be assigned the highest severity level.

Issue severity has to do with the impact of the defect in question to system end-users. Software is developed to achieve a purpose; issues get in the way of achieving that intention. Just how much the issue obstructs achieving the goal determines the severity of the issue.

Our company uses five levels of severity:

  • Showstopper--either a safety issue or an issue that affects a central requirement for which there is no workaround. It prevents either use or testing of the system.
  • High--an issue that affects a central requirement for which there is a workaround. Use or testing of the system can proceed in a degraded mode.
  • Medium--an issue that affects a non-central requirement for which there is no workaround. The feature cannot be used.
  • Low--an issue that affects a non-central requirement for which there is a workaround.
  • Cosmetic--information is correctly shown but the appearance is wrong, such as misspelled words, wrong font, wrong indentation, etc.

Issue priority is the order in which issues are addressed by developers. In our company, issues are triaged by supervisory personnel, who may adjust the severity level and who will then assign the issue a priority and dispatch it to a developer for remediation.

Our company uses four levels of priority: urgent (drop whatever else you are working on and fix this now), high, medium, and low. Developers are expected to remediate issues in priority order: first urgent; then high; then medium; and finally low.

We don't always fix issues in order of severity. I may assign less-experienced developers a medium or low issue, even though outstanding showstopper issues still remain. I may do this because they have no experience with the sub-system where the showstopper resides, and there is no one available to mentor them as they fix it. Or I may have a low severity issue that is on a customer punchlist and must be cleared before we can get paid, so I give it a high priority.

The notion of issue triage requires an adjustment by developers. Often, they will want to work on their own pet issues first, rather than working on issues that I assign to them in the order that I want them addressed. They will attempt to get around these restrictions by inflating the severity of an issue when they log the issue into the tracking system. This is why supervisory personnel adjust the issue severity prior to assigning it. It is expected that supervisors will be more dispassionate than the developers, and that they have sufficient domain experience to correctly assess an issue's impact.

There is little point in having a separate issue prioritization scheme if you do not have a single point of issue triage. If developers are selecting their own issues from the issue tracking system, it is usually sufficient to have them address issues in order of decreasing severity.
Our issue tracking system has the concept of deferred issues. A deferred issue is one that we are not going to address at this time. A feature may not be needed in a particular release of software, or it may occur so infrequently that we deem it unnecessary to ever fix it. Yet we don't close out the issue, so it will keep some level of visibility in our system. Deferred issues may be addressed under a product development project, or if I need to find work for personnel during a slow period.

Issue severity gives management a good impression of the current state of the software being developed. It also serves as a foundation for establishing issue priority. Issue prioritization gives developers a clear understanding of the order in which they should work on their defects.

Despite the risk of "technical radicalism," when issue severity and priority travel hand in hand, managing developers can make informed decisions on work order and can support their choices with documented information.

About The Author: Peter Clark

Peter Clark has twenty years of experience in industrial automation. He currently manages teams working in materials handling, especially baggagehandling systems. A regular columnist on StickyMinds.com, Peter can be reached at pclark@jerviswebb.com.

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

#1

>> I have a doubt regarding this severity and priority. As per the Priority definition, for those bugs which need to be fixed in later releases, we should assign the low priority level.yn, I wouldn't recommend adjusting something to low that doesn't need to be there ('low' often becomes 'we'll never address these bugs at all').If you have a bug that's a medium or high - but not for the current release - then check your bug tracking software to see if you can assign a target milestone/release/version/whatever to it. It may not be a priority for *this* release, but you want to be able to mark it appropriately for future releases so it doesn't fall through the cracks.

#2

hi, I am undergoing the Software Testing Training. I have a doubt regarding this severity and priority. As per the Priority definition, for those bugs which need to be fixed in later releases, we should assign the low priority level. But why do we test such functionalities now, which will come into picture in the later releases ? Can you please explain with relevant example ?Thank you.

#3

Although looking at issue severity seems to make common sense, I've always struggled with large projects with large numbers of issues. In these sorts of systems, minor usability issues, gui issues, issues with phrasing of terms on the screen, etc., seem to get relegated to lowest priority. For the companies I've worked in that bucket might as well be labeled "will never be fixed." The resulting product, while not containing any show stoppers, has rough fit and finish. Sort of like buying a car that runs every day, but who's seats won't adjust and the doors squeek, and, and... you get the picture. Any suggestions on handling this effect?

#4

"There is little point in having a separate issue prioritization scheme if you do not have a single point of issue triage."Peter, I think that pretty much nails it all on the head. There can be different scales and meanings for Severity and Priority. But all involved parties must "buy in" to them and use them appropriately. A third-party group (representatives of the affected groups: Dev., Test, Support, Proj. Mgt., Sales/Marketing, and Users) is needed to sort out the issues and get them corrected in order of importance to the End-User and Company. Otherwise it can be a political mess, and nothing gets done.

#5

I am a big fan of two separate ratings of bugs, and I like this approach. I implemented a similar version of this concept in a past position. When I started as manager of the test team, our biggest waste of time during stress-filled release cycles was arguing about the severity, which ranged from 1(highest severity) to 5(lowest severity). The development team endlessly attacked testers for not understanding the description of each severity level, and for good reason. A release was considered delayed if on the release date there were any severity 1 or 2 issues outstanding. Testers could not distinguish severity 2 and 3 bugs, according to development - they kept giving sev 3's a severity of 2 (and potentially delaying a release). Funny, but the assignment of other severity levels was never questioned! This was clearly a battle where the entire battlefield needed to be destroyed, so we added a column to our bug tables called Priority. The Product Manager for the release would assign the priority, which was either R(required for release) or NR(not required for release). It was the tester's job to assess the impact of the bug on the software itself, not the impact on the customer. The metric driving preparedness for release became the count of bugs with "R" priority. This allowed testers to focus on what they truly understood - how bad off is the program when the bug occurs? The argument over readiness for release changed into a judgement call by the Product Manager, who had the best insight into impact of bugs on customers (and overall accountability for the project). This also allowed us to distinguish typos that were critical (e.g., misspelled company name on a web portal's home page) and crashes that were not (e.g., reserving a particular hotel fo a particular number of guests on a specific day in time crashed the app). Our use of severity was not as good as the one Peter suggests, though - we were just giving the programmer a little more info about the nature of the bug. Peter's point is important - customer impact of a bug does not always drive priority in terms of a fix - and we did not address this in our solution. What we learned, in addition to this, was that the best person to determine customer impact might exist outside the technical team. We had much smoother releases after our change, too. Thanks for a great example of best practices in a fundamental process.

#6

We use a similar method for determining priority. Priority is a numerical value defined as the product of the Severity (1 to 4) and the Urgency (1 to 4), giving you possible values of 1 to 16 (with 1 being "fix first"). Managers are then permitted to adjust that number if necessary. This gives us the ability to ensure high urgency, low severity and low urgency, high severity defects are not swept under the carpet. We use this for defect, problem, and project issue management. Because it's fairly simple and injects additional objectivity into the process, it has been fairly well-received.

#7

I would say if it is an established language to the developer, it is fine to adopt any system. In my practice, severity rating are formed up using 2x2 matrix of exposure and impact of an issue. Hence, the final rating, I would say already depicts the urgency of an issue. Tackling the issues by prioritising the severity level seems sound to me and less complex to the developers.

#8

Good article Peter. We do something very similar. I am the QA manager and we publish our Severity / Priority definitions on our internal Wiki and each developer / tester has a copy at their desk for reference. These are our definitions:SEVERITY / PRIORITY STANDARDSSEVERITYSeverity is the measurement of how seriously the defect affects the application and it's use. 1. Critical (Affects the use of the entire application) eg:o Application cannot be installed o Application Crashes o Calculated algorithms / values are noticeably incorrect o Product Licensing does not work o No Workaround exists 2. High (Affects the use of one or more major features within the application) o Calculated algorithms / values are slightly off o All issues can be reproduced but require more repro steps or more test passes o Work around may be available 3. Medium (Affects functional areas within major features) o Error or omissions in function specs o Broken Links / Navigation o Missing Math References o Input validation checks not being done or are not returning correct results o Serious Usability Issues 4. Low (Cosmetic issues) o Typos, grammatical errors, spelling errors, incorrect fonts PRIORITYPriority sets the ranking in which a defect must be fixed 1. Urgent o Must be fixed immediately. Either a customer is seriously affected or Testing cannot continue. May require a patch release. 2. High o Must be fixed in the current release. May require a patch release. 3. Medium o Fix in the current release. (Can be deferred but may impact users) 4. Low o Only schedule for the current release if time / resources permit (Can be deferred with little user impact) 5. Future Release o Schedule for the next or a future release

#9

Peter, Nice article. I remember one group I worked in had only severity tags. The Priority 1 severity finally ended up with 4 or 5 additional gradations which were in effect priorities. Overly complicated and also failed to give the triage team a clear set of rules to make decisions without a lot of convolution.Your description is straightforward and easy to use!

#10

The example I always use when conducting the ISEB course is something I came across when working in market research as a statistical typist. We did a report for AT&T, but as this was the late 70's its formal name was "American Telephone and Telegraph". We wrote a report for them in which every mention of the company was its name in full. The report came zinging back to us with the request "Please, our corporate identity is now 'AT&T' and we'd like all references in the report to be changed." So if your customer's name on the splash screen is misspelled, it might be a Level 4 severity (cosmetic) but a Level 1 priority (You misspelled the customer's name, darn it! Fix it now!).Severity is the issue reporter's responsibility (with management oversight). Priority is completely the call of management. Unless your project manager is also doing some testing, the test analyst should leave "Severity" blank.

#11

The most important thing in assigning priorities in defect ttracking is to have clear and percise design requirements. If the requirements are not met, then I totally agree with Peter's process of assigning priorties. Sometimes, there is a total disagreement between the developer and the tester of how high the priority should be. The tester should be customer as well as company centered to ensure the highest quality possible and the best customer satification.Basil McDougald QA Analyst Hummingbird, Ltd.

#12

If Risk Based system (e.g. 2x2 matrix) for assigning defect severity is excellent then why I never see it implemented in a defect tracking system. Is it something for product managers and not tester (I am a tester)?

#13

I've found that it is best to assign priority to a release point which helps us measure readiness for individual releases. We use a High, Medium, Low and Defer scheme where high is the issue must be fixed for the next planned release, medium the one to follow, low before final release. Defer - don't fix unless nothing else is left.This also gives the developers a fixed point of reference for when a bug is absolutely required. It can avoid having a developer work on a lower priority issue because it's easier (it does happen).

#14

hi Peter I am a student,learning testing.I had confusion about priority and severity concept but now it is cleared as your language is very easy to understand. But about severity i have still some problem if you kindly give some explanation with example it'll really easy for me to know the concept. thank you

#15

To comment further on Jeff Patton's comment - In queuing systems, a job that stays in the system but doesn't get attention because jobs with higher priorities hog the resources 'starves'. When a job starves, its priority is automatically increased. Thus a low priority job that doesn't get attention will gradually increase in priority until it is addressed. Wouldn't the addition of a starvation mechanism in the bug tracker help to prevent low priority issues never receiving attention?

#16

What I'd like to know is if the concept of priority and severity can be assigned in a similar manner to test cases (and the associated test case result). We have implemented such a concept in QaTraq (the freely available Open Source Test case management tool, www.testmanagement.com). However, I'm not sure if this concept transfers well to test scripts and results. I'd be interested to hear what other think.