Defects in Software Testing

Objective of Software testing is to provide independent view of the software to allow the business to appreciate and understand the risks of software implementation. It is an investigation conducted to provide stakeholders with information about the quality of the product or service under test.

Software testing can be stated as the process of validating and verifying that a computer program/application/product:

1. meets the requirements that guided its design and development,
2. works as expected,
3. can be implemented with the same characteristics,
4. and satisfies the needs of stakeholders

A Software Defect / Bug is a condition in a software product which does not meet a software requirement (as stated in the requirement specifications) or end-user expectations (which may not be specified but are reasonable). In other words, a defect is an error in coding or logic that causes a program to malfunction or to produce incorrect/unexpected results.

Software Defects/ Bugs are normally classified as per:

Severity / Impact- Defect Severity or Impact is a classification of software defect (bug) to indicate the degree of negative impact on the quality of software. Severity or impact is broadly categorized in 4 classes.

• Critical (Severity 1): The defect affects critical functionality or critical data. It does not have a workaround. Example: Unsuccessful installation, complete failure of a feature.
• Major (Severity 2): The defect affects major functionality or major data. It has a workaround but is not obvious and is difficult. Example: A feature is not functional from one module but the task is doable if 10 complicated indirect steps are followed in another module/s.
• Minor (Severity 3): The defect affects minor functionality or non-critical data. It has an easy workaround. Example: A minor feature that is not functional in one module but the same task is easily doable from another module.
• Trivial (Severity 4): The defect does not affect functionality or data. It does not even need a workaround. It does not impact productivity or efficiency. It is merely an inconvenience. Example: Petty layout discrepancies, spelling/grammatical errors

Probability / Visibility - Defect Probability indicates the likelihood of a user encountering the defect / bug. If the user easily identifies a bug in rarely used functionality then it will be considered as High priority and vice-versa. Defect probability is broadly classified in 3 categories:

1. High: Encountered by all or almost all the users of the feature
2. Medium: Encountered by about 50% of the users of the feature
3. Low: Encountered by very few or no users of the feature

Priority / Urgency - Defect Priority indicates the importance or urgency of fixing a defect. Though priority may be initially set by the Software Tester, it is usually finalized by the Project/Product Manager.Priority is quite a subjective decision; do not take the categorizations above as authoritative. However, at a high level, priority is determined by considering Business requirement, severity type, probability, available time and resources. Priority can be categorized in following levels:

1. Urgent: Must be fixed in the next build.
2. High: Must be fixed in any of the upcoming builds but should be included in the release.
3. Medium: May be fixed after the release / in the next release.
4. Low: May or may not be fixed at all. 

· 

Related Dimension of Quality – When someone says “This software is of a very high quality.” you might want to ask “In which dimension of quality?”Software has below dimensions of quality.

• Accessibility:The degree to which software can be used comfortably by a wide variety of people, including those who require assistive technologies like screen magnifiers or voice recognition.
• Compatibility:The suitability of software for use in different environments like different Operating Systems, Browsers, etc.
• Concurrency: The ability of software to service multiple requests to the same resources at the same time.
• Efficiency: The ability of software to perform well or achieve a result without wasted energy, resources, effort, time or money.
• Functionality: The ability of software to carry out the functions as specified or desired.
• Installability: The ability of software to be installed inspecified environment.
• Localizability: The ability of software to be used in different languages, time zones etc.
• Maintainability: The ease with which software can be modified (adding features, enhancing features, fixing bugs, etc.)
• Performance: The speed at which software performs under a particular load.
• Portability: The ability of software to be transferred easily from one location to another.
• Reliability: The ability of software to perform a required function under stated conditions for stated period of time without any errors.
• Scalability: The measure of software’s ability to increase or decrease in performance in response to changes in software’s processing demands.
• Security: The extent of protection of software against unauthorized access, invasion of privacy, theft, loss of data, etc.
• Testability: The ability of software to be easily tested.
• Usability: The degree of software’s ease of use.

Related Module / Component– This defect category further classify the defect in module or component where defect lies and reported. This give clear idea to developers where to look for the defect and accordingly run the simulation process.

Phase Detected– This is very important when we consider the System Development life cycle and testing. In testing phase, there are four phases involved viz. Unit testing, Integration Testing, System Testing and Acceptance testing. The phase, in which defect detected and type of defect reported in line with severity, probability has severe consequence in implementation to production.

Phase Injected - Phase Injected can be known only after a proper root-cause analysis of the bug.Phase Injected indicates the phase in the software development lifecycle where the bug was introduced. There are 3 phases of code development viz.

Requirements Development – 

Requirements are the details describing an application's externally-perceived functionality and properties. Requirements are ideally clear, complete, reasonably detailed, cohesive, attainable, and testable. Determining and organizing requirements details in a useful and efficient way can be a difficult effort; different methods and software tools are available depending on the particular project.

If the defect is injected at the time of requirement finalization, then entire code is built on the wrong ground. This code will not be acceptable by the user and will discover at very later stage of testing. The flaws in the requirement definition will be carried forward the in the code design, coding and formulation and deployment. The testers won’t trace the defect being the Business requirement document will the base documentation in the devising the testwares. They are sub-categorized as :

• Business Requirement - The functionality is not as per the desired requirement specified in the Business requirement document or there may be addition in the requirement due to change in the delivery timelines. Defect reported may be on account of difficult to understand the requirement, non-compliance to regulatory standards, contradictory statements in the documents, incorrect statement etc.
• Non-Functional - The requirement though important for the functinality however does not impact the functionality of the application as per the scope document.
• Operational - There may be change in the functionity requirement or modification as per the changing market conditions or operational requirement.

High Level Design&Detailed Design - 

Good Design' could refer to  'functional design' or 'internal design'. Good internal design is indicated by software code whose overall structure is clear, understandable, easily modifiable, and maintainable; is robust with sufficient error-handling and status logging capability; and works as expected when implemented. Good functional design is indicated by an application whose functionality can be traced back to customer and end-user requirements or user stories.

The defects introduced in design level are harder to deal with; developers built exactly what they were told to but unfortunately the designer made some mistakes so there are defects in the design. Unless Programme Managerchecks against the requirements definition, tester will not spot those defects during testing. When such defects are reported, they will be hard to fix because design changes will be required. They are sub-categorized as :

• Functional :If the functionality of the application is not working as per the Software requirement specification(SRS), it is functionality defect.
• Technical :A technical defect exists if the product “departs from its intended design even though all possible care was exercised in the preparation and marketing of the product So, if the product was inadequately assembled or if the materials or components were not consistent with those specified design.

Coding& Deployment – 

'Good code' is code that works, is reasonably bug free, secure, and is readable and maintainable. Some organizations have coding 'standards' that all developers are supposed to adhere to, but everyone has different ideas about what's best, or what is too many or too few rules. It should be kept in mind that excessive use of standards and rules can stifle productivity and creativity. 'Peer reviews', 'buddy checks' pair programming, code analysis tools, etc. can be used to check for problems and enforce standards.

Issues related to coding & deployment are not severe being these are easily spotted and corrected during testing, because tester can see the product does not meet its design specification. Coding defects can be broadly summarized as :

• Build/Code : A failure or flaw in a program that produces undesired or incorrect results. It’s an error that prevents the application from functioning as it should.
• Configuration :System software/application need to be configured at compile time to tailor it with respect to a broad range of supported hardware architectures and application domains. If the there is no such compilation resulting to error in application performance.
• Documentation: A defect in a user manual, reference manual or on-line help that gives incorrect information or fails to give information relevant to a problem. Requirement is not clear to the reviewer. Also includes ambiguous use of words – e.g. Like, such as, may be, could be, might etc.

Deploy-Test environment - 

Test environments differ from production environments in terms of operating systems, patch levels, software versions, configuration, etc. The wider the gap between the test and production environments, the greater the chance of an application failing after being deployed or a defect leaking into live systems. Further, configuration changes made in response to errors often go untracked, causing errors when applications are moved into production. They are sub-classified as :

1. Connectivity/network : Internal or external to application interfacing error, Incorrect handling of passing parameters, Incorrect alignment, incorrect/misplaced fields/objects, un friendly window/screen positions
2. Hardware/Operating system-Application framework: Hardware issues, memory leaks, Windows-type interfaces, client-server and distributed applications, data communications, enormous relational databases, and sheer size of applications will result in defect related to Operating system/application framework.

Test Defects - 

These are the defects generally reported on account of Testing error. Mis-understanding or mis-communication about the testing requirement viz. test scripting error. They are sub-classified in :

• Data - Local to Application: Incorrect data population / update in database. Error in the database schema/Design.
• Duplicate Defects: Defects which are linked to either of previously logged defect and resolution to that defect will automatically resolve the issue.
• Mainframe Data: Defects reported while connecting to Mainframe through web based applications.
• No Fault found: While re-testing the defect at devlopers end, if the defect is not reported on the code, such defect logged can be classified as 'No Fault found'.
• Reference data: Error reported in connection with reference data on account of incorrect data, inconsistent data, matching or linking or batch/real time connections.
• Script Error: Inadequate/ incorrect/ misleading or missing comments in the test script. Missing or Inadequate or irrelevant or ambiguous functionality in scripting.
• Unreproducible: Bug is not reproducible, in essence we are indicating that at the present time, insufficient detail is known about the whole set of circumstances. The bug is able to be reproduced, but we aren't able to do it.
• User error: The requirements have been performed falsely. Wrong procedure observed by user while testing.