Risks in Software Testing

Testing is inherently a risk-based activity. In software testing, RISKs are the possible problems that might endanger the objectives of the project stakeholders. It is the possibility of a negative or undesirable outcome. As testing is the last part of the project, it’s always under pressure and have time constraint. To save time and money you should be able to prioritize your testing work. How will prioritize testing work? - Exhaustive testing is impractical for most applications under development. Exceptions include applications that support very high risk processes, such as air traffic control, nuclear station operations, defense systems, and so on. The project team must design a test strategy that utilizes a balance of testing techniques to cover a representative sample of the system in order to minimize risk while still delivering the application to production in a timely manner.

Test manager’s responsibility is to determine the test methodology to achieve the greatest level of confidence in the application under development. Risk is a major driver in the test planning activity. A risk has some probability between 0% and 100%; it is a possibility, not a certainty. When conducting risk analysis, two major components are taken into consideration:

• The probability that the negative event will occur.
• The potential loss or impact associated with the event.

The test manager must determine the appropriate amount of testing to perform based upon the risks associated with the application. These risks can arise from the newness and reliability of the technology being used, the nature of the application, or from the priority of the business functions under test. The amount of testing that should be performed is directly related to the amount of risk involved.

There are numerous testing approaches used by Test Managers while assessing risk in the Testing like Analytical, Model Based, Methodical, Standard compliant, Dynamic or exploratory and regression-averse. Understanding the risk-based nature of testing is the key to dealing with the chronic problem related to application testing.

Some of the primary testing risks include:·

Not Enough Training/Lack of Test Competency

Competencies are the knowledge, understanding, practical and thinking skills needed to perform effectively to the standards required in employment. They are identified and demonstrated through sets of behaviors that encompass the skills, knowledge, abilities and personal attributes that are critical to successful role accomplishment. Developers are often trained to design and code the application however tester are not. The majority of IT personnel have not been formally trained in testing, and only about half of full-time independent testing personnel have been trained in testing techniques. This causes a great deal of misunderstanding and misapplication of testing techniques. Many tester confirms that they learnt testing technique on the job while carrying out testing on the projects.
Test Environment

The work environment is not conducive to effective and efficient testing. A testing environment is a setup of software and hardware on which the testing team is going to perform the testing of the newly built software product. This setup consists of the physical setup which includes hardware, and logical setup that includes Server Operating system, client operating system, database server, front end running environment, browser (if web application), IIS (version on server side) or any other software components required to run this software product. This testing setup is to be built on both the ends – i.e. the server and client. Sometime issues reported in Internal or external to application interfacing error, Incorrect handling of passing parameters, incorrect alignment, incorrect/misplaced fields/objects, and unfriendly window/screen positions. Hardware and operating system issues like Windows-type interfaces, client-server and distributed applications, data communications, enormous relational databases, and sheer size of applications will result in defect related to Operating system/application framework.

Test environments differ from production environments in terms of operating systems, patch levels, software versions, configuration, etc. The wider the gap between the test and production environments, the greater the chance of an application failing after being deployed or a defect leaking into live systems. Further, configuration changes made in response to errors often go untracked, causing errors when applications are moved into production.
New technology

The new focus on client/server, Intranet, and Internet applications has introduced even more risk to the test process. These multi-tiered systems are more complex than traditional mainframe systems, and therefore have a higher level of risk associated with their testing and implementation. Security, performance, availability, complex component integration, and a team new to the technology are just a few of these new risk factors. With advancement of technology the developers have privilege in the learning the new technology the coding and designing concepts. Training programmes, walk-though sessions, ample access to literature made available to developers for upgrading their knowledge base, whereas the tester carry on testing on the earned skills-sets from on-job training.
Us versus Them Mentality

This common problem arises when developers and testers are onopposite sides of the testing issue. In many organizations, testing is an Us Vs. Them mentality. Testers are treated like Bad guys. Each whether developer or tester feels that it is “out to get” the other. Often, the political infighting takes up energy, sidetracks the project, and accomplishes little except to negatively impact relationships. Some developers write sloppy coding initially and let the testing reports the defects. Such defects are then cured, and code released for production with credit to developers for high quality system.
Requirement study
Requirements are the details describing an application's externally-perceived functionality and properties. Requirements are ideally clear, complete, reasonably detailed, cohesive, attainable, and testable. Determining and organizing requirements details in a useful and efficient way can be a difficult effort; different methods and software tools are available depending on the particular project. If there is no clarity on the testing scope or the business requirement document, tester will be at confused status because of difference in Business Requirement Document and scope definition during business discussion or there may be addition in the requirement due to change in the delivery timelines. Defect reported may be on account of difficult to understand the requirement, non-compliance to regulatory standards, contradictory statements in the documents, incorrect statement etc.
Code Design

Good Design' could refer to ‘functional design' or 'internal design'. Good internal design is indicated by software code whose overall structure is clear, understandable, easily modifiable, and maintainable; is robust with sufficient error-handling and status logging capability; and works as expected when implemented. Good functional design is indicated by an application whose functionality can be traced back to customer and end-user requirements or user stories. Since the testers lack the adequate training opportunities toward knowledge up gradation required for testing an application, defects reported may get in-appropriately categorized resulting to blame on the tester on in-adequate quality in testing.
Lack of Test Tools

Testing is normally treated as art rather than an engineering discipline. Management prefers to labour intensive technique instead use of testing tools. IT management may have the attitude that test tools are a luxury. Manual testing can be an overwhelming task. Although more than just tools are needed, trying to test effectively without tools is like trying to dig a trench with a spoon. Sometimes tools are available but integrating them with actual testing is challenge for the test manager. When system grows larger, testing without use of tools becoming logistically impossible.
Lack of Management Understanding and Support of Testing

Testing is always looked as secondary activity whereas Development is classified as Creative or desirable endeavor. Support for testing must come from the top, otherwise staff will not takethe job seriously and testers’ morale will suffer. Management support goes beyond financial provisions; management must also make the tough calls to deliver the software on time with defects or take a little longer and do the job right.
Lack of Customer and User Involvement

Users and customers may be shut out of the testing process, or perhapsthey don’t want to be involved. Users and customers play one of the most critical roles in testing; making sure the software works from a business perspective. Here Customers are those who sponsor the development of application so as to ensure that objectives are met in the designing application whereas Users of application want ease of use and operability.
Not Enough Schedule or Budget for Testing

This is a common complaint. The challenge is to prioritize the plan to test the right things in the given time. When the schedules are established, time is allotted for testing however many a time either developers usurp the testing timelines for development phase or there is inadequate staffing in testing work load. Testers are placed under excessive stress when they are categorized as the group which stands in Go-Live timelines for application going in production. Since testing comes in last phase of SDLC, testers are blamed for delays in timelines despite developers have over utilized schedule. Due to stressed timelines, there may be possibility of inadequate test coverage. Show-stoppers, delayed response of the system due to connectivity issues, delay in running batch runs etc. may impact the Project Testing plan.

There is also quite disparity in management attitude towards the allocation budgets for the development and testing.
Over Reliance on Independent Testers

Sometimes called the “throw it over the wall” syndrome, developers know that independent testers will check their work, so they focus on coding and let the testers do the testing. Unfortunately, this results in higher defect levels and longer testing times. Traditionally, once developers are finished with coding the application; the system is handed over to testing team. There is no effective communication between developers and testing team to understand the repeated testing defects and issues reported in testing an application.
Rapid Change

In some technologies, especially Rapid Application Development (RAD), the software is created and modified faster than the testers can test it. This highlights the need for automation, but also for version and release management.
Testers are in A Lose-Lose Situation

Testers are always on the loser end in the entire system life cycle. If thorough testing is done and defects are reported which has resulted in delayed implementation, they are blamed for delaying the project. Conversely, if the testers do not find the critical defects and approve the implementation and typical problems reported in the production, they are blamed for poor quality. Some developers try to isolate designing from the defects. The defects reported are considered as tester’s responsibility and if the same flow to the production, the testers are held responsible for inadequate code testing. Tester are often held responsible for delay in fixing the defects.
Having to Say “No”

Having to say, “No, the software is not ready for production,” is the single toughest dilemma for testers. Nobody on the project likes to hear that and frequently testers succumb to the pressures of schedule and cost. Many a times tester face the challenge of telling the developers and customer about the in-adequacy of code, status of applications, defects and recommendation on implementing in production.
New developmental processes

Along with these new delivery platforms come new methods for software development. Project teams have moved away from traditional “waterfall” methods, and follow a much more iterative approach to development and delivery. An integral part of this iterative approach is that testing activities are more integrated across the development cycle than before. Traditional system testing is not just conducted at the end of development, but may be conducted multiple times throughout the project for major components, and then completed once all development is complete. Based on the risk assessment of the risks associated with the test process, changes to the test process may be needed. For example, if the testers lack training in the test process, that training should be provided prior to testing. If the training cannot occur then extra supervision would be required. If inadequate test resources and schedule time is not available the number of tests may need to be reduced. Risk analysis will determine the risks associated with the software being tested and performing the test process. Risk analysis should determine the magnitude of the risks, and prioritize them in importance. Test planning will then consider those risks in developing the Test Plan.
Programmatic Risk

These are the external risks beyond the operational limits. These are all uncertain risks are outside the control of the program like running out of funds, market development, change in product and market strategy, changes in government rules and regulations.
Technical risks

Technical risks generally lead to failure of functionality and performance like Continuous changing requirements, no advanced technology available or the existing technology is in initial stages, Product is complex to implement, difficult project modules integration, failure to identify complex functionalities and time required to develop those testing scripts.
No Say on the Big bang Approach

Sometimes Project Management decides to adopt Big Bang approach and devise schedule accordingly, testing teams in such scenario are hardly taken in consideration when they have bigger role to play. When a new system needs to be implemented in an organization, there are three different ways to adopt this new system: The big bang adoption, phased adoption and parallel adoption. With the big bang adoption, the switch between using the old system and using the new system happens at one single date, the so-called instant changeover of the system.

Everybody starts to use the new system at the same date and the old system will not be used anymore from that moment on. There is real challenge to the testing team on ensuring appropriate functioning of new system. Testing team has to learn all applications involved, their interfaces, consolidation of all system etc. Tester has to stress on defects present at the interfaces of components are identified well in advance, identify the defect impact whether defect is in component or interface. There is high probability of missing some critical defects which might surface in production. With timelines its too stressful for testing team to cover up all the integration. To be more efficient and accurate, testing team has to take care in defining the user-like workloads for creating realistic scenarios in exercising the environment to give confidence that the integrated environment will work as expected for the target customers.