Artificial Neural Network Applications for Software Reliability Prediction

By Manjubala Bisi and Neeraj Kumar Goyal

Artificial neural network (ANN) has proven to be a universal approximator for any non-linear continuous function with arbitrary accuracy. This book presents how to apply ANN to measure various software reliability indicators: number of failures in a given time, time between successive failures, fault-prone modules and development efforts. The application of machine learning algorithm i.e. artificial neural networks application in software reliability prediction during testing phase as well as early phases of software development process is presented as well. Applications of artificial neural network for the above purposes are discussed with experimental results in this book so that practitioners can easily use ANN models for predicting software reliability indicators.

• Language: English
• Publisher: Wiley
• Release date: Sep 18, 2017
• ISBN: 9781119223924

Book preview

Chapter 1

Introduction

Computer systems controlled by software play a major role in day-to-day activities of the modern society. Applications of software has now reached diverse fields such as home appliances, hospital health care unit, industrial control, nuclear reactor plant, air traffic control, aircraft, shopping and many more. Many commercial and government organizations depend on software to increase their efficiency and effectiveness. The demand for application of software is increasing day-by-day. The failures in the software may lead to minor inconvenience and customer dissatisfaction, economic loss, loss of life and total system failure. Some of the tragic consequences of software failures are: i) Therac-25 accident: It is a computer controlled radiation therapy machine. Software failures led to death of six patients due to therapeutic overdose ii) Obama care health insurance portal: Due to software failures, this portal shallows one third insurance applications which leads to customer dissatisfaction iii) Social network: Due to automatic generation and recommendation of add friend request, private data of six million members of the network was made public iv) 22 people wrongly arrested in Australia due to failures in NZ million courts computer system v) Telephone outage: After changing three lines of code in a signaling program which contains millions lines of code, the local telephone systems in California stopped. Therefore, software professionals are constantly trying to develop software which are not only functionally attractive but also safe and reliable.

According to ANSI (American National Standards Institute), software reliability is defined as the probability of failure-free software operation for a specified period of time in a specified environment. It is also an important attribute of software quality, together with functionality, usability, performance, serviceability, capability, install ability, maintainability and documentation.

Software reliability prediction is an important step in the development of reliable software. It aids to develop reliable software within given time and budget constraints. Software reliability prediction is carried throughout software development process. Different terminologies such as fault, error, failure, bug, mistake, malfunction and defect are associated with software reliability problems. A fault is created when programmer makes some error during requirement, design, coding and testing phase. Software contains fault, if for some input, the output is incorrect. It can be removed by correcting the erroneous part of the software. Failure occurs in software, if the faulty part is executed. Wrong input, incorrect printing of output and misinterpretation of output may cause failures in software. The failures are classified as transient, permanent, recoverable and unrecoverable depending upon severity. Transient failures occur only for certain input values while permanent failures occur for all input values. When recoverable failure occurs, the system recovers with or without operator intervention. The system may have to be restarted when unrecoverable failure occurs.

Software reliability measurement is gaining importance in software industry. So, various techniques have been developed to predict software failures with time, which characterize the software failure behavior. The existing software reliability prediction models are divided into two categories. First category of methods utilize failure data recorded during testing to estimate current reliability of the software and assess the efforts required to achieve certain level of software reliability before releasing it. These models generally model the software reliability growth considering impact of testing and debugging efforts. Second category of models assess development efforts to achieve a certain level of reliability utilizing software quality indices. These models are generally called early software reliability prediction models as they try to assess software reliability before testing phase allowing better planning and utilization of resources. These models allow to monitor software reliability throughout development process and provide feedback at right time during development resulting in lesser development cost and achieving quality and reliable software within budget and time constraint. However, early software reliability prediction models show larger prediction errors, in general, compared to growth models as the testing data provides more information about software reliability built in the system.

Predicting cumulative number of failures and time between successive software failures help software professionals for determining release time of software. If the number of failures are not increasing after testing the software for a certain period of time and the time between successive failures are very high, then software managers can stop testing and release the software. Large and complex software systems are developed by integrating various software modules. All modules are neither equally important nor do they contain an equal amount of faults. Therefore, researchers started focusing on the classification of software modules as fault-prone (FP) and not fault-prone modules (NFP). Fault-proneness of a module can be determined from software quality metrics. Some metrics are interrelated and redundant with each other. Therefore, determination of important software metrics is required to predict fault-pone modules. Software development efforts need to be accurately predicted during early phases of software development process using software quality metrics affecting efforts known as effort multipliers. It helps to produce reliable software within time and budget constraint. If a software is released with higher reliability than the desired reliability, additional testing resources are required which reduces the profit margin for the developer and delay in development schedule. If a software is released with lower reliability than the desired reliability, then the maintenance cost will be more than testing cost.

This book presents prediction of above software reliability applications: number of failures in a given time, time between failures, classification of fault-prone modules and development efforts prediction. The major content of this book are as follows:

Predicting Cumulative Number of Software Failures in a Given Time: In this work, two artificial neural network (ANN) models are presented to predict cumulative number of failures in software. These models use single input (testing time), single output (failure count) and experimented with multiple data sets. The results show better prediction accuracy. These models help to determine time to stop testing and release time of software.

Predicting Time between Successive Software Failures: In this work, two ANN based models using time series approach are presented to predict time between successive software failures. These models use multiple delayed inputs, single output and their application on multiple data sets also show better prediction accuracy.

Predicting Software Fault-Prone Modules: In this work, ANN models are presented to predict fault-prone modules in software. These models are configured as multiple inputs (software quality metrics) and single output to predict fault-prone modules. Sensitivity analysis and principal component analysis (PCA) are studied for the purpose of input dimension reduction and PCA is found to be providing better accuracy.

Predicting Software Development Efforts: In this work, ANN models are presented to predict software development efforts from effort multipliers. Effect of optimizing ANN architecture using genetic algorithm (GA) on prediction accuracy is studied and found to be providing better accuracy.

This chapter explains the overview and necessity of software reliability. The overview of software reliability modeling and its limitations are also described in this chapter. This chapter defines objectives of this book. Finally, organization of the book is presented in this chapter.

The rest of this chapter is organized as follows. Overview of software reliability modeling and its limitations are discussed in Section 1.1. The objectives and overview of the book are presented in Section 1.2. The organization of book is presented in Section 1.3.

1.1 Overview of Software Reliability Prediction and Its Limitation

In this section, an overview of software reliability prediction and its limitations are presented.

In software industry, time, budget and testing resources are limited. So, it is necessary to develop reliable software within budget and time constraint. Software reliability is hard to achieve, because the complexity of software tends to be high. Software reliability is measured to determine quality of current product, to predict quality of product and to improve quality of a product. However, it is a difficult problem to measure software reliability because we do not have a good understanding of the nature of software. It can be measured as estimation, prediction and certification of software reliability. Software reliability estimation is used to quantify current reliability level. Some statistical inferences are applied to failure data obtained during software testing to determine whether the applied reliability model is good enough for the current software project. Software reliability prediction is used to forecast the future reliability. The future failure behavior is predicted from present and past failures. Software reliability certification ensures a certain reliability level.

Software reliability modeling and prediction is an important step to develop reliable software. Reliability prediction is carried throughout the software development process. Reliability prediction carried out before testing phase (requirement, design and coding phase) is called early reliability prediction. Testing data are not available during these phases. Therefore, using some software development characteristics, software project, use/test characteristics and fault statistics obtained through technical review of requirement, design and coding phase, reliability is predicted. Early reliability prediction models provide early identification of cost overruns, software development process issues and optimal development strategies. A number of early reliability prediction models exist in literature (Agrest 1992, Rome 1992). Reliability prediction carried out during testing is called reliability growth model. These models use the testing data to predict reliability. During testing phase, failures occur and faults that caused failures are identified and removed. The reliability of software increases during testing phase as more and more faults are removed. The reliability improvement phenomenon is called as reliability growth. Software reliability growth models (SRGMs) are useful for timely testing process control and help software project managers to make timely and cost-effective decisions, adapting testing strategies and resource allocation policy. A number of SRGMs exist in literature (Goel 1979, Jelnski 1972, Pham 2000, Littlehood 1973, Chaterjee 1997).

Existing software reliability growth models are classified into two major groups such as parametric and non-parametric models. Parametric models are based on certain assumptions about the software development process, software use/test characteristics and software failure process. Some of the assumptions of parametric models may not be valid in real situations. Different models are based on different assumptions. The software development process also varies from one software project to another. Therefore, the predictive capability of models differ from each other. The predictive capability of a model also varies from one software project to another software project. A single model which can be applied to all cases with acceptable prediction accuracy does not exist in literature.

To overcome limitations of parametric models, non-parametric models such as support vector machine (SVM), artificial neural network (ANN), fuzzy logic and genetic algorithm (GA) are used for software reliability assessment. The non-parametric models are not bounded by unrealistic assumptions and use similar project failure data to build models and to predict reliability for future project. Among several non-parametric models, artificial neural network (ANN) has better predictive capability and wide applicability such as number of failure prediction, time between failure prediction, fault-localization, testing effort and cost effort prediction and in so many fields. ANNs are generally able to capture the unknown complex relationship that exist between input and output.

There exists a wide number of ANN based models for software reliability assessment (Karunanithi 1992a, Neumann 2002, Lyu). The efficiency of ANN models depend upon the architecture of ANN. As the architecture becomes complex, the computational complexity increases and convergence speed decreases. Existing ANN based models use different architecture for different data sets. Any user who wants to apply these models first determine the architecture of the data set and then apply the model on it. As a result, the predictive capability of ANN based models also vary from one data set to another. Therefore, there is a need to develop ANN model which gives consistent result for different data sets so that in real cases, user can use the models without doing lots of experimentation to find ANN architecture.

1.2 Overview of the Book

Overview of the book is presented in this section. In Section 1.2.1, predicting cumulative number of software failures in a given time is presented. Section 1.2.2 describes prediction of time between successive software failures. In Section 1.2.3, prediction of software fault-prone modules is given. Section 1.2.4 describes the prediction of software development efforts.

Numerous factors like software development process, software development organization, software test/use characteristics, software complexity, and nature of software faults and possibility of occurrence of failure affect the software reliability behavior. These factors represent non-linear pattern. As ANN models normally approximate any non-linear continuous function, more attention is given to ANN based methods now-a-days. ANN has better predictive capability for software reliability assessment. Realizing importance of ANN for software reliability prediction, the objectives of this book are formulated as:

Design a fixed and simple architecture based ANN model to predict cumulative number of failures in software providing good accuracy to a number of data sets.

To predict time between failures in software using ANN with time series approach with good accuracy using a simple and efficient approach.

Develop an ANN model to predict software fault-prone modules with good prediction accuracy.

Design an ANN based model to predict software development efforts from effort multipliers and study the effect of dimension reduction, ANN architecture optimization to achieve higher accuracy.

1.2.1 Predicting Cumulative Number of Software Failures in a Given Time

There exist no software which is free of faults, independent of its size and complexity. Software is written by human beings. Due to complex human nature, errors will always occur during software development process. Mistakes are made by software developers during different phases of software development process due to large problem complexity, wrong or incomplete requirement specification and lack of time. When humans make such type of mistakes, faults are introduced in the software which lead to make incorrect piece of software. When the faulty software is executed, there is a deviation from the actual behavior producing incorrect result. This is known as software