System Test Architecture Evaluation: A Probabilistic Modeling Approach

Mahmoud Efatmaneshnik, Shraga Shoval, Keith Joiner

Research output: Contribution to journalArticlepeer-review

10 Scopus citations

Abstract

In this paper, we study the effects of testing architecture on system quality using a probabilistic model of unit testing. The Markovian view of the testing process relates unit quality and test quality to the expected unit/system quality, and to the expected number of tests. The model is based on replicating testing where the test is replicated only after a fail test outcome. A set of equations are generalized for many component systems. The study enables the costs and associated benefits for particular groups of components to be considered for module testing at different levels of system hierarchy. Simulation results show that the selection of an appropriate testing architecture and modular architecture can greatly enhance the efficiency and effectiveness of system testing. The model is applied to several testing architectures and test patterns to illustrate the tradeoff comparisons that can be made between different system architectures in terms of the overall test cost and the resulting system quality. Several heuristics are derived to assist in planning tests of complex systems for optimizing quality and cost before systems are built. This Markovian model of testability in systems shows promise for use from the modular unit build-level in engineering manufacture development through to strategizing portfolio-level integration and information assurance testing of new capability projects with legacy systems in a family-of-systems. The early abstract testability work reinforces the benefits of iterating testing as early and often as possible. Following trials to validate the model in different contexts and to refine the interface with systems engineers, this testability model could be added to other design for six-sigma tools and techniques to better enable systems engineering practitioners to tradeoff on testability options well before testing starts.

Original languageEnglish
Article number8667376
Pages (from-to)3651-3662
Number of pages12
JournalIEEE Systems Journal
Volume13
Issue number4
DOIs
StatePublished - Dec 2019

Keywords

  • Latent defect
  • Markov process
  • modular testing
  • probabilistic modeling
  • residual defect
  • system architecture
  • testability
  • testing architecture

Fingerprint

Dive into the research topics of 'System Test Architecture Evaluation: A Probabilistic Modeling Approach'. Together they form a unique fingerprint.

Cite this