Compact Functional Test Pattern Generation for DNNs Using Evolution Strategies
Paper Title: Compact Functional Test Pattern Generation for DNNs Using Evolution Strategies
Link to Paper: https://doi.org/10.1109/VTS69484.2026.11563375 (VTS 2026)
Date: 2026
Paper Type: Automatic Test Pattern Generation, Deep Neural Networks, Evolution Strategies, Black-Box Testing
Short Abstract: This paper proposes a black-box functional testing framework that detects hardware faults in DNNs exclusively through their observable effects on network outputs. Instead of the typical two-phase pipeline (generate patterns first, compact them second), the method uses Covariance Matrix Adaptation Evolution Strategy (CMA-ES) to jointly optimize fault coverage and test-set size in a single step. This is important because the compactness criterion makes the objective non-differentiable, ruling out gradient-based approaches. Across five architectures including CNNs, ResNets, a Vision Transformer, and a segmentation network, the method achieves on average 98.7% fault coverage while using 9× fewer test patterns than a gradient-based baseline, with up to 16.3× compaction in the best case.