12 - Metacognition with Neural Network Verification and Repair Using Veritex

Summary

This chapter presents a metacognitive AI approach via formal verification and repair of neural networks (NNs). We observe that a neural network repair is a form of metacognition, where trained AI systems relearn until specifications hold. We detail Veritex, a tool for reachability analysis and repair of deep NNs (DNNs). Veritex includes methods for exact and over-approximative reachability analysis of DNNs. The exact methods can compute the exact output reachable domain, as well as the exact unsafe input space that causes safety violations of DNNs. Based on the exact unsafe input–output reachable domain, Veritex can repair unsafe DNNs on multiple safety properties with negligible performance degradation, by updating the DNN parameters via retraining. Veritex primarily addresses the synthesis of provably safe DNNs, which is not yet significantly addressed in the literature. Veritex is evaluated for safety verification and DNN repair. Benchmarks for verification include ACAS Xu, and benchmarks for the repair include an unsafe ACAS Xu and an unsafe agent trained in deep reinforcement learning (DRL), where it is able to modify the NNs until safety is proven.