Information Robust Dirichlet Networks for Predictive Uncertainty Estimation

Uncertainty in data input to a neural network is a critical aspect that could undermine the network's performance and accuracy. Particularly in applications dependent on high-stakes decisions like healthcare or finance, the ability to effectively interpret model uncertainty carries significant import. Thus, devising a method that enhances the accuracy of neural networks in detecting uncertainty is imperative. Existing approaches to handling uncertainty often depend largely upon adaptation and retraining of neural networks, which can prove both time-consuming and computationally expensive. Also, these methods often fail to explicitly account for adversarial examples, and as such may miss opportunities to enhance model robustness. The current landscape of solutions fails to address these problems adequately, hence the need for improved techniques.

Technology Description

The method described involves an application that generates weights for a neural network, capable of dynamically generating a training regime to more accurately detect uncertainty in data input. The system determines a training loss for the neural network with the objective of minimizing the expected Lp norm of any prediction error. Training is structured so that prediction probabilities follow a Dirichlet distribution. A mathematical shorthand for the training loss is then determined. What sets this neural network apart is its ability to infer parameters of the Dirichlet distribution, essentially enabling it to learn and map distributions over class probability vectors. This anility makes it capable of regularizing the Dirichlet distribution through an information divergence method. As a final component of the process, a maximum entropy penalty is applied to adversarial examples, thereby maximizing uncertainty near the fringes of the Dirichlet distribution.