Signal Confidence Limits from a Neural Network Data Analysis

Abstract

This paper deals with a situation of some importance for the analysis of experimental data via Neural Network (NN) or similar devices: Let $N$ data be given, such that $N=N_s+N_b$, where $N_s$ is the number of signals, $N_b$ the number of background events, both unknown. Assume that a NN has been trained, such that it will tag signals with efficiency $F_s$, $(0<F_s<1)$ and background data with $F_b$, $(0<F_b<1)$. Applying the NN yields $N^Y$ tagged events. We demonstrate that the knowledge of $N^Y$ is sufficient to calculate confidence bounds for the signal likelihood, which have the same statistical interpretation as the Clopper-Pearson bounds for the well-studied case of direct signal observation. Subsequently, we discuss rigorous bounds for the a-posteriori distribution function of the signal probability, as well as for the (closely related) likelihood that there are $N_s$ signals in the data. We compare them with results obtained by starting off with a maximum entropy type assumption for the a-priori likelihood that there are $N_s$ signals in the data and applying the Bayesian theorem. Difficulties are encountered with the latter method.