Feature-Based Likelihood Ratios for Forensic Science: Combining Neural Networks with Bayesian Probability Calculus

Abstract

In forensic science, when crime-scene evidence (CSE) and suspect-related evidence (SRE) is present, it is customary to report on the value of this evidence in the form of a likelihood ratio (LR). The LR can be calculated as the probability of CSE given SRE divided by the probability of CSE given that it was generated by a randomly selected person from an alternative culprit population. In forensic science, this is known as a feature-based LR, and intuitively the LR contrasts "similarity" by "typicality". Since it is generally a problem for feature-based LRs to find appropriate models for the data, one either resorts to score-based LRs or to adjusting the feature-based output post-hoc to well-calibrated output. Either way, the above definition of the LR is broken and interpretation of the LR as similarity between CSE and SRE divided by typicality of CSE is destroyed. Here, we report on progress in obtaining instantly well-performing feature-based LRs using gradient descent in combination with Bayesian probability theory to train a two-level model, the main LR model in forensic science for describing distributions of continuous data. For a dataset of laser-ablation inductively-coupled-plasma mass-spectrometry measurements on glass fragments from forensic casework, we show that our best model on validation data yields much better calibrated feature-based LRs on the test set when compared to state-of-the-art feature-based LR systems trained on the same type of data, and that it improves a factor of 4.5 on average on $C_{\mathrm{llr}}$. For LRs interpretable in terms of "similarity" contrasting "typicality", this is a major advancement. However, a state-of-the-art LR system still performs a factor of 1.5 better on $C_{\mathrm{llr}}$ for this data. We also present future plans to close this remaining gap. In order to facilitate collaboration, we have put relevant code on GitHub.