arXiv Science⌕ Search

arXiv subjects

David Evans

Publications and source records attributed to David Evans.

At least 19 recordsLinked to original sources
arXiv2026

A SoK for SoCs: Reading the TI Leaves on AI for Cyber Threat Intelligence Generation and Sharing

Cyber Threat Intelligence (CTI) is essential for defending mission-critical infrastructure, yet the process of transforming raw attack evidence into shareable CTI remains fragmented and understudied. We conduct a literature survey of academic papers, organizing the CTI lifecycle into three stages: Threat Data Collection, CTI Generation and Sharing, and CTI Consumption. The first and third stages are well represented in the literature, whereas only a small number of papers address CTI Generation and Sharing. To learn how this stage is practiced, we survey practitioners across multiple organizations who routinely generate and share CTI. They describe a largely manual process with four recurring challenges: preventing the exposure of sensitive information, extracting indicators from noisy attack data, correlating observed behavior with standardized tactics, techniques, and procedures (TTPs), and translating CTI into the formats that sharing platforms require. Using the insights from the practitioner survey, we divide the CTI Generation and Sharing stage into four steps: Intelligence Extraction, Normalization and Enrichment, Codification, and Distribution. We then conduct pilot studies that probe the feasibility of current Large Language Models (LLMs) for each step. The pilot studies show that LLMs can assist an analyst in each of the four steps. However, the models recover only a fraction of the indicators the evidence contains, struggle to ground every claim in the supplied evidence, and do not judge what keeps shared intelligence useful to its recipients. Each step therefore requires expert supervision. Based on these observations, we derive three research directions for automating the production of shareable intelligence.

cs.CR↗
arXiv2026

Complexity-Scalable Direct Geolocation and Cancellation of Terrestrial GNSS Jammers: Single-Satellite and Multi-Antenna Experiments in Low Earth Orbit

Monitoring the radio-frequency (RF) spectrum from space imposes demanding requirements to satellite platforms in terms of communication bandwidth and computational resources, which are necessary for the downlink, the storage, and the processing of high-throughput I/Q samples. This paper analyzes in depth the quasi-direct geolocation (QDG) as a technique to enable the exploitation of satellites of opportunity in low Earth orbit (LEO) to sense the spectrum in the bands of global navigation satellite systems (GNSS). This is a technique of passive RF geolocation and consists of an ensemble of signal processing algorithms, which compress the I/Q samples and process the compressed data through fast delay-Doppler shift matching and interferometry in a quantized time-frequency domain. These algorithms speed up the exhaustive search of multiple RF sources in the position domain. The efficiency gain addresses the bottleneck that prevents the employment of satellites, which are limited in downlink capacity and on-board computational power. These satellites are usually constrained in size, weight and power (SWaP) and represent most of the spacecrafts in LEO. The ability to exploit assets as such for the geolocation of terrestrial GNSS jammers in near real time is instrumental the performance of a multi-constellation GNSS RFI monitoring system. The present work describes the mathematical framework and precision bounds, introduces single- and multi-antenna uses cases, combines different compression methods, and evaluates the geolocation accuracy with real data. The I/Q samples were collected by a repurposed GNSS reflectometry (GNSS-R) satellite, OPS-SAT PRETTY, in a dedicated test session during Jammertest 2025. The experimental results demonstrate the capability to geolocate GNSS jammers with different signal-to-noise ratios (SNR) with extremely high compression ratios.

eess.SP↗
arXiv2026

When RLHF Fails: A Mechanistic Taxonomy of Reward Hacking, Collapse, and Evaluator Gaming

RLHF evaluation should track how failures emerge, where they localize, and which warning signals appear before external quality degrades. We study this problem with a compact RLHF pipeline built for this paper, including PPO, DPO, uncertainty-penalized PPO (UP-PPO), reward-model uncertainty, approximate policy drift, diversity and repetition diagnostics, and two external LLM judges. Rather than treating reward hacking as a single terminal event, we classify matched checkpoint and prompt-level transitions by the directions of learned reward R_phi, judge scores R_dag and R2_dag, and their average R_dag. The main empirical findings are that aggressive PPO produces the clearest localized reward-hacking signal, UP-PPO reduces but does not eliminate that signal, row-level diagnostics reveal failures hidden by checkpoint averages, and pre-transition features partially anticipate future localized reward hacking. The central conclusion is methodological: RLHF failures are training dynamics that can be classified, localized, and partially anticipated, not only final-model pathologies. The repository is available at github.com/zabahana/rlhf-failure-modes-diagnostics. The pipeline is also deployed as a live interactive web demo for model comparison and diagnostic views at rlhf-failures.zelalem.ai.

cs.LG↗
arXiv2026

Evolution of Log-Based Detection Rules in Public Repositories

Log-based detection rules remain central to modern security operations, encoding domain expertise that analysts iteratively refine to balance detection coverage against alert volume. Yet while prior work has examined the evolution of network intrusion detection signatures, the longitudinal behavior of log-based detection rules has received little empirical study. We present the first longitudinal analysis of detection rule evolution across two widely used repositories: the community-driven Sigma project and the curated Splunk Security Content (SSC). To compare rule versions based on detection logic rather than surface syntax, we introduce a predicate graph intermediate representation that canonicalizes the logical structure of a rule, together with a tree alignment procedure for analyzing changes across revisions. We apply this method to 6,859 rule histories from Sigma and SSC and find that roughly 56% of rules undergo at least one revision on detection logic. Across rule lifetimes, evolution is predominantly non-monotonic, with over half of rules both adding and removing clauses over time. We further observe recurring reversions, indicating that changes are often revisited rather than strictly accumulated. Combining structural analysis with LLM-based inference and human validation of operational intent shows that roughly a quarter to a third of rules alternate between expanding coverage and reducing false positives, rather than converging toward a stable form. Together, these results reveal that detection rule evolution in public repositories reflects ongoing operational trade-offs rather than steady convergence. Our study raises questions about why rules change the way they do and supports research towards better processes for devising and deploying security rules.

cs.CR↗
arXiv2026

NoisyCoconut: Counterfactual Consensus via Latent Space Reasoning

This paper presents NoisyCoconut, a novel inference-time method that enhances large language model (LLM) reliability by manipulating internal representations. Unlike fine-tuning methods that require extensive retraining, NoisyCoconut operates directly on model representations during inference and requires no retraining. Rather than training models to reason in latent space, we inject controlled noise into latent trajectories to generate diverse reasoning paths. Agreement among these paths provides a confidence signal, enabling models to abstain when uncertain. We demonstrate that this approach achieves effective coverage-accuracy tradeoffs across multiple reasoning benchmarks without requiring access to training data or modification of model parameters. This approach provides a practical pathway to improving the reliability of LLM outputs while maintaining compatibility with existing models. Our experiments show that unanimous agreement among noise-perturbed paths reduces error rates from 40-70% to below 15%, enabling models to exceed 95% accuracy on mathematical reasoning tasks through selective abstention.

cs.LG↗
arXiv2026

Beyond Indistinguishability: Measuring Extraction Risk in LLM APIs

Indistinguishability properties such as differential privacy bounds or low empirically measured membership inference are widely treated as proxies to show a model is sufficiently protected against broader memorization risks. However, we show that indistinguishability properties are neither sufficient nor necessary for preventing data extraction in LLM APIs. We formalize a privacy-game separation between extraction and indistinguishability-based privacy, showing that indistinguishability and inextractability are incomparable: upper-bounding distinguishability does not upper-bound extractability. To address this gap, we introduce $(l, b)$-inextractability as a definition that requires at least $2^b$ expected queries for any black-box adversary to induce the LLM API to emit a protected $l$-gram substring. We instantiate this via a worst-case extraction game and derive a rank-based extraction risk upper bound for targeted exact extraction, as well as extensions to cover untargeted and approximate extraction. The resulting estimator captures the extraction risk over multiple attack trials and prefix adaptations. We show that it can provide a tight and efficient estimation for standard greedy extraction and an upper bound on the probabilistic extraction risk given any decoding configuration. We empirically evaluate extractability across different models, clarifying its connection to distinguishability, demonstrating its advantage over existing extraction risk estimators, and providing actionable mitigation guidelines across model training, API access, and decoding configurations in LLM API deployment. Our code is publicly available at: https://github.com/Emory-AIMS/Inextractability.

cs.CR↗
arXiv2026

White-Box Sensitivity Auditing with Steering Vectors

Algorithmic audits are essential tools for examining systems for properties required by regulators or desired by operators. Current audits of large language models (LLMs) primarily rely on black-box evaluations that assess model behavior only through input-output testing. These methods are limited to tests constructed in the input space, often generated by heuristics. In addition, many socially relevant model properties (e.g., gender bias) are abstract and difficult to measure through text-based inputs alone. To address these limitations, we propose a white-box sensitivity auditing framework for LLMs that leverages activation steering to conduct more rigorous assessments through model internals. Our auditing method conducts internal sensitivity tests by manipulating key concepts relevant to the model's intended function for the task. We demonstrate its application to bias audits in four simulated high-stakes LLM decision tasks. Our method consistently indicates substantial dependence on protected attributes in model predictions, even in settings where standard black-box evaluations suggest little or no bias. Our code is openly available at https://github.com/hannahxchen/llm-steering-audit

cs.CY↗
arXiv2025

Truck drivers and automation: A methodology for identifying and supporting workforce transition in the Australian road freight sector

Transition to autonomous trucks (ATs) is coming, and is expected to create both challenges and opportunities for the driver workforce. This paper presents a novel methodology for identifying viable occupational transitions for truck drivers as transport automation advances. Unlike traditional workforce transition analyses that focus primarily on skill similarity, wages, and employment demand, this methodology incorporates four integrated components: task-level automation analysis, skill similarity assessment, labour market conditions analysis, and empirical validation using historical transition patterns. Applying this methodology to Australian truck drivers shows that while ATs will automate core driving tasks, many non-driving responsibilities will continue requiring a human, suggesting occupational evolution rather than wholesale displacement. A skill similarity analysis identifies 17 occupations with high transferability, while labour market analysis reveals significant trade-offs between wage levels and job availability across potential transition pathways. Key findings indicate that bus and coach driving, along with earthmoving plant operation, emerge as high-priority transition options, offering comparable wages and positive employment growth. Delivery and forklift driving present medium-priority pathways with abundant opportunities but lower wages. A regression analysis of historical transitions confirms that skill similarity, wage differentials, geographic accessibility, and qualification requirements all significantly influence actual transition patterns, with some viable pathways currently underutilised. The research provides policymakers, industry stakeholders, and educational institutions with evidence-based guidance for supporting workforce adaptation to technological change. The proposed methodology is generalisable beyond trucking to other sectors facing automation.

cs.HC↗
arXiv2025

Steering the CensorShip: Uncovering Representation Vectors for LLM "Thought" Control

Large language models (LLMs) have transformed the way we access information. These models are often tuned to refuse to comply with requests that are considered harmful and to produce responses that better align with the preferences of those who control the models. To understand how this "censorship" works. We use representation engineering techniques to study open-weights safety-tuned models. We present a method for finding a refusal--compliance vector that detects and controls the level of censorship in model outputs. We also analyze recent reasoning LLMs, distilled from DeepSeek-R1, and uncover an additional dimension of censorship through "thought suppression". We show a similar approach can be used to find a vector that suppresses the model's reasoning process, allowing us to remove censorship by applying the negative multiples of this vector. Our code is publicly available at: https://github.com/hannahxchen/llm-censorship-steering

cs.CL↗
arXiv2025

Inferring Events from Time Series using Language Models

A common goal in analyzing time series data is to understand how events cause observed variations. We study whether Large Language Models (LLMs) can infer natural language events associated with time series data. We introduce an automated method for generating tasks that test a model's ability to reason about events associated with time series data based on sports data, and develop a new benchmarking method. In experiments spanning 18 LLMs, we prompt LLMs to infer unobserved events given time series data and observe surprising successes, even when providing minimal context. We then show that combining distillation with Reinforcement Learning (RL) can improve the performance for small language models to approach that of large proprietary reasoning models. All resources needed to reproduce our work are available: https://github.com/hartvigsen-group/GAMETime

cs.AI↗
arXiv2025

Numerical methods for unraveling inter-particle potentials in colloidal suspensions: A comparative study for two-dimensional suspensions

We compare three model-free numerical methods for inverting structural data to obtain interaction potentials, namely iterative Boltzmann inversion (IBI), test-particle insertion (TPI), and a machine-learning (ML) approach called ActiveNet. Three archetypal models of two-dimensional colloidal systems are used as test cases: Weeks--Chandler--Anderson short-ranged repulsion, the Lennard-Jones potential, and a repulsive shoulder interaction with two length scales. Additionally, data on an experimental suspension of colloidal spheres are acquired by optical microscopy and used to test the inversion methods. The methods have different merits. IBI is the only choice when the radial distribution function is known but particle coordinates are unavailable. TPI requires snapshots with particle positions and can extract both pair- and higher-body potentials without the need for simulation. The ML approach can only be used when particles can be tracked in time and it returns the force rather than the potential. However, it can unravel pair interactions from any one-body forces (such as drag or propulsion) and does not rely on equilibrium distributions for its derivation. Our results may serve as a guide when a numerical method is needed for application to experimental data, and as a reference for further development of the methodology itself.

cond-mat.soft↗
arXiv2025

Unsupervised Concept Vector Extraction for Bias Control in LLMs

Large language models (LLMs) are known to perpetuate stereotypes and exhibit biases. Various strategies have been proposed to mitigate these biases, but most work studies biases as a black-box problem without considering how concepts are represented within the model. We adapt techniques from representation engineering to study how the concept of "gender" is represented within LLMs. We introduce a new method that extracts concept representations via probability weighting without labeled data and efficiently selects a steering vector for measuring and manipulating the model's representation. We develop a projection-based method that enables precise steering of model predictions and demonstrate its effectiveness in mitigating gender bias in LLMs and show that it also generalizes to racial bias. Our code is available at: https://github.com/hannahxchen/gender-bias-steering

cs.CL↗
arXiv2025

Toward a Principled Framework for Disclosure Avoidance

Responsible disclosure limitation is an iterative exercise in risk assessment and mitigation. From time to time, as disclosure risks grow and evolve and as data users' needs change, agencies must consider redesigning the disclosure avoidance system(s) they use. Discussions about candidate systems often conflate inherent features of those systems with implementation decisions independent of those systems. For example, a system's ability to calibrate the strength of protection to suit the underlying disclosure risk of the data (e.g., by varying suppression thresholds), is a worthwhile feature regardless of the independent decision about how much protection is actually necessary. Having a principled discussion of candidate disclosure avoidance systems requires a framework for distinguishing these inherent features of the systems from the implementation decisions that need to be made independent of the system selected. For statistical agencies, this framework must also reflect the applied nature of these systems, acknowledging that candidate systems need to be adaptable to requirements stemming from the legal, scientific, resource, and stakeholder environments within which they would be operating. This paper proposes such a framework. No approach will be perfectly adaptable to every potential system requirement. Because the selection of some methodologies over others may constrain the resulting systems' efficiency and flexibility to adapt to particular statistical product specifications, data user needs, or disclosure risks, agencies may approach these choices in an iterative fashion, adapting system requirements, product specifications, and implementation parameters as necessary to ensure the resulting quality of the statistical product.

stat.AP↗
arXiv2024

Re-entrant percolation in active Brownian hard disks

Non-equilibrium clustering and percolation are investigated in an archetypal model of two-dimensional active matter using dynamic simulations of self-propelled Brownian repulsive particles. We concentrate on the single-phase region up to moderate levels of activity, before motility-induced phase separation (MIPS) sets in. Weak activity promotes cluster formation and lowers the percolation threshold. However, driving the system further out of equilibrium partly reverses this effect, resulting in a minimum in the critical density for the formation of system-spanning clusters and introducing re-entrant percolation as a function of activity in the pre-MIPS regime. This non-monotonic behaviour arises from competition between activity-induced effective attraction (which eventually leads to MIPS) and activity-driven cluster breakup. Using an adapted iterative Boltzmann inversion method, we derive effective potentials to map weakly active cases onto a passive (equilibrium) model with conservative attraction, which can be characterised by Monte Carlo simulations. While the active and passive systems have practically identical radial distribution functions, we find decisive differences in higher-order structural correlations, to which the percolation threshold is highly sensitive. For sufficiently strong activity, no passive pairwise potential can reproduce the radial distribution function of the active system.

cond-mat.stat-mech↗
arXiv2024

Do Prevalent Bias Metrics Capture Allocational Harms from LLMs?

Allocational harms occur when resources or opportunities are unfairly withheld from specific groups. Many proposed bias measures ignore the discrepancy between predictions, which are what the proposed methods consider, and decisions that are made as a result of those predictions. Our work examines the reliability of current bias metrics in assessing allocational harms arising from predictions of large language models (LLMs). We evaluate their predictive validity and utility for model selection across ten LLMs and two allocation tasks. Our results reveal that commonly-used bias metrics based on average performance gap and distribution distance fail to reliably capture group disparities in allocation outcomes. Our work highlights the need to account for how model predictions are used in decisions, in particular in contexts where they are influenced by how limited resources are allocated.

cs.CL↗
arXiv2024

The OPS-SAT benchmark for detecting anomalies in satellite telemetry

Detecting anomalous events in satellite telemetry is a critical task in space operations. This task, however, is extremely time-consuming, error-prone and human dependent, thus automated data-driven anomaly detection algorithms have been emerging at a steady pace. However, there are no publicly available datasets of real satellite telemetry accompanied with the ground-truth annotations that could be used to train and verify anomaly detection supervised models. In this article, we address this research gap and introduce the AI-ready benchmark dataset (OPSSAT-AD) containing the telemetry data acquired on board OPS-SAT -- a CubeSat mission which has been operated by the European Space Agency which has come to an end during the night of 22--23 May 2024 (CEST). The dataset is accompanied with the baseline results obtained using 30 supervised and unsupervised classic and deep machine learning algorithms for anomaly detection. They were trained and validated using the training-test dataset split introduced in this work, and we present a suggested set of quality metrics which should be always calculated to confront the new algorithms for anomaly detection while exploiting OPSSAT-AD. We believe that this work may become an important step toward building a fair, reproducible and objective validation procedure that can be used to quantify the capabilities of the emerging anomaly detection techniques in an unbiased and fully transparent way.

eess.SP↗
arXiv2024

Do Parameters Reveal More than Loss for Membership Inference?

Membership inference attacks are used as a key tool for disclosure auditing. They aim to infer whether an individual record was used to train a model. While such evaluations are useful to demonstrate risk, they are computationally expensive and often make strong assumptions about potential adversaries' access to models and training environments, and thus do not provide tight bounds on leakage from potential attacks. We show how prior claims around black-box access being sufficient for optimal membership inference do not hold for stochastic gradient descent, and that optimal membership inference indeed requires white-box access. Our theoretical results lead to a new white-box inference attack, IHA (Inverse Hessian Attack), that explicitly uses model parameters by taking advantage of computing inverse-Hessian vector products. Our results show that both auditors and adversaries may be able to benefit from access to model parameters, and we advocate for further research into white-box methods for membership inference.

cs.LG↗
arXiv2024

DP-RuL: Differentially-Private Rule Learning for Clinical Decision Support Systems

Serious privacy concerns arise with the use of patient data in rule-based clinical decision support systems (CDSS). The goal of a privacy-preserving CDSS is to learn a population ruleset from individual clients' local rulesets, while protecting the potentially sensitive information contained in the rulesets. We present the first work focused on this problem and develop a framework for learning population rulesets with local differential privacy (LDP), suitable for use within a distributed CDSS and other distributed settings. Our rule discovery protocol uses a Monte-Carlo Tree Search (MCTS) method integrated with LDP to search a rule grammar in a structured way and find rule structures clients are likely to have. Randomized response queries are sent to clients to determine promising paths to search within the rule grammar. In addition, we introduce an adaptive budget allocation method which dynamically determines how much privacy loss budget to use at each query, resulting in better privacy-utility trade-offs. We evaluate our approach using three clinical datasets and find that we are able to learn population rulesets with high coverage (breadth of rules) and clinical utility even at low privacy loss budgets.

cs.CR↗