arXiv Science⌕ Search

arXiv subjects

Lewis Mitchell

Publications and source records attributed to Lewis Mitchell.

At least 19 recordsLinked to original sources

When Are Predictions Enough? An Evaluation Protocol for Frozen Expert Composition

Practitioners often combine independently trained models while keeping their weights frozen. These experts are commonly combined through their final predictions, although their internal features may retain useful information that those predictions omit. We ask when combining features improves on combining predictions, and how to make that comparison fair. We introduce a reusable evaluation protocol, apply it to 21 frozen deepfake detectors from DF40 (Yan et al., 2024), and audit how evaluation choices change the conclusions. The protocol uses strong prediction baselines, compares simple and larger feature combiners, varies architecture and training data separately, and reports results by held-out generator with train/test overlap controls. Under leave-one-generator-out evaluation, logistic regression on the features of six architecture-diverse experts improves macro AUROC by 0.081 over the best tested prediction combiner in each fold, with gains on seven of eight generators. The larger network shows no clear improvement over the linear model (approximately 3.5M versus 10K parameters), suggesting that access to richer information matters more than additional combiner capacity on this pool. Features may help when they retain information that the predictions omit and that transfers to new generators. Predictions may be enough when the strongest prediction baseline already performs well, while neither approach reliably compensates for poor expert transfer. Weakening prediction baselines on the overlap-affected standard split inflates the apparent feature advantage by up to 0.172 AUROC. Overlap-aware evaluation also changes the estimated gain, though the contrast does not isolate overlap. Results by generator and expert pool show where the advantage holds, weakens, or reverses. Without these controls, the same benchmark can support different conclusions.

cs.LG↗

An Information-Theoretic Model for the Evolution of a Social Network

Echo chambers and filter bubbles provide a framework to view communication and connection in a social network age. Such systems are characterised by isolated communities regurgitating information which reinforces prior beliefs. Theoretical analysis of echo chambers is suited for a network science approach to understand the mechanics of information influences between users. Here, we propose an evolutionary network model that extends the Quoter Model, rewiring edges according to cross-entropy between text-generating nodes. Parameters characterise binary reinforcement vs exploration node intents, determining whether a node will seek high or low novelty neighbours, measured by cross-entropy. Through an empirical simulation study, we observe emergent model attributes. Specifically, we compare to a Watts--Strogatz model, highlighting parameter regions of high clustering and hyper-exploration. Further, we identify a quality of disjoint communities akin to rudimentary echo chamber structures. These findings underscore user behaviours that drive echo chamber entrenchment, contextualising future social network analysis.

cs.SI↗

No Model Required: Text Entropy Rate Filtering Mitigates Iterative Fine-Tuning Collapse

Iterative fine-tuning on synthetic data causes \emph{model collapse}: output diversity narrows as rare patterns are progressively lost, a signature most visible as phrase-level repetition. Existing mitigations either require model log-probabilities, an external oracle, or continued access to real human data. Here we develop a new approach grounded in mathematical information theory: the non-parametric Kontoyiannis entropy rate estimator $h_k$, computed entirely from raw text via match-length statistics, with no model of any kind. We show that this is in fact a \emph{superior} training-data filter on text-diversity metrics in a fully-synthetic, single-lineage fine-tuning setting. In a six-generation QLoRA collapse experiment on Llama-3.1-8B, logprob-based filtering (the most established model-access-requiring baseline) provides no significant text-diversity benefit on any metric ($p > 0.23$), whereas $h_k$-filtering yields $+42\%$ unique trigrams, $+30\%$ vocabulary, and $-19\%$ repetition (all $p < 0.001$). We validate $h_k$ as a cross-domain entropy proxy ($β= 0.924$, $R^2 = 0.746$) and collapse detector ($ρ= +0.454$, $p < 0.0001$) across 4~domains, 2~temperatures, 2~generator--scorer model pairs, and 1{,}520 generated documents. Our results demonstrate that information theoretic approaches to collapse mitigation are efficient, and suggest new approaches for maintaining multi-agent diversity.

cs.CL↗

Repeatability is not recovery: Quantifying algorithmic stability and topic recovery in Latent Dirichlet Allocation

Topic models are often judged by the consistency of their outputs across repeated runs, implicitly assuming that repeatable topic output is a successful recovery of the underlying topics. We show that this assumption is false: repeatability is not recovery. We introduce a stability framework that jointly measures consistency among repeated runs and accuracy relative to known ground truth. Because real-world corpora lack known topic structures, we generate synthetic corpora using the Latent Dirichlet Allocation (LDA) generative process, enabling direct evaluation of topic recovery. Across 50 repeated LDA runs on each corpus, we find that LDA reliably identifies the correct number of topics and frequently converges to highly consistent topic solutions. However, these repeatable solutions frequently fail to recover the true generating topics. Thus, internal stability should not be interpreted as evidence of correctness. Our results illustrate that stability and recovery are distinct properties of topic models and should be evaluated separately. Consequently, topic-model outputs should be validated using multiple complementary criteria before supporting substantive conclusions, particularly in high-stakes applications.

cs.CL↗

Best Preprocessing Techniques for Sentiment Analysis

Sentiment analysis in Twitter datasets is important because it enables monitoring public opinion on products and analysis of political and social movements. One critical step is preprocessing: the automated processing of text for machine learning algorithms. Preprocessing plays a critical role in reducing noise and improving efficiency. However, little research has systematically examined the order in which preprocessing techniques are implemented. We find that, when accounting for order, spelling correction is the least impactful preprocessing technique, whereas tokenisation is the most impactful. Stemming and stop-word removal are interchangeable, and it is better to remove stop words without removing negation. The best order for applying the preprocessing techniques was tokenisation, text cleaning, stemming, and then stopword removal. Our results provide a systematic approach for practitioners to deploy preprocessing to improve model output without the costly preprocessing exploratory phase.

cs.CL↗

Learning Moral Diversity: Modelling Individual Perspectives in Moral Classification of Texts

Understanding moral values in social media text offers insight into moral judgement formation, and supervised NLP models trained on crowdsourced data have achieved strong classification performance. However, most approaches simplify the problem by aggregating multiple annotators' labels into a single "ground truth", overlooking the inherent subjectivity of the task. In practice, there are disagreements between annotators caused by personal viewpoint or inherent ambiguities, particularly for short tweets. Here, we extend a pretrained language model with a layer that learns annotator-specific features. Our model improves predictions of individual annotations and yields representations that reveal meaningful insights into annotators' moral perspectives. We show that models trained on aggregated labels may hide variation and give a misleading impression of performance. Overall, we demonstrate that disagreement reflects the inherent subjectivity of the task and that modelling individual perspectives creates benefits for moral classification of texts.

cs.CL↗

Demographic Dependence of Vaccine Adoption under Opinion Persuasion

Inspired by contagion models of social belief formation, we develop an epistemically-informed modeling framework, SIS-Vo, in which vaccine-related information propagates on a signed opinion network. Our model allows for heterogeneous treatment effects of policy messages across subpopulations through demographic-specific responses. We derive fixed-point characterizations of the healthy (disease-free) and endemic equilibria of this model, and obtain conditions for local stability of the healthy state in terms of the contact network and opinion-dependent vaccination capacities. Using numerical simulations, we illustrate how suitably targeted policy interventions, acting through opinion dynamics, can stabilize the epidemic process by moving the system towards the healthy regime. The SIS-Vo framework thus provides a natural basis for control-theoretic analysis of vaccination policies that remain robust even when misinformation targets specific subgroups.

physics.soc-ph↗

A Hybrid Theory and Data-driven Approach to Persuasion Detection with Large Language Models

Traditional psychological models of belief revision focus on face-to-face interactions, but with the rise of social media, more effective models are needed to capture belief revision at scale, in this rich text-based online discourse. Here, we use a hybrid approach, utilizing large language models (LLMs) to develop a model that predicts successful persuasion using features derived from psychological experiments. Our approach leverages LLM generated ratings of features previously examined in the literature to build a random forest classification model that predicts whether a message will result in belief change. Of the eight features tested, \textit{epistemic emotion} and \textit{willingness to share} were the top-ranking predictors of belief change in the model. Our findings provide insights into the characteristics of persuasive messages and demonstrate how LLMs can enhance models of successful persuasion based on psychological theory. Given these insights, this work has broader applications in fields such as online influence detection and misinformation mitigation, as well as measuring the effectiveness of online narratives.

cs.CL↗

Data Denoising and Derivative Estimation for Data-Driven Modeling of Nonlinear Dynamical Systems

Data-driven modeling of nonlinear dynamical systems is often hampered by measurement noise. We propose a denoising framework, called Runge-Kutta and Total Variation Based Implicit Neural Representation (RKTV-INR), that represents the state trajectory with an implicit neural representation (INR) fitted directly to noisy observations. Runge-Kutta integration and total variation are imposed as constraints to ensure that the reconstructed state is a trajectory of a dynamical system that remains close to the original data. The trained INR yields a clean, continuous trajectory and provides accurate first-order derivatives via automatic differentiation. These denoised states and derivatives are then supplied to Sparse Identification of Nonlinear Dynamics (SINDy) to recover the governing equations. Experiments demonstrate effective noise suppression, precise derivative estimation, and reliable system identification.

cs.LG↗

Analysis of Voluntarily Reported Data Post Mesh Implantation for Detecting Public Emotion and Identifying Concern Reports

Mesh implants are widely utilized in hernia repair surgeries, but postoperative complications present a significant concern. This study analyzes patient reports from the Manufacturer and User Facility Device Experience (MAUDE) database spanning 2000 to 2021 to investigate the emotional aspects of patients following mesh implantation using Natural Language Processing (NLP). Employing the National Research Council Canada (NRC) Emotion Lexicon and TextBlob for sentiment analysis, the research categorizes patient narratives into eight emotions (anger, fear, anticipation, trust, surprise, sadness, joy, and disgust) and assesses sentiment polarity. The goal is to discern patterns in patient sentiment over time and to identify reports signaling urgent concerns, referred to as "Concern Reports," thereby understanding shifts in patient experiences in relation to changes in medical device regulation and technological advancements in healthcare. The study detected an increase in Concern Reports and higher emotional intensity during the periods of 2011-2012 and 2017-2018. Through temporal analysis of Concern Reports and overall sentiment, this research provides valuable insights for healthcare practitioners, enhancing their understanding of patient experiences post-surgery, which is critical for improving preoperative counselling, postoperative care, and preparing patients for mesh implant surgeries. The study underscores the importance of emotional considerations in medical practices and the potential for sentiment analysis to inform and enhance patient care.

cs.CL↗

Discovering Coordinated Processes From Social Online Networks

The rapid growth of social media presents a unique opportunity to study coordinated agent behavior in an unfiltered environment. Online processes often exhibit complex structures that reflect the nature of the user behavior, whether it is authentic and genuine, or part of a coordinated effort by malicious agents to spread misinformation and disinformation. Detection of AI-generated content can be extremely challenging due to the high quality of large language model-generated text. Therefore, approaches that use metadata like post timings are required to effectively detect coordinated AI-driven campaigns. Existing work that models the spread of information online is limited in its ability to represent different control flows that occur within the network in practice. Process mining offers techniques for the discovery of process models with different routing constructs and are yet to be applied to social networks. We propose to leverage process mining methods for the discovery of AI and human agent behavior within social networks. Applying process mining techniques to real-world Twitter (now X) event data, we demonstrate how the structural and behavioral properties of discovered process models can reveal coordinated AI and human behaviors online.

cs.SI↗

Hierarchical Representations for Evolving Acyclic Vector Autoregressions (HEAVe)

Causal networks offer an intuitive framework to understand influence structures within time series systems. However, the presence of cycles can obscure dynamic relationships and hinder hierarchical analysis. These networks are typically identified through multivariate predictive modelling, but enforcing acyclic constraints significantly increases computational and analytical complexity. Despite recent advances, there remains a lack of simple, flexible approaches that are easily tailorable to specific problem instances. We propose an evolutionary approach to fitting acyclic vector autoregressive processes and introduces a novel hierarchical representation that directly models structural elements within a time series system. On simulated datasets, our model retains most of the predictive accuracy of unconstrained models and outperforms permutation-based alternatives. When applied to a dataset of 100 cryptocurrency return series, our method generates acyclic causal networks capturing key structural properties of the unconstrained model. The acyclic networks are approximately sub-graphs of the unconstrained networks, and most of the removed links originate from low-influence nodes. Given the high levels of feature preservation, we conclude that this cryptocurrency price system functions largely hierarchically. Our findings demonstrate a flexible, intuitive approach for identifying hierarchical causal networks in time series systems, with broad applications to fields like econometrics and social network analysis.

q-fin.ST↗

Performance Analysis: Discovering Semi-Markov Models From Event Logs

Process mining is a well-established discipline of data analysis focused on the discovery of process models from information systems' event logs. Recently, an emerging subarea of process mining, known as stochastic process discovery, has started to evolve. Stochastic process discovery considers frequencies of events in the event data and allows for a more comprehensive analysis. In particular, when the durations of activities are presented in the event log, performance characteristics of the discovered stochastic models can be analyzed, e.g., the overall process execution time can be estimated. Existing performance analysis techniques usually discover stochastic process models from event data, and then simulate these models to evaluate their execution times. These methods rely on empirical approaches. This paper proposes analytical techniques for performance analysis that allow for the derivation of statistical characteristics of the overall processes' execution times in the presence of arbitrary time distributions of events modeled by semi-Markov processes. The proposed methods include express analysis, focused on the mean execution time estimation, and full analysis techniques that build probability density functions (PDFs) of process execution times in both continuous and discrete forms. These methods are implemented and tested on real-world event data, demonstrating their potential for what-if analysis by providing solutions without resorting to simulation. Specifically, we demonstrated that the discrete approach is more time-efficient for small duration support sizes compared to the simulation technique. Furthermore, we showed that the continuous approach, with PDFs represented as Mixtures of Gaussian Models (GMMs), facilitates the discovery of more compact and interpretable models.

cs.DB↗

Community Detection of Complex Network Based on Graph Convolution Iterative Algorithm

Community detection can reveal the underlying structure and patterns of complex networks, identify sets of nodes with specific functions or similar characteristics, and study the evolution process and development trends of networks. Despite the myriad community detection methods that have been proposed, researchers continue to strive for ways to enhance the accuracy and efficiency of these methods. Graph convolutional neural networks can continuously aggregate the features of multiple neighboring nodes and have become an important tool in many fields. In view of this, this paper proposes a community detection method for complex networks based on graph convolution iteration algorithm. Firstly, the candidate community centers are determined by random sampling and the node attribute matrix is obtained based on the distances of nodes to community centers. Next, the graph convolution operation is implemented to obtain the convolutional node attribute matrix. Then, community partitioning method according to the convolutional node attribute matrix is presented and the effectiveness of community partitioning is measured through modularity. The method proposed in this paper is applied into to multiple random and real-world networks. The comparison results with some baseline methods demonstrate its effectiveness.

cs.SI↗

Personality Profiling: How informative are social media profiles in predicting personal information?

Personality profiling has been utilised by companies for targeted advertising, political campaigns and public health campaigns. However, the accuracy and versatility of such models remains relatively unknown. Here we explore the extent to which peoples' online digital footprints can be used to profile their Myers-Briggs personality type. We analyse and compare four models: logistic regression, naive Bayes, support vector machines (SVMs) and random forests. We discover that a SVM model achieves the best accuracy of 20.95% for predicting a complete personality type. However, logistic regression models perform only marginally worse and are significantly faster to train and perform predictions. Moreover, we develop a statistical framework for assessing the importance of different sets of features in our models. We discover some features to be more informative than others in the Intuitive/Sensory (p = 0.032) and Thinking/Feeling (p = 0.019) models. Many labelled datasets present substantial class imbalances of personal characteristics on social media, including our own. We therefore highlight the need for attentive consideration when reporting model performance on such datasets and compare a number of methods to fix class-imbalance problems.

cs.CL↗

Enhancing Causal Discovery in Financial Networks with Piecewise Quantile Regression

Financial networks can be constructed using statistical dependencies found within the price series of speculative assets. Across the various methods used to infer these networks, there is a general reliance on predictive modelling to capture cross-correlation effects. These methods usually model the flow of mean-response information, or the propagation of volatility and risk within the market. Such techniques, though insightful, don't fully capture the broader distribution-level causality that is possible within speculative markets. This paper introduces a novel approach, combining quantile regression with a piecewise linear embedding scheme - allowing us to construct causality networks that identify the complex tail interactions inherent to financial markets. Applying this method to 260 cryptocurrency return series, we uncover significant tail-tail causal effects and substantial causal asymmetry. We identify a propensity for coins to be self-influencing, with comparatively sparse cross variable effects. Assessing all link types in conjunction, Bitcoin stands out as the primary influencer - a nuance that is missed in conventional linear mean-response analyses. Our findings introduce a comprehensive framework for modelling distributional causality, paving the way towards more holistic representations of causality in financial markets.

q-fin.ST↗

Modified CMA-ES Algorithm for Multi-Modal Optimization: Incorporating Niching Strategies and Dynamic Adaptation Mechanism

This study modifies the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) algorithm for multi-modal optimization problems. The enhancements focus on addressing the challenges of multiple global minima, improving the algorithm's ability to maintain diversity and explore complex fitness landscapes. We incorporate niching strategies and dynamic adaptation mechanisms to refine the algorithm's performance in identifying and optimizing multiple global optima. The algorithm generates a population of candidate solutions by sampling from a multivariate normal distribution centered around the current mean vector, with the spread determined by the step size and covariance matrix. Each solution's fitness is evaluated as a weighted sum of its contributions to all global minima, maintaining population diversity and preventing premature convergence. We implemented the algorithm on 8 tunable composite functions for the GECCO 2024 Competition on Benchmarking Niching Methods for Multi-Modal Optimization (MMO), adhering to the competition's benchmarking framework. The results are presenting in many ways such as Peak Ratio, F1 score on various dimensions. They demonstrate the algorithm's robustness and effectiveness in handling both global optimization and MMO- specific challenges, providing a comprehensive solution for complex multi-modal optimization problems.

cs.NE↗

Orthogonally Initiated Particle Swarm Optimization with Advanced Mutation for Real-Parameter Optimization

This article introduces an enhanced particle swarm optimizer (PSO), termed Orthogonal PSO with Mutation (OPSO-m). Initially, it proposes an orthogonal array-based learning approach to cultivate an improved initial swarm for PSO, significantly boosting the adaptability of swarm-based optimization algorithms. The article further presents archive-based self-adaptive learning strategies, dividing the population into regular and elite subgroups. Each subgroup employs distinct learning mechanisms. The regular group utilizes efficient learning schemes derived from three unique archives, which categorize individuals based on their quality levels. Additionally, a mutation strategy is implemented to update the positions of elite individuals. Comparative studies are conducted to assess the effectiveness of these learning strategies in OPSO-m, evaluating its optimization capacity through exploration-exploitation dynamics and population diversity analysis. The proposed OPSO-m model is tested on real-parameter challenges from the CEC 2017 suite in 10, 30, 50, and 100-dimensional search spaces, with its results compared to contemporary state-of-the-art algorithms using a sensitivity metric. OPSO-m exhibits distinguished performance in the precision of solutions, rapidity of convergence, efficiency in search, and robust stability, thus highlighting its superior aptitude for resolving intricate optimization issues.

cs.NE↗