arXiv subjects

Mark Shapiro

Publications and source records attributed to Mark Shapiro.

4 recordsLinked to original sources

arXiv2026

Retrofitting Recurrent Depth into a Pretrained Language Model: Installation, Extrapolation, Transfer, and Retention at Two Parameter Budgets

A dense, pretrained language model can be retrofitted with recurrent depth and learn an iterative latent transition that persists after outcome-only annealing. Qwen2.5-0.5B-Instruct is split into a Prelude, a weight-tied Recurrent Block, and a Coda, with an identity-preserving one-loop path and a re-entry bridge on later loops. At loop 1 the retrofit remains non-inferior to its base on a preregistered ARC battery. Three findings. First, the mechanism is a reusable procedure rather than terminal-answer lookup, and installs at two budgets: 6M trained parameters over frozen base weights and 180M full-block. With intermediate-step supervision, the model computes one task step per loop and persists when only final answers are graded. The adapter matched the full block overall (83.8% versus 84.0%), led through depth 11, and trailed beyond. Verbal fine-tuning reached 79-86% on controlled verbal renderings (zero-shot transfer was minimal), and adapter verbal training begun from the installed mechanism outpaced matched fresh training by 18.6 points, including on a held-out test set. Second, the operation extrapolates to roughly 1.5 times its supervised depth, holding 70% accuracy through depth 18. Third, a same-size scratchpad-trained model matched the recurrent model within its learned horizon but collapsed beyond it. The recurrent model won overall, 84% versus 72%, retained 53% versus 2.5% beyond depth 10, and answered 7.6 times faster. An iterative transformer can therefore perform deeper reasoning in latent space faster than comparable or larger models fine-tuned on the same task, in a system-level comparison. A second task, running the rule in reverse, exposed the limits: the inverse was learnable in isolation, but no continuation acquired it while preserving the installed mechanism and general capability, a catastrophic-interference boundary. Learned depth selection remains open.

cs.CL↗

arXiv2025

Heterogeneous Causal Discovery of Repeated Undesirable Health Outcomes

Understanding the factors that trigger or prevent undesirable health outcomes across patient subpopulations is essential for designing targeted interventions. While randomized controlled trials and expert-led patient interviews are standard methods for identifying these factors, they can be time-consuming or infeasible. Causal discovery offers an alternative to conventional approaches by generating cause-and-effect hypotheses from observational data, yet its practical utility is limited by strong or untestable assumptions. This work presents a novel, end-to-end framework that uniquely integrates an ensemble of causal structure learning (CSL) algorithms with heterogeneous causal effect estimation. By aggregating results across multiple algorithms, the framework identifies robust causal relationships that persist under different modeling assumptions while simultaneously revealing how these effects vary across specific patient contexts. The proposed heterogeneous causal discovery framework improves robustness and provides practitioners with a prioritized set of actionable, clinically interpretable hypotheses. We demonstrate the framework's effectiveness through two large-scale healthcare applications: identifying drivers and inhibitors of repeat emergency department visits among diabetic patients and hospital readmissions among ICU patients, using insurance claims and electronic health record datasets. Our results, across both settings, identify chronic disease management and care coordination as key interventions, while revealing that intervention effectiveness depends on specific patient-level modifiers. We employ a multi-layered validation strategy, including ground-truth recovery via simulations, alignment with clinical literature, validation by expert clinicians, and portability in modern healthcare systems using an external dataset, to demonstrate the framework's practical utility.

cs.AI↗

arXiv2022

Feature Acquisition using Monte Carlo Tree Search

Feature acquisition algorithms address the problem of acquiring informative features while balancing the costs of acquisition to improve the learning performances of ML models. Previous approaches have focused on calculating the expected utility values of features to determine the acquisition sequences. Other approaches formulated the problem as a Markov Decision Process (MDP) and applied reinforcement learning based algorithms. In comparison to previous approaches, we focus on 1) formulating the feature acquisition problem as a MDP and applying Monte Carlo Tree Search, 2) calculating the intermediary rewards for each acquisition step based on model improvements and acquisition costs and 3) simultaneously optimizing model improvement and acquisition costs with multi-objective Monte Carlo Tree Search. With Proximal Policy Optimization and Deep Q-Network algorithms as benchmark, we show the effectiveness of our proposed approach with experimental study.

cs.LG↗

arXiv2022

Weight Set Decomposition for Weighted Rank Aggregation: An interpretable and visual decision support tool

The problem of interpreting or aggregating multiple rankings is common to many real-world applications. Perhaps the simplest and most common approach is a weighted rank aggregation, wherein a (convex) weight is applied to each input ranking and then ordered. This paper describes a new tool for visualizing and displaying ranking information for the weighted rank aggregation method. Traditionally, the aim of rank aggregation is to summarize the information from the input rankings and provide one final ranking that hopefully represents a more accurate or truthful result than any one input ranking. While such an aggregated ranking is, and clearly has been, useful to many applications, it also obscures information. In this paper, we show the wealth of information that is available for the weighted rank aggregation problem due to its structure. We apply weight set decomposition to the set of convex multipliers, study the properties useful for understanding this decomposition, and visualize the indifference regions. This methodology reveals information--that is otherwise collapsed by the aggregated ranking--into a useful, interpretable, and intuitive decision support tool. Included are multiple illustrative examples, along with heuristic and exact algorithms for computing the weight set decomposition.

cs.IR↗