arXiv Science⌕ Search

arXiv · physics/0401007

The CAVES Project - Exploring Virtual Data Concepts for Data Analysis

Abstract

The Collaborative Analysis Versioning Environment System (CAVES) project concentrates on the interactions between users performing data and/or computing intensive analyses on large data sets, as encountered in many contemporary scientific disciplines. In modern science increasingly larger groups of researchers collaborate on a given topic over extended periods of time. The logging and sharing of knowledge about how analyses are performed or how results are obtained is important throughout the lifetime of a project. Here is where virtual data concepts play a major role. The ability to seamlessly log, exchange and reproduce results and the methods, algorithms and computer programs used in obtaining them enhances in a qualitative way the level of collaboration in a group or between groups in larger organizations. The CAVES project takes a pragmatic approach in assessing the needs of a community of scientists by building series of prototypes with increasing sophistication. In extending the functionality of existing data analysis packages with virtual data capabilities these prototypes provide an easy and habitual entry point for researchers to explore virtual data concepts in real life applications and to provide valuable feedback for refining the system design. The architecture is modular based on Web, Grid and other services which can be plugged in as desired. As a proof of principle we build a first system by extending the very popular data analysis framework ROOT, widely used in high energy physics and other fields, making it virtual data enabled.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Dimitri Bourilkov. 2004-01-04. The CAVES Project - Exploring Virtual Data Concepts for Data Analysis. https://arxiv.science/abs/physics/0401007

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

A Paired Monte Carlo Intensity-Ratio Method for Incorporating Detector Resolution in Amplitude Analysis

Finite detector resolution can bias resonance parameters and fit fractions extracted from amplitude analyses. We present the paired Monte Carlo intensity-ratio method for incorporating resolution effects into amplitude analysis. The method uses ratios of reference intensities evaluated at paired truth-level and reconstructed kinematics to reweight the Monte Carlo normalization and approximately account for detector resolution. A resampling study of simulated $D^+\to K^+π^+π^-π^0$ decays shows that the method substantially reduces resolution-induced shifts in fitted resonance widths. Likelihood evaluation uses intensity calculations and normalization sums, avoiding a response convolution at each data event.

physics.data-an↗

Distinguishing thyroid nodule malignancy using digital palpation

Accurate pre-operative diagnosis of thyroid nodules remains a major clinical challenge. Current diagnostic techniques rely on fine-needle aspiration and demonstrate high rates of inconclusive results, often leading to avoidable thyroidectomies with lifelong implications. Here, we introduce a mechanics-based diagnostic method that leverages Digital Palpation to determine malignancy and identify tissue subtypes in human thyroid nodules. From a cohort of patients undergoing thyroidectomies, we evaluated ex vivo thyroid specimens and processed the resulting mechanical data from 36 tests in 10 thyroid nodules. Principal Component Analysis reveals that these experimental results are distributed within a three-dimensional space of interpretable scalar features reflecting tissue grittiness, strength, and toughness, which map closely to descriptions of malignant and benign nodules given by clinicians. Our results further demonstrate that malignant nodules exhibit distinct mechanical signatures compared to their benign counterparts. This revealed a statistically significant difference (p = 9 x 10^-4) and separability (95% lower-bound Hedges' g = 1.08) between benign and malignant nodules along a direction in principal component space representing effective stiffness. Preliminary clustering patterns were also observed among malignant subtypes, including follicular, oncocytic, and papillary thyroid carcinomas. Together, these results highlight the potential of Digital Palpation as a novel mechanics-based method for tissue characterization and motivate its further development toward diagnostic applications.

physics.data-an↗

Numerical Integration Instabilities in Likelihood Fits: Mechanisms and Remedies

Numerical instabilities in the normalization of probability density functions can produce artificially lower negative log-likelihood (NLL) values than the true minimum of a likelihood fit, even when the resulting fit provides a poorer description of the data. Motivated by anomalously low NLL values found in non-converged fits associated with a recent CMS search for fully charmed tetraquark candidates, we use a pseudo-dataset generated from a likelihood model based on that analysis to investigate how numerical integration can yield unreliable normalization results, how these errors affect likelihood minimization, and how they can be addressed. We find that, even at fixed integration tolerance, the stopping decision of a numerical integrator can change with the fit parameters, introducing discontinuities in the normalization integrals that propagate to the likelihood surface. Such discontinuities can trap the minimizer at a numerical step and yield an artificially lower NLL. Tightening the integration tolerance mitigates these numerical discontinuities, while adaptive integration methods, such as Gauss--Kronrod quadrature, can also improve the numerical stability. A more severe failure occurs when the integration grid cannot adequately resolve a sharply localized contribution. In this case, the integrator may satisfy its estimated-error criterion despite substantially underestimating the normalization, allowing the minimizer to enter a region with an artificially low NLL. Tightening integration tolerances or using alternative adaptive integration methods does not necessarily resolve sharply localized contributions. We demonstrate that a suitable variable transformation can improve their numerical resolution, yielding more accurate normalization integrals and restoring fit convergence.

physics.data-an↗