arXiv subjects

Jing Luo

Publications and source records attributed to Jing Luo.

At least 73 records · Page 4Linked to original sources

arXiv2024

Small Tunes Transformer: Exploring Macro & Micro-Level Hierarchies for Skeleton-Conditioned Melody Generation

Recently, symbolic music generation has become a focus of numerous deep learning research. Structure as an important part of music, contributes to improving the quality of music, and an increasing number of works start to study the hierarchical structure. In this study, we delve into the multi-level structures within music from macro-level and micro-level hierarchies. At the macro-level hierarchy, we conduct phrase segmentation algorithm to explore how phrases influence the overall development of music, and at the micro-level hierarchy, we design skeleton notes extraction strategy to explore how skeleton notes within each phrase guide the melody generation. Furthermore, we propose a novel Phrase-level Cross-Attention mechanism to capture the intrinsic relationship between macro-level hierarchy and micro-level hierarchy. Moreover, in response to the current lack of research on Chinese-style music, we construct our Small Tunes Dataset: a substantial collection of MIDI files comprising 10088 Small Tunes, a category of traditional Chinese Folk Songs. This dataset serves as the focus of our study. We generate Small Tunes songs utilizing the extracted skeleton notes as conditions, and experiment results indicate that our proposed model, Small Tunes Transformer, outperforms other state-of-the-art models. Besides, we design three novel objective evaluation metrics to evaluate music from both rhythm and melody dimensions.

cs.SD↗

arXiv2024

Investigating the Impact of Model Complexity in Large Language Models

Large Language Models (LLMs) based on the pre-trained fine-tuning paradigm have become pivotal in solving natural language processing tasks, consistently achieving state-of-the-art performance. Nevertheless, the theoretical understanding of how model complexity influences fine-tuning performance remains challenging and has not been well explored yet. In this paper, we focus on autoregressive LLMs and propose to employ Hidden Markov Models (HMMs) to model them. Based on the HMM modeling, we investigate the relationship between model complexity and the generalization capability in downstream tasks. Specifically, we consider a popular tuning paradigm for downstream tasks, head tuning, where all pre-trained parameters are frozen and only individual heads are trained atop pre-trained LLMs. Our theoretical analysis reveals that the risk initially increases and then decreases with rising model complexity, showcasing a "double descent" phenomenon. In this case, the initial "descent" is degenerate, signifying that the "sweet spot" where bias and variance are balanced occurs when the model size is zero. Obtaining the presented in this study conclusion confronts several challenges, primarily revolving around effectively modeling autoregressive LLMs and downstream tasks, as well as conducting a comprehensive risk analysis for multivariate regression. Our research is substantiated by experiments conducted on data generated from HMMs, which provided empirical support and alignment with our theoretical insights.

cs.LG↗

arXiv2024

Determination of crystal structure and physical properties of Ru2Al5 intermetallic from first-principles calculations

Novel ordered intermetallic compounds have stimulated much interest. Ru-Al alloys are a prominent class of high-temperature structural materials, but the experimentally reported crystal structure of the intermetallic Ru2Al5 phase remains elusive and debatable. To resolve this controversy, we extensively explored the crystal structures of Ru2Al5 using first-principles calculations combined with crystal structure prediction technique. Among the calculated X-ray diffraction patterns and lattice parameters of five candidate Ru2Al5 structures, those of the orthorhombic Pmmn structure best aligned with recent experimental results. The structural stabilities of the five Ru2Al5 structures were confirmed through formation energy, elastic constants, and phonon spectrum calculations. We also comprehensively analyzed the mechanical and electronic properties of the five candidates. This work can guide the exploration of novel ordered intermetallic compounds in Ru-Al alloys.

cond-mat.mtrl-sci↗

arXiv2024

Mirror contrastive loss based sliding window transformer for subject-independent motor imagery based EEG signal recognition

While deep learning models have been extensively utilized in motor imagery based EEG signal recognition, they often operate as black boxes. Motivated by neurological findings indicating that the mental imagery of left or right-hand movement induces event-related desynchronization (ERD) in the contralateral sensorimotor area of the brain, we propose a Mirror Contrastive Loss based Sliding Window Transformer (MCL-SWT) to enhance subject-independent motor imagery-based EEG signal recognition. Specifically, our proposed mirror contrastive loss enhances sensitivity to the spatial location of ERD by contrasting the original EEG signals with their mirror counterparts-mirror EEG signals generated by interchanging the channels of the left and right hemispheres of the EEG signals. Moreover, we introduce a temporal sliding window transformer that computes self-attention scores from high temporal resolution features, thereby improving model performance with manageable computational complexity. We evaluate the performance of MCL-SWT on subject-independent motor imagery EEG signal recognition tasks, and our experimental results demonstrate that MCL-SWT achieved accuracies of 66.48% and 75.62%, surpassing the state-of-the-art (SOTA) model by 2.82% and 2.17%, respectively. Furthermore, ablation experiments confirm the effectiveness of the proposed mirror contrastive loss. A code demo of MCL-SWT is available at https://github.com/roniusLuo/MCL_SWT.

eess.SP↗

arXiv2024

PINT: Maximum-likelihood estimation of pulsar timing noise parameters

PINT is a pure-Python framework for high-precision pulsar timing developed on top of widely used and well-tested Python libraries, supporting both interactive and programmatic data analysis workflows. We present a new frequentist framework within PINT to characterize the single-pulsar noise processes present in pulsar timing datasets. This framework enables the parameter estimation for both uncorrelated and correlated noise processes as well as the model comparison between different timing and noise models in a computationally inexpensive way. We demonstrate the efficacy of the new framework by applying it to simulated datasets as well as a real dataset of PSR B1855+09. We also describe the new features implemented in PINT since it was first described in the literature.

astro-ph.IM↗

arXiv2024

The NANOGrav 15 yr Data Set: Chromatic Gaussian Process Noise Models for Six Pulsars

Pulsar timing arrays (PTAs) are designed to detect low-frequency gravitational waves (GWs). GWs induce achromatic signals in PTA data, meaning that the timing delays do not depend on radio-frequency. However, pulse arrival times are also affected by radio-frequency dependent "chromatic" noise from sources such as dispersion measure (DM) and scattering delay variations. Furthermore, the characterization of GW signals may be influenced by the choice of chromatic noise model for each pulsar. To better understand this effect, we assess if and how different chromatic noise models affect achromatic noise properties in each pulsar. The models we compare include existing DM models used by NANOGrav and noise models used for the European PTA Data Release 2 (EPTA DR2). We perform this comparison using a subsample of six pulsars from the NANOGrav 15 yr data set, selecting the same six pulsars as from the EPTA DR2 six-pulsar dataset. We find that the choice of chromatic noise model noticeably affects the achromatic noise properties of several pulsars. This is most dramatic for PSR J1713+0747, where the amplitude of its achromatic red noise lowers from $\log_{10}A_{\text{RN}} = -14.1^{+0.1}_{-0.1}$ to $-14.7^{+0.3}_{-0.5}$, and the spectral index broadens from $γ_{\text{RN}} = 2.6^{+0.5}_{-0.4}$ to $γ_{\text{RN}} = 3.5^{+1.2}_{-0.9}$. We also compare each pulsar's noise properties with those inferred from the EPTA DR2, using the same models. From the discrepancies, we identify potential areas where the noise models could be improved. These results highlight the potential for custom chromatic noise models to improve PTA sensitivity to GWs.

astro-ph.HE↗

arXiv2024

API Is Enough: Conformal Prediction for Large Language Models Without Logit-Access

This study aims to address the pervasive challenge of quantifying uncertainty in large language models (LLMs) without logit-access. Conformal Prediction (CP), known for its model-agnostic and distribution-free features, is a desired approach for various LLMs and data distributions. However, existing CP methods for LLMs typically assume access to the logits, which are unavailable for some API-only LLMs. In addition, logits are known to be miscalibrated, potentially leading to degraded CP performance. To tackle these challenges, we introduce a novel CP method that (1) is tailored for API-only LLMs without logit-access; (2) minimizes the size of prediction sets; and (3) ensures a statistical guarantee of the user-defined coverage. The core idea of this approach is to formulate nonconformity measures using both coarse-grained (i.e., sample frequency) and fine-grained uncertainty notions (e.g., semantic similarity). Experimental results on both close-ended and open-ended Question Answering tasks show our approach can mostly outperform the logit-based CP baselines.

cs.CL↗

arXiv2024

An unusual pulse shape change event in PSR J1713+0747 observed with the Green Bank Telescope and CHIME

The millisecond pulsar J1713+0747 underwent a sudden and significant pulse shape change between April 16 and 17, 2021 (MJDs 59320 and 59321). Subsequently, the pulse shape gradually recovered over the course of several months. We report the results of continued multi-frequency radio observations of the pulsar made using the Canadian Hydrogen Intensity Mapping Experiment (CHIME) and the 100-meter Green Bank Telescope (GBT) in a three-year period encompassing the shape change event, between February 2020 and February 2023. As of February 2023, the pulse shape had returned to a state similar to that seen before the event, but with measurable changes remaining. The amplitude of the shape change and the accompanying TOA residuals display a strong non-monotonic dependence on radio frequency, demonstrating that the event is neither a glitch (the effects of which should be independent of radio frequency, $ν$) nor a change in dispersion measure (DM) alone (which would produce a delay proportional to $ν^{-2}$). However, it does bear some resemblance to the two previous "chromatic timing events" observed in J1713+0747 (Demorest et al. 2013; Lam et al. 2016), as well as to a similar event observed in PSR J1643-1224 in 2015 (Shannon et al. 2016).

astro-ph.HE↗

arXiv2024

Learning a Structural Causal Model for Intuition Reasoning in Conversation

Reasoning, a crucial aspect of NLP research, has not been adequately addressed by prevailing models including Large Language Model. Conversation reasoning, as a critical component of it, remains largely unexplored due to the absence of a well-designed cognitive model. In this paper, inspired by intuition theory on conversation cognition, we develop a conversation cognitive model (CCM) that explains how each utterance receives and activates channels of information recursively. Besides, we algebraically transformed CCM into a structural causal model (SCM) under some mild assumptions, rendering it compatible with various causal discovery methods. We further propose a probabilistic implementation of the SCM for utterance-level relation reasoning. By leveraging variational inference, it explores substitutes for implicit causes, addresses the issue of their unobservability, and reconstructs the causal representations of utterances through the evidence lower bounds. Moreover, we constructed synthetic and simulated datasets incorporating implicit causes and complete cause labels, alleviating the current situation where all available datasets are implicit-causes-agnostic. Extensive experiments demonstrate that our proposed method significantly outperforms existing methods on synthetic, simulated, and real-world datasets. Finally, we analyze the performance of CCM under latent confounders and propose theoretical ideas for addressing this currently unresolved issue.

cs.CL↗

arXiv2024

The NANOGrav 12.5-year data set: A computationally efficient eccentric binary search pipeline and constraints on an eccentric supermassive binary candidate in 3C 66B

The radio galaxy 3C 66B has been hypothesized to host a supermassive black hole binary (SMBHB) at its center based on electromagnetic observations. Its apparent 1.05-year period and low redshift ($\sim0.02$) make it an interesting testbed to search for low-frequency gravitational waves (GWs) using Pulsar Timing Array (PTA) experiments. This source has been subjected to multiple searches for continuous GWs from a circular SMBHB, resulting in progressively more stringent constraints on its GW amplitude and chirp mass. In this paper, we develop a pipeline for performing Bayesian targeted searches for eccentric SMBHBs in PTA data sets, and test its efficacy by applying it on simulated data sets with varying injected signal strengths. We also search for a realistic eccentric SMBHB source in 3C 66B using the NANOGrav 12.5-year data set employing PTA signal models containing Earth term-only as well as Earth+Pulsar term contributions using this pipeline. Due to limitations in our PTA signal model, we get meaningful results only when the initial eccentricity $e_0<0.5$ and the symmetric mass ratio $η>0.1$. We find no evidence for an eccentric SMBHB signal in our data, and therefore place 95% upper limits on the PTA signal amplitude of $88.1\pm3.7$ ns for the Earth term-only and $81.74\pm0.86$ ns for the Earth+Pulsar term searches for $e_0<0.5$ and $η>0.1$. Similar 95% upper limits on the chirp mass are $(1.98 \pm 0.05) \times 10^9\,M_{\odot}$ and $(1.81 \pm 0.01) \times 10^9\,M_{\odot}$. These upper limits, while less stringent than those calculated from a circular binary search in the NANOGrav 12.5-year data set, are consistent with the SMBHB model of 3C 66B developed from electromagnetic observations.

astro-ph.HE↗

arXiv2023

How to Detect an Astrophysical Nanohertz Gravitational-Wave Background

Analysis of pulsar timing data have provided evidence for a stochastic gravitational wave background in the nHz frequency band. The most plausible source of such a background is the superposition of signals from millions of supermassive black hole binaries. The standard statistical techniques used to search for such a background and assess its significance make several simplifying assumptions, namely: i) Gaussianity; ii) isotropy; and most often iii) a power-law spectrum. However, a stochastic background from a finite collection of binaries does not exactly satisfy any of these assumptions. To understand the effect of these assumptions, we test standard analysis techniques on a large collection of realistic simulated datasets. The dataset length, observing schedule, and noise levels were chosen to emulate the NANOGrav 15-year dataset. Simulated signals from millions of binaries drawn from models based on the Illustris cosmological hydrodynamical simulation were added to the data. We find that the standard statistical methods perform remarkably well on these simulated datasets, despite their fundamental assumptions not being strictly met. They are able to achieve a confident detection of the background. However, even for a fixed set of astrophysical parameters, different realizations of the universe result in a large variance in the significance and recovered parameters of the background. We also find that the presence of loud individual binaries can bias the spectral recovery of the background if we do not account for them.

gr-qc↗

arXiv2023

Asymptotic in a class of network models with an increasing sub-Gamma degree sequence

For the differential privacy under the sub-Gamma noise, we derive the asymptotic properties of a class of network models with binary values with a general link function. In this paper, we release the degree sequences of the binary networks under a general noisy mechanism with the discrete Laplace mechanism as a special case. We establish the asymptotic result including both consistency and asymptotically normality of the parameter estimator when the number of parameters goes to infinity in a class of network models. Simulations and a real data example are provided to illustrate asymptotic results.

math.ST↗

arXiv2023

The NANOGrav 15-year data set: Search for Transverse Polarization Modes in the Gravitational-Wave Background

Recently we found compelling evidence for a gravitational wave background with Hellings and Downs (HD) correlations in our 15-year data set. These correlations describe gravitational waves as predicted by general relativity, which has two transverse polarization modes. However, more general metric theories of gravity can have additional polarization modes which produce different interpulsar correlations. In this work we search the NANOGrav 15-year data set for evidence of a gravitational wave background with quadrupolar Hellings and Downs (HD) and Scalar Transverse (ST) correlations. We find that HD correlations are the best fit to the data, and no significant evidence in favor of ST correlations. While Bayes factors show strong evidence for a correlated signal, the data does not strongly prefer either correlation signature, with Bayes factors $\sim 2$ when comparing HD to ST correlations, and $\sim 1$ for HD plus ST correlations to HD correlations alone. However, when modeled alongside HD correlations, the amplitude and spectral index posteriors for ST correlations are uninformative, with the HD process accounting for the vast majority of the total signal. Using the optimal statistic, a frequentist technique that focuses on the pulsar-pair cross-correlations, we find median signal-to-noise-ratios of 5.0 for HD and 4.6 for ST correlations when fit for separately, and median signal-to-noise-ratios of 3.5 for HD and 3.0 for ST correlations when fit for simultaneously. While the signal-to-noise-ratios for each of the correlations are comparable, the estimated amplitude and spectral index for HD are a significantly better fit to the total signal, in agreement with our Bayesian analysis.

gr-qc↗

arXiv2023

How to Enhance Causal Discrimination of Utterances: A Case on Affective Reasoning

Our investigation into the Affective Reasoning in Conversation (ARC) task highlights the challenge of causal discrimination. Almost all existing models, including large language models (LLMs), excel at capturing semantic correlations within utterance embeddings but fall short in determining the specific causal relationships. To overcome this limitation, we propose the incorporation of \textit{i.i.d.} noise terms into the conversation process, thereby constructing a structural causal model (SCM). It explores how distinct causal relationships of fitted embeddings can be discerned through independent conditions. To facilitate the implementation of deep learning, we introduce the cogn frameworks to handle unstructured conversation data, and employ an autoencoder architecture to regard the unobservable noise as learnable "implicit causes." Moreover, we curate a synthetic dataset that includes i.i.d. noise. Through comprehensive experiments, we validate the effectiveness and interpretability of our approach. Our code is available in https://github.com/Zodiark-ch/mater-of-our-EMNLP2023-paper.

cs.CL↗

arXiv2023

Inter-Rater Uncertainty Quantification in Medical Image Segmentation via Rater-Specific Bayesian Neural Networks

Automated medical image segmentation inherently involves a certain degree of uncertainty. One key factor contributing to this uncertainty is the ambiguity that can arise in determining the boundaries of a target region of interest, primarily due to variations in image appearance. On top of this, even among experts in the field, different opinions can emerge regarding the precise definition of specific anatomical structures. This work specifically addresses the modeling of segmentation uncertainty, known as inter-rater uncertainty. Its primary objective is to explore and analyze the variability in segmentation outcomes that can occur when multiple experts in medical imaging interpret and annotate the same images. We introduce a novel Bayesian neural network-based architecture to estimate inter-rater uncertainty in medical image segmentation. Our approach has three key advancements. Firstly, we introduce a one-encoder-multi-decoder architecture specifically tailored for uncertainty estimation, enabling us to capture the rater-specific representation of each expert involved. Secondly, we propose Bayesian modeling for the new architecture, allowing efficient capture of the inter-rater distribution, particularly in scenarios with limited annotations. Lastly, we enhance the rater-specific representation by integrating an attention module into each decoder. This module facilitates focused and refined segmentation results for each rater. We conduct extensive evaluations using synthetic and real-world datasets to validate our technical innovations rigorously. Our method surpasses existing baseline methods in five out of seven diverse tasks on the publicly available \emph{QUBIQ} dataset, considering two evaluation metrics encompassing different uncertainty aspects. Our codes, models, and the new dataset are available through our GitHub repository: https://github.com/HaoWang420/bOEMD-net .

eess.IV↗

arXiv2023

The NANOGrav 12.5-Year Data Set: Dispersion Measure Mis-Estimation with Varying Bandwidths

Noise characterization for pulsar-timing applications accounts for interstellar dispersion by assuming a known frequency-dependence of the delay it introduces in the times of arrival (TOAs). However, calculations of this delay suffer from mis-estimations due to other chromatic effects in the observations. The precision in modeling dispersion is dependent on the observed bandwidth. In this work, we calculate the offsets in infinite-frequency TOAs due to mis-estimations in the modeling of dispersion when using varying bandwidths at the Green Bank Telescope. We use a set of broadband observations of PSR J1643-1224, a pulsar with an excess of chromatic noise in its timing residuals. We artificially restricted these observations to a narrowband frequency range, then used both data sets to calculate residuals with a timing model that does not include short-scale dispersion variations. By fitting the resulting residuals to a dispersion model, and comparing the ensuing fitted parameters, we quantify the dispersion mis-estimations. Moreover, by calculating the autocovariance function of the parameters we obtained a characteristic timescale over which the dispersion mis-estimations are correlated. For PSR J1643-1224, which has one of the highest dispersion measures (DM) in the NANOGrav pulsar timing array, we find that the infinite-frequency TOAs suffer from a systematic offset of ~22 microseconds due to DM mis-estimations, with correlations over ~1 month. For lower-DM pulsars, the offset is ~7 microseconds. This error quantification can be used to provide more robust noise modeling in NANOGrav's data, thereby increasing sensitivity and improving parameter estimation in gravitational wave searches.

astro-ph.HE↗

arXiv2023

The NANOGrav 12.5-year Data Set: Search for Gravitational Wave Memory

We present the results of a Bayesian search for gravitational wave (GW) memory in the NANOGrav 12.5-yr data set. We find no convincing evidence for any gravitational wave memory signals in this data set (Bayes factor = 2.8). As such, we go on to place upper limits on the strain amplitude of GW memory events as a function of sky location and event epoch. These upper limits are computed using a signal model that assumes the existence of a common, spatially uncorrelated red noise in addition to a GW memory signal. The median strain upper limit as a function of sky position is approximately $3.3 \times 10^{-14}$. We also find that there are some differences in the upper limits as a function of sky position centered around PSR J0613$-$0200. This suggests that this pulsar has some excess noise which can be confounded with GW memory. Finally, the upper limits as a function of burst epoch continue to improve at later epochs. This improvement is attributable to the continued growth of the pulsar timing array.

gr-qc↗

arXiv2023

The NANOGrav 15-year Data Set: Constraints on Supermassive Black Hole Binaries from the Gravitational Wave Background

The NANOGrav 15-year data set shows evidence for the presence of a low-frequency gravitational-wave background (GWB). While many physical processes can source such low-frequency gravitational waves, here we analyze the signal as coming from a population of supermassive black hole (SMBH) binaries distributed throughout the Universe. We show that astrophysically motivated models of SMBH binary populations are able to reproduce both the amplitude and shape of the observed low-frequency gravitational-wave spectrum. While multiple model variations are able to reproduce the GWB spectrum at our current measurement precision, our results highlight the importance of accurately modeling binary evolution for producing realistic GWB spectra. Additionally, while reasonable parameters are able to reproduce the 15-year observations, the implied GWB amplitude necessitates either a large number of parameters to be at the edges of expected values, or a small number of parameters to be notably different from standard expectations. While we are not yet able to definitively establish the origin of the inferred GWB signal, the consistency of the signal with astrophysical expectations offers a tantalizing prospect for confirming that SMBH binaries are able to form, reach sub-parsec separations, and eventually coalesce. As the significance grows over time, higher-order features of the GWB spectrum will definitively determine the nature of the GWB and allow for novel constraints on SMBH populations.

astro-ph.HE↗