arXiv subjects

Jiarui Liu

Publications and source records attributed to Jiarui Liu.

69 records · Page 4Linked to original sources

arXiv2024

Synatra: Turning Indirect Knowledge into Direct Demonstrations for Digital Agents at Scale

LLMs can now act as autonomous agents that interact with digital environments and complete specific objectives (e.g., arranging an online meeting). However, accuracy is still far from satisfactory, partly due to a lack of large-scale, direct demonstrations for digital tasks. Obtaining supervised data from humans is costly, and automatic data collection through exploration or reinforcement learning relies on complex environmental and content setup, resulting in datasets that lack comprehensive coverage of various scenarios. On the other hand, there is abundant knowledge that may indirectly assist task completion, such as online tutorials that were created for human consumption. In this work, we present Synatra, an approach that effectively transforms this indirect knowledge into direct supervision at scale. We define different types of indirect knowledge, and carefully study the available sources to obtain it, methods to encode the structure of direct demonstrations, and finally methods to transform indirect knowledge into direct demonstrations. We use 100k such synthetically-created demonstrations to finetune a 7B CodeLlama, and demonstrate that the resulting agent surpasses all comparably sized models on three web-based task benchmarks Mind2Web, MiniWoB++ and WebArena, as well as surpassing GPT-3.5 on WebArena and Mind2Web. In addition, while synthetic demonstrations prove to be only 3% the cost of human demonstrations (at $0.031 each), we show that the synthetic demonstrations can be more effective than an identical number of human demonstrations collected from limited domains.

cs.AI↗

arXiv2024

Implicit Personalization in Language Models: A Systematic Study

Implicit Personalization (IP) is a phenomenon of language models inferring a user's background from the implicit cues in the input prompts and tailoring the response based on this inference. While previous work has touched upon various instances of this problem, there lacks a unified framework to study this behavior. This work systematically studies IP through a rigorous mathematical formulation, a multi-perspective moral reasoning framework, and a set of case studies. Our theoretical foundation for IP relies on a structural causal model and introduces a novel method, indirect intervention, to estimate the causal effect of a mediator variable that cannot be directly intervened upon. Beyond the technical approach, we also introduce a set of moral reasoning principles based on three schools of moral philosophy to study when IP may or may not be ethically appropriate. Equipped with both mathematical and ethical insights, we present three diverse case studies illustrating the varied nature of the IP problem and offer recommendations for future research. Our code is at https://github.com/jiarui-liu/IP, and our data is at https://huggingface.co/datasets/Jerry999/ImplicitPersonalizationData.

cs.CL↗

arXiv2024

Charge-Density-Wave State in Extremely Overdoped Cuprates Driven by Phonons

Recent resonant x-ray scattering (RXS) experiments revealed a novel charge order in extremely overdoped La$_{2-x}$Sr$_x$CuO$_4$ (LSCO) [Phys. Rev. Lett. 131,116002]. The observed charge order appears around the $(π/3,0)$ wavevector, distinct from the well-known stripe fluctuations near 1/8 doping, and persists from cryogenic temperatures to room temperature. To investigate the origin of this charge order in the overdoped regime, we use determinant quantum Monte Carlo (DQMC) simulations to examine correlated models with various interactions. We demonstrate that this distinctive CDW originates from remnant correlations in extremely overdoped cuprates, with its specific pattern shaped by interactions beyond the Hubbard model, particularly electron-phonon couplings. The persistence of the $(π/3,0)$ wavevector across different doping levels indicates the presence of nonlocal couplings. Our study reveals the significant role of phonons in cuprates, which assist correlated electrons in the formation of unconventional phases.

cond-mat.str-el↗

arXiv2024

Signatures of the Attractive Interaction in Spin Spectra of One-dimensional Cuprate Chains

Identifying the minimal model for cuprates is crucial for explaining the high-$T_c$ pairing mechanism. Recent photoemission experiments have suggested a significant near-neighbor attractive interaction $V$ in cuprate chains, favoring pairing instability. To determine its strength, we systematically investigate the dynamical spin structure factors $S(q,ω)$ using the density matrix renormalization group. Our analysis quantitatively reveals a notable softening in the two-spinon continuum, particularly evident in the intense spectrum at large momentum. This softening is primarily driven by the renormalization of the superexchange interaction, as determined by a comparison with the slave-boson theory. We also demonstrate the feasibility of detecting this spectral shift in thin-film samples using resonant inelastic x-ray scattering. Therefore, this provides a distinctive fingerprint for the attractive interaction, motivating future experiments to unveil essential ingredients in cuprates.

cond-mat.str-el↗

arXiv2024

Automatic Generation of Model and Data Cards: A Step Towards Responsible AI

In an era of model and data proliferation in machine learning/AI especially marked by the rapid advancement of open-sourced technologies, there arises a critical need for standardized consistent documentation. Our work addresses the information incompleteness in current human-generated model and data cards. We propose an automated generation approach using Large Language Models (LLMs). Our key contributions include the establishment of CardBench, a comprehensive dataset aggregated from over 4.8k model cards and 1.4k data cards, coupled with the development of the CardGen pipeline comprising a two-step retrieval process. Our approach exhibits enhanced completeness, objectivity, and faithfulness in generated model and data cards, a significant step in responsible AI documentation practices ensuring better accountability and traceability.

cs.CL↗

arXiv2024

Chumor 1.0: A Truly Funny and Challenging Chinese Humor Understanding Dataset from Ruo Zhi Ba

Existing humor datasets and evaluations predominantly focus on English, lacking resources for culturally nuanced humor in non-English languages like Chinese. To address this gap, we construct Chumor, a dataset sourced from Ruo Zhi Ba (RZB), a Chinese Reddit-like platform dedicated to sharing intellectually challenging and culturally specific jokes. We annotate explanations for each joke and evaluate human explanations against two state-of-the-art LLMs, GPT-4o and ERNIE Bot, through A/B testing by native Chinese speakers. Our evaluation shows that Chumor is challenging even for SOTA LLMs, and the human explanations for Chumor jokes are significantly better than explanations generated by the LLMs.

cs.CL↗

arXiv2024

Multi-Stream Keypoint Attention Network for Sign Language Recognition and Translation

Sign language serves as a non-vocal means of communication, transmitting information and significance through gestures, facial expressions, and bodily movements. The majority of current approaches for sign language recognition (SLR) and translation rely on RGB video inputs, which are vulnerable to fluctuations in the background. Employing a keypoint-based strategy not only mitigates the effects of background alterations but also substantially diminishes the computational demands of the model. Nevertheless, contemporary keypoint-based methodologies fail to fully harness the implicit knowledge embedded in keypoint sequences. To tackle this challenge, our inspiration is derived from the human cognition mechanism, which discerns sign language by analyzing the interplay between gesture configurations and supplementary elements. We propose a multi-stream keypoint attention network to depict a sequence of keypoints produced by a readily available keypoint estimator. In order to facilitate interaction across multiple streams, we investigate diverse methodologies such as keypoint fusion strategies, head fusion, and self-distillation. The resulting framework is denoted as MSKA-SLR, which is expanded into a sign language translation (SLT) model through the straightforward addition of an extra translation network. We carry out comprehensive experiments on well-known benchmarks like Phoenix-2014, Phoenix-2014T, and CSL-Daily to showcase the efficacy of our methodology. Notably, we have attained a novel state-of-the-art performance in the sign language translation task of Phoenix-2014T. The code and models can be accessed at: https://github.com/sutwangyan/MSKA.

cs.CV↗

arXiv2024

Analyzing the Role of Semantic Representations in the Era of Large Language Models

Traditionally, natural language processing (NLP) models often use a rich set of features created by linguistic expertise, such as semantic representations. However, in the era of large language models (LLMs), more and more tasks are turned into generic, end-to-end sequence generation problems. In this paper, we investigate the question: what is the role of semantic representations in the era of LLMs? Specifically, we investigate the effect of Abstract Meaning Representation (AMR) across five diverse NLP tasks. We propose an AMR-driven chain-of-thought prompting method, which we call AMRCoT, and find that it generally hurts performance more than it helps. To investigate what AMR may have to offer on these tasks, we conduct a series of analysis experiments. We find that it is difficult to predict which input examples AMR may help or hurt on, but errors tend to arise with multi-word expressions, named entities, and in the final inference step where the LLM must connect its reasoning over the AMR to its prediction. We recommend focusing on these areas for future work in semantic representations for LLMs. Our code: https://github.com/causalNLP/amr_llm.

cs.CL↗

arXiv2024

Can Large Language Models Infer Causation from Correlation?

Causal inference is one of the hallmarks of human intelligence. While the field of CausalNLP has attracted much interest in the recent years, existing causal inference datasets in NLP primarily rely on discovering causality from empirical knowledge (e.g., commonsense knowledge). In this work, we propose the first benchmark dataset to test the pure causal inference skills of large language models (LLMs). Specifically, we formulate a novel task Corr2Cause, which takes a set of correlational statements and determines the causal relationship between the variables. We curate a large-scale dataset of more than 200K samples, on which we evaluate seventeen existing LLMs. Through our experiments, we identify a key shortcoming of LLMs in terms of their causal inference skills, and show that these models achieve almost close to random performance on the task. This shortcoming is somewhat mitigated when we try to re-purpose LLMs for this skill via finetuning, but we find that these models still fail to generalize -- they can only perform causal inference in in-distribution settings when variable names and textual expressions used in the queries are similar to those in the training set, but fail in out-of-distribution settings generated by perturbing these queries. Corr2Cause is a challenging task for LLMs, and would be helpful in guiding future research on improving LLMs' pure reasoning skills and generalizability. Our data is at https://huggingface.co/datasets/causalnlp/corr2cause. Our code is at https://github.com/causalNLP/corr2cause.

cs.CL↗

arXiv2024

Bias Amplification Enhances Minority Group Performance

Neural networks produced by standard training are known to suffer from poor accuracy on rare subgroups despite achieving high accuracy on average, due to the correlations between certain spurious features and labels. Previous approaches based on worst-group loss minimization (e.g. Group-DRO) are effective in improving worse-group accuracy but require expensive group annotations for all the training samples. In this paper, we focus on the more challenging and realistic setting where group annotations are only available on a small validation set or are not available at all. We propose BAM, a novel two-stage training algorithm: in the first stage, the model is trained using a bias amplification scheme via introducing a learnable auxiliary variable for each training sample; in the second stage, we upweight the samples that the bias-amplified model misclassifies, and then continue training the same model on the reweighted dataset. Empirically, BAM achieves competitive performance compared with existing methods evaluated on spurious correlation benchmarks in computer vision and natural language processing. Moreover, we find a simple stopping criterion based on minimum class accuracy difference that can remove the need for group annotations, with little or no loss in worst-group accuracy. We perform extensive analyses and ablations to verify the effectiveness and robustness of our algorithm in varying class and group imbalance ratios.

cs.LG↗

arXiv2024

Differentiable Quantum Architecture Search For Job Shop Scheduling Problem

The Job shop scheduling problem (JSSP) plays a pivotal role in industrial applications, such as signal processing (SP) and steel manufacturing, involving sequencing machines and jobs to maximize scheduling efficiency. Before, JSSP was solved using manually defined circuits by variational quantum algorithm (VQA). Finding a good circuit architecture is task-specific and time-consuming. Differentiable quantum architecture search (DQAS) is a gradient-based framework that can automatically design circuits. However, DQAS is only tested on quantum approximate optimization algorithm (QAOA) and error mitigation tasks. Whether DQAS applies to JSSP based on a more flexible algorithm, such as variational quantum eigensolver (VQE), is still open for optimization problems. In this work, we redefine the operation pool and extend DQAS to a framework JSSP-DQAS by evaluating circuits to generate circuits for JSSP automatically. The experiments conclude that JSSP-DQAS can automatically find noise-resilient circuit architectures that perform much better than manually designed circuits. It helps to improve the efficiency of solving JSSP.

quant-ph↗

arXiv2023

Interactive Molecular Discovery with Natural Language

Natural language is expected to be a key medium for various human-machine interactions in the era of large language models. When it comes to the biochemistry field, a series of tasks around molecules (e.g., property prediction, molecule mining, etc.) are of great significance while having a high technical threshold. Bridging the molecule expressions in natural language and chemical language can not only hugely improve the interpretability and reduce the operation difficulty of these tasks, but also fuse the chemical knowledge scattered in complementary materials for a deeper comprehension of molecules. Based on these benefits, we propose the conversational molecular design, a novel task adopting natural language for describing and editing target molecules. To better accomplish this task, we design ChatMol, a knowledgeable and versatile generative pre-trained model, enhanced by injecting experimental property information, molecular spatial knowledge, and the associations between natural and chemical languages into it. Several typical solutions including large language models (e.g., ChatGPT) are evaluated, proving the challenge of conversational molecular design and the effectiveness of our knowledge enhancement method. Case observations and analysis are conducted to provide directions for further exploration of natural-language interaction in molecular discovery.

cs.CL↗

arXiv2022

Prevailing charge order in overdoped cuprates beyond the superconducting dome

The extremely overdoped cuprates are generally considered to be Fermi liquid metals without exotic orders, whereas the underdoped cuprates harbor intertwined states. Contrary to this conventional wisdom, using Cu $L_3$ edge and O $K$ edge resonant x-ray scattering, we reveal a charge order (CO) in overdoped La$_{2-x}$Sr$_x$CuO$_4$ (0.35 $\leq$ x $\leq$ 0.6) beyond the superconducting dome. This CO has a periodicity of $\sim$ 6 lattice units with correlation lengths of $\sim 3 - 20$ lattice units. It shows similar in-plane momentum and polarization dependence and dispersive excitations as the CO of underdoped cuprates, but its maximum intensity differs along the c-direction and persists up to 300 K. This CO cannot be explained by either the Fermi surface instability or the doped Hubbard model and its origin remains to be understood. Our results suggest that CO is prevailing in the overdoped metallic regime and superconductivity emerges out of the CO phase upon decreasing hole carriers.

cond-mat.supr-con↗

arXiv2021

Sequential Search Models: A Pairwise Maximum Rank Approach

This paper studies sequential search models that (1) incorporate unobserved product quality, which can be correlated with endogenous observable characteristics (such as price) and endogenous search cost variables (such as product rankings in online search intermediaries); and (2) do not require researchers to know the true distribution of the match value between consumers and products. A likelihood approach to estimate such models gives biased results. Therefore, I propose a new estimator -- pairwise maximum rank (PMR) estimator -- for both preference and search cost parameters. I show that the PMR estimator is consistent using only data on consumers' search order among one pair of products rather than data on consumers' full consideration set or final purchase. Additionally, we can use the PMR estimator to test for the true match value distribution in the data. In the empirical application, I apply the PMR estimator to quantify the effect of rankings in Expedia hotel search using two samples of the data set, to which consumers are randomly assigned. I find the position effect to be \$0.11-\$0.36, and the effect estimated using the sample with randomly generated rankings is close to the effect estimated using the sample with endogenous rankings. Moreover, I find that the true match value distribution in the data is unlikely to be N(0,1). Likelihood estimation ignoring endogeneity gives an upward bias of at least \$1.17; misspecification of match value distribution as N(0,1) gives an upward bias of at least \$2.99.

econ.EM↗

arXiv2019

Copositivity for 3rd order symmetric tensors and applications

The strict opositivity of 4th order symmetric tensor may apply to detect vacuum stability of general scalar potential. For finding analytical expressions of (strict) opositivity of 4th order symmetric tensor, we may reduce its order to 3rd order to better deal with it. So, it is provided that several analytically sufficient conditions for the copositivity of 3th order 2 dimensional (3 dimensional) symmetric tensors. Subsequently, applying these conclusions to 4th order tensors, the analytically sufficient conditions of copositivity are proved for 4th order 2 dimensional and 3 dimensional symmetric tensors. Finally, we apply these results to present analytical vacuum stability conditions for vacuum stability for $\mathbb{Z}_3$ scalar dark matter.

math-ph↗