arXiv subjects

Lingjun Li

Publications and source records attributed to Lingjun Li.

6 recordsLinked to original sources

arXiv2025

A Perception CNN for Facial Expression Recognition

Convolutional neural networks (CNNs) can automatically learn data patterns to express face images for facial expression recognition (FER). However, they may ignore effect of facial segmentation of FER. In this paper, we propose a perception CNN for FER as well as PCNN. Firstly, PCNN can use five parallel networks to simultaneously learn local facial features based on eyes, cheeks and mouth to realize the sensitive capture of the subtle changes in FER. Secondly, we utilize a multi-domain interaction mechanism to register and fuse between local sense organ features and global facial structural features to better express face images for FER. Finally, we design a two-phase loss function to restrict accuracy of obtained sense information and reconstructed face images to guarantee performance of obtained PCNN in FER. Experimental results show that our PCNN achieves superior results on several lab and real-world FER benchmarks: CK+, JAFFE, FER2013, FERPlus, RAF-DB and Occlusion and Pose Variant Dataset. Its code is available at https://github.com/hellloxiaotian/PCNN.

cs.CV↗

arXiv2024

Yuan 2.0-M32: Mixture of Experts with Attention Router

Yuan 2.0-M32, with a similar base architecture as Yuan-2.0 2B, uses a mixture-of-experts architecture with 32 experts of which 2 experts are active. A new router network, Attention Router, is proposed and adopted for a more efficient selection of experts, which improves the accuracy compared to the model with classical router network. Yuan 2.0-M32 is trained with 2000B tokens from scratch, and the training computation consumption is only 9.25% of a dense model at the same parameter scale. Yuan 2.0-M32 demonstrates competitive capability on coding, math, and various domains of expertise, with only 3.7B active parameters of 40B in total, and 7.4 GFlops forward computation per token, both of which are only 1/19 of Llama3-70B. Yuan 2.0-M32 surpass Llama3-70B on MATH and ARC-Challenge benchmark, with accuracy of 55.89 and 95.8 respectively. The models and source codes of Yuan 2.0-M32 are released at Github1.

cs.AI↗

arXiv2023

YUAN 2.0: A Large Language Model with Localized Filtering-based Attention

In this work, we develop and release Yuan 2.0, a series of large language models with parameters ranging from 2.1 billion to 102.6 billion. The Localized Filtering-based Attention (LFA) is introduced to incorporate prior knowledge of local dependencies of natural language into Attention. A data filtering and generating system is presented to build pre-training and fine-tuning dataset in high quality. A distributed training method with non-uniform pipeline parallel, data parallel, and optimizer parallel is proposed, which greatly reduces the bandwidth requirements of intra-node communication, and achieves good performance in large-scale distributed training. Yuan 2.0 models display impressive ability in code generation, math problem-solving, and chatting compared with existing models. The latest version of YUAN 2.0, including model weights and source code, is accessible at Github.

cs.CL↗

arXiv2019

Online Change-Point Detection in High-Dimensional Covariance Structure with Application to Dynamic Networks

In this paper, we develop an online change-point detection procedure in the covariance structure of high-dimensional data. A new stopping rule is proposed to terminate the process as early as possible when a change in covariance structure occurs. The stopping rule allows temporal dependence and can be applied to non-Gaussian data. An explicit expression for the average run length (ARL) is derived, so that the level of threshold in the stopping rule can be easily obtained with no need to run time-consuming Monte Carlo simulations. We also establish an upper bound for the expected detection delay (EDD), the expression of which demonstrates the impact of data dependence and magnitude of change in the covariance structure. Simulation studies are provided to confirm accuracy of the theoretical results. The practical usefulness of the proposed procedure is illustrated by detecting the change of brain's covariance network in a resting-state fMRI dataset.

stat.ME↗

arXiv2019

Change Point Detection in the Mean of High-Dimensional Time Series Data under Dependence

High-dimensional time series are characterized by a large number of measurements and complex dependence, and often involve abrupt change points. We propose a new procedure to detect change points in the mean of high-dimensional time series data. The proposed procedure incorporates spatial and temporal dependence of data and is able to test and estimate the change point occurred on the boundary of time series. We study its asymptotic properties under mild conditions. Simulation studies demonstrate its robust performance through the comparison with other existing methods. Our procedure is applied to an fMRI dataset.

stat.ME↗

arXiv2017

A Neighborhood-Assisted Hotelling's $T^2$ Test for High-Dimensional Means

Many tests have been proposed to remedy the classical Hotelling's $T^2$ test in the "large $p$, small $n$" paradigm, but the existence of an optimal sum-of-squares type test has not been explored. This paper shows that under certain conditions, the population Hotelling's $T^2$ test with the known $\Sigma^{-1}$ attains the best power among all the $L_2$-norm based tests with the data transformation by $\Sigma^{\eta}$ for $\eta \in (-\infty, \infty)$. To extend the result to the case of unknown $\Sigma^{-1}$, we propose a Neighborhood-Assisted Hotelling's $T^2$ statistic obtained by replacing the inverse of sample covariance matrix in the classical Hotelling's $T^2$ statistic with a regularized covariance estimator. Utilizing a regression model, we establish its asymptotic normality under mild conditions. We show that the proposed test is able to match the performance of the population Hotelling's $T^2$ test under certain conditions, and thus possesses certain optimality. Moreover, it can adaptively attain the best power by empirically choosing a neighborhood size to maximize its signal-to-noise ratio. Simulation experiments and case studies are given to demonstrate the empirical performance of the proposed test.

stat.ME↗