arXiv subjects

Haoran Luo

Publications and source records attributed to Haoran Luo.

87 records · Page 5Linked to original sources

arXiv2024

CP-Prompt: Composition-Based Cross-modal Prompting for Domain-Incremental Continual Learning

The key challenge of cross-modal domain-incremental learning (DIL) is to enable the learning model to continuously learn from novel data with different feature distributions under the same task without forgetting old ones. However, existing top-performing methods still cause high forgetting rates, by lacking intra-domain knowledge extraction and inter-domain common prompting strategy. In this paper, we propose a simple yet effective framework, CP-Prompt, by training limited parameters to instruct a pre-trained model to learn new domains and avoid forgetting existing feature distributions. CP-Prompt captures intra-domain knowledge by compositionally inserting personalized prompts on multi-head self-attention layers and then learns the inter-domain knowledge with a common prompting strategy. CP-Prompt shows superiority compared with state-of-the-art baselines among three widely evaluated DIL tasks. The source code is available at https://github.com/dannis97500/CP_Prompt.

cs.CL↗

arXiv2024

CRE-LLM: A Domain-Specific Chinese Relation Extraction Framework with Fine-tuned Large Language Model

Domain-Specific Chinese Relation Extraction (DSCRE) aims to extract relations between entities from domain-specific Chinese text. Despite the rapid development of PLMs in recent years, especially LLMs, DSCRE still faces three core challenges: complex network structure design, poor awareness, and high consumption of fine-tuning. Given the impressive performance of large language models (LLMs) in natural language processing, we propose a new framework called CRE-LLM. This framework is based on fine-tuning open-source LLMs, such as Llama-2, ChatGLM2, and Baichuan2. CRE-LLM enhances the logic-awareness and generative capabilities of the model by constructing an appropriate prompt and utilizing open-source LLMs for instruction-supervised fine-tuning. And then it directly extracts the relations of the given entities in the input textual data, which improving the CRE approach. To demonstrate the effectiveness of the proposed framework, we conducted extensive experiments on two domain-specific CRE datasets, FinRE and SanWen. The experimental results show that CRE-LLM is significantly superior and robust, achieving state-of-the-art (SOTA) performance on the FinRE dataset. This paper introduces a novel approach to domain-specific relation extraction (DSCRE) tasks that are semantically more complex by combining LLMs with triples. Our code is publicly available.

cs.CL↗

arXiv2024

Turán density of long tight cycle minus one hyperedge

Denote by $\mathcal{C}^-_{\ell}$ the $3$-uniform hypergraph obtained by removing one hyperedge from the tight cycle on $\ell$ vertices. It is conjectured that the Turán density of $\mathcal{C}^-_{5}$ is $1/4$. In this paper, we make progress toward this conjecture by proving that the Turán density of $\mathcal{C}^-_{\ell}$ is $1/4$, for every sufficiently large $\ell$ not divisible by $3$. One of the main ingredients of our proof is a forbidden-subhypergraph characterization of the hypergraphs, for which there exists a tournament on the same vertex set such that every hyperedge is a cyclic triangle in this tournament. A byproduct of our method is a human-checkable proof for the upper bound on the maximum number of almost similar triangles in a planar point set, which was recently proved using the method of flag algebras by Balogh, Clemen, and Lidický.

math.CO↗

arXiv2023

TFLEX: Temporal Feature-Logic Embedding Framework for Complex Reasoning over Temporal Knowledge Graph

Multi-hop logical reasoning over knowledge graph (KG) plays a fundamental role in many artificial intelligence tasks. Recent complex query embedding (CQE) methods for reasoning focus on static KGs, while temporal knowledge graphs (TKGs) have not been fully explored. Reasoning over TKGs has two challenges: 1. The query should answer entities or timestamps; 2. The operators should consider both set logic on entity set and temporal logic on timestamp set. To bridge this gap, we define the multi-hop logical reasoning problem on TKGs. With generated three datasets, we propose the first temporal CQE named Temporal Feature-Logic Embedding framework (TFLEX) to answer the temporal complex queries. We utilize vector logic to compute the logic part of Temporal Feature-Logic embeddings, thus naturally modeling all First-Order Logic (FOL) operations on entity set. In addition, our framework extends vector logic on timestamp set to cope with three extra temporal operators (After, Before and Between). Experiments on numerous query patterns demonstrate the effectiveness of our method.

cs.LG↗

arXiv2023

On the Maximum $F_5$-free Subhypergraphs of a Random Hypergraph

Denote by $F_5$ the $3$-uniform hypergraph on vertex set $\{1,2,3,4,5\}$ with hyperedges $\{123,124,345\}$. Balogh, Butterfield, Hu, and Lenz proved that if $p > K \log n / n$ for some large constant $K$, then every maximum $F_5$-free subhypergraph of $G^3(n,p)$ is tripartite with high probability, and showed that if $p_0 = 0.1\sqrt{\log n} / n$, then with high probability there exists a maximum $F_5$-free subhypergraph of $G^3(n,p_0)$ that is not tripartite. In this paper, we sharpen the upper bound to be best possible up to a constant factor. We prove that if $p > C \sqrt{\log n} / n $ for some large constant $C$, then every maximum $F_5$-free subhypergraph of $G^3(n, p)$ is tripartite with high probability.

math.CO↗

arXiv2023

HAHE: Hierarchical Attention for Hyper-Relational Knowledge Graphs in Global and Local Level

Link Prediction on Hyper-relational Knowledge Graphs (HKG) is a worthwhile endeavor. HKG consists of hyper-relational facts (H-Facts), composed of a main triple and several auxiliary attribute-value qualifiers, which can effectively represent factually comprehensive information. The internal structure of HKG can be represented as a hypergraph-based representation globally and a semantic sequence-based representation locally. However, existing research seldom simultaneously models the graphical and sequential structure of HKGs, limiting HKGs' representation. To overcome this limitation, we propose a novel Hierarchical Attention model for HKG Embedding (HAHE), including global-level and local-level attention. The global-level attention can model the graphical structure of HKG using hypergraph dual-attention layers, while the local-level attention can learn the sequential structure inside H-Facts via heterogeneous self-attention layers. Experiment results indicate that HAHE achieves state-of-the-art performance in link prediction tasks on HKG standard datasets. In addition, HAHE addresses the issue of HKG multi-position prediction for the first time, increasing the applicability of the HKG link prediction task. Our code is publicly available.

cs.AI↗

arXiv2023

FLEX: Feature-Logic Embedding Framework for CompleX Knowledge Graph Reasoning

Current best performing models for knowledge graph reasoning (KGR) introduce geometry objects or probabilistic distributions to embed entities and first-order logical (FOL) queries into low-dimensional vector spaces. They can be summarized as a center-size framework (point/box/cone, Beta/Gaussian distribution, etc.). However, they have limited logical reasoning ability. And it is difficult to generalize to various features, because the center and size are one-to-one constrained, unable to have multiple centers or sizes. To address these challenges, we instead propose a novel KGR framework named Feature-Logic Embedding framework, FLEX, which is the first KGR framework that can not only TRULY handle all FOL operations including conjunction, disjunction, negation and so on, but also support various feature spaces. Specifically, the logic part of feature-logic framework is based on vector logic, which naturally models all FOL operations. Experiments demonstrate that FLEX significantly outperforms existing state-of-the-art methods on benchmark datasets.

cs.AI↗

arXiv2023

NQE: N-ary Query Embedding for Complex Query Answering over Hyper-Relational Knowledge Graphs

Complex query answering (CQA) is an essential task for multi-hop and logical reasoning on knowledge graphs (KGs). Currently, most approaches are limited to queries among binary relational facts and pay less attention to n-ary facts (n>=2) containing more than two entities, which are more prevalent in the real world. Moreover, previous CQA methods can only make predictions for a few given types of queries and cannot be flexibly extended to more complex logical queries, which significantly limits their applications. To overcome these challenges, in this work, we propose a novel N-ary Query Embedding (NQE) model for CQA over hyper-relational knowledge graphs (HKGs), which include massive n-ary facts. The NQE utilizes a dual-heterogeneous Transformer encoder and fuzzy logic theory to satisfy all n-ary FOL queries, including existential quantifiers, conjunction, disjunction, and negation. We also propose a parallel processing algorithm that can train or predict arbitrary n-ary FOL queries in a single batch, regardless of the kind of each query, with good flexibility and extensibility. In addition, we generate a new CQA dataset WD50K-NFOL, including diverse n-ary FOL queries over WD50K. Experimental results on WD50K-NFOL and other standard CQA datasets show that NQE is the state-of-the-art CQA method over HKGs with good generalization capability. Our code and dataset are publicly available.

cs.AI↗

arXiv2023

DHGE: Dual-View Hyper-Relational Knowledge Graph Embedding for Link Prediction and Entity Typing

In the field of representation learning on knowledge graphs (KGs), a hyper-relational fact consists of a main triple and several auxiliary attribute-value descriptions, which is considered more comprehensive and specific than a triple-based fact. However, currently available hyper-relational KG embedding methods in a single view are limited in application because they weaken the hierarchical structure that represents the affiliation between entities. To overcome this limitation, we propose a dual-view hyper-relational KG structure (DH-KG) that contains a hyper-relational instance view for entities and a hyper-relational ontology view for concepts that are abstracted hierarchically from the entities. This paper defines link prediction and entity typing tasks on DH-KG for the first time and constructs two DH-KG datasets, JW44K-6K, extracted from Wikidata, and HTDM based on medical data. Furthermore, we propose DHGE, a DH-KG embedding model based on GRAN encoders, HGNNs, and joint learning. DHGE outperforms baseline models on DH-KG, according to experimental results. Finally, we provide an example of how this technology can be used to treat hypertension. Our model and new datasets are publicly available.

cs.AI↗

arXiv2023

Maximal $3$-wise intersecting families

A family $\mathcal{F}$ on ground set $[n]:=\{1,2,\ldots, n\}$ is maximal $k$-wise intersecting if every collection of at most $k$ sets in $\mathcal{F}$ has non-empty intersection, and no other set can be added to $\mathcal{F}$ while maintaining this property. In 1974, Erdős and Kleitman asked for the minimum size of a maximal $k$-wise intersecting family. We answer their question for $k=3$ and sufficiently large $n$. We show that the unique minimum family is obtained by partitioning the ground set $[n]$ into two sets $A$ and $B$ with almost equal sizes and taking the family consisting of all the proper supersets of $A$ and of $B$.

math.CO↗

arXiv2022

Non-degenerate Hypergraphs with Exponentially Many Extremal Constructions

For every integer $t \ge 0$, denote by $F_5^t$ the hypergraph on vertex set $\{1,2,\ldots, 5+t\}$ with hyperedges $\{123,124\} \cup \{34k : 5 \le k \le 5+t\}$. We determine $\mathrm{ex}(n,F_5^t)$ for every $t\ge 0$ and sufficiently large $n$ and characterize the extremal $F_5^t$-free hypergraphs. In particular, if $n$ satisfies certain divisibility conditions, then the extremal $F_5^t$-free hypergraphs are exactly the balanced complete tripartite hypergraphs with additional hyperedges inside each of the three parts $(V_1,V_2,V_3)$ in the partition; each part $V_i$ spans a $(|V_i|,3,2,t)$-design. This generalizes earlier work of Frankl and Füredi on the Turán number of $F_5:=F_5^0$. Our results extend a theory of Erdős and Simonovits about the extremal constructions for certain fixed graphs. In particular, the hypergraphs $F_5^{6t}$, for $t\geq 1$, are the first examples of hypergraphs with exponentially many extremal constructions and positive Turán density.

math.CO↗

arXiv2022

Maximal 3-wise Intersecting Families with Minimum Size: the Odd Case

A family $\mathcal{F}$ on ground set $\{1,2,\ldots, n\}$ is maximal $k$-wise intersecting if every collection of $k$ sets in $\mathcal{F}$ has non-empty intersection, and no other set can be added to $\mathcal{F}$ while maintaining this property. Erdős and Kleitman asked for the minimum size of a maximal $k$-wise intersecting family. Complementing earlier work of Hendrey, Lund, Tompkins and Tran, who answered this question for $k=3$ and large even $n$, we answer it for $k=3$ and large odd $n$. We show that the unique minimum family is obtained by partitioning the ground set into two sets $A$ and $B$ with almost equal sizes and taking the family consisting of all the proper supersets of $A$ and of $B$. A key ingredient of our proof is the stability result by Ellis and Sudakov about the so-called $2$-generator set systems.

math.CO↗

arXiv2022

Sharp threshold for the Erdős-Ko-Rado theorem

For positive integers $n$ and $k$ with $n\geq 2k+1$, the Kneser graph $K(n,k)$ is the graph with vertex set consisting of all $k$-sets of $\{1,\dots,n\}$, where two $k$-sets are adjacent exactly when they are disjoint. The independent sets of $K(n,k)$ are $k$-uniform intersecting families, and hence the maximum size independent sets are given by the Erdős-Ko-Rado Theorem. Let $K_p(n,k)$ be a random spanning subgraph of $K(n,k)$ where each edge is included independently with probability $p$. Bollobás, Narayanan, and Raigorodskii asked for what $p$ does $K_p(n,k)$ have the same independence number as $K(n,k)$ with high probability. For $n=2k+1$, we prove a hitting time result, which gives a sharp threshold for this problem at $p=3/4$. Additionally, completing work of Das and Tran and work of Devlin and Kahn, we determine a sharp threshold function for all $n>2k+1$.

math.CO↗

arXiv2021

EchoEA: Echo Information between Entities and Relations for Entity Alignment

Entity alignment (EA) plays an important role in automatically integrating knowledge graphs (KGs) from multiple sources. Recent approaches based on Graph Neural Network (GNN) obtain entity representation from relation information and have achieved promising results. Besides, more and more methods introduce semi-supervision to ask for more labeled training data. However, two challenges still exist in GNN-based EA methods: (1) Deeper GNN Encoder: The GNN encoder of current methods has limited depth (usually 2-layers). (2) Low-quality Bootstrapping: The generated semi-supervised data is of low quality. In this paper, we propose a novel framework, Echo Entity Alignment (EchoEA), which leverages 4-levels self-attention mechanism to spread entity information to relations and echo back to entities. Furthermore, we propose attribute-combined bi-directional global-filtered strategy (ABGS) to improve bootstrapping, reduce false samples and generate high-quality training data. The experimental results on three real-world cross-lingual datasets are stable at around 96\% at hits@1 on average, showing that our approach not only significantly outperforms the state-of-the-art GNN-based methods, but also is universal and transferable for existing EA methods.

cs.CL↗

arXiv2021

On the number of forests and connected spanning subgraphs

Let $F(G)$ be the number of forests of a graph $G$. Similarly let $C(G)$ be the number of connected spanning subgraphs of a connected graph $G$. We bound $F(G)$ and $C(G)$ for regular graphs and for graphs with fixed average degree. Among many other things we study $f_d=\sup_{G\in \mathcal{G}_d}F(G)^{1/v(G)}$, where $\mathcal{G}_d$ is the family of $d$--regular graphs, and $v(G)$ denotes the number of vertices of a graph $G$. We show that $f_3=2^{3/2}$, and if $(G_n)_n$ is a sequence of $3$--regular graphs with length of the shortest cycle tending to infinity, then $\lim_{n\to \infty}F(G_n)^{1/v(G_n)}=2^{3/2}$. We also improve on the previous best bounds on $f_d$ for $4\leq d\leq 9$.

math.CO↗