arXiv subjects

Jing Jiang

Publications and source records attributed to Jing Jiang.

At least 217 records · Page 12Linked to original sources

arXiv2018

Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling

Many natural language processing tasks solely rely on sparse dependencies between a few tokens in a sentence. Soft attention mechanisms show promising performance in modeling local/global dependencies by soft probabilities between every two tokens, but they are not effective and efficient when applied to long sentences. By contrast, hard attention mechanisms directly select a subset of tokens but are difficult and inefficient to train due to their combinatorial nature. In this paper, we integrate both soft and hard attention into one context fusion model, "reinforced self-attention (ReSA)", for the mutual benefit of each other. In ReSA, a hard attention trims a sequence for a soft self-attention to process, while the soft attention feeds reward signals back to facilitate the training of the hard one. For this purpose, we develop a novel hard attention called "reinforced sequence sampling (RSS)", selecting tokens in parallel and trained via policy gradient. Using two RSS modules, ReSA efficiently extracts the sparse dependencies between each pair of selected tokens. We finally propose an RNN/CNN-free sentence-encoding model, "reinforced self-attention network (ReSAN)", solely based on ReSA. It achieves state-of-the-art performance on both Stanford Natural Language Inference (SNLI) and Sentences Involving Compositional Knowledge (SICK) datasets.

cs.CL↗

arXiv2018

Reduced Transmission in Multi-Server Coded Caching

Coded caching has been widely used in the wireless network for shifting the some transmissions during the peak traffic times to the off-peak traffic times. Multi-server coded caching, which can share responsibility for the total amount of transmission in the wireless network during the peak traffic times by means of the collaboration among these servers, can be seen everywhere in our life. The three servers setting (two data servers and one parity check server) is used in practice, e.g. redundant array of independent disks-4. In this scenario, there are total $N$ files which are equally stored in two data servers respectively and $K$ users each of which has the memory size of $M$ files. Each server connects to users by an independently channel. During the off-peak traffic times, two data servers place some parts of each files in each user's cache. In that time, servers do not know users' requests in future. During the peak traffic times each user just requests one file from $N$ files. Luo et al. in 2016 proposed the first coded caching scheme for this setting. In this paper, we proposed some method that further reduces the amount of transmission in each channel when $\frac{KM}{N}$ is odd. This method also improves the transmission rate for systems with general multiply servers.

cs.IT↗

arXiv2018

Constructions of Coded Caching Schemes with Flexible Memory Size

Coded caching scheme recently has become quite popular in the wireless network due to its effectively reducing the transmission amount (denote such an amount by $R$) during peak traffic times. However to realize a coded caching scheme, each file must be divided into $F$ packets which usually increases the computation complexity of a coded caching scheme. So we prefer to construct a caching scheme that decreases the order of $F$ for practical implementations. In this paper, we construct four classes of new schemes where two classes can significantly reduce the value of $F$ by increasing a little $R$ comparing with the well known scheme proposed by Maddah-Ali and Niesen, and $F$ in the other two classes grows sub-exponentially with $K$ by sacrificing more $R$. It is worth noting that a tradeoff between $R$ and $F$, which is a hot topic in the field of caching scheme, is proposed by our constructions. In addition, our constructions include all the results constructed by Yan et al., (IEEE Trans. Inf. Theory 63, 5821-5833, 2017) and some main results obtained by Shangguan et al., (arXiv preprint arXiv:1608.03989v1) as the special cases.

cs.IT↗

arXiv2018

A Co-Matching Model for Multi-choice Reading Comprehension

Multi-choice reading comprehension is a challenging task, which involves the matching between a passage and a question-answer pair. This paper proposes a new co-matching approach to this problem, which jointly models whether a passage can match both a question and a candidate answer. Experimental results on the RACE dataset demonstrate that our approach achieves state-of-the-art performance.

cs.CL↗

arXiv2018

ARUM: Polar Coded HARQ Scheme based on Incremental Channel Polarization

A hybrid ARQ (HARQ) scheme for polar code, which is called active-bit relocation under masks (ARUM), is proposed. In each transmission, the data bits are encoded and bit-wisely XOR-masked using a binary vector before being transmitted through the channel. The masking process combines multiple transmissions together which forms another step of inter-transmission channel transform. The reliabilities are updated after every transmission, and the less reliable bits in earlier ones are relocated to the more reliable positions at the latest transmitted block. ARUM is a very flexible HARQ scheme which allows each transmission to have a different mother code length and to adopt independent rate-matching scheme with sufficient channel state feedback in HARQ process. Simulation shows that ARUM can obtain near-optimal coding gain.

cs.IT↗

arXiv2018

Evidence Aggregation for Answer Re-Ranking in Open-Domain Question Answering

A popular recent approach to answering open-domain questions is to first search for question-related passages and then apply reading comprehension models to extract answers. Existing methods usually extract answers from single passages independently. But some questions require a combination of evidence from across different sources to answer correctly. In this paper, we propose two models which make use of multiple passages to generate their answers. Both use an answer-reranking approach which reorders the answer candidates generated by an existing state-of-the-art QA model. We propose two methods, namely, strength-based re-ranking and coverage-based re-ranking, to make use of the aggregated evidence from different passages to better determine the answer. Our models have achieved state-of-the-art results on three public open-domain QA datasets: Quasar-T, SearchQA and the open-domain version of TriviaQA, with about 8 percentage points of improvement over the former two datasets.

cs.CL↗

arXiv2018

Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Modeling

Recurrent neural networks (RNN), convolutional neural networks (CNN) and self-attention networks (SAN) are commonly used to produce context-aware representations. RNN can capture long-range dependency but is hard to parallelize and not time-efficient. CNN focuses on local dependency but does not perform well on some tasks. SAN can model both such dependencies via highly parallelizable computation, but memory requirement grows rapidly in line with sequence length. In this paper, we propose a model, called "bi-directional block self-attention network (Bi-BloSAN)", for RNN/CNN-free sequence encoding. It requires as little memory as RNN but with all the merits of SAN. Bi-BloSAN splits the entire sequence into blocks, and applies an intra-block SAN to each block for modeling local context, then applies an inter-block SAN to the outputs for all blocks to capture long-range dependency. Thus, each SAN only needs to process a short sequence, and only a small amount of memory is required. Additionally, we use feature-level attention to handle the variation of contexts around the same word, and use forward/backward masks to encode temporal order information. On nine benchmark datasets for different NLP tasks, Bi-BloSAN achieves or improves upon state-of-the-art accuracy, and shows better efficiency-memory trade-off than existing RNN/CNN/SAN.

cs.CL↗

arXiv2017

A Novel Recursive Construction for Coded Caching Schemes

As a strategy to further reduce the transmission pressure during the peak traffic times in wireless network, coded caching has been widely studied recently. And several coded caching schemes are constructed focusing on the two core problems in practice, i.e., the rate transmitted during the peak traffic times and the packet number of each file divided during the off peak traffic times. It is well known that there exits a tradeoff between the rate and the packet number. In this paper, a novel recursive construction is proposed. As an application, several new schemes are obtained. Comparing with previously known schemes, new schemes could further reduce packet number by increasing little rate. And for some parameters in coded caching systems, the packet number of our new schemes are smaller than that of schemes generated by memory sharing method which is widely used in the field of caching. By the way our new schemes include all the results constructed by Tang et al., (IEEE ISIT, 2790-2794, 2017) as special cases.

cs.IT↗

arXiv2017

Modelling Domain Relationships for Transfer Learning on Retrieval-based Question Answering Systems in E-commerce

In this paper, we study transfer learning for the PI and NLI problems, aiming to propose a general framework, which can effectively and efficiently adapt the shared knowledge learned from a resource-rich source domain to a resource- poor target domain. Specifically, since most existing transfer learning methods only focus on learning a shared feature space across domains while ignoring the relationship between the source and target domains, we propose to simultaneously learn shared representations and domain relationships in a unified framework. Furthermore, we propose an efficient and effective hybrid model by combining a sentence encoding- based method and a sentence interaction-based method as our base model. Extensive experiments on both paraphrase identification and natural language inference demonstrate that our base model is efficient and has promising performance compared to the competing models, and our transfer learning method can help to significantly boost the performance. Further analysis shows that the inter-domain and intra-domain relationship captured by our model are insightful. Last but not least, we deploy our transfer learning model for PI into our online chatbot system, which can bring in significant improvements over our existing system. Finally, we launch our new system on the chatbot platform Eva in our E-commerce site AliExpress.

cs.CL↗

arXiv2017

R$^3$: Reinforced Reader-Ranker for Open-Domain Question Answering

In recent years researchers have achieved considerable success applying neural network methods to question answering (QA). These approaches have achieved state of the art results in simplified closed-domain settings such as the SQuAD (Rajpurkar et al., 2016) dataset, which provides a pre-selected passage, from which the answer to a given question may be extracted. More recently, researchers have begun to tackle open-domain QA, in which the model is given a question and access to a large corpus (e.g., wikipedia) instead of a pre-selected passage (Chen et al., 2017a). This setting is more complex as it requires large-scale search for relevant passages by an information retrieval component, combined with a reading comprehension model that "reads" the passages to generate an answer to the question. Performance in this setting lags considerably behind closed-domain performance. In this paper, we present a novel open-domain QA system called Reinforced Ranker-Reader $(R^3)$, based on two algorithmic innovations. First, we propose a new pipeline for open-domain QA with a Ranker component, which learns to rank retrieved passages in terms of likelihood of generating the ground-truth answer to a given question. Second, we propose a novel method that jointly trains the Ranker along with an answer-generation Reader model, based on reinforcement learning. We report extensive experimental results showing that our method significantly improves on the state of the art for multiple open-domain QA datasets.

cs.CL↗

arXiv2017

DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding

Recurrent neural nets (RNN) and convolutional neural nets (CNN) are widely used on NLP tasks to capture the long-term and local dependencies, respectively. Attention mechanisms have recently attracted enormous interest due to their highly parallelizable computation, significantly less training time, and flexibility in modeling dependencies. We propose a novel attention mechanism in which the attention between elements from input sequence(s) is directional and multi-dimensional (i.e., feature-wise). A light-weight neural net, "Directional Self-Attention Network (DiSAN)", is then proposed to learn sentence embedding, based solely on the proposed attention without any RNN/CNN structure. DiSAN is only composed of a directional self-attention with temporal order encoded, followed by a multi-dimensional attention that compresses the sequence into a vector representation. Despite its simple form, DiSAN outperforms complicated RNN models on both prediction quality and time efficiency. It achieves the best test accuracy among all sentence encoding methods and improves the most recent best result by 1.02% on the Stanford Natural Language Inference (SNLI) dataset, and shows state-of-the-art test accuracy on the Stanford Sentiment Treebank (SST), Multi-Genre natural language inference (MultiNLI), Sentences Involving Compositional Knowledge (SICK), Customer Review, MPQA, TREC question-type classification and Subjectivity (SUBJ) datasets.

cs.CL↗

arXiv2017

Coded Caching Schemes with Low Rate and Subpacketizations

Coded caching scheme, which is an effective technique to increase the transmission efficiency during peak traffic times, has recently become quite popular among the coding community. Generally rate can be measured to the transmission in the peak traffic times, i.e., this efficiency increases with the decreasing of rate. In order to implement a coded caching scheme, each file in the library must be split in a certain number of packets. And this number directly reflects the complexity of a coded caching scheme, i.e., the complexity increases with the increasing of the packet number. However there exists a tradeoff between the rate and packet number. So it is meaningful to characterize this tradeoff and design the related Pareto-optimal coded caching schemes with respect to both parameters. Recently, a new concept called placement delivery array (PDA) was proposed to characterize the coded caching scheme. However as far as we know no one has yet proved that one of the previously known PDAs is Pareto-optimal. In this paper, we first derive two lower bounds on the rate under the framework of PDA. Consequently, the PDA proposed by Maddah-Ali and Niesen is Pareto-optimal, and a tradeoff between rate and packet number is obtained for some parameters. Then, from the above observations and the view point of combinatorial design, two new classes of Pareto-optimal PDAs are obtained. Based on these PDAs, the schemes with low rate and packet number are obtained. Finally the performance of some previously known PDAs are estimated by comparing with these two classes of schemes.

cs.IT↗

arXiv2017

Bounds and Constructions for $\overline{3}$-Strongly Separable Codes with Length $3$

As separable code (SC, IEEE Trans Inf Theory 57:4843-4851, 2011) and frameproof code (FPC, IEEE Trans Inf Theory 44:1897-1905, 1998) do in multimedia fingerprinting, strongly separable code (SSC, Des. Codes and Cryptogr.79:303-318, 2016) can be also used to construct anti-collusion codes. Furthermore, SSC is better than FPC and SC in the applications for multimedia fingerprinting since SSC has lower tracing complexity than that of SC (the same complexity as FPC) and weaker structure than that of FPC. In this paper, we first derive several upper bounds on the number of codewords of $\overline{t}$-SSC. Then we focus on $\overline{3}$-SSC with codeword length $3$, and obtain the following two main results: (1) An equivalence between an SSC and an SC. %Consequently a more tighter upper bound $(3q^2/4)$ and lower bound $(q^{3/2})$ on the number of codewords are obtained; (2) An improved lower bound $Ω(q^{5/3}+q^{4/3}-q)$ on the size of a $q$-ary SSC when $q=q_1^6$ for any prime power $q_1\equiv\ 1 \pmod 6$, better than the before known bound $\lfloor\sqrt{q}\rfloor^{3}$, which is obtained by means of difference matrix and the known result on the subset of $\mathbb{F}^{n}_q$ containing no three points on a line.

cs.DM↗

arXiv2017

The Optimal Placement Delivery Arrays

In wireless networks, coded caching is an effective technique to reduce network congestion during peak traffic times. Recently, a new concept called placement delivery array (PDA) was proposed to characterize the coded caching scheme. So far, only one class of PDAs by Maddah-Ali and Niesen is known to be optimal. In this paper, we mainly focus on constructing optimal PDAs. Firstly, we derive some lower bounds. Next, we present several infinite classes of PDAs, which are shown to be optimal with respect to the new bounds.

cs.IT↗

arXiv2016

Unified Framework for the Effective Rate Analysis of Wireless Communication Systems over MISO Fading Channels

This paper proposes a unified framework for the effective rate analysis over arbitrary correlated and not necessarily identical multiple inputs single output (MISO) fading channels, which uses moment generating function (MGF) based approach and H transform representation. The proposed framework has the potential to simplify the cumbersome analysis procedure compared to the probability density function (PDF) based approach. Moreover, the effective rates over two specific fading scenarios are investigated, namely independent but not necessarily identical distributed (i.n.i.d.) MISO hyper Fox's H fading channels and arbitrary correlated generalized K fading channels. The exact analytical representations for these two scenarios are also presented. By substituting corresponding parameters, the effective rates in various practical fading scenarios, such as Rayleigh, Nakagami-m, Weibull/Gamma and generalized K fading channels, are readily available. In addition, asymptotic approximations are provided for the proposed H transform and MGF based approach as well as for the effective rate over i.n.i.d. MISO hyper Fox's H fading channels. Simulations under various fading scenarios are also presented, which support the validity of the proposed method.

cs.IT↗

arXiv2016

Learning Natural Language Inference with LSTM

Natural language inference (NLI) is a fundamentally important task in natural language processing that has many applications. The recently released Stanford Natural Language Inference (SNLI) corpus has made it possible to develop and evaluate learning-centered methods such as deep neural networks for natural language inference (NLI). In this paper, we propose a special long short-term memory (LSTM) architecture for NLI. Our model builds on top of a recently proposed neural attention model for NLI but is based on a significantly different idea. Instead of deriving sentence embeddings for the premise and the hypothesis to be used for classification, our solution uses a match-LSTM to perform word-by-word matching of the hypothesis with the premise. This LSTM is able to place more emphasis on important word-level matching results. In particular, we observe that this LSTM remembers important mismatches that are critical for predicting the contradiction or the neutral relationship label. On the SNLI corpus, our model achieves an accuracy of 86.1%, outperforming the state of the art.

cs.CL↗

arXiv2016

Machine Comprehension Using Match-LSTM and Answer Pointer

Machine comprehension of text is an important problem in natural language processing. A recently released dataset, the Stanford Question Answering Dataset (SQuAD), offers a large number of real questions and their answers created by humans through crowdsourcing. SQuAD provides a challenging testbed for evaluating machine comprehension algorithms, partly because compared with previous datasets, in SQuAD the answers do not come from a small set of candidate answers and they have variable lengths. We propose an end-to-end neural architecture for the task. The architecture is based on match-LSTM, a model we proposed previously for textual entailment, and Pointer Net, a sequence-to-sequence model proposed by Vinyals et al.(2015) to constrain the output tokens to be from the input sequences. We propose two ways of using Pointer Net for our task. Our experiments show that both of our two models substantially outperform the best results obtained by Rajpurkar et al.(2016) using logistic regression and manually crafted features.

cs.CL↗

arXiv2016

A Compare-Aggregate Model for Matching Text Sequences

Many NLP tasks including machine comprehension, answer selection and text entailment require the comparison between sequences. Matching the important units between sequences is a key to solve these problems. In this paper, we present a general "compare-aggregate" framework that performs word-level matching followed by aggregation using Convolutional Neural Networks. We particularly focus on the different comparison functions we can use to match two vectors. We use four different datasets to evaluate the model. We find that some simple comparison functions based on element-wise operations can work better than standard neural network and neural tensor network.

cs.CL↗