arXiv subjects

Xin Yao

Publications and source records attributed to Xin Yao.

At least 127 records · Page 7Linked to original sources

arXiv2020

Federated Learning with Unbiased Gradient Aggregation and Controllable Meta Updating

Federated learning (FL) aims to train machine learning models in the decentralized system consisting of an enormous amount of smart edge devices. Federated averaging (FedAvg), the fundamental algorithm in FL settings, proposes on-device training and model aggregation to avoid the potential heavy communication costs and privacy concerns brought by transmitting raw data. However, through theoretical analysis we argue that 1) the multiple steps of local updating will result in gradient biases and 2) there is an inconsistency between the expected target distribution and the optimization objectives following the training paradigm in FedAvg. To tackle these problems, we first propose an unbiased gradient aggregation algorithm with the keep-trace gradient descent and the gradient evaluation strategy. Then we introduce an additional controllable meta updating procedure with a small set of data samples, indicating the expected target distribution, to provide a clear and consistent optimization objective. Both the two improvements are model- and task-agnostic and can be applied individually or together. Experimental results demonstrate that the proposed methods are faster in convergence and achieve higher accuracy with different network architectures in various FL settings.

cs.LG↗

arXiv2020

How to Evaluate Solutions in Pareto-based Search-Based Software Engineering? A Critical Review and Methodological Guidance

With modern requirements, there is an increasing tendency of considering multiple objectives/criteria simultaneously in many Software Engineering (SE) scenarios. Such a multi-objective optimization scenario comes with an important issue -- how to evaluate the outcome of optimization algorithms, which typically is a set of incomparable solutions (i.e., being Pareto non-dominated to each other). This issue can be challenging for the SE community, particularly for practitioners of Search-Based SE (SBSE). On one hand, multi-objective optimization could still be relatively new to SE/SBSE researchers, who may not be able to identify the right evaluation methods for their problems. On the other hand, simply following the evaluation methods for general multi-objective optimization problems may not be appropriate for specific SE problems, especially when the problem nature or decision maker's preferences are explicitly/implicitly available. This has been well echoed in the literature by various inappropriate/inadequate selection and inaccurate/misleading use of evaluation methods. In this paper, we first carry out a systematic and critical review of quality evaluation for multi-objective optimization in SBSE. We survey 717 papers published between 2009 and 2019 from 36 venues in seven repositories, and select 95 prominent studies, through which we identify five important but overlooked issues in the area. We then conduct an in-depth analysis of quality evaluation indicators/methods and general situations in SBSE, which, together with the identified issues, enables us to codify a methodological guidance for selecting and using evaluation methods in different SBSE scenarios.

cs.SE↗

arXiv2020

Muon Spin Relaxation and fluctuating magnetism in the pseudogap phase of YBa$_{2}$Cu$_{3}$O$_{y}$

We report results of a muon spin relaxation study of slow magnetic fluctuations in the pseudogap phase of underdoped single-crystalline YBa$_{2}$Cu$_{3}$O$_{y}$, $y = 6.77$ and 6.83. The dependence of the dynamic muon spin relaxation rate on applied magnetic field yields the rms magnitude~$B\mathrm{_{loc}^{rms}}$ and correlation time~$τ_c$ of fluctuating local fields at muon sites. The observed relaxation rates do not decrease with decreasing temperature~$T$ below the pseudogap onset at $T^\ast$, as would be expected for a conventional magnetic transition; both $B\mathrm{_{loc}^{rms}}$ and $τ_c$ are roughly constant in the pseudogap phase down to the superconducting transition. Corresponding NMR relaxation rates are estimated to be too small to be observable. Our results put strong constraints on theories of the anomalous pseudogap magnetism in YBa$_{2}$Cu$_{3}$O$_{y}$.

cond-mat.str-el↗

arXiv2020

Parallel Exploration via Negatively Correlated Search

Effective exploration is a key to successful search. The recently proposed Negatively Correlated Search (NCS) tries to achieve this by parallel exploration, where a set of search processes are driven to be negatively correlated so that different promising areas of the search space can be visited simultaneously. Various applications have verified the advantages of such novel search behaviors. Nevertheless, the mathematical understandings are still lacking as the previous NCS was mostly devised by intuition. In this paper, a more principled NCS is presented, explaining that the parallel exploration is equivalent to the explicit maximization of both the population diversity and the population solution qualities, and can be optimally obtained by partially gradient descending both models with respect to each search process. For empirical assessments, the reinforcement learning tasks that largely demand exploration ability is considered. The new NCS is applied to the popular reinforcement learning problems, i.e., playing Atari games, to directly train a deep convolution network with 1.7 million connection weights in the environments with uncertain and delayed rewards. Empirical results show that the significant advantages of NCS over the compared state-of-the-art methods can be highly owed to the effective parallel exploration ability.

cs.NE↗

arXiv2020

Zero-Dimensional Organic-Inorganic Hybrid Material with Ultra-Narrow-Red Emission at Room Temperature

Recently, low-dimensional organic-inorganic hybrid halide compounds have aroused great attention in the optoelectronic field, due to the unique topology and optical properties. Herein, we report an Mn4+ doped [N(CH3)4]2TiF6 zero-dimensional organic-inorganic hybrid phosphor, which could not only exhibit very narrow and pure red emission, but also maintain efficient emission intensity at room temperature. The crystal structure, photoluminescence properties and temperature sensing application are discussed. The excellent temperature dependent luminescent properties are attributed to the rigid structure and isolated MnF62- octahedra in the total crystal framework. These results will help design suitable materials and devices in both warm white light emitting diodes and optical sensors.

cond-mat.mtrl-sci↗

arXiv2020

A Hybrid Evolutionary Algorithm for Reliable Facility Location Problem

The reliable facility location problem (RFLP) is an important research topic of operational research and plays a vital role in the decision-making and management of modern supply chain and logistics. Through solving RFLP, the decision-maker can obtain reliable location decisions under the risk of facilities' disruptions or failures. In this paper, we propose a novel model for the RFLP. Instead of assuming allocating a fixed number of facilities to each customer as in the existing works, we set the number of allocated facilities as an independent variable in our proposed model, which makes our model closer to the scenarios in real life but more difficult to be solved by traditional methods. To handle it, we propose EAMLS, a hybrid evolutionary algorithm, which combines a memorable local search (MLS) method and an evolutionary algorithm (EA). Additionally, a novel metric called l3-value is proposed to assist the analysis of the algorithm's convergence speed and exam the process of evolution. The experimental results show the effectiveness and superior performance of our EAMLS, compared to a CPLEX solver and a Genetic Algorithm (GA), on large-scale problems.

cs.NE↗

arXiv2020

Continual Local Training for Better Initialization of Federated Models

Federated learning (FL) refers to the learning paradigm that trains machine learning models directly in the decentralized systems consisting of smart edge devices without transmitting the raw data, which avoids the heavy communication costs and privacy concerns. Given the typical heterogeneous data distributions in such situations, the popular FL algorithm \emph{Federated Averaging} (FedAvg) suffers from weight divergence and thus cannot achieve a competitive performance for the global model (denoted as the \emph{initial performance} in FL) compared to centralized methods. In this paper, we propose the local continual training strategy to address this problem. Importance weights are evaluated on a small proxy dataset on the central server and then used to constrain the local training. With this additional term, we alleviate the weight divergence and continually integrate the knowledge on different local clients into the global model, which ensures a better generalization ability. Experiments on various FL settings demonstrate that our method significantly improves the initial performance of federated models with few extra communication costs.

cs.LG↗

arXiv2020

Knee Point Identification Based on Trade-Off Utility

Knee points, characterised as their smallest trade-off loss at all objectives, are attractive to decision makers in multi-criterion decision-making. In contrast, other Pareto-optimal solutions are less attractive since a small improvement on one objective can lead to a significant degradation on at least one of the other objectives. In this paper, we propose a simple and effective knee point identification method based on trade-off utility, dubbed KPITU, to help decision makers identify knee points from a given set of trade-off solutions. The basic idea of KPITU is to sequentially validate whether a solution is a knee point or not by comparing its trade-off utility with others within its neighbourhood. In particular, a solution is a knee point if and only if it has the best trade-off utility among its neighbours. Moreover, we implement a GPU version of KPITU that carries out the knee point identification in a parallel manner. This GPU version reduces the worst-case complexity from quadratic to linear. To validate the effectiveness of KPITU, we compare its performance with five state-of-the-art knee point identification methods on 134 test problem instances. Empirical results fully demonstrate the outstanding performance of KPITU especially on problems with many local knee points. At the end, we further validate the usefulness of KPITU for guiding EMO algorithms to search for knee points on the fly during the evolutionary process.

math.OC↗

arXiv2020

A Novel CNet-assisted Evolutionary Level Repairer and Its Applications to Super Mario Bros

Applying latent variable evolution to game level design has become more and more popular as little human expert knowledge is required. However, defective levels with illegal patterns may be generated due to the violation of constraints for level design. A traditional way of repairing the defective levels is programming specific rule-based repairers to patch the flaw. However, programming these constraints is sometimes complex and not straightforward. An autonomous level repairer which is capable of learning the constraints is needed. In this paper, we propose a novel approach, CNet, to learn the probability distribution of tiles giving its surrounding tiles on a set of real levels, and then detect the illegal tiles in generated new levels. Then, an evolutionary repairer is designed to search for optimal replacement schemes equipped with a novel search space being constructed with the help of CNet and a novel heuristic function. The proposed approaches are proved to be effective in our case study of repairing GAN-generated and artificially destroyed levels of Super Mario Bros. game. Our CNet-assisted evolutionary repairer can also be easily applied to other games of which the levels can be represented by a matrix of objects or tiles.

cs.AI↗

arXiv2020

Synergizing Domain Expertise with Self-Awareness in Software Systems: A Patternized Architecture Guideline

To promote engineering self-aware and self-adaptive software systems in a reusable manner, architectural patterns and the related methodology provide an unified solution to handle the recurring problems in the engineering process. However, in existing patterns and methods, domain knowledge and engineers' expertise that is built over time are not explicitly linked to the self-aware processes. This linkage is important, as the knowledge is a valuable asset for the related problems and its absence would cause unnecessary overhead, possibly misleading results and unwise waste of the tremendous benefit that could have been brought by the domain expertise. This paper highlights the importance of synergizing domain expertise and the self-awareness to enable better self-adaptation in software systems, relying on well-defined expertise representation, algorithms and techniques. In particular, we present a holistic framework of notions, enriched patterns and methodology, dubbed DBASES, that offers a principled guideline for the engineers to perform difficulty and benefit analysis on possible synergies, in an attempt to keep "engineers-in-the-loop". Through three tutorial case studies, we demonstrate how DBASES can be applied in different domains, within which a carefully selected set of candidates with different synergies can be used for quantitative investigation, providing more informed decisions of the design choices.

cs.SE↗

arXiv2020

Kernel Truncated Regression Representation for Robust Subspace Clustering

Subspace clustering aims to group data points into multiple clusters of which each corresponds to one subspace. Most existing subspace clustering approaches assume that input data lie on linear subspaces. In practice, however, this assumption usually does not hold. To achieve nonlinear subspace clustering, we propose a novel method, called kernel truncated regression representation. Our method consists of the following four steps: 1) projecting the input data into a hidden space, where each data point can be linearly represented by other data points; 2) calculating the linear representation coefficients of the data representations in the hidden space; 3) truncating the trivial coefficients to achieve robustness and block-diagonality; and 4) executing the graph cutting operation on the coefficient matrix by solving a graph Laplacian problem. Our method has the advantages of a closed-form solution and the capacity of clustering data points that lie on nonlinear subspaces. The first advantage makes our method efficient in handling large-scale datasets, and the second one enables the proposed method to conquer the nonlinear subspace clustering challenge. Extensive experiments on six benchmarks demonstrate the effectiveness and the efficiency of the proposed method in comparison with current state-of-the-art approaches.

cs.CV↗

arXiv2020

Dynamic Multi-objective Optimization of the Travelling Thief Problem

Investigation of detailed and complex optimisation problem formulations that reflect realistic scenarios is a burgeoning field of research. A growing body of work exists for the Travelling Thief Problem, including multi-objective formulations and comparisons of exact and approximate methods to solve it. However, as many realistic scenarios are non-static in time, dynamic formulations have yet to be considered for the TTP. Definition of dynamics within three areas of the TTP problem are addressed; in the city locations, availability map and item values. Based on the elucidation of solution conservation between initial sets and obtained non-dominated sets, we define a range of initialisation mechanisms using solutions generated via solvers, greedily and randomly. These are then deployed to seed the population after a change and the performance in terms of hypervolume and spread is presented for comparison. Across a range of problems with varying TSP-component and KP-component sizes, we observe interesting trends in line with existing conclusions; there is little benefit to using randomisation as a strategy for initialisation of solution populations when the optimal TSP and KP component solutions can be exploited. Whilst these separate optima don't guarantee good TTP solutions, when combined, provide better initial performance and therefore in some examined instances, provides the best response to dynamic changes. A combined approach that mixes solution generation methods to provide a composite population in response to dynamic changes provides improved performance in some instances for the different dynamic TTP formulations. Potential for further development of a more cooperative combined method are realised to more cohesively exploit known information about the problems.

cs.NE↗

arXiv2020

An entanglement-based quantum network based on symmetric dispersive optics quantum key distribution

Quantum key distribution (QKD) is a crucial technology for information security in the future. Developing simple and efficient ways to establish QKD among multiple users are important to extend the applications of QKD in communication networks. Herein, we proposed a scheme of symmetric dispersive optics QKD (DO-QKD) and demonstrated an entanglement-based quantum network based on it. In the experiment, a broadband entanglement photon pair source was shared by end users via wavelength and space division multiplexing. The wide spectrum of generated entangled photon pairs was divided into 16 combinations of frequency-conjugate channels. Photon pairs in each channel combination supported a fully-connected subnet with 8 users by a passive beam splitter. Eventually, it showed that an entanglement-based QKD network over 100 users could be supported by one entangled photon pair source in this architecture. It has great potential on applications of local quantum networks with large user number.

quant-ph↗

arXiv2019

Comyco: Quality-Aware Adaptive Video Streaming via Imitation Learning

Learning-based Adaptive Bit Rate~(ABR) method, aiming to learn outstanding strategies without any presumptions, has become one of the research hotspots for adaptive streaming. However, it typically suffers from several issues, i.e., low sample efficiency and lack of awareness of the video quality information. In this paper, we propose Comyco, a video quality-aware ABR approach that enormously improves the learning-based methods by tackling the above issues. Comyco trains the policy via imitating expert trajectories given by the instant solver, which can not only avoid redundant exploration but also make better use of the collected samples. Meanwhile, Comyco attempts to pick the chunk with higher perceptual video qualities rather than video bitrates. To achieve this, we construct Comyco's neural network architecture, video datasets and QoE metrics with video quality features. Using trace-driven and real-world experiments, we demonstrate significant improvements of Comyco's sample efficiency in comparison to prior work, with 1700x improvements in terms of the number of samples required and 16x improvements on training time required. Moreover, results illustrate that Comyco outperforms previously proposed methods, with the improvements on average QoE of 7.5% - 16.79%. Especially, Comyco also surpasses state-of-the-art approach Pensieve by 7.37% on average video quality under the same rebuffering time.

cs.MM↗

arXiv2019

Competitive Coevolution as an Adversarial Approach to Dynamic Optimization

Dynamic optimization, for which the objective functions change over time, has attracted intensive investigations due to the inherent uncertainty associated with many real-world problems. For its robustness with respect to noise, Evolutionary Algorithms (EAs) have been expected to have great potential for dynamic optimization. On the other hand, EAs are also criticized for its high computational complexity, which appears to be contradictory to the core requirement of real-world dynamic optimization, i.e., fast adaptation (typically in terms of wall-clock time) to the environmental change. So far, whether EAs would indeed lead to a truly effective approach for real-world dynamic optimization remain unclear. In this paper, a new framework of employing EAs in the context of dynamic optimization is explored. We suggest that, instead of online evolving (searching) solutions for the ever-changing objective function, EAs are more suitable for acquiring an archive of solutions in an offline way, which could be adopted to construct a system to provide high-quality solutions efficiently in a dynamic environment. To be specific, we first re-formulate dynamic optimization problems as static set-oriented optimization problems. Then, a particular type of EAs, namely competitive coevolution, is employed to search for the archive of solutions in an adversarial way. The general framework is instantiated for continuous dynamic constrained optimization problems, and the empirical results showed the potential of the proposed framework.

cs.NE↗

arXiv2019

Adaptive Initialization Method for K-means Algorithm

The K-means algorithm is a widely used clustering algorithm that offers simplicity and efficiency. However, the traditional K-means algorithm uses the random method to determine the initial cluster centers, which make clustering results prone to local optima and then result in worse clustering performance. Many initialization methods have been proposed, but none of them can dynamically adapt to datasets with various characteristics. In our previous research, an initialization method for K-means based on hybrid distance was proposed, and this algorithm can adapt to datasets with different characteristics. However, it has the following drawbacks: (a) When calculating density, the threshold cannot be uniquely determined, resulting in unstable results. (b) Heavily depending on adjusting the parameter, the parameter must be adjusted five times to obtain better clustering results. (c) The time complexity of the algorithm is quadratic, which is difficult to apply to large datasets. In the current paper, we proposed an adaptive initialization method for the K-means algorithm (AIMK) to improve our previous work. AIMK can not only adapt to datasets with various characteristics but also obtain better clustering results within two interactions. In addition, we then leverage random sampling in AIMK, which is named as AIMK-RS, to reduce the time complexity. AIMK-RS is easily applied to large and high-dimensional datasets. We compared AIMK and AIMK-RS with 10 different algorithms on 16 normal and six extra-large datasets. The experimental results show that AIMK and AIMK-RS outperform the current initialization methods and several well-known clustering algorithms. Furthermore, AIMK-RS can significantly reduce the complexity of applying it to extra-large datasets with high dimensions. The time complexity of AIMK-RS is O(n).

cs.LG↗

arXiv2019

On Performance Estimation in Automatic Algorithm Configuration

Over the last decade, research on automated parameter tuning, often referred to as automatic algorithm configuration (AAC), has made significant progress. Although the usefulness of such tools has been widely recognized in real world applications, the theoretical foundations of AAC are still very weak. This paper addresses this gap by studying the performance estimation problem in AAC. More specifically, this paper first proves the universal best performance estimator in a practical setting, and then establishes theoretical bounds on the estimation error, i.e., the difference between the training performance and the true performance for a parameter configuration, considering finite and infinite configuration spaces respectively. These findings were verified in extensive experiments conducted on four algorithm configuration scenarios involving different problem domains. Moreover, insights for enhancing existing AAC methods are also identified.

cs.LG↗

arXiv2019

Voronoi-based Efficient Surrogate-assisted Evolutionary Algorithm for Very Expensive Problems

Very expensive problems are very common in practical system that one fitness evaluation costs several hours or even days. Surrogate assisted evolutionary algorithms (SAEAs) have been widely used to solve this crucial problem in the past decades. However, most studied SAEAs focus on solving problems with a budget of at least ten times of the dimension of problems which is unacceptable in many very expensive real-world problems. In this paper, we employ Voronoi diagram to boost the performance of SAEAs and propose a novel framework named Voronoi-based efficient surrogate assisted evolutionary algorithm (VESAEA) for very expensive problems, in which the optimization budget, in terms of fitness evaluations, is only 5 times of the problem's dimension. In the proposed framework, the Voronoi diagram divides the whole search space into several subspace and then the local search is operated in some potentially better subspace. Additionally, in order to trade off the exploration and exploitation, the framework involves a global search stage developed by combining leave-one-out cross-validation and radial basis function surrogate model. A performance selector is designed to switch the search dynamically and automatically between the global and local search stages. The empirical results on a variety of benchmark problems demonstrate that the proposed framework significantly outperforms several state-of-art algorithms with extremely limited fitness evaluations. Besides, the efficacy of Voronoi-diagram is furtherly analyzed, and the results show its potential to optimize very expensive problems.

cs.NE↗