arXiv subjects

Ayanendranath Basu

Publications and source records attributed to Ayanendranath Basu.

At least 37 records · Page 2Linked to original sources

arXiv2021

A Scale-invariant Generalization of the Rényi Entropy, Associated Divergences and their Optimizations under Tsallis' Nonextensive Framework

Entropy and relative or cross entropy measures are two very fundamental concepts in information theory and are also widely used for statistical inference across disciplines. The related optimization problems, in particular the maximization of the entropy and the minimization of the cross entropy or relative entropy (divergence), are essential for general logical inference in our physical world. In this paper, we discuss a two parameter generalization of the popular Rényi entropy and associated optimization problems. We derive the desired entropic characteristics of the new generalized entropy measure including its positivity, expandability, extensivity and generalized (sub-)additivity. More importantly, when considered over the class of sub-probabilities, our new family turns out to be scale-invariant. We also propose the corresponding cross entropy and relative entropy measures and discuss their geometric properties including generalized Pythagorean results over $β$-convex sets. The maximization of the new entropy and the minimization of the corresponding cross or relative entropy measures are carried out explicitly under the non-extensive (`third-choice') constraints given by the Tsallis' normalized $q$ expectations which also correspond to the $β$-linear family of probability distributions. Important properties of the associated forward and reverse projection rules are discussed along with their existence and uniqueness. In this context, we have come up with, for the first time, a class of entropy measures -- a subfamily of our two-parameter generalization -- that leads to the classical (extensive) exponential family of MaxEnt distributions under the non-extensive constraints. Other members of the new entropy family, however, lead to the power-law type generalized $q$-exponential MaxEnt distributions which is in conformity with Tsallis' nonextensive theory.

math.ST↗

arXiv2021

The extended Bregman divergence and parametric estimation

Minimization of suitable statistical distances~(between the data and model densities) has proved to be a very useful technique in the field of robust inference. Apart from the class of $ϕ$-divergences of \cite{a} and \cite{b}, the Bregman divergence (\cite{c}) has been extensively used for this purpose. However, since the data density must have a linear presence in the cross product term of the Bregman divergence involving both the data and model densities, several useful divergences cannot be captured by the usual Bregman form. In this respect, we provide an extension of the ordinary Bregman divergence by considering an exponent of the density function as the argument rather than the density function itself. We demonstrate that many useful divergence families, which are not ordinarily Bregman divergences, can be accommodated within this extended description. Using this formulation, one can develop many new families of divergences which may be useful in robust inference. In particular, through an application of this extension, we propose the new class of the GSB divergence family. We explore the applicability of the minimum GSB divergence estimator in discrete parametric models. Simulation studies as well as conforming real data examples are given to demonstrate the performance of the estimator and to substantiate the theory developed.

math.ST↗

arXiv2020

Robust Inference Using the Exponential-Polynomial Divergence

Density-based minimum divergence procedures represent popular techniques in parametric statistical inference. They combine strong robustness properties with high (sometimes full) asymptotic efficiency. Among density-based minimum distance procedures, the methods based on the Bregman-divergence have the attractive property that the empirical formulation of the divergence does not require the use of any non-parametric smoothing technique such as kernel density estimation. The methods based on the density power divergence (DPD) represent the current standard in this area of research. In this paper, we will present a more generalized divergence that subsumes the DPD as a special case and produces several new options providing better compromises between robustness and efficiency.

stat.ME↗

arXiv2020

Robust Estimation under Linear Mixed Models: The Minimum Density Power Divergence Approach

Many real-life data sets can be analyzed using Linear Mixed Models (LMMs). Since these are ordinarily based on normality assumptions, under small deviations from the model the inference can be highly unstable when the associated parameters are estimated by classical methods. On the other hand, the density power divergence (DPD) family, which measures the discrepancy between two probability density functions, has been successfully used to build robust estimators with high stability associated with minimal loss in efficiency. Here, we develop the minimum DPD estimator (MDPDE) for independent but non identically distributed observations in LMMs. We prove the theoretical properties, including consistency and asymptotic normality. The influence function and sensitivity measures are studied to explore the robustness properties. As a data based choice of the MDPDE tuning parameter $α$ is very important, we propose two candidates as "optimal" choices, where optimality is in the sense of choosing the strongest downweighting that is necessary for the particular data set. We conduct a simulation study comparing the proposed MDPDE, for different values of $α$, with the S-estimators, M-estimators and the classical maximum likelihood estimator, considering different levels of contamination. Finally, we illustrate the performance of our proposal on a real-data example.

stat.ME↗

arXiv2020

Robust Hypothesis Testing and Model Selection for Parametric Proportional Hazard Regression Models

The semi-parametric Cox proportional hazards regression model has been widely used for many years in several applied sciences. However, a fully parametric proportional hazards model, if appropriately assumed, can often lead to more efficient inference. To tackle the extreme non-robustness of the traditional maximum likelihood estimator in the presence of outliers in the data under such fully parametric proportional hazard models, a robust estimation procedure has recently been proposed extending the concept of the minimum density power divergence estimator (MDPDE) under this set-up. In this paper, we consider the problem of statistical inference under the parametric proportional hazards model and develop robust Wald-type hypothesis testing and model selection procedures using the MDPDEs. We have also derived the necessary asymptotic results which are used to construct the testing procedure for general composite hypothesis and study its asymptotic powers. The claimed robustness properties are studied theoretically via appropriate influence function analyses. We have studied the finite sample level and power of the proposed MDPDE based Wald type test through extensive simulations where comparisons are also made with the existing semi-parametric methods. The important issue of the selection of appropriate robustness tuning parameter is also discussed. The practical usefulness of the proposed robust testing and model selection procedures is finally illustrated through three interesting real data examples.

stat.ME↗

arXiv2020

General Robust Bayes Pseudo-Posterior: Exponential Convergence results with Applications

Although Bayesian inference is an immensely popular paradigm among a large segment of scientists including statisticians, most applications consider objective priors and need critical investigations (Efron, 2013, Science). While it has several optimal properties, a major drawback of Bayesian inference is the lack of robustness against data contamination and model misspecification, which becomes pernicious in the use of objective priors. This paper presents the general formulation of a Bayes pseudo-posterior distribution yielding robust inference. Exponential convergence results related to the new pseudo-posterior and the corresponding Bayes estimators are established under the general parametric set-up and illustrations are provided for the independent stationary as well as non-homogeneous models. Several additional details and properties of the procedure are described, including the estimation under fixed-design regression models.

math.ST↗

arXiv2020

On minimum Bregman divergence inference

In this paper a new family of minimum divergence estimators based on the Bregman divergence is proposed. The popular density power divergence (DPD) class of estimators is a sub-class of Bregman divergences. We propose and study a new sub-class of Bregman divergences called the exponentially weighted divergence (EWD). Like the minimum DPD estimator, the minimum EWD estimator is recognised as an M-estimator. This characterisation is useful while discussing the asymptotic behaviour as well as the robustness properties of this class of estimators. Performances of the two classes are compared -- both through simulations as well as through real life examples. We develop an estimation process not only for independent and homogeneous data, but also for non-homogeneous data. General tests of parametric hypotheses based on the Bregman divergences are also considered. We establish the asymptotic null distribution of our proposed test statistic and explore its behaviour when applied to real data. The inference procedures generated by the new EWD divergence appear to be competitive or better that than the DPD based procedures.

math.ST↗

arXiv2020

Robust Wald-type test in GLM with random design based on minimum density power divergence estimators

We consider the problem of robust inference under the generalized linear model (GLM) with stochastic covariates. We derive the properties of the minimum density power divergence estimator of the parameters in GLM with random design and use this estimator to propose robust Wald-type tests for testing any general composite null hypothesis about the GLM. The asymptotic and robustness properties of the proposed tests are also examined for the GLM with random design. Application of the proposed robust inference procedures to the popular Poisson regression model for analyzing count data is discussed in detail both theoretically and numerically through simulation studies and real data examples.

stat.ME↗

arXiv2019

On Robust Pseudo-Bayes Estimation for the Independent Non-homogeneous Set-up

The ordinary Bayes estimator based on the posterior density suffers from the potential problems of non-robustness under data contamination or outliers. In this paper, we consider the general set-up of independent but non-homogeneous (INH) observations and study a robustified pseudo-posterior based estimation for such parametric INH models. In particular, we focus on the $R^{(α)}$-posterior developed by Ghosh and Basu (2016) for IID data and later extended by Ghosh and Basu (2017) for INH set-up, where its usefulness and desirable properties have been numerically illustrated. In this paper, we investigate the detailed theoretical properties of this robust pseudo Bayes $R^{(α)}$-posterior and associated $R^{(α)}$-Bayes estimate under the general INH set-up with applications to fixed-design regressions. We derive a Bernstein von-Mises types asymptotic normality results and Laplace type asymptotic expansion of the $R^{(α)}$-posterior as well as the asymptotic distributions of the expected $R^{(α)}$-posterior estimators. The required conditions and the asymptotic results are simplified for linear regressions with known or unknown error variance and logistic regression models with fixed covariates. The robustness of the $R^{(α)}$-posterior and associated estimators are theoretically examined through appropriate influence function analyses under general INH set-up; illustrations are provided for the case of linear regression. A high breakdown point result is derived for the expected $R^{(α)}$-posterior estimators of the location parameter under a location-scale type model. Some interesting real life data examples illustrate possible applications.

math.ST↗

arXiv2019

Density Power Downweighting and Robust Inference: Some New Strategies

Preserving the robustness of the procedure has, at the present time, become almost a default requirement for statistical data analysis. Since efficiency at the model and robustness under misspecification of the model are often in conflict, it is important to choose such inference procedures which provide the best compromise between these two concepts. Some minimum Bregman divergence estimators and related tests of hypothesis seem to be able to do well in this respect, with the procedures based on the density power divergence providing the existing standard. In this paper we propose a new family of Bregman divergences which is a superfamily encompassing the density power divergence. This paper describes the inference procedures resulting from this new family of divergences, and makes a strong case for the utility of this divergence family in statistical inference.

math.ST↗

arXiv2019

Robust Inference for Skewed data in Health Sciences

Health data are often not symmetric to be adequately modeled through the usual normal distributions; most of them exhibit skewed patterns. They can indeed be modeled better through the larger family of skew-normal distributions covering both skewed and symmetric cases. However, the existing likelihood based inference, that is routinely performed in these cases, is extremely non-robust against data contamination/outliers. Since outliers are not uncommon in complex real-life experimental datasets, a robust methodology automatically taking care of the noises in the data would be of great practical value to produce stable and more precise research insights leading to better policy formulation. In this paper, we develop a class of robust estimators and testing procedures for the family of skew-normal distributions using the minimum density power divergence approach with application to health data. In particular, a robust procedure for testing of symmetry is discussed in the presence of outliers. Two efficient computational algorithms are discussed. Besides deriving the asymptotic and robustness theory for the proposed methods, their advantages and utilities are illustrated through simulations and a couple of real-life applications for health data of athletes from Australian Institute of Sports and AIDS clinical trial data.

stat.AP↗

arXiv2019

Statistical Inference Based on a New Weighted Likelihood Approach

We discuss a new weighted likelihood method for parametric estimation. The method is motivated by the need for generating a simple estimation strategy which provides a robust solution that is simultaneously fully efficient when the model is correctly specified. This is achieved by appropriately weighting the score function at each observation in the maximum likelihood score equation. The weight function determines the compatibility of each observation with the model in relation to the remaining observations and applies a downweighting only if it is necessary, rather than automatically downweighting a proportion of the observations all the time. This allows the estimators to retain full asymptotic efficiency at the model. We establish all the theoretical properties of the proposed estimators and substantiate the theory developed through simulation and real data examples. Our approach provides an alternative to the weighted likelihood method of Markatou et al. (1997, 1998).

stat.ME↗

arXiv2019

A Robust Generalization of the Rao Test

This paper presents new families of Rao-type test statistics based on the minimum density power divergence estimators which provide robust generalizations for testing simple and composite null hypotheses. The asymptotic null distributions of the proposed tests are obtained and their robustness properties are also theoretically studied. Numerical illustrations are provided to substantiate the theory developed. On the whole, the proposed tests are seen to be excellent alternatives to the classical Rao test.

stat.ME↗

arXiv2019

Robust Wald-Type Tests under Random Censoring with Applications to Clinical Trial Analyses

Randomly censored survival data are frequently encountered in applied sciences including biomedical or reliability applications and clinical trial analyses. Testing the significance of statistical hypotheses is crucial in such analyses to get conclusive inference but the existing likelihood based tests, under a fully parametric model, are extremely non-robust against outliers in the data. Although, there exists a few robust parameter estimators (e.g., M-estimators and minimum density power divergence estimators) given randomly censored data, there is hardly any robust testing procedure available in the literature in this context. One of the major difficulties in this context is the construction of a suitable consistent estimator of the asymptotic variance of M estimators; the latter is a function of the unknown censoring distribution. In this paper, we take the first step in this direction by proposing a consistent estimator of asymptotic variance of the M-estimators based on randomly censored data without any assumption on the form of the censoring scheme. We then describe and study a class of robust Wald-type tests for parametric statistical hypothesis, both simple as well as composite, under such set-up, along with their general asymptotic and robustness properties. Robust tests for comparing two independent randomly censored samples and robust tests against one sided alternatives are also discussed. Their advantages and usefulness are demonstrated for the tests based on the minimum density power divergence estimators with specific attention to clinical trial analyses.

stat.ME↗

arXiv2018

The B-Exponential Divergence and its Generalizations with Applications to Parametric Estimation

In this paper a new family of minimum divergence estimators based on the Bregman divergence is proposed, where the defining convex function has an exponential nature. These estimators avoid the necessity of using an intermediate kernel density and many of them also have strong robustness properties. It is further demonstrated that the proposed approach can be extended to construct a class of generalized estimating equations, where the pool of the resultant estimators encompass a large variety of minimum divergence estimators and range from highly robust to fully efficient based on the choice of the tuning parameters. All of the resultant estimators are M-estimators, where the defining functions make explicit use of the form of the parametric model. The properties of these estimators are discussed in detail; the theoretical results are substantiated by simulation and real data examples. It is observed that in many cases, certain robust estimators from the above generalized class provide better compromises between robustness and efficiency compared to the existing standards.

stat.ME↗

arXiv2018

Robust and Efficient Estimation in the Parametric Cox Regression Model under Random Censoring

Cox proportional hazard regression model is a popular tool to analyze the relationship between a censored lifetime variable with other relevant factors. The semi-parametric Cox model is widely used to study different types of data arising from applied disciplines like medical science, biology, reliability studies and many more. A fully parametric version of the Cox regression model, if properly specified, can yield more efficient parameter estimates leading to better insight generation. However, the existing maximum likelihood approach of generating inference under the fully parametric Cox regression model is highly non-robust against data-contamination which restricts its practical usage. In this paper we develop a robust estimation procedure for the parametric Cox regression model based on the minimum density power divergence approach. The proposed minimum density power divergence estimator is seen to produce highly robust estimates under data contamination with only a slight loss in efficiency under pure data. Further, they are always seen to generate more precise inference than the likelihood based estimates under the semi-parametric Cox models or their existing robust versions. We also sketch the derivation of the asymptotic properties of the proposed estimator using the martingale approach and justify their robustness theoretically through the influence function analysis. The practical applicability and usefulness of the proposal are illustrated through simulations and a real data example.

stat.ME↗

arXiv2018

Power and Level Robustness of A Composite Hypothesis Testing under Independent Non-Homogeneous Data

Robust tests of general composite hypothesis under non-identically distributed observations is always a challenge. Ghosh and Basu (2018, Statistica Sinica, 28, 1133--1155) have proposed a new class of test statistics for such problems based on the density power divergence, but their robustness with respect to the size and power are not studied in detail. This note fills this gap by providing a rigorous derivation of power and level influence functions of these tests to theoretically justify their robustness. Applications to the fixed-carrier linear regression model are also provided with empirical illustrations.

math.ST↗

arXiv2018

A Robust Wald-type Test for Testing the Equality of Two Means from Log-Normal Samples

The log-normal distribution is one of the most common distributions used for modeling skewed and positive data. It frequently arises in many disciplines of science, specially in the biological and medical sciences. The statistical analysis for comparing the means of two independent log-normal distributions is an issue of significant interest. In this paper we present a robust test for this problem. The unknown parameters of the model are estimated by minimum density power divergence estimators (Basu et al 1998, Biometrika, 85(3), 549-559). The robustness as well as the asymptotic properties of the proposed test statistics are rigorously established. The performance of the test is explored through simulations and real data analysis. The test is compared with some existing methods, and it is demonstrated that the proposed test outperforms the others in the presence of outliers.

stat.ME↗