Influence Function and Bootstrap Methods of Estimating the Standard Errors of the Estimators of Mixture Exponential Distribution

This work estimated the standard error of the maximum likelihood estimator (MLE) and the robust estimators of the exponential mixture parameter ((cid:2)) using the influence function and the bootstrap approaches. Mixture exponential random samples of sizes 10, 15, 20, 25, 50, and 100 were generated using 3 mixture exponential models at 2%, 5% and 10% contamination levels. The selected estimators namely: mean, median, alpha-trimmed mean, Huber M-estimate and their standard errors ( (cid:4) (cid:5) ) were estimated using the two approaches at the indicated sample sizes and contamination levels. The results were compared using the coefficient of variation, confidence interval and the asymptotic relative efficiency of (cid:4) (cid:5) in order to find out which approach yields the more reliable, precise and efficient estimate of (cid:4) (cid:5) . The results of the analysis show that the two approaches do not equally perform at all conditions. From the results, the bootstrap method was found to be more reliable and efficient method of estimating the standard error of the arithmetic mean at all sample sizes and contamination levels. In estimating the standard error of the median, the influence function method was found to be more effective especially when the sample size is small and yet contamination is high. The influence function based approach yielded more reliable, precise and efficient estimates of the standard errors of the alpha-trimmed mean and the Huber M-estimate for all sample sizes and levels of contamination although the reliability of the bootstrap method improved better as sample size increased to 50 and above. All simulations and analysis were carried out in R programming language.


Introduction
The exponential model has many practical applications especially in life testing. Blischke and Prabhakar Murthy [1]; and Murthy et al. [2] in their different works opined that the exponential distribution is the most commonly employed model in reliability and life testing analysis. Epstein [3] stated that the exponential distribution plays a role in life testing analogous to that of the normal distribution in other areas of statistics. A study on the estimation of exponential distribution parameter is therefore very important.
The maximum likelihood estimator (MLE) of the exponential distribution parameter ( ) is the arithmetic mean and the measure of its long run accuracy is the squared error which is also based on the mean. If the perfect exponential fit assumptions of the classical statistics are fully met the MLE is expected to be optimal in estimating the population parameter in terms of attaining the lowest possible asymptotic variance among a reasonable class of estimators. In the presence of outliers or other forms of contaminations however, the classical estimate can be very suboptimal (Ripley [4]). Writing on the effect of mild deviations from a parametric model on classical estimates, Hampel et al. [5] noted that the effect of contaminations on the squared error is even worse than that on the arithmetic mean. 115 exponential distribution parameter using the influence function and the bootstrap approaches and compared the estimates on the basis of their coefficient of variation (CV), asymptotic relative efficiency (ARE) and confidence interval (CI) at various sample sizes and levels of contamination.

Mixture exponential models for contamination
One may view mixture models as a weighted average of some probability models. They arise in real life situations and are used to represent chance contamination.
If we presume that is from model F with a probability 1 − and from a contaminating distribution G with a small probability we can write: We may simply take the contaminating distribution to be the point mass distribution at y so that = and ( ) = 1 if > and 0 otherwise, and as such, the equation (1) can be written as ~ (1 − ) + as a simple model for proportion of outliers or contaminants at y and , = (1 − ) + ( ) is the mixture model.
Sometimes, contamination could be as a result of a proportion of observation coming from the same distribution family say exponential but with a larger scale. When this happens in the case of exponential distribution, it may be referred to as exponential contamination or exponential mixture. Staudte and Sheather [6] noted that if $ is not very large, this kind of contamination is difficult to detect. They further asserted that goodness of fit test has power less than 0.15 at 5% alpha to distinguish this form of mixture distribution from exponential distribution even though such mixture models still undermines the optimality of the MLE estimator of and its standard error.

116
There are two approaches mostly employed in estimating the standard error of the estimators of the parameter of mixture exponential distribution are the influence function and the bootstrap based methods.

Variance of the estimators of the parameters of the mixture exponential distribution via the influence function (IF)
The Influence Function was introduced by Hampel ( [7], [8]). It is the relationship between the influence a particular data point has on the value of an estimator and the distance of the data point from the estimate. Huber [9] traced the contributions of robust statistics to classical statistics and noted that among the basic robustness concepts influence function has become a standard tool. Hampel et al. [5] carried out a comprehensive treatment of influence function.
There are two approaches to calculating the influence function.
(1) By estimating a sample parameter ( ) from a sample of ) − 1 points and estimating the same parameter ( ) from a sample of n points obtained by one additional point x (a new observation) or (2) by estimating ( ) from a sample of n points and estimating the same parameter ( ) still from a sample of n points but by replacing one of the points say with another point say * .
The difference between the two estimates using any of the procedures is the influence of the new point. The approximation of this difference as a function of the normalized distance of the new point x from ( ) yields the influence function (Wainer [10]).  P(4,7) (Hampel et al. [5], Staudte and Sheather [6], Huber and Ronchetti [11]).

The arithmetic mean
The maximum likelihood estimator of the exponential distribution parameter is the arithmetic mean R and it is standard error is S ̅ (Hampel et al. [5], Staudte and Sheather ; if has mean Y = ( ); , has mean (1 − )Y + , and Ronchetti [11]).

The trimmed mean (Z [ \, )
Let … be a set of observations such that ≤ B … ≤ is the observation in ascending order and let Therefore to estimate the standard error of the trimmed mean using the influence function approach: 1. Compute the winsorized observations s .

The Huber's M-Estimate
Let ≤ B … ≤ be a set of observations arranged in ascending order then the Huber's M-estimator ̅ xly is given by:

The qth quarntile estimator
Given a set of observations ≤ B … ≤ , arranged in ascending order, for any ", 0 ≤ " ≤ 1, the qth quantile " may be defined by W6 ≤ " 9 = ". If we define a = "d) + 1f where "d. f is the greatest integer less than or equal to "() + 1), " may simply be estimated as € " = (x) or in our usual notation as (") = (x) that is the mth observation. If X is continuous and 6 " 9 > 0, the influence function (IF) of the qth quantile is given as: So that (") = € " = " + ∑ : " ( ) with a remainder term that tends to zero as n tends to infinity and IF based estimated variance is given as A6 € " 9 = "( ") ? §6 ¨9 @ n .
For the exponential distribution the programme took cognizance of the lower limit of X which is 0.

Bootstrap estimation of the standard error of « \
The bootstrap method for estimating a standard error is the second approach we employed for estimating the standard error of the estimates. Efron defined a re-sampling procedure that he coined as bootstrap (Efron [14]).
The idea of bootstrap is to use only what one knows from the data and not introduce extraneous assumptions about the population distribution.

Simulation study
Based on the models, indicated sample sizes and the levels of contamination, we 120 generated data through simulation, estimated the standard error of the estimators of theta, estimated the standard error of the standard errors SET n and carried out our analysis.
The contaminated data were generated considering the case of contamination due to increased scale for exponential mixture models. Given the general mixture model employed in this research work, we choose $ = constant = 3, 6, and 9. Specifically: We worked at 2%, 5%, 10% contamination levels and for sample sizes of, 10, 15, 20, 25, 50, and 100 to make for small, medium and large sample sizes. The multiplicative constants were chosen to make the estimators unbiased for . Without loss of generality, we assumed = 1 since our estimators are scale equivariant. The "MASS", "mixtools", and the "WRS2" packages for R programming language were used for the simulation and analysis.

Method of Comparison
Generally, in evaluating the standard error estimates, what is important to us is for the standard error to be small relative to the estimate itself (Staudte and Sheather [6]). We will therefore compare the methods based on their ability to render the least standard error of the estimate.

Coefficient of variation (CV)
The relative standard error or the coefficient of variation of is given by: where k( ) is estimated by itself.
The coefficient of variation of the standard error of the estimator itself is a guide to its effectiveness. Staudte and Sheather [6] recommended the use of CV in evaluating the 121 standard error of estimates. The estimator has a variability which we want to be small relative to what it is estimating. Therefore the approach that returns larger CV is considered less reliable and the one that delivers mostly unreliable estimates will be deemed to be performing poorer than the other.

Confidence interval (CI)
The ^% confidence interval using the bootstrap method is given as: where * is an estimate of based on the bootstrap method, ℓ = ® 2 ¾ ; ¿ = ® − ℓ; ^ is the confidence level. The procedure that delivers a shorter interval is preferred to the other (Wilcox [12]).

Result and Discussion
The columns of the tables indicate the approach used for the particular estimation (influence function or bootstrap approach), the sample size used, the model employed, as well as the value of so that: : Å x , AE means influence function based estimate when ) =` for model 1 at = Ç% and ® Å x , AE = bootstrap based estimate at ) =`, for model 1 and at = Ç%.
An abridged version of the results of the analysis for the coefficient of variation and the confidence interval are included in the body of the work while the unabridged results are provide as appendix. This is to avoid producing an unnecessarily long report.

Coefficient of variation (CV) of the standard error estimate « \
When CV is used as a tool for analysis, interest is typically to find out which procedure returns the smaller CV.  Table 1 above, the highest coefficient of variation values returned by the influence function approach for model 1, model 2 and model 3 are for the standard error of the arithmetic mean and mostly at the lower levels of contamination and sample sizes.

123
The highest CV values returned by the bootstrap approach are for the standard error of the Huber's M-estimate at lower sample sizes. The bootstrap approach was however more reliable than the influence function for the standard error of the arithmetic mean at all sample sizes and levels of contamination.
The influence function method was more reliable for the standard error of the median, the 10% and the 20% trimmed mean as well as the Huber's M-estimate even though the bootstrap approach gained more reliability with Huber's M-estimate with increased sample size. This implies that for a mixture exponential distribution the bootstrap approach is more effective than the influence function approach essentially for the standard error of the arithmetic mean under the conditions covered in this study. Our interest is to find out which approach returns a shorter CI for . From Table 2, the bootstrap approach returned shorter confidence intervals for the mean for all three levels of contamination and all sample sizes. At the lower contamination levels and sample sizes, the bootstrap method returned shorter interval for the standard error of the median than the influence function approach, as the contamination level increased however this advantage in precision was lost. The influence function approach was more precise for the standard error of the 10%, and 20% trimmed mean as well as the Huber's M-estimate for a mixture exponential distribution.

The asymptotic relative efficiency (ARE) of the standard error of « \
The efficiency of the bootstrap based estimates relative to the influence function approach is presented on the table below.   Table 3, we noticed that the bootstrap method is more efficient than the influence function approach for the estimation of the standard error of the mean, the influence function approach is more efficient for the estimation of the standard error of the median, the 10%, & 20% trimmed mean and the Huber's m-estimator for a mixture exponential distribution.

Summary of finding
The following were the findings of the study: The bootstrap approach yielded a more reliable, more precise and more efficient result for the estimation of the standard error of the arithmetic mean than the influence function approach did. This could possibly be explained by the claims of Chernick and LaBudde ( [15]) that the arithmetic mean is the theoretical mean of the bootstrap distribution.
The coefficient of variation for the standard error of the arithmetic mean was high even more than 100% in some cases at lower sample sizes and higher contamination levels when the influence function approach was used which indicates a poor 127 performance. This however improved with increase in the sample size especially for the IF method.
The influence function approach was more reliable for the standard error of the median especially at the higher levels of contamination.
For the 10% and 20% trimmed mean, the influence function approach also yielded more reliable, more precise and more efficient estimates than the bootstrap approach however, the results of the 20% trimmed mean are preferred.
At lower sample sizes and particularly with higher contamination levels, the influence function approach performed better than the bootstrap approach for the standard error of the Huber's m-estimate particularly, in terms of effectiveness and reliability. This however greatly improved for the bootstrap method as the sample size increased to 50 and above.

Conclusion
In conclusion, the two procedures for estimating the standard error considered in this work do not equally perform under all conditions. For mixture exponential distribution and under the stipulated conditions, the influence function approach is generally preferred in estimating the standard error having been found more efficient, more reliable and precise in estimation in most cases especially at higher levels of contamination except for the arithmetic mean.
We therefore recommend that the bootstrap method should be the first choice preferred to the influence function approach in estimating the standard error of the arithmetic mean for mixture exponential distribution. For the standard error of the median, the alpha trimmed mean and the Huber M-estimator, the influence function approach should be the first choice. The standard error of 20% trimming performed better than 10% trimming.