Completeness (statistics)

In statistics, completeness is a property of a statistic in relation to a model for a set of observed data. In essence, it is a condition which ensures that the parameters of the probability distribution representing the model can all be estimated on the basis of the statistic: it ensures that the distributions corresponding to different values of the parameters are distinct.

It is closely related to the idea of identifiability, but in statistical theory it is often found as a condition imposed on a sufficient statistic from which certain optimality results are derived.

Definition

Consider a random variable X whose probability distribution belongs to a parametric family of probability distributions P_θ parametrized by θ.

Formally, a statistic s is a measurable function of X; thus, a statistic s is evaluated on a random variable X, taking the value s(X), which is itself a random variable. A given realization of the random variable X(ω) is a data-point (datum), on which the statistic s takes the value s(X(ω)).

The statistic s is said to be complete for the distribution of X if, for every measurable function g,^[1]

if E(g(s(X))) = 0 for all θ then P_θ(g(s(X)) = 0) = 1 for all θ.

The statistic s is said to be boundedly complete for the distribution of X if this implication holds for every measurable function g that is also bounded.

Example 1: Bernoulli model

The Bernoulli model admits a complete statistic.^[2] Let X be a random sample of size n such that each X_i has the same Bernoulli distribution with parameter p. Let T be the number of 1s observed in the sample. T is a statistic of X which has a binomial distribution with parameters (n,p). If the parameter space for p is (0,1), then T is a complete statistic. To see this, note that

\operatorname {E}(g(T))=\sum _{{t=0}}^{n}{g(t){n \choose t}p^{{t}}(1-p)^{{n-t}}}=(1-p)^{n}\sum _{{t=0}}^{n}{g(t){n \choose t}\left({\frac {p}{1-p}}\right)^{t}}.

Observe also that neither p nor 1 − p can be 0. Hence $E(g(T))=0$ if and only if:

\sum _{{t=0}}^{n}g(t){n \choose t}\left({\frac {p}{1-p}}\right)^{t}=0.

On denoting p/(1 − p) by r, one gets:

\sum _{{t=0}}^{n}g(t){n \choose t}r^{t}=0.

First, observe that the range of r is the positive reals. Also, E(g(T)) is a polynomial in r and, therefore, can only be identical to 0 if all coefficients are 0, that is, g(t) = 0 for all t.

It is important to notice that the result that all coefficients must be 0 was obtained because of the range of r. Had the parameter space been finite and with a number of elements smaller than n, it might be possible to solve the linear equations in g(t) obtained by substituting the values of r and get solutions different from 0. For example, if n = 1 and the parametric space is {0.5}, a single observation, T is not complete. Observe that, with the definition:

g(t)=2(t-0.5),\,

then, E(g(T)) = 0 although g(t) is not 0 for t = 0 nor for t = 1.

Example 2: Sum of normals

This example will show that, in a sample of size 2 from a normal distribution with known variance, the statistic X1+X2 is complete and sufficient. Suppose (X₁, X₂) are independent, identically distributed random variables, normally distributed with expectation θ and variance 1. The sum

s((X_{1},X_{2}))=X_{1}+X_{2}\,\!

is a complete statistic for θ.

To show this, it is sufficient to demonstrate that there is no non-zero function $g$ such that the expectation of

g(s(X_{1},X_{2}))=g(X_{1}+X_{2})\,\!

remains zero regardless of the value of θ.

That fact may be seen as follows. The probability distribution of X₁ + X₂ is normal with expectation 2θ and variance 2. Its probability density function in $x$ is therefore proportional to

\exp \left(-(x-2\theta )^{2}/4\right).

The expectation of g above would therefore be a constant times

\int _{{-\infty }}^{\infty }g(x)\exp \left(-(x-2\theta )^{2}/4\right)\,dx.

A bit of algebra reduces this to

k(\theta )\int _{{-\infty }}^{\infty }h(x)e^{{x\theta }}\,dx\,\!

where k(θ) is nowhere zero and

h(x)=g(x)e^{{-x^{2}/4}}.\,\!

As a function of θ this is a two-sided Laplace transform of h(X), and cannot be identically zero unless h(x) is zero almost everywhere.^[3] The exponential is not zero, so this can only happen if g(x) is zero almost everywhere.

Relation to sufficient statistics

For some parametric families, a complete sufficient statistic does not exist (for example, see Galili and Meilijson 2016 ^[4]). Also, a minimal sufficient statistic need not exist. (A case in which there is no minimal sufficient statistic was shown by Bahadur in 1957. ) Under mild conditions, a minimal sufficient statistic does always exist. In particular, these conditions always hold if the random variables (associated with P_θ ) are all discrete or are all continuous.

Importance of completeness

The notion of completeness has many applications in statistics, particularly in the following two theorems of mathematical statistics.

Lehmann–Scheffé theorem

Completeness occurs in the Lehmann–Scheffé theorem, which states that if a statistic that is unbiased, complete and sufficient for some parameter θ, then it is the best mean-unbiased estimator for θ. In other words, this statistic has a smaller expected loss for any convex loss function; in many practical applications with the squared loss-function, it has a smaller mean squared error among any estimators with the same expected value.

Examples exists that when the minimal sufficient statistic is not complete then several alternative statistics exists for unbiased estimation of θ, while some of them have lower variance than others.^[5]

Basu's theorem

Bounded completeness occurs in Basu's theorem,^[6] which states that a statistic which is both boundedly complete and sufficient is independent of any ancillary statistic.

Bahadur's theorem

Bounded completeness also occurs in Bahadur's theorem. If a statistic is sufficient and boundedly complete, then it is minimal sufficient.

Notes

↑ Young, G. A. and Smith, R. L. (2005). Essentials of Statistical Inference. (p. 94). Cambridge University Press.
↑ Casella, G. and Berger, R. L. (2001). Statistical Inference. (pp. 285-286). Duxbury Press.
↑ Orloff, Jeremy. "Uniqueness of Laplace Transform" (PDF).
↑ Tal Galili & Isaac Meilijson (31 Mar 2016). "An Example of an Improvable Rao–Blackwell Improvement, Inefficient Maximum Likelihood Estimator, and Unbiased Generalized Bayes Estimator". The American Statistician. 70 (1): 108–113. doi:10.1080/00031305.2015.1100683.
↑ Tal Galili & Isaac Meilijson (31 Mar 2016). "An Example of an Improvable Rao–Blackwell Improvement, Inefficient Maximum Likelihood Estimator, and Unbiased Generalized Bayes Estimator". The American Statistician. 70 (1): 108–113. doi:10.1080/00031305.2015.1100683.
↑ Casella, G. and Berger, R. L. (2001). Statistical Inference. (pp. 287). Duxbury Press.

References

Basu, D. (1988). J. K. Ghosh, ed. Statistical information and likelihood : A collection of critical essays by Dr. D. Basu. Lecture Notes in Statistics. 45. Springer. ISBN 0-387-96751-6. MR 953081.
Bickel, Peter J.; Doksum, Kjell A. (2001). Mathematical statistics, Volume 1: Basic and selected topics (Second (updated printing 2007) of the Holden-Day 1976 ed.). Pearson Prentice–Hall. ISBN 0-13-850363-X. MR 443141.
E. L., Lehmann; Romano, Joseph P. (2005). Testing statistical hypotheses. Springer Texts in Statistics (Third ed.). New York: Springer. pp. xiv+784. ISBN 0-387-98864-5. MR 2135927.
Lehmann, E.L.; Scheffé, H. (1950). "Completeness, similar regions, and unbiased estimation. I.". Sankhyā: the Indian Journal of Statistics. 10 (4): 305–340. JSTOR 25048038. MR 39201.
Lehmann, E.L.; Scheffé, H. (1955). "Completeness, similar regions, and unbiased estimation. II". Sankhyā: the Indian Journal of Statistics. 15 (3): 219–236. JSTOR 25048243. MR 72410.

Statistics

Descriptive statistics

Continuous data

Center	Mean arithmetic geometric harmonic Median Mode

Dispersion	Variance Standard deviation Coefficient of variation Percentile Range Interquartile range

Shape	Moments Skewness Kurtosis L-moments

Count data

Index of dispersion

Summary tables

Dependence

Graphics

Data collection

Study design	Population Statistic Effect size Statistical power Sample size determination Missing data

Survey methodology	Sampling Standard error stratified cluster Opinion poll Questionnaire

Controlled experiments	Design control optimal Controlled trial Randomized Random assignment Replication Blocking Interaction Factorial experiment

Uncontrolled studies	Observational study Natural experiment Quasi-experiment

Statistical inference

Statistical theory

Frequentist inference

Point estimation	Estimating equations Maximum likelihood Method of moments M-estimator Minimum distance Unbiased estimators Mean-unbiased minimum-variance Rao–Blackwellization Lehmann–Scheffé theorem Median unbiased Plug-in

Interval estimation	Confidence interval Pivot Likelihood interval Prediction interval Tolerance interval Resampling Bootstrap Jackknife

Testing hypotheses	1- & 2-tails Power Uniformly most powerful test Permutation test Randomization test Multiple comparisons

Parametric tests	Likelihood-ratio Wald Score

Specific tests

Z (normal) Student's t-test F

Goodness of fit	Chi-squared Kolmogorov–Smirnov Anderson–Darling Normality (Shapiro–Wilk) Likelihood-ratio test Model selection Cross validation AIC BIC

Rank statistics	Sign Sample median Signed rank (Wilcoxon) Hodges–Lehmann estimator Rank sum (Mann–Whitney) Nonparametric anova 1-way (Kruskal–Wallis) 2-way (Friedman) Ordered alternative (Jonckheere–Terpstra)

Bayesian inference

Correlation	Pearson product–moment Partial correlation Confounding variable Coefficient of determination

Regression analysis	Errors and residuals Regression model validation Mixed effects models Simultaneous equations models Multivariate adaptive regression splines (MARS)

Linear regression	Simple linear regression Ordinary least squares General linear model Bayesian regression

Non-standard predictors	Nonlinear regression Nonparametric Semiparametric Isotonic Robust Heteroscedasticity Homoscedasticity

Generalized linear model	Exponential families Logistic (Bernoulli) / Binomial / Poisson regressions

Partition of variance	Analysis of variance (ANOVA, anova) Analysis of covariance Multivariate ANOVA Degrees of freedom

Categorical / Multivariate / Time-series / Survival analysis

Categorical

Multivariate

Time-series

General	Decomposition Trend Stationarity Seasonal adjustment Exponential smoothing Cointegration Structural break Granger causality

Specific tests	Dickey–Fuller Johansen Q-statistic (Ljung–Box) Durbin–Watson Breusch–Godfrey

Time domain	Autocorrelation (ACF) partial (PACF) Cross-correlation (XCF) ARMA model ARIMA model (Box–Jenkins) Autoregressive conditional heteroskedasticity (ARCH) Vector autoregression (VAR)

Frequency domain	Spectral density estimation Fourier analysis Wavelet

Survival

Survival function	Kaplan–Meier estimator (product limit) Proportional hazards models Accelerated failure time (AFT) model First hitting time

Hazard function	Nelson–Aalen estimator

Test	Log-rank test

Applications

Biostatistics	Bioinformatics Clinical trials / studies Epidemiology Medical statistics

Engineering statistics	Chemometrics Methods engineering Probabilistic design Process / quality control Reliability System identification

Social statistics	Actuarial science Census Crime statistics Demography Econometrics National accounts Official statistics Population statistics Psychometrics

Spatial statistics	Cartography Environmental statistics Geographic information system Geostatistics Kriging

Category
Portal
Commons
WikiProject

This article is issued from Wikipedia - version of the 8/23/2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.