In the architecture of statistical theory, consistency sits near the foundation. It addresses a fundamental question about what happens to an estimator when the volume of data available for analysis grows without bound. To ask if an estimator is consistent is to inquire whether it possesses the property of asymptotic fidelity, converging in probability to the true parameter value it seeks to estimate. This behavior transforms a procedure from a mere computational tool into a reliable mechanism for uncovering truth as the sample size approaches infinity.
The Formal Definition of Consistency
Contrast with Unbiasedness
It is crucial to distinguish consistency from unbiasedness, as these properties address different aspects of an estimator's performance. An unbiased estimator achieves exact accuracy on average, with its expected value equal to the true parameter for any finite sample size. However, an estimator can be biased in finite samples yet still be consistent. As the sample size expands, the bias—the systematic error—must diminish to zero. Therefore, while unbiasedness concerns the center of the sampling distribution, consistency concerns the concentration of that distribution around the true value as the data accumulates.
Mechanisms Driving Consistency
The convergence required for consistency is typically achieved through the compounding effects of the Law of Large Numbers and the Central Limit Theorem. The Law of Large Numbers ensures that sample averages or frequencies stabilize around their expected values as observations increase. For maximum likelihood estimators, consistency often arises from the optimization of a log-likelihood function that becomes sharply peaked around the true parameter value with more data. This peaking effect reduces the variance of the estimator, allowing it to lock onto the correct value despite initial variability.
Illustrative Examples
Concrete examples help solidify the abstract definition. The sample mean serves as a primary illustration of a consistent estimator for the population mean. According to the Law of Large Numbers, the average of a large number of independent observations will be close to the expected value, and this proximity increases as the sample size grows. Similarly, the sample variance, when calculated with the denominator $n$ rather than $n-1$, provides a consistent estimate of the population variance, demonstrating how the correction for degrees of freedom becomes negligible with large $n$.
