An alternative to computing formulaic test statistics which uses numerical simulation techniques for the purpose of hypothesis testing
(Paul Grootendorst, University of Toronto, PPG2010)
Bootstrap analysis is performed by deriving the sampling distribution's shape, location and width based on various assumptions including the central limit theorem, probability limits and other results from statistics. This technique is useful for estimating the shape of a sampling distribution because we can never actually observe the sampling distribution.
Bootstrapping allows the analyst to estimate the width of a sampling distribution of a statistic if his sample data is reflective of the underlying population data. If this condition is satisfied, the analyst can take the following steps to perform a bootstrap analysis.
1.) Create a large number of bootstrap samples by "re-sampling," which is to say sampling from the dataset with replacements.
2.) For each bootstrap sample, compute the statistic of interest.
3.) Use the histogram of these bootstrap samples to estimate the width of the sampling distribution.
There are two main reasons that an analyst may choose to "bootstrap" his data.
1.) It is a useful tool when the variance of a statistic is difficult to derive analytically. Essentially, the approach substitutes "computer horsepower" for the analyst's "brainpower."
2.) The bootstrap sampling distribution is itself an approximation that is strictly valid only as the number of observations approaches infinity; however, it appears that bootstrapping is often a better approximation in samples of moderate size than are analytic formulas.
Paul Grootendorst, University of Toronto, PPG2010