Minitab Macros for Resampling Methods

Introduction

We have created a set of downloadable macros which perform computationally-intensive inference in Minitab for a range of widely used statistical techniques, as well as some more specialist techniques in spatial statistics.

Resampling methods in statistics

Resampling methods are a class of statistical techniques for drawing inferences based on the variability present within a dataset. Resampling methods (sometimes known as computationally intensive methods) include:

  • Bootstrapping
  • Randomization tests (also known as permutation tests)
  • Monte Carlo tests and related procedures

The common concept underlying all resampling methods is that we can assess the variability by drawing a large number of samples, each having the same size as the original dataset, from the observed data (this is the process of resampling); we then compare the properties of the observed data to the properties of the resampled datasets.

When should resampling methods be used ?

Resampling methods are useful for obtaining assessments of variability - this means that they are principally used to calculate confidence intervals and p-values. Resampling methods have become increasingly popular in recent year, partly because of increasing computer power. Resampling methods are usually used instead of - or alongside - standard techniques for drawing inferences from data. Standard techniques usually rely upon statistical theory and assumptions about the distribution of the data (for example, that the data are normally distributed). Resampling methods do not make these assumptions, and so should be more reliable in those situations in which the standard assumptions are false. If the assumptions underlying standard theory are valid, resampling and standard techniques should give very similar results.

In fact, resampling methods can give similar results to standard theory even if the assumptions underlying standard theory are not valid. Possibly the most interesting feature of resampling methods is their generality - they may be used to tackle a wide variety of practical statistical problems, including problems for which standard theory does not yet exist, in a fairly straightforward way.

Content of the macros

In creating the macros we have largely followed the range and approach of Manly (1997). This book provides a clear, non-technical introduction to the application of resampling methods to biology and ecology. For further details about resampling methodology, see Efron & Tibshirani (1993) and Davison & Hinkley (1997).The macros are designed to provide resampling versions of standard Minitab functions, when these exist, and the names of the macros reflect this.

We also include a number of macros to perform more complicated procedures in spatial statistics, for which standard Minitab functions are not available.

Limitations to resampling

Although resampling methods are widely applicable, there is no guarantee that they will give meaningful or correct results in any particular circumstance.

Users are advised that :

  • Careful examination of the data is (as always) required. In particular, it should be verified that any outlying or unusual points are correct.
  • If standard methods are available they should be applied to the data alongside resampling methods, and results of using the two methods should then be compared. In the case of serious discrepancy, further analysis should be conducted, and references should be consulted regarding the adequacy of standard and resampling methods for the case at hand.
  • If highly unusual datasets are being used, or if a non-standard statistical procedure is being attempted, the user should consult references to see whether the assumptions underlying resampling are likely to be valid.

Monte Carlo variation

  • Unlike macros for other statistical procedures in Minitab, if you repeatedly use one of the macros in this library over the same dataset then it will not (in general) give the same result each time. This variability, known as Monte Carlo variation, is an inevitable feature of resampling methods, and should not be a cause for concern.
  • In general, the number of resamples used should be kept high (the default settings are 999 resamples for hypothesis testing and 2000 resamples for confidence intervals). So long as this advice is followed, Monte Carlo variation should be small.
  • Because the procedures used are computationally intensive, some of the macros may take a long time to run (especially with large datasets).

Disclaimer

Neither CEH nor the authors of the macros take responsibility for any problems encountered whilst downloading or using the macros.

Acknowledgements

We wish to thank all authors whose datasets we have made use of.

All sources of data are fully cited.

Reference

Manly, B. F. J. (1997) Randomization, bootstrap and Monte Carlo methods in biology, 2nd edn, Chapman & Hall, London.

CEH Minitab Macro Download Request

Please complete the following form before downloading the Allocation Programme. This information is for our monitoring purposes and will only be used to keep you updated about any changes to the checklist.

Name:

Email Address: