Home > Python > Introduction to Pandas for Physics

Ch. 8.6 Descriptive Statistics & Uncertainty Analysis

Chapters Index
Program 1

Descriptive Statistics & Uncertainty Analysis

Python 3.11 Ready

                                
Click "Run Code" to execute script and render figures.

Mathematical Problem Formulation

Statistical Metrics in Experimental Physics:

Given $N$ independent measurements $x_1, x_2, \dots, x_N$ of a physical observable:

1. Sample Mean ($\bar{x}$):

$$\bar{x} = \frac{1}{N} \sum_{i=1}^N x_i$$

2. Sample Standard Deviation ($s_x$ with Bessel's Correction):

$$s_x = \sqrt{\frac{1}{N - 1} \sum_{i=1}^N (x_i - \bar{x})^2}$$

3. Standard Error of the Mean ($\mathrm{SEM}$):

$$\mathrm{SEM} = \frac{s_x}{\sqrt{N}}$$

4. Outlier Rejection via Tukey's Interquartile Range (IQR):

Let $Q_1$ and $Q_3$ represent the 25th and 75th percentiles. A measurement is classified as a gross experimental error (blunder) if it falls outside the Tukey fences:

$$x_{\text{outlier}} \notin [Q_1 - 1.5 \cdot \mathrm{IQR},\; Q_3 + 1.5 \cdot \mathrm{IQR}]$$

5. Error Propagation to Derived Quantities:

For a simple pendulum of length $L$ and period $T$, $g = \frac{4\pi^2 L}{T^2}$. The propagated uncertainty is:

$$\delta g = g \cdot 2 \frac{\mathrm{SEM}_T}{\bar{T}}$$

Theoretical Background & Explanation

1. Precision vs Uncertainty in Experimental Physics:

In scientific measurement, stating a result as a single number without an uncertainty interval is meaningless. Pandas provides specialized functions for statistical reduction:

  • .mean(): Best estimate of the true physical parameter.
  • .std(ddof=1): Sample standard deviation reflecting experimental spread (uses $N-1$ degrees of freedom by default).
  • .sem(): Standard error of the mean, quantifying the uncertainty in the mean estimate.

2. Automated Outlier Rejection:

Student laboratory sessions often suffer from blunders (e.g. premature stopwatch clicks or electrical glitches). Tukey's IQR rule provides a principled, objective method to prune outliers without subjective investigator bias.

3. Error Propagation:

Combining the calculated $\mathrm{SEM}$ with analytical error propagation yields the final experimental report formatted according to international metrology standards: $g = (\bar{g} \pm \delta g)\,\text{m/s}^2$.