Outliers

In a nutshell

Outliers are extreme values outside the trend of a data set. They can be calculated in different ways.

Variable definitions

Variable Name	symbol
Upper quartile	$Q_3$
Lower quartile	$Q_1$
Constant	$k$

Equations

Description	Equation
Outliers that are beyond the maximum.	$Q_3 + k(Q_3-Q_1)$
Outliers that are below the minimum.	$Q_1 - k(Q_3-Q_1)$
The interquartile range.	$Q_3 -Q_1$

Finding outliers

Use the given equations to find outliers from a data set. The $k$ value will be provided or implied within the question.

$\boxed{Q_3 + k(Q_3-Q_1) \ \ \ and \ \ \ Q_1 - k(Q_3-Q_1)}$

Example 1

The mass of $30$ bunnies is recorded. The results in $kg$ are shown below:

$1.6 , \ 2.3, \ 2.4, \ 2.4 ,\ 2.6, \ 2.7, \ 2.9, \ 3.1, \ 3.7, \ 3.8, \ 3.8, \ 3.8, \ 3.9, \ 3.9, \ 3.9, \\ 4.1, \ 4.1, \ 4.1, \ 4.3, \ 4.6, \ 4.8, \ 5.0, \ 5.0, \ 5.0, \ 5.2, \ 5.2, \ 5.3, \ 5.4 , \ 5.5, \ 5.8$

An outlier is observed at $1.5 \times \text{interquartile range}$ above the upper quartile or $1.5 \times \text{interquartile range}$ below the lower quartile. Does the data set have any outliers?

Find the lower quartile which a quarter of the way through the data points:

$30 \times \dfrac1 4 = 7.5 \rightarrow 8^{th} \ \textit{term}$

$8^{th} \ \textit{term} = 3.1$ 

Find the upper quartile which three quarters of the way through the data points:

$30 \times \dfrac34 = 22.5 \rightarrow 23^{rd} \ \textit{term}$

$23^{rd} \ \textit{term} = 5.0$

Find the interquartile range:

$5.0 - 3.1 = 1.9$ 

Outliers are values less than:

$3.1 - (1.5 \times 1.9) = 0.25$

Outliers are values greater than:

$5.0 + (1.5 \times 1.9) = 7.85$ 

There are no outliers in the data set.

Note: When calculating the position of a given quartile always round up.

Cleaning data

Questions may use different definitions for outliers. This may involve the application of different equations like those for mean and standard deviation. Once outliers are identified anomalies can then be removed from a data set with a justifiable reason. This is known as cleaning data.

Example 2

The height of $10$ tigers are given in the data set below in metres:

$1.2, \ 1.5, \ 3.1, \ 3.2, \ 3.5, \ 3.7, \ 4.0, \ 4.1, \ 5.3, \ 8.4$ 

Two sums are also given:

$\sum (x) = 38 \quad \quad \sum(x^2) = 180.94$ 

Outliers are more than a standard deviation away from the mean. Find the outliers and use this information to clean the data set, justified with reasons.

Calculate the mean:

$\text{Mean} =\overline{x}= \dfrac{\sum x}{n} = \dfrac{38}{10} = 3.8$

Calculate the standard deviation:

$\text{Variance} = \dfrac {\sum x^2}{n} - \overline{x}^2 = \dfrac {180.94}{10} - 3.8^2 = 3.654$

$\sqrt{\text{Variance}} = \text{Standard deviation} = \sqrt{3.654} = 1.9\ (1 \ d.p.)$

Find the outliers:

smaller than $3.8 - 1.9 = 1.9$

bigger than $3.8+1.9 = 5.7$

Hence the outliers are $\underline{1.2, \ 1.5, \ 8.4}$ .

Identify the anomaly with a reason:

$1.2$ and $1.5$ may not be anomalous as it is possible these are juvenile tigers.

It is highly unlikely a tiger is $8.4 \ m$ because this is a significantly larger than $\overline{x}+\sigma$ with an almost $3\ m$ difference.

Rewrite the cleaned data set:

$\underline{1.2, \ 1.5, \ 3.1, \ 3.2, \ 3.5, \ 3.7, \ 4.0, \ 4.1, \ 5.3}$ 