Difference between variance, covariance and correlation

Many blogs explain these topics separately but not many explain the difference between them.

In simple words:

Variance tells us how much a quantity varies w.r.t. its mean. Its the spread of data around the mean value. You only know the magnitude here, as in how much the data is spread.

Covariance tells us direction in which two quantities vary with each other.

Correlation shows us both, the direction and magnitude of how two quantities vary with each other.

Variance is fairly simple. We now elaborate on covariance and correlation.

Covariance

Directional relationship between two variables.

When we calculate the time taken for a ball to reach the ground when thrown from a fixed height, we know that the mass does not affect the time taken. Hence mass has zero covariance to the time taken. Now, if air resistance is high, the ball takes longer to fall. More the resistance, more the time it takes. This is a positive covariance. If the ball is thrown with more force, lesser the time it takes to reach the ground. This is a negative covariance.

Correlation

Hence, Corr(no. of products,profit)>Corr(ads posted,profit).

One must note that both will be positive correlations, with ‘no. of products’ having a higher positive value.

This is used in feature selection to determine which feature affects the result the most.

A correlation of +1 indicates a perfect positive correlation.

A correlation of -1 indicates a perfect negative correlation.

A correlation of 0 indicates that there is no relationship between the different variables (mass of a ball does not affect time taken to fall).

Values between -1 and 1 denote the strength of the correlation.

Lets take a look at the formulae:

Variance

Var(X) = E(X.X) — E(X).E(X)

Covariance

Cov(X,Y) = E(X.Y) — E(X).E(Y)

Correlation

--

--

Budding blogger | Software Engineer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store