Correlation and regression
Correlation measures the strength and direction of a linear link, from −1 to +1. Regression fits a line to predict the dependent variable: Y = a + bX.
Kinds and measures of correlation
| Item | Meaning |
|---|---|
| Positive or negative | Variables move together, or in opposite directions (price and quantity demanded: negative) |
| Linear or non-linear | A constant ratio of change, or a changing one (curvilinear) |
| Simple, partial or multiple | Two variables; two with others held constant; one with two or more taken together |
| Scatter diagram | A plot of the points; the quickest visual check |
| Karl Pearson's coefficient | Product-moment r for interval or ratio data |
| Spearman's rank coefficient | For ranks or qualities such as honesty or taste |
| Concurrent deviation method | Uses only the direction of change; quick and rough |
| Probable error | PE = 0.6745 × (1 − r²) ÷ √n; r is significant if it exceeds 6 PE |
Correlation vs regression
Sir Francis Galton coined 'regression' after finding that children's heights regressed towards the average of the population.
Properties NET tests
- r is independent of change of origin and scale; only its sign flips if one variable is multiplied by a negative number.
- Regression coefficients are independent of change of origin but not of scale.
- r is the geometric mean of bYX and bXY. When both are positive, their arithmetic mean is at least r.
- bYX × bXY = r² ≤ 1, so the two coefficients cannot both exceed 1 in absolute value.
- bYX = r × σY ÷ σX and bXY = r × σX ÷ σY.
- The two regression lines cross at (x̄, ȳ). If r = 0 they are perpendicular; if r = ±1 they coincide.
- r = 0 means no linear relation; a curved relation may still exist. A high r between unrelated series is spurious (nonsense) correlation.
- Tied ranks: give each tied item the average of the ranks it shares, then add (m³ − m) ÷ 12 to Σd² for each group of m ties before using Spearman's formula.
Worked example 1: Pearson's r and the regression line
Advertising spend X (₹ lakh) and sales Y (₹ crore) for five months: X = 1, 2, 3, 4, 5; Y = 2, 4, 5, 4, 5. Find r and the regression of Y on X, and estimate Y when X = 6.
- x̄ = 15 ÷ 5 = 3; ȳ = 20 ÷ 5 = 4.
- Deviations: x = −2, −1, 0, 1, 2; y = −2, 0, 1, 0, 1.
- Σxy = 4 + 0 + 0 + 0 + 2 = 6; Σx² = 10; Σy² = 4 + 0 + 1 + 0 + 1 = 6.
- r = 6 ÷ √(10 × 6) = 6 ÷ 7.746 = 0.7746 (4 decimal places), so r² = 0.6.
- bYX = Σxy ÷ Σx² = 6 ÷ 10 = 0.6; a = ȳ − b x̄ = 4 − 0.6 × 3 = 2.2. So Y = 2.2 + 0.6X.
- At X = 6: Y = 2.2 + 0.6 × 6 = 2.2 + 3.6 = 5.8.
r ≈ 0.77, a strong positive link; advertising explains 60% of the variation in sales. Y = 2.2 + 0.6X gives ₹5.8 crore at X = 6.
Worked example 2: Spearman's rank correlation
Two judges rank five TV ads for an FMCG brand. Judge 1: 1, 2, 3, 4, 5. Judge 2: 2, 1, 4, 3, 5. Find the rank correlation.
- d = R1 − R2 = −1, 1, −1, 1, 0.
- Σd² = 1 + 1 + 1 + 1 + 0 = 4; n = 5.
- R = 1 − (6 × 4) ÷ [5 × (25 − 1)] = 1 − 24 ÷ 120 = 1 − 0.2 = 0.8.
R = 0.8: the judges largely agree.
Worked example 3: from regression coefficients to r and a line
bYX = 0.8, bXY = 0.45, x̄ = 10 and ȳ = 20. Find r and the regression equation of Y on X.
- r² = bYX × bXY = 0.8 × 0.45 = 0.36, so r = +0.6, positive because both coefficients are positive.
- Y − ȳ = bYX(X − x̄): Y − 20 = 0.8(X − 10).
- Y = 20 + 0.8X − 8 = 12 + 0.8X.
r = 0.6; Y = 12 + 0.8X.
NET asks for r from the two regression coefficients, Spearman's R from Σd², a prediction from a regression line, and properties: the limits of r, the effect of origin and scale, and where the two lines meet.
Typical MCQ: “bYX = −0.6 and bXY = −0.6. Find r.” r = −√0.36 = −0.6. Trap: giving +0.6; r takes the sign of the coefficients.
Typical MCQ: “Can bYX = 1.6 and bXY = 0.8 hold for the same data?” No. Their product, 1.28, would be r², which cannot exceed 1. Trap: rejecting 1.6 alone; one coefficient may exceed 1, but the product may not.
Typical MCQ: “r = 0 between X and Y. Are they unrelated?” Only linearly. A curved relation may still exist. Trap: reading r = 0 as 'no relation at all'.
Correlation = how closely; regression = how much. r is the geometric mean of the two b's, and both regression lines pass through the means.