Morgan and Winship
(2007: p. 66) illustrate Pearl's (2009) concern about conditioning on a collider variable using a simple example.
The general problem of conditioning on a collider is as follows. Consider three variables A, B, and C, with both A and B being causes of C: A → C ← B. (Formally, any variable C that has two arrows pointing to it along a given path is a collider.) Unlike a confounder (an uncontrolled common cause of A and B), a collider does not induce a zero-order correlation between A and B. However, when handled inappropriately, a collider can induce a conditional correlation between A and B.
Morgan and Winship's example shows just that: A college admits applicants based on their SAT scores and ratings of their motivation based on an interview. Those in the top 15 per cent of the sum of SAT and motivation ratings are being admitted. SAT scores and motivation ratings are largely uncorrelated.
// Generate and label two variables
drawnorm sat motivation, ///
n(250) ///
means(.007, -.053) ///
sds(1.01, 1.02) ///
corr(1, .035, 1) cstorage(lower) ///
clear seed(1)
label var sat "SAT"
label var motivation "Motivation"
// Only the 15 per cent at the top are admitted
gen admission_sc = sat + motivation
_pctile admission_sc, percentiles(85)
gen admission = (admission_sc > r(r1))
label var admission "Admission status"
label define admission 1 "Admitted applicant" ///
0 "Rejected applicant"
label val admission admission
drop admission_sc
// Plot as in Morgan and Winship, p. 67:
twoway (scatter motivation sat if admission == 1) ///
(scatter motivation sat if admission == 0) ///
, ///
legend(label(1 "Admitted applicants") ///
label(2 "Rejected applicants") ///
pos(5) ring(0)) ///
ylabel(none) xlabel(none) ///
name(collider1, replace)
// Enhanced plot with fitted lines and correlations
quietly cor motivation sat
local r_overall = round(r(rho), .01)
quietly cor motivation sat if admission == 1
local r_admitted = round(r(rho), .01)
quietly cor motivation sat if admission == 0
local r_rejected = round(r(rho), .01)
twoway (scatter motivation sat if admission == 1) ///
(scatter motivation sat if admission == 0) ///
(lfit motivation sat) ///
(lfit motivation sat if admission == 1) ///
(lfit motivation sat if admission == 0) ///
, ///
legend(label(1 "Admitted applicants") ///
label(2 "Rejected applicants") ///
label(3 "Overall fit, {it:r} = `r_overall'") ///
label(4 "Fit for admitted, {it:r} = `r_admitted'") ///
label(5 "Fit for rejected, {it:r} = `r_rejected'") ///
pos(5) ring(0)) ///
ylabel(none) xlabel(none) ytitle("Motivation")
What the Figures show is that the very small correlation between motivation and SAT score for the overall group turns out to be much larger when conditioning for admission status.
This also shows in an OLS regression:
regress motivation sat, beta
estimates store m1
regress motivation sat admission, beta
estimates store m2
estimates table m1 m2, b(%7.2f) se(%7.2f) stats(N) label
----------------------------------------------
Variable | m1 m2
-------------------------+--------------------
SAT | 0.10 -0.21
| 0.06 0.06
Admission status | 1.60
| 0.17
Constant | -0.12 -0.36
| 0.06 0.06
-------------------------+--------------------
N | 250 250
----------------------------------------------
legend: b/se
Cole et al.
(2010) present additional illustrations for this problem.
References
Cole, Stephen R., Robert W. Platt, Enrique F. Schisterman, Haitao Chu, Daniel Westreich, David Richardson, and Charles Poole. 2010. "Illustrating Bias Due to Conditioning on a Collider."
International Journal of Epidemiology 39(2):417-420. doi: 10.1093/ije/dyp334
Morgan, Stephen L., and Christopher Winship. 2007.
Counterfactuals and Causal Inference. Methods and Principles for Social Research. Cambridge University Press.
Pearl, Judea. 2009.
Causality. Models, Reasoning, and Inference, 2nd ed. Cambridge University Press.
// Figure 2
set seed 1
// Create file to store simulation results
tempname foo
postfile `foo' r2_50 r2_100 r2_150 r2_200 using overfitting, replace
// Simulate analyses
forvalues i = 1/10000 {
drop _all
set obs 10000
generate y = rnormal()
foreach i of numlist 1/15 {
generate x`i' = rnormal()
}
foreach j of numlist 50 100 150 200 {
preserve
sample `j', count
qui reg y x*
local r2_`j' = e(r2)
restore
}
post `foo' (`r2_50') (`r2_100') (`r2_150') (`r2_200')
}
postclose `foo'
// Open results from simulations
use overfitting, clear
// Plot
twoway (kdensity r2_200) ///
(kdensity r2_150) ///
(kdensity r2_100) ///
(kdensity r2_50) ///
, ytitle("Percent of samples") ///
xtitle("R-square value from regression model") ///
xlabel(0 (.1) .6) ///
ylabel(0 (2) 20) ///
legend(order(1 "ca. 13 cases/predictor ({it:N} = 200)" ///
2 "10 cases/predictor ({it:N} = 150)" ///
3 "ca. 7 cases/predictor ({it:N} = 100)" ///
4 "ca. 3 cases/predictor ({it:N} = 50)") ///
pos(2) ring(0)) name(figure2, replace)
// Figure 2
set seed 1
// Create file to store simulation results
tempname foo
postfile `foo' r2_50 r2_100 r2_150 r2_200 using overfitting, replace
// Simulate analyses
forvalues i = 1/10000 {
drop _all
set obs 10000
generate y = rnormal()
foreach i of numlist 1/15 {
generate x`i' = rnormal()
}
foreach j of numlist 50 100 150 200 {
preserve
sample `j', count
qui reg y x*
local r2_`j' = e(r2)
restore
}
post `foo' (`r2_50') (`r2_100') (`r2_150') (`r2_200')
}
postclose `foo'
// Open results from simulations
use overfitting, clear
// Plot
twoway (kdensity r2_200) ///
(kdensity r2_150) ///
(kdensity r2_100) ///
(kdensity r2_50) ///
, ytitle("Percent of samples") ///
xtitle("R-square value from regression model") ///
xlabel(0 (.1) .6) ///
ylabel(0 (2) 20) ///
legend(order(1 "ca. 13 cases/predictor ({it:N} = 200)" ///
2 "10 cases/predictor ({it:N} = 150)" ///
3 "ca. 7 cases/predictor ({it:N} = 100)" ///
4 "ca. 3 cases/predictor ({it:N} = 50)") ///
pos(2) ring(0)) name(figure2, replace)
// Figure 4 (actually Table 1)
clear
set seed 1
// Create file to store simulation results
tempname foo
postfile `foo' n correlation typei using dichotomization, replace
// Simulate analyses
forvalues i = 1/10000 {
drop _all
foreach j of numlist 50 100 200 {
foreach k of numlist 0 .3 .5 .7 {
qui drawnorm x1 x2, ///
n(`j') ///
corr(1, `k', 1) cstorage(lower) ///
clear
generate e = rnormal()
generate y = .5*x1 + 0*x2 + e
qui sum x1, detail
generate x1s = (x1 > r(p50))
qui sum x2, detail
generate x2s = (x2 > r(p50))
qui reg y x1s x2s
local typei = _b[x2s]/_se[x2s]
local sig = (abs(`typei') > 1.96)
*di _b[x2s] _skip(5) _se[x2s] _skip(5) `typei' _skip(5) `sig'
post `foo' (`j') (`k') (`sig')
}
}
}
postclose `foo'
// Open results from simulations
use dichotomization, clear
collapse typei, by(n correlation)
graph hbar typei, over(n, relabel(1 "{it:N} = 50" 2 "{it:N} = 100" 3 "{it:N} = 200")) ///
over(correlation, relabel(1 "{it:Corr(x{sub:1}, x{sub:2})} = 0" ///
2 "{it:Corr(x{sub:1}, x{sub:2})} = .3" ///
3 "{it:Corr(x{sub:1}, x{sub:2})} = .5" ///
4 "{it:Corr(x{sub:1}, x{sub:2})} = .7")) ///
ytitle("Type I error rate") yscale(alt) ylabel(, format(%6.2f)) ///
name(figure4, replace)
