Centering predictor variables in OLS regression

clear 

clear 

set seed 1
set obs 1000

// Generate variables
generate female = (runiform() > .48)
generate health = 1 - female * rnormal(0.5, 1) + 0.1 * rnormal(3, 1)

// Recode predictor variable
center female
recode female  (0 = -1), gen(e_female)

eststo clear
eststo: regress health female
eststo: regress health c_female
eststo: regress health e_female

esttab, cells(b(fmt(2))) rename(c_female female e_female female) ///
        mtitle("Dummy" "Centering" "Effect") nonumber ///
		coeflabel(female "Female (ref. male)" _cons "Intercept") ///
		varwidth(18) collabel("") stats(N, fmt(%9.0gc)) ///
		title(Comparing different types of centering)

Standard deviation of binary variables

clear

twoway (function y = sqrt(x * (1 - x)), range(0 1)), ///
        ytitle("Sample standard deviation") xtitle("Proportion") ///
        xlabel(0 (.25) 1, format(%6.2f)) ///
        name(figure1, replace)

-list- using a random sample of data set

preserve           // Temporarily saves data set
sample 100, count  // Randomly samples a hundred cases of data set, discards rest
list hb030 nr_resp in 1/100, nol
restore            // Restores temporarily saved data set

-merge- crib sheet

Since Stata 11, -merge- comes with a more precise syntax, distinguishing between different types of matching. The general command structure is as follows:

use master
merge type idvars using using, options 

The so-called "master" file is being matched with the "using" file based on the list of ID variable(s) "idvars". "Type" distinguishes between four types of merging:

• One-to-one:
  When using -merge 1:1-, Stata merges one observation in the "master" file to the corresponding observation in the "using" file.
  (However, cases that could be matched will nonetheless be included in the merged file.In order to prevent that, the option -assert(match)- has to be added to the command. However, if ID's are not unique, -merge 1:1- will produce an error message.)
• Many-to-one:
  When using -merge m:1-, Stata merges many observations from the "master" data set to one corresponding obervation in the "using" file. An example for this would be to have individual-level data in the "master" file and country- or household-level information in the "using" file.
• One-to-many:
  -merge 1:m- is just the reverse of many-to-one; e.g. country-level information in master file is matched to respondent information comprising the "using" file.
• Many-to-many:
  According to the Stata Data-Management Reference Manual [D], -merge m:m- "is allowed for completeness, but it is difficult to imagine an example of when it would be useful. Use of -merge m:m- is not encouraged."

 

Troubleshooting merges

Still have syntax with the old -merge- command? See the old help file here.

William Gould has the following suggestions for merges gone bad:

• Check whether the ID variable is stored properly, e.g. if it's a long number it might not be sufficient to store it as a float. Stata might start rounding the long numbers if they are too long for the storage type.
• Check the uniqueness of ID's in both files to be merged:

by id, sort: assert _N == 1

• Merge on all common variables: If you have doubts about your ID variable, add another variable that should be constant within units, for instance gender.

Replicating simple analyses with -corr2data-

A correlation table is a pivotal feature of academic papers in many disciplines such as psychology. The reason for this is that many basic statistical methods rely on correlation matrices only. Thus, the vector of means and standard deviations plus the correlation matrix and the sample size are sufficient to replicate many standard analyses such as OLS regression or principal component analysis. -corr2data- allows to do so by creating an artficial data set based on the correlation matrix (or, covariance matrix), means, standard deviations, and sample size, which can then be used for replicating or modifying the analyses.

Table 1 of Huff-Corzine et al. (1986)

In order to try out -corr2data-, we will replicate a randomly chosen research paper, namely Huff-Corzine et al. (1986). Huff-Corzine and colleagues deal with the North–South differences in homicide in the US, and, more importantly for our purposes, they provide a full correlation table and conduct OLS regression analyses.


First, we need to read in the correlation matrix as reported in Figure 1. (N.B. Stata 12 now comes with the -ssd- command which might allow for a more convenient way of reading in summary statistics.)

version 12
clear
capture which estout
if _rc ssc install estout

// Read in correlation table
#delimit ;
input str30 varnames double (a1-a9);
"1 Homicide rate"
    1.00  0.91  0.83  0.88  0.43 -0.06 -0.30  0.65  0.83;
"2 Structural poverty index"
    0.91  1.00  0.78  0.90  0.40 -0.04 -0.17  0.65  0.84;
"3 Southernness index"
    0.83  0.78  1.00  0.72  0.34  0.14 -0.44  0.71  0.93;
"4 Perc nonwhite"
    0.88  0.90  0.72  1.00  0.32  0.08 -0.14  0.68  0.76;
"5 Perc ages 20-34"
    0.43  0.40  0.34  0.32  1.00 -0.14 -0.21 -0.09  0.32;
"6 Perc rural"
   -0.06 -0.04  0.14  0.08 -0.14  1.00  0.14  0.28  0.10;
"7 Hospital beds/100K"
   -0.30 -0.17 -0.44 -0.14 -0.21  0.14  1.00 -0.20 -0.30;
"8 Gini index"
    0.65  0.65  0.71  0.68 -0.09  0.28 -0.20  1.00  0.73;
"9 Perc born in South"
    0.83  0.84  0.93  0.76  0.32  0.10 -0.30  0.73  1.00;
end;
#delimit cr

// Convert to a matrix called M
mkmat a1-a9, matrix(M) 
matrix list M

// Read in variable names
levelsof(varnames), local(names)

local ednames = ""
  // Create local "ednames" that includes all variable names,
  // use function -strtoname()- to bring variable names into 
  // proper Stata format
foreach x of local names {
  *di strtoname("`x'", 1)
  local ed_name1 = strtoname("`x'", 1)
  *di "`ed_name1 '"
  local ed_name2 = substr("`ed_name1'", 4, .)
  *di "`ed_name2 '"
  local ednames = "`ednames'" + " " + "`ed_name2'"
}

In this step, we create the artificial data set based on the correlation matrix and means and s.d.'s:

// Create data based on correlation matrix M, vectors of
// means and sd's, specify no. of cases and variable names
corr2data `ednames', n(48) clear ///
     corr(M) ///
    means(7.12 7.12 17.71 10.42 19.75 34.22 7.81 .38 7.18) ///
   sds(4.25 3.88  9.22  8.85  1.27 14.37 1.57 .02 1.44)

One shouldn't underestimate the importance of fine-looking labels in a data set:

// Create nice-looking value labels based on the variable names
local varlabels = ""

foreach var of varlist _all {
  local varname = "`var'"
  *di "`varname'"
  local varname = subinstr("`varname'", "_"   , " ", .)
  *di "`varname'"
  local varname = subinstr("`varname'", "Perc", "%", .)
  *di "`varname'"
  lab variable `var'  "`varname'"
  local varlabels = "`varlabels'" + " " + "`varname'"
}

A brief check shows that every thing seems to be alright:

corr _all

tabstat _all, statistics(mean sd)

However, replicating the analyses yields different results:

eststo clear

// Run regression

eststo: regress Homicide_rate ///
          Structural_poverty_index Southernness_index ///
    Perc_nonwhite Perc_ages_20_34 Perc_rural ///
    Hospital_beds_100K Gini_index, beta  
    
eststo: regress Homicide_rate ///
          Structural_poverty_index ///
    Perc_nonwhite Perc_ages_20_34 Perc_rural ///
    Hospital_beds_100K Gini_index Perc_born_in_South, beta
    
estadd beta: est1 est2

esttab, r2 nonumbers mlabels("South index" "% born South") ///
        cells ((b(fmt(3) star) beta(fmt(3)))) label not)

Download the do-file here.

These results differ substantially from those reported by Huff-Corzine and others (see below), however I have a feeling that I should rather be trusting my own result.

Table 2 of Huff-Corzine et al. (1986)

A robustness check offered by -corr2data- is the -seed(#)- option, which allows for generating different artificial data sets. Specifying different seeds and comparing the results can serve as a check whether the summary statistics are really sufficient for replicating an analysis. Using different seeds in our example always leads to the same result.

Reference

Huff-Corzine, Lin, Jay Corzine, and David C. Moore. 1986. "Southern Exposure. Deciphering the South's Influence on Homicide Rates." Social Forces 64(4):906-924. doi: 10.1093/sf/64.4.906

Publication-style correlation tables in Stata

Quite a number of Stata users have engaged in programming commands to create readymade correlation tables to use in publications. Stata's own -correlate- and -pwcorr- lack many desirable features; for instance, means and standard deviations cannot be included in the table automatically and columns cannot be numbered. Furthermore, variable labels cannot be displayed, only variable names can be shown in the table.

sysuse auto
correlate price-foreign, means

-estpost-, -mkcorr-, -corrtab-, and -makematrix- are all user-written commands which aim to improve the Stata default table.

 

-estpost-

-estpost- stems from Ben Jann's mighty -estout- package. The general use of the command is as follows:

sysuse auto
capture which estout
if _rc ssc install estout
estpost correlate price-trunk, matrix
esttab, unstack not nonum compress noobs

In order to display both the upper and the lower triangle of the matrix, the -nohalf- option of -estpost- needs to be used:

estpost correlate price-trunk, matrix nohalf

However, getting -esttab- to use variable labels or to include descriptive statistics doesn't seem to be a trivial task, although the latter can be done. Finally, it should be noted that the default of -estpost correlate- is pairwise deletion, not listwise deletion; so that it is -pwcorr- rather than -correlate-. In order to obtain a correlation matrix based on listwise deletion, the -listwise- option needs to be specified.

 

-mkcorr-

-mkcorr- by Glenn Hoetker comes with many nice features and only few drawbacks.

sysuse auto, clear
capture which mkcorr
if _rc ssc install mkcorr
mkcorr price-trunk, log(corr_table)

It allows including summary statistics (mean, SD, minimum, and maximum) into the table:

mkcorr price-trunk, log(corr_table) replace means

The use of variable labels:

mkcorr price-trunk, log(corr_table) replace lab

The use of numbers in the column headers:

mkcorr price-trunk, log(corr_table) replace num

The inclusion of p-values:

mkcorr price-trunk, log(corr_table) replace means sig 

Furthermore, the number of decimal places for correlation can be manipulated via -cdec(#)- (-mdec(#)- for summary statistics).
The main drawback is that the table is not being displayed in the results window; instead it can only be written into a tab-separated text-only file that then needs additional formatting in Word or Excel. Also, the p-values are rather ugly; an option to include stars to denote statistical significance would be a great improvement. Again, the default setting is to provide correlations based on pairwise deletion; the option -casewise- will yield results based on listwise deletion.

-corrtab-

-corrtab- by Fred Wolfe is a tool that has a somewhat more limited functionality, or, more functions that I don't find to be of great use.

sysuse auto, clear
capture which corrtab
if _rc ssc install corrtab
corrtab price-trunk 

-corrtab- also allows surpressing correlations based on their p-values when using the option -print(#)-:

corrtab price-trunk, print (.10)

Or, based on the absolute value of the correlation coefficient when specifying -above(#)-:

corrtab price-trunk, above(.4)

Correlations can also be surpressed based on the position in the list of variables. The option -var(#)- only lists the first # variables in the columns, later variables are only included in the rows of the table:

corrtab price-trunk, var(3)

Correlations can be sorted by size for a single variable specified in -vsort()-:

corrtab price-trunk, vsort(price)

In order to use variable labels, some complicated additional commands need to be used; and the default for treating missing values is also pairwise deletion.
In sum, there seems to be little reason to ever use -corrtab-.

 

-makematrix-

-makematrix- by Nick Cox is a very flexible tool for all sort of applications, one of them being correlation tables. He explains it a bit in Cox (2003), but it seems to require too much tweaking to get readymade publication-style correlation tables in a short amount of time. Furthermore, there's a bug in the command: The option -listwise- will yield a pairwise correlation matrix instead of a correlation based on listwise deletion.

In sum, the commands reviewed here all leave a lot to be desired; -mkcorr- is the command that seems to be the most useful one to me.

References

Cox, Nicholas J. 2003. "Speaking Stata. Problems with Tables, Part II." Stata Journal 3(4):420-439.

Execution time of Stata commands

When conducting more complex analyses, keeping track of how long it took to run a certain command can be a crucial information. Of course, one can just keep their computer running all night for a moderatley complex analysis; however, sometimes it can make a difference whether a particular analysis took three or ten hours. Stata has a build-in command -rmsg- that allows for doing so:

set rmsg on

The more complex command -timer- gives further options.

Improved frequency tables with -fre-

A command that allows to display frequency tables in SPSS style (labels and values at the same time, percentages including and excluding missing values) is the command -fre- written by Ben Jann. It has many neat additional functions, such as the possibility to sort the output (e.g. -descending-) and to store it in various formats (e.g. tab-delimited via -using filename.tab-). Michael Norman Mitchell has some rave comments about it.

ssc install fre