Debian Codegraph @

inst/doc/panel-data-CRVE.html

raw
1 <!DOCTYPE html>
2
3 <html>
4
5 <head>
6
7 <meta charset="utf-8" />
8 <meta name="generator" content="pandoc" />
9 <meta http-equiv="X-UA-Compatible" content="IE=EDGE" />
10
11 <meta name="viewport" content="width=device-width, initial-scale=1" />
12
13 <meta name="author" content="James E. Pustejovsky" />
14
15 <meta name="date" content="2026-05-01" />
16
17 <title>Cluster-robust standard errors and hypothesis tests in panel data models</title>
18
19 <script>// Pandoc 2.9 adds attributes on both header and div. We remove the former (to
20 // be compatible with the behavior of Pandoc < 2.8).
21 document.addEventListener('DOMContentLoaded', function(e) {
22 var hs = document.querySelectorAll("div.section[class*='level'] > :first-child");
23 var i, h, a;
24 for (i = 0; i < hs.length; i++) {
25 h = hs[i];
26 if (!/^h[1-6]$/i.test(h.tagName)) continue; // it should be a header h1-h6
27 a = h.attributes;
28 while (a.length > 0) h.removeAttribute(a[0].name);
29 }
30 });
31 </script>
32
33 <style type="text/css">
34 code{white-space: pre-wrap;}
35 span.smallcaps{font-variant: small-caps;}
36 span.underline{text-decoration: underline;}
37 div.column{display: inline-block; vertical-align: top; width: 50%;}
38 div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
39 ul.task-list{list-style: none;}
40 </style>
41
42
43
44 <style type="text/css">
45 code {
46 white-space: pre;
47 }
48 .sourceCode {
49 overflow: visible;
50 }
51 </style>
52 <style type="text/css" data-origin="pandoc">
53 html { -webkit-text-size-adjust: 100%; }
54 pre > code.sourceCode { white-space: pre; position: relative; }
55 pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
56 pre > code.sourceCode > span:empty { height: 1.2em; }
57 .sourceCode { overflow: visible; }
58 code.sourceCode > span { color: inherit; text-decoration: inherit; }
59 div.sourceCode { margin: 1em 0; }
60 pre.sourceCode { margin: 0; }
61 @media screen {
62 div.sourceCode { overflow: auto; }
63 }
64 @media print {
65 pre > code.sourceCode { white-space: pre-wrap; }
66 pre > code.sourceCode > span { text-indent: -5em; padding-left: 5em; }
67 }
68 pre.numberSource code
69 { counter-reset: source-line 0; }
70 pre.numberSource code > span
71 { position: relative; left: -4em; counter-increment: source-line; }
72 pre.numberSource code > span > a:first-child::before
73 { content: counter(source-line);
74 position: relative; left: -1em; text-align: right; vertical-align: baseline;
75 border: none; display: inline-block;
76 -webkit-touch-callout: none; -webkit-user-select: none;
77 -khtml-user-select: none; -moz-user-select: none;
78 -ms-user-select: none; user-select: none;
79 padding: 0 4px; width: 4em;
80 color: #aaaaaa;
81 }
82 pre.numberSource { margin-left: 3em; border-left: 1px solid #aaaaaa; padding-left: 4px; }
83 div.sourceCode
84 { }
85 @media screen {
86 pre > code.sourceCode > span > a:first-child::before { text-decoration: underline; }
87 }
88 code span.al { color: #ff0000; font-weight: bold; }
89 code span.an { color: #60a0b0; font-weight: bold; font-style: italic; }
90 code span.at { color: #7d9029; }
91 code span.bn { color: #40a070; }
92 code span.bu { color: #008000; }
93 code span.cf { color: #007020; font-weight: bold; }
94 code span.ch { color: #4070a0; }
95 code span.cn { color: #880000; }
96 code span.co { color: #60a0b0; font-style: italic; }
97 code span.cv { color: #60a0b0; font-weight: bold; font-style: italic; }
98 code span.do { color: #ba2121; font-style: italic; }
99 code span.dt { color: #902000; }
100 code span.dv { color: #40a070; }
101 code span.er { color: #ff0000; font-weight: bold; }
102 code span.ex { }
103 code span.fl { color: #40a070; }
104 code span.fu { color: #06287e; }
105 code span.im { color: #008000; font-weight: bold; }
106 code span.in { color: #60a0b0; font-weight: bold; font-style: italic; }
107 code span.kw { color: #007020; font-weight: bold; }
108 code span.op { color: #666666; }
109 code span.ot { color: #007020; }
110 code span.pp { color: #bc7a00; }
111 code span.sc { color: #4070a0; }
112 code span.ss { color: #bb6688; }
113 code span.st { color: #4070a0; }
114 code span.va { color: #19177c; }
115 code span.vs { color: #4070a0; }
116 code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; }
117 </style>
118 <script>
119 // apply pandoc div.sourceCode style to pre.sourceCode instead
120 (function() {
121 var sheets = document.styleSheets;
122 for (var i = 0; i < sheets.length; i++) {
123 if (sheets[i].ownerNode.dataset["origin"] !== "pandoc") continue;
124 try { var rules = sheets[i].cssRules; } catch (e) { continue; }
125 var j = 0;
126 while (j < rules.length) {
127 var rule = rules[j];
128 // check if there is a div.sourceCode rule
129 if (rule.type !== rule.STYLE_RULE || rule.selectorText !== "div.sourceCode") {
130 j++;
131 continue;
132 }
133 var style = rule.style.cssText;
134 // check if color or background-color is set
135 if (rule.style.color === '' && rule.style.backgroundColor === '') {
136 j++;
137 continue;
138 }
139 // replace div.sourceCode by a pre.sourceCode rule
140 sheets[i].deleteRule(j);
141 sheets[i].insertRule('pre.sourceCode{' + style + '}', j);
142 }
143 }
144 })();
145 </script>
146
147
148
149
150 <style type="text/css">body {
151 background-color: #fff;
152 margin: 1em auto;
153 max-width: 700px;
154 overflow: visible;
155 padding-left: 2em;
156 padding-right: 2em;
157 font-family: "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;
158 font-size: 14px;
159 line-height: 1.35;
160 }
161 #TOC {
162 clear: both;
163 margin: 0 0 10px 10px;
164 padding: 4px;
165 width: 400px;
166 border: 1px solid #CCCCCC;
167 border-radius: 5px;
168 background-color: #f6f6f6;
169 font-size: 13px;
170 line-height: 1.3;
171 }
172 #TOC .toctitle {
173 font-weight: bold;
174 font-size: 15px;
175 margin-left: 5px;
176 }
177 #TOC ul {
178 padding-left: 40px;
179 margin-left: -1.5em;
180 margin-top: 5px;
181 margin-bottom: 5px;
182 }
183 #TOC ul ul {
184 margin-left: -2em;
185 }
186 #TOC li {
187 line-height: 16px;
188 }
189 table {
190 margin: 1em auto;
191 border-width: 1px;
192 border-color: #DDDDDD;
193 border-style: outset;
194 border-collapse: collapse;
195 }
196 table th {
197 border-width: 2px;
198 padding: 5px;
199 border-style: inset;
200 }
201 table td {
202 border-width: 1px;
203 border-style: inset;
204 line-height: 18px;
205 padding: 5px 5px;
206 }
207 table, table th, table td {
208 border-left-style: none;
209 border-right-style: none;
210 }
211 table thead, table tr.even {
212 background-color: #f7f7f7;
213 }
214 p {
215 margin: 0.5em 0;
216 }
217 blockquote {
218 background-color: #f6f6f6;
219 padding: 0.25em 0.75em;
220 }
221 hr {
222 border-style: solid;
223 border: none;
224 border-top: 1px solid #777;
225 margin: 28px 0;
226 }
227 dl {
228 margin-left: 0;
229 }
230 dl dd {
231 margin-bottom: 13px;
232 margin-left: 13px;
233 }
234 dl dt {
235 font-weight: bold;
236 }
237 ul {
238 margin-top: 0;
239 }
240 ul li {
241 list-style: circle outside;
242 }
243 ul ul {
244 margin-bottom: 0;
245 }
246 pre, code {
247 background-color: #f7f7f7;
248 border-radius: 3px;
249 color: #333;
250 white-space: pre-wrap;
251 }
252 pre {
253 border-radius: 3px;
254 margin: 5px 0px 10px 0px;
255 padding: 10px;
256 }
257 pre:not([class]) {
258 background-color: #f7f7f7;
259 }
260 code {
261 font-family: Consolas, Monaco, 'Courier New', monospace;
262 font-size: 85%;
263 }
264 p > code, li > code {
265 padding: 2px 0px;
266 }
267 div.figure {
268 text-align: center;
269 }
270 img {
271 background-color: #FFFFFF;
272 padding: 2px;
273 border: 1px solid #DDDDDD;
274 border-radius: 3px;
275 border: 1px solid #CCCCCC;
276 margin: 0 5px;
277 }
278 h1 {
279 margin-top: 0;
280 font-size: 35px;
281 line-height: 40px;
282 }
283 h2 {
284 border-bottom: 4px solid #f7f7f7;
285 padding-top: 10px;
286 padding-bottom: 2px;
287 font-size: 145%;
288 }
289 h3 {
290 border-bottom: 2px solid #f7f7f7;
291 padding-top: 10px;
292 font-size: 120%;
293 }
294 h4 {
295 border-bottom: 1px solid #f7f7f7;
296 margin-left: 8px;
297 font-size: 105%;
298 }
299 h5, h6 {
300 border-bottom: 1px solid #ccc;
301 font-size: 105%;
302 }
303 a {
304 color: #0033dd;
305 text-decoration: none;
306 }
307 a:hover {
308 color: #6666ff; }
309 a:visited {
310 color: #800080; }
311 a:visited:hover {
312 color: #BB00BB; }
313 a[href^="http:"] {
314 text-decoration: underline; }
315 a[href^="https:"] {
316 text-decoration: underline; }
317
318 code > span.kw { color: #555; font-weight: bold; }
319 code > span.dt { color: #902000; }
320 code > span.dv { color: #40a070; }
321 code > span.bn { color: #d14; }
322 code > span.fl { color: #d14; }
323 code > span.ch { color: #d14; }
324 code > span.st { color: #d14; }
325 code > span.co { color: #888888; font-style: italic; }
326 code > span.ot { color: #007020; }
327 code > span.al { color: #ff0000; font-weight: bold; }
328 code > span.fu { color: #900; font-weight: bold; }
329 code > span.er { color: #a61717; background-color: #e3d2d2; }
330 </style>
331
332
333
334
335 </head>
336
337 <body>
338
339
340
341
342 <h1 class="title toc-ignore">Cluster-robust standard errors and
343 hypothesis tests in panel data models</h1>
344 <h4 class="author">James E. Pustejovsky</h4>
345 <h4 class="date">2026-05-01</h4>
346
347
348
349 <p>The importance of using cluster-robust variance estimators (i.e.,
350 “clustered standard errors”) in panel models is now widely recognized.
351 Less widely recognized is the fact that standard methods for
352 constructing hypothesis tests and confidence intervals based on CRVE can
353 perform quite poorly in when based on a limited number of independent
354 clusters. Furthermore, it can be difficult to determine what counts as a
355 large-enough sample to trust standard CRVE methods, because the
356 finite-sample behavior of the variance estimators and test statistics
357 depends on the configuration of the covariates, not just the total
358 number of clusters.</p>
359 <p>One solution to this problem is to use bias-reduced linearization
360 (BRL), which was proposed by Bell and McCaffrey (2002) and has recently
361 begun to receive attention in the econometrics literature (e.g., Cameron
362 &amp; Miller, 2015; Imbens &amp; Kolesar, 2015). The idea of BRL is to
363 correct the bias of standard CRVE based on a working model, and then to
364 use a degrees-of-freedom correction for Wald tests based on the
365 bias-reduced CRVE. That may seem silly (after all, the whole point of
366 CRVE is to avoid making distributional assumptions about the errors in
367 your model), but it turns out that the correction can help quite a bit,
368 even when the working model is wrong. The degrees-of-freedom correction
369 is based on a standard Satterthwaite-type approximation, and also relies
370 on the working model.</p>
371 <p>A problem with Bell and McCaffrey’s original formulation of BRL is
372 that it does not work in some very common models for panel data, such as
373 state-by-year panels that include fixed effects for each state and each
374 year (Angrist and Pischke, 2009, point out this issue in their chapter
375 on “non-standard standard error issues”; see also Young, 2016). However,
376 Pustejovsky and Tipton (2016) proposed a generalization of BRL that
377 works even in models with arbitrary sets of fixed effects, and this
378 generalization is implemented in <code>clubSandwich</code> as CRVE type
379 <code>CR2</code>. The package also implements small-sample corrections
380 for multiple-constraint hypothesis tests based on an approximation
381 proposed by Pustejovsky and Tipton (2016). For one-parameter
382 constraints, the test reduces to a t-test with Satterthwaite degrees of
383 freedom, and so it is a natural extension of BRL.</p>
384 <p>The following example demonstrates how to use
385 <code>clubSandwich</code> to do cluster-robust inference for a
386 state-by-year panel model with fixed effects in both dimensions,
387 clustering by states.</p>
388 <div id="effects-of-changing-the-minimum-legal-drinking-age" class="section level2">
389 <h2>Effects of changing the minimum legal drinking age</h2>
390 <p>Carpenter and Dobkin (2011) analyzed the effects of changes in the
391 minimum legal drinking age on rates of motor vehicle fatalities among
392 18-20 year olds, using state-level panel data from the National Highway
393 Traffic Administration’s Fatal Accident Reporting System. In their new
394 textbook, Angrist and Pischke (2014) developed a stylized example based
395 on Carpenter and Dobkin’s work. The following example uses Angrist and
396 Pischke’s data and follows their analysis because their data are <a href="https://www.masteringmetrics.com/resources/">easily
397 available</a>.</p>
398 <p>The outcome is the incidence of deaths in motor vehicle crashes among
399 18-20 year-olds (per 100,000 residents), for each state plus the
400 District of Columbia, over the period 1970 to 1983. There were several
401 changes in the minimum legal drinking age during this time period, with
402 variability in the timing of changes across states. Angrist and Pischke
403 (following Carpenter and Dobkin) use a difference-in-differences
404 strategy to estimate the effects of lowering the minimum legal drinking
405 age from 21 to 18. Their specification is</p>
406 <p><span class="math display">\[y_{it} = \alpha_i + \beta_t + \gamma
407 b_{it} + \delta d_{it} + \epsilon_{it},\]</span></p>
408 <p>for <span class="math inline">\(i\)</span> = 1,…,51 and <span class="math inline">\(t\)</span> = 1970,…,1983. In this model, <span class="math inline">\(\alpha_i\)</span> is a state-specific fixed
409 effect, <span class="math inline">\(\beta_t\)</span> is a year-specific
410 fixed effect, <span class="math inline">\(b_{it}\)</span> is the current
411 rate of beer taxation in state <span class="math inline">\(i\)</span> in
412 year <span class="math inline">\(t\)</span>, <span class="math inline">\(d_{it}\)</span> is the proportion of 18-20
413 year-olds in state <span class="math inline">\(i\)</span> in year <span class="math inline">\(t\)</span> who are legally allowed to drink, and
414 <span class="math inline">\(\delta\)</span> captures the effect of
415 shifting the minimum legal drinking age from 21 to 18. Following Angrist
416 and Pischke’s analysis, we estimate this model both by (unweighted) OLS
417 and by weighted least squares with weights corresponding to population
418 size in a given state and year. We also demonstrate random effects
419 estimation and implement a cluster-robust Hausman specification
420 test.</p>
421 </div>
422 <div id="unweighted-ols" class="section level2">
423 <h2>Unweighted OLS</h2>
424 <p>The following code does some simple data-munging and the estimates
425 the model by OLS:</p>
426 <div class="sourceCode" id="cb1"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1-1"><a href="#cb1-1" tabindex="-1"></a><span class="fu">library</span>(clubSandwich)</span>
427 <span id="cb1-2"><a href="#cb1-2" tabindex="-1"></a><span class="fu">data</span>(MortalityRates)</span>
428 <span id="cb1-3"><a href="#cb1-3" tabindex="-1"></a></span>
429 <span id="cb1-4"><a href="#cb1-4" tabindex="-1"></a><span class="co"># subset for deaths in motor vehicle accidents, 1970-1983</span></span>
430 <span id="cb1-5"><a href="#cb1-5" tabindex="-1"></a>MV_deaths <span class="ot">&lt;-</span> <span class="fu">subset</span>(MortalityRates, cause<span class="sc">==</span><span class="st">&quot;Motor Vehicle&quot;</span> <span class="sc">&amp;</span> </span>
431 <span id="cb1-6"><a href="#cb1-6" tabindex="-1"></a> year <span class="sc">&lt;=</span> <span class="dv">1983</span> <span class="sc">&amp;</span> <span class="sc">!</span><span class="fu">is.na</span>(beertaxa))</span>
432 <span id="cb1-7"><a href="#cb1-7" tabindex="-1"></a></span>
433 <span id="cb1-8"><a href="#cb1-8" tabindex="-1"></a><span class="co"># fit by OLS</span></span>
434 <span id="cb1-9"><a href="#cb1-9" tabindex="-1"></a>lm_unweighted <span class="ot">&lt;-</span> <span class="fu">lm</span>(mrate <span class="sc">~</span> <span class="dv">0</span> <span class="sc">+</span> legal <span class="sc">+</span> beertaxa <span class="sc">+</span> </span>
435 <span id="cb1-10"><a href="#cb1-10" tabindex="-1"></a> <span class="fu">factor</span>(state) <span class="sc">+</span> <span class="fu">factor</span>(year), <span class="at">data =</span> MV_deaths)</span></code></pre></div>
436 <p>The <code>coef_test</code> function from <code>clubSandwich</code>
437 can then be used to test the hypothesis that changing the minimum legal
438 drinking age has no effect on motor vehicle deaths in this cohort (i.e.,
439 <span class="math inline">\(H_0: \delta = 0\)</span>). The usual way to
440 test this is to cluster the standard errors by state, calculate the
441 robust Wald statistic, and compare that to a standard normal reference
442 distribution. The code and results are as follows:</p>
443 <div class="sourceCode" id="cb2"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb2-1"><a href="#cb2-1" tabindex="-1"></a><span class="fu">coef_test</span>(lm_unweighted, <span class="at">vcov =</span> <span class="st">&quot;CR1&quot;</span>, </span>
444 <span id="cb2-2"><a href="#cb2-2" tabindex="-1"></a> <span class="at">cluster =</span> MV_deaths<span class="sc">$</span>state, <span class="at">test =</span> <span class="st">&quot;naive-t&quot;</span>)[<span class="dv">1</span><span class="sc">:</span><span class="dv">2</span>,]</span></code></pre></div>
445 <pre><code>## Alternative hypothesis: two-sided
446 ## Coef. Estimate SE Null value t-stat d.f. (naive-t) p-val (naive-t) Sig.
447 ## legal 7.59 2.44 0 3.108 49 0.00313 **
448 ## beertaxa 3.82 5.14 0 0.743 49 0.46128</code></pre>
449 <p>A better approach would be to use the generalized, bias-reduced
450 linearization CRVE, together with Satterthwaite degrees of freedom. In
451 the <code>clubSandwich</code> package, the BRL adjustment is called
452 “CR2” because it is directly analogous to the HC2 correction used in
453 heteroskedasticity-robust variance estimation. When applied to an OLS
454 model estimated by <code>lm</code>, the default working model is an
455 identity matrix, which amounts to the “working” assumption that the
456 errors are all uncorrelated and homoskedastic. Here’s how to apply this
457 approach in the example:</p>
458 <div class="sourceCode" id="cb4"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb4-1"><a href="#cb4-1" tabindex="-1"></a><span class="fu">coef_test</span>(lm_unweighted, <span class="at">vcov =</span> <span class="st">&quot;CR2&quot;</span>, </span>
459 <span id="cb4-2"><a href="#cb4-2" tabindex="-1"></a> <span class="at">cluster =</span> MV_deaths<span class="sc">$</span>state, <span class="at">test =</span> <span class="st">&quot;Satterthwaite&quot;</span>)[<span class="dv">1</span><span class="sc">:</span><span class="dv">2</span>,]</span></code></pre></div>
460 <pre><code>## Alternative hypothesis: two-sided
461 ## Coef. Estimate SE Null value t-stat d.f. (Satt) p-val (Satt) Sig.
462 ## legal 7.59 2.51 0 3.019 24.58 0.00583 **
463 ## beertaxa 3.82 5.27 0 0.725 5.77 0.49663</code></pre>
464 <p>The Satterthwaite degrees of freedom are different for each
465 coefficient in the model, and so the <code>coef_test</code> function
466 reports them right alongside the standard error. For the effect of legal
467 drinking age, the degrees of freedom are about half of what might be
468 expected, given that there are 51 clusters. The p-value for the
469 CR2+Satterthwaite test is about twice as large as the p-value based on
470 the standard Wald test, although the coefficient is still statistically
471 significant at conventional levels. Note, however, that the degrees of
472 freedom on the beer taxation rate are considerably smaller because there
473 are only a few states with substantial variability in taxation rates
474 over time.</p>
475 </div>
476 <div id="unweighted-within-estimation" class="section level2">
477 <h2>Unweighted “within” estimation</h2>
478 <p>The <code>plm</code> package in R provides another way to estimate
479 the same model. It is convenient because it absorbs the state and year
480 fixed effects before estimating the effect of <code>legal</code>. The
481 <code>clubSandwich</code> package works with fitted <code>plm</code>
482 models too:</p>
483 <div class="sourceCode" id="cb6"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb6-1"><a href="#cb6-1" tabindex="-1"></a><span class="fu">library</span>(plm)</span>
484 <span id="cb6-2"><a href="#cb6-2" tabindex="-1"></a>plm_unweighted <span class="ot">&lt;-</span> <span class="fu">plm</span>(mrate <span class="sc">~</span> legal <span class="sc">+</span> beertaxa, <span class="at">data =</span> MV_deaths, </span>
485 <span id="cb6-3"><a href="#cb6-3" tabindex="-1"></a> <span class="at">effect =</span> <span class="st">&quot;twoways&quot;</span>, <span class="at">index =</span> <span class="fu">c</span>(<span class="st">&quot;state&quot;</span>,<span class="st">&quot;year&quot;</span>))</span>
486 <span id="cb6-4"><a href="#cb6-4" tabindex="-1"></a><span class="fu">coef_test</span>(plm_unweighted, <span class="at">vcov =</span> <span class="st">&quot;CR1&quot;</span>, <span class="at">cluster =</span> <span class="st">&quot;individual&quot;</span>, <span class="at">test =</span> <span class="st">&quot;naive-t&quot;</span>)</span></code></pre></div>
487 <pre><code>## Alternative hypothesis: two-sided
488 ## Coef. Estimate SE Null value t-stat d.f. (naive-t) p-val (naive-t) Sig.
489 ## legal 7.59 2.44 0 3.108 49 0.00313 **
490 ## beertaxa 3.82 5.14 0 0.743 49 0.46128</code></pre>
491 <div class="sourceCode" id="cb8"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb8-1"><a href="#cb8-1" tabindex="-1"></a><span class="fu">coef_test</span>(plm_unweighted, <span class="at">vcov =</span> <span class="st">&quot;CR2&quot;</span>, <span class="at">cluster =</span> <span class="st">&quot;individual&quot;</span>, <span class="at">test =</span> <span class="st">&quot;Satterthwaite&quot;</span>)</span></code></pre></div>
492 <pre><code>## Alternative hypothesis: two-sided
493 ## Coef. Estimate SE Null value t-stat d.f. (Satt) p-val (Satt) Sig.
494 ## legal 7.59 2.51 0 3.019 24.58 0.00583 **
495 ## beertaxa 3.82 5.27 0 0.725 5.77 0.49663</code></pre>
496 </div>
497 <div id="population-weighted-estimation" class="section level2">
498 <h2>Population-weighted estimation</h2>
499 <p>The difference between the standard method and the new method are not
500 terribly exciting in the above example. However, things change quite a
501 bit if the model is estimated using population weights. We go back to
502 fitting in <code>lm</code> with dummies for all the fixed effects
503 because <code>plm</code> does not handle weighted least squares.</p>
504 <div class="sourceCode" id="cb10"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb10-1"><a href="#cb10-1" tabindex="-1"></a>lm_weighted <span class="ot">&lt;-</span> <span class="fu">lm</span>(mrate <span class="sc">~</span> <span class="dv">0</span> <span class="sc">+</span> legal <span class="sc">+</span> beertaxa <span class="sc">+</span> <span class="fu">factor</span>(state) <span class="sc">+</span> <span class="fu">factor</span>(year), </span>
505 <span id="cb10-2"><a href="#cb10-2" tabindex="-1"></a> <span class="at">weights =</span> pop, <span class="at">data =</span> MV_deaths)</span>
506 <span id="cb10-3"><a href="#cb10-3" tabindex="-1"></a><span class="fu">coef_test</span>(lm_weighted, <span class="at">vcov =</span> <span class="st">&quot;CR1&quot;</span>, </span>
507 <span id="cb10-4"><a href="#cb10-4" tabindex="-1"></a> <span class="at">cluster =</span> MV_deaths<span class="sc">$</span>state, <span class="at">test =</span> <span class="st">&quot;naive-t&quot;</span>)[<span class="dv">1</span><span class="sc">:</span><span class="dv">2</span>,]</span></code></pre></div>
508 <pre><code>## Alternative hypothesis: two-sided
509 ## Coef. Estimate SE Null value t-stat d.f. (naive-t) p-val (naive-t) Sig.
510 ## legal 7.78 2.01 0 3.87 49 &lt;0.001 ***
511 ## beertaxa 11.16 4.20 0 2.66 49 0.0106 *</code></pre>
512 <div class="sourceCode" id="cb12"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb12-1"><a href="#cb12-1" tabindex="-1"></a><span class="fu">coef_test</span>(lm_weighted, <span class="at">vcov =</span> <span class="st">&quot;CR2&quot;</span>, </span>
513 <span id="cb12-2"><a href="#cb12-2" tabindex="-1"></a> <span class="at">cluster =</span> MV_deaths<span class="sc">$</span>state, <span class="at">test =</span> <span class="st">&quot;Satterthwaite&quot;</span>)[<span class="dv">1</span><span class="sc">:</span><span class="dv">2</span>,]</span></code></pre></div>
514 <pre><code>## Alternative hypothesis: two-sided
515 ## Coef. Estimate SE Null value t-stat d.f. (Satt) p-val (Satt) Sig.
516 ## legal 7.78 2.13 0 3.64 8.52 0.00588 **
517 ## beertaxa 11.16 4.37 0 2.55 6.85 0.03854 *</code></pre>
518 <p>Using population weights slightly reduces the point estimate of the
519 effect, while also slightly increasing its precision. If you were
520 following the standard approach, you would probably be happy with the
521 weighted estimates and wouldn’t think about it any further. However,
522 using the CR2 variance estimator and Satterthwaite correction produces a
523 p-value that is an order of magnitude larger (though still significant
524 at the conventional 5% level). The degrees of freedom are just
525 8.5—drastically smaller than would be expected based on the number of
526 clusters.</p>
527 <p>Even with weights, the <code>coef_test</code> function uses an
528 “independent, homoskedastic” working model as a default for
529 <code>lm</code> objects. In the present example, the outcome is a
530 standardized rate and so a better assumption might be that the error
531 variances are inversely proportional to population size. The following
532 code uses this alternate working model:</p>
533 <div class="sourceCode" id="cb14"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb14-1"><a href="#cb14-1" tabindex="-1"></a><span class="fu">coef_test</span>(lm_weighted, <span class="at">vcov =</span> <span class="st">&quot;CR2&quot;</span>, </span>
534 <span id="cb14-2"><a href="#cb14-2" tabindex="-1"></a> <span class="at">cluster =</span> MV_deaths<span class="sc">$</span>state, <span class="at">target =</span> <span class="dv">1</span> <span class="sc">/</span> MV_deaths<span class="sc">$</span>pop, </span>
535 <span id="cb14-3"><a href="#cb14-3" tabindex="-1"></a> <span class="at">test =</span> <span class="st">&quot;Satterthwaite&quot;</span>)[<span class="dv">1</span><span class="sc">:</span><span class="dv">2</span>,]</span></code></pre></div>
536 <pre><code>## Alternative hypothesis: two-sided
537 ## Coef. Estimate SE Null value t-stat d.f. (Satt) p-val (Satt) Sig.
538 ## legal 7.78 2.03 0 3.83 12.64 0.00221 **
539 ## beertaxa 11.16 4.17 0 2.68 5.06 0.04333 *</code></pre>
540 <p>The new working model leads to slightly smaller standard errors and a
541 couple of additional degrees of freedom, though they remain in
542 small-sample territory.</p>
543 </div>
544 <div id="random-effects-estimation" class="section level2">
545 <h2>Random effects estimation</h2>
546 <p>If the unobserved effects <span class="math inline">\(\alpha_1,...,\alpha_{51}\)</span> are uncorrelated
547 with the regressors, then a more efficient way to estimate <span class="math inline">\(\gamma,\delta\)</span> is by weighted least
548 squares, with weights based on a random effects model. We still treat
549 the year effects as fixed.</p>
550 <div class="sourceCode" id="cb16"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb16-1"><a href="#cb16-1" tabindex="-1"></a>plm_random <span class="ot">&lt;-</span> <span class="fu">plm</span>(mrate <span class="sc">~</span> <span class="dv">0</span> <span class="sc">+</span> legal <span class="sc">+</span> beertaxa <span class="sc">+</span> year, <span class="at">data =</span> MV_deaths, </span>
551 <span id="cb16-2"><a href="#cb16-2" tabindex="-1"></a> <span class="at">effect =</span> <span class="st">&quot;individual&quot;</span>, <span class="at">index =</span> <span class="fu">c</span>(<span class="st">&quot;state&quot;</span>,<span class="st">&quot;year&quot;</span>),</span>
552 <span id="cb16-3"><a href="#cb16-3" tabindex="-1"></a> <span class="at">model =</span> <span class="st">&quot;random&quot;</span>)</span>
553 <span id="cb16-4"><a href="#cb16-4" tabindex="-1"></a><span class="fu">coef_test</span>(plm_random, <span class="at">vcov =</span> <span class="st">&quot;CR1&quot;</span>, <span class="at">test =</span> <span class="st">&quot;naive-t&quot;</span>)[<span class="dv">1</span><span class="sc">:</span><span class="dv">2</span>,]</span></code></pre></div>
554 <pre><code>## Alternative hypothesis: two-sided
555 ## Coef. Estimate SE Null value t-stat d.f. (naive-t) p-val (naive-t) Sig.
556 ## legal 7.31 2.39 0 3.054 49 0.00364 **
557 ## beertaxa 3.37 5.11 0 0.661 49 0.51202</code></pre>
558 <div class="sourceCode" id="cb18"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb18-1"><a href="#cb18-1" tabindex="-1"></a><span class="fu">coef_test</span>(plm_random, <span class="at">vcov =</span> <span class="st">&quot;CR2&quot;</span>, <span class="at">test =</span> <span class="st">&quot;Satterthwaite&quot;</span>)[<span class="dv">1</span><span class="sc">:</span><span class="dv">2</span>,]</span></code></pre></div>
559 <pre><code>## Alternative hypothesis: two-sided
560 ## Coef. Estimate SE Null value t-stat d.f. (Satt) p-val (Satt) Sig.
561 ## legal 7.31 2.46 0 2.966 25.18 0.00652 **
562 ## beertaxa 3.37 5.22 0 0.647 5.78 0.54258</code></pre>
563 <p>With random effects estimation, the effect of legal drinking age is
564 smaller by about 1 death per 100,000. As a procedural aside, note that
565 <code>coef_test</code> infers that <code>state</code> is the clustering
566 variable because the call to plm includes only one type of effects
567 (random state effects).</p>
568 </div>
569 <div id="robust-hausman-test" class="section level2">
570 <h2>Robust Hausman test</h2>
571 <p>CRVE is also used in specification tests, as in the artificial
572 Hausman-type test for endogeneity of unobserved effects (Arellano,
573 1993). As noted above, random effects estimation is more efficient than
574 fixed effects estimation, but requires the assumption that the
575 unobserved effects are uncorrelated with the regressors. However, if the
576 unobserved effects covary with <span class="math inline">\(\mathbf{b}_i,
577 \mathbf{d}_i\)</span>, then the random-effects estimator will be
578 biased.</p>
579 <p>We can test for whether endogeneity is a problem by including
580 group-centered covariates as additional regressors. Let <span class="math inline">\(\tilde{d}_{it} = d_{it} - \frac{1}{T}\sum_t
581 d_{it}\)</span>, with <span class="math inline">\(\tilde{b}_{it}\)</span> defined analogously. Now
582 estimate the regression</p>
583 <p><span class="math display">\[y_{it} = \beta_t + \gamma_1 b_{it} +
584 \gamma_2 \tilde{b}_{it} + \delta_1 d_{it} + \delta_2 \tilde{d}_{it} +
585 \epsilon_{it},\]</span></p>
586 <p>which does not include state fixed effects. The parameters <span class="math inline">\(\gamma_2,\delta_2\)</span> represent the
587 differences between the within-groups and between-groups estimands of
588 <span class="math inline">\(\gamma_1, \delta_1\)</span>. If these are
589 both zero, then the random effects estimator is unbiased. Thus, the
590 joint test for <span class="math inline">\(H_0: \gamma_2 = \delta_2 =
591 0\)</span> amounts to a test for exogeneity of the unobserved
592 effects.</p>
593 <p>For efficiency, we estimate this specification using weighted least
594 squares (although OLS would be valid too):</p>
595 <div class="sourceCode" id="cb20"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb20-1"><a href="#cb20-1" tabindex="-1"></a>MV_deaths <span class="ot">&lt;-</span> <span class="fu">within</span>(MV_deaths, {</span>
596 <span id="cb20-2"><a href="#cb20-2" tabindex="-1"></a> legal_cent <span class="ot">&lt;-</span> legal <span class="sc">-</span> <span class="fu">tapply</span>(legal, state, mean)[<span class="fu">factor</span>(state)]</span>
597 <span id="cb20-3"><a href="#cb20-3" tabindex="-1"></a> beer_cent <span class="ot">&lt;-</span> beertaxa <span class="sc">-</span> <span class="fu">tapply</span>(beertaxa, state, mean)[<span class="fu">factor</span>(state)]</span>
598 <span id="cb20-4"><a href="#cb20-4" tabindex="-1"></a>})</span>
599 <span id="cb20-5"><a href="#cb20-5" tabindex="-1"></a></span>
600 <span id="cb20-6"><a href="#cb20-6" tabindex="-1"></a>plm_Hausman <span class="ot">&lt;-</span> <span class="fu">plm</span>(mrate <span class="sc">~</span> <span class="dv">0</span> <span class="sc">+</span> legal <span class="sc">+</span> beertaxa <span class="sc">+</span> legal_cent <span class="sc">+</span> beer_cent <span class="sc">+</span> <span class="fu">factor</span>(year), </span>
601 <span id="cb20-7"><a href="#cb20-7" tabindex="-1"></a> <span class="at">data =</span> MV_deaths,</span>
602 <span id="cb20-8"><a href="#cb20-8" tabindex="-1"></a> <span class="at">effect =</span> <span class="st">&quot;individual&quot;</span>, <span class="at">index =</span> <span class="fu">c</span>(<span class="st">&quot;state&quot;</span>,<span class="st">&quot;year&quot;</span>),</span>
603 <span id="cb20-9"><a href="#cb20-9" tabindex="-1"></a> <span class="at">model =</span> <span class="st">&quot;random&quot;</span>)</span>
604 <span id="cb20-10"><a href="#cb20-10" tabindex="-1"></a><span class="fu">coef_test</span>(plm_Hausman, <span class="at">vcov =</span> <span class="st">&quot;CR2&quot;</span>, <span class="at">test =</span> <span class="st">&quot;Satterthwaite&quot;</span>)[<span class="dv">1</span><span class="sc">:</span><span class="dv">4</span>,]</span></code></pre></div>
605 <pre><code>## Alternative hypothesis: two-sided
606 ## Coef. Estimate SE Null value t-stat d.f. (Satt) p-val (Satt) Sig.
607 ## legal -9.180 7.62 0 -1.2042 24.94 0.2398
608 ## beertaxa 3.395 9.40 0 0.3613 6.44 0.7295
609 ## legal_cent 16.768 8.53 0 1.9665 25.44 0.0602 .
610 ## beer_cent 0.424 9.25 0 0.0458 6.42 0.9648</code></pre>
611 <p>To conduct a joint test on the centered covariates, we can use the
612 <code>Wald_test</code> function. The usual way to test this hypothesis
613 would be to use the <code>CR1</code> variance estimator to calculate the
614 robust Wald statistic, then use a <span class="math inline">\(\chi^2_2\)</span> reference distribution (or
615 equivalently, compare a re-scaled Wald statistic to an <span class="math inline">\(F(2,\infty)\)</span> distribution). The
616 <code>Wald_test</code> function reports the latter version:</p>
617 <div class="sourceCode" id="cb22"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb22-1"><a href="#cb22-1" tabindex="-1"></a><span class="fu">Wald_test</span>(plm_Hausman, </span>
618 <span id="cb22-2"><a href="#cb22-2" tabindex="-1"></a> <span class="at">constraints =</span> <span class="fu">constrain_zero</span>(<span class="fu">c</span>(<span class="st">&quot;legal_cent&quot;</span>,<span class="st">&quot;beer_cent&quot;</span>)), </span>
619 <span id="cb22-3"><a href="#cb22-3" tabindex="-1"></a> <span class="at">vcov =</span> <span class="st">&quot;CR1&quot;</span>, <span class="at">test =</span> <span class="st">&quot;chi-sq&quot;</span>)</span></code></pre></div>
620 <pre><code>## test Fstat df_num df_denom p_val sig
621 ## chi-sq 2.93 2 Inf 0.0534 .</code></pre>
622 <p>The test is just shy of significance at the 5% level. If we instead
623 use the <code>CR2</code> variance estimator and our newly proposed
624 approximate F-test (which is the default in <code>Wald_test</code>),
625 then we get:</p>
626 <div class="sourceCode" id="cb24"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb24-1"><a href="#cb24-1" tabindex="-1"></a><span class="fu">Wald_test</span>(plm_Hausman, </span>
627 <span id="cb24-2"><a href="#cb24-2" tabindex="-1"></a> <span class="at">constraints =</span> <span class="fu">constrain_zero</span>(<span class="fu">c</span>(<span class="st">&quot;legal_cent&quot;</span>,<span class="st">&quot;beer_cent&quot;</span>)), </span>
628 <span id="cb24-3"><a href="#cb24-3" tabindex="-1"></a> <span class="at">vcov =</span> <span class="st">&quot;CR2&quot;</span>)</span></code></pre></div>
629 <pre><code>## test Fstat df_num df_denom p_val sig
630 ## HTZ 2.56 2 11.7 0.12</code></pre>
631 <p>The low degrees of freedom of the test indicate that we’re definitely
632 in small-sample territory and should not trust the asymptotic <span class="math inline">\(\chi^2\)</span> approximation.</p>
633 </div>
634 <div id="references" class="section level2">
635 <h2>References</h2>
636 <p>Angrist, J. D., &amp; Pischke, J. S. (2009). <em>Mostly Harmless
637 Econometrics: An Empiricist’s Companion</em>. Princeton, NJ: Princeton
638 University Press.</p>
639 <p>Angrist, J. D., &amp; Pischke, J. S. (2014). <em>Mastering ’Metrics:
640 The Path from Cause to Effect</em>. Princeton, NJ: Princeton University
641 Press.</p>
642 <p>Arellano, M. (1993). On the testing of correlated effects with panel
643 data. Journal of Econometrics, 59(1-2), 87-97. doi: <a href="https://doi.org/10.1016/0304-4076(93)90040-C">10.1016/0304-4076(93)90040-C</a></p>
644 <p>Bell, R. M., &amp; McCaffrey, D. F. (2002). Bias reduction in
645 standard errors for linear regression with multi-stage samples.
646 <em>Survey Methodology, 28</em>(2), 169-181.</p>
647 <p>Cameron, A. C., &amp; Miller, D. L. (2015). A practitioner’s guide to
648 cluster-robust inference. <em>Journal of Human Resources, 50</em>(2),
649 317-372. doi: <a href="https://doi.org/10.3368/jhr.50.2.317">10.3368/jhr.50.2.317</a></p>
650 <p>Carpenter, C., &amp; Dobkin, C. (2011). The minimum legal drinking
651 age and public health. <em>Journal of Economic Perspectives, 25</em>(2),
652 133-156. doi: <a href="https://doi.org/10.1257/jep.25.2.133">10.1257/jep.25.2.133</a></p>
653 <p>Imbens, G. W., &amp; Kolesar, M. (2015). Robust standard errors in
654 small samples: Some practical advice. doi: <a href="https://doi.org/10.1162/REST_a_00552">10.1162/REST_a_00552</a></p>
655 <p>Pustejovsky, J. E. &amp; Tipton, E. (2016). Small sample methods for
656 cluster-robust variance estimation and hypothesis testing in fixed
657 effects models. arXiv: <a href="https://arxiv.org/abs/1601.01981">1601.01981</a> [stat.ME]</p>
658 <p>Young, A. (2016). Improved, nearly exact, statistical inference with
659 robust and clustered covariance matrices using effective degrees of
660 freedom corrections.</p>
661 </div>
662
663
664
665 <!-- code folding -->
666
667
668 <!-- dynamically load mathjax for compatibility with self-contained -->
669 <script>
670 (function () {
671 var script = document.createElement("script");
672 script.type = "text/javascript";
673 script.src = "https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML";
674 document.getElementsByTagName("head")[0].appendChild(script);
675 })();
676 </script>
677
678 </body>
679 </html>