Debian Codegraph @

inst/doc/Wald-tests-in-clubSandwich.html

raw
1 <!DOCTYPE html>
2
3 <html>
4
5 <head>
6
7 <meta charset="utf-8" />
8 <meta name="generator" content="pandoc" />
9 <meta http-equiv="X-UA-Compatible" content="IE=EDGE" />
10
11 <meta name="viewport" content="width=device-width, initial-scale=1" />
12
13 <meta name="author" content="James E. Pustejovsky" />
14
15 <meta name="date" content="2026-05-01" />
16
17 <title>Wald tests of multiple-constraint null hypotheses</title>
18
19 <script>// Pandoc 2.9 adds attributes on both header and div. We remove the former (to
20 // be compatible with the behavior of Pandoc < 2.8).
21 document.addEventListener('DOMContentLoaded', function(e) {
22 var hs = document.querySelectorAll("div.section[class*='level'] > :first-child");
23 var i, h, a;
24 for (i = 0; i < hs.length; i++) {
25 h = hs[i];
26 if (!/^h[1-6]$/i.test(h.tagName)) continue; // it should be a header h1-h6
27 a = h.attributes;
28 while (a.length > 0) h.removeAttribute(a[0].name);
29 }
30 });
31 </script>
32
33 <style type="text/css">
34 code{white-space: pre-wrap;}
35 span.smallcaps{font-variant: small-caps;}
36 span.underline{text-decoration: underline;}
37 div.column{display: inline-block; vertical-align: top; width: 50%;}
38 div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
39 ul.task-list{list-style: none;}
40 </style>
41
42
43
44 <style type="text/css">
45 code {
46 white-space: pre;
47 }
48 .sourceCode {
49 overflow: visible;
50 }
51 </style>
52 <style type="text/css" data-origin="pandoc">
53 html { -webkit-text-size-adjust: 100%; }
54 pre > code.sourceCode { white-space: pre; position: relative; }
55 pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
56 pre > code.sourceCode > span:empty { height: 1.2em; }
57 .sourceCode { overflow: visible; }
58 code.sourceCode > span { color: inherit; text-decoration: inherit; }
59 div.sourceCode { margin: 1em 0; }
60 pre.sourceCode { margin: 0; }
61 @media screen {
62 div.sourceCode { overflow: auto; }
63 }
64 @media print {
65 pre > code.sourceCode { white-space: pre-wrap; }
66 pre > code.sourceCode > span { text-indent: -5em; padding-left: 5em; }
67 }
68 pre.numberSource code
69 { counter-reset: source-line 0; }
70 pre.numberSource code > span
71 { position: relative; left: -4em; counter-increment: source-line; }
72 pre.numberSource code > span > a:first-child::before
73 { content: counter(source-line);
74 position: relative; left: -1em; text-align: right; vertical-align: baseline;
75 border: none; display: inline-block;
76 -webkit-touch-callout: none; -webkit-user-select: none;
77 -khtml-user-select: none; -moz-user-select: none;
78 -ms-user-select: none; user-select: none;
79 padding: 0 4px; width: 4em;
80 color: #aaaaaa;
81 }
82 pre.numberSource { margin-left: 3em; border-left: 1px solid #aaaaaa; padding-left: 4px; }
83 div.sourceCode
84 { }
85 @media screen {
86 pre > code.sourceCode > span > a:first-child::before { text-decoration: underline; }
87 }
88 code span.al { color: #ff0000; font-weight: bold; }
89 code span.an { color: #60a0b0; font-weight: bold; font-style: italic; }
90 code span.at { color: #7d9029; }
91 code span.bn { color: #40a070; }
92 code span.bu { color: #008000; }
93 code span.cf { color: #007020; font-weight: bold; }
94 code span.ch { color: #4070a0; }
95 code span.cn { color: #880000; }
96 code span.co { color: #60a0b0; font-style: italic; }
97 code span.cv { color: #60a0b0; font-weight: bold; font-style: italic; }
98 code span.do { color: #ba2121; font-style: italic; }
99 code span.dt { color: #902000; }
100 code span.dv { color: #40a070; }
101 code span.er { color: #ff0000; font-weight: bold; }
102 code span.ex { }
103 code span.fl { color: #40a070; }
104 code span.fu { color: #06287e; }
105 code span.im { color: #008000; font-weight: bold; }
106 code span.in { color: #60a0b0; font-weight: bold; font-style: italic; }
107 code span.kw { color: #007020; font-weight: bold; }
108 code span.op { color: #666666; }
109 code span.ot { color: #007020; }
110 code span.pp { color: #bc7a00; }
111 code span.sc { color: #4070a0; }
112 code span.ss { color: #bb6688; }
113 code span.st { color: #4070a0; }
114 code span.va { color: #19177c; }
115 code span.vs { color: #4070a0; }
116 code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; }
117 </style>
118 <script>
119 // apply pandoc div.sourceCode style to pre.sourceCode instead
120 (function() {
121 var sheets = document.styleSheets;
122 for (var i = 0; i < sheets.length; i++) {
123 if (sheets[i].ownerNode.dataset["origin"] !== "pandoc") continue;
124 try { var rules = sheets[i].cssRules; } catch (e) { continue; }
125 var j = 0;
126 while (j < rules.length) {
127 var rule = rules[j];
128 // check if there is a div.sourceCode rule
129 if (rule.type !== rule.STYLE_RULE || rule.selectorText !== "div.sourceCode") {
130 j++;
131 continue;
132 }
133 var style = rule.style.cssText;
134 // check if color or background-color is set
135 if (rule.style.color === '' && rule.style.backgroundColor === '') {
136 j++;
137 continue;
138 }
139 // replace div.sourceCode by a pre.sourceCode rule
140 sheets[i].deleteRule(j);
141 sheets[i].insertRule('pre.sourceCode{' + style + '}', j);
142 }
143 }
144 })();
145 </script>
146
147
148
149 <style type="text/css">
150
151 div.csl-bib-body { }
152 div.csl-entry {
153 clear: both;
154 margin-bottom: 0em;
155 }
156 .hanging div.csl-entry {
157 margin-left:2em;
158 text-indent:-2em;
159 }
160 div.csl-left-margin {
161 min-width:2em;
162 float:left;
163 }
164 div.csl-right-inline {
165 margin-left:2em;
166 padding-left:1em;
167 }
168 div.csl-indent {
169 margin-left: 2em;
170 }
171 </style>
172
173 <style type="text/css">body {
174 background-color: #fff;
175 margin: 1em auto;
176 max-width: 700px;
177 overflow: visible;
178 padding-left: 2em;
179 padding-right: 2em;
180 font-family: "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;
181 font-size: 14px;
182 line-height: 1.35;
183 }
184 #TOC {
185 clear: both;
186 margin: 0 0 10px 10px;
187 padding: 4px;
188 width: 400px;
189 border: 1px solid #CCCCCC;
190 border-radius: 5px;
191 background-color: #f6f6f6;
192 font-size: 13px;
193 line-height: 1.3;
194 }
195 #TOC .toctitle {
196 font-weight: bold;
197 font-size: 15px;
198 margin-left: 5px;
199 }
200 #TOC ul {
201 padding-left: 40px;
202 margin-left: -1.5em;
203 margin-top: 5px;
204 margin-bottom: 5px;
205 }
206 #TOC ul ul {
207 margin-left: -2em;
208 }
209 #TOC li {
210 line-height: 16px;
211 }
212 table {
213 margin: 1em auto;
214 border-width: 1px;
215 border-color: #DDDDDD;
216 border-style: outset;
217 border-collapse: collapse;
218 }
219 table th {
220 border-width: 2px;
221 padding: 5px;
222 border-style: inset;
223 }
224 table td {
225 border-width: 1px;
226 border-style: inset;
227 line-height: 18px;
228 padding: 5px 5px;
229 }
230 table, table th, table td {
231 border-left-style: none;
232 border-right-style: none;
233 }
234 table thead, table tr.even {
235 background-color: #f7f7f7;
236 }
237 p {
238 margin: 0.5em 0;
239 }
240 blockquote {
241 background-color: #f6f6f6;
242 padding: 0.25em 0.75em;
243 }
244 hr {
245 border-style: solid;
246 border: none;
247 border-top: 1px solid #777;
248 margin: 28px 0;
249 }
250 dl {
251 margin-left: 0;
252 }
253 dl dd {
254 margin-bottom: 13px;
255 margin-left: 13px;
256 }
257 dl dt {
258 font-weight: bold;
259 }
260 ul {
261 margin-top: 0;
262 }
263 ul li {
264 list-style: circle outside;
265 }
266 ul ul {
267 margin-bottom: 0;
268 }
269 pre, code {
270 background-color: #f7f7f7;
271 border-radius: 3px;
272 color: #333;
273 white-space: pre-wrap;
274 }
275 pre {
276 border-radius: 3px;
277 margin: 5px 0px 10px 0px;
278 padding: 10px;
279 }
280 pre:not([class]) {
281 background-color: #f7f7f7;
282 }
283 code {
284 font-family: Consolas, Monaco, 'Courier New', monospace;
285 font-size: 85%;
286 }
287 p > code, li > code {
288 padding: 2px 0px;
289 }
290 div.figure {
291 text-align: center;
292 }
293 img {
294 background-color: #FFFFFF;
295 padding: 2px;
296 border: 1px solid #DDDDDD;
297 border-radius: 3px;
298 border: 1px solid #CCCCCC;
299 margin: 0 5px;
300 }
301 h1 {
302 margin-top: 0;
303 font-size: 35px;
304 line-height: 40px;
305 }
306 h2 {
307 border-bottom: 4px solid #f7f7f7;
308 padding-top: 10px;
309 padding-bottom: 2px;
310 font-size: 145%;
311 }
312 h3 {
313 border-bottom: 2px solid #f7f7f7;
314 padding-top: 10px;
315 font-size: 120%;
316 }
317 h4 {
318 border-bottom: 1px solid #f7f7f7;
319 margin-left: 8px;
320 font-size: 105%;
321 }
322 h5, h6 {
323 border-bottom: 1px solid #ccc;
324 font-size: 105%;
325 }
326 a {
327 color: #0033dd;
328 text-decoration: none;
329 }
330 a:hover {
331 color: #6666ff; }
332 a:visited {
333 color: #800080; }
334 a:visited:hover {
335 color: #BB00BB; }
336 a[href^="http:"] {
337 text-decoration: underline; }
338 a[href^="https:"] {
339 text-decoration: underline; }
340
341 code > span.kw { color: #555; font-weight: bold; }
342 code > span.dt { color: #902000; }
343 code > span.dv { color: #40a070; }
344 code > span.bn { color: #d14; }
345 code > span.fl { color: #d14; }
346 code > span.ch { color: #d14; }
347 code > span.st { color: #d14; }
348 code > span.co { color: #888888; font-style: italic; }
349 code > span.ot { color: #007020; }
350 code > span.al { color: #ff0000; font-weight: bold; }
351 code > span.fu { color: #900; font-weight: bold; }
352 code > span.er { color: #a61717; background-color: #e3d2d2; }
353 </style>
354
355
356
357
358 </head>
359
360 <body>
361
362
363
364
365 <h1 class="title toc-ignore">Wald tests of multiple-constraint null
366 hypotheses</h1>
367 <h4 class="author">James E. Pustejovsky</h4>
368 <h4 class="date">2026-05-01</h4>
369
370
371 <div id="TOC">
372 <ul>
373 <li><a href="#the-wald-test-function" id="toc-the-wald-test-function"><span class="toc-section-number">1</span> The Wald test function</a>
374 <ul>
375 <li><a href="#testing-treatment-effects" id="toc-testing-treatment-effects"><span class="toc-section-number">1.1</span> Testing treatment effects</a></li>
376 <li><a href="#constrain_zero" id="toc-constrain_zero"><span class="toc-section-number">1.2</span>
377 <code>constrain_zero()</code></a></li>
378 <li><a href="#constrain_equal" id="toc-constrain_equal"><span class="toc-section-number">1.3</span>
379 <code>constrain_equal()</code></a></li>
380 </ul></li>
381 <li><a href="#testing-an-interaction" id="toc-testing-an-interaction"><span class="toc-section-number">2</span> Testing an interaction</a>
382 <ul>
383 <li><a href="#lists-of-constraints" id="toc-lists-of-constraints"><span class="toc-section-number">2.1</span> Lists of constraints</a></li>
384 </ul></li>
385 <li><a href="#pairwise-t-tests" id="toc-pairwise-t-tests"><span class="toc-section-number">3</span> Pairwise t-tests</a></li>
386 <li><a href="#remark" id="toc-remark"><span class="toc-section-number">4</span> Remark</a></li>
387 <li><a href="#references" id="toc-references">References</a></li>
388 </ul>
389 </div>
390
391 <p>Version 0.5.0 of <code>clubSandwich</code> introduced a new syntax
392 for <code>Wald_test()</code>, a function for conducting tests of
393 multiple-constraint hypotheses. In previous versions, this function was
394 poorly documented and, consequently, probably little used. This vignette
395 will demonstrate the new syntax.</p>
396 <p>For purposes of illustration, I will use the <code>STAR</code> data
397 (available in the <code>AER</code> package), which is drawn from a
398 randomized trial evaluating the effects of elementary school class size
399 on student achievement. The data consist of individual-level measures
400 for students in each of several dozen schools. For purposes of
401 illustration, I will look at effects on math performance in first grade.
402 Treatment conditions (the variable called <code>stark</code>) were
403 assigned at the classroom level, and consisted of either a) a
404 regular-size class, b) a small-size class, or c) a regular-size class
405 but with the addition of a teacher’s aide. In all of what follows, I
406 will cluster standard errors by school in order to allow for
407 generalization to a super-population of schools.</p>
408 <div class="sourceCode" id="cb1"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1-1"><a href="#cb1-1" tabindex="-1"></a><span class="fu">library</span>(clubSandwich)</span>
409 <span id="cb1-2"><a href="#cb1-2" tabindex="-1"></a></span>
410 <span id="cb1-3"><a href="#cb1-3" tabindex="-1"></a><span class="fu">data</span>(STAR, <span class="at">package =</span> <span class="st">&quot;AER&quot;</span>)</span>
411 <span id="cb1-4"><a href="#cb1-4" tabindex="-1"></a></span>
412 <span id="cb1-5"><a href="#cb1-5" tabindex="-1"></a><span class="co"># clean up a few variables</span></span>
413 <span id="cb1-6"><a href="#cb1-6" tabindex="-1"></a><span class="fu">levels</span>(STAR<span class="sc">$</span>stark)[<span class="dv">3</span>] <span class="ot">&lt;-</span> <span class="st">&quot;aide&quot;</span></span>
414 <span id="cb1-7"><a href="#cb1-7" tabindex="-1"></a><span class="fu">levels</span>(STAR<span class="sc">$</span>schoolk)[<span class="dv">1</span>] <span class="ot">&lt;-</span> <span class="st">&quot;urban&quot;</span></span>
415 <span id="cb1-8"><a href="#cb1-8" tabindex="-1"></a>STAR <span class="ot">&lt;-</span> <span class="fu">subset</span>(STAR, </span>
416 <span id="cb1-9"><a href="#cb1-9" tabindex="-1"></a> <span class="sc">!</span><span class="fu">is.na</span>(schoolidk),</span>
417 <span id="cb1-10"><a href="#cb1-10" tabindex="-1"></a> <span class="at">select =</span> <span class="fu">c</span>(schoolidk, schoolk, stark, gender, ethnicity, math1, lunchk))</span>
418 <span id="cb1-11"><a href="#cb1-11" tabindex="-1"></a><span class="fu">head</span>(STAR)</span></code></pre></div>
419 <pre><code>## schoolidk schoolk stark gender ethnicity math1 lunchk
420 ## 1137 63 rural small female cauc 538 non-free
421 ## 1143 20 suburban small female afam 592 non-free
422 ## 1183 19 urban aide male afam NA free
423 ## 1277 69 rural regular male cauc 584 non-free
424 ## 1292 79 rural small male cauc 545 free
425 ## 1308 5 rural regular male cauc 553 free</code></pre>
426 <div id="the-wald-test-function" class="section level1" number="1">
427 <h1><span class="header-section-number">1</span> The Wald test
428 function</h1>
429 <p>The <code>Wald_test()</code> function can be used to conduct
430 hypothesis tests that involve multiple constraints on the regression
431 coefficients. Consider a linear model for an outcome <span class="math inline">\(Y_{ij}\)</span> regressed on a <span class="math inline">\(1 \times p\)</span> row vector of predictors <span class="math inline">\(\mathbf{x}_{ij}\)</span> (which might include a
432 constant intercept term): <span class="math display">\[
433 Y_{ij} = \mathbf{x}_{ij} \boldsymbol\beta + \epsilon_{ij}
434 \]</span> The regression coefficient vector is <span class="math inline">\(\boldsymbol\beta\)</span>. In quite general terms,
435 a set of constraints on the regression coefficient vector can be
436 expressed in terms of a <span class="math inline">\(q \times p\)</span>
437 matrix <span class="math inline">\(\mathbf{C}\)</span>, where each row
438 of <span class="math inline">\(\mathbf{C}\)</span> corresponds to one
439 constraint. A joint null hypothesis is then <span class="math inline">\(H_0: \mathbf{C} \boldsymbol\beta =
440 \mathbf{0}\)</span>, where <span class="math inline">\(\mathbf{0}\)</span> is a <span class="math inline">\(q \times 1\)</span> vector of zeros.<a href="#fn1" class="footnote-ref" id="fnref1"><sup>1</sup></a></p>
441 <p>Wald-type test are based on the test statistic <span class="math display">\[
442 Q = \left(\mathbf{C}\boldsymbol{\hat\beta}\right)&#39; \left(\mathbf{C}
443 \mathbf{V}^{CR} \mathbf{C}&#39;\right)^{-1}
444 \left(\mathbf{C}\boldsymbol{\hat\beta}\right),
445 \]</span> where <span class="math inline">\(\boldsymbol{\hat\beta}\)</span> is the estimated
446 regression coefficient vector and <span class="math inline">\(\mathbf{V}^{CR}\)</span> is a cluster-robust
447 variance matrix. If the number of clusters is sufficiently large, then
448 the distribution of <span class="math inline">\(Q\)</span> under the
449 null hypothesis is approximately <span class="math inline">\(\chi^2(q)\)</span>. <span class="citation">Tipton
450 &amp; Pustejovsky (2015)</span> investigated a wide range of other
451 approximations to the null distribution of <span class="math inline">\(Q\)</span>, many of which are included as options
452 in <code>Wald_test()</code>. Based on a large simulation, they (…er…we…)
453 recommended a method called the “approximate Hotelling’s <span class="math inline">\(T^2\)</span>-Z” test, or “AHZ.” This test
454 approximates the distribution of <span class="math inline">\(Q /
455 q\)</span> by a <span class="math inline">\(T^2\)</span> distribution,
456 which is a multiple of an <span class="math inline">\(F\)</span>
457 distribution, with numerator degrees of freedom <span class="math inline">\(q\)</span> and denominator degrees of freedom
458 based on a generalization of the Satterthwaite approximation.</p>
459 <p>The <code>Wald_test()</code> function has three main arguments:</p>
460 <div class="sourceCode" id="cb3"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb3-1"><a href="#cb3-1" tabindex="-1"></a><span class="fu">args</span>(Wald_test)</span></code></pre></div>
461 <pre><code>## function (obj, constraints, vcov, null_constant = 0, test = &quot;HTZ&quot;,
462 ## tidy = FALSE, adjustment_method = &quot;none&quot;, ...)
463 ## NULL</code></pre>
464 <ul>
465 <li>The <code>obj</code> argument is used to specify the estimated
466 regression model on which to perform the test,</li>
467 <li>the <code>constraints</code> argument is a <span class="math inline">\(\mathbf{C}\)</span> matrix expressing the set of
468 constraints to test, and</li>
469 <li>the <code>vcov</code> argument is a cluster-robust variance matrix,
470 which is used to construct the test statistic. (Alternately,
471 <code>vcov</code> can be the type of cluster-robust variance matrix to
472 construct, in which case it will be computed internally.)</li>
473 </ul>
474 <p>By default, <code>Wald_test()</code> will use the HTZ small-sample
475 approximation. Other options are available (via the <code>test</code>
476 argument) but not recommended for routine use. The optional
477 <code>tidy</code> argument will be demonstrated below.</p>
478 <div id="testing-treatment-effects" class="section level2" number="1.1">
479 <h2><span class="header-section-number">1.1</span> Testing treatment
480 effects</h2>
481 <p>Returning to the STAR data, let’s suppose we want to examine
482 differences in math performance across class sizes. This can be done
483 with a simple linear regression model, while clustering the standard
484 errors by <code>schoolidk</code>. The estimating equation is <span class="math display">\[
485 \left(\text{Math}\right)_{ij} = \beta_0 + \beta_1
486 \left(\text{small}\right)_{ij} + \beta_2 \left(\text{aide}\right)_{ij} +
487 e_{ij},
488 \]</span> which can be estimated in R as follows:</p>
489 <div class="sourceCode" id="cb5"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb5-1"><a href="#cb5-1" tabindex="-1"></a>lm_trt <span class="ot">&lt;-</span> <span class="fu">lm</span>(math1 <span class="sc">~</span> stark, <span class="at">data =</span> STAR)</span>
490 <span id="cb5-2"><a href="#cb5-2" tabindex="-1"></a>V_trt <span class="ot">&lt;-</span> <span class="fu">vcovCR</span>(lm_trt, <span class="at">cluster =</span> STAR<span class="sc">$</span>schoolidk, <span class="at">type =</span> <span class="st">&quot;CR2&quot;</span>)</span>
491 <span id="cb5-3"><a href="#cb5-3" tabindex="-1"></a><span class="fu">coef_test</span>(lm_trt, <span class="at">vcov =</span> V_trt)</span></code></pre></div>
492 <pre><code>## Alternative hypothesis: two-sided
493 ## Coef. Estimate SE Null value t-stat d.f. (Satt) p-val (Satt) Sig.
494 ## (Intercept) 531.727 2.78 0 191.506 59.9 &lt;0.001 ***
495 ## starksmall 9.469 2.30 0 4.114 65.6 &lt;0.001 ***
496 ## starkaide -0.483 1.86 0 -0.259 65.6 0.796</code></pre>
497 <p>In this estimating equation, the coefficients <span class="math inline">\(\beta_1\)</span> and <span class="math inline">\(\beta_2\)</span> represent treatment effects, or
498 differences in average math scores relative to the reference level of
499 <code>stark</code>, which in this case is a regular-size class. The
500 t-statistics and p-values reported by <code>coef_test</code> are
501 separate tests of the null hypotheses that each of these coefficients
502 are equal to zero, meaning that there is no difference between the
503 specified treatment condition and the reference level. We might want to
504 instead test the <em>joint</em> null hypothesis that there are no
505 differences among <em>any</em> of the conditions. This null can be
506 expressed by a set of multiple constraints on the parameters: <span class="math inline">\(\beta_1 = 0\)</span> and <span class="math inline">\(\beta_2 = 0\)</span>.</p>
507 <p>To test the null hypothesis that <span class="math inline">\(\beta_1
508 = \beta_2 = 0\)</span> based on the treatment effects model
509 specification, we can use:</p>
510 <div class="sourceCode" id="cb7"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb7-1"><a href="#cb7-1" tabindex="-1"></a>C_trt <span class="ot">&lt;-</span> <span class="fu">matrix</span>(<span class="fu">c</span>(<span class="dv">0</span>,<span class="dv">0</span>,<span class="dv">1</span>,<span class="dv">0</span>,<span class="dv">0</span>,<span class="dv">1</span>), <span class="dv">2</span>, <span class="dv">3</span>)</span>
511 <span id="cb7-2"><a href="#cb7-2" tabindex="-1"></a>C_trt</span></code></pre></div>
512 <pre><code>## [,1] [,2] [,3]
513 ## [1,] 0 1 0
514 ## [2,] 0 0 1</code></pre>
515 <div class="sourceCode" id="cb9"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb9-1"><a href="#cb9-1" tabindex="-1"></a><span class="fu">Wald_test</span>(lm_trt, <span class="at">constraints =</span> C_trt, <span class="at">vcov =</span> V_trt)</span></code></pre></div>
516 <pre><code>## test Fstat df_num df_denom p_val sig
517 ## HTZ 10.2 2 65.3 &lt;0.001 ***</code></pre>
518 <p>The result includes details about the form of <code>test</code>
519 computed, the <span class="math inline">\(F\)</span>-statistic, the
520 numerator and denominator degrees of freedom used to compute the
521 reference distribution, and the <span class="math inline">\(p\)</span>-value corresponding to the specified
522 null hypothesis. In this example, <span class="math inline">\(p =
523 0.000141\)</span>, so we can rule out the null hypothesis that there are
524 no differences in math performance across conditions.</p>
525 <p>The representation of null hypotheses as arbitrary constraint
526 matrices is useful for developing theory about how to test such
527 hypotheses, but it is not all that helpful for actually running
528 tests—constructing constraint matrices “by hand” is just too cumbersome
529 of an exercise. Moreover, <span class="math inline">\(\mathbf{C}\)</span> matrices typically follow one
530 of a small number of patterns. Two common use cases are a) constraining
531 a set of <span class="math inline">\(q &gt; 1\)</span> parameters to all
532 be equal to zero and b) constraining a set of <span class="math inline">\(q + 1\)</span> parameters to be equal to a common
533 value. The <code>clubSandwich</code> package now includes a set of
534 helper functions to create constraint matrices for these common use
535 cases.</p>
536 </div>
537 <div id="constrain_zero" class="section level2" number="1.2">
538 <h2><span class="header-section-number">1.2</span>
539 <code>constrain_zero()</code></h2>
540 <p>To constrain a set of <span class="math inline">\(q\)</span>
541 regression coefficients to all be equal to zero, the simplest form of
542 the <span class="math inline">\(\mathbf{C}\)</span> matrix would consist
543 of a set of <span class="math inline">\(q\)</span> rows, where a single
544 entry in each row would be equal to 1 and the remaining entries would
545 all be zero. For the <code>lm_trt</code> model, the C matrix would look
546 like this: <span class="math display">\[
547 \mathbf{C} = \left[\begin{array}{ccc} 0 &amp; 1 &amp; 0 \\ 0 &amp; 0
548 &amp; 1 \end{array} \right],
549 \]</span> so that <span class="math display">\[
550 \mathbf{C}\boldsymbol\beta = \left[\begin{array}{ccc} 0 &amp; 1 &amp; 0
551 \\ 0 &amp; 0 &amp; 1 \end{array} \right] \left[\begin{array}{c} \beta_0
552 \\ \beta_1 \\ \beta_2 \end{array} \right] = \left[\begin{array}{c}
553 \beta_1 \\ \beta_2 \end{array} \right],
554 \]</span> which is set equal to <span class="math inline">\(\left[\begin{array}{c} 0 \\ 0 \end{array}
555 \right]\)</span>.</p>
556 <p>The <code>constrain_zero()</code> function will create matrices like
557 this automatically. The function takes two main arguments:</p>
558 <div class="sourceCode" id="cb11"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb11-1"><a href="#cb11-1" tabindex="-1"></a><span class="fu">args</span>(constrain_zero)</span></code></pre></div>
559 <pre><code>## function (constraints, coefs, reg_ex = FALSE)
560 ## NULL</code></pre>
561 <ul>
562 <li>The <code>constraints</code> argument is used to specify
563 <em>which</em> coefficients in a regression model to set equal to
564 zero.</li>
565 <li>The <code>coefs</code> argument is the set of estimated regression
566 coefficients, for which to calculate the constraints.</li>
567 </ul>
568 <p>Constraints can be specified by position index, by name, or via a
569 regular expression. To test the joint null hypothesis that average math
570 performance is equal across the three treatment conditions, we need to
571 constrain the second and third coefficients to zero:</p>
572 <div class="sourceCode" id="cb13"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb13-1"><a href="#cb13-1" tabindex="-1"></a><span class="fu">constrain_zero</span>(<span class="dv">2</span><span class="sc">:</span><span class="dv">3</span>, <span class="at">coefs =</span> <span class="fu">coef</span>(lm_trt))</span></code></pre></div>
573 <pre><code>## [,1] [,2] [,3]
574 ## [1,] 0 1 0
575 ## [2,] 0 0 1</code></pre>
576 <p>Or equivalently:</p>
577 <div class="sourceCode" id="cb15"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb15-1"><a href="#cb15-1" tabindex="-1"></a><span class="fu">constrain_zero</span>(<span class="fu">c</span>(<span class="st">&quot;starksmall&quot;</span>,<span class="st">&quot;starkaide&quot;</span>), <span class="at">coefs =</span> <span class="fu">coef</span>(lm_trt))</span></code></pre></div>
578 <pre><code>## [,1] [,2] [,3]
579 ## [1,] 0 1 0
580 ## [2,] 0 0 1</code></pre>
581 <p>or</p>
582 <div class="sourceCode" id="cb17"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb17-1"><a href="#cb17-1" tabindex="-1"></a><span class="fu">constrain_zero</span>(<span class="st">&quot;^stark&quot;</span>, <span class="at">coefs =</span> <span class="fu">coef</span>(lm_trt), <span class="at">reg_ex =</span> <span class="cn">TRUE</span>)</span></code></pre></div>
583 <pre><code>## [,1] [,2] [,3]
584 ## [1,] 0 1 0
585 ## [2,] 0 0 1</code></pre>
586 <p>Note that if <code>constraints</code> is a regular expression, then
587 the <code>reg_ex</code> argument needs to be set to
588 <code>TRUE</code>.</p>
589 <p>The result of <code>constrain_zero()</code> can then be fed into the
590 <code>Wald_test()</code> function:</p>
591 <div class="sourceCode" id="cb19"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb19-1"><a href="#cb19-1" tabindex="-1"></a>C_trt <span class="ot">&lt;-</span> <span class="fu">constrain_zero</span>(<span class="dv">2</span><span class="sc">:</span><span class="dv">3</span>, <span class="at">coefs =</span> <span class="fu">coef</span>(lm_trt))</span>
592 <span id="cb19-2"><a href="#cb19-2" tabindex="-1"></a><span class="fu">Wald_test</span>(lm_trt, <span class="at">constraints =</span> C_trt, <span class="at">vcov =</span> V_trt)</span></code></pre></div>
593 <pre><code>## test Fstat df_num df_denom p_val sig
594 ## HTZ 10.2 2 65.3 &lt;0.001 ***</code></pre>
595 <p>To reduce redundancy in the syntax, we can also omit the
596 <code>coefs</code> argument to <code>constrain_zero</code>, so long as
597 we call it inside of <code>Wald_test</code><a href="#fn2" class="footnote-ref" id="fnref2"><sup>2</sup></a>:</p>
598 <div class="sourceCode" id="cb21"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb21-1"><a href="#cb21-1" tabindex="-1"></a><span class="fu">Wald_test</span>(lm_trt, <span class="at">constraints =</span> <span class="fu">constrain_zero</span>(<span class="dv">2</span><span class="sc">:</span><span class="dv">3</span>), <span class="at">vcov =</span> V_trt)</span></code></pre></div>
599 <pre><code>## test Fstat df_num df_denom p_val sig
600 ## HTZ 10.2 2 65.3 &lt;0.001 ***</code></pre>
601 </div>
602 <div id="constrain_equal" class="section level2" number="1.3">
603 <h2><span class="header-section-number">1.3</span>
604 <code>constrain_equal()</code></h2>
605 <p>Another common type of constraints involve setting a set of <span class="math inline">\(q + 1\)</span> regression coefficients to be all
606 equal to a common (but unknown) value (<span class="math inline">\(q +
607 1\)</span> because it takes <span class="math inline">\(q\)</span>
608 constraints to do this). There are many equivalent ways to express such
609 a set of constraints in terms of a <span class="math inline">\(\mathbf{C}\)</span> matrix. One fairly simple form
610 consists of a set of <span class="math inline">\(q\)</span> rows, where
611 the entry corresponding to one of the coefficients of interest is equal
612 to -1 and the entry corresponding to another coefficient of interest is
613 equal to 1.</p>
614 <p>To see how this works, let’s look at a different way of
615 parameterizing our simple model for the STAR data, by using separate
616 intercepts for each treatment condition. The estimating equation would
617 then be <span class="math display">\[
618 \left(\text{Math}\right)_{ij} = \beta_0 \left(\text{regular}\right)_{ij}
619 + \beta_1 \left(\text{small}\right)_{ij} + \beta_2
620 \left(\text{aide}\right)_{ij} + e_{ij}.
621 \]</span> This model can be estimated in R by dropping the intercept
622 term:</p>
623 <div class="sourceCode" id="cb23"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb23-1"><a href="#cb23-1" tabindex="-1"></a>lm_sep <span class="ot">&lt;-</span> <span class="fu">lm</span>(math1 <span class="sc">~</span> <span class="dv">0</span> <span class="sc">+</span> stark, <span class="at">data =</span> STAR)</span>
624 <span id="cb23-2"><a href="#cb23-2" tabindex="-1"></a>V_sep <span class="ot">&lt;-</span> <span class="fu">vcovCR</span>(lm_sep, <span class="at">cluster =</span> STAR<span class="sc">$</span>schoolidk, <span class="at">type =</span> <span class="st">&quot;CR2&quot;</span>)</span>
625 <span id="cb23-3"><a href="#cb23-3" tabindex="-1"></a><span class="fu">coef_test</span>(lm_sep, <span class="at">vcov =</span> V_sep)</span></code></pre></div>
626 <pre><code>## Alternative hypothesis: two-sided
627 ## Coef. Estimate SE Null value t-stat d.f. (Satt) p-val (Satt) Sig.
628 ## starkregular 532 2.78 0 192 59.9 &lt;0.001 ***
629 ## starksmall 541 2.89 0 187 65.0 &lt;0.001 ***
630 ## starkaide 531 2.72 0 195 64.3 &lt;0.001 ***</code></pre>
631 <p>In this parameterization, the coefficients <span class="math inline">\(\beta_0\)</span>, <span class="math inline">\(\beta_1\)</span>, and <span class="math inline">\(\beta_2\)</span> represent the average math
632 performance levels of students in each of the treatment conditions. The
633 t-tests and p-values now have a very different interpretation because
634 they pertain to the null hypothesis that the average performance level
635 for a given condition is equal to zero. With this separate-intercepts
636 model, the joint null hypothesis that performance levels are equal
637 across conditions amounts to constraining the intercepts to be equal to
638 each other: <span class="math inline">\(\beta_0 = \beta_1\)</span> and
639 <span class="math inline">\(\beta_0 = \beta_2\)</span> (note that we
640 don’t need the constraint <span class="math inline">\(\beta_1 =
641 \beta_2\)</span> because it is implied by the first two).</p>
642 <p>For the <code>lm_sep</code> model, which has separate intercepts
643 <span class="math inline">\(\beta_0\)</span>, <span class="math inline">\(\beta_1\)</span>, and <span class="math inline">\(\beta_2\)</span>, the C matrix would look like
644 this: <span class="math display">\[
645 \mathbf{C} = \left[\begin{array}{ccc} -1 &amp; 1 &amp; 0 \\ -1 &amp; 0
646 &amp; 1 \end{array} \right],
647 \]</span> so that <span class="math display">\[
648 \mathbf{C}\boldsymbol\beta = \left[\begin{array}{ccc} -1 &amp; 1 &amp; 0
649 \\ -1 &amp; 0 &amp; 1 \end{array} \right] \left[\begin{array}{c} \beta_0
650 \\ \beta_1 \\ \beta_2 \end{array} \right] = \left[\begin{array}{c}
651 \beta_1 - \beta_0 \\ \beta_2 - \beta_0 \end{array} \right],
652 \]</span> which is set equal to <span class="math inline">\(\left[\begin{array}{c} 0 \\ 0 \end{array}
653 \right]\)</span>.</p>
654 <p>The <code>constrain_equal()</code> function will create matrices like
655 this automatically, given a set of coefficients to constrain. The syntax
656 is identical to <code>constrain_zero()</code>:</p>
657 <div class="sourceCode" id="cb25"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb25-1"><a href="#cb25-1" tabindex="-1"></a><span class="fu">args</span>(constrain_equal)</span></code></pre></div>
658 <pre><code>## function (constraints, coefs, reg_ex = FALSE)
659 ## NULL</code></pre>
660 <p>To test the joint null hypothesis that average math performance is
661 equal across the three treatment conditions, we can constrain all three
662 coefficients of <code>lm_sep</code> to be equal:</p>
663 <div class="sourceCode" id="cb27"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb27-1"><a href="#cb27-1" tabindex="-1"></a><span class="fu">constrain_equal</span>(<span class="dv">1</span><span class="sc">:</span><span class="dv">3</span>, <span class="at">coefs =</span> <span class="fu">coef</span>(lm_sep))</span></code></pre></div>
664 <pre><code>## [,1] [,2] [,3]
665 ## [1,] -1 1 0
666 ## [2,] -1 0 1</code></pre>
667 <p>Or equivalently:</p>
668 <div class="sourceCode" id="cb29"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb29-1"><a href="#cb29-1" tabindex="-1"></a><span class="fu">constrain_equal</span>(<span class="fu">c</span>(<span class="st">&quot;starkregular&quot;</span>,<span class="st">&quot;starksmall&quot;</span>,<span class="st">&quot;starkaide&quot;</span>), <span class="at">coefs =</span> <span class="fu">coef</span>(lm_sep))</span></code></pre></div>
669 <pre><code>## [,1] [,2] [,3]
670 ## [1,] -1 1 0
671 ## [2,] -1 0 1</code></pre>
672 <p>or</p>
673 <div class="sourceCode" id="cb31"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb31-1"><a href="#cb31-1" tabindex="-1"></a><span class="fu">constrain_equal</span>(<span class="st">&quot;^stark&quot;</span>, <span class="at">coefs =</span> <span class="fu">coef</span>(lm_sep), <span class="at">reg_ex =</span> <span class="cn">TRUE</span>)</span></code></pre></div>
674 <pre><code>## [,1] [,2] [,3]
675 ## [1,] -1 1 0
676 ## [2,] -1 0 1</code></pre>
677 <p>Just as with <code>constrain_zero</code>, if <code>constraints</code>
678 is a regular expression, then the <code>reg_ex</code> argument needs to
679 be set to <code>TRUE</code>.</p>
680 <p>This constraint matrix can then be fed into
681 <code>Wald_test()</code>:</p>
682 <div class="sourceCode" id="cb33"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb33-1"><a href="#cb33-1" tabindex="-1"></a>C_sep <span class="ot">&lt;-</span> <span class="fu">constrain_equal</span>(<span class="st">&quot;^stark&quot;</span>, <span class="at">coefs =</span> <span class="fu">coef</span>(lm_sep), <span class="at">reg_ex =</span> <span class="cn">TRUE</span>)</span>
683 <span id="cb33-2"><a href="#cb33-2" tabindex="-1"></a><span class="fu">Wald_test</span>(lm_sep, <span class="at">constraints =</span> C_sep, <span class="at">vcov =</span> V_sep)</span></code></pre></div>
684 <pre><code>## test Fstat df_num df_denom p_val sig
685 ## HTZ 10.2 2 65.3 &lt;0.001 ***</code></pre>
686 <p>or equivalently:</p>
687 <div class="sourceCode" id="cb35"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb35-1"><a href="#cb35-1" tabindex="-1"></a><span class="fu">Wald_test</span>(lm_sep, <span class="at">constraints =</span> <span class="fu">constrain_equal</span>(<span class="dv">1</span><span class="sc">:</span><span class="dv">3</span>), <span class="at">vcov =</span> V_sep)</span></code></pre></div>
688 <pre><code>## test Fstat df_num df_denom p_val sig
689 ## HTZ 10.2 2 65.3 &lt;0.001 ***</code></pre>
690 <p>Note that these test results are exactly equal to the tests based on
691 <code>lm_trt</code> with <code>constrain_zero()</code>. They’re
692 algebraically equivalent—just different ways of parameterizing the same
693 model and constraints.</p>
694 </div>
695 </div>
696 <div id="testing-an-interaction" class="section level1" number="2">
697 <h1><span class="header-section-number">2</span> Testing an
698 interaction</h1>
699 <p>Let’s now consider how these functions can be applied in a more
700 complex model. Suppose that we are interested in understanding whether
701 the effect of being in a small class is consistent across schools in
702 different areas, where areas are categorized as urban, suburban, or
703 rural. To answer this question, we need to test for an interaction
704 between urbanicity and treatment condition. One estimating equation that
705 would let us examine this question is: <span class="math display">\[
706 \begin{aligned}
707 \left(\text{Math}\right)_{ij} &amp;= \beta_0 + \beta_1
708 \left(\text{suburban}\right)_{ij} + \beta_2
709 \left(\text{rural}\right)_{ij} \\
710 &amp; \quad + \beta_3 \left(\text{small}\right)_{ij} + \beta_4
711 \left(\text{aide}\right)_{ij} \\
712 &amp; \quad\quad + \beta_5
713 \left(\text{small}\right)(\text{suburban})_{ij} + \beta_6
714 \left(\text{aide}\right)(\text{suburban})_{ij} \\
715 &amp; \quad\quad\quad + \beta_{7}
716 \left(\text{small}\right)(\text{rural})_{ij} + \beta_{8}
717 \left(\text{aide}\right)(\text{rural})_{ij} \\
718 &amp; \quad\quad\quad\quad + \mathbf{x}_{ij} \boldsymbol\gamma +
719 e_{ij},
720 \end{aligned}
721 \]</span> where <span class="math inline">\(\mathbf{x}_{ij}\)</span> is
722 a row vector of student characteristics, included just to make the
723 regression look fancier. In this specification, <span class="math inline">\(\beta_3\)</span> and <span class="math inline">\(\beta_4\)</span> represent the effects of being in
724 a small class or aide class, compared to being in a regular class, but
725 <em>only for the reference level of urbanicity</em>—in this case, urban
726 schools. The coefficients <span class="math inline">\(\beta_5, \beta_6,
727 \beta_7, \beta_8\)</span> all represent <em>interactions</em> between
728 treatment condition and urbanicity. Here’s the model, estimated in
729 R:</p>
730 <div class="sourceCode" id="cb37"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb37-1"><a href="#cb37-1" tabindex="-1"></a>lm_urbanicity <span class="ot">&lt;-</span> <span class="fu">lm</span>(math1 <span class="sc">~</span> schoolk <span class="sc">*</span> stark <span class="sc">+</span> gender <span class="sc">+</span> ethnicity <span class="sc">+</span> lunchk, <span class="at">data =</span> STAR)</span>
731 <span id="cb37-2"><a href="#cb37-2" tabindex="-1"></a>V_urbanicity <span class="ot">&lt;-</span> <span class="fu">vcovCR</span>(lm_urbanicity, <span class="at">cluster =</span> STAR<span class="sc">$</span>schoolidk, <span class="at">type =</span> <span class="st">&quot;CR2&quot;</span>)</span>
732 <span id="cb37-3"><a href="#cb37-3" tabindex="-1"></a><span class="fu">coef_test</span>(lm_urbanicity, <span class="at">vcov =</span> V_urbanicity)</span></code></pre></div>
733 <pre><code>## Alternative hypothesis: two-sided
734 ## Coef. Estimate SE Null value t-stat d.f. (Satt)
735 ## (Intercept) 542.62 5.91 0 91.8599 21.70
736 ## schoolksuburban 2.77 6.76 0 0.4100 28.35
737 ## schoolkrural 1.03 6.38 0 0.1616 30.74
738 ## starksmall 9.42 4.56 0 2.0649 17.10
739 ## starkaide -4.27 2.17 0 -1.9631 16.73
740 ## genderfemale 2.14 1.20 0 1.7773 67.14
741 ## ethnicityafam -16.79 4.19 0 -4.0026 34.94
742 ## ethnicityasian 13.19 11.02 0 1.1963 6.23
743 ## ethnicityhispanic 39.23 20.62 0 1.9028 1.01
744 ## ethnicityother 8.86 18.78 0 0.4720 3.02
745 ## lunchkfree -19.37 2.04 0 -9.4848 57.38
746 ## schoolksuburban:starksmall 3.03 6.39 0 0.4746 28.94
747 ## schoolkrural:starksmall -0.31 5.58 0 -0.0555 34.04
748 ## schoolksuburban:starkaide 5.10 3.72 0 1.3711 28.64
749 ## schoolkrural:starkaide 8.16 3.16 0 2.5857 34.30
750 ## p-val (Satt) Sig.
751 ## &lt;0.001 ***
752 ## 0.6849
753 ## 0.8727
754 ## 0.0544 .
755 ## 0.0665 .
756 ## 0.0800 .
757 ## &lt;0.001 ***
758 ## 0.2751
759 ## 0.3067
760 ## 0.6690
761 ## &lt;0.001 ***
762 ## 0.6386
763 ## 0.9560
764 ## 0.1810
765 ## 0.0141 *</code></pre>
766 <p>With this specification, there are several different null hypotheses
767 that we might want to test. For one, perhaps we want to see if there is
768 <em>any</em> variation in treatment effects across different levels of
769 urbanicity. This can be expressed in the null hypothesis that all four
770 interaction terms are zero, or <span class="math inline">\(H_{0A}:
771 \beta_5 = \beta_6 = \beta_7 = \beta_8 = 0\)</span>. With Wald test:</p>
772 <div class="sourceCode" id="cb39"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb39-1"><a href="#cb39-1" tabindex="-1"></a><span class="fu">Wald_test</span>(lm_urbanicity, </span>
773 <span id="cb39-2"><a href="#cb39-2" tabindex="-1"></a> <span class="at">constraints =</span> <span class="fu">constrain_zero</span>(<span class="st">&quot;schoolk.+:stark&quot;</span>, <span class="at">reg_ex =</span> <span class="cn">TRUE</span>),</span>
774 <span id="cb39-3"><a href="#cb39-3" tabindex="-1"></a> <span class="at">vcov =</span> V_urbanicity)</span></code></pre></div>
775 <pre><code>## test Fstat df_num df_denom p_val sig
776 ## HTZ 1.96 4 37.5 0.121</code></pre>
777 <p>Another possibility is that we might want to focus on variation in
778 the effect of being in a small class or regular class, while ignoring
779 whatever is going on in the aide class condition. Here, the null
780 hypothesis would be simply <span class="math inline">\(H_{0B}: \beta_5 =
781 \beta_6 = 0\)</span>, tested as:</p>
782 <div class="sourceCode" id="cb41"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb41-1"><a href="#cb41-1" tabindex="-1"></a><span class="fu">Wald_test</span>(lm_urbanicity, </span>
783 <span id="cb41-2"><a href="#cb41-2" tabindex="-1"></a> <span class="at">constraints =</span> <span class="fu">constrain_zero</span>(<span class="st">&quot;schoolk.+:starksmall&quot;</span>, <span class="at">reg_ex =</span> <span class="cn">TRUE</span>),</span>
784 <span id="cb41-3"><a href="#cb41-3" tabindex="-1"></a> <span class="at">vcov =</span> V_urbanicity)</span></code></pre></div>
785 <pre><code>## test Fstat df_num df_denom p_val sig
786 ## HTZ 0.189 2 34.5 0.828</code></pre>
787 <div id="lists-of-constraints" class="section level2" number="2.1">
788 <h2><span class="header-section-number">2.1</span> Lists of
789 constraints</h2>
790 <p>In models like the urbanicity-by-treatment interaction specification,
791 we may need to run multiple tests on the same estimating equation. This
792 can be accomplished with <code>Wald_test</code> by providing a
793 <em>list</em> of constraints to the <code>constraints</code> argument.
794 For example, we could test the hypotheses described above by creating a
795 list of several constraint matrices and then passing it to
796 <code>Wald_test</code>:</p>
797 <div class="sourceCode" id="cb43"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb43-1"><a href="#cb43-1" tabindex="-1"></a>C_list <span class="ot">&lt;-</span> <span class="fu">list</span>(</span>
798 <span id="cb43-2"><a href="#cb43-2" tabindex="-1"></a> <span class="st">`</span><span class="at">Any interaction</span><span class="st">`</span> <span class="ot">=</span> <span class="fu">constrain_zero</span>(<span class="st">&quot;schoolk.+:stark&quot;</span>, </span>
799 <span id="cb43-3"><a href="#cb43-3" tabindex="-1"></a> <span class="fu">coef</span>(lm_urbanicity), <span class="at">reg_ex =</span> <span class="cn">TRUE</span>),</span>
800 <span id="cb43-4"><a href="#cb43-4" tabindex="-1"></a> <span class="st">`</span><span class="at">Small vs regular</span><span class="st">`</span> <span class="ot">=</span> <span class="fu">constrain_zero</span>(<span class="st">&quot;schoolk.+:starksmall&quot;</span>, </span>
801 <span id="cb43-5"><a href="#cb43-5" tabindex="-1"></a> <span class="fu">coef</span>(lm_urbanicity), <span class="at">reg_ex =</span> <span class="cn">TRUE</span>)</span>
802 <span id="cb43-6"><a href="#cb43-6" tabindex="-1"></a>)</span>
803 <span id="cb43-7"><a href="#cb43-7" tabindex="-1"></a></span>
804 <span id="cb43-8"><a href="#cb43-8" tabindex="-1"></a><span class="fu">Wald_test</span>(lm_urbanicity, </span>
805 <span id="cb43-9"><a href="#cb43-9" tabindex="-1"></a> <span class="at">constraints =</span> C_list,</span>
806 <span id="cb43-10"><a href="#cb43-10" tabindex="-1"></a> <span class="at">vcov =</span> V_urbanicity)</span></code></pre></div>
807 <pre><code>## $`Any interaction`
808 ## test Fstat df_num df_denom p_val sig
809 ## HTZ 1.96 4 37.5 0.121
810 ##
811 ## $`Small vs regular`
812 ## test Fstat df_num df_denom p_val sig
813 ## HTZ 0.189 2 34.5 0.828</code></pre>
814 <p>Setting the option <code>tidy = TRUE</code> will arrange the output
815 of all the tests into a single data frame:</p>
816 <div class="sourceCode" id="cb45"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb45-1"><a href="#cb45-1" tabindex="-1"></a><span class="fu">Wald_test</span>(lm_urbanicity, </span>
817 <span id="cb45-2"><a href="#cb45-2" tabindex="-1"></a> <span class="at">constraints =</span> C_list,</span>
818 <span id="cb45-3"><a href="#cb45-3" tabindex="-1"></a> <span class="at">vcov =</span> V_urbanicity, </span>
819 <span id="cb45-4"><a href="#cb45-4" tabindex="-1"></a> <span class="at">tidy =</span> <span class="cn">TRUE</span>)</span></code></pre></div>
820 <pre><code>## hypothesis test Fstat df_num df_denom p_val sig
821 ## Any interaction HTZ 1.960 4 37.5 0.121
822 ## Small vs regular HTZ 0.189 2 34.5 0.828</code></pre>
823 <p>The list of constraints can also be created inside
824 <code>Wald_test</code>, so that the <code>coefs</code> argument can be
825 omitted from <code>constrain_zero()</code>:</p>
826 <div class="sourceCode" id="cb47"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb47-1"><a href="#cb47-1" tabindex="-1"></a><span class="fu">Wald_test</span>(</span>
827 <span id="cb47-2"><a href="#cb47-2" tabindex="-1"></a> lm_urbanicity, </span>
828 <span id="cb47-3"><a href="#cb47-3" tabindex="-1"></a> <span class="at">constraints =</span> <span class="fu">list</span>(</span>
829 <span id="cb47-4"><a href="#cb47-4" tabindex="-1"></a> <span class="st">`</span><span class="at">Any interaction</span><span class="st">`</span> <span class="ot">=</span> <span class="fu">constrain_zero</span>(<span class="st">&quot;schoolk.+:stark&quot;</span>, <span class="at">reg_ex =</span> <span class="cn">TRUE</span>),</span>
830 <span id="cb47-5"><a href="#cb47-5" tabindex="-1"></a> <span class="st">`</span><span class="at">Small vs regular</span><span class="st">`</span> <span class="ot">=</span> <span class="fu">constrain_zero</span>(<span class="st">&quot;schoolk.+:starksmall&quot;</span>, <span class="at">reg_ex =</span> <span class="cn">TRUE</span>)</span>
831 <span id="cb47-6"><a href="#cb47-6" tabindex="-1"></a> ),</span>
832 <span id="cb47-7"><a href="#cb47-7" tabindex="-1"></a> <span class="at">vcov =</span> V_urbanicity, </span>
833 <span id="cb47-8"><a href="#cb47-8" tabindex="-1"></a> <span class="at">tidy =</span> <span class="cn">TRUE</span></span>
834 <span id="cb47-9"><a href="#cb47-9" tabindex="-1"></a>)</span></code></pre></div>
835 <pre><code>## hypothesis test Fstat df_num df_denom p_val sig
836 ## Any interaction HTZ 1.960 4 37.5 0.121
837 ## Small vs regular HTZ 0.189 2 34.5 0.828</code></pre>
838 </div>
839 </div>
840 <div id="pairwise-t-tests" class="section level1" number="3">
841 <h1><span class="header-section-number">3</span> Pairwise t-tests</h1>
842 <p>The <code>clubSandwich</code> package also provides a further
843 convenience function, <code>constrain_pairwise()</code> that can be used
844 in combination with <code>Wald_test()</code> to conduct pairwise
845 comparisons among a set of regression coefficients. This function
846 differs from the other two <code>constrain_*()</code> functions because
847 it returns a <em>list</em> of constraint matrices, each of which
848 corresponds to a single linear combination of covariates. Specifically,
849 the <code>constrain_pairwise()</code> function provides a list of
850 constraints that represent the differences between every possible pair
851 among a specified set of coefficients. The syntax is very similar to the
852 other <code>constrain_*()</code> functions.</p>
853 <p>To demonstrate, consider the separate-intercepts specification of the
854 simpler regression model:</p>
855 <div class="sourceCode" id="cb49"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb49-1"><a href="#cb49-1" tabindex="-1"></a><span class="fu">coef_test</span>(lm_sep, <span class="at">vcov =</span> V_sep)</span></code></pre></div>
856 <pre><code>## Alternative hypothesis: two-sided
857 ## Coef. Estimate SE Null value t-stat d.f. (Satt) p-val (Satt) Sig.
858 ## starkregular 532 2.78 0 192 59.9 &lt;0.001 ***
859 ## starksmall 541 2.89 0 187 65.0 &lt;0.001 ***
860 ## starkaide 531 2.72 0 195 64.3 &lt;0.001 ***</code></pre>
861 <p>This specification is nice because it lets us simply read off the
862 average outcomes for each group. However, we will naturally also want to
863 know about whether there are differences between the groups, so we’ll
864 want to compare the small-class condition to the regular-size class
865 condition, the aide condition to the regular-size class condition, and
866 the small-class condition to the aide condition. Thus, we’ll want
867 comparisons among all three coefficients:</p>
868 <div class="sourceCode" id="cb51"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb51-1"><a href="#cb51-1" tabindex="-1"></a>C_pairs <span class="ot">&lt;-</span> <span class="fu">constrain_pairwise</span>(<span class="dv">1</span><span class="sc">:</span><span class="dv">3</span>, <span class="at">coefs =</span> <span class="fu">coef</span>(lm_sep))</span>
869 <span id="cb51-2"><a href="#cb51-2" tabindex="-1"></a>C_pairs</span></code></pre></div>
870 <pre><code>## $`starksmall - starkregular`
871 ## [,1] [,2] [,3]
872 ## [1,] -1 1 0
873 ##
874 ## $`starkaide - starkregular`
875 ## [,1] [,2] [,3]
876 ## [1,] -1 0 1
877 ##
878 ## $`starkaide - starksmall`
879 ## [,1] [,2] [,3]
880 ## [1,] 0 -1 1</code></pre>
881 <p>Feeding these constraints into <code>Wald_test()</code> gives us
882 significance tests for each pair:</p>
883 <div class="sourceCode" id="cb53"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb53-1"><a href="#cb53-1" tabindex="-1"></a><span class="fu">Wald_test</span>(lm_sep, <span class="at">constraints =</span> C_pairs, <span class="at">vcov =</span> V_sep, <span class="at">tidy =</span> <span class="cn">TRUE</span>)</span></code></pre></div>
884 <pre><code>## hypothesis test Fstat df_num df_denom p_val sig
885 ## starksmall - starkregular HTZ 16.9238 1 65.6 &lt;0.001 ***
886 ## starkaide - starkregular HTZ 0.0673 1 65.6 0.796
887 ## starkaide - starksmall HTZ 17.8137 1 66.9 &lt;0.001 ***</code></pre>
888 <p>The first two of these tests are equivalent to the tests of the
889 treatment effect coefficients in the other parameterization of the
890 model. Indeed, the denominator degrees of freedom are identical to the
891 results of <code>coef_test(lm_trt, vcov = V_trt)</code>; the
892 <code>Fstat</code>s here are equal to the squared t-statistics from the
893 first model:</p>
894 <div class="sourceCode" id="cb55"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb55-1"><a href="#cb55-1" tabindex="-1"></a>t_stats <span class="ot">&lt;-</span> <span class="fu">coef_test</span>(lm_trt, <span class="at">vcov =</span> V_trt)<span class="sc">$</span>tstat[<span class="dv">2</span><span class="sc">:</span><span class="dv">3</span>]</span>
895 <span id="cb55-2"><a href="#cb55-2" tabindex="-1"></a>F_stats <span class="ot">&lt;-</span> <span class="fu">Wald_test</span>(lm_sep, <span class="at">constraints =</span> C_pairs, <span class="at">vcov =</span> V_sep, <span class="at">tidy =</span> <span class="cn">TRUE</span>)<span class="sc">$</span>Fstat[<span class="dv">1</span><span class="sc">:</span><span class="dv">2</span>]</span>
896 <span id="cb55-3"><a href="#cb55-3" tabindex="-1"></a><span class="fu">all.equal</span>(t_stats<span class="sc">^</span><span class="dv">2</span>, F_stats)</span></code></pre></div>
897 <pre><code>## [1] TRUE</code></pre>
898 <p>It is important to note that the p-values from the pairwise
899 comparisons are <em>not</em> corrected for multiplicity.<a href="#fn3" class="footnote-ref" id="fnref3"><sup>3</sup></a> For now, please
900 correct-your-own using <code>p.adjust()</code> or your preferred
901 method.</p>
902 <p>Pairwise comparisons might also be of use in the model with
903 treatment-by-urbanicity interactions. Here’s the model results
904 again:</p>
905 <div class="sourceCode" id="cb57"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb57-1"><a href="#cb57-1" tabindex="-1"></a><span class="fu">coef_test</span>(lm_urbanicity, <span class="at">vcov =</span> V_urbanicity)</span></code></pre></div>
906 <pre><code>## Alternative hypothesis: two-sided
907 ## Coef. Estimate SE Null value t-stat d.f. (Satt)
908 ## (Intercept) 542.62 5.91 0 91.8599 21.70
909 ## schoolksuburban 2.77 6.76 0 0.4100 28.35
910 ## schoolkrural 1.03 6.38 0 0.1616 30.74
911 ## starksmall 9.42 4.56 0 2.0649 17.10
912 ## starkaide -4.27 2.17 0 -1.9631 16.73
913 ## genderfemale 2.14 1.20 0 1.7773 67.14
914 ## ethnicityafam -16.79 4.19 0 -4.0026 34.94
915 ## ethnicityasian 13.19 11.02 0 1.1963 6.23
916 ## ethnicityhispanic 39.23 20.62 0 1.9028 1.01
917 ## ethnicityother 8.86 18.78 0 0.4720 3.02
918 ## lunchkfree -19.37 2.04 0 -9.4848 57.38
919 ## schoolksuburban:starksmall 3.03 6.39 0 0.4746 28.94
920 ## schoolkrural:starksmall -0.31 5.58 0 -0.0555 34.04
921 ## schoolksuburban:starkaide 5.10 3.72 0 1.3711 28.64
922 ## schoolkrural:starkaide 8.16 3.16 0 2.5857 34.30
923 ## p-val (Satt) Sig.
924 ## &lt;0.001 ***
925 ## 0.6849
926 ## 0.8727
927 ## 0.0544 .
928 ## 0.0665 .
929 ## 0.0800 .
930 ## &lt;0.001 ***
931 ## 0.2751
932 ## 0.3067
933 ## 0.6690
934 ## &lt;0.001 ***
935 ## 0.6386
936 ## 0.9560
937 ## 0.1810
938 ## 0.0141 *</code></pre>
939 <p>Suppose that we are interested in the effect of small versus regular
940 size classes, and in particular whether this effect varies across
941 schools in different areas. The coefficients on
942 <code>schoolksuburban:starksmall</code> and
943 <code>schoolkrural:starksmall</code> already give us the differences in
944 treatment effects between suburban schools versus urban schools and
945 between rural schools versus urban schools. The difference between these
946 coefficients gives us the difference in treatment effects between
947 suburban schools and rural schools. We can look at all three of these
948 contrasts using <code>constrain_pairwise()</code> by setting the option
949 <code>with_zero = TRUE</code>:</p>
950 <div class="sourceCode" id="cb59"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb59-1"><a href="#cb59-1" tabindex="-1"></a><span class="fu">Wald_test</span>(lm_urbanicity, </span>
951 <span id="cb59-2"><a href="#cb59-2" tabindex="-1"></a> <span class="at">constraints =</span> <span class="fu">constrain_pairwise</span>(<span class="st">&quot;:starksmall&quot;</span>, <span class="at">reg_ex =</span> <span class="cn">TRUE</span>, <span class="at">with_zero =</span> <span class="cn">TRUE</span>),</span>
952 <span id="cb59-3"><a href="#cb59-3" tabindex="-1"></a> <span class="at">vcov =</span> V_urbanicity,</span>
953 <span id="cb59-4"><a href="#cb59-4" tabindex="-1"></a> <span class="at">tidy =</span> <span class="cn">TRUE</span>)</span></code></pre></div>
954 <pre><code>## hypothesis test Fstat df_num
955 ## schoolksuburban:starksmall HTZ 0.22526 1
956 ## schoolkrural:starksmall HTZ 0.00308 1
957 ## schoolkrural:starksmall - schoolksuburban:starksmall HTZ 0.36471 1
958 ## df_denom p_val sig
959 ## 28.9 0.639
960 ## 34.0 0.956
961 ## 24.4 0.551</code></pre>
962 <p>Again, the results of the first two tests are identical to the
963 t-tests reported in <code>coef_test()</code>.</p>
964 </div>
965 <div id="remark" class="section level1" number="4">
966 <h1><span class="header-section-number">4</span> Remark</h1>
967 <p>All of the preceding examples were based on ordinary linear
968 regression models with clustered standard errors. However,
969 <code>Wald_test()</code> and its helper functions all work identically
970 for all of the other models with supporting <code>clubSandwich</code>
971 methods, including <code>nlme::lme()</code>, <code>nlme::gls()</code>,
972 <code>lme4::lmer()</code>, <code>rma.uni()</code>,
973 <code>rma.mv()</code>, and <code>robu()</code>, among others.</p>
974 </div>
975 <div id="references" class="section level1 unnumbered">
976 <h1 class="unnumbered">References</h1>
977 <div id="refs" class="references csl-bib-body hanging-indent" entry-spacing="0" line-spacing="2">
978 <div id="ref-pustejovsky2018small" class="csl-entry">
979 Pustejovsky, J. E., &amp; Tipton, E. (2018). Small-<span>Sample</span>
980 <span>Methods</span> for <span>Cluster</span>-<span>Robust</span>
981 <span>Variance</span> <span>Estimation</span> and
982 <span>Hypothesis</span> <span>Testing</span> in <span>Fixed</span>
983 <span>Effects</span> <span>Models</span>. <em>Journal of Business &amp;
984 Economic Statistics</em>, <em>36</em>(4), 672–683. <a href="https://doi.org/10.1080/07350015.2016.1247004">https://doi.org/10.1080/07350015.2016.1247004</a>
985 </div>
986 <div id="ref-tipton2015small" class="csl-entry">
987 Tipton, E., &amp; Pustejovsky, J. E. (2015). Small-sample adjustments
988 for tests of moderators and model fit using robust variance estimation
989 in meta-regression. <em>Journal of Educational and Behavioral
990 Statistics</em>, <em>40</em>(6), 604–634. <a href="https://doi.org/10.3102/1076998615606099">https://doi.org/10.3102/1076998615606099</a>
991 </div>
992 </div>
993 </div>
994 <div class="footnotes footnotes-end-of-document">
995 <hr />
996 <ol>
997 <li id="fn1"><p>In <span class="citation">Pustejovsky &amp; Tipton
998 (2018)</span> we used a more general formulation of multiple-constraint
999 null hypotheses, expressed as <span class="math inline">\(H_0:
1000 \mathbf{C} \boldsymbol\beta = \mathbf{d}\)</span> for some fixed <span class="math inline">\(q \times 1\)</span> vector <span class="math inline">\(\mathbf{d}\)</span>. In practice, it’s often
1001 possible to modify the <span class="math inline">\(\mathbf{C}\)</span>
1002 matrix so that <span class="math inline">\(\mathbf{d}\)</span> can
1003 always be set to <span class="math inline">\(\mathbf{0}\)</span>.<a href="#fnref1" class="footnote-back">↩︎</a></p></li>
1004 <li id="fn2"><p>How does this work? If we omit the <code>coefs</code>
1005 argument, <code>constrain_zero()</code> acts as a functional, by
1006 returning a function equivalent to
1007 <code>function(coefs) constrain_zero(constraints, coefs = coefs)</code>.
1008 If this function is fed into the <code>constraints</code> argument of
1009 <code>Wald_test()</code>, <code>Wald_test()</code> recognizes that it is
1010 a function and evaluates the function with <code>coef(obj)</code>. It’s
1011 a kinda-sorta hacky substitute for lazy evaluation. If you have
1012 suggestions for how to do this more elegantly, please send them my
1013 way.<a href="#fnref2" class="footnote-back">↩︎</a></p></li>
1014 <li id="fn3"><p>Options to include multiplicity corrections (Bonferroni,
1015 Holm, Benjamini-Hochberg, etc.) might be included in a <a href="https://github.com/jepusto/clubSandwich/issues/33">future
1016 release</a>. Reach out if this is of interest to you.<a href="#fnref3" class="footnote-back">↩︎</a></p></li>
1017 </ol>
1018 </div>
1019
1020
1021
1022 <!-- code folding -->
1023
1024
1025 <!-- dynamically load mathjax for compatibility with self-contained -->
1026 <script>
1027 (function () {
1028 var script = document.createElement("script");
1029 script.type = "text/javascript";
1030 script.src = "https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML";
1031 document.getElementsByTagName("head")[0].appendChild(script);
1032 })();
1033 </script>
1034
1035 </body>
1036 </html>