Mixed designs

class: center, middle, title-slide, inverse, no-scribble
layout: false

# Mixed designs and the GLM
## Professor Andy Field

<div>
  <img style="vertical-align:middle; width:30px; height:30px" src="media/twitter_60.png">
  <span style="line-height:40px;">@profandyfield</span>
</div>
<div>
  <img style="vertical-align:middle; width:60px" src="media/youtube.png">
  <span style="line-height:40px;">www.youtube.com/user/ProfAndyField/</span>
</div>
<div>
  <img style="vertical-align:middle; width:30px; height:30px" src="media/ds_com_fav.png">
  <span style="line-height:40px;">www.discoveringstatistics.com</span>
</div>
<div>
  <img style="vertical-align:middle; width:30px; height:30px" src="media/milton_grey_fav.png">
  <span style="line-height:40px;">www.milton-the-cat.rocks</span>
</div>
<div>
  <img style="vertical-align:middle; width:30px; height:30px" src="media/discovr_fav.png">
  <span style="line-height:40px;">www.discovr.rocks</span>
</div>

???
Music: Deafheaven Irresistable

h or ?: Toggle the help window
j: Jump to next slide
k: Jump to previous slide
b: Toggle blackout mode
m: Toggle mirrored mode.
p: Toggle PresenterMode
f: Toggle Fullscreen
t: Reset presentation timer
<number> + <Return>: Jump to slide <number>
c: Create a clone presentation on a new window

---

class: center

![:scale 83%](media/spine_content_map.png)
???

We've seen this map of the process of fitting models before

---
class: center

![:scale 83%](media/spine_lec_04.png)

???

Today we focus back on the model itself to look at the form of the model we're fitting. The faded stuff still applies though - we'll look at bias, robust models, and of course samples and estimation and so on. But they are the same as for other models, what is different is the form of the model we're fitting.

---
#  Learning outcomes

* Describe what is meant by a mixed design

* Apply a GLM to a mixed designs using `afex`

* Interpret a GLM for a mixed design (up to three factors)

* Making sense of three-way interactions

---

#  A speed-dating example

* Dating strategies

+ If someone is committed to pursuing a relationship with a person who plays hard to get, they will find that person more desirable but less likeable (Dai, Dong, and Jia , 2014)
  + Field (2017) fictitious study to look at the interplay between looks, personality and dating strategies on evaluations of a date

* Design

+ Predictor 1 (**looks**): above average (high), average, below average (low)
  + Predictor 2 (**personality**): high charisma, average charisma, low charisma
  + Predictor 3 (**strategy**): acting normal or playing hard to get

* Outcome

+ P’s percentage desire for a full **date** (0-100%)

---
class: center

![:scale 100%](media/speed_date_design_a.png)

---
class: center

![:scale 100%](media/speed_date_design_b.png)
---
class: center

![:scale 100%](media/speed_date_design_c.png)
---
background-image: none

# The data in <svg style="height: 1em; top:.04em; position: relative; fill: #136CB9;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg>

<div class="datatables html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-6e920e777723c065527d" style="width:100%;height:auto;"></div>
<script type="application/json" data-for="htmlwidget-6e920e777723c065527d">{"x":{"filter":"none","vertical":false,"caption":"<caption>Table 1: Data for the speed dating example<\/caption>","data":[["1","2","3","4","5","6","7","8","9","10","11","12","13","14","15","16","17","18","19","20","21","22","23","24","25","26","27","28","29","30","31","32","33","34","35","36","37","38","39","40","41","42","43","44","45","46","47","48","49","50","51","52","53","54","55","56","57","58","59","60","61","62","63","64","65","66","67","68","69","70","71","72","73","74","75","76","77","78","79","80","81","82","83","84","85","86","87","88","89","90","91","92","93","94","95","96","97","98","99","100","101","102","103","104","105","106","107","108","109","110","111","112","113","114","115","116","117","118","119","120","121","122","123","124","125","126","127","128","129","130","131","132","133","134","135","136","137","138","139","140","141","142","143","144","145","146","147","148","149","150","151","152","153","154","155","156","157","158","159","160","161","162","163","164","165","166","167","168","169","170","171","172","173","174","175","176","177","178","179","180"],["2rl3f","2rl3f","2rl3f","2rl3f","2rl3f","2rl3f","2rl3f","2rl3f","2rl3f","2h53u","2h53u","2h53u","2h53u","2h53u","2h53u","2h53u","2h53u","2h53u","qa78v","qa78v","qa78v","qa78v","qa78v","qa78v","qa78v","qa78v","qa78v","lvws1","lvws1","lvws1","lvws1","lvws1","lvws1","lvws1","lvws1","lvws1","nti4g","nti4g","nti4g","nti4g","nti4g","nti4g","nti4g","nti4g","nti4g","mkt03","mkt03","mkt03","mkt03","mkt03","mkt03","mkt03","mkt03","mkt03","382re","382re","382re","382re","382re","382re","382re","382re","382re","4mue7","4mue7","4mue7","4mue7","4mue7","4mue7","4mue7","4mue7","4mue7","m487l","m487l","m487l","m487l","m487l","m487l","m487l","m487l","m487l","57v26","57v26","57v26","57v26","57v26","57v26","57v26","57v26","57v26","w363q","w363q","w363q","w363q","w363q","w363q","w363q","w363q","w363q","e5065","e5065","e5065","e5065","e5065","e5065","e5065","e5065","e5065","7kw62","7kw62","7kw62","7kw62","7kw62","7kw62","7kw62","7kw62","7kw62","4rjbx","4rjbx","4rjbx","4rjbx","4rjbx","4rjbx","4rjbx","4rjbx","4rjbx","kym4j","kym4j","kym4j","kym4j","kym4j","kym4j","kym4j","kym4j","kym4j","98h23","98h23","98h23","98h23","98h23","98h23","98h23","98h23","98h23","8xdj8","8xdj8","8xdj8","8xdj8","8xdj8","8xdj8","8xdj8","8xdj8","8xdj8","0l4rt","0l4rt","0l4rt","0l4rt","0l4rt","0l4rt","0l4rt","0l4rt","0l4rt","o8esu","o8esu","o8esu","o8esu","o8esu","o8esu","o8esu","o8esu","o8esu","2dm8c","2dm8c","2dm8c","2dm8c","2dm8c","2dm8c","2dm8c","2dm8c","2dm8c"],["Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Hard to get","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal","Normal"],["High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low","High","Average","Low"],["High","High","High","Average","Average","Average","Low","Low","Low","High","High","High","Average","Average","Average","Low","Low","Low","High","High","High","Average","Average","Average","Low","Low","Low","High","High","High","Average","Average","Average","Low","Low","Low","High","High","High","Average","Average","Average","Low","Low","Low","High","High","High","Average","Average","Average","Low","Low","Low","High","High","High","Average","Average","Average","Low","Low","Low","High","High","High","Average","Average","Average","Low","Low","Low","High","High","High","Average","Average","Average","Low","Low","Low","High","High","High","Average","Average","Average","Low","Low","Low","High","High","High","Average","Average","Average","Low","Low","Low","High","High","High","Average","Average","Average","Low","Low","Low","High","High","High","Average","Average","Average","Low","Low","Low","High","High","High","Average","Average","Average","Low","Low","Low","High","High","High","Average","Average","Average","Low","Low","Low","High","High","High","Average","Average","Average","Low","Low","Low","High","High","High","Average","Average","Average","Low","Low","Low","High","High","High","Average","Average","Average","Low","Low","Low","High","High","High","Average","Average","Average","Low","Low","Low","High","High","High","Average","Average","Average","Low","Low","Low"],[86,84,67,88,69,50,97,48,47,91,83,53,83,74,48,86,50,46,89,88,48,99,70,48,90,45,48,89,69,58,86,77,40,87,47,53,80,81,57,88,71,50,82,50,45,80,84,51,96,63,42,92,48,43,89,85,61,87,79,44,86,50,45,100,94,56,86,71,54,84,54,47,90,74,54,92,71,58,78,38,45,89,86,63,80,73,49,91,48,39,89,91,93,88,65,54,55,48,52,84,90,85,95,70,60,50,44,45,99,100,89,80,79,53,51,48,44,86,89,83,86,74,58,52,48,47,89,87,80,83,74,43,58,50,48,80,81,79,86,59,47,51,47,40,82,92,85,81,66,47,50,45,47,97,69,87,95,72,51,45,48,46,95,92,90,98,64,53,54,53,45,95,93,96,79,66,46,52,39,47]],"container":"<table class=\"display\">\n  <thead>\n    <tr>\n      <th> <\/th>\n      <th>id<\/th>\n      <th>strategy<\/th>\n      <th>looks<\/th>\n      <th>personality<\/th>\n      <th>date<\/th>\n    <\/tr>\n  <\/thead>\n<\/table>","options":{"dom":"tp","columnDefs":[{"className":"dt-center","targets":[1,2,3]},{"className":"dt-right","targets":5},{"orderable":false,"targets":0}],"pageLength":10,"order":[],"autoWidth":false,"orderClasses":false},"selection":{"mode":"multiple","selected":null,"target":"row","selectable":null}},"evals":[],"jsHooks":[]}</script>

---
# Mixed designs and the linear model

* A bare minimum
  + Modelling variation in ratings overall across participants
  + Not modelling variation in the effect of repeated measures predictors across participants
  + Not explicitly listing  the dummy variables for **looks** and **personality** (they'll be transformed into two dummy variables each)

.center[.ong[
$$
`\begin{aligned}
\text{date}_{ij} &= \hat{b}_{0j} + \hat{b}_{1}\text{strategy}_{i} + \hat{b}_{2}\text{looks}_{ij} + \hat{b}_{3}\text{personality}_{ij} + \dots \\
&\dots + \hat{b}_{4}\text{strategy}_{i} \times \text{looks}_{ij} + \hat{b}_{5}\text{strategy}_{i} \times \text{personality}_{ij} + \hat{b}_{6}\text{looks}_{ij} \times \text{personality}_{ij} + \dots \\
&\dots + \hat{b}_{7}\text{strategy}_{i} \times \text{looks}_{ij} \times \text{personality}_{ij} + e_{ij} + \hat{u}_{0j}
\end{aligned}`
$$
]]

---
# Summary of effects

.pull-left[
We will get an *F*-statistic for the following effects:

* Main effects

+ looks
  + personality
  + strategy

* Two-way interactions
  
  + strategy × looks
  + strategy × personality
  + looks × personality

* Three-way Interaction

+ strategy × looks × personality
]

.pull-right[.eq_lrge[.warning[

**"It is never sensible to interpret main effects in the presence of a significant interaction effect."**

**"It is not sensible to interpret interaction effects in the presence of a significant higher-order interaction effect."**

]]]
---
# Fitting the model

.pull-left[
* The `afex::aov_4()` function

+ Specify the repeated measures with `(rm_predictors|id_var)`
  
  + Automatically sets contrasts
  
  + Built in interaction plot with `afex_plot()`
  
  + But ... no parameter estimates, diagnostic plots, or robust methods

```r
date_afx <- afex::aov_4(
* date ~ strategy*looks*personality + (looks*personality|id),
  data = sniff_tib
  )
```

]

.pull-right[ 
* Use `lmer()` from the `lme4` package

+ A trickier option (we don't teach you it)
  
  + Manually set contrasts
  
  + Can get parameter estimates, diagnostic plots, and robust methods
  
  + Comes with free mental anguish
]

---
# Contrasts

* For both **looks** and **personality** there are three categories (high, average, low). These variables need to be coded into two dummy/contrast variables.

* We'd expect the desirability of a full date to be different at the extremes to the 'average'. Therefore, for both predictors these contrasts would work:

+ **Contrast 1**: {high} vs. {average}
  + **Contrast 2**: {low} vs. {average}

* We can extract these using `emmeans::contrast()`

```r
emmeans::contrast(my_emm_object, method = "eff", adjust = "holm")
```

* Use `method = `
  + `"eff"`: each category is compared to the average of all categories (default).
  + `"pairwise"`: each category is compared to all others.
  + `"poly"`: polynomial contrasts (i.e. tests for an ordered trend in category means).
  + `"trt.vs.ctrl"`: compares each category to a declared reference category (by default the first category). Use `ref = x` to make `x` the reference category.
  + `"consec"`: compares each level/category (except the first) to the previous

---
background-image: none
background-color: #000000
class: no-scribble

---
background-image: none

# Overall model summary

```r
afex::aov_4(date ~ strategy*looks*personality + (looks*personality|id), data = date_tib)
```

<table class="table" style="margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;">   </th>
   <th style="text-align:right;"> num Df </th>
   <th style="text-align:right;"> den Df </th>
   <th style="text-align:right;"> MSE </th>
   <th style="text-align:right;"> F </th>
   <th style="text-align:right;"> ges </th>
   <th style="text-align:right;"> Pr(&gt;F) </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> strategy </td>
   <td style="text-align:right;"> 1.00 </td>
   <td style="text-align:right;"> 18.00 </td>
   <td style="text-align:right;"> 42.23 </td>
   <td style="text-align:right;"> 0.00 </td>
   <td style="text-align:right;"> 0.00 </td>
   <td style="text-align:right;"> 0.95 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> looks </td>
   <td style="text-align:right;"> 1.92 </td>
   <td style="text-align:right;"> 34.62 </td>
   <td style="text-align:right;"> 25.50 </td>
   <td style="text-align:right;"> 423.73 </td>
   <td style="text-align:right;"> 0.81 </td>
   <td style="text-align:right;"> 0.00 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> strategy:looks </td>
   <td style="text-align:right;"> 1.92 </td>
   <td style="text-align:right;"> 34.62 </td>
   <td style="text-align:right;"> 25.50 </td>
   <td style="text-align:right;"> 80.43 </td>
   <td style="text-align:right;"> 0.45 </td>
   <td style="text-align:right;"> 0.00 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> personality </td>
   <td style="text-align:right;"> 1.87 </td>
   <td style="text-align:right;"> 33.62 </td>
   <td style="text-align:right;"> 37.89 </td>
   <td style="text-align:right;"> 328.25 </td>
   <td style="text-align:right;"> 0.83 </td>
   <td style="text-align:right;"> 0.00 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> strategy:personality </td>
   <td style="text-align:right;"> 1.87 </td>
   <td style="text-align:right;"> 33.62 </td>
   <td style="text-align:right;"> 37.89 </td>
   <td style="text-align:right;"> 62.45 </td>
   <td style="text-align:right;"> 0.47 </td>
   <td style="text-align:right;"> 0.00 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> looks:personality </td>
   <td style="text-align:right;"> 3.20 </td>
   <td style="text-align:right;"> 57.55 </td>
   <td style="text-align:right;"> 34.62 </td>
   <td style="text-align:right;"> 36.63 </td>
   <td style="text-align:right;"> 0.45 </td>
   <td style="text-align:right;"> 0.00 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: yellow !important;"> strategy:looks:personality </td>
   <td style="text-align:right;background-color: yellow !important;"> 3.20 </td>
   <td style="text-align:right;background-color: yellow !important;"> 57.55 </td>
   <td style="text-align:right;background-color: yellow !important;"> 34.62 </td>
   <td style="text-align:right;background-color: yellow !important;"> 24.12 </td>
   <td style="text-align:right;background-color: yellow !important;"> 0.35 </td>
   <td style="text-align:right;background-color: yellow !important;"> 0.00 </td>
  </tr>
</tbody>
</table>

</br>

.infobox[

???

Note we use generalized eta-squared, which is better for repeated measures designs.

---
# Main effect of strategy

.pull-left[
![](mixed_designs_files/figure-html/unnamed-chunk-7-1.png)
]

.pull-right[

```r
emmeans::emmeans(date_afx, ~strategy, model = "multivariate")
```

<table>
 <thead>
  <tr>
   <th style="text-align:left;"> strategy </th>
   <th style="text-align:right;"> emmean </th>
   <th style="text-align:right;"> SE </th>
   <th style="text-align:right;"> df </th>
   <th style="text-align:right;"> lower.CL </th>
   <th style="text-align:right;"> upper.CL </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Normal </td>
   <td style="text-align:right;"> 68.53 </td>
   <td style="text-align:right;"> 0.69 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 67.09 </td>
   <td style="text-align:right;"> 69.97 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Hard to get </td>
   <td style="text-align:right;"> 68.60 </td>
   <td style="text-align:right;"> 0.69 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 67.16 </td>
   <td style="text-align:right;"> 70.04 </td>
  </tr>
</tbody>
</table>

]

---
# Main effect of looks

.pull-left[
![](mixed_designs_files/figure-html/unnamed-chunk-10-1.png)
]

.pull-right[

```r
emmeans::emmeans(date_afx, ~looks, model = "multivariate")
```

<table>
 <thead>
  <tr>
   <th style="text-align:left;"> looks </th>
   <th style="text-align:right;"> emmean </th>
   <th style="text-align:right;"> SE </th>
   <th style="text-align:right;"> df </th>
   <th style="text-align:right;"> lower.CL </th>
   <th style="text-align:right;"> upper.CL </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:right;"> 55.82 </td>
   <td style="text-align:right;"> 0.65 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 54.45 </td>
   <td style="text-align:right;"> 57.18 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:right;"> 67.78 </td>
   <td style="text-align:right;"> 0.82 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 66.06 </td>
   <td style="text-align:right;"> 69.51 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> High </td>
   <td style="text-align:right;"> 82.10 </td>
   <td style="text-align:right;"> 0.65 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 80.73 </td>
   <td style="text-align:right;"> 83.47 </td>
  </tr>
</tbody>
</table>

]

---
# Main effect of personality

.pull-left[
![](mixed_designs_files/figure-html/unnamed-chunk-13-1.png)
]

.pull-right[

```r
emmeans::emmeans(date_afx, ~personality, model = "multivariate")
```

<table>
 <thead>
  <tr>
   <th style="text-align:left;"> personality </th>
   <th style="text-align:right;"> emmean </th>
   <th style="text-align:right;"> SE </th>
   <th style="text-align:right;"> df </th>
   <th style="text-align:right;"> lower.CL </th>
   <th style="text-align:right;"> upper.CL </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:right;"> 54.3 </td>
   <td style="text-align:right;"> 0.57 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 53.10 </td>
   <td style="text-align:right;"> 55.50 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:right;"> 69.3 </td>
   <td style="text-align:right;"> 0.73 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 67.76 </td>
   <td style="text-align:right;"> 70.84 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> High </td>
   <td style="text-align:right;"> 82.1 </td>
   <td style="text-align:right;"> 1.01 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 79.98 </td>
   <td style="text-align:right;"> 84.22 </td>
  </tr>
</tbody>
</table>

]

---
# Strategy × looks interaction

.pull-left[
![](mixed_designs_files/figure-html/unnamed-chunk-16-1.png)
]

.pull-right[

```r
emmeans::emmeans(date_afx, c("strategy", "looks"), model = "multivariate")
```

<table class="table" style="font-size: 14px; margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;"> strategy </th>
   <th style="text-align:left;"> looks </th>
   <th style="text-align:right;"> emmean </th>
   <th style="text-align:right;"> SE </th>
   <th style="text-align:right;"> df </th>
   <th style="text-align:right;"> lower.CL </th>
   <th style="text-align:right;"> upper.CL </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Normal </td>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:right;"> 61.33 </td>
   <td style="text-align:right;"> 0.92 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 59.40 </td>
   <td style="text-align:right;"> 63.27 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Hard to get </td>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:right;"> 50.30 </td>
   <td style="text-align:right;"> 0.92 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 48.37 </td>
   <td style="text-align:right;"> 52.23 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Normal </td>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:right;"> 68.10 </td>
   <td style="text-align:right;"> 1.16 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 65.66 </td>
   <td style="text-align:right;"> 70.54 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Hard to get </td>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:right;"> 67.47 </td>
   <td style="text-align:right;"> 1.16 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 65.03 </td>
   <td style="text-align:right;"> 69.90 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Normal </td>
   <td style="text-align:left;"> High </td>
   <td style="text-align:right;"> 76.17 </td>
   <td style="text-align:right;"> 0.92 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 74.23 </td>
   <td style="text-align:right;"> 78.11 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Hard to get </td>
   <td style="text-align:left;"> High </td>
   <td style="text-align:right;"> 88.03 </td>
   <td style="text-align:right;"> 0.92 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 86.09 </td>
   <td style="text-align:right;"> 89.97 </td>
  </tr>
</tbody>
</table>

]

---
# Strategy × personality interaction

.pull-left[
![](mixed_designs_files/figure-html/unnamed-chunk-19-1.png)
]

.pull-right[

```r
emmeans::emmeans(date_afx, c("strategy", "personality"), model = "multivariate")
```

<table class="table" style="font-size: 14px; margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;"> strategy </th>
   <th style="text-align:left;"> personality </th>
   <th style="text-align:right;"> emmean </th>
   <th style="text-align:right;"> SE </th>
   <th style="text-align:right;"> df </th>
   <th style="text-align:right;"> lower.CL </th>
   <th style="text-align:right;"> upper.CL </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Normal </td>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:right;"> 48.30 </td>
   <td style="text-align:right;"> 0.81 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 46.60 </td>
   <td style="text-align:right;"> 50.00 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Hard to get </td>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:right;"> 60.30 </td>
   <td style="text-align:right;"> 0.81 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 58.60 </td>
   <td style="text-align:right;"> 62.00 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Normal </td>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:right;"> 69.07 </td>
   <td style="text-align:right;"> 1.03 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 66.89 </td>
   <td style="text-align:right;"> 71.24 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Hard to get </td>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:right;"> 69.53 </td>
   <td style="text-align:right;"> 1.03 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 67.36 </td>
   <td style="text-align:right;"> 71.71 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Normal </td>
   <td style="text-align:left;"> High </td>
   <td style="text-align:right;"> 88.23 </td>
   <td style="text-align:right;"> 1.43 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 85.23 </td>
   <td style="text-align:right;"> 91.23 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Hard to get </td>
   <td style="text-align:left;"> High </td>
   <td style="text-align:right;"> 75.97 </td>
   <td style="text-align:right;"> 1.43 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 72.97 </td>
   <td style="text-align:right;"> 78.97 </td>
  </tr>
</tbody>
</table>

]

---
# Looks × personality interaction

.pull-left[
![](mixed_designs_files/figure-html/unnamed-chunk-22-1.png)
]

.pull-right[

```r
emmeans::emmeans(date_afx, c("looks", "personality"), model = "multivariate")
```

<table class="table" style="font-size: 14px; margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;"> looks </th>
   <th style="text-align:left;"> personality </th>
   <th style="text-align:right;"> emmean </th>
   <th style="text-align:right;"> SE </th>
   <th style="text-align:right;"> df </th>
   <th style="text-align:right;"> lower.CL </th>
   <th style="text-align:right;"> upper.CL </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:right;"> 45.95 </td>
   <td style="text-align:right;"> 0.75 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 44.38 </td>
   <td style="text-align:right;"> 47.52 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:right;"> 47.40 </td>
   <td style="text-align:right;"> 0.89 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 45.54 </td>
   <td style="text-align:right;"> 49.26 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> High </td>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:right;"> 69.55 </td>
   <td style="text-align:right;"> 1.02 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 67.41 </td>
   <td style="text-align:right;"> 71.69 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:right;"> 49.75 </td>
   <td style="text-align:right;"> 1.21 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 47.21 </td>
   <td style="text-align:right;"> 52.29 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:right;"> 70.35 </td>
   <td style="text-align:right;"> 1.17 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 67.89 </td>
   <td style="text-align:right;"> 72.81 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> High </td>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:right;"> 87.80 </td>
   <td style="text-align:right;"> 1.41 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 84.84 </td>
   <td style="text-align:right;"> 90.76 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:left;"> High </td>
   <td style="text-align:right;"> 71.75 </td>
   <td style="text-align:right;"> 1.25 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 69.13 </td>
   <td style="text-align:right;"> 74.37 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:left;"> High </td>
   <td style="text-align:right;"> 85.60 </td>
   <td style="text-align:right;"> 1.72 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 81.98 </td>
   <td style="text-align:right;"> 89.22 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> High </td>
   <td style="text-align:left;"> High </td>
   <td style="text-align:right;"> 88.95 </td>
   <td style="text-align:right;"> 1.38 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 86.04 </td>
   <td style="text-align:right;"> 91.86 </td>
  </tr>
</tbody>
</table>

]

---
background-image: none
background-color: #000000
class: no-scribble

---
# Strategy × looks × personality interaction

.pull-left[
![](mixed_designs_files/figure-html/unnamed-chunk-25-1.png)
]

.pull-right[

<table class="table" style="font-size: 10px; margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;"> strategy </th>
   <th style="text-align:left;"> looks </th>
   <th style="text-align:left;"> personality </th>
   <th style="text-align:right;"> emmean </th>
   <th style="text-align:right;"> SE </th>
   <th style="text-align:right;"> df </th>
   <th style="text-align:right;"> lower.CL </th>
   <th style="text-align:right;"> upper.CL </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Normal </td>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:right;"> 46.1 </td>
   <td style="text-align:right;"> 1.06 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 43.88 </td>
   <td style="text-align:right;"> 48.32 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Hard to get </td>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:right;"> 45.8 </td>
   <td style="text-align:right;"> 1.06 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 43.58 </td>
   <td style="text-align:right;"> 48.02 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Normal </td>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:right;"> 47.0 </td>
   <td style="text-align:right;"> 1.26 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 44.36 </td>
   <td style="text-align:right;"> 49.64 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Hard to get </td>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:right;"> 47.8 </td>
   <td style="text-align:right;"> 1.26 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 45.16 </td>
   <td style="text-align:right;"> 50.44 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Normal </td>
   <td style="text-align:left;"> High </td>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:right;"> 51.8 </td>
   <td style="text-align:right;"> 1.44 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 48.77 </td>
   <td style="text-align:right;"> 54.83 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Hard to get </td>
   <td style="text-align:left;"> High </td>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:right;"> 87.3 </td>
   <td style="text-align:right;"> 1.44 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 84.27 </td>
   <td style="text-align:right;"> 90.33 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Normal </td>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:right;"> 51.2 </td>
   <td style="text-align:right;"> 1.71 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 47.60 </td>
   <td style="text-align:right;"> 54.80 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Hard to get </td>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:right;"> 48.3 </td>
   <td style="text-align:right;"> 1.71 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 44.70 </td>
   <td style="text-align:right;"> 51.90 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Normal </td>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:right;"> 68.9 </td>
   <td style="text-align:right;"> 1.66 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 65.42 </td>
   <td style="text-align:right;"> 72.38 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Hard to get </td>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:right;"> 71.8 </td>
   <td style="text-align:right;"> 1.66 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 68.32 </td>
   <td style="text-align:right;"> 75.28 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Normal </td>
   <td style="text-align:left;"> High </td>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:right;"> 87.1 </td>
   <td style="text-align:right;"> 1.99 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 82.92 </td>
   <td style="text-align:right;"> 91.28 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Hard to get </td>
   <td style="text-align:left;"> High </td>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:right;"> 88.5 </td>
   <td style="text-align:right;"> 1.99 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 84.32 </td>
   <td style="text-align:right;"> 92.68 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Normal </td>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:left;"> High </td>
   <td style="text-align:right;"> 86.7 </td>
   <td style="text-align:right;"> 1.77 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 82.99 </td>
   <td style="text-align:right;"> 90.41 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Hard to get </td>
   <td style="text-align:left;"> Low </td>
   <td style="text-align:left;"> High </td>
   <td style="text-align:right;"> 56.8 </td>
   <td style="text-align:right;"> 1.77 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 53.09 </td>
   <td style="text-align:right;"> 60.51 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Normal </td>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:left;"> High </td>
   <td style="text-align:right;"> 88.4 </td>
   <td style="text-align:right;"> 2.43 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 83.29 </td>
   <td style="text-align:right;"> 93.51 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Hard to get </td>
   <td style="text-align:left;"> Average </td>
   <td style="text-align:left;"> High </td>
   <td style="text-align:right;"> 82.8 </td>
   <td style="text-align:right;"> 2.43 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 77.69 </td>
   <td style="text-align:right;"> 87.91 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Normal </td>
   <td style="text-align:left;"> High </td>
   <td style="text-align:left;"> High </td>
   <td style="text-align:right;"> 89.6 </td>
   <td style="text-align:right;"> 1.96 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 85.49 </td>
   <td style="text-align:right;"> 93.71 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Hard to get </td>
   <td style="text-align:left;"> High </td>
   <td style="text-align:left;"> High </td>
   <td style="text-align:right;"> 88.3 </td>
   <td style="text-align:right;"> 1.96 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 84.19 </td>
   <td style="text-align:right;"> 92.41 </td>
  </tr>
</tbody>
</table>

]

---
# Contrasts across the interaction

```r
three_way_emm <- emmeans::emmeans(date_afx, c("strategy", "looks", "personality"), model = "multivariate")

emmeans::contrast(
  three_way_emm,
  interaction = c(strategy = "trt.vs.ctrl",
                  looks = "trt.vs.ctrl",
                  personality = "trt.vs.ctrl"),
  ref = 2,
  adjust = "holm"
  )
```

<table>
 <thead>
  <tr>
   <th style="text-align:left;"> Contrast </th>
   <th style="text-align:left;"> strategy_trt.vs.ctrl </th>
   <th style="text-align:left;"> looks_trt.vs.ctrl </th>
   <th style="text-align:left;"> personality_trt.vs.ctrl </th>
   <th style="text-align:right;"> estimate </th>
   <th style="text-align:right;"> SE </th>
   <th style="text-align:right;"> df </th>
   <th style="text-align:right;"> t.ratio </th>
   <th style="text-align:right;"> p.value </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> Normal - Hard to get </td>
   <td style="text-align:left;"> Low - Average </td>
   <td style="text-align:left;"> Low - Average </td>
   <td style="text-align:right;background-color: yellow !important;"> -4.7 </td>
   <td style="text-align:right;"> 4.07 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> -1.16 </td>
   <td style="text-align:right;background-color: yellow !important;"> 0.53 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 2 </td>
   <td style="text-align:left;"> Normal - Hard to get </td>
   <td style="text-align:left;"> High - Average </td>
   <td style="text-align:left;"> Low - Average </td>
   <td style="text-align:right;background-color: yellow !important;"> -36.2 </td>
   <td style="text-align:right;"> 4.65 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> -7.79 </td>
   <td style="text-align:right;background-color: yellow !important;"> 0.00 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 3 </td>
   <td style="text-align:left;"> Normal - Hard to get </td>
   <td style="text-align:left;"> Low - Average </td>
   <td style="text-align:left;"> High - Average </td>
   <td style="text-align:right;background-color: yellow !important;"> 18.5 </td>
   <td style="text-align:right;"> 5.41 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 3.42 </td>
   <td style="text-align:right;background-color: yellow !important;"> 0.01 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 4 </td>
   <td style="text-align:left;"> Normal - Hard to get </td>
   <td style="text-align:left;"> High - Average </td>
   <td style="text-align:left;"> High - Average </td>
   <td style="text-align:right;background-color: yellow !important;"> -5.8 </td>
   <td style="text-align:right;"> 6.02 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> -0.96 </td>
   <td style="text-align:right;background-color: yellow !important;"> 0.53 </td>
  </tr>
</tbody>
</table>

???

Note p-values adjusted for 4 tests, they are not adjusted on the next slides (so the ps change)

---
## Contrast 1
.center[

![](mixed_designs_files/figure-html/unnamed-chunk-31-1.png)
]
---
## Contrast 1

.center[
<table>
 <thead>
  <tr>
   <th style="text-align:left;"> Contrast </th>
   <th style="text-align:left;"> strategy_trt.vs.ctrl </th>
   <th style="text-align:left;"> looks_trt.vs.ctrl </th>
   <th style="text-align:left;"> personality_trt.vs.ctrl </th>
   <th style="text-align:right;"> estimate </th>
   <th style="text-align:right;"> SE </th>
   <th style="text-align:right;"> df </th>
   <th style="text-align:right;"> t.ratio </th>
   <th style="text-align:right;"> p.value </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> Normal - Hard to get </td>
   <td style="text-align:left;"> Low - Average </td>
   <td style="text-align:left;"> Low - Average </td>
   <td style="text-align:right;background-color: yellow !important;"> -4.7 </td>
   <td style="text-align:right;"> 4.07 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> -1.16 </td>
   <td style="text-align:right;background-color: yellow !important;"> 0.53 </td>
  </tr>
</tbody>
</table>

![](mixed_designs_files/figure-html/unnamed-chunk-33-1.png)
]

---
## Contrast 2
.center[

![](mixed_designs_files/figure-html/unnamed-chunk-35-1.png)
]

---
## Contrast 2

.center[
<table>
 <thead>
  <tr>
   <th style="text-align:left;"> Contrast </th>
   <th style="text-align:left;"> strategy_trt.vs.ctrl </th>
   <th style="text-align:left;"> looks_trt.vs.ctrl </th>
   <th style="text-align:left;"> personality_trt.vs.ctrl </th>
   <th style="text-align:right;"> estimate </th>
   <th style="text-align:right;"> SE </th>
   <th style="text-align:right;"> df </th>
   <th style="text-align:right;"> t.ratio </th>
   <th style="text-align:right;"> p.value </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> 2 </td>
   <td style="text-align:left;"> Normal - Hard to get </td>
   <td style="text-align:left;"> High - Average </td>
   <td style="text-align:left;"> Low - Average </td>
   <td style="text-align:right;background-color: yellow !important;"> -36.2 </td>
   <td style="text-align:right;"> 4.65 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> -7.79 </td>
   <td style="text-align:right;background-color: yellow !important;"> 0 </td>
  </tr>
</tbody>
</table>

![](mixed_designs_files/figure-html/unnamed-chunk-37-1.png)
]

---
## Contrast 3
.center[
<table>
 <thead>
  <tr>
   <th style="text-align:left;"> Contrast </th>
   <th style="text-align:left;"> strategy_trt.vs.ctrl </th>
   <th style="text-align:left;"> looks_trt.vs.ctrl </th>
   <th style="text-align:left;"> personality_trt.vs.ctrl </th>
   <th style="text-align:right;"> estimate </th>
   <th style="text-align:right;"> SE </th>
   <th style="text-align:right;"> df </th>
   <th style="text-align:right;"> t.ratio </th>
   <th style="text-align:right;"> p.value </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> 3 </td>
   <td style="text-align:left;"> Normal - Hard to get </td>
   <td style="text-align:left;"> Low - Average </td>
   <td style="text-align:left;"> High - Average </td>
   <td style="text-align:right;background-color: yellow !important;"> 18.5 </td>
   <td style="text-align:right;"> 5.41 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 3.42 </td>
   <td style="text-align:right;background-color: yellow !important;"> 0.01 </td>
  </tr>
</tbody>
</table>

![](mixed_designs_files/figure-html/unnamed-chunk-39-1.png)
]
---
## Contrast 3

.center[
<table>
 <thead>
  <tr>
   <th style="text-align:left;"> Contrast </th>
   <th style="text-align:left;"> strategy_trt.vs.ctrl </th>
   <th style="text-align:left;"> looks_trt.vs.ctrl </th>
   <th style="text-align:left;"> personality_trt.vs.ctrl </th>
   <th style="text-align:right;"> estimate </th>
   <th style="text-align:right;"> SE </th>
   <th style="text-align:right;"> df </th>
   <th style="text-align:right;"> t.ratio </th>
   <th style="text-align:right;"> p.value </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> 3 </td>
   <td style="text-align:left;"> Normal - Hard to get </td>
   <td style="text-align:left;"> Low - Average </td>
   <td style="text-align:left;"> High - Average </td>
   <td style="text-align:right;background-color: yellow !important;"> 18.5 </td>
   <td style="text-align:right;"> 5.41 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 3.42 </td>
   <td style="text-align:right;background-color: yellow !important;"> 0.01 </td>
  </tr>
</tbody>
</table>

![](mixed_designs_files/figure-html/unnamed-chunk-41-1.png)
]

---
## Contrast 4
.center[
<table>
 <thead>
  <tr>
   <th style="text-align:left;"> Contrast </th>
   <th style="text-align:left;"> strategy_trt.vs.ctrl </th>
   <th style="text-align:left;"> looks_trt.vs.ctrl </th>
   <th style="text-align:left;"> personality_trt.vs.ctrl </th>
   <th style="text-align:right;"> estimate </th>
   <th style="text-align:right;"> SE </th>
   <th style="text-align:right;"> df </th>
   <th style="text-align:right;"> t.ratio </th>
   <th style="text-align:right;"> p.value </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> 4 </td>
   <td style="text-align:left;"> Normal - Hard to get </td>
   <td style="text-align:left;"> High - Average </td>
   <td style="text-align:left;"> High - Average </td>
   <td style="text-align:right;background-color: yellow !important;"> -5.8 </td>
   <td style="text-align:right;"> 6.02 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> -0.96 </td>
   <td style="text-align:right;background-color: yellow !important;"> 0.53 </td>
  </tr>
</tbody>
</table>

![](mixed_designs_files/figure-html/unnamed-chunk-43-1.png)
]

---
## Contrast 4

.center[
<table>
 <thead>
  <tr>
   <th style="text-align:left;"> Contrast </th>
   <th style="text-align:left;"> strategy_trt.vs.ctrl </th>
   <th style="text-align:left;"> looks_trt.vs.ctrl </th>
   <th style="text-align:left;"> personality_trt.vs.ctrl </th>
   <th style="text-align:right;"> estimate </th>
   <th style="text-align:right;"> SE </th>
   <th style="text-align:right;"> df </th>
   <th style="text-align:right;"> t.ratio </th>
   <th style="text-align:right;"> p.value </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> 4 </td>
   <td style="text-align:left;"> Normal - Hard to get </td>
   <td style="text-align:left;"> High - Average </td>
   <td style="text-align:left;"> High - Average </td>
   <td style="text-align:right;background-color: yellow !important;"> -5.8 </td>
   <td style="text-align:right;"> 6.02 </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> -0.96 </td>
   <td style="text-align:right;background-color: yellow !important;"> 0.53 </td>
  </tr>
</tbody>
</table>

![](mixed_designs_files/figure-html/unnamed-chunk-45-1.png)
]

---
# Summary

* Mixed designs combine at least one predictor measured using repeated measures, and at least one measured using different entities

* Sphericity
  + Correct using Greenhouse-Geisser, report it and forget about it
  + `afex` does this by default

* The output is much the same as an other designs
	+ An *F*-statistic for each effect
	+ Interpret as you would any *F*-statistic
	+ Pick apart the highest order interaction
		
* Interactions
  + Use built in contrasts to tease apart what the interaction shows