-
Notifications
You must be signed in to change notification settings - Fork 1
Expand file tree
/
Copy pathGLM_3_Bernoulli_exercise_solutions.html
More file actions
1723 lines (1621 loc) · 146 KB
/
GLM_3_Bernoulli_exercise_solutions.html
File metadata and controls
1723 lines (1621 loc) · 146 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<meta name="generator" content="pandoc" />
<meta http-equiv="X-UA-Compatible" content="IE=EDGE" />
<title>Day 3: Bernoulli GLMs</title>
<script src="site_libs/header-attrs-2.29/header-attrs.js"></script>
<script src="site_libs/jquery-3.6.0/jquery-3.6.0.min.js"></script>
<meta name="viewport" content="width=device-width, initial-scale=1" />
<link href="site_libs/bootstrap-3.3.5/css/flatly.min.css" rel="stylesheet" />
<script src="site_libs/bootstrap-3.3.5/js/bootstrap.min.js"></script>
<script src="site_libs/bootstrap-3.3.5/shim/html5shiv.min.js"></script>
<script src="site_libs/bootstrap-3.3.5/shim/respond.min.js"></script>
<style>h1 {font-size: 34px;}
h1.title {font-size: 38px;}
h2 {font-size: 30px;}
h3 {font-size: 24px;}
h4 {font-size: 18px;}
h5 {font-size: 16px;}
h6 {font-size: 12px;}
code {color: inherit; background-color: rgba(0, 0, 0, 0.04);}
pre:not([class]) { background-color: white }</style>
<script src="site_libs/navigation-1.1/tabsets.js"></script>
<script src="site_libs/navigation-1.1/codefolding.js"></script>
<link href="site_libs/font-awesome-6.5.2/css/all.min.css" rel="stylesheet" />
<link href="site_libs/font-awesome-6.5.2/css/v4-shims.min.css" rel="stylesheet" />
<style type="text/css">
code{white-space: pre-wrap;}
span.smallcaps{font-variant: small-caps;}
span.underline{text-decoration: underline;}
div.column{display: inline-block; vertical-align: top; width: 50%;}
div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
ul.task-list{list-style: none;}
</style>
<style type="text/css">
code {
white-space: pre;
}
.sourceCode {
overflow: visible;
}
</style>
<style type="text/css" data-origin="pandoc">
pre > code.sourceCode { white-space: pre; position: relative; }
pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
pre > code.sourceCode > span:empty { height: 1.2em; }
.sourceCode { overflow: visible; }
code.sourceCode > span { color: inherit; text-decoration: inherit; }
div.sourceCode { margin: 1em 0; }
pre.sourceCode { margin: 0; }
@media screen {
div.sourceCode { overflow: auto; }
}
@media print {
pre > code.sourceCode { white-space: pre-wrap; }
pre > code.sourceCode > span { text-indent: -5em; padding-left: 5em; }
}
pre.numberSource code
{ counter-reset: source-line 0; }
pre.numberSource code > span
{ position: relative; left: -4em; counter-increment: source-line; }
pre.numberSource code > span > a:first-child::before
{ content: counter(source-line);
position: relative; left: -1em; text-align: right; vertical-align: baseline;
border: none; display: inline-block;
-webkit-touch-callout: none; -webkit-user-select: none;
-khtml-user-select: none; -moz-user-select: none;
-ms-user-select: none; user-select: none;
padding: 0 4px; width: 4em;
color: #aaaaaa;
}
pre.numberSource { margin-left: 3em; border-left: 1px solid #aaaaaa; padding-left: 4px; }
div.sourceCode
{ background-color: #f8f8f8; }
@media screen {
pre > code.sourceCode > span > a:first-child::before { text-decoration: underline; }
}
code span.al { color: #ef2929; } /* Alert */
code span.an { color: #8f5902; font-weight: bold; font-style: italic; } /* Annotation */
code span.at { color: #204a87; } /* Attribute */
code span.bn { color: #0000cf; } /* BaseN */
code span.cf { color: #204a87; font-weight: bold; } /* ControlFlow */
code span.ch { color: #4e9a06; } /* Char */
code span.cn { color: #8f5902; } /* Constant */
code span.co { color: #8f5902; font-style: italic; } /* Comment */
code span.cv { color: #8f5902; font-weight: bold; font-style: italic; } /* CommentVar */
code span.do { color: #8f5902; font-weight: bold; font-style: italic; } /* Documentation */
code span.dt { color: #204a87; } /* DataType */
code span.dv { color: #0000cf; } /* DecVal */
code span.er { color: #a40000; font-weight: bold; } /* Error */
code span.ex { } /* Extension */
code span.fl { color: #0000cf; } /* Float */
code span.fu { color: #204a87; font-weight: bold; } /* Function */
code span.im { } /* Import */
code span.in { color: #8f5902; font-weight: bold; font-style: italic; } /* Information */
code span.kw { color: #204a87; font-weight: bold; } /* Keyword */
code span.op { color: #ce5c00; font-weight: bold; } /* Operator */
code span.ot { color: #8f5902; } /* Other */
code span.pp { color: #8f5902; font-style: italic; } /* Preprocessor */
code span.sc { color: #ce5c00; font-weight: bold; } /* SpecialChar */
code span.ss { color: #4e9a06; } /* SpecialString */
code span.st { color: #4e9a06; } /* String */
code span.va { color: #000000; } /* Variable */
code span.vs { color: #4e9a06; } /* VerbatimString */
code span.wa { color: #8f5902; font-weight: bold; font-style: italic; } /* Warning */
.sourceCode .row {
width: 100%;
}
.sourceCode {
overflow-x: auto;
}
.code-folding-btn {
margin-right: -30px;
}
</style>
<script>
// apply pandoc div.sourceCode style to pre.sourceCode instead
(function() {
var sheets = document.styleSheets;
for (var i = 0; i < sheets.length; i++) {
if (sheets[i].ownerNode.dataset["origin"] !== "pandoc") continue;
try { var rules = sheets[i].cssRules; } catch (e) { continue; }
var j = 0;
while (j < rules.length) {
var rule = rules[j];
// check if there is a div.sourceCode rule
if (rule.type !== rule.STYLE_RULE || rule.selectorText !== "div.sourceCode") {
j++;
continue;
}
var style = rule.style.cssText;
// check if color or background-color is set
if (rule.style.color === '' && rule.style.backgroundColor === '') {
j++;
continue;
}
// replace div.sourceCode by a pre.sourceCode rule
sheets[i].deleteRule(j);
sheets[i].insertRule('pre.sourceCode{' + style + '}', j);
}
}
})();
</script>
<style type = "text/css">
.main-container {
max-width: 940px;
margin-left: auto;
margin-right: auto;
}
img {
max-width:100%;
}
.tabbed-pane {
padding-top: 12px;
}
.html-widget {
margin-bottom: 20px;
}
button.code-folding-btn:focus {
outline: none;
}
summary {
display: list-item;
}
details > summary > p:only-child {
display: inline;
}
pre code {
padding: 0;
}
</style>
<style type="text/css">
.dropdown-submenu {
position: relative;
}
.dropdown-submenu>.dropdown-menu {
top: 0;
left: 100%;
margin-top: -6px;
margin-left: -1px;
border-radius: 0 6px 6px 6px;
}
.dropdown-submenu:hover>.dropdown-menu {
display: block;
}
.dropdown-submenu>a:after {
display: block;
content: " ";
float: right;
width: 0;
height: 0;
border-color: transparent;
border-style: solid;
border-width: 5px 0 5px 5px;
border-left-color: #cccccc;
margin-top: 5px;
margin-right: -10px;
}
.dropdown-submenu:hover>a:after {
border-left-color: #adb5bd;
}
.dropdown-submenu.pull-left {
float: none;
}
.dropdown-submenu.pull-left>.dropdown-menu {
left: -100%;
margin-left: 10px;
border-radius: 6px 0 6px 6px;
}
</style>
<script type="text/javascript">
// manage active state of menu based on current page
$(document).ready(function () {
// active menu anchor
href = window.location.pathname
href = href.substr(href.lastIndexOf('/') + 1)
if (href === "")
href = "index.html";
var menuAnchor = $('a[href="' + href + '"]');
// mark the anchor link active (and if it's in a dropdown, also mark that active)
var dropdown = menuAnchor.closest('li.dropdown');
if (window.bootstrap) { // Bootstrap 4+
menuAnchor.addClass('active');
dropdown.find('> .dropdown-toggle').addClass('active');
} else { // Bootstrap 3
menuAnchor.parent().addClass('active');
dropdown.addClass('active');
}
// Navbar adjustments
var navHeight = $(".navbar").first().height() + 15;
var style = document.createElement('style');
var pt = "padding-top: " + navHeight + "px; ";
var mt = "margin-top: -" + navHeight + "px; ";
var css = "";
// offset scroll position for anchor links (for fixed navbar)
for (var i = 1; i <= 6; i++) {
css += ".section h" + i + "{ " + pt + mt + "}\n";
}
style.innerHTML = "body {" + pt + "padding-bottom: 40px; }\n" + css;
document.head.appendChild(style);
});
</script>
<!-- tabsets -->
<style type="text/css">
.tabset-dropdown > .nav-tabs {
display: inline-table;
max-height: 500px;
min-height: 44px;
overflow-y: auto;
border: 1px solid #ddd;
border-radius: 4px;
}
.tabset-dropdown > .nav-tabs > li.active:before, .tabset-dropdown > .nav-tabs.nav-tabs-open:before {
content: "\e259";
font-family: 'Glyphicons Halflings';
display: inline-block;
padding: 10px;
border-right: 1px solid #ddd;
}
.tabset-dropdown > .nav-tabs.nav-tabs-open > li.active:before {
content: "\e258";
font-family: 'Glyphicons Halflings';
border: none;
}
.tabset-dropdown > .nav-tabs > li.active {
display: block;
}
.tabset-dropdown > .nav-tabs > li > a,
.tabset-dropdown > .nav-tabs > li > a:focus,
.tabset-dropdown > .nav-tabs > li > a:hover {
border: none;
display: inline-block;
border-radius: 4px;
background-color: transparent;
}
.tabset-dropdown > .nav-tabs.nav-tabs-open > li {
display: block;
float: none;
}
.tabset-dropdown > .nav-tabs > li {
display: none;
}
</style>
<!-- code folding -->
<style type="text/css">
.code-folding-btn { margin-bottom: 4px; }
</style>
</head>
<body>
<div class="container-fluid main-container">
<div class="navbar navbar-default navbar-fixed-top" role="navigation">
<div class="container">
<div class="navbar-header">
<button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-bs-toggle="collapse" data-target="#navbar" data-bs-target="#navbar">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="index.html">PGR-GLM</a>
</div>
<div id="navbar" class="navbar-collapse collapse">
<ul class="nav navbar-nav">
<li>
<a href="index.html">
<span class="fa fa-home"></span>
Home
</a>
</li>
<li>
<a href="setup.html">
<span class="fa fa-cog"></span>
Setup
</a>
</li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" data-bs-toggle="dropdown" aria-expanded="false">
<span class="fa fa-book"></span>
R Book
<span class="caret"></span>
</a>
<ul class="dropdown-menu" role="menu">
<li>
<a href="https://intro2r.com">
<span class="fa fa-firefox"></span>
Web book
</a>
</li>
<li class="divider"></li>
<li>
<a href="https://github.com/alexd106/Rbook/raw/master/docs/Rbook.pdf">
<span class="fa fa-file-pdf"></span>
PDF book
</a>
</li>
</ul>
</li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" data-bs-toggle="dropdown" aria-expanded="false">
<span class="fa fa-book"></span>
Exercises
<span class="caret"></span>
</a>
<ul class="dropdown-menu" role="menu">
<li>
<a href="exercises.html">
<span class="fa fa-book"></span>
Exercises
</a>
</li>
<li class="divider"></li>
<li>
<a href="exercise_solutions.html">
<span class="fa fa-book"></span>
Exercise Solutions
</a>
</li>
</ul>
</li>
<li>
<a href="data.html">
<span class="fa fa-download"></span>
Data
</a>
</li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" data-bs-toggle="dropdown" aria-expanded="false">
<span class="fa fa-question-circle"></span>
Info
<span class="caret"></span>
</a>
<ul class="dropdown-menu" role="menu">
<li>
<a href="syllabus.html">
<span class="fa fa-graduation-cap"></span>
Syllabus
</a>
</li>
<li>
<a href="People.html">
<span class="fa fa-user-friends"></span>
People
</a>
</li>
<li class="divider"></li>
<li>
<a href="resources.html">
<span class="fa fa-book"></span>
Resources
</a>
</li>
<li class="dropdown-header">Feedback</li>
<li>
<a href="People.html">
<span class="fa fa-envelope fa-lg"></span>
Contact
</a>
</li>
<li class="divider"></li>
<li>
<a href="https://github.com/alexd106/PGR-GLM">
<span class="fa fa-github fa-lg"></span>
Source code
</a>
</li>
</ul>
</li>
</ul>
<ul class="nav navbar-nav navbar-right">
</ul>
</div><!--/.nav-collapse -->
</div><!--/.container -->
</div><!--/.navbar -->
<div id="header">
<div class="btn-group pull-right float-right">
<button type="button" class="btn btn-default btn-xs btn-secondary btn-sm dropdown-toggle" data-toggle="dropdown" data-bs-toggle="dropdown" aria-haspopup="true" aria-expanded="false"><span>Code</span> <span class="caret"></span></button>
<ul class="dropdown-menu dropdown-menu-right" style="min-width: 50px;">
<li><a id="rmd-show-all-code" href="#">Show All Code</a></li>
<li><a id="rmd-hide-all-code" href="#">Hide All Code</a></li>
</ul>
</div>
<h1 class="title toc-ignore">Day 3: Bernoulli GLMs</h1>
</div>
<p> </p>
<div
id="bernoulli-glm---predicting-if-netflix-users-will-watch-lord-of-the-rings-the-fellowship-of-the-ring"
class="section level1">
<h1>Bernoulli GLM - Predicting if Netflix users will watch Lord of the
Rings: The Fellowship of the Ring</h1>
<p> </p>
<p>For the GLM exercises, we’ll use the workflow we suggested in the
first GLM overview lecture as a template, specifically:</p>
<ol style="list-style-type: decimal">
<li><p>Know your research question!</p></li>
<li><p>Think about your response variable (stochastic).</p></li>
<li><p>Think about the process behind the data (deterministic).</p></li>
<li><p>Understand the data that you’ve collected (plot it!)</p></li>
<li><p>Combine into a model that can answer your question.</p></li>
<li><p>Fit the model.</p></li>
<li><p>Check your assumption(s).</p></li>
<li><p>Answer your question.</p></li>
</ol>
<p><br />
</p>
<div id="know-your-research-question" class="section level3">
<h3>1. Know your research question!</h3>
<p><br />
</p>
<p>For this final exercise, we’ll do something that’s, hopefully, a bit
fun and silly. We’re going to work with a relatively small Netflix
dataset, where we’re going to try and <em>predict</em> if a user will
will watch Lord of the Rings: The Fellowship of the Ring. I will admit
to a number of ulterior motivations in ending the course using this
dataset.</p>
<ol style="list-style-type: decimal">
<li><p>There is a common misconception that Machine Learning is for
predicting and Statistics is for understanding. I want to show that this
distinction is meaningless. In truth, machine learning is, in general,
statistics performed (or in some cases developed) by computer
scientists. For example, the GLMs you’ve learned to use on this course
are often called Supervised Machine Learning Models in Machine Learning
courses. The distinction between the two methods largely comes down to
who is teaching you (and who taught them). Indeed, some “classic”
machine learning methods such as Random Forest models are just GLMs with
a few extra bells and whistles in much the same way that some “classic”
statistical methods such as Mixed Effects models are just GLMs with a
few extra bells and whistles.</p></li>
<li><p>Because people tend to think of there being some sort of
distinction between Machine Learning and Statistics, it can result in
people, such as yourselves, not realising they have skill-sets that lots
of employers (including non-academic) are looking for. By using this
Netflix example I want to show that, having taken this course, your
employment options have broadened.</p></li>
<li><p>In the final and corresponding <span
class="math inline">\(Bernoulli\)</span> GLM lecture, I showed an
example where AIC fails miserably. In doing so, I’m at risk of leaving
you with the impression that AIC (and also other forms of model and
variable selection) should never be used. Therefore, in this exercise
we’re going to use an example where AIC is wonderfully well suited for
helping achieve our objective. It will also give me an opportunity to
talk a little more about how AIC works, and also the controversy and
differing opinions on AIC.</p></li>
<li><p>I really like Lord of the Rings.</p></li>
</ol>
<p>While this exercise takes on an industry focus, this is not specific
to <span class="math inline">\(Bernoulli\)</span> GLMs. Indeed, like all
GLMs and LMs, <span class="math inline">\(Bernoulli\)</span> GLMs can be
used for any of the three broad purposes we described in the first
lecture: Causal inference, prediction or description.</p>
<p>The data you’ve been provided includes various information that
Netflix has on hand about their users, for the purposes of predicting if
a user will watch Lord of the Rings (LoTR). To do so, our response
variable if whether not a given viewer watched LoTR (1) or not (0).</p>
<p>The covariates we have are:</p>
<ul>
<li><p><code>lotr</code> - if that user watched LoTR on Netflix</p></li>
<li><p><code>premium</code> - if the user has Netflix Premium or
Standard</p></li>
<li><p><code>age</code> - the age of the user at the time of data
collection</p></li>
<li><p><code>genre_likes</code> - how many fantasy films/shows the user
had previously liked</p></li>
<li><p><code>actor_likes</code> - how many films/shows the user had
previously liked that included cast members of LoTR</p></li>
<li><p><code>similar_user_ratins</code> - what lotr of “similar” users
had liked LoTR (“similar” is undefined)</p></li>
<li><p><code>mean_time_genre</code> - what is the average length of time
users watch fantasy media</p></li>
<li><p><code>user_time</code> - Time of day for user at time of data
collection (<code>Morning</code>, <code>Afternoon</code> and
<code>Evening</code>)</p></li>
<li><p><code>user_day</code> - Day of week for user at time of data
collection (<code>Monday</code>, <code>Tuesday</code>, etc.)</p></li>
<li><p><code>user_device</code> - Type of device user was logged in from
(<code>TV</code>, <code>PC</code>, <code>Mobile Device</code> and
<code>Gaming Device</code>)</p></li>
<li><p><code>fam_members</code> - Number of family member accounts tied
to subscription</p></li>
<li><p><code>country</code> - Country of subscription</p></li>
</ul>
<p>As concisely as possible, write down the research question that can
be answered with our analysis, keeping in mind that Netflix are not
interested in understanding causation or writing a scientific paper,
they want predictive power.</p>
<div class="sourceCode" id="cb1"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1-1"><a href="#cb1-1" tabindex="-1"></a><span class="co"># Using the available covariates, can Lord of the Rings engagement be predicted </span></span>
<span id="cb1-2"><a href="#cb1-2" tabindex="-1"></a><span class="co"># and can we make this prediction as efficient as possible?</span></span></code></pre></div>
<p><br />
</p>
</div>
<div id="think-about-your-response-variable-the-stochastic-element."
class="section level3">
<h3>2. Think about your response variable (the stochastic element).</h3>
<p><br />
</p>
<p>From the information provided in the description above, we can
determine that a <span class="math inline">\(Bernoulli\)</span>
distribution is a sensible one to use here; our data is 0 (user did not
watch LoTR) or 1 (user did watch LoTR). Our stochastic element of the
model would therefore be:</p>
<p><span class="math inline">\(y_i \sim Bernoulli(p_i)\)</span></p>
<p>Where <span class="math inline">\(y\)</span> is if user <span
class="math inline">\(i\)</span> watched LoTR or not, generated
according to a <span class="math inline">\(Bernoulli\)</span>
distribution, with probability <span
class="math inline">\(p\)</span>.</p>
<p><br />
</p>
</div>
<div
id="think-about-the-process-behind-the-data-the-deterministic-element."
class="section level3">
<h3>3. Think about the process behind the data (the deterministic
element).</h3>
<p><br />
</p>
<p>Just like the drinking coffee example from the first lecture, the
<em>Data Generating Process</em> (DGP) for whether or not someone
watches a particular film on Netflix is going to be incredibly complex.
A nuance here is that our objective for this analysis, and intended use,
is prediction. When that’s the case, we can be a bit more relaxed about
understanding the deep complexity underlying the DGP and instead feel
relatively comfortable adding any and all covariates for which we have
data, with an important caveat. We must be confident that we are not
including any covariates which have no conceivable connection to our
response (i.e. variables which may be spuriously correlated, that is
correlated by pure chance). For example, we wouldn’t want to include the
number of springbok observed by that day in Kruger National Park as a
covariate, because it shouldn’t give us any predictive power. The only
way it would is if we’re unlucky; it may be correlated (by pure chance)
with LoTR viewership and thus we may think it helps us do predictions
(see <a href="https://www.tylervigen.com/spurious-correlations">this</a>
website for fun examples of such spurious correlations).</p>
<p>In this dataset, Netflix have collected a total of 11 covariates with
the explicit purpose of predicting viewership, so we can be confident
that at least <em>someone</em> thinks these variables might give us some
predictive power and aren’t spurious correlations. That said, it is
still important to convince yourself of this.</p>
<p>Our model will, nevertheless, still have a deterministic equation
that describes the associations that we’re going to exploit to make
predictions. Given that we have 11 covariates that we could include in
the model, would be cruel to task you with writing the equation out in
full for a model that contained all of these covariates. Instead, assume
our model was <code>lotr ~ age + user_time</code>. Use this simplified
version of the model to write out the equation that would underpin it.
Note that <code>age</code> is a continuous variable, and
<code>user_time</code> is a categorical variable with the levels
<code>Morning</code>, <code>Afternoon</code> and
<code>Evening</code>.</p>
<p>For this equation I’m not giving any hints; so the difficulty does
spike here. If you’re struggling, ask for help. It really is worthwhile
learning how the syntax <code>lotr ~ age + user_time</code> is
translated into an equation, especially when you’re ready to go beyond
GLMs.</p>
<p><span class="math inline">\(logit(p_i) = \text{___}\)</span></p>
<div class="sourceCode" id="cb2"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb2-1"><a href="#cb2-1" tabindex="-1"></a><span class="co"># With the blanks filled in, the equation should be:</span></span>
<span id="cb2-2"><a href="#cb2-2" tabindex="-1"></a><span class="co"># logit(p_i) = beta_0 + beta_1 * Age_i + beta_2 * Evening_i + beta_3 * Morning_i</span></span>
<span id="cb2-3"><a href="#cb2-3" tabindex="-1"></a></span>
<span id="cb2-4"><a href="#cb2-4" tabindex="-1"></a><span class="co"># Why have I included Evening and then Morning, and not Afternoon?</span></span>
<span id="cb2-5"><a href="#cb2-5" tabindex="-1"></a><span class="co"># Because R will take the level that appears first alphanumerically, and use </span></span>
<span id="cb2-6"><a href="#cb2-6" tabindex="-1"></a><span class="co"># that as the intercept (here *A*fternoon)</span></span>
<span id="cb2-7"><a href="#cb2-7" tabindex="-1"></a><span class="co"># If we wanted Morning to be the reference level, what R code would we use?</span></span></code></pre></div>
<p> </p>
</div>
<div id="understand-the-data-that-youve-collected-plot-it"
class="section level3">
<h3>4. Understand the data that you’ve collected (plot it!)</h3>
<p><br />
</p>
<p>Import the data file ‘netflix.txt’ into R. Take a look at the
structure of the response variable and covariates. A special note here
is that a few of our covariates have a natural order to them, for
instance, <code>user_day</code> is day of the week. Consider if you want
to alter this variable such that the levels in these factors follow a
natural order (e.g. Monday to Sunday) to make interpretation more
intuitive rather than leaving these in alphabetical order.</p>
<div class="sourceCode" id="cb3"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb3-1"><a href="#cb3-1" tabindex="-1"></a><span class="co"># Load in the data</span></span>
<span id="cb3-2"><a href="#cb3-2" tabindex="-1"></a>netflix <span class="ot"><-</span> <span class="fu">read.table</span>(<span class="at">file=</span> <span class="st">"./data/netflix.txt"</span>, <span class="at">header =</span> <span class="cn">TRUE</span>)</span>
<span id="cb3-3"><a href="#cb3-3" tabindex="-1"></a></span>
<span id="cb3-4"><a href="#cb3-4" tabindex="-1"></a><span class="co"># Check the structure of the data</span></span>
<span id="cb3-5"><a href="#cb3-5" tabindex="-1"></a><span class="co"># When we do so, we can see a few covariates have been read in a characters</span></span>
<span id="cb3-6"><a href="#cb3-6" tabindex="-1"></a><span class="co"># We want to convert these to factors, and impart an order to some of them</span></span>
<span id="cb3-7"><a href="#cb3-7" tabindex="-1"></a><span class="co"># Including an order will make our figures more legible and our results more</span></span>
<span id="cb3-8"><a href="#cb3-8" tabindex="-1"></a><span class="co"># intuitive.</span></span>
<span id="cb3-9"><a href="#cb3-9" tabindex="-1"></a><span class="fu">str</span>(netflix)</span>
<span id="cb3-10"><a href="#cb3-10" tabindex="-1"></a><span class="do">## 'data.frame': 10000 obs. of 12 variables:</span></span>
<span id="cb3-11"><a href="#cb3-11" tabindex="-1"></a><span class="do">## $ lotr : int 0 0 0 0 0 0 0 1 0 0 ...</span></span>
<span id="cb3-12"><a href="#cb3-12" tabindex="-1"></a><span class="do">## $ premium : chr "Standard" "Standard" "Standard" "Premium" ...</span></span>
<span id="cb3-13"><a href="#cb3-13" tabindex="-1"></a><span class="do">## $ age : int 30 26 27 39 37 37 43 40 36 38 ...</span></span>
<span id="cb3-14"><a href="#cb3-14" tabindex="-1"></a><span class="do">## $ genre_likes : int 1 1 1 4 2 6 2 3 0 3 ...</span></span>
<span id="cb3-15"><a href="#cb3-15" tabindex="-1"></a><span class="do">## $ actor_likes : int 0 1 3 2 1 4 5 2 1 0 ...</span></span>
<span id="cb3-16"><a href="#cb3-16" tabindex="-1"></a><span class="do">## $ similar_user_ratings: num 0.65 0.37 0.17 0.31 0.69 0.5 0.68 0.92 0.57 0.47 ...</span></span>
<span id="cb3-17"><a href="#cb3-17" tabindex="-1"></a><span class="do">## $ mean_time_genre : num 153.6 84.5 129.1 102.6 237.9 ...</span></span>
<span id="cb3-18"><a href="#cb3-18" tabindex="-1"></a><span class="do">## $ user_time : chr "Morning" "Afternoon" "Evening" "Evening" ...</span></span>
<span id="cb3-19"><a href="#cb3-19" tabindex="-1"></a><span class="do">## $ user_day : chr "Saturday" "Wednesday" "Sunday" "Monday" ...</span></span>
<span id="cb3-20"><a href="#cb3-20" tabindex="-1"></a><span class="do">## $ user_device : chr "PC" "TV" "TV" "TV" ...</span></span>
<span id="cb3-21"><a href="#cb3-21" tabindex="-1"></a><span class="do">## $ fam_members : int 3 2 2 2 1 5 1 2 2 4 ...</span></span>
<span id="cb3-22"><a href="#cb3-22" tabindex="-1"></a><span class="do">## $ country : chr "FR" "DE" "US" "US" ...</span></span>
<span id="cb3-23"><a href="#cb3-23" tabindex="-1"></a></span>
<span id="cb3-24"><a href="#cb3-24" tabindex="-1"></a><span class="co"># Change variables from characters to factors</span></span>
<span id="cb3-25"><a href="#cb3-25" tabindex="-1"></a><span class="co"># These factors have no particular order to them</span></span>
<span id="cb3-26"><a href="#cb3-26" tabindex="-1"></a> <span class="co"># so I am happy to just leave these in alphabetical order</span></span>
<span id="cb3-27"><a href="#cb3-27" tabindex="-1"></a>netflix<span class="sc">$</span>premium <span class="ot"><-</span> <span class="fu">factor</span>(netflix<span class="sc">$</span>premium)</span>
<span id="cb3-28"><a href="#cb3-28" tabindex="-1"></a>netflix<span class="sc">$</span>user_device <span class="ot"><-</span> <span class="fu">factor</span>(netflix<span class="sc">$</span>user_device)</span>
<span id="cb3-29"><a href="#cb3-29" tabindex="-1"></a>netflix<span class="sc">$</span>country <span class="ot"><-</span> <span class="fu">factor</span>(netflix<span class="sc">$</span>country)</span>
<span id="cb3-30"><a href="#cb3-30" tabindex="-1"></a></span>
<span id="cb3-31"><a href="#cb3-31" tabindex="-1"></a><span class="co"># For time and day, however, I do want the natural order to be implemented</span></span>
<span id="cb3-32"><a href="#cb3-32" tabindex="-1"></a><span class="co"># To do so, we use the levels = argument, and specify the order we want</span></span>
<span id="cb3-33"><a href="#cb3-33" tabindex="-1"></a><span class="co"># by writing it out within the c() function.</span></span>
<span id="cb3-34"><a href="#cb3-34" tabindex="-1"></a><span class="co"># Note you cannot have typos here - be sure of your spelling</span></span>
<span id="cb3-35"><a href="#cb3-35" tabindex="-1"></a>netflix<span class="sc">$</span>user_time <span class="ot"><-</span> <span class="fu">factor</span>(netflix<span class="sc">$</span>user_time, </span>
<span id="cb3-36"><a href="#cb3-36" tabindex="-1"></a> <span class="at">levels =</span> <span class="fu">c</span>(<span class="st">"Morning"</span>, <span class="st">"Afternoon"</span>, <span class="st">"Evening"</span>))</span>
<span id="cb3-37"><a href="#cb3-37" tabindex="-1"></a>netflix<span class="sc">$</span>user_day <span class="ot"><-</span> <span class="fu">factor</span>(netflix<span class="sc">$</span>user_day, </span>
<span id="cb3-38"><a href="#cb3-38" tabindex="-1"></a> <span class="at">levels =</span> <span class="fu">c</span>(<span class="st">"Monday"</span>, <span class="st">"Tuesday"</span>, <span class="st">"Wednesday"</span>, </span>
<span id="cb3-39"><a href="#cb3-39" tabindex="-1"></a> <span class="st">"Thursday"</span>, <span class="st">"Friday"</span>, <span class="st">"Saturday"</span>, </span>
<span id="cb3-40"><a href="#cb3-40" tabindex="-1"></a> <span class="st">"Sunday"</span>))</span></code></pre></div>
<p>A bit of advice for visualising <span
class="math inline">\(Bernoulli\)</span> data; adding a little noise to
the x and y-axis values for each point (i.e. jittering) helps immensely
for visualising. To do so using <code>base</code> <code>R</code>, the
code would be:</p>
<pre><code>plot(jitter(my_data$y) ~ my_data$x)
# To add more noise:
plot(jitter(my_data$y, 20) ~ my_data$x)</code></pre>
<p>As in yesterday’s lecture and exercise, my figures below are produced
using <code>ggplot2</code> and are included purely for inspiration. In
today’s document, I am also using an additional package,
<code>patchwork</code>, to stitch together multiple figures into a
single panel. If you would like to use <code>ggplot2</code>, there is a
dedicated chapter that explains the underlying logic of
<code>ggplot2</code> in the Intro2R book.</p>
<div class="sourceCode" id="cb5"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb5-1"><a href="#cb5-1" tabindex="-1"></a><span class="co"># install.packages("ggplot2") # Run this line of code if you do not have ggplot installed</span></span>
<span id="cb5-2"><a href="#cb5-2" tabindex="-1"></a><span class="co"># Once installed, load the package</span></span>
<span id="cb5-3"><a href="#cb5-3" tabindex="-1"></a><span class="fu">library</span>(ggplot2)</span>
<span id="cb5-4"><a href="#cb5-4" tabindex="-1"></a></span>
<span id="cb5-5"><a href="#cb5-5" tabindex="-1"></a><span class="co"># I'll use another package called `patchwork` that helps combine ggplot figure together</span></span>
<span id="cb5-6"><a href="#cb5-6" tabindex="-1"></a><span class="co"># install.packages("patchwork") # Run this line of code if you do not have patchwork installed</span></span>
<span id="cb5-7"><a href="#cb5-7" tabindex="-1"></a><span class="co"># Once installed, load the package</span></span>
<span id="cb5-8"><a href="#cb5-8" tabindex="-1"></a><span class="fu">library</span>(patchwork)</span>
<span id="cb5-9"><a href="#cb5-9" tabindex="-1"></a></span>
<span id="cb5-10"><a href="#cb5-10" tabindex="-1"></a><span class="co"># Not needed but I'm downloading netflix icon to add to figure to give that corporate feel</span></span>
<span id="cb5-11"><a href="#cb5-11" tabindex="-1"></a><span class="co"># If you want to do this, you'll need to install the `png` package</span></span>
<span id="cb5-12"><a href="#cb5-12" tabindex="-1"></a><span class="fu">library</span>(png)</span>
<span id="cb5-13"><a href="#cb5-13" tabindex="-1"></a>logo <span class="ot"><-</span> <span class="fu">readPNG</span>(<span class="st">"./images/netflix.png"</span>)</span>
<span id="cb5-14"><a href="#cb5-14" tabindex="-1"></a></span>
<span id="cb5-15"><a href="#cb5-15" tabindex="-1"></a><span class="co"># Bar chart showing total number of views according to account type</span></span>
<span id="cb5-16"><a href="#cb5-16" tabindex="-1"></a>p1 <span class="ot"><-</span> <span class="fu">ggplot</span>(netflix) <span class="sc">+</span></span>
<span id="cb5-17"><a href="#cb5-17" tabindex="-1"></a> <span class="fu">geom_jitter</span>(<span class="fu">aes</span>(<span class="at">x =</span> premium, <span class="at">y =</span> lotr), </span>
<span id="cb5-18"><a href="#cb5-18" tabindex="-1"></a> <span class="at">width =</span> <span class="fl">0.3</span>, <span class="at">height =</span> <span class="fl">0.3</span>,</span>
<span id="cb5-19"><a href="#cb5-19" tabindex="-1"></a> <span class="at">colour =</span> <span class="st">"#E50914"</span>, <span class="at">alpha =</span> <span class="fl">0.05</span>) <span class="sc">+</span></span>
<span id="cb5-20"><a href="#cb5-20" tabindex="-1"></a> <span class="fu">labs</span>(<span class="at">x =</span> <span class="st">"Subscription type"</span>,</span>
<span id="cb5-21"><a href="#cb5-21" tabindex="-1"></a> <span class="at">y =</span> <span class="st">"Viewed</span><span class="sc">\n</span><span class="st">Lord of The Rings"</span>) <span class="sc">+</span> </span>
<span id="cb5-22"><a href="#cb5-22" tabindex="-1"></a> <span class="fu">theme_classic</span>() <span class="sc">+</span></span>
<span id="cb5-23"><a href="#cb5-23" tabindex="-1"></a> <span class="fu">annotation_raster</span>(logo, </span>
<span id="cb5-24"><a href="#cb5-24" tabindex="-1"></a> <span class="at">xmin =</span> <span class="fl">0.5</span>, <span class="at">xmax =</span> <span class="dv">1</span>,</span>
<span id="cb5-25"><a href="#cb5-25" tabindex="-1"></a> <span class="at">ymin =</span> <span class="dv">4000</span>, <span class="at">ymax =</span> <span class="dv">6000</span>)</span>
<span id="cb5-26"><a href="#cb5-26" tabindex="-1"></a></span>
<span id="cb5-27"><a href="#cb5-27" tabindex="-1"></a><span class="co"># A scatterplot for lotr ~ age, jittered</span></span>
<span id="cb5-28"><a href="#cb5-28" tabindex="-1"></a>p2 <span class="ot"><-</span> <span class="fu">ggplot</span>(netflix) <span class="sc">+</span></span>
<span id="cb5-29"><a href="#cb5-29" tabindex="-1"></a> <span class="fu">geom_jitter</span>(<span class="fu">aes</span>(<span class="at">x =</span> age, <span class="at">y =</span> lotr), </span>
<span id="cb5-30"><a href="#cb5-30" tabindex="-1"></a> <span class="at">width =</span> <span class="dv">0</span>, <span class="at">height =</span> <span class="fl">0.3</span>,</span>
<span id="cb5-31"><a href="#cb5-31" tabindex="-1"></a> <span class="at">colour =</span> <span class="st">"#E50914"</span>, <span class="at">alpha =</span> <span class="fl">0.05</span>) <span class="sc">+</span></span>
<span id="cb5-32"><a href="#cb5-32" tabindex="-1"></a> <span class="fu">scale_y_continuous</span>(<span class="at">labels =</span> <span class="cf">function</span>(y) <span class="fu">ifelse</span>(y <span class="sc">==</span> <span class="dv">0</span>, <span class="st">"No"</span>, <span class="fu">ifelse</span>(y <span class="sc">==</span> <span class="dv">1</span>, <span class="st">"Yes"</span>, y)),</span>
<span id="cb5-33"><a href="#cb5-33" tabindex="-1"></a> <span class="at">breaks =</span> <span class="fu">c</span>(<span class="dv">0</span>, <span class="dv">1</span>)) <span class="sc">+</span></span>
<span id="cb5-34"><a href="#cb5-34" tabindex="-1"></a> <span class="fu">labs</span>(<span class="at">x =</span> <span class="st">"User age"</span>,</span>
<span id="cb5-35"><a href="#cb5-35" tabindex="-1"></a> <span class="at">y =</span> <span class="st">"Viewed</span><span class="sc">\n</span><span class="st">Lord of The Rings"</span>) <span class="sc">+</span></span>
<span id="cb5-36"><a href="#cb5-36" tabindex="-1"></a> <span class="fu">theme_classic</span>()</span>
<span id="cb5-37"><a href="#cb5-37" tabindex="-1"></a></span>
<span id="cb5-38"><a href="#cb5-38" tabindex="-1"></a><span class="co"># A scatterplot for lotr ~ genre_likes, jittered</span></span>
<span id="cb5-39"><a href="#cb5-39" tabindex="-1"></a>p3 <span class="ot"><-</span> <span class="fu">ggplot</span>(netflix) <span class="sc">+</span></span>
<span id="cb5-40"><a href="#cb5-40" tabindex="-1"></a> <span class="fu">geom_jitter</span>(<span class="fu">aes</span>(<span class="at">x =</span> genre_likes, <span class="at">y =</span> lotr), </span>
<span id="cb5-41"><a href="#cb5-41" tabindex="-1"></a> <span class="at">width =</span> <span class="fl">0.3</span>, <span class="at">height =</span> <span class="fl">0.3</span>,</span>
<span id="cb5-42"><a href="#cb5-42" tabindex="-1"></a> <span class="at">colour =</span> <span class="st">"#E50914"</span>, <span class="at">alpha =</span> <span class="fl">0.05</span>) <span class="sc">+</span></span>
<span id="cb5-43"><a href="#cb5-43" tabindex="-1"></a> <span class="fu">scale_y_continuous</span>(<span class="at">labels =</span> <span class="cf">function</span>(y) <span class="fu">ifelse</span>(y <span class="sc">==</span> <span class="dv">0</span>, <span class="st">"No"</span>, <span class="fu">ifelse</span>(y <span class="sc">==</span> <span class="dv">1</span>, <span class="st">"Yes"</span>, y)),</span>
<span id="cb5-44"><a href="#cb5-44" tabindex="-1"></a> <span class="at">breaks =</span> <span class="fu">c</span>(<span class="dv">0</span>, <span class="dv">1</span>)) <span class="sc">+</span></span>
<span id="cb5-45"><a href="#cb5-45" tabindex="-1"></a> <span class="fu">labs</span>(<span class="at">x =</span> <span class="st">"Number of fantasy likes"</span>,</span>
<span id="cb5-46"><a href="#cb5-46" tabindex="-1"></a> <span class="at">y =</span> <span class="st">"Viewed</span><span class="sc">\n</span><span class="st">Lord of The Rings"</span>) <span class="sc">+</span></span>
<span id="cb5-47"><a href="#cb5-47" tabindex="-1"></a> <span class="fu">theme_classic</span>()</span>
<span id="cb5-48"><a href="#cb5-48" tabindex="-1"></a></span>
<span id="cb5-49"><a href="#cb5-49" tabindex="-1"></a><span class="co"># A scatterplot for lotr ~ actor_likes, jittered</span></span>
<span id="cb5-50"><a href="#cb5-50" tabindex="-1"></a>p4 <span class="ot"><-</span> <span class="fu">ggplot</span>(netflix) <span class="sc">+</span></span>
<span id="cb5-51"><a href="#cb5-51" tabindex="-1"></a> <span class="fu">geom_jitter</span>(<span class="fu">aes</span>(<span class="at">x =</span> actor_likes, <span class="at">y =</span> lotr), </span>
<span id="cb5-52"><a href="#cb5-52" tabindex="-1"></a> <span class="at">width =</span> <span class="fl">0.3</span>, <span class="at">height =</span> <span class="fl">0.3</span>,</span>
<span id="cb5-53"><a href="#cb5-53" tabindex="-1"></a> <span class="at">colour =</span> <span class="st">"#E50914"</span>, <span class="at">alpha =</span> <span class="fl">0.05</span>) <span class="sc">+</span></span>
<span id="cb5-54"><a href="#cb5-54" tabindex="-1"></a> <span class="fu">scale_y_continuous</span>(<span class="at">labels =</span> <span class="cf">function</span>(y) <span class="fu">ifelse</span>(y <span class="sc">==</span> <span class="dv">0</span>, <span class="st">"No"</span>, <span class="fu">ifelse</span>(y <span class="sc">==</span> <span class="dv">1</span>, <span class="st">"Yes"</span>, y)),</span>
<span id="cb5-55"><a href="#cb5-55" tabindex="-1"></a> <span class="at">breaks =</span> <span class="fu">c</span>(<span class="dv">0</span>, <span class="dv">1</span>)) <span class="sc">+</span></span>
<span id="cb5-56"><a href="#cb5-56" tabindex="-1"></a> <span class="fu">labs</span>(<span class="at">x =</span> <span class="st">"Number of films with same actor liked"</span>,</span>
<span id="cb5-57"><a href="#cb5-57" tabindex="-1"></a> <span class="at">y =</span> <span class="st">"Viewed</span><span class="sc">\n</span><span class="st">Lord of The Rings"</span>) <span class="sc">+</span></span>
<span id="cb5-58"><a href="#cb5-58" tabindex="-1"></a> <span class="fu">theme_classic</span>()</span>
<span id="cb5-59"><a href="#cb5-59" tabindex="-1"></a></span>
<span id="cb5-60"><a href="#cb5-60" tabindex="-1"></a><span class="co"># Using the patchwork package I stich p1 to p4 together into a single figure</span></span>
<span id="cb5-61"><a href="#cb5-61" tabindex="-1"></a>(p1 <span class="sc">+</span> p2) <span class="sc">/</span> (p3 <span class="sc">+</span> p4)</span></code></pre></div>
<p><img src="GLM_3_Bernoulli_exercise_solutions_files/figure-html/Q4.1-1.png" width="672" /></p>
<div class="sourceCode" id="cb6"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb6-1"><a href="#cb6-1" tabindex="-1"></a></span>
<span id="cb6-2"><a href="#cb6-2" tabindex="-1"></a><span class="co"># A scatterplot for lotr ~ similar_user_ratings, jittered</span></span>
<span id="cb6-3"><a href="#cb6-3" tabindex="-1"></a>p5 <span class="ot"><-</span> <span class="fu">ggplot</span>(netflix) <span class="sc">+</span></span>
<span id="cb6-4"><a href="#cb6-4" tabindex="-1"></a> <span class="fu">geom_jitter</span>(<span class="fu">aes</span>(<span class="at">x =</span> similar_user_ratings, <span class="at">y =</span> lotr), </span>
<span id="cb6-5"><a href="#cb6-5" tabindex="-1"></a> <span class="at">width =</span> <span class="dv">0</span>, <span class="at">height =</span> <span class="fl">0.3</span>,</span>
<span id="cb6-6"><a href="#cb6-6" tabindex="-1"></a> <span class="at">colour =</span> <span class="st">"#E50914"</span>, <span class="at">alpha =</span> <span class="fl">0.05</span>) <span class="sc">+</span></span>
<span id="cb6-7"><a href="#cb6-7" tabindex="-1"></a> <span class="fu">scale_y_continuous</span>(<span class="at">labels =</span> <span class="cf">function</span>(y) <span class="fu">ifelse</span>(y <span class="sc">==</span> <span class="dv">0</span>, <span class="st">"No"</span>, <span class="fu">ifelse</span>(y <span class="sc">==</span> <span class="dv">1</span>, <span class="st">"Yes"</span>, y)),</span>
<span id="cb6-8"><a href="#cb6-8" tabindex="-1"></a> <span class="at">breaks =</span> <span class="fu">c</span>(<span class="dv">0</span>, <span class="dv">1</span>)) <span class="sc">+</span></span>
<span id="cb6-9"><a href="#cb6-9" tabindex="-1"></a> <span class="fu">labs</span>(<span class="at">x =</span> <span class="st">"Proportion of similar users who liked LoTR"</span>,</span>
<span id="cb6-10"><a href="#cb6-10" tabindex="-1"></a> <span class="at">y =</span> <span class="st">"Viewed</span><span class="sc">\n</span><span class="st">Lord of The Rings"</span>) <span class="sc">+</span></span>
<span id="cb6-11"><a href="#cb6-11" tabindex="-1"></a> <span class="fu">theme_classic</span>()</span>
<span id="cb6-12"><a href="#cb6-12" tabindex="-1"></a></span>
<span id="cb6-13"><a href="#cb6-13" tabindex="-1"></a></span>
<span id="cb6-14"><a href="#cb6-14" tabindex="-1"></a><span class="co"># A scatterplot for lotr ~ mean_time_genre, jittered</span></span>
<span id="cb6-15"><a href="#cb6-15" tabindex="-1"></a>p6 <span class="ot"><-</span> <span class="fu">ggplot</span>(netflix) <span class="sc">+</span></span>
<span id="cb6-16"><a href="#cb6-16" tabindex="-1"></a> <span class="fu">geom_jitter</span>(<span class="fu">aes</span>(<span class="at">x =</span> mean_time_genre, <span class="at">y =</span> lotr), </span>
<span id="cb6-17"><a href="#cb6-17" tabindex="-1"></a> <span class="at">width =</span> <span class="fl">0.3</span>, <span class="at">height =</span> <span class="fl">0.3</span>,</span>
<span id="cb6-18"><a href="#cb6-18" tabindex="-1"></a> <span class="at">colour =</span> <span class="st">"#E50914"</span>, <span class="at">alpha =</span> <span class="fl">0.05</span>) <span class="sc">+</span></span>
<span id="cb6-19"><a href="#cb6-19" tabindex="-1"></a> <span class="fu">scale_y_continuous</span>(<span class="at">labels =</span> <span class="cf">function</span>(y) <span class="fu">ifelse</span>(y <span class="sc">==</span> <span class="dv">0</span>, <span class="st">"No"</span>, <span class="fu">ifelse</span>(y <span class="sc">==</span> <span class="dv">1</span>, <span class="st">"Yes"</span>, y)),</span>
<span id="cb6-20"><a href="#cb6-20" tabindex="-1"></a> <span class="at">breaks =</span> <span class="fu">c</span>(<span class="dv">0</span>, <span class="dv">1</span>)) <span class="sc">+</span></span>
<span id="cb6-21"><a href="#cb6-21" tabindex="-1"></a> <span class="fu">labs</span>(<span class="at">x =</span> <span class="st">"Mean time spent watching fantasy films"</span>,</span>
<span id="cb6-22"><a href="#cb6-22" tabindex="-1"></a> <span class="at">y =</span> <span class="st">"Viewed</span><span class="sc">\n</span><span class="st">Lord of The Rings"</span>) <span class="sc">+</span></span>
<span id="cb6-23"><a href="#cb6-23" tabindex="-1"></a> <span class="fu">theme_classic</span>()</span>
<span id="cb6-24"><a href="#cb6-24" tabindex="-1"></a></span>
<span id="cb6-25"><a href="#cb6-25" tabindex="-1"></a><span class="co"># A scatterplot for lotr ~ user_time, jittered</span></span>
<span id="cb6-26"><a href="#cb6-26" tabindex="-1"></a>p7 <span class="ot"><-</span> <span class="fu">ggplot</span>(netflix) <span class="sc">+</span></span>
<span id="cb6-27"><a href="#cb6-27" tabindex="-1"></a> <span class="fu">geom_jitter</span>(<span class="fu">aes</span>(<span class="at">x =</span> user_time, <span class="at">y =</span> lotr), </span>
<span id="cb6-28"><a href="#cb6-28" tabindex="-1"></a> <span class="at">width =</span> <span class="fl">0.3</span>, <span class="at">height =</span> <span class="fl">0.3</span>,</span>
<span id="cb6-29"><a href="#cb6-29" tabindex="-1"></a> <span class="at">colour =</span> <span class="st">"#E50914"</span>, <span class="at">alpha =</span> <span class="fl">0.05</span>) <span class="sc">+</span></span>
<span id="cb6-30"><a href="#cb6-30" tabindex="-1"></a> <span class="fu">scale_y_continuous</span>(<span class="at">labels =</span> <span class="cf">function</span>(y) <span class="fu">ifelse</span>(y <span class="sc">==</span> <span class="dv">0</span>, <span class="st">"No"</span>, <span class="fu">ifelse</span>(y <span class="sc">==</span> <span class="dv">1</span>, <span class="st">"Yes"</span>, y)),</span>
<span id="cb6-31"><a href="#cb6-31" tabindex="-1"></a> <span class="at">breaks =</span> <span class="fu">c</span>(<span class="dv">0</span>, <span class="dv">1</span>)) <span class="sc">+</span></span>
<span id="cb6-32"><a href="#cb6-32" tabindex="-1"></a> <span class="fu">labs</span>(<span class="at">x =</span> <span class="st">"Time of day"</span>,</span>
<span id="cb6-33"><a href="#cb6-33" tabindex="-1"></a> <span class="at">y =</span> <span class="st">"Viewed</span><span class="sc">\n</span><span class="st">Lord of The Rings"</span>) <span class="sc">+</span></span>
<span id="cb6-34"><a href="#cb6-34" tabindex="-1"></a> <span class="fu">theme_classic</span>()</span>
<span id="cb6-35"><a href="#cb6-35" tabindex="-1"></a></span>
<span id="cb6-36"><a href="#cb6-36" tabindex="-1"></a><span class="co"># A scatterplot for lotr ~ user_day, jittered</span></span>
<span id="cb6-37"><a href="#cb6-37" tabindex="-1"></a>p8 <span class="ot"><-</span> <span class="fu">ggplot</span>(netflix) <span class="sc">+</span></span>
<span id="cb6-38"><a href="#cb6-38" tabindex="-1"></a> <span class="fu">geom_jitter</span>(<span class="fu">aes</span>(<span class="at">x =</span> user_day, <span class="at">y =</span> lotr), </span>
<span id="cb6-39"><a href="#cb6-39" tabindex="-1"></a> <span class="at">width =</span> <span class="fl">0.3</span>, <span class="at">height =</span> <span class="fl">0.3</span>,</span>
<span id="cb6-40"><a href="#cb6-40" tabindex="-1"></a> <span class="at">colour =</span> <span class="st">"#E50914"</span>, <span class="at">alpha =</span> <span class="fl">0.05</span>) <span class="sc">+</span></span>
<span id="cb6-41"><a href="#cb6-41" tabindex="-1"></a> <span class="fu">scale_y_continuous</span>(<span class="at">labels =</span> <span class="cf">function</span>(y) <span class="fu">ifelse</span>(y <span class="sc">==</span> <span class="dv">0</span>, <span class="st">"No"</span>, <span class="fu">ifelse</span>(y <span class="sc">==</span> <span class="dv">1</span>, <span class="st">"Yes"</span>, y)),</span>
<span id="cb6-42"><a href="#cb6-42" tabindex="-1"></a> <span class="at">breaks =</span> <span class="fu">c</span>(<span class="dv">0</span>, <span class="dv">1</span>)) <span class="sc">+</span></span>
<span id="cb6-43"><a href="#cb6-43" tabindex="-1"></a> <span class="fu">labs</span>(<span class="at">x =</span> <span class="st">"Day of week"</span>,</span>
<span id="cb6-44"><a href="#cb6-44" tabindex="-1"></a> <span class="at">y =</span> <span class="st">"Viewed</span><span class="sc">\n</span><span class="st">Lord of The Rings"</span>) <span class="sc">+</span></span>
<span id="cb6-45"><a href="#cb6-45" tabindex="-1"></a> <span class="fu">theme_classic</span>() <span class="sc">+</span></span>
<span id="cb6-46"><a href="#cb6-46" tabindex="-1"></a> <span class="fu">theme</span>(<span class="at">axis.text.x =</span> <span class="fu">element_text</span>(<span class="at">angle =</span> <span class="dv">30</span>, <span class="at">vjust =</span> <span class="fl">0.5</span>, <span class="at">hjust =</span> <span class="fl">0.3</span>))</span>
<span id="cb6-47"><a href="#cb6-47" tabindex="-1"></a></span>
<span id="cb6-48"><a href="#cb6-48" tabindex="-1"></a></span>
<span id="cb6-49"><a href="#cb6-49" tabindex="-1"></a><span class="co"># Using the patchwork package I stich p1 to p4 together into a single figure</span></span>
<span id="cb6-50"><a href="#cb6-50" tabindex="-1"></a>(p5 <span class="sc">+</span> p6) <span class="sc">/</span> (p7 <span class="sc">+</span> p8)</span></code></pre></div>
<p><img src="GLM_3_Bernoulli_exercise_solutions_files/figure-html/Q4.1-2.png" width="672" /></p>
<div class="sourceCode" id="cb7"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb7-1"><a href="#cb7-1" tabindex="-1"></a></span>
<span id="cb7-2"><a href="#cb7-2" tabindex="-1"></a><span class="co"># A scatterplot for lotr ~ fam_members, jittered</span></span>
<span id="cb7-3"><a href="#cb7-3" tabindex="-1"></a>p9 <span class="ot"><-</span> <span class="fu">ggplot</span>(netflix) <span class="sc">+</span></span>
<span id="cb7-4"><a href="#cb7-4" tabindex="-1"></a> <span class="fu">annotation_raster</span>(logo,</span>
<span id="cb7-5"><a href="#cb7-5" tabindex="-1"></a> <span class="at">xmin =</span> <span class="fl">8.3</span>, <span class="at">xmax =</span> <span class="fl">9.6</span>,</span>
<span id="cb7-6"><a href="#cb7-6" tabindex="-1"></a> <span class="at">ymin =</span> <span class="sc">-</span><span class="fl">0.3</span>, <span class="at">ymax =</span> <span class="fl">0.3</span>) <span class="sc">+</span></span>
<span id="cb7-7"><a href="#cb7-7" tabindex="-1"></a> <span class="fu">geom_jitter</span>(<span class="fu">aes</span>(<span class="at">x =</span> fam_members, <span class="at">y =</span> lotr), </span>
<span id="cb7-8"><a href="#cb7-8" tabindex="-1"></a> <span class="at">width =</span> <span class="fl">0.3</span>, <span class="at">height =</span> <span class="fl">0.3</span>,</span>
<span id="cb7-9"><a href="#cb7-9" tabindex="-1"></a> <span class="at">colour =</span> <span class="st">"#E50914"</span>, <span class="at">alpha =</span> <span class="fl">0.05</span>) <span class="sc">+</span></span>
<span id="cb7-10"><a href="#cb7-10" tabindex="-1"></a> <span class="fu">scale_y_continuous</span>(<span class="at">labels =</span> <span class="cf">function</span>(y) <span class="fu">ifelse</span>(y <span class="sc">==</span> <span class="dv">0</span>, <span class="st">"No"</span>, <span class="fu">ifelse</span>(y <span class="sc">==</span> <span class="dv">1</span>, <span class="st">"Yes"</span>, y)),</span>
<span id="cb7-11"><a href="#cb7-11" tabindex="-1"></a> <span class="at">breaks =</span> <span class="fu">c</span>(<span class="dv">0</span>, <span class="dv">1</span>)) <span class="sc">+</span></span>
<span id="cb7-12"><a href="#cb7-12" tabindex="-1"></a> <span class="fu">labs</span>(<span class="at">x =</span> <span class="st">"Number of accounts"</span>,</span>
<span id="cb7-13"><a href="#cb7-13" tabindex="-1"></a> <span class="at">y =</span> <span class="st">"Viewed</span><span class="sc">\n</span><span class="st">Lord of The Rings"</span>) <span class="sc">+</span></span>
<span id="cb7-14"><a href="#cb7-14" tabindex="-1"></a> <span class="fu">theme_classic</span>()</span>
<span id="cb7-15"><a href="#cb7-15" tabindex="-1"></a> </span>
<span id="cb7-16"><a href="#cb7-16" tabindex="-1"></a><span class="co"># A scatterplot for lotr ~ user_device, jittered</span></span>
<span id="cb7-17"><a href="#cb7-17" tabindex="-1"></a>p10 <span class="ot"><-</span> <span class="fu">ggplot</span>(netflix) <span class="sc">+</span></span>
<span id="cb7-18"><a href="#cb7-18" tabindex="-1"></a> <span class="fu">geom_jitter</span>(<span class="fu">aes</span>(<span class="at">x =</span> user_device, <span class="at">y =</span> lotr), </span>
<span id="cb7-19"><a href="#cb7-19" tabindex="-1"></a> <span class="at">width =</span> <span class="fl">0.3</span>, <span class="at">height =</span> <span class="fl">0.3</span>,</span>
<span id="cb7-20"><a href="#cb7-20" tabindex="-1"></a> <span class="at">colour =</span> <span class="st">"#E50914"</span>, <span class="at">alpha =</span> <span class="fl">0.05</span>) <span class="sc">+</span></span>
<span id="cb7-21"><a href="#cb7-21" tabindex="-1"></a> <span class="fu">scale_y_continuous</span>(<span class="at">labels =</span> <span class="cf">function</span>(y) <span class="fu">ifelse</span>(y <span class="sc">==</span> <span class="dv">0</span>, <span class="st">"No"</span>, <span class="fu">ifelse</span>(y <span class="sc">==</span> <span class="dv">1</span>, <span class="st">"Yes"</span>, y)),</span>
<span id="cb7-22"><a href="#cb7-22" tabindex="-1"></a> <span class="at">breaks =</span> <span class="fu">c</span>(<span class="dv">0</span>, <span class="dv">1</span>)) <span class="sc">+</span></span>
<span id="cb7-23"><a href="#cb7-23" tabindex="-1"></a> <span class="fu">labs</span>(<span class="at">x =</span> <span class="st">"User device"</span>,</span>
<span id="cb7-24"><a href="#cb7-24" tabindex="-1"></a> <span class="at">y =</span> <span class="st">"Viewed</span><span class="sc">\n</span><span class="st">Lord of The Rings"</span>) <span class="sc">+</span></span>
<span id="cb7-25"><a href="#cb7-25" tabindex="-1"></a> <span class="fu">theme_classic</span>()</span>
<span id="cb7-26"><a href="#cb7-26" tabindex="-1"></a></span>
<span id="cb7-27"><a href="#cb7-27" tabindex="-1"></a><span class="co"># A scatterplot for lotr ~ country, jittered</span></span>
<span id="cb7-28"><a href="#cb7-28" tabindex="-1"></a>p11 <span class="ot"><-</span> <span class="fu">ggplot</span>(netflix) <span class="sc">+</span></span>
<span id="cb7-29"><a href="#cb7-29" tabindex="-1"></a> <span class="fu">geom_jitter</span>(<span class="fu">aes</span>(<span class="at">x =</span> country, <span class="at">y =</span> lotr), </span>
<span id="cb7-30"><a href="#cb7-30" tabindex="-1"></a> <span class="at">width =</span> <span class="fl">0.3</span>, <span class="at">height =</span> <span class="fl">0.3</span>,</span>
<span id="cb7-31"><a href="#cb7-31" tabindex="-1"></a> <span class="at">colour =</span> <span class="st">"#E50914"</span>, <span class="at">alpha =</span> <span class="fl">0.05</span>) <span class="sc">+</span></span>
<span id="cb7-32"><a href="#cb7-32" tabindex="-1"></a> <span class="fu">scale_y_continuous</span>(<span class="at">labels =</span> <span class="cf">function</span>(y) <span class="fu">ifelse</span>(y <span class="sc">==</span> <span class="dv">0</span>, <span class="st">"No"</span>, <span class="fu">ifelse</span>(y <span class="sc">==</span> <span class="dv">1</span>, <span class="st">"Yes"</span>, y)),</span>
<span id="cb7-33"><a href="#cb7-33" tabindex="-1"></a> <span class="at">breaks =</span> <span class="fu">c</span>(<span class="dv">0</span>, <span class="dv">1</span>)) <span class="sc">+</span></span>
<span id="cb7-34"><a href="#cb7-34" tabindex="-1"></a> <span class="fu">labs</span>(<span class="at">x =</span> <span class="st">"Country of registration"</span>,</span>
<span id="cb7-35"><a href="#cb7-35" tabindex="-1"></a> <span class="at">y =</span> <span class="st">"Viewed</span><span class="sc">\n</span><span class="st">Lord of The Rings"</span>) <span class="sc">+</span></span>
<span id="cb7-36"><a href="#cb7-36" tabindex="-1"></a> <span class="fu">theme_classic</span>()</span>
<span id="cb7-37"><a href="#cb7-37" tabindex="-1"></a></span>
<span id="cb7-38"><a href="#cb7-38" tabindex="-1"></a><span class="co"># Using the patchwork package I stich p1 to p4 together into a single figure</span></span>
<span id="cb7-39"><a href="#cb7-39" tabindex="-1"></a>(p10 <span class="sc">+</span> p11) <span class="sc">/</span> p9</span></code></pre></div>
<p><img src="GLM_3_Bernoulli_exercise_solutions_files/figure-html/Q4.1-3.png" width="672" /></p>
<p><br />
</p>
</div>
<div id="combine-into-a-model-that-can-answer-your-question."
class="section level3">
<h3>5. Combine into a model that can answer your question.</h3>
<p><br />
</p>
<p>Having gone through the previous steps, it’s now time to run our
model. Keep in mind that with this model, our objective is prediction,
so it would make sense to use all covariates at our disposal. Keep in
mind, with predictive modelling, we don’t care why a covariate
influences our response - we just want it to be repeatable, as this
means we can predict it again in the future.</p>
<p>Run the full model, using <code>glm()</code>.</p>
<ul>
<li>Hints:
<ul>
<li>Because <span class="math inline">\(Bernoulli\)</span> is really
just a special case of the <span class="math inline">\(Binomial\)</span>
distribution, we specify <code>family = binomial</code></li>
<li>Do we need to specify number of success and number of failures like
we did for <span class="math inline">\(Binomial\)</span> GLMs?</li>
<li>What is the default link function used by <span
class="math inline">\(Bernoulli\)</span> GLMs?</li>
</ul></li>
</ul>
<div class="sourceCode" id="cb8"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb8-1"><a href="#cb8-1" tabindex="-1"></a>mod1 <span class="ot"><-</span> <span class="fu">glm</span>(lotr <span class="sc">~</span> premium <span class="sc">+</span> age <span class="sc">+</span> genre_likes <span class="sc">+</span> actor_likes <span class="sc">+</span> similar_user_ratings <span class="sc">+</span></span>
<span id="cb8-2"><a href="#cb8-2" tabindex="-1"></a> mean_time_genre <span class="sc">+</span> user_time <span class="sc">+</span> user_day <span class="sc">+</span> user_device <span class="sc">+</span> fam_members <span class="sc">+</span> country, </span>
<span id="cb8-3"><a href="#cb8-3" tabindex="-1"></a> <span class="at">family =</span> <span class="fu">binomial</span>(<span class="at">link =</span> <span class="st">"logit"</span>), </span>
<span id="cb8-4"><a href="#cb8-4" tabindex="-1"></a> <span class="at">data =</span> netflix)</span></code></pre></div>
<p><br />
</p>
</div>
<div id="variable-selection" class="section level3">
<h3>6. Variable selection</h3>
<p>Given that in the lecture I showed an instance where AIC and variable
selection failed quite miserably, in this exercise, I wanted to include
a scenario where it performs very well.</p>
<p>Note that this discussion below is also true for the alternative
method we’ve shown to do variable selection: Likelihood Ratio Testing
(LRT). In fact, AIC ends up being the equivalent to LRT, where the
P-value threshold to 0.187 rather than 0.05 (for more information, see
Sutherland et al., <a
href="https://doi.org/10.1098/rspb.2023.1261%5D">2023</a>).</p>
<p>Given we didn’t have enough time to go into how AIC works during the
lectures, I’ll use this as an oppurtunity to explain it in a bit more
detail - starting with how it is calculated:</p>
<p><span class="math inline">\(AIC = 2K -
2ln(\mathcal{\hat{L}})\)</span></p>
<p>where <span class="math inline">\(K\)</span> is the number of
parameters in our model, <span class="math inline">\(ln\)</span> is the
natural log, and <span class="math inline">\(\mathcal{\hat{L}}\)</span>
is the maximum likelihood value for our given model.</p>
<p>Think back to the <span class="math inline">\(Poisson\)</span> GLM
lecture where we walked through how a likelihood value (<span
class="math inline">\(\mathcal{\hat{L}}\)</span>) was derived. In that
example, we were just trying different values for each parameter until
we hit the “sweet spot” - where our predicted values of number of birds
were closest to our observed values - using a given combination of
parameter values.</p>
<p>AIC uses this <span class="math inline">\(\mathcal{\hat{L}}\)</span>
value and asks; “For this likelihood value, do we really need all of
these parameters?” That’s why AIC includes the <span
class="math inline">\(2K\)</span> bit; Each parameter has a “cost”
associated with it of <span class="math inline">\(+2 AIC\)</span> units.
AIC is a fit versus cost trade-off measure. If the cost of including a
parameter in our model does not sufficiently improve our likelihood,
then it is considered to be uninformative and AIC tells us “it’s not
worth including in the model”. The conventional threshold is that if the
difference in AIC of two models is greater than, or equal to, <span
class="math inline">\(2 AIC\)</span> units, then we would conclude that
the lower scoring model has sufficient support as the most parsimonious
model.</p>
<p>(Note, the difference in AIC between two models is often referred to
in papers and some software as <span class="math inline">\(\Delta
AIC\)</span>, where <span class="math inline">\(\Delta\)</span> just
means difference from the lowest [i.e. “better”] AIC scoring model and
the model you are comparing it to.)</p>
<p>If we go through this process, fitting various models and removing
variables that increase AIC above 2 units, we are left with the most
parsimonious model. “The most parsimonious model” just means that the
model has the best (of the models considered) level of accuracy (or
prediction) and uses the fewest covariates as possible to do so. This is
why some statisticians (myself included) tend to think of AIC as a
“with-in sample predictive” tool. If you take this view, then AIC is
simply a tool that evaluates the predictive ability of a model;
Specifically, in predicting the data that you used to fit the model.
With this view of AIC, a parsimonious model simply represents a model
that has “good” predictive ability. Importantly, however, such models
are not intended to offer insights into causation. Indeed, for causal
inference, AIC can be quite harmful.</p>
<p>For example, consider the following scenario. We are interested in
the association between ice cream sales and shark attacks. There is no
direct causal relationship between these two variables. Instead, it is
the number of beach visitors that determines <em>both</em> ice cream
sales and shark attacks.</p>
<p>Regardless of causal mechanisms, finding a set of parameter values
that captures the association between ice cream sales and shark attacks
is trivial for <code>glm()</code>. This is what <code>glm()</code> is
designed to do after all - estimate associations. If we were to use AIC
on this hypothetical model it would strongly support including ice cream
sales in the model, because knowing how many ice creams have been sold
is pretty much telling us how many people were at the beach.</p>
<p>Test it out if your skeptical (and also run a Bernoulli GLM!):</p>