transcripts/en/AI_lectures_talks/srt/0X-NdPtFKq0.srt

﻿1
00:00:00,600 --> 00:00:05,600
And welcome to the second Royal 
Television Society and institution of 

2
00:00:05,600 --> 00:00:07,590
Engineering and technology public 
lecture.

3
00:00:07,830 --> 00:00:10,800
I'm Tim Davie.
I'm sharing tonight and you know when we

4
00:00:10,801 --> 00:00:15,801
conceived of this lecture series,
our aim was to hear from some of the 

5
00:00:15,801 --> 00:00:19,521
world's finest minds who work in the 
spaces that bridge cutting edge science 

6
00:00:19,741 --> 00:00:22,950
and technology with human creativity and
media.

7
00:00:23,820 --> 00:00:26,710
The amazing response,
and we were lucky to be there last year,

8
00:00:26,711 --> 00:00:31,711
whose those of us who saw Mike Lynch in 
full flow suggested we were onto 

9
00:00:31,711 --> 00:00:36,141
something and the excitement in the room
tonight speaks to the enduring appeal in

10
00:00:36,211 --> 00:00:41,211
my mind,
a public lectures in sparking our 

11
00:00:41,211 --> 00:00:42,930
imagination in many ways.
And I really believe this.

12
00:00:42,990 --> 00:00:47,130
We are lucky enough to be enjoying 
another golden age of advancement.

13
00:00:47,131 --> 00:00:52,131
The demands public engagement and debate
over 200 years.

14
00:00:52,561 --> 00:00:57,561
It was a rather precocious Humphry Davy 
that was wowing crowds and causing 

15
00:00:58,050 --> 00:01:03,050
carries traffic jams with sellout 
lectures on the nature of human progress

16
00:01:03,720 --> 00:01:05,220
and scientific knowledge.

17
00:01:05,760 --> 00:01:10,620
Noticeably as a friend of Coleridge,
he was fascinated by the intersection of

18
00:01:10,621 --> 00:01:14,190
the subjective and the technical and the
possibilities,

19
00:01:14,280 --> 00:01:16,770
the endless possibilities that this 
throws up.

20
00:01:17,310 --> 00:01:19,560
So tonight at this wonderful 
institution,

21
00:01:19,860 --> 00:01:23,100
we continue that tradition with our very
special speaker,

22
00:01:23,400 --> 00:01:26,220
Dennis Hassabis,
that's the best dentist,

23
00:01:26,250 --> 00:01:31,250
will talk for 35 minutes or so.
Then we'll open up for q and a and I'll 

24
00:01:31,250 --> 00:01:35,481
be conducting will have the lights up 
and were really up for a good old 

25
00:01:35,481 --> 00:01:36,450
discussion with you guys.
As you know,

26
00:01:36,840 --> 00:01:40,710
Dennis is acknowledged as a world 
leading thinker in the realm of Ai,

27
00:01:41,280 --> 00:01:46,280
which in recent years has become where 
the hottest topics dominating the media 

28
00:01:46,280 --> 00:01:48,990
and the imagination of the public.
You know my world.

29
00:01:48,991 --> 00:01:53,991
We run a recent and BBC survey which was
looking at which jobs are at risk of 

30
00:01:53,991 --> 00:01:58,971
computerization.
We attracted two point 3 million page 

31
00:01:58,971 --> 00:02:02,991
views.
People viewing that and seeing how safe 

32
00:02:02,991 --> 00:02:02,991
they were.

33
00:02:03,480 --> 00:02:07,110
Liz needs little introduction,
but I have to say his achievements,

34
00:02:07,111 --> 00:02:10,200
but even the highest achievers cv db in 
the shade,

35
00:02:10,780 --> 00:02:15,780
chess master,
30 World Games Championship winner five 

36
00:02:15,780 --> 00:02:17,160
times running.
Successful poker player,

37
00:02:17,161 --> 00:02:20,820
particularly impressive double first in 
computer science at Cambridge.

38
00:02:21,270 --> 00:02:26,270
A pioneering video games developer lead 
of numerous important pieces of research

39
00:02:27,360 --> 00:02:32,360
in the field of neuroscience.
Notably his landmark paper on the 

40
00:02:32,400 --> 00:02:37,400
similarities of how we shape memory and 
how we imagine the future was a landmark

41
00:02:38,491 --> 00:02:43,491
and breakthrough piece of research and 
then of course founder of deep mind in 

42
00:02:43,491 --> 00:02:47,511
2011,
which was then sold to Google in 2014 

43
00:02:47,511 --> 00:02:51,810
with Dennis becoming vp engineering with
special responsibility for ai.

44
00:02:52,920 --> 00:02:56,760
Deep mind has set out a goal of solving 
intelligence.

45
00:02:56,840 --> 00:03:01,840
A humble objective specifically dennis 
just said he is involved in building 

46
00:03:01,901 --> 00:03:05,890
something that can expect the unexpected
gracefully.

47
00:03:07,060 --> 00:03:10,660
I think that's probably a great brief 
for an audience of a public lecture.

48
00:03:11,210 --> 00:03:12,460
Dennis,
the floor is yours.

49
00:03:20,550 --> 00:03:25,550
Well,
thanks very much tim for that very 

50
00:03:25,550 --> 00:03:27,411
generous introduction.
So it's a real pleasure to be here and 

51
00:03:27,411 --> 00:03:30,641
I'm giving this lecture and I'm going to
talk about artificial intelligence and 

52
00:03:31,260 --> 00:03:33,600
its impact on the future.
In fact,

53
00:03:33,601 --> 00:03:36,120
it could be the relatively near term 
future.

54
00:03:37,920 --> 00:03:42,920
So Ai is really the science of making 
machines smart and I got into ai firstly

55
00:03:45,151 --> 00:03:50,130
through the medium of games and Games 
started for me with chess.

56
00:03:50,340 --> 00:03:55,340
As Tim said,
I started playing chess when I was very 

57
00:03:55,340 --> 00:03:57,651
young at the age of four and I think if 
you play chess seriously from such a 

58
00:03:57,651 --> 00:03:59,220
young age and you're quite an 
introspective kid,

59
00:03:59,221 --> 00:04:04,221
which I was,
then you start thinking a lot about how 

60
00:04:04,221 --> 00:04:05,310
is it that your brain is coming up with 
these moves,

61
00:04:05,311 --> 00:04:06,710
these ideas,
um,

62
00:04:06,840 --> 00:04:09,450
that allow you to play this game and win
these games.

63
00:04:09,840 --> 00:04:14,190
And I started thinking a lot about this 
as I got into my teenage years.

64
00:04:16,010 --> 00:04:17,840
And allied with that,
um,

65
00:04:17,960 --> 00:04:18,500
I,
uh,

66
00:04:18,560 --> 00:04:21,370
got into computing or actually bought my
first computer,

67
00:04:21,371 --> 00:04:26,371
is that x spectrum here,
48 k with some winnings from a chess 

68
00:04:26,371 --> 00:04:26,670
tournament.
And uh,

69
00:04:26,690 --> 00:04:30,170
when I was about eight years old and I 
started teaching myself how to program.

70
00:04:30,440 --> 00:04:33,440
And I think very early on in the 
engineers and the audience will,

71
00:04:33,500 --> 00:04:35,270
I think,
resonate with this.

72
00:04:35,450 --> 00:04:40,450
I sort of realized I'm on an intuitive 
level that this competed a kind of 

73
00:04:40,820 --> 00:04:43,310
special type of machine.
You know,

74
00:04:43,311 --> 00:04:48,311
most machines like cars and planes,
they allow us to extend our physical 

75
00:04:48,311 --> 00:04:48,920
capabilities.
You know,

76
00:04:48,921 --> 00:04:52,070
cars allow us to move faster than we can
run planes allow us to fly,

77
00:04:52,340 --> 00:04:54,740
um,
but I think computers do that,

78
00:04:54,890 --> 00:04:56,810
but in the realm of our minds,
um,

79
00:04:56,870 --> 00:05:01,870
they really extend the capabilities of 
the brain and this really came clear to 

80
00:05:01,870 --> 00:05:06,781
me when I used to write my first 
programs and did sort of basic math 

81
00:05:06,781 --> 00:05:07,780
calculations and other things which,
um,

82
00:05:07,810 --> 00:05:12,810
it really struck me.
You could set something running 

83
00:05:12,810 --> 00:05:15,151
overnight and then go to sleep and then 
you'd wake up the next morning and your 

84
00:05:15,151 --> 00:05:18,100
computer will solve some problem for you
whilst you were asleep.

85
00:05:18,490 --> 00:05:21,100
So this felt like a really powerful in 
some ways,

86
00:05:21,101 --> 00:05:21,580
magical.
Yeah.

87
00:05:23,330 --> 00:05:28,330
Say My love of computers and my love of 
games obviously came together in a,

88
00:05:29,091 --> 00:05:30,930
in a kind of obvious way,
uh,

89
00:05:30,980 --> 00:05:35,980
in the designing of video games.
And actually this is one of the reasons 

90
00:05:35,980 --> 00:05:37,370
I accepted to do this lecture is I love 
the idea,

91
00:05:37,371 --> 00:05:42,371
as tim said,
of the confluence of bringing together 

92
00:05:42,371 --> 00:05:44,870
rts and the creative arts and the iet 
and sort of engineering.

93
00:05:45,180 --> 00:05:50,180
Um,
and that's why I got into commercial 

94
00:05:50,180 --> 00:05:50,180
video games.
Um,

95
00:05:50,180 --> 00:05:53,890
because at the time,
this is sort of like the early and mid 

96
00:05:53,890 --> 00:05:57,251
nineties and computer games are really 
pushing the cutting edge of engineering 

97
00:05:57,251 --> 00:06:01,340
and even the machines that were being 
built to run these games.

98
00:06:01,550 --> 00:06:04,040
So I remember the debates in the 
nineties about,

99
00:06:04,150 --> 00:06:09,150
you know,
Intel is bringing out their new pentium 

100
00:06:09,150 --> 00:06:09,150
processes and people were sort of 
saying,

101
00:06:09,150 --> 00:06:13,240
well,
how much more power do we need to run 

102
00:06:13,240 --> 00:06:13,310
our work processes and spreadsheets and 
um,

103
00:06:13,311 --> 00:06:18,311
you know,
haven't we got all the computing power 

104
00:06:18,311 --> 00:06:18,311
we need?

105
00:06:18,311 --> 00:06:18,380
And actually one of the answers was 
that,

106
00:06:18,381 --> 00:06:20,710
um,
if we wanted more and more realistic and

107
00:06:20,870 --> 00:06:25,870
complex games,
then we would require more and more 

108
00:06:25,870 --> 00:06:27,260
powerful computers with larger memory 
and things like graphics chips.

109
00:06:27,440 --> 00:06:32,440
So for a long while games were actually 
driving the development of a cutting 

110
00:06:32,541 --> 00:06:35,540
edge hardware.
And furthermore,

111
00:06:35,960 --> 00:06:40,960
the games that I used to sort of design 
and program all involved ai as a core 

112
00:06:41,871 --> 00:06:46,871
gameplay mechanic.
So probably my best known game was 

113
00:06:46,871 --> 00:06:49,370
called theme park and uh,
which is some screenshots of it has came

114
00:06:49,371 --> 00:06:54,371
out in 94 and was very successful.
And it was actually the first game of 

115
00:06:54,441 --> 00:06:57,020
its type.
So the idea here was that,

116
00:06:57,021 --> 00:07:02,021
um,
you designed your own Disney world and 

117
00:07:02,021 --> 00:07:04,460
thousands of little people would come in
to your Disney world and kind of play on

118
00:07:04,461 --> 00:07:08,570
your rides and how enjoyable they 
thought your theme park was,

119
00:07:08,780 --> 00:07:09,530
would,
um,

120
00:07:09,560 --> 00:07:12,470
sort of have an impact on their emotions
and how happy they were.

121
00:07:12,620 --> 00:07:16,310
And then that fed into them and 
economics model about how much you could

122
00:07:16,311 --> 00:07:18,170
charge them for the hamburgers and the 
balloons.

123
00:07:18,320 --> 00:07:20,220
So the better design your,
um,

124
00:07:20,270 --> 00:07:21,890
thing park was,
um,

125
00:07:21,920 --> 00:07:23,640
the more money it made,
and then that allows you,

126
00:07:23,641 --> 00:07:25,370
of course,
to expand the theme park further.

127
00:07:25,850 --> 00:07:26,550
So,
um,

128
00:07:26,600 --> 00:07:31,600
this game and actually another game 
called Sim city with the first sort of 

129
00:07:31,600 --> 00:07:34,301
games that had ai as a core game play 
component and really spawned a whole 

130
00:07:34,301 --> 00:07:36,920
genre of management simulation games as 
they're called.

131
00:07:37,430 --> 00:07:41,990
And one of the reasons these gains are 
so popular is that the ai adapted to the

132
00:07:41,991 --> 00:07:46,991
way the player played the game.
So that means that every single person 

133
00:07:46,991 --> 00:07:49,700
who played this game had a unique 
experience.

134
00:07:50,180 --> 00:07:50,660
And,
uh,

135
00:07:50,750 --> 00:07:53,300
people used to send in our member into 
magazines,

136
00:07:53,301 --> 00:07:58,301
gay magazines and write into us,
I'm showing what's the end state they 

137
00:07:58,341 --> 00:07:59,630
got their theme park into.

138
00:07:59,780 --> 00:08:04,780
And there was all these amazing designs 
that people are created that we had no 

139
00:08:04,780 --> 00:08:07,100
idea could be done even as the inventors
of this game.

140
00:08:07,510 --> 00:08:12,510
Um,
so that really struck a chord with me 

141
00:08:12,510 --> 00:08:12,510
when I was around 16,
17 years old when I wrote this game.

142
00:08:12,680 --> 00:08:14,720
And I'm thinking about,
you know,

143
00:08:14,750 --> 00:08:17,600
maybe if I devoted my career to ai 
advancing ai,

144
00:08:17,720 --> 00:08:20,030
um,
what an incredible technology that could

145
00:08:20,031 --> 00:08:25,031
be.
So then after having a career and 

146
00:08:25,031 --> 00:08:25,640
getting games and running my own games 
companies and things,

147
00:08:25,880 --> 00:08:30,290
um,
I then went back to academia to do a phd

148
00:08:30,650 --> 00:08:35,650
in neuroscience,
which I felt was another piece of the 

149
00:08:35,650 --> 00:08:36,470
puzzle that I needed before launching an
effort.

150
00:08:36,471 --> 00:08:41,471
Like deep mind.
I wanted to understand a bit more about 

151
00:08:41,471 --> 00:08:41,471
how the brain,
um,

152
00:08:41,471 --> 00:08:44,630
solved a tough problems like imagination
and memory.

153
00:08:44,810 --> 00:08:49,810
And I specifically pick those topics to 
do my phd on because those are things 

154
00:08:49,810 --> 00:08:54,671
that at least back into mid two 
thousands where were we will not very 

155
00:08:54,671 --> 00:08:59,250
good at doing in computer algorithms.
So I wanted to look at the way the brain

156
00:08:59,251 --> 00:09:00,390
sold.
Somebody is very,

157
00:09:00,391 --> 00:09:03,780
very tough problems that we didn't know 
yet how to imbue our machines with.

158
00:09:04,710 --> 00:09:08,080
And I'll come back to that towards the 
second half of my talk.

159
00:09:09,220 --> 00:09:13,090
So all of these different experiences 
then culminated in finally in setting up

160
00:09:13,091 --> 00:09:14,320
deep mind,
uh,

161
00:09:14,390 --> 00:09:16,730
in 2010 and really,
um,

162
00:09:16,810 --> 00:09:19,990
it's been a 20 year plus journey for me 
to get to this point,

163
00:09:20,210 --> 00:09:25,210
uh,
and have enough of what I thought were 

164
00:09:25,210 --> 00:09:27,241
the basic ingredients both on an 
algorithmic level but also in terms of 

165
00:09:27,241 --> 00:09:31,441
the founding scientific team and making 
those contacts to actually put together 

166
00:09:31,441 --> 00:09:33,250
something like the mind and,
um,

167
00:09:33,370 --> 00:09:36,970
and plausibly go after those big 
emission as solving intelligence.

168
00:09:38,460 --> 00:09:40,100
So another way we look at,
um,

169
00:09:40,210 --> 00:09:45,210
uh,
the company is as an Apollo program for 

170
00:09:45,210 --> 00:09:48,501
ai as sort of moonshot project that 
really focuses on the very ambitious 

171
00:09:48,751 --> 00:09:53,751
longterm goals.
And we've collected together a 100 more 

172
00:09:53,751 --> 00:09:55,530
than $100,
knew 150 now of the world's top research

173
00:09:55,531 --> 00:10:00,531
scientists in this area.
So I think the mind is by far now the 

174
00:10:00,531 --> 00:10:03,120
biggest collection of machine learning 
experts anywhere in the world.

175
00:10:04,430 --> 00:10:05,920
And another thing we're experimenting 
with,

176
00:10:05,921 --> 00:10:10,921
of course,
apart from trying to build ai is 

177
00:10:10,921 --> 00:10:12,260
actually a new ways to organize 
scientific endeavor.

178
00:10:12,620 --> 00:10:16,300
So what we've tried to do with deep mind
is really combined the best form,

179
00:10:16,470 --> 00:10:19,880
um,
silicon valley startups together with,

180
00:10:20,010 --> 00:10:22,340
uh,
the best parts of that you find in the,

181
00:10:22,341 --> 00:10:24,710
in the best academic institutes like 
Mit,

182
00:10:24,711 --> 00:10:25,730
ucl,
Cambridge,

183
00:10:25,731 --> 00:10:28,730
and so on,
and see if we can infuse that into a new

184
00:10:28,731 --> 00:10:33,731
hybrid way of doing science,
which is more productive and extremely 

185
00:10:33,731 --> 00:10:36,380
efficient,
but still allows for extreme creativity.

186
00:10:38,680 --> 00:10:40,500
So our mission then,
as Tim said,

187
00:10:40,600 --> 00:10:43,840
we articulate it in a kind of to step 
way.

188
00:10:43,930 --> 00:10:46,780
So firstly we talk about solving 
intelligence.

189
00:10:46,970 --> 00:10:48,730
Um,
and we use the word solve,

190
00:10:48,850 --> 00:10:52,270
which is a kind of ambiguous word there 
because actually what we mean,

191
00:10:52,271 --> 00:10:55,390
what we're interested in is,
is understanding natural intelligence.

192
00:10:56,230 --> 00:11:00,400
So the human mind.
But also recreating that on intelligence

193
00:11:00,401 --> 00:11:03,080
artificially.
And then step two,

194
00:11:03,290 --> 00:11:07,010
we want to use that technology to help 
us solve everything else.

195
00:11:07,340 --> 00:11:08,340
Now,
um,

196
00:11:08,420 --> 00:11:13,420
you know,
that might seem a little bit far 

197
00:11:13,420 --> 00:11:14,791
fetched,
possibly a little bit fanciful to some 

198
00:11:14,791 --> 00:11:14,791
of you,
um,

199
00:11:14,791 --> 00:11:18,130
but we really believe actually that step
to naturally follows on from step one.

200
00:11:18,320 --> 00:11:21,980
If you can solve intelligence,
and I hope by the end of this talk,

201
00:11:22,180 --> 00:11:23,030
uh,
you know,

202
00:11:23,031 --> 00:11:25,040
you'll,
you'll agree with this conjecture.

203
00:11:26,960 --> 00:11:30,230
So more prosaically,
how are we going to solve intelligence?

204
00:11:30,410 --> 00:11:35,410
Well,
what we're trying to do at deep mind is 

205
00:11:35,410 --> 00:11:37,160
to construct the world's first general 
purpose learning machine.

206
00:11:38,060 --> 00:11:41,090
And the key aspects of this are the word
general and learning.

207
00:11:41,390 --> 00:11:45,020
So we're at deep mind,
we're only interested in algorithms that

208
00:11:45,021 --> 00:11:50,021
learn for themselves,
so they learn automatically from raw 

209
00:11:50,021 --> 00:11:53,090
experience or war data,
so they're not preprogrammed in any way.

210
00:11:53,560 --> 00:11:57,250
So what we're talking about here is 
autonomous learning system.

211
00:11:58,930 --> 00:12:01,210
The second thing is this idea of 
generality.

212
00:12:01,540 --> 00:12:02,560
So,
um,

213
00:12:02,650 --> 00:12:04,930
what we're interested in is the same 
system.

214
00:12:05,110 --> 00:12:10,110
I'm actually being able to operate 
across a wide range of tasks and 

215
00:12:10,110 --> 00:12:12,160
environments out of the box with no 
reconfiguration,

216
00:12:12,940 --> 00:12:15,550
so of course we have an example of such 
a general learning system.

217
00:12:15,580 --> 00:12:20,140
It's the human mind where we're able to 
apply our minds to almost endless number

218
00:12:20,141 --> 00:12:24,850
of different tasks.
Now I should say most of ai today,

219
00:12:24,851 --> 00:12:28,240
although it's a huge buzzword right now 
and is very fashionable,

220
00:12:28,450 --> 00:12:32,650
most of it,
ai is not of this type of technology,

221
00:12:33,010 --> 00:12:38,010
so we call most ai actually internally 
at deep mind narrow ai and what we mean 

222
00:12:38,010 --> 00:12:42,541
by that is preprogrammed ai that has 
been built for in a bespoke way for one 

223
00:12:42,971 --> 00:12:47,971
specific task and actually most of the 
Ai we interact with everyday from Siri 

224
00:12:47,971 --> 00:12:52,711
on your phone to self driving cars is 
actually a of this preprogram type of Ai

225
00:12:55,250 --> 00:12:59,210
and what we're interested in is what we 
call artificial general intelligence.

226
00:12:59,420 --> 00:13:01,580
This idea of a general learning system

227
00:13:03,120 --> 00:13:05,880
and perhaps still the most famous and 
clearest example I can give her.

228
00:13:05,881 --> 00:13:09,820
This is the famous deepblue match 
against Garry Kasparov.

229
00:13:10,100 --> 00:13:15,100
Of course,
this was a watershed moment in ai when 

230
00:13:15,100 --> 00:13:19,161
in the late nineties,
IBM's deep blue beat Kasparov in a six 

231
00:13:19,161 --> 00:13:20,250
game chest match.
Um,

232
00:13:20,370 --> 00:13:22,650
but the interesting thing is I came away
from that match,

233
00:13:22,651 --> 00:13:27,450
actually more impressed by Garry 
Kasparov mind than the deep blue machine

234
00:13:27,720 --> 00:13:29,080
because,
um,

235
00:13:29,150 --> 00:13:34,150
you know,
of course it was an impressive 

236
00:13:34,150 --> 00:13:35,511
engineering feat,
but deep blue was programmed by an 

237
00:13:35,511 --> 00:13:39,321
amazing team of programmers along with a
bunch of chess grandmasters trying to 

238
00:13:39,321 --> 00:13:42,330
distill chest knowledge into an 
algorithmic sort of construct.

239
00:13:42,570 --> 00:13:47,570
And those programmers were directly 
programming in the sort of ideas and 

240
00:13:47,570 --> 00:13:49,830
solutions into the machine.
And of course,

241
00:13:49,890 --> 00:13:52,920
what that meant is that deep blue,
although it was very good at chess,

242
00:13:53,040 --> 00:13:56,010
it was no use for absolutely anything 
else,

243
00:13:56,220 --> 00:13:59,670
including strictly simpler things like,
for example,

244
00:13:59,671 --> 00:14:04,671
playing noughts and crosses which any 
chess grandmaster you could trivially 

245
00:14:04,671 --> 00:14:05,520
teach them how to play noughts and 
crosses.

246
00:14:05,730 --> 00:14:07,950
But obviously deep blue,
nothing that deep blue,

247
00:14:07,951 --> 00:14:10,690
new or in its code would help it with,
um,

248
00:14:10,890 --> 00:14:12,540
even something strictly simple like 
that,

249
00:14:12,720 --> 00:14:13,710
let alone,
um,

250
00:14:13,910 --> 00:14:18,910
uh,
other kinds of domains like speaking 

251
00:14:18,910 --> 00:14:20,931
languages or driving cause all these 
other things that of course Gary 

252
00:14:20,931 --> 00:14:20,931
Kasparov could do effortlessly.

253
00:14:22,930 --> 00:14:27,930
So instead of that,
we think about intelligence in the 

254
00:14:27,930 --> 00:14:28,210
framework of what's called reinforcement
learning.

255
00:14:28,840 --> 00:14:33,840
So I'm just going to illustrate what the
main basic parts of that in this little 

256
00:14:33,840 --> 00:14:36,430
cartoon diagram because it's important 
for what I'm going to show next in terms

257
00:14:36,431 --> 00:14:37,570
of the videos of the,
of the,

258
00:14:37,630 --> 00:14:41,620
of the algorithms working.
So you start off with your agent system.

259
00:14:41,810 --> 00:14:46,630
I'm represented by this little humanoid 
character and that agent finds itself in

260
00:14:46,631 --> 00:14:50,020
an environment which could be virtual or
real world.

261
00:14:50,021 --> 00:14:52,090
If it's real world,
the agent probably be a robot.

262
00:14:52,100 --> 00:14:57,100
If it's virtual,
the agent will be an Avatar and the 

263
00:14:57,100 --> 00:14:57,290
agent has some kind of goal that has 
been given to,

264
00:14:57,291 --> 00:15:00,020
that is trying to achieve in that 
environment.

265
00:15:00,830 --> 00:15:03,680
And the agent only interacts with the 
environment in two ways.

266
00:15:03,950 --> 00:15:06,380
One is that it gets observations through
it.

267
00:15:06,381 --> 00:15:11,381
Sensory operators,
observations about the world and we 

268
00:15:11,381 --> 00:15:14,531
mostly use vision at the moment,
but we're also looking to use other 

269
00:15:14,531 --> 00:15:14,531
sensory modalities soon.

270
00:15:14,540 --> 00:15:17,960
And those observations are always 
incomplete and noisy.

271
00:15:18,290 --> 00:15:20,540
So you never get full information about 
the world,

272
00:15:20,570 --> 00:15:24,310
unlike say a game of chess where it's a 
perfect state information.

273
00:15:24,320 --> 00:15:26,930
You see everything that's in the game 
world,

274
00:15:27,140 --> 00:15:27,770
uh,
in,

275
00:15:27,771 --> 00:15:29,120
in the real world.
Of course,

276
00:15:29,780 --> 00:15:34,780
you don't get to see all the information
and that one of the jobs of the agent 

277
00:15:34,780 --> 00:15:38,951
system is to build a,
as accurate a model as possible of the 

278
00:15:38,951 --> 00:15:41,810
environment out there based solely on 
these noisy,

279
00:15:41,870 --> 00:15:46,870
incomplete observations.
And the agent is doing this in real 

280
00:15:46,870 --> 00:15:46,870
time.
These,

281
00:15:46,870 --> 00:15:49,030
these observations are coming in every 
time step and it's,

282
00:15:49,080 --> 00:15:53,510
and the agent is continually updating 
its model of the world based on this new

283
00:15:53,511 --> 00:15:58,511
evidence that it gets.
And the second job of the agent is to 

284
00:15:58,511 --> 00:16:02,681
then pick a what action it should take,
what's the best action it can take in 

285
00:16:02,691 --> 00:16:05,420
that particular moment in time that will
guess best,

286
00:16:05,421 --> 00:16:07,270
get it towards its goal,
um,

287
00:16:07,730 --> 00:16:09,980
from the current situation that it finds
itself in.

288
00:16:10,370 --> 00:16:12,530
And once it's decided what the actions 
should be,

289
00:16:12,610 --> 00:16:15,110
an outpost that action action gets 
executed,

290
00:16:15,260 --> 00:16:17,600
and that then may drive a change in the 
environment,

291
00:16:17,601 --> 00:16:21,680
which will then drive a new observation.
And this goes round in a,

292
00:16:21,681 --> 00:16:23,570
um,
endless sort of cycle.

293
00:16:24,950 --> 00:16:29,390
Now this diagram is a very simple to 
sort of explain,

294
00:16:29,590 --> 00:16:32,090
actually hides an incredible amount of 
complexity.

295
00:16:32,300 --> 00:16:36,650
So we know that if you could solve all 
the problems behind the,

296
00:16:36,651 --> 00:16:38,360
um,
the underlie this diagram,

297
00:16:38,540 --> 00:16:43,540
this representation,
then that will be enough for true 

298
00:16:43,540 --> 00:16:45,860
artificial intelligence.
And we know that because this is the way

299
00:16:45,861 --> 00:16:47,990
that biological systems learn,
um,

300
00:16:48,080 --> 00:16:50,770
including humans and most mammals.
Um,

301
00:16:51,020 --> 00:16:56,020
and in fact in humans is the dopamine 
system that implements a form of 

302
00:16:56,020 --> 00:16:56,270
reinforcement learning.

303
00:16:58,850 --> 00:17:00,520
So we go,
we went onto,

304
00:17:00,560 --> 00:17:02,540
um,
test these kinds of systems,

305
00:17:02,680 --> 00:17:07,680
um,
and actually we chose to test the 

306
00:17:07,680 --> 00:17:10,061
intelligence of our systems.
I'm on computer games now,

307
00:17:10,850 --> 00:17:13,080
a true thinking machine,
uh,

308
00:17:13,100 --> 00:17:17,240
we believe would have to be embedded in 
a sensory motor data stream.

309
00:17:17,480 --> 00:17:22,480
Um,
you can't have true intelligence and 

310
00:17:22,480 --> 00:17:25,361
true thinking unless you have the 
ability to affect the world that you're 

311
00:17:25,361 --> 00:17:26,600
in.
And the ability to sense that world.

312
00:17:27,020 --> 00:17:29,110
And uh,
and so usually the,

313
00:17:29,120 --> 00:17:31,810
so this is called embodied cognition and
um,

314
00:17:31,820 --> 00:17:34,640
usually when people subscribe to this 
view of Ai,

315
00:17:34,850 --> 00:17:37,910
they normally start working on robots,
will robots.

316
00:17:38,030 --> 00:17:41,120
I'm based in obviously in real world 
environments,

317
00:17:41,930 --> 00:17:44,750
but robots are very tricky to use.
They're very expensive.

318
00:17:44,870 --> 00:17:49,870
They're very slow and they break down.
So if you talk to anyone who's who's 

319
00:17:49,870 --> 00:17:52,050
used or to try to develop robots,
your,

320
00:17:52,070 --> 00:17:57,070
your hair,
a lot of the work actually goes into 

321
00:17:57,070 --> 00:17:59,310
fixing the mechanics of the robot,
the motors and the sensors and so on.

322
00:17:59,490 --> 00:18:01,500
And actually we didn't want to be 
distracted by that.

323
00:18:01,501 --> 00:18:04,920
We wanted to focus on the intelligence 
algorithms themselves.

324
00:18:05,760 --> 00:18:10,020
So what we decided to use was video 
games in the first instance.

325
00:18:10,260 --> 00:18:15,260
And of course it's a little bit to do 
with my background where it came in 

326
00:18:15,260 --> 00:18:15,350
useful here in video games,
um,

327
00:18:15,540 --> 00:18:20,540
and use it.
And we purpose the games as a platform 

328
00:18:20,540 --> 00:18:23,490
for testing the intelligence and the 
capabilities of our Ai Algorithms.

329
00:18:24,710 --> 00:18:29,210
Now Games are really good because 
obviously you can run them in the cloud,

330
00:18:29,300 --> 00:18:32,000
you can run them much faster than real 
time,

331
00:18:32,570 --> 00:18:35,060
you can run millions of experiments in 
parallel,

332
00:18:35,250 --> 00:18:40,250
um,
and it's very easy to measure progress 

333
00:18:40,250 --> 00:18:41,780
because most games fortunately have game
scores,

334
00:18:41,870 --> 00:18:46,870
so you can see very conveniently if your
algorithmic tweaks are gaining you an 

335
00:18:47,091 --> 00:18:52,091
advantage and whether you're heading in 
the right direction based on the 

336
00:18:52,091 --> 00:18:56,321
performance in those environments.
And that's something that's very 

337
00:18:56,321 --> 00:18:59,741
important,
especially for a very long time mission 

338
00:18:59,741 --> 00:18:59,741
like we have,
um,

339
00:18:59,741 --> 00:19:02,000
and very ambitious mission is to be able
to break down a,

340
00:19:02,040 --> 00:19:07,040
an ambitious mission into smaller chunks
that are very easy to measure the 

341
00:19:07,040 --> 00:19:07,610
progress on.

342
00:19:09,490 --> 00:19:14,490
The other key thing about games is that 
obviously they were designed by other 

343
00:19:14,490 --> 00:19:18,301
people and other other engineering teams
and they weren't designed specifically 

344
00:19:18,301 --> 00:19:20,020
for ai testing.
And uh,

345
00:19:20,021 --> 00:19:25,021
so what that means is that you have to 
deal with all kinds of interesting 

346
00:19:25,021 --> 00:19:27,841
problems that you would never have dealt
with a designed yourself as an ai 

347
00:19:27,841 --> 00:19:28,990
designer.
Um,

348
00:19:29,050 --> 00:19:34,050
and I think that's actually makes sure 
that there isn't any bias in the types 

349
00:19:34,050 --> 00:19:38,461
of problems that you apply your ai to.
So one of the big problems of in ai 

350
00:19:38,461 --> 00:19:41,020
research that has been over the last few
decades is that generally speaking,

351
00:19:41,140 --> 00:19:46,140
is the ai designers that also designed 
the problems are and subconsciously 

352
00:19:46,140 --> 00:19:51,001
whether you like it or not,
you end up designing problems that you 

353
00:19:51,001 --> 00:19:51,310
know,
your air go isms are well suited to.

354
00:19:53,340 --> 00:19:58,340
So what we started off with was actually
Atari Games from the eighties,

355
00:19:58,500 --> 00:20:03,500
which were really the first iconic 
platform that had a lot of very popular 

356
00:20:03,570 --> 00:20:06,690
challenging games on it and we decided 
to start with that.

357
00:20:06,890 --> 00:20:10,920
And what we did is we started with an 
open source emulator for Atari Games and

358
00:20:10,921 --> 00:20:11,670
we,
um,

359
00:20:11,910 --> 00:20:16,910
uh,
a souped it up and made it more robust 

360
00:20:16,910 --> 00:20:19,551
and made it run faster.
And then we plugged it in our ai 

361
00:20:19,551 --> 00:20:19,551
algorithms into the system.

362
00:20:19,970 --> 00:20:24,970
Now I'm going to show you a couple of 
videos of the AI system working and 

363
00:20:24,970 --> 00:20:25,490
then.
But before I do that,

364
00:20:25,491 --> 00:20:28,460
I always wanted to explain to you what 
it is you're going to see.

365
00:20:29,600 --> 00:20:34,130
So the AI system here,
I'm only gets the raw pixels as inputs,

366
00:20:34,280 --> 00:20:37,160
so it's almost as if we'd set up a video
camera,

367
00:20:37,310 --> 00:20:39,770
a observing the screen,
and I'm,

368
00:20:39,830 --> 00:20:44,830
the only information that he gets is the
raw pixels so it doesn't know anything 

369
00:20:44,830 --> 00:20:44,830
about,
uh,

370
00:20:44,830 --> 00:20:48,100
what it's controlling.
It doesn't know what the of the game is.

371
00:20:48,220 --> 00:20:53,220
Um,
it doesn't know how to get points and 

372
00:20:53,220 --> 00:20:55,610
all it's been told is that it needs to,
it's goal is to maximize the score and 

373
00:20:55,610 --> 00:21:00,180
everything else is learned from scratch.
And then there's a sort of generality 

374
00:21:02,050 --> 00:21:07,050
component comes in again where we 
require a single system to play all the 

375
00:21:07,050 --> 00:21:11,241
different games out of the box.
And there's obviously dozens and dozens 

376
00:21:11,241 --> 00:21:12,390
of very different Atari Games.

377
00:21:13,390 --> 00:21:13,820
Okay.

378
00:21:13,930 --> 00:21:15,780
So the first thing you're going to show 
you is space invaders,

379
00:21:15,790 --> 00:21:18,450
probably the most iconic game that 
they're,

380
00:21:18,451 --> 00:21:19,510
you know,
there's ever been.

381
00:21:19,780 --> 00:21:23,560
And I'm going to show you sort of two 
parts of this video.

382
00:21:23,760 --> 00:21:24,830
Um,
so in the beginning,

383
00:21:24,831 --> 00:21:29,831
as I roll the video now you know,
you'll see what the agent looks like 

384
00:21:29,831 --> 00:21:31,480
when it first encounters this 
environment.

385
00:21:31,650 --> 00:21:36,650
Now,
as controlling the rocket at the bottom 

386
00:21:36,650 --> 00:21:38,101
of this screen,
obviously it's trying actions randomly 

387
00:21:38,101 --> 00:21:40,951
because it has no idea what it's 
supposed to be doing and it loses its 

388
00:21:40,951 --> 00:21:41,770
three lives almost immediately.
Now,

389
00:21:41,771 --> 00:21:46,771
if you leave the machine training 
overnight and you come back the next 

390
00:21:46,771 --> 00:21:46,771
day,
um,

391
00:21:46,771 --> 00:21:48,290
the,
the machine now is superhuman at,

392
00:21:48,870 --> 00:21:49,950
at the,
at the,

393
00:21:50,010 --> 00:21:52,600
at the game.
So every single shot it,

394
00:21:52,660 --> 00:21:54,430
it,
fires hit something,

395
00:21:55,470 --> 00:21:57,290
it can't be killed anymore.
Um,

396
00:21:57,310 --> 00:21:59,520
it's,
it's worked out that the pink mothership

397
00:21:59,530 --> 00:22:04,530
of the top of the screen is worth a lot 
of points that does these amazing 

398
00:22:04,530 --> 00:22:04,810
accurate shots.

399
00:22:05,050 --> 00:22:07,810
And you can see that the model is built 
of the world is,

400
00:22:07,860 --> 00:22:11,530
is extremely accurate.
So those you play space invaders back in

401
00:22:11,531 --> 00:22:13,000
the eighties wall,
remember that the,

402
00:22:13,020 --> 00:22:13,460
the,
the,

403
00:22:13,480 --> 00:22:14,920
as there's less of them,
they get faster.

404
00:22:14,921 --> 00:22:19,921
If you just watched the last shot,
you'll see that they sort of predicted 

405
00:22:19,921 --> 00:22:20,020
where,
um,

406
00:22:20,050 --> 00:22:25,050
where that is going to wear that term.
Space invaders going to end up and far 

407
00:22:25,050 --> 00:22:28,951
as the shortest shots ahead of time.
So I'm going to show you a second video 

408
00:22:28,951 --> 00:22:30,460
now,
which is the game of breakout.

409
00:22:30,580 --> 00:22:34,360
It's my favorite video where they show a
few more gradations of the agent getting

410
00:22:34,361 --> 00:22:36,600
better and more capable.
Um,

411
00:22:36,601 --> 00:22:39,160
so in this game,
the agent is controlling the,

412
00:22:39,180 --> 00:22:40,870
the,
the pink bat and ball.

413
00:22:41,020 --> 00:22:44,860
And the aim of the game is to break 
through this rainbow colored brick wall,

414
00:22:44,920 --> 00:22:45,700
brick by brick.

415
00:22:47,120 --> 00:22:49,220
So to start off with,
after 100 games,

416
00:22:49,430 --> 00:22:50,580
um,
the,

417
00:22:50,610 --> 00:22:52,730
the AI system has,
you know,

418
00:22:52,731 --> 00:22:57,731
it's not very good.
You can see it's missing the ball most 

419
00:22:57,731 --> 00:23:00,161
of the time,
but you can see maybe you can convince 

420
00:23:00,161 --> 00:23:02,201
yourself.
It's starting to get the hang of the 

421
00:23:02,201 --> 00:23:02,201
idea that it should be moving the bat 
towards the ball.

422
00:23:02,201 --> 00:23:03,530
Now,
after 300 games,

423
00:23:04,060 --> 00:23:09,060
you can see that the,
the system has now got pretty much as 

424
00:23:09,060 --> 00:23:12,431
good as any human complainants and 
almost always gets the ball back even 

425
00:23:12,431 --> 00:23:13,880
when it's coming back at very vertical 
angles.

426
00:23:14,240 --> 00:23:16,130
So we thought that was pretty cool,
but we thought,

427
00:23:16,131 --> 00:23:21,131
well,
what would happen if we left the agent 

428
00:23:21,131 --> 00:23:21,890
running for longer and playdoh another 
200 games?

429
00:23:22,010 --> 00:23:27,010
And then this unexpected thing happened,
it discovered the optimal strategy was 

430
00:23:27,010 --> 00:23:30,371
to dig a tunnel around the site and send
the ball around the back of the wall.

431
00:23:30,890 --> 00:23:31,870
And um,
you know,

432
00:23:32,000 --> 00:23:37,000
it's doing that again with sort of super
human accuracy in terms of the motor 

433
00:23:37,000 --> 00:23:38,700
control then and strategy.
I wonder,

434
00:23:38,730 --> 00:23:42,110
the funny things is,
is that although the researchers on that

435
00:23:42,240 --> 00:23:45,170
are amazing,
programmers and engineers,

436
00:23:45,930 --> 00:23:49,700
they're not so good at playing Atari 
Games so they didn't actually know about

437
00:23:49,701 --> 00:23:51,100
that strategy.
So it's,

438
00:23:51,110 --> 00:23:51,710
I think,
you know,

439
00:23:51,711 --> 00:23:56,711
an example of a system that you've 
created actually teaching you something 

440
00:23:56,711 --> 00:23:58,790
which is quite an a watershed moment for
us.

441
00:23:59,510 --> 00:24:01,450
So efficient in that work.
Uh,

442
00:24:01,490 --> 00:24:06,490
we,
this was then fully published in nature 

443
00:24:06,490 --> 00:24:09,221
and the front cover earlier this year 
and we actually even released the code 

444
00:24:09,231 --> 00:24:10,130
as well.
So you can,

445
00:24:10,220 --> 00:24:12,290
you can have a look at that and play 
with that yourselves.

446
00:24:12,770 --> 00:24:16,490
Um,
so now we're moving onto three d games,

447
00:24:17,090 --> 00:24:21,650
go a robot simulators and of course we 
are interested in robots,

448
00:24:21,770 --> 00:24:25,460
but as a developed as a sort of 
application rather than as a development

449
00:24:25,670 --> 00:24:26,210
platform.

450
00:24:27,500 --> 00:24:30,500
Now I'll just show you one thing on the,
a three d stuff,

451
00:24:30,740 --> 00:24:32,540
um,
so we'll have a lot more announcements,

452
00:24:32,541 --> 00:24:35,000
new announcements to make her in the 
next year.

453
00:24:35,240 --> 00:24:36,170
Um,
but we,

454
00:24:36,171 --> 00:24:41,171
we,
I'll show this sort of run this little 

455
00:24:41,171 --> 00:24:41,171
video of,
um,

456
00:24:41,171 --> 00:24:43,930
the same agent that you saw playing the 
Atari Games actually now driving a 

457
00:24:43,930 --> 00:24:46,070
racing car around the track in a three d
game.

458
00:24:46,490 --> 00:24:47,210
So,
um,

459
00:24:47,211 --> 00:24:50,120
again,
the only inputs here are the pixels,

460
00:24:50,121 --> 00:24:54,020
the raw pixels and the steering wheel 
controls and um,

461
00:24:54,050 --> 00:24:57,560
it's learned just from your experience 
driving the car around how to drive,

462
00:24:57,730 --> 00:25:02,730
um,
and even do things like it's overtaking 

463
00:25:02,730 --> 00:25:03,020
the other cars that sort of 200 
kilometers an hour.

464
00:25:03,320 --> 00:25:04,460
Um,
and again,

465
00:25:04,461 --> 00:25:09,461
just from the raw pixel data.
So we're now moving towards much more 

466
00:25:09,891 --> 00:25:13,460
advanced three d environments where 
we're looking at May's problems,

467
00:25:13,630 --> 00:25:14,360
uh,
and,

468
00:25:14,361 --> 00:25:15,230
uh,
all kinds of,

469
00:25:15,260 --> 00:25:16,750
a much more complex,
uh,

470
00:25:16,810 --> 00:25:18,230
a path finding problems.

471
00:25:19,490 --> 00:25:24,490
Now,
I spoke about neuroscience at the 

472
00:25:24,490 --> 00:25:25,340
beginning of the talk and just want to 
come back and touch on that now.

473
00:25:25,341 --> 00:25:27,270
So we talk a lot about,
um,

474
00:25:27,380 --> 00:25:32,380
the Ai that we build it.
The mind has been neuroscience inspired 

475
00:25:32,380 --> 00:25:35,800
and in fact,
many of the other research areas that 

476
00:25:35,800 --> 00:25:38,201
we're looking at now,
we're looking to neuroscience very 

477
00:25:38,201 --> 00:25:38,350
closely for inspiration about,
um,

478
00:25:38,590 --> 00:25:40,280
uh,
for new types of algorithms,

479
00:25:40,460 --> 00:25:42,110
uh,
as to how the brain works.

480
00:25:42,350 --> 00:25:44,720
So we're looking at memory,
attention,

481
00:25:44,840 --> 00:25:49,310
concepts,
planning a navigation and imagination.

482
00:25:50,570 --> 00:25:52,010
Now because this is a,
you know,

483
00:25:52,011 --> 00:25:55,120
each one of those areas we'd probably 
need a whole talk to,

484
00:25:55,150 --> 00:26:00,150
to sort of get into.
So I'm just going to focus on 

485
00:26:00,150 --> 00:26:00,770
imagination because I think that's most 
relevant for the audience here.

486
00:26:01,070 --> 00:26:03,440
Um,
and it's also what I did for my phd.

487
00:26:03,980 --> 00:26:08,980
Now it turns out the imagination is 
quite dependent on an area of the brain 

488
00:26:08,980 --> 00:26:10,250
called the hippocampus,
which is actually here,

489
00:26:10,340 --> 00:26:14,150
this area and pink hair at the center of
your brain in this,

490
00:26:14,210 --> 00:26:15,800
um,
in this diagram of,

491
00:26:15,830 --> 00:26:16,670
of the human brain.

492
00:26:17,780 --> 00:26:19,970
Now the hippocampus,
it's very important,

493
00:26:19,971 --> 00:26:22,100
right?
It's quite a small part of the brain,

494
00:26:22,250 --> 00:26:27,250
a small brain region,
but it's very critical problem brain 

495
00:26:27,250 --> 00:26:27,530
region and it's been known for,
you know,

496
00:26:27,770 --> 00:26:31,400
more than 50 years now that if you 
damage the hippocampus,

497
00:26:31,600 --> 00:26:33,680
um,
then you um,

498
00:26:33,681 --> 00:26:38,681
how you become amnesic.
So it's well known that the hippocampus 

499
00:26:38,681 --> 00:26:42,230
is vital for episodic memory.
But what wasn't known was,

500
00:26:42,290 --> 00:26:44,640
what else was the hippocampus useful 
for?

501
00:26:44,800 --> 00:26:45,960
Um,
for example,

502
00:26:46,020 --> 00:26:51,020
was it involved with imagination now?
I suspected that it might be because 

503
00:26:53,250 --> 00:26:57,060
when I started reading the literature on
memory and hippocampus when I started my

504
00:26:57,061 --> 00:27:02,061
phd,
I sort of came across this literature 

505
00:27:02,061 --> 00:27:04,521
that was talking about memory as being a
reconstructive process rather than like 

506
00:27:04,561 --> 00:27:06,780
something like a video tape.
So,

507
00:27:06,781 --> 00:27:11,781
uh,
and that's actually the way that memory 

508
00:27:11,781 --> 00:27:11,781
works when you,
if you remember this lecture tomorrow,

509
00:27:11,781 --> 00:27:12,930
it won't,
it's not kind of like a store,

510
00:27:12,931 --> 00:27:14,910
like a video tape somewhere in your 
mind.

511
00:27:15,120 --> 00:27:20,120
Actually,
you will be combining it from all sorts 

512
00:27:20,120 --> 00:27:20,250
of components of things and experiences 
that you've had before.

513
00:27:20,251 --> 00:27:25,251
Other lectures,
perhaps other visits to the British 

514
00:27:25,251 --> 00:27:27,510
Museum as well as specific pieces of 
content that are to do with this evening

515
00:27:27,511 --> 00:27:32,511
specifically,
and what your brain does in the 

516
00:27:32,511 --> 00:27:32,511
hippocampus is always is reconstructing 
that,

517
00:27:32,511 --> 00:27:34,650
pulling all those parts together into a 
coherent whole,

518
00:27:34,770 --> 00:27:39,770
which then is recognized by the rest of 
your brain as a actually an episodic 

519
00:27:39,770 --> 00:27:40,740
memory.
So I was thinking,

520
00:27:40,741 --> 00:27:45,741
well,
if memory works as a reconstructive 

521
00:27:45,741 --> 00:27:47,751
process,
then if we think about imagination as 

522
00:27:47,751 --> 00:27:51,381
being a similar process,
but in this case is a constructive 

523
00:27:51,381 --> 00:27:54,020
process.
If we think of memory as trying to put 

524
00:27:54,020 --> 00:27:57,261
your components that you have together 
in a way that your brain thinks looks 

525
00:27:57,261 --> 00:27:59,050
and judges as familiar,
perhaps creativity's,

526
00:27:59,070 --> 00:28:04,070
that is the converse of that.
You're still bringing together those 

527
00:28:04,070 --> 00:28:06,651
components,
but now you're trying to create 

528
00:28:06,651 --> 00:28:07,380
something novel that actually your brain
judges as unfamiliar.

529
00:28:08,010 --> 00:28:09,060
Um,
so I was thinking,

530
00:28:09,061 --> 00:28:14,061
well,
if memory is heavily depend on the 

531
00:28:14,061 --> 00:28:16,071
hippocampus,
then maybe imagination is also a very 

532
00:28:16,071 --> 00:28:19,050
heavily dependent on the same brain 
structure and the same processes.

533
00:28:20,310 --> 00:28:23,490
So the way we decided to test this was 
actually by getting,

534
00:28:23,670 --> 00:28:24,730
um,
uh,

535
00:28:24,870 --> 00:28:29,870
going around the country to interview 
patients who had damage to the 

536
00:28:30,301 --> 00:28:35,301
hippocampus,
but only the hippocampus and there's 

537
00:28:35,301 --> 00:28:38,601
very rare sorts of diseases that cause 
that although things like Alzheimer's 

538
00:28:38,601 --> 00:28:41,730
actually do attack the hippocampus,
but also other brain structures,

539
00:28:41,880 --> 00:28:46,880
but what we needed were patients that 
had only specific damage only to this 

540
00:28:46,880 --> 00:28:50,721
one brain region and we tested those 
patients on their imaginative abilities 

541
00:28:50,820 --> 00:28:55,820
rather than their episodic memory.
And what we did is gave them fairly a 

542
00:28:55,820 --> 00:28:57,480
kind of simple,
imaginative task.

543
00:28:57,730 --> 00:29:01,080
We would give them a word cues like the 
following.

544
00:29:01,290 --> 00:29:04,680
Imagine you're lying on a white sandy 
beach in a beautiful tropical bay.

545
00:29:04,920 --> 00:29:09,420
Describe in as much detail as you can,
what you can see and hear and experience

546
00:29:09,421 --> 00:29:12,060
around you.
And what happened was,

547
00:29:12,061 --> 00:29:15,480
is when we compared and broke down their
descriptions,

548
00:29:15,570 --> 00:29:20,570
we scored it a in a very complex scoring
system to break that how rich that 

549
00:29:20,570 --> 00:29:25,311
description was.
And we compared it to age and educated 

550
00:29:25,311 --> 00:29:29,220
an ICU matched control subjects.
So they were actually in every way,

551
00:29:29,221 --> 00:29:33,270
except obviously they had intact and 
healthy hipaa campuses.

552
00:29:33,510 --> 00:29:35,240
We found that,
um,

553
00:29:35,410 --> 00:29:40,410
on our richness measure experiential 
index measure that the imagine scenes 

554
00:29:40,990 --> 00:29:45,990
that be hippocampal patients with 
describing what hugely impoverished 

555
00:29:45,990 --> 00:29:50,020
compared to the healthy controls.
And you can see that in these two.

556
00:29:50,110 --> 00:29:53,230
Then if you can see this laser pointer 
in these two bar charts.

557
00:29:53,300 --> 00:29:56,230
So on the left hand side here is the 
patients,

558
00:29:56,290 --> 00:30:01,290
the five patients,
and then here are the 10 match controls 

559
00:30:01,290 --> 00:30:04,951
to these patients.
And you can see this is the sort of 

560
00:30:04,951 --> 00:30:04,951
richness index,
if you like,

561
00:30:04,951 --> 00:30:09,810
of the described seeing and you can see 
that the patients are massively 

562
00:30:09,810 --> 00:30:11,950
deficient compared to the,
um,

563
00:30:12,100 --> 00:30:13,270
to compare to the controls.

564
00:30:14,860 --> 00:30:15,850
So,
um,

565
00:30:15,900 --> 00:30:20,900
so it seems indeed that the hippocampus 
is very important for imagining the 

566
00:30:20,900 --> 00:30:22,000
future.
And,

567
00:30:22,001 --> 00:30:27,001
uh,
actually the new scientists reported on 

568
00:30:27,001 --> 00:30:27,001
this,
uh,

569
00:30:27,001 --> 00:30:27,001
work,
uh,

570
00:30:27,001 --> 00:30:27,490
and it became quite a big study.
Um,

571
00:30:27,700 --> 00:30:32,050
that was also listed in sciences or as 
one of the big breakthroughs with 2007.

572
00:30:32,320 --> 00:30:37,320
And the new sites has had quite a nice 
headline talking about Hipaa camp or 

573
00:30:37,880 --> 00:30:41,050
patients being stuck in the present.
So the idea was,

574
00:30:41,080 --> 00:30:46,080
you know,
they know they can't remember the past 

575
00:30:46,080 --> 00:30:46,840
very well and now it turns out they 
can't imagine the future either.

576
00:30:47,050 --> 00:30:47,780
So,
um,

577
00:30:47,830 --> 00:30:49,240
but yeah,
if you were to talk to them,

578
00:30:49,360 --> 00:30:50,260
they seem,
you know,

579
00:30:50,290 --> 00:30:55,290
for a few minutes they would seem 
completely normal to you and you'd be 

580
00:30:55,290 --> 00:30:58,681
able to converse with them completely 
normally because they're processing the 

581
00:30:58,681 --> 00:31:02,071
present.
I'm in the same way that a healthy 

582
00:31:02,071 --> 00:31:02,071
person.

583
00:31:02,071 --> 00:31:02,071
What.

584
00:31:02,071 --> 00:31:07,070
So we then follow this up in Mri and we 
found out of course the hippocampus 

585
00:31:07,070 --> 00:31:09,150
doesn't support imagination on it's own.
It's part of a,

586
00:31:09,151 --> 00:31:14,151
it's a critical ct solar core part of a 
much larger brain network that includes 

587
00:31:14,250 --> 00:31:18,660
all sorts of other brain regions from 
the been or to the medial frontal Cortex

588
00:31:18,790 --> 00:31:23,790
to the medial temporal lobe,
lateral temporal cortex and parietal 

589
00:31:23,790 --> 00:31:27,261
cortex.
So all these regions reliably come on 

590
00:31:27,261 --> 00:31:30,741
when you scan somebody in a brain 
scanner and you'd get them to imagine 

591
00:31:30,741 --> 00:31:31,350
scenes.
Um,

592
00:31:31,380 --> 00:31:32,020
and,
and the,

593
00:31:32,040 --> 00:31:34,950
this network is the imagination network,
if you like.

594
00:31:36,440 --> 00:31:38,180
So they're much more recently I was 
thinking,

595
00:31:38,181 --> 00:31:39,080
well,
okay,

596
00:31:39,081 --> 00:31:41,360
so,
so this is how humans imagine.

597
00:31:41,500 --> 00:31:42,570
Um,
but um,

598
00:31:42,620 --> 00:31:45,170
what about animals?
Can they imagine as well,

599
00:31:45,260 --> 00:31:47,330
for example,
kind of rat imagine.

600
00:31:47,620 --> 00:31:48,660
Um,
uh,

601
00:31:48,680 --> 00:31:49,040
we,
you know,

602
00:31:49,041 --> 00:31:51,350
again,
we know that rats have memory,

603
00:31:51,470 --> 00:31:53,200
very good memory in fact.
Um,

604
00:31:53,390 --> 00:31:56,360
but can they do things like imagine the 
future?

605
00:31:57,320 --> 00:31:57,650
Yeah.

606
00:31:57,750 --> 00:32:02,340
So before I show you the study that we 
did to investigate that,

607
00:32:02,520 --> 00:32:07,020
I just need to take you through a couple
of things about that we know about rats,

608
00:32:07,180 --> 00:32:07,990
um,
um,

609
00:32:08,160 --> 00:32:11,790
and we know that rats can do.
So the first thing to tell you about his

610
00:32:11,791 --> 00:32:16,791
place sells.
Now I'm play cells are you can really 

611
00:32:16,791 --> 00:32:20,130
think of them as the GPS coordinates in 
a rat's brain about where they are.

612
00:32:21,000 --> 00:32:24,930
So if we imagine a box that a rat might 
be in a box.

613
00:32:24,931 --> 00:32:27,210
So here we're looking top down on this 
environment,

614
00:32:27,540 --> 00:32:30,750
then the rat might be roaming around 
this box.

615
00:32:31,050 --> 00:32:32,550
And uh,
you know,

616
00:32:32,551 --> 00:32:37,551
in these experiments you record directly
from the rat's brain while they're 

617
00:32:37,551 --> 00:32:38,520
roaming around.
And what you find is,

618
00:32:38,670 --> 00:32:39,640
is,
um,

619
00:32:39,800 --> 00:32:44,800
cells in the hippocampus fire,
I'm in specific places in the 

620
00:32:44,800 --> 00:32:46,190
environment.
So for example,

621
00:32:46,310 --> 00:32:47,180
a cell,
a cell,

622
00:32:47,181 --> 00:32:52,181
a might fire only it when the rat is 
traversing through this particular part 

623
00:32:52,730 --> 00:32:55,310
of the box environment.
Conversely,

624
00:32:55,340 --> 00:32:59,030
selby be my only fire in another part of
the environment.

625
00:32:59,870 --> 00:33:00,940
Now,
the,

626
00:33:01,150 --> 00:33:03,050
the amazing person who discovered this,
John O'keefe,

627
00:33:03,051 --> 00:33:06,260
who discovered these place cells in the 
seventies just around the corner at Ucl,

628
00:33:06,261 --> 00:33:09,140
won the Nobel prize for this last year.
Um,

629
00:33:09,290 --> 00:33:11,590
and I was lucky enough to have him as 
my,

630
00:33:11,591 --> 00:33:13,790
uh,
on my vibe of committee for my phd.

631
00:33:13,970 --> 00:33:18,170
So I got to know him well and um,
the entire rap literature very well.

632
00:33:18,440 --> 00:33:21,300
And what I started thinking about is 
since that discovery,

633
00:33:21,301 --> 00:33:24,350
a place cells,
people have found that actually,

634
00:33:24,460 --> 00:33:26,600
um,
sequences of place,

635
00:33:26,601 --> 00:33:28,360
cells fire in,
in,

636
00:33:28,390 --> 00:33:31,790
in kind of sequence when a wrap moves 
through an environment.

637
00:33:32,120 --> 00:33:34,660
So for example,
let's take a new environment here,

638
00:33:34,730 --> 00:33:39,730
a linear track.
So this is like a linear box and the 

639
00:33:39,730 --> 00:33:39,810
rats moving from,
um,

640
00:33:40,040 --> 00:33:42,560
from left to right here.
And what you find,

641
00:33:42,561 --> 00:33:44,750
if you record from the brains of these 
rats,

642
00:33:44,751 --> 00:33:47,960
is that place cells will fire in order 
a,

643
00:33:47,961 --> 00:33:48,260
b,
c,

644
00:33:48,261 --> 00:33:50,990
and d,
depending mimicking the way,

645
00:33:51,120 --> 00:33:56,120
um,
and matching the way that the rat is 

646
00:33:56,120 --> 00:33:57,020
moving through that environment.
And in the nineties,

647
00:33:57,140 --> 00:34:02,140
other people's found that when they 
recorded from the brains of these rats,

648
00:34:02,480 --> 00:34:07,480
while they were asleep,
after they had walked around and 

649
00:34:07,480 --> 00:34:12,071
navigate it in one of these amazing like
environments that the rats would replay 

650
00:34:13,760 --> 00:34:17,750
the trajectories they'd experienced in 
their wake session before.

651
00:34:17,900 --> 00:34:22,900
So you could,
you could really think about this as 

652
00:34:22,900 --> 00:34:25,301
rats dreaming,
so they would replay this trajectory of 

653
00:34:25,301 --> 00:34:27,740
Abcd,
but now this would be just,

654
00:34:27,860 --> 00:34:32,860
um,
they will be sleeping soundly and it'll 

655
00:34:32,860 --> 00:34:32,860
be just their,
their brains replaying this.

656
00:34:32,860 --> 00:34:33,950
And what's interesting is that the 
brain,

657
00:34:33,990 --> 00:34:36,350
so these rats actually replay these 
trajectories,

658
00:34:36,780 --> 00:34:41,480
an order of magnitude faster than they 
actually experienced it in real life.

659
00:34:41,930 --> 00:34:46,930
So if you think about dreaming as maybe 
helping the rats learn about the 

660
00:34:46,930 --> 00:34:49,140
environment that they're in,
then they actually,

661
00:34:49,410 --> 00:34:54,410
I'm learning from this much more 
efficiently than they can experience it 

662
00:34:54,410 --> 00:34:55,580
when they are awake.
So that's,

663
00:34:55,660 --> 00:35:00,660
uh,
shows that rats have memory and perhaps 

664
00:35:00,660 --> 00:35:03,431
they dream,
but it's not showing that they actually 

665
00:35:03,431 --> 00:35:05,720
imagine new experiences that they 
haven't experienced while they're awake.

666
00:35:06,200 --> 00:35:11,200
So we wanted to show that unequivocally 
and recently we published a study with 

667
00:35:11,200 --> 00:35:15,491
some colleagues of mine at Ucl in ie 
life that I think a unequivocally shows 

668
00:35:16,251 --> 00:35:17,480
that rats do imagine.

669
00:35:18,770 --> 00:35:20,690
So we designed this simple but,
um,

670
00:35:20,960 --> 00:35:24,560
I think quite elegant design to test 
this hypothesis out.

671
00:35:24,860 --> 00:35:27,650
So what we had here is a teammate is 
this time.

672
00:35:27,651 --> 00:35:29,810
So again,
we're looking top down on the,

673
00:35:29,830 --> 00:35:32,570
on the environment,
and the team is,

674
00:35:32,660 --> 00:35:34,370
has a barrier.
So,

675
00:35:34,371 --> 00:35:37,320
um,
and this barrier here stops the rat.

676
00:35:37,710 --> 00:35:40,460
The rat starts off in the stem of the 
teammates,

677
00:35:40,630 --> 00:35:45,630
um,
and it stops the rat moving to the arms 

678
00:35:45,630 --> 00:35:45,630
of the teammates,
but it's safe.

679
00:35:45,630 --> 00:35:49,230
Lou the barrier so the rat can see past 
to the arms and see what's on the arms.

680
00:35:50,640 --> 00:35:55,640
So the rise initially in the first 
session running up and down the stem of 

681
00:35:55,640 --> 00:35:59,601
the t mates.
And what we do to make the rat really 

682
00:35:59,601 --> 00:36:01,650
interested in the arms is we put some 
food,

683
00:36:01,860 --> 00:36:05,200
uh,
rice pellet on one of the arms here,

684
00:36:05,520 --> 00:36:10,520
depicted by this yellow dot and on the 
right hand arm and the rack can see this

685
00:36:11,850 --> 00:36:15,450
when it gets to the barrier,
but it can't reach the rice pellet.

686
00:36:15,660 --> 00:36:20,660
So obviously he's very motivated to 
think about a to try and get the rice 

687
00:36:20,660 --> 00:36:20,790
pellet,
but it can't get past the barrier.

688
00:36:21,750 --> 00:36:23,970
So then after it's experienced that 
environment for a while,

689
00:36:23,971 --> 00:36:26,370
we let the rat go to sleep and that 
will,

690
00:36:26,371 --> 00:36:31,371
of course,
we recording from the rat's brain while 

691
00:36:31,371 --> 00:36:31,371
this is going on.
Um,

692
00:36:31,371 --> 00:36:34,050
and then we wake up again and now we put
it back in the environment.

693
00:36:34,230 --> 00:36:35,570
But this time,
um,

694
00:36:35,730 --> 00:36:40,530
we remove that barrier.
So now it's free to run around the whole

695
00:36:40,531 --> 00:36:43,110
teammates.
So it does that happily,

696
00:36:43,170 --> 00:36:44,800
it moves around,
uh,

697
00:36:44,820 --> 00:36:45,400
the,
the,

698
00:36:45,430 --> 00:36:48,930
the stem and the arms,
both the left arm and the right arm,

699
00:36:49,120 --> 00:36:54,120
uh,
and it fully explored this whole 

700
00:36:54,120 --> 00:36:56,031
environment.
So obviously as I've just told you with 

701
00:36:56,031 --> 00:36:57,000
the place cells,
what we can do is we find that there are

702
00:36:57,001 --> 00:36:59,670
play cells,
let's say so a and sell DDI.

703
00:36:59,850 --> 00:37:03,900
That fire,
I'm on the arm section of the maze.

704
00:37:05,280 --> 00:37:10,280
Now what we can do is then go back to 
look at the data we collected when the 

705
00:37:10,410 --> 00:37:15,410
rat was asleep and see if the rack was 
imagining about those trajectories 

706
00:37:16,621 --> 00:37:20,670
towards the rice pellet before it ever 
had experienced it in reality.

707
00:37:20,910 --> 00:37:25,530
So they forget when it was sleeping in 
the rat had never experienced walking on

708
00:37:25,531 --> 00:37:27,980
this or a that.
Only seen that all.

709
00:37:28,590 --> 00:37:31,100
And what happens is we find our 
conjectures,

710
00:37:31,101 --> 00:37:34,650
we're sort of proven that actually you 
get,

711
00:37:34,740 --> 00:37:37,470
we find if we go back and analyze the 
sleep data,

712
00:37:37,620 --> 00:37:41,220
you get this replay or pre play if you 
like,

713
00:37:41,370 --> 00:37:43,430
of this trajectory,
abcd.

714
00:37:43,740 --> 00:37:47,700
And what's more is this is not just 
random pre play.

715
00:37:47,940 --> 00:37:52,940
You actually get more significantly more
prepared to the right hand arm then to 

716
00:37:52,940 --> 00:37:57,171
the left handle,
which is exactly what you would expect 

717
00:37:57,171 --> 00:37:57,171
if it's behaviorally consequential.

718
00:37:57,171 --> 00:37:58,090
Right?
So you can.

719
00:37:58,170 --> 00:38:00,360
I mean,
of course this is anthropomorphizing the

720
00:38:00,361 --> 00:38:05,361
rat,
but you could imagine the rats really 

721
00:38:05,361 --> 00:38:08,360
wanting to get to that rice pellet and 
is imagining plans of how could it get 

722
00:38:08,360 --> 00:38:08,360
there,
right?

723
00:38:08,360 --> 00:38:11,340
Almost imagining yourself walking to the
writer Paler and then eating it.

724
00:38:11,870 --> 00:38:16,870
And so,
and then it just dreaming about imagine 

725
00:38:16,870 --> 00:38:16,870
experiences.
Right?

726
00:38:16,870 --> 00:38:17,190
So,
um,

727
00:38:17,220 --> 00:38:19,230
so,
and that's really what's going on here.

728
00:38:19,231 --> 00:38:20,040
I think,
uh,

729
00:38:20,120 --> 00:38:25,120
and of course we're now going to look 
into this further and we have a number 

730
00:38:25,120 --> 00:38:26,310
of plans to look at a follow on studies 
from this,

731
00:38:26,440 --> 00:38:28,440
um,
with more complex environments.

732
00:38:29,760 --> 00:38:30,690
So,
you know,

733
00:38:30,691 --> 00:38:32,700
humans imagine rats.
Imagine.

734
00:38:32,820 --> 00:38:34,560
So what about,
um,

735
00:38:34,720 --> 00:38:36,250
uh,
machines.

736
00:38:36,490 --> 00:38:40,540
So this is something that's key 
imagination to planning for the future,

737
00:38:40,541 --> 00:38:43,000
making plans,
good plans about the future.

738
00:38:43,180 --> 00:38:46,360
So this is obviously something that we 
also want our machines to be able to do.

739
00:38:47,140 --> 00:38:47,900
So,
um,

740
00:38:48,040 --> 00:38:49,540
you know,
I've been titled this slide do,

741
00:38:49,660 --> 00:38:54,660
do androids dream of electric sheep,
which is of course is a reference to 

742
00:38:54,660 --> 00:38:57,901
Philip K,
Dick [inaudible] famous book and one of 

743
00:38:57,901 --> 00:38:57,901
my favorite,
one of my favorite films,

744
00:38:57,901 --> 00:38:58,840
blade runner.
And um,

745
00:38:59,050 --> 00:39:04,050
and this is really,
I'm just going to give you a very short 

746
00:39:04,050 --> 00:39:05,380
excerpt here with this video of which is
a little bit of an insight into the mind

747
00:39:05,381 --> 00:39:09,280
of the machine that you saw earlier 
playing space invaders.

748
00:39:09,490 --> 00:39:14,490
So I told you that one of the purposes 
of the agent system is to build a model 

749
00:39:14,490 --> 00:39:16,570
of the world so they can predict the 
future of what's going to happen in that

750
00:39:16,571 --> 00:39:17,950
game world.
And here,

751
00:39:18,190 --> 00:39:23,190
what I'm going to show in this sort of 
ten second video is the machine getting 

752
00:39:23,190 --> 00:39:24,490
an initial input from space invaders 
like this.

753
00:39:24,520 --> 00:39:29,520
This is the game position and then 
freely imagining or dreaming about what 

754
00:39:29,520 --> 00:39:33,710
might happen over the next 10 seconds.
So you can see it's,

755
00:39:33,750 --> 00:39:35,150
it's dream out moving.
It's,

756
00:39:35,210 --> 00:39:36,160
it's,
uh,

757
00:39:36,210 --> 00:39:41,210
the rocket and it's dreaming about 
getting points and shooting some of the 

758
00:39:41,210 --> 00:39:45,251
space invaders.
Now it's quite fuzzy because there's 

759
00:39:45,251 --> 00:39:45,251
uncertainty about what might happen in 
the world.

760
00:39:45,251 --> 00:39:46,280
It's all probabilistic.
So,

761
00:39:46,281 --> 00:39:51,281
um,
it's not as certain as seeing an actual 

762
00:39:51,281 --> 00:39:51,281
screen.
Um,

763
00:39:51,281 --> 00:39:54,140
but it's the beginnings,
I think of imagination based planning.

764
00:39:55,890 --> 00:39:57,690
Now I'm just going to end by talking a 
little bit about,

765
00:39:57,691 --> 00:40:00,720
so that's imagination.
So once you start thinking,

766
00:40:00,721 --> 00:40:02,430
well,
could machines have imagination?

767
00:40:02,431 --> 00:40:07,431
What about creativity?
I just something I get asked about all 

768
00:40:07,431 --> 00:40:08,400
the time and of course is very relevant 
to a lot of the work that people in this

769
00:40:08,401 --> 00:40:10,650
room do.
And um,

770
00:40:10,710 --> 00:40:15,710
you know,
I think we're a long way away from 

771
00:40:15,710 --> 00:40:17,880
machines being truly creative,
but I don't think it's impossible and I 

772
00:40:17,880 --> 00:40:22,731
think that um,
when we start to understand what this 

773
00:40:22,731 --> 00:40:25,191
process is,
this mysterious process of creativity 

774
00:40:25,191 --> 00:40:25,191
is,
um,

775
00:40:25,191 --> 00:40:28,820
I think it will become actually more 
obvious how to implement that in an 

776
00:40:28,820 --> 00:40:33,531
algorithm.
So I just want to show a couple of 

777
00:40:33,531 --> 00:40:33,531
little hints of things that might 
surprise you.

778
00:40:33,531 --> 00:40:37,930
Um,
so let's just take a picture of the 

779
00:40:38,670 --> 00:40:40,950
British Museum,
the front of this building and uh,

780
00:40:40,951 --> 00:40:45,510
if we then say to the machine,
and this is a new type of algorithm that

781
00:40:45,511 --> 00:40:50,511
was actually first a very recently 
invented at Max Planck Institute in 

782
00:40:50,521 --> 00:40:50,970
Germany.

783
00:40:50,970 --> 00:40:53,610
And then we've implemented our own 
version of this internally.

784
00:40:53,910 --> 00:40:58,910
And what you can do is you can give it a
autistic picture like this van Gough 

785
00:40:59,311 --> 00:41:02,370
picture and say you wanted to,
you want the,

786
00:41:02,390 --> 00:41:07,390
that photo redrawn in the style of Van 
Gough Rights and um,

787
00:41:07,730 --> 00:41:12,730
and actually so you ends up with outputs
that I like this where you can actually,

788
00:41:12,841 --> 00:41:15,210
it's not ready yet to be hung at the 
Louvre,

789
00:41:15,390 --> 00:41:20,390
but you can sort of start thinking it's 
pretty surprising when I saw these 

790
00:41:20,390 --> 00:41:23,850
things like how a actually coherent the 
output can be.

791
00:41:24,380 --> 00:41:25,750
And then,
you know,

792
00:41:25,830 --> 00:41:30,830
we can look at other examples.
So actually this is a concept piece of 

793
00:41:30,830 --> 00:41:32,190
concept art for um,
uh,

794
00:41:32,240 --> 00:41:37,130
a new google that's been built in,
in Silicon Valley and we give it a syrup

795
00:41:37,131 --> 00:41:40,820
painting and then we ask it's output in 
is one of my favorite ones.

796
00:41:40,970 --> 00:41:41,990
And you know,
producers,

797
00:41:41,991 --> 00:41:44,780
pretty good version of the,
um,

798
00:41:44,840 --> 00:41:46,120
of the original,
but in,

799
00:41:46,260 --> 00:41:47,420
in the start of syrup.

800
00:41:47,750 --> 00:41:50,000
And this isn't true creativity,
right in,

801
00:41:50,001 --> 00:41:55,001
in some senses is a politic because what
we're doing here with deconstructing 

802
00:41:55,001 --> 00:41:55,001
the,
the,

803
00:41:55,001 --> 00:41:59,780
the,
the features of both the original photo 

804
00:41:59,780 --> 00:41:59,780
and the,
um,

805
00:41:59,780 --> 00:42:02,570
the painting.
And then we're swapping those features,

806
00:42:02,720 --> 00:42:07,720
a overwriting of the photo features with
the painting features and that gives 

807
00:42:07,720 --> 00:42:07,720
this,
uh,

808
00:42:07,720 --> 00:42:09,440
this,
these kinds of outputs.

809
00:42:09,710 --> 00:42:14,710
But the surprising thing here is that 
there isn't much sort of what you would 

810
00:42:14,710 --> 00:42:18,400
regard as creativity here and yet you 
get these kind of very interesting 

811
00:42:18,400 --> 00:42:22,661
outputs.
So it may be that creativity isn't as 

812
00:42:22,661 --> 00:42:24,740
mysterious as it seems to us when we 
have ultimately find out what it is

813
00:42:26,280 --> 00:42:31,280
now I'm just going to end by talking a 
little bit about the bigger picture and 

814
00:42:31,280 --> 00:42:32,670
sort of the impact that ai might have in
the future.

815
00:42:32,970 --> 00:42:34,230
And uh,
so,

816
00:42:34,290 --> 00:42:39,290
you know,
I think some of the big problems are 

817
00:42:39,290 --> 00:42:40,671
facing us as a society or information 
overload and system complexity.

818
00:42:41,280 --> 00:42:42,240
So,
you know,

819
00:42:42,241 --> 00:42:45,450
everywhere we go now it daily use by 
inflammation.

820
00:42:45,630 --> 00:42:47,100
So things like,
um,

821
00:42:47,190 --> 00:42:49,080
obviously genomics,
big data in general,

822
00:42:49,081 --> 00:42:51,420
but in the world of TV know 
entertainment.

823
00:42:51,421 --> 00:42:54,720
I mean there's so many TV channels now 
and modes of watching things.

824
00:42:54,721 --> 00:42:57,360
How can you really find what it is that 
you're interested in?

825
00:42:57,750 --> 00:43:00,990
And personalization is one kind of 
technology that might help,

826
00:43:00,991 --> 00:43:05,991
but it doesn't really work because it's 
really based at the moment on quite 

827
00:43:05,991 --> 00:43:09,381
primitive sort of wisdom of the crowds,
collaborative filtering technology and 

828
00:43:09,381 --> 00:43:12,300
that doesn't give you unique 
recommendations that are unique to your,

829
00:43:12,360 --> 00:43:14,340
what I would call long tail of 
interests.

830
00:43:14,880 --> 00:43:19,880
And then in terms of system complexity,
the kinds of systems we would like to 

831
00:43:19,880 --> 00:43:19,880
master,
you know,

832
00:43:19,880 --> 00:43:21,210
climate disease,
energy,

833
00:43:21,211 --> 00:43:26,211
macroeconomics,
even particle physics are becoming so 

834
00:43:26,211 --> 00:43:29,301
complex now that even teams of the,
of the best and brightest human experts 

835
00:43:30,090 --> 00:43:35,090
are having difficulty comprehending the 
implications are of these systems and 

836
00:43:35,281 --> 00:43:37,080
actually making useful predictions about
them.

837
00:43:38,290 --> 00:43:43,290
So I think solving intelligence,
solving ai is potentially a kind of 

838
00:43:43,290 --> 00:43:44,830
metal solution to all these problems.
If we can solve intelligence,

839
00:43:44,980 --> 00:43:47,860
then maybe we can use it to help us,
um,

840
00:43:48,010 --> 00:43:49,900
as human experts,
uh,

841
00:43:49,930 --> 00:43:52,300
get a better handle on all these other 
systems.

842
00:43:52,570 --> 00:43:57,570
And my dream really the thing I'd like 
to use a January ai for is to build ai 

843
00:43:57,570 --> 00:44:00,550
scientists or to make ai assisted 
science possible.

844
00:44:02,480 --> 00:44:04,760
And of course,
if we have something this powerful,

845
00:44:04,970 --> 00:44:08,040
then obviously we need to think about 
the ethics of that,

846
00:44:08,041 --> 00:44:13,041
of the use of it.
And as with all new powerful 

847
00:44:13,041 --> 00:44:13,970
technologies.
And I think ai is no different from many

848
00:44:13,971 --> 00:44:15,770
other technologies in the past in this 
regard.

849
00:44:16,010 --> 00:44:21,010
We have to be a very cognizant about 
using these technologies ethically and 

850
00:44:21,010 --> 00:44:23,690
responsibly.
And although human level Ai,

851
00:44:23,691 --> 00:44:26,150
I think general ai is there many decades
away,

852
00:44:26,300 --> 00:44:29,820
I think we should start the debate now 
and when we,

853
00:44:29,920 --> 00:44:34,920
that's what we're doing,
both is our own internal ethics 

854
00:44:34,920 --> 00:44:36,210
committees,
but also by supporting academic work and

855
00:44:36,211 --> 00:44:41,211
academic conferences on these topics.
And then finally with a nod to 

856
00:44:41,430 --> 00:44:43,710
neuroscience,
I think building Ai,

857
00:44:43,830 --> 00:44:45,330
uh,
actually in this way,

858
00:44:45,331 --> 00:44:50,331
this neuroscience inspired way now help 
us better understand the mysteries and 

859
00:44:50,331 --> 00:44:53,400
the workings of our own minds.
And I think in the future,

860
00:44:53,530 --> 00:44:58,530
you know,
I think we're on part of the journey 

861
00:44:58,530 --> 00:45:00,021
we're on,
is that as we try to distill 

862
00:45:00,021 --> 00:45:00,021
intelligence into an algorithmic 
construct,

863
00:45:00,021 --> 00:45:02,490
if we then compare that with the 
capabilities of the human mind,

864
00:45:02,670 --> 00:45:07,670
I think we'll better understand about 
what's unique and special about our own 

865
00:45:07,670 --> 00:45:11,271
minds,
like dreaming creativity and perhaps 

866
00:45:11,271 --> 00:45:11,271
even the great consciousness question.

867
00:45:11,271 --> 00:45:12,180
Um,
and there's firemen said,

868
00:45:12,181 --> 00:45:15,120
one of my all time scientific heroes,
what I cannot build,

869
00:45:15,300 --> 00:45:17,760
I do not truly understand.
Thanks for listening.

870
00:45:24,100 --> 00:45:27,670
Thank you.

871
00:45:30,340 --> 00:45:32,560
That was brilliant.
If we get the lights up,

872
00:45:32,561 --> 00:45:36,970
I'm going to ask one question but not 
hog the limelight here.

873
00:45:36,971 --> 00:45:39,100
And I'll hand over to the audience.
We're going to have 20 minutes.

874
00:45:39,101 --> 00:45:44,101
So I'm having have a little pause.
Have a think about what you want to ask 

875
00:45:44,101 --> 00:45:46,900
Dennis because they're probably provoked
a so much kind of thinking.

876
00:45:47,220 --> 00:45:49,300
One question I've got very simple one 
actually is.

877
00:45:50,990 --> 00:45:52,170
Well,
I was amazed by,

878
00:45:52,190 --> 00:45:53,610
was that what you,
what your details,

879
00:45:53,611 --> 00:45:54,970
your 20 year plan,
you know,

880
00:45:55,560 --> 00:45:58,240
get your chest skills sorted,
you know,

881
00:45:58,470 --> 00:46:00,750
in terms of those virtual environments 
in gaming.

882
00:46:00,751 --> 00:46:02,010
Then,
then you needed the nude,

883
00:46:02,310 --> 00:46:05,860
but that was the first 20 years and you 
just talked about decades away because I

884
00:46:05,870 --> 00:46:10,870
think people are pretty obsessed with 
taking that raw data in building the 

885
00:46:10,870 --> 00:46:12,630
picture of the environment where we're 
at Atari Games.

886
00:46:13,050 --> 00:46:15,000
But I knew that breakout trick,
but anyway,

887
00:46:15,880 --> 00:46:20,880
we're Atari Games and then the other end
of the spectrum is the AI scientists 

888
00:46:20,880 --> 00:46:21,880
taking raw.
What's,

889
00:46:21,940 --> 00:46:26,940
what.
So just give us a sense in the 

890
00:46:26,940 --> 00:46:29,580
generation you'll read what's a 
realistic moon landing in your term?

891
00:46:30,130 --> 00:46:33,300
Yeah.
I touched on some of those things.

892
00:46:33,301 --> 00:46:35,550
Um,
in that neuroscience slide of the things

893
00:46:35,551 --> 00:46:39,000
with big things I think are important to
solve that beyond like the Atari Games,

894
00:46:39,001 --> 00:46:44,001
of course,
we're sort of on the first rung of the 

895
00:46:44,001 --> 00:46:45,561
ladder that the Atari thing was 
significant because it was the first 

896
00:46:45,561 --> 00:46:46,470
time anyone built what we call an end to
end agent.

897
00:46:46,650 --> 00:46:50,400
So something that took data and then 
make decisions.

898
00:46:50,430 --> 00:46:52,890
And did that in one big cycle.
Um,

899
00:46:52,920 --> 00:46:57,920
obviously I think the next big 
breakthroughs will be kainate really 

900
00:46:57,920 --> 00:46:58,150
learn abstract concepts,
um,

901
00:46:58,220 --> 00:47:03,000
and go beyond just perceptual inputs and
have a wheel underlying understanding of

902
00:47:03,001 --> 00:47:05,070
the semantics of the world it finds 
itself in.

903
00:47:05,370 --> 00:47:06,240
So that's,
I think,

904
00:47:06,390 --> 00:47:07,350
um,
you know,

905
00:47:07,351 --> 00:47:08,890
for us is our big kind of thing.

906
00:47:09,350 --> 00:47:11,790
I mean,
is this going to be exponential like the

907
00:47:11,791 --> 00:47:14,070
way computers developed that we're gonna
be in 20 years,

908
00:47:14,071 --> 00:47:15,370
be bowled over by the,

909
00:47:15,700 --> 00:47:17,720
the speed of progression?
I think it's hard to

910
00:47:17,900 --> 00:47:20,060
because we've only got one of you and 
you're not going to be here forever.

911
00:47:20,960 --> 00:47:22,610
We want to know how much we can do.

912
00:47:23,890 --> 00:47:26,500
I think it's hard to predict because,
um,

913
00:47:26,540 --> 00:47:31,540
you know,
we need at least a dozen really huge 

914
00:47:31,540 --> 00:47:31,570
breakthroughs and uh,
I think to get all that way and,

915
00:47:31,571 --> 00:47:32,000
and,
and,

916
00:47:32,001 --> 00:47:37,001
and research breakthroughs are 
notoriously hard to putting the 

917
00:47:37,001 --> 00:47:37,001
timescales off.
So I think we'll,

918
00:47:37,001 --> 00:47:37,170
we'll have,
you know,

919
00:47:37,180 --> 00:47:39,730
several very surprising things over the 
next few years.

920
00:47:39,870 --> 00:47:41,350
Um,
but you know,

921
00:47:41,351 --> 00:47:43,230
as to how far we'll get all the way.
I think it's,

922
00:47:43,231 --> 00:47:48,231
it's too hard to say from here,
I'm being a personalized timescales for 

923
00:47:48,231 --> 00:47:48,231
that.

924
00:47:48,231 --> 00:47:52,460
Brilliant.
I'm going to open it up because it's 

925
00:47:52,460 --> 00:47:52,460
easy for me.
I could be here all night,

926
00:47:52,460 --> 00:47:53,450
but why don't we start there?
We'll get a mic to you.

927
00:47:53,490 --> 00:47:56,970
That gentleman there with me,
then we'll come down to the front of how

928
00:47:57,040 --> 00:47:58,260
many we've got going.

929
00:47:59,740 --> 00:48:00,960
Hmm.

930
00:48:03,670 --> 00:48:05,660
Tim Marshall.
If the squeamish,

931
00:48:05,670 --> 00:48:07,080
we'll just close their eyes for a 
second.

932
00:48:07,081 --> 00:48:09,450
A few weeks ago I was at a conference,
uh,

933
00:48:09,451 --> 00:48:14,451
where a robot was performing a prostate 
operation a more than just performing 

934
00:48:14,451 --> 00:48:19,131
the operation there,
could actually understand the tumor and 

935
00:48:19,131 --> 00:48:19,131
make a decision whether to proceed or 
not.

936
00:48:19,590 --> 00:48:22,890
Today we've seen in the news the 
challenges of providing healthcare.

937
00:48:23,460 --> 00:48:24,630
Uh,
where do you think,

938
00:48:24,631 --> 00:48:29,631
what role do you think ai can play in 
diagnosis and treatment in health 

939
00:48:29,631 --> 00:48:30,450
because,
uh,

940
00:48:30,480 --> 00:48:33,900
the way we practice medicine at the 
moment is a 19th century paradigm.

941
00:48:34,420 --> 00:48:36,060
So I'd be interested in your thoughts on
that.

942
00:48:36,150 --> 00:48:36,720
Yeah,
it's great.

943
00:48:36,721 --> 00:48:38,880
Actually,
a healthcare is actually one of the main

944
00:48:39,260 --> 00:48:41,340
application areas we're focusing on.
First,

945
00:48:41,490 --> 00:48:42,800
I think,
um,

946
00:48:42,960 --> 00:48:47,130
we could probably revolutionize the sort
of quality of the care and efficiency of

947
00:48:47,131 --> 00:48:48,280
it,
um,

948
00:48:48,380 --> 00:48:49,170
you know,
as you say,

949
00:48:49,171 --> 00:48:52,710
we're still using kind of 19th century 
methods and uh,

950
00:48:52,711 --> 00:48:53,670
I think,
uh,

951
00:48:53,700 --> 00:48:58,700
having this sort of latest information 
available in a digestible,

952
00:48:58,711 --> 00:49:01,770
actionable way to surgeons and gps and 
so on.

953
00:49:01,990 --> 00:49:02,760
Um,
you know,

954
00:49:02,820 --> 00:49:05,310
I think we'll must really help that 
whole,

955
00:49:05,370 --> 00:49:06,060
uh,
you know,

956
00:49:06,061 --> 00:49:11,061
the whole healthcare space.
So it's something we're looking to get 

957
00:49:11,061 --> 00:49:12,270
heavily involved with in the next few 
years.

958
00:49:13,760 --> 00:49:13,880
Okay.

959
00:49:15,490 --> 00:49:20,490
Why did you decide to publish the code 
and were you ever worried or concerned 

960
00:49:20,490 --> 00:49:22,940
that when the ethics that might get into
the wrong hands?

961
00:49:23,400 --> 00:49:28,400
Yeah,
I mean we try to be as open as possible 

962
00:49:28,400 --> 00:49:29,370
about what we're doing.
So we generally publish.

963
00:49:29,430 --> 00:49:32,880
I'm almost everything that we do and 
where we can,

964
00:49:32,881 --> 00:49:36,630
we do open source things as well.
So actually our neural network libraries

965
00:49:36,631 --> 00:49:38,050
called torch that we,
um,

966
00:49:38,070 --> 00:49:42,120
build our algorithms on top of that.
That's open source and uh,

967
00:49:42,121 --> 00:49:47,121
we felt and there was a demand for some 
of the nature of reviewers and editors 

968
00:49:47,121 --> 00:49:47,700
that,
um,

969
00:49:47,730 --> 00:49:52,730
you know,
it'd be nice if we could release our 

970
00:49:52,730 --> 00:49:52,730
code.
So we thought about it and we decided in

971
00:49:52,730 --> 00:49:54,570
that case that was fine.
Um,

972
00:49:54,600 --> 00:49:55,470
but,
uh,

973
00:49:55,471 --> 00:49:57,330
you know,
that that may not always be the case for

974
00:49:57,331 --> 00:50:02,331
the stuff we do a and we'll obviously 
have to consider that on a case by case 

975
00:50:02,331 --> 00:50:02,910
basis,
but in general where we can,

976
00:50:02,911 --> 00:50:07,911
we like to engage and support the 
general academic community and we think 

977
00:50:07,911 --> 00:50:11,460
it's important that knowledge is shared 
and I think that's the way that humanity

978
00:50:11,461 --> 00:50:13,380
can advance as quickly as possible.

979
00:50:15,290 --> 00:50:20,290
And the second road,
you're one of the largest brains on the 

980
00:50:20,290 --> 00:50:23,260
planet and I'm good now bought here and 
um,

981
00:50:23,580 --> 00:50:28,580
I'm gonna ask you that question,
which I'm sure you're expecting Stephen 

982
00:50:28,910 --> 00:50:28,910
Hawking

983
00:50:28,940 --> 00:50:33,940
thing that the concern about ai that um,
once you let the genie out of the 

984
00:50:34,011 --> 00:50:35,500
bottle,
um,

985
00:50:36,160 --> 00:50:41,160
we're all fucked.
What are you doing to try?

986
00:50:45,820 --> 00:50:46,900
This is the regular.

987
00:50:47,530 --> 00:50:48,370
Yes,
sure.

988
00:50:48,410 --> 00:50:49,310
I mean,
I,

989
00:50:49,400 --> 00:50:51,480
I've actually spoken,
um,

990
00:50:51,630 --> 00:50:56,630
uh,
I had a long chat with Stephen Hawking 

991
00:50:56,630 --> 00:50:58,250
about this a few months ago and um,
I think he was,

992
00:50:58,730 --> 00:50:59,340
well,
he was,

993
00:50:59,341 --> 00:51:01,300
I think he lives very enjoyable and I 
think he,

994
00:51:01,301 --> 00:51:02,730
he,
we spent hours together.

995
00:51:02,731 --> 00:51:04,350
We only spend half an hour,
but he was,

996
00:51:04,351 --> 00:51:09,351
he had so many questions and I think he 
was quite reassured after we talked 

997
00:51:09,351 --> 00:51:11,250
about how we will,
we specifically were approaching it.

998
00:51:11,580 --> 00:51:12,780
Um,
and I think,

999
00:51:12,781 --> 00:51:13,260
look,
you know,

1000
00:51:13,261 --> 00:51:15,360
there are big,
big issues here,

1001
00:51:15,361 --> 00:51:19,500
very big issues about um,
autonomous learning systems.

1002
00:51:19,501 --> 00:51:22,110
What goals should we give them?
What value system should we give them?

1003
00:51:22,111 --> 00:51:27,111
How can we make sure that,
that those are exactly what we want and 

1004
00:51:27,111 --> 00:51:28,320
there are very tough pieces of research 
that needs to be done.

1005
00:51:28,670 --> 00:51:30,780
Um,
there hasn't been much work done in that

1006
00:51:30,781 --> 00:51:35,781
area yet,
partly because there'd be no systems to 

1007
00:51:35,781 --> 00:51:35,781
really try this out on.

1008
00:51:35,781 --> 00:51:36,570
So it's all been thought experiments and
I think if you,

1009
00:51:36,571 --> 00:51:37,380
do,
you know,

1010
00:51:37,410 --> 00:51:42,410
mostly the people thinking about this 
and worrying about this to that extent 

1011
00:51:42,410 --> 00:51:45,860
or not in the AI field right there.
Either philosophers or there other very 

1012
00:51:45,860 --> 00:51:50,450
famous scientists or,
or industrialist but not actually 

1013
00:51:50,450 --> 00:51:51,780
working on ai themselves.
And if they were working,

1014
00:51:51,781 --> 00:51:54,810
I think they would see that the problems
are much more prosaic at the moment.

1015
00:51:55,200 --> 00:51:56,120
And it's easy,
I think,

1016
00:51:56,160 --> 00:51:58,920
to get carried away with science fiction
scenarios that are,

1017
00:51:58,950 --> 00:52:00,150
you know,
many decades away.

1018
00:52:00,420 --> 00:52:01,860
My,
I have confidence that,

1019
00:52:01,970 --> 00:52:06,970
um,
as we get bill more powerful systems 

1020
00:52:06,970 --> 00:52:09,470
will have much better ideas about the 
answers to these questions that I just 

1021
00:52:09,470 --> 00:52:10,680
talked about,
value systems and so on.

1022
00:52:10,860 --> 00:52:12,120
Uh,
and um,

1023
00:52:12,240 --> 00:52:17,240
you know,
mathematical proofs of empirical work 

1024
00:52:17,240 --> 00:52:17,240
that will,
um,

1025
00:52:17,240 --> 00:52:19,950
allow us to have much better idea of how
to keep these systems on the control

1026
00:52:20,230 --> 00:52:21,910
when I'm pleased to see that you've got 
the,

1027
00:52:21,911 --> 00:52:26,911
um,
the ethics committee that onboard and 

1028
00:52:26,911 --> 00:52:26,911
you're thinking about these issues 
issues,

1029
00:52:26,911 --> 00:52:29,730
but you have taken the Yankee dollar.
And I'm,

1030
00:52:30,010 --> 00:52:35,010
I am worried about this because you are 
so smart and I hope that everything you 

1031
00:52:35,010 --> 00:52:38,590
do actually improves the society route 
kills us off.

1032
00:52:39,090 --> 00:52:40,020
Well,
so do I.

1033
00:52:40,210 --> 00:52:40,930
But uh,
but,

1034
00:52:40,931 --> 00:52:41,690
uh,
um,

1035
00:52:41,920 --> 00:52:42,440
you know,
I,

1036
00:52:42,441 --> 00:52:43,540
I think ai,
you know,

1037
00:52:43,541 --> 00:52:46,720
it could be the greatest thing for 
humanity and the sense of if we build it

1038
00:52:46,721 --> 00:52:51,721
right,
we'll solve all these big issues that 

1039
00:52:51,721 --> 00:52:56,520
corporate responsibility versus showing 
through.

1040
00:52:56,621 --> 00:52:58,270
I think that's a big one,
isn't it,

1041
00:52:58,271 --> 00:53:00,820
in terms of where power lies in.
Sure.

1042
00:53:00,850 --> 00:53:03,280
I mean I should probably make a little 
bit about that.

1043
00:53:03,281 --> 00:53:07,060
So obviously we spent a long time doing 
due diligence ourselves on Google.

1044
00:53:07,090 --> 00:53:12,090
Right.
And we had a lot of other options 

1045
00:53:12,090 --> 00:53:13,531
including stay independent and we 
decided to join forces with them partly 

1046
00:53:13,531 --> 00:53:16,741
because the people higher up at Google 
agreed with things like the ethics 

1047
00:53:16,741 --> 00:53:19,741
committee and thought it was a good idea
that governed the use of the technology 

1048
00:53:19,741 --> 00:53:21,430
of deep mines technology.
We've already out ruled out things,

1049
00:53:21,431 --> 00:53:25,060
obvious things like military or 
intelligence applications.

1050
00:53:25,090 --> 00:53:26,130
Um,
so,

1051
00:53:26,131 --> 00:53:28,590
you know,
the by default deep mind staff doesn't,

1052
00:53:28,620 --> 00:53:31,320
cannot be used for those things.
And then obviously we,

1053
00:53:31,321 --> 00:53:36,321
you know,
we've had our inaugural meeting of the 

1054
00:53:36,321 --> 00:53:37,560
committee ethics committee.
There are very big illuminaries on that,

1055
00:53:37,740 --> 00:53:42,740
many of whom are some of the people that
you've mentioned who are worried about 

1056
00:53:42,740 --> 00:53:43,890
this stuff,
not just the people who think positively

1057
00:53:43,891 --> 00:53:48,891
about it.
And a big part of that actually because 

1058
00:53:48,891 --> 00:53:48,891
you know,
we are decades away,

1059
00:53:48,891 --> 00:53:53,181
is to just start educating everyone on 
what the real issues are and separate 

1060
00:53:53,181 --> 00:53:54,660
sort of fact from science fiction.
Um,

1061
00:53:54,750 --> 00:53:59,750
and I think that's the first time point 
and then we can actually get to the hub 

1062
00:53:59,750 --> 00:54:01,440
of the really core technical difficult 
questions there are.

1063
00:54:02,280 --> 00:54:03,250
And there are some,
but I,

1064
00:54:03,260 --> 00:54:06,960
I'm very confident if we apply enough 
brainpower onto it with enough time,

1065
00:54:07,200 --> 00:54:08,730
uh,
will solve those problems.

1066
00:54:09,090 --> 00:54:10,430
Right.
You,

1067
00:54:11,390 --> 00:54:13,560
you've been in Google now for about a 
year.

1068
00:54:13,850 --> 00:54:14,610
Um,
can you,

1069
00:54:14,640 --> 00:54:19,640
can you give us a couple of examples or 
anecdotes about how deep mind has 

1070
00:54:19,741 --> 00:54:24,741
changed the company?
I'm taking over some processes changed 

1071
00:54:24,741 --> 00:54:29,481
the way the company works your going 
forward and just to tag onto the ethics 

1072
00:54:30,031 --> 00:54:32,490
committee.
Why haven't you publicized or published?

1073
00:54:32,491 --> 00:54:33,780
Who's,
who's on it?

1074
00:54:34,690 --> 00:54:35,340
So,
um,

1075
00:54:35,470 --> 00:54:40,470
first question is I'm almost,
nothing's changed and that's the whole 

1076
00:54:40,470 --> 00:54:40,780
point of it.
That was one of the main agreement.

1077
00:54:40,781 --> 00:54:41,480
So we,
we,

1078
00:54:41,610 --> 00:54:46,610
our,
our headquarters is still in the UK and 

1079
00:54:46,610 --> 00:54:46,960
tinkering around King's cross.
We've built,

1080
00:54:46,961 --> 00:54:51,961
invested in the research team there,
so the whole of the minus still UK side 

1081
00:54:52,180 --> 00:54:53,560
and uh,
you know,

1082
00:54:53,561 --> 00:54:58,561
we're very,
we work as a kind of semi autonomous 

1083
00:54:58,561 --> 00:55:01,410
type of unit.
The plus size are the amount of compute 

1084
00:55:01,410 --> 00:55:01,430
power that we have access to,
uh,

1085
00:55:01,450 --> 00:55:04,560
has really accelerated our progress and 
obviously the other resources that.

1086
00:55:04,880 --> 00:55:06,290
Sorry,
I meant the other way around.

1087
00:55:06,291 --> 00:55:09,200
How is deepmind to,
to Google as a come?

1088
00:55:09,260 --> 00:55:09,860
Oh,
I see.

1089
00:55:09,980 --> 00:55:11,180
Um,
well,

1090
00:55:11,181 --> 00:55:11,980
so,
um,

1091
00:55:12,080 --> 00:55:14,050
that's harder to say.
I mean Google is very big,

1092
00:55:14,580 --> 00:55:19,580
but I think that we have actually a 
effect to that in some senses a the way 

1093
00:55:20,001 --> 00:55:22,910
that some other parts of Google research
do that work.

1094
00:55:23,030 --> 00:55:26,550
So there's actually thousands of people 
in Google research and there's thousands

1095
00:55:26,551 --> 00:55:28,870
of people working on machine learning.
Um,

1096
00:55:28,970 --> 00:55:33,970
but we were sort of have a more coherent
a specific mission than the more applied

1097
00:55:35,301 --> 00:55:37,070
machine learning against done elsewhere 
and google.

1098
00:55:37,220 --> 00:55:42,220
So I think we bring,
we bring together a kind of longer term 

1099
00:55:42,220 --> 00:55:42,450
research focus that,
um,

1100
00:55:42,470 --> 00:55:47,470
I think maybe google wants more of now 
and that requires quite different 

1101
00:55:47,470 --> 00:55:49,520
organizational structures and management
processes,

1102
00:55:49,521 --> 00:55:50,230
which,
um,

1103
00:55:50,480 --> 00:55:51,350
you know,
some of,

1104
00:55:51,370 --> 00:55:56,370
uh,
some of which has been adopted over in 

1105
00:55:56,370 --> 00:55:56,370
mountain view in Silicon Valley.
Now.

1106
00:55:56,370 --> 00:55:58,220
I'm,
so your second question was about the,

1107
00:55:58,221 --> 00:55:59,360
why don't we publicize stuff?

1108
00:55:59,540 --> 00:56:02,290
Well,
firstly I'm a,

1109
00:56:02,360 --> 00:56:04,760
you know,
we're very early days and uh,

1110
00:56:04,800 --> 00:56:07,070
there's a lot of scrutiny on this and 
um,

1111
00:56:07,120 --> 00:56:11,100
there's nothing at the moment,
it's about simply about educating people

1112
00:56:11,120 --> 00:56:13,460
are ever getting everyone up to speed 
with the issues.

1113
00:56:13,790 --> 00:56:16,190
Um,
once you start making things public,

1114
00:56:16,310 --> 00:56:18,890
then immediately that changes,
that can change the debate.

1115
00:56:19,070 --> 00:56:24,070
And I wanted to have a period of um,
sort of quiet behind the scenes,

1116
00:56:25,060 --> 00:56:26,170
a calm,
calm,

1117
00:56:26,171 --> 00:56:31,171
collected debate before we additionally 
on ourselves this additional sort of 

1118
00:56:31,241 --> 00:56:36,241
public scrutiny.
So at some point I think we will 

1119
00:56:36,241 --> 00:56:36,241
announce who,
you know,

1120
00:56:36,241 --> 00:56:40,530
these people are,
and also a little bit about what the 

1121
00:56:40,530 --> 00:56:40,900
issues are that are being discussed.
Having said that,

1122
00:56:40,901 --> 00:56:45,901
we already do lots of public things.
So that was a big conference in Puerto 

1123
00:56:46,540 --> 00:56:51,540
Rico that was talking about ai ethics 
and safety as another one at New York 

1124
00:56:51,540 --> 00:56:56,490
University in January that we're 
sponsoring and I'm keynoting and I'm on 

1125
00:56:56,490 --> 00:56:56,920
a program committee of,
along with Facebook,

1126
00:56:57,100 --> 00:56:59,980
uh,
the heads of ais there and Microsoft and

1127
00:56:59,981 --> 00:57:04,981
some of the other companies.
So I think probably the next stage next 

1128
00:57:04,981 --> 00:57:05,290
year will be to create a cross industry 
panel,

1129
00:57:05,510 --> 00:57:08,410
um,
and bring together all the big companies

1130
00:57:08,411 --> 00:57:13,411
and academic labs that are working on 
this in addition to our own internal 

1131
00:57:13,411 --> 00:57:14,030
committee with your list.
Let's correct

1132
00:57:14,090 --> 00:57:15,560
for a few more.
We've got about 10 minutes.

1133
00:57:15,561 --> 00:57:18,660
So I think someone's got a mike here 
though and then will go up to mother.

1134
00:57:19,100 --> 00:57:19,800
Thank you.
Yes,

1135
00:57:19,830 --> 00:57:22,490
that was a brilliant presentation.
Thank you very much for that.

1136
00:57:22,960 --> 00:57:26,480
I'm a was an engineer.
I'm now a slightly aged academic.

1137
00:57:26,930 --> 00:57:31,930
Um,
we work on the general a video gaming 

1138
00:57:31,930 --> 00:57:34,040
machines,
so I appreciate that very much.

1139
00:57:34,041 --> 00:57:36,950
But when you go the next day up to 
imagination,

1140
00:57:37,240 --> 00:57:41,900
then that must be so many individual 
random coordinates,

1141
00:57:41,901 --> 00:57:46,901
if you like,
because I think we all imagined 

1142
00:57:46,901 --> 00:57:46,901
differently,
um,

1143
00:57:46,901 --> 00:57:50,260
that you'll spend an infinite amount of 
time trying to analyze these to actually

1144
00:57:50,631 --> 00:57:53,750
make machines if you're like,
imagine implant fashion.

1145
00:57:53,751 --> 00:57:56,330
So how do,
how do you cope with these coordinates?

1146
00:57:56,331 --> 00:58:01,331
You can't build it as well because a lot
of the audience and for the television 

1147
00:58:01,331 --> 00:58:01,331
audience here,
I can see,

1148
00:58:01,331 --> 00:58:03,520
I suspect there was a bit of a shudder 
around east kidding.

1149
00:58:03,521 --> 00:58:05,480
He's the creative,
which is,

1150
00:58:05,490 --> 00:58:06,440
yeah,
that's a bit clunky.

1151
00:58:06,441 --> 00:58:11,030
We should kind of merging the course 
versus when the first director,

1152
00:58:11,660 --> 00:58:13,230
the first writer,
when those,

1153
00:58:13,310 --> 00:58:15,040
the invite the points in the 
environment,

1154
00:58:15,060 --> 00:58:18,640
the choices become financial versus a 
space invaders.

1155
00:58:18,900 --> 00:58:21,570
Exactly.
It's like the rice on the chess board.

1156
00:58:21,930 --> 00:58:22,920
Yeah.
So I think,

1157
00:58:22,921 --> 00:58:27,921
uh,
I think most people's jobs in here safe 

1158
00:58:27,921 --> 00:58:29,770
for a long time ago is quite hard.
So I don't think it's going to be any 

1159
00:58:29,770 --> 00:58:30,030
directors,
you know,

1160
00:58:30,031 --> 00:58:31,440
directing something,
you know,

1161
00:58:31,441 --> 00:58:33,210
the quality of Ridley Scott or 
something,

1162
00:58:33,500 --> 00:58:34,770
you know,
that's probably going to be one of the,

1163
00:58:34,771 --> 00:58:37,070
if,
if ever one of the last things that that

1164
00:58:37,110 --> 00:58:42,110
competes will be able to do.
So we asked talking about very 

1165
00:58:42,110 --> 00:58:42,110
constrained things where,
um,

1166
00:58:42,110 --> 00:58:47,000
you know,
that's a very difficult thing that 

1167
00:58:47,000 --> 00:58:47,280
humans of course do better than,
than way better than computers is we,

1168
00:58:47,281 --> 00:58:52,281
you know,
they can have this rudimentary kind of 

1169
00:58:52,281 --> 00:58:53,940
brute force imagination.
But one of the big things that humans do

1170
00:58:53,970 --> 00:58:56,040
is they have aesthetic judgment,
right?

1171
00:58:56,220 --> 00:59:01,220
They know that I'm not,
all Paul's are equal and some of them 

1172
00:59:01,220 --> 00:59:02,790
are likely to be more fruitful than 
others.

1173
00:59:02,850 --> 00:59:07,850
Even if you compare chess,
grandmaster playing the chess compared 

1174
00:59:07,850 --> 00:59:08,220
to a computer,
they,

1175
00:59:08,240 --> 00:59:08,910
they,
you know,

1176
00:59:08,911 --> 00:59:12,030
computer might look at millions of moves
to make that one decision.

1177
00:59:12,180 --> 00:59:15,570
Whereas a transplant not only look at a 
few hundred but judicial ones,

1178
00:59:15,720 --> 00:59:18,970
and they in some sense our brains even 
filter out a,

1179
00:59:19,020 --> 00:59:24,020
our low level power brain or any of the 
kind of moves or trajectories that are 

1180
00:59:24,410 --> 00:59:26,780
not going to yield anything useful.
Um,

1181
00:59:26,810 --> 00:59:28,040
and uh,
you know,

1182
00:59:28,050 --> 00:59:28,300
the,

1183
00:59:28,420 --> 00:59:30,100
when you do that filtrate,
are you working on.

1184
00:59:30,300 --> 00:59:35,300
Well,
we are talking that filtration part of 

1185
00:59:35,300 --> 00:59:35,300
that is,
is to do with how well you

1186
00:59:35,300 --> 00:59:39,060
model of the world that you're in.
So if you're better at modeling the 

1187
00:59:39,060 --> 00:59:42,031
world,
then what that means is you should make 

1188
00:59:42,031 --> 00:59:44,460
better predictions about what are going 
to be useful things to spend your 

1189
00:59:44,460 --> 00:59:45,760
compute time,
uh,

1190
00:59:45,790 --> 00:59:48,390
imagining or thinking about a and at a 
moment where,

1191
00:59:48,391 --> 00:59:49,900
you know,
we're still very early stages of that.

1192
00:59:51,270 --> 00:59:52,940
Sorry.
Hi,

1193
00:59:53,250 --> 00:59:55,030
David Abraham from Channel Four.
Um,

1194
00:59:55,530 --> 00:59:58,530
you touched on the challenge of,
um,

1195
00:59:59,160 --> 01:00:04,160
how many choices people have,
an entertainment and something that we 

1196
01:00:04,631 --> 01:00:06,900
in our industry is spending a lot of 
time thinking about.

1197
01:00:07,570 --> 01:00:09,210
Um,
are you,

1198
01:00:09,250 --> 01:00:14,070
um,
working more specifically on the area of

1199
01:00:14,100 --> 01:00:18,180
recommendation engines and are you going
to be,

1200
01:00:18,420 --> 01:00:23,420
as it were,
capturing the power of that algorithm 

1201
01:00:23,420 --> 01:00:23,420
and behalf of Google?

1202
01:00:23,530 --> 01:00:25,780
Yeah,
we are looking at recommendation systems

1203
01:00:25,840 --> 01:00:27,640
and um,
you know,

1204
01:00:27,910 --> 01:00:30,670
uh,
in all forms actually all sorts of forms

1205
01:00:30,910 --> 01:00:33,100
and a,
I think it's a very interesting area and

1206
01:00:33,101 --> 01:00:35,590
it's something that our technology is 
quite,

1207
01:00:35,740 --> 01:00:37,300
uh,
quite sort of applicable to.

1208
01:00:37,720 --> 01:00:39,130
Again,
it's about,

1209
01:00:39,190 --> 01:00:44,190
you know,
can you model a user journeys and 

1210
01:00:44,190 --> 01:00:44,980
trajectories through things and in a way
that,

1211
01:00:44,981 --> 01:00:45,700
um,
you know,

1212
01:00:45,701 --> 01:00:49,480
then delivers much more compelling 
content or recommendations and I think,

1213
01:00:49,720 --> 01:00:50,740
uh,
you know,

1214
01:00:50,741 --> 01:00:52,690
the current systems we have are not good
enough.

1215
01:00:52,850 --> 01:00:54,370
Um,
and you know,

1216
01:00:54,371 --> 01:00:55,630
we're,
we're experimenting in that.

1217
01:00:55,631 --> 01:00:56,240
Again,
we don't,

1218
01:00:56,280 --> 01:01:01,280
we're,
we're quite early days with that and 

1219
01:01:01,280 --> 01:01:01,280
we're looking at that for things both 
internally at Google and external

1220
01:01:02,270 --> 01:01:07,270
dcx external because I think that will 
be deeply intriguing to a number of us 

1221
01:01:07,270 --> 01:01:11,081
in the room who are working in the media
business about how to serve up stuff in 

1222
01:01:11,081 --> 01:01:11,750
a world where choices.
Yeah.

1223
01:01:11,780 --> 01:01:15,290
Exploded.
And the general application of those few

1224
01:01:15,290 --> 01:01:17,840
years know,
think it'd be announcing stuff,

1225
01:01:17,870 --> 01:01:22,870
products coming out.
I think maybe it might be the wrong 

1226
01:01:22,870 --> 01:01:22,870
word,
forgive my ignorance,

1227
01:01:22,870 --> 01:01:25,430
but systems by which the likes of 
channel for the BBC,

1228
01:01:25,431 --> 01:01:30,431
other broadcasts in the room,
independent companies conserve their 

1229
01:01:30,431 --> 01:01:30,431
content that we think that's not far 
away.

1230
01:01:30,431 --> 01:01:30,431
Yeah.

1231
01:01:30,431 --> 01:01:34,560
Yeah.
I think in the next couple of years 

1232
01:01:34,560 --> 01:01:35,060
you'll start seeing under the hood a 
algorithms helping the,

1233
01:01:35,061 --> 01:01:40,061
these kinds of recommendation systems 
and then maybe four or five years our 

1234
01:01:40,061 --> 01:01:43,201
actual hope,
totally new systems that you might 

1235
01:01:43,201 --> 01:01:43,201
interact with in a different way than we
do now.

1236
01:01:43,201 --> 01:01:43,440
Personally.

1237
01:01:44,530 --> 01:01:46,570
We got,
we're coming towards the last few,

1238
01:01:46,571 --> 01:01:48,190
but we'll get through as many as I can

1239
01:01:49,530 --> 01:01:54,530
from IBM.
I think it the other big investors in 

1240
01:01:54,530 --> 01:01:55,220
Ai,
and I liked your comments around the big

1241
01:01:55,221 --> 01:01:57,990
breakthroughs is probably takes more 
than one player to go to,

1242
01:01:57,991 --> 01:01:59,690
is to,
to the future space.

1243
01:01:59,710 --> 01:02:00,970
Um,
uh,

1244
01:02:00,980 --> 01:02:04,430
had a couple of questions in treatment 
in your talk.

1245
01:02:04,720 --> 01:02:07,070
One was go.
So,

1246
01:02:07,100 --> 01:02:12,100
so years ago,
I think there was a big effort around 

1247
01:02:12,100 --> 01:02:12,110
going who was the big one that was hard 
for computers to solve.

1248
01:02:12,111 --> 01:02:13,880
It might be a bit esoteric for this 
audience.

1249
01:02:14,690 --> 01:02:16,820
And then the second question,
just to bring this to the point,

1250
01:02:16,821 --> 01:02:19,260
is a,
for rts,

1251
01:02:19,320 --> 01:02:24,320
how can ai be used not to supplant 
creativity but to enhance and support it

1252
01:02:24,730 --> 01:02:27,510
and allow us to do more creative things 
as humans,

1253
01:02:27,511 --> 01:02:28,460
not as computers.

1254
01:02:28,520 --> 01:02:33,520
Let's do a quick one on the first one 
because you might wanna do that 

1255
01:02:33,520 --> 01:02:33,520
afterwards.
And it's really.

1256
01:02:33,520 --> 01:02:33,520
Yeah.
So go,

1257
01:02:33,520 --> 01:02:38,400
go for it.
As you don't know is it is an oriental 

1258
01:02:38,400 --> 01:02:40,520
board game,
which is probably the most complex game 

1259
01:02:40,520 --> 01:02:42,771
there is.
This is what they play in China and 

1260
01:02:42,771 --> 01:02:42,771
Japan instead of a career instead of 
chest.

1261
01:02:42,840 --> 01:02:47,840
And um,
one reason it's been so hard for 

1262
01:02:47,840 --> 01:02:47,840
computers to crack is that the branching
factor,

1263
01:02:47,840 --> 01:02:51,830
the number of choices you have in each 
booth is the order of 100,

1264
01:02:51,930 --> 01:02:55,200
whereas in chat is more like 20.
So as you start planning that branch,

1265
01:02:55,201 --> 01:03:00,201
in fact explode.
So if you're going to do it in a brute 

1266
01:03:00,201 --> 01:03:00,201
force way,
there aren't enough.

1267
01:03:00,201 --> 01:03:02,230
I think atoms in the university describe
how many go positions there are.

1268
01:03:02,240 --> 01:03:04,020
For example,
bet you're good at.

1269
01:03:04,021 --> 01:03:06,620
Go on.
I'm reasonably good at go where you will

1270
01:03:07,640 --> 01:03:09,890
champion champion.
But,

1271
01:03:09,960 --> 01:03:10,940
but,
but um,

1272
01:03:11,270 --> 01:03:14,700
and then the second problem is that uh,
in chess,

1273
01:03:14,701 --> 01:03:16,820
because chess is a very materialistic 
games,

1274
01:03:16,821 --> 01:03:18,930
so the queen is worth more than a real 
console.

1275
01:03:19,110 --> 01:03:24,110
It's quite easy to hand program and 
evaluation function to tell you whether 

1276
01:03:24,110 --> 01:03:27,660
your program is winning or losing or how
well it's doing in that position.

1277
01:03:27,900 --> 01:03:30,720
Whereas in go,
all the pieces are worth the same.

1278
01:03:30,721 --> 01:03:35,721
They're just,
there's just one piece and so whether 

1279
01:03:35,721 --> 01:03:36,390
you're winning or not as much more about
the overall pattern of the board.

1280
01:03:36,630 --> 01:03:38,460
So it's a much more beautiful game in 
some sense,

1281
01:03:38,461 --> 01:03:43,461
very aesthetically pleasing,
but it's much harder to hand code and 

1282
01:03:43,461 --> 01:03:45,060
evaluation function.
So,

1283
01:03:45,061 --> 01:03:45,660
um,
yeah,

1284
01:03:45,661 --> 01:03:50,661
we have,
we're going to have some very big 

1285
01:03:50,661 --> 01:03:50,661
announcements to make ongoing.
I mean,

1286
01:03:50,661 --> 01:03:53,781
it's sort of been the holy grail for the
AI research community for the last 20 

1287
01:03:53,781 --> 01:03:55,280
years since the blue actually be cast 
off.

1288
01:03:55,530 --> 01:03:56,080
Um,

1289
01:03:56,720 --> 01:03:57,200
sorry.

1290
01:03:58,880 --> 01:03:59,750
Yeah,
sure.

1291
01:03:59,990 --> 01:04:00,800
So,
uh,

1292
01:04:00,830 --> 01:04:01,820
yeah,
and then so then,

1293
01:04:01,850 --> 01:04:02,630
then the,
the,

1294
01:04:02,631 --> 01:04:07,631
the,
the last question was on a train that's 

1295
01:04:09,500 --> 01:04:12,320
really the same thing I'm thinking about
in science to write and,

1296
01:04:12,500 --> 01:04:17,500
and with doctors and with one reading as
ai surfacing the right information for 

1297
01:04:17,671 --> 01:04:22,671
you in a much more digestible way so you
can just leverage that for whatever it 

1298
01:04:22,671 --> 01:04:22,671
is

1299
01:04:23,130 --> 01:04:28,130
we took the recommendation area.
Is there anything else in your head you 

1300
01:04:28,130 --> 01:04:30,540
might just springs to mind in terms of 
the creative process,

1301
01:04:30,541 --> 01:04:31,830
the creation of media?

1302
01:04:32,170 --> 01:04:37,170
I think that's a lot tougher.
So I think the recommendation is the 

1303
01:04:37,170 --> 01:04:37,720
obvious one.
We are looking at things like music,

1304
01:04:37,721 --> 01:04:42,721
which is a,
a kind of more constrained domain for a 

1305
01:04:42,721 --> 01:04:42,721
computer than visuals.
I mean,

1306
01:04:42,721 --> 01:04:43,690
which is incredibly hard,
right?

1307
01:04:43,990 --> 01:04:48,990
And uh,
this is a very interesting work being 

1308
01:04:48,990 --> 01:04:48,990
done in music,
music composition,

1309
01:04:48,990 --> 01:04:50,230
a music analysis,
uh,

1310
01:04:50,260 --> 01:04:55,260
which I think is pretty promising.
So I would imagine that would be the 

1311
01:04:55,260 --> 01:04:55,260
next place.

1312
01:04:55,260 --> 01:04:59,030
Very good.
I'm going to take three more because 

1313
01:04:59,030 --> 01:04:59,030
we're really running out of time.
I'm sorry,

1314
01:04:59,030 --> 01:05:03,510
because we could just keep going.
We're not here to take you guys weren't 

1315
01:05:03,510 --> 01:05:03,510
here.
Is there,

1316
01:05:03,510 --> 01:05:06,691
and got a microphone on them because 
we'll just get the nice gentleman in the

1317
01:05:07,110 --> 01:05:08,240
back.
Want to have A.

1318
01:05:08,240 --> 01:05:11,490
Because I've been very front focus that 
we don't want to be on the road right at

1319
01:05:11,491 --> 01:05:12,030
the back there.

1320
01:05:13,470 --> 01:05:15,510
IBM,
we've been sheep.

1321
01:05:15,511 --> 01:05:18,100
The from the search of pain,
pleasure during

1322
01:05:18,160 --> 01:05:22,480
our own cultural history and it's shaped
the way that we're thinking,

1323
01:05:22,520 --> 01:05:26,530
the way we are behaving.
How can you teach a pain and pleasure to

1324
01:05:26,531 --> 01:05:27,070
a machine?

1325
01:05:28,950 --> 01:05:30,100
Well,
one question is whether we,

1326
01:05:30,101 --> 01:05:32,850
whether we need to,
but it's also,

1327
01:05:33,090 --> 01:05:35,490
or whether we should and um,
uh,

1328
01:05:35,500 --> 01:05:40,500
but there is sort of,
this speaks to this idea actually that 

1329
01:05:40,500 --> 01:05:41,100
we look at internally of intrinsic 
motivation.

1330
01:05:41,101 --> 01:05:41,940
We call it.
So,

1331
01:05:42,180 --> 01:05:44,460
you know,
there are emotions and other things that

1332
01:05:44,461 --> 01:05:47,160
drive human behavior,
not just external rewards.

1333
01:05:47,400 --> 01:05:48,480
Um,
so,

1334
01:05:48,660 --> 01:05:49,230
uh,
you know,

1335
01:05:49,231 --> 01:05:51,570
and at the moment our machines don't 
have anything like that.

1336
01:05:51,840 --> 01:05:52,500
But,
um,

1337
01:05:52,530 --> 01:05:55,450
maybe to do more complex tasks or you 
know,

1338
01:05:55,500 --> 01:06:00,010
where the mobile working in game worlds,
where there is conveniently a score most

1339
01:06:00,060 --> 01:06:05,060
most of the time,
but even if you start going to more 

1340
01:06:05,060 --> 01:06:07,431
complex games,
mope all open ended things like 

1341
01:06:07,431 --> 01:06:07,920
minecraft.
Now there isn't a score anymore,

1342
01:06:08,130 --> 01:06:13,130
right?
So how are you going to decide what you 

1343
01:06:13,130 --> 01:06:13,130
should do?
What is,

1344
01:06:13,130 --> 01:06:14,400
what's useful,
what's good that you're making progress?

1345
01:06:14,401 --> 01:06:16,020
And I'm talking about the agent system 
here.

1346
01:06:16,320 --> 01:06:17,550
So,
um,

1347
01:06:17,610 --> 01:06:22,610
and of course that's more like the real 
world for us as as humans and yet 

1348
01:06:22,610 --> 01:06:26,271
somehow we have our own internal drives 
probably that had been evolved to that 

1349
01:06:26,271 --> 01:06:31,251
help influence our behavior.
So I think it's interesting to think 

1350
01:06:31,251 --> 01:06:31,251
about.
Um,

1351
01:06:31,251 --> 01:06:31,770
and you know,
we have,

1352
01:06:31,771 --> 01:06:36,771
neuroscientists are experts in these 
areas who work with us as consultants 

1353
01:06:36,771 --> 01:06:37,200
and it's something I'm very fascinated 
by.

1354
01:06:37,620 --> 01:06:38,430
Um,
but,

1355
01:06:38,490 --> 01:06:39,030
uh,
you know,

1356
01:06:39,031 --> 01:06:40,890
we don't have a definite answer on that 
yet.

1357
01:06:41,460 --> 01:06:42,810
Thank you.
Two more.

1358
01:06:42,870 --> 01:06:44,860
So we'll take the gentleman at the back 
of it.

1359
01:06:44,870 --> 01:06:47,360
You almost don't want to have a question
about,

1360
01:06:48,410 --> 01:06:50,190
I got hand up the.
Yes,

1361
01:06:50,191 --> 01:06:51,840
there is rather back there.
I just feel,

1362
01:06:52,730 --> 01:06:57,720
and then we'll take the gentleman in the
red as the last question.

1363
01:06:58,530 --> 01:06:59,010
No pressure.

1364
01:07:00,850 --> 01:07:02,510
My,
my question leads on quite well from the

1365
01:07:02,511 --> 01:07:04,160
last.
Actually I was thinking,

1366
01:07:04,161 --> 01:07:06,830
have you thought about generalizing the 
goals?

1367
01:07:06,860 --> 01:07:09,590
So you talked about how the,
the uh,

1368
01:07:09,920 --> 01:07:11,760
observations and you have,
um,

1369
01:07:12,290 --> 01:07:15,560
actions that you take,
but presumably you define the goal.

1370
01:07:15,561 --> 01:07:19,310
Then you tell the system how to measure 
its goals and if we're thinking about ai

1371
01:07:19,311 --> 01:07:22,370
as something which is a servant to,
to a human intelligence,

1372
01:07:22,790 --> 01:07:27,790
then have you thought about ai which can
derive its goals from the environment.

1373
01:07:28,480 --> 01:07:29,440
Can,
can also,

1374
01:07:29,450 --> 01:07:31,490
and also as humans,
we segment our goals.

1375
01:07:31,491 --> 01:07:36,491
We might have life goals,
but we focused on sub goals to get 

1376
01:07:36,491 --> 01:07:36,960
there.
And also I'm not just know,

1377
01:07:36,980 --> 01:07:39,680
I'm sort of imagining a human saying to 
a system,

1378
01:07:40,010 --> 01:07:45,010
can you help me with this and that and 
the ai being able to derive its goal 

1379
01:07:45,010 --> 01:07:48,680
from the things that it hears,
but also going further than that,

1380
01:07:48,710 --> 01:07:52,340
being able to derive goals before 
they're specifically instructed.

1381
01:07:52,700 --> 01:07:53,500
So,
um,

1382
01:07:54,020 --> 01:07:57,130
being able to anticipate goals that 
people might want that to ai.

1383
01:07:57,140 --> 01:07:57,850
So we realize it

1384
01:07:57,930 --> 01:07:58,800
go to high school.

1385
01:07:59,090 --> 01:07:59,930
That's fine.
I mean,

1386
01:07:59,931 --> 01:08:00,710
that's very,
you know,

1387
01:08:00,711 --> 01:08:03,230
that's a great question.
And actually it's a fascinating research

1388
01:08:03,231 --> 01:08:05,720
areas.
Can the machines learn their own goals?

1389
01:08:05,950 --> 01:08:08,430
A bite through observation of,
you know,

1390
01:08:08,450 --> 01:08:10,910
learn what it is you like through 
observing you for example.

1391
01:08:10,911 --> 01:08:12,650
Right?
And then trying to,

1392
01:08:12,670 --> 01:08:14,810
you know,
maybe even be able to preemptively guess

1393
01:08:14,811 --> 01:08:18,080
what is that you need before you even 
ask for it.

1394
01:08:18,470 --> 01:08:19,580
Um,
so I think,

1395
01:08:19,730 --> 01:08:20,600
uh,
you know,

1396
01:08:20,601 --> 01:08:25,601
those systems are very interesting.
I mean even there you will still have 

1397
01:08:25,601 --> 01:08:26,990
some kind of top level goal which is to 
satisfy the user,

1398
01:08:26,991 --> 01:08:29,330
right?
Although it may learn what the sub goals

1399
01:08:29,331 --> 01:08:34,331
are and that's another very active area 
of research is how do you break down a 

1400
01:08:34,331 --> 01:08:36,290
large go into,
into automatically into sub goals.

1401
01:08:36,500 --> 01:08:38,660
And of course that's something our minds
do effortlessly.

1402
01:08:38,890 --> 01:08:41,540
You know,
if you're going to plan to um,

1403
01:08:41,541 --> 01:08:43,660
you know,
a trip to Paris from hair,

1404
01:08:43,920 --> 01:08:48,920
um,
your brain is not going to plan over 

1405
01:08:48,920 --> 01:08:49,340
your muscle fiber movements all the way 
from here to Paris,

1406
01:08:49,400 --> 01:08:49,630
right?

1407
01:08:49,650 --> 01:08:51,830
It's the yet.
That's how robotics works.

1408
01:08:51,831 --> 01:08:54,650
And it might feel like they have no 
defining of hierarchy,

1409
01:08:54,800 --> 01:08:56,410
well your act,
but it's actually going to do is like,

1410
01:08:56,420 --> 01:09:01,420
you know,
at high level you need to get to the 

1411
01:09:01,420 --> 01:09:01,420
Eurostar terminal and then take a train 
there and so on,

1412
01:09:01,420 --> 01:09:04,040
and then only at the point where you get
up off that chair,

1413
01:09:04,100 --> 01:09:07,790
does your brain then go and unpack the 
muscle fiber movement of get,

1414
01:09:07,791 --> 01:09:10,190
you know,
get up off the chair and uh,

1415
01:09:10,191 --> 01:09:11,780
and walk.
So,

1416
01:09:11,840 --> 01:09:13,100
um,
whereas at the moment,

1417
01:09:13,101 --> 01:09:18,010
because we haven't solved this problem 
of automatically generating subgoals I'm

1418
01:09:18,170 --> 01:09:19,100
a robot,
for example,

1419
01:09:19,101 --> 01:09:24,101
trying to do that task,
we'd have to plan over the primitive 

1420
01:09:24,101 --> 01:09:26,531
action movements all the way to Paris.
And of course this is not feasible and 

1421
01:09:26,531 --> 01:09:27,040
therefore not tractable.
Um,

1422
01:09:27,080 --> 01:09:28,760
because it ends up becoming,
you know,

1423
01:09:28,761 --> 01:09:33,761
speaks to the other gentleman's question
about you imagine all those paths from 

1424
01:09:33,761 --> 01:09:34,280
here or muscle muscle fiber movements.
There's an infinite,

1425
01:09:34,330 --> 01:09:35,660
there's basically an infinite number of 
them.

1426
01:09:36,110 --> 01:09:37,610
So,
um,

1427
01:09:37,700 --> 01:09:42,700
so we need,
that's one of the key things we need to 

1428
01:09:42,700 --> 01:09:42,700
solve is the sub goal problem.

1429
01:09:42,700 --> 01:09:43,870
The last question.
Oh,

1430
01:09:44,470 --> 01:09:46,750
sorry.
Get that across.

1431
01:09:48,420 --> 01:09:49,700
Hmm.

1432
01:09:51,060 --> 01:09:56,060
Malcolm have a broadcast engineer 
following off from an earlier one and 

1433
01:09:56,060 --> 01:09:56,400
your list of memory,
navigation,

1434
01:09:56,430 --> 01:09:57,750
imagination,
etc.

1435
01:09:58,200 --> 01:10:03,200
You didn't have emotions and that's 
where we're going to be dealing with an 

1436
01:10:03,200 --> 01:10:06,800
interacting with humans.
How explosive is that with Your 

1437
01:10:06,800 --> 01:10:08,100
conflicts of ethics and parameters?

1438
01:10:08,660 --> 01:10:09,230
Yeah,
I mean,

1439
01:10:09,231 --> 01:10:09,790
again,
this,

1440
01:10:09,800 --> 01:10:10,630
this,
um,

1441
01:10:10,670 --> 01:10:14,450
relates to the question identifying down
here about emotions.

1442
01:10:14,490 --> 01:10:16,010
We currently.
I'm,

1443
01:10:16,360 --> 01:10:16,970
uh,
you know,

1444
01:10:16,971 --> 01:10:19,190
there is no equivalent of that in our 
systems.

1445
01:10:19,460 --> 01:10:20,360
Um,
but,

1446
01:10:20,440 --> 01:10:23,030
uh,
if you think of inch in emotions,

1447
01:10:23,031 --> 01:10:27,590
and probably this is too simplistic,
but part of emotions are internal drives

1448
01:10:27,770 --> 01:10:32,770
a give us internal drives,
then that's something we do need to 

1449
01:10:32,770 --> 01:10:32,770
explore,
um,

1450
01:10:32,870 --> 01:10:37,870
and try and work out what ones might be 
needed and it might be we need similar 

1451
01:10:37,870 --> 01:10:40,370
ones so that these systems can empathize
with humans.

1452
01:10:40,610 --> 01:10:41,810
Um,
you know,

1453
01:10:41,811 --> 01:10:43,340
obviously there's a great channel for 
series.

1454
01:10:43,341 --> 01:10:45,650
I think that people,
some people in the audience with humans,

1455
01:10:45,651 --> 01:10:47,720
which I really love and that's 
interesting.

1456
01:10:47,721 --> 01:10:52,721
You know,
they're trying to empathize with the 

1457
01:10:52,721 --> 01:10:52,721
humans that they serve.
Alternatively,

1458
01:10:52,721 --> 01:10:56,360
you might want to have systems that have
no a very different types of drives that

1459
01:10:56,361 --> 01:10:59,930
help them be complimentary to what 
humans are good at,

1460
01:11:00,110 --> 01:11:02,930
so I could imagine we might need both 
types of daily,

1461
01:11:03,410 --> 01:11:07,070
so motion.
Just want to fish on the using essential

1462
01:11:07,071 --> 01:11:12,071
that you're going to have to cope with 
emotion as a driver of if you'd like 

1463
01:11:12,071 --> 01:11:12,390
some types of emotions,
relations,

1464
01:11:12,490 --> 01:11:16,170
so there's two reasons you might want 
the one so that we can see these systems

1465
01:11:16,171 --> 01:11:18,870
can empathize and work better.
Hand in hand with humans.

1466
01:11:18,940 --> 01:11:20,720
Yeah.
The other thing is if they.

1467
01:11:20,730 --> 01:11:22,350
It turns out the environments they're 
in,

1468
01:11:22,380 --> 01:11:24,690
there aren't many external reward 
signals.

1469
01:11:24,870 --> 01:11:28,740
They have to have some internal drive to
get them going in the right direction.

1470
01:11:30,300 --> 01:11:35,300
It was a privilege.
I'm going to hand to nomi climate from 

1471
01:11:35,300 --> 01:11:36,210
the president,
the iet in a second,

1472
01:11:36,211 --> 01:11:38,520
but dennis has been one year.
Thank you.

1473
01:11:41,790 --> 01:11:46,790
Thank you

1474
01:11:53,380 --> 01:11:55,420
dennis.
As the president of the iet,

1475
01:11:55,660 --> 01:12:00,660
we're absolutely delighted to have co 
hosted this lecture with the royal 

1476
01:12:00,660 --> 01:12:03,871
television society and delighted to have
someone of your extraordinary caliber 

1477
01:12:03,871 --> 01:12:07,870
miss events like today for fill,
part of the iet charitable remit,

1478
01:12:07,871 --> 01:12:09,160
which is to inspire,
inform,

1479
01:12:09,161 --> 01:12:11,710
and influence people and I don't know 
about you,

1480
01:12:11,711 --> 01:12:16,711
but you sure as hell have inspired,
informed and influenced me on a topic 

1481
01:12:16,711 --> 01:12:17,350
that I believe is going to change the 
world.

1482
01:12:17,650 --> 01:12:22,650
I love the idea that your ai journey 
started with games and the machine 

1483
01:12:22,650 --> 01:12:27,421
learning is done through play.
Pretty mucH the same as it is for 

1484
01:12:27,421 --> 01:12:29,590
humans.
I've enjoyed the way that you've made it

1485
01:12:29,591 --> 01:12:34,591
all sound pretty straightforward 
actually from our hippocampus to 

1486
01:12:34,591 --> 01:12:38,290
imagining rats to the quest for machine 
creativity and it just sounded quite 

1487
01:12:39,611 --> 01:12:43,060
logical that your journey to the to the 
point to have your mission,

1488
01:12:43,061 --> 01:12:47,710
which is use ai to solve everything else
seems quite reasonable.

1489
01:12:47,950 --> 01:12:52,950
Your quest for the ai scientist to 
really tackle some of those big 

1490
01:12:52,950 --> 01:12:54,100
important challenges.
So listening to you,

1491
01:12:54,101 --> 01:12:56,800
it all sounds incredibly real and 
feasible.

1492
01:12:56,801 --> 01:13:01,801
That ai can make a positive difference 
to humanity and even if it's not going 

1493
01:13:01,811 --> 01:13:06,040
to be directing any movies anytime soon.
So once again,

1494
01:13:06,070 --> 01:13:08,520
please join me in thanking dennis 
hassabis,

1495
01:13:08,800 --> 01:13:11,920
founder of deep mind for an absolutely 
fantastic.

1496
01:13:12,150 --> 01:13:17,150
Sure.
Thanks and I will do.

1497
01:13:25,280 --> 01:13:28,730
I'd like to thank our feisty chair.
You did an excellent job.

1498
01:13:28,731 --> 01:13:32,660
Tyndale be the ceo of bbc worldwide.
I think we've all been inspired,

1499
01:13:32,661 --> 01:13:33,790
informed,
and influenced,

1500
01:13:33,800 --> 01:13:36,260
so it's been very nice for me to be part
of this.

1501
01:13:36,560 --> 01:13:41,560
Thank you very much for coming here.
We did like to to hear that drinks are 

1502
01:13:41,560 --> 01:13:41,560
now served outside.
Thank you.

1503
01:13:41,560 --> 01:13:43,870
thank you.
