Moore’s law in peril and the future of computing

1
00:00:04,850 --> 00:00:05,980
Physics world.

2
00:00:06,430 --> 00:00:10,380
Hello and welcome to the Physics World
Stories Podcast. I'm Andrew g Tron.

3
00:00:10,380 --> 00:00:11,213
In this episode,

4
00:00:11,390 --> 00:00:15,220
we're gonna be exploring Moore's Law
named after the late Gordon Moore,

5
00:00:15,240 --> 00:00:19,580
who observed that the number of
transistors in a dense integrated circuit

6
00:00:19,970 --> 00:00:22,780
doubled about every two years of late.

7
00:00:22,950 --> 00:00:26,420
There have been suggestions
that the law might be dead.

8
00:00:26,670 --> 00:00:31,540
We'll explore that and look
into why this matters and what,

9
00:00:32,120 --> 00:00:35,020
if anything, might replace
it. Later in the podcast,

10
00:00:35,270 --> 00:00:38,580
we'll hear from two researchers
looking into the future of computing.

11
00:00:39,080 --> 00:00:42,860
Two of the biggest questions that they're
looking into are whether the increase

12
00:00:42,860 --> 00:00:47,460
of computing power can keep up with the
ever increasing demands of generative

13
00:00:47,900 --> 00:00:48,820
artificial intelligence,

14
00:00:49,600 --> 00:00:54,580
and can that happen in a way that doesn't
threaten the planet due to the vast

15
00:00:54,720 --> 00:00:58,500
energy demands? But first,
here's Louis Barson.

16
00:00:59,140 --> 00:01:00,220
I am, uh, director of Science,

17
00:01:00,220 --> 00:01:02,380
innovation and Skills at
the Institute of Physics.

18
00:01:02,380 --> 00:01:05,060
Semiconductors are important
because they're the basic, uh,

19
00:01:05,180 --> 00:01:08,100
building blocks of computer chips. Um,

20
00:01:08,240 --> 00:01:10,820
and the more transistors
you have on a chip, uh,

21
00:01:10,820 --> 00:01:12,660
the more numbers you can crunch, uh,

22
00:01:12,660 --> 00:01:17,540
the smaller and more powerful your
digital devices become. Uh, and, uh,

23
00:01:17,540 --> 00:01:20,860
that's important. Cause that's what
really powers the, the digital revolution.

24
00:01:20,860 --> 00:01:23,900
It's the miniaturization,
uh, and the increasing, uh,

25
00:01:23,900 --> 00:01:28,500
power and efficiency of digital devices
that is transforming industries,

26
00:01:28,760 --> 00:01:33,420
uh, in and everything from aerospace
and automotive to, uh, healthcare, uh,

27
00:01:33,680 --> 00:01:36,700
and defense. And I guess
it's Moore's Law, um,

28
00:01:36,830 --> 00:01:40,620
which tells the story of this
doubling, uh, of, of that,

29
00:01:40,620 --> 00:01:44,140
that capacity that really charts the
development of the digital transformation

30
00:01:44,140 --> 00:01:44,973
that we're seeing.

31
00:01:45,160 --> 00:01:49,660
Can you tell me a bit more about Moore's
law and crucially, is it actually dead?

32
00:01:49,820 --> 00:01:50,980
I guess this really comes from a,

33
00:01:50,980 --> 00:01:55,300
an article he published in 1965 in
the American Journal, uh, electronics.

34
00:01:56,000 --> 00:01:58,940
Uh, and this is back when he was
director of the R&d Lab at Fairchild's

35
00:01:58,940 --> 00:02:02,900
Semiconductors, so before Intel
and all that. Um, and the,

36
00:02:02,900 --> 00:02:05,580
the article and the prediction is about
the number of components on a chip.

37
00:02:06,040 --> 00:02:07,700
So it's a pretty understated article.

38
00:02:07,860 --> 00:02:09,300
I would say It's sort
of highlights that the,

39
00:02:09,520 --> 00:02:12,660
the number of the components on a chip
had doubled every year for the last three

40
00:02:12,660 --> 00:02:16,140
years. And he said, you know, he was,
he was open and said that, you know,

41
00:02:16,140 --> 00:02:17,900
there's a bit of uncertainty
of the future growth rates,

42
00:02:18,160 --> 00:02:21,060
but he expected that doubling
would continue every year,

43
00:02:21,080 --> 00:02:25,980
at least for 10 more years. Um, so he
then updated his prediction in 1975.

44
00:02:26,300 --> 00:02:26,340
Actually,

45
00:02:26,340 --> 00:02:29,660
to save this doubling would take place
every two years rather than every year.

46
00:02:30,080 --> 00:02:31,860
So this was, this was the
observation of a trend.

47
00:02:31,920 --> 00:02:34,740
It was charting it into the
midterm and, uh, you know,

48
00:02:34,980 --> 00:02:37,300
updating the prediction to make
it a bit more realistic, but not,

49
00:02:37,320 --> 00:02:40,780
not a prophecy by, uh, by
stretching the imagination. Uh,

50
00:02:40,780 --> 00:02:42,540
and I guess it was probably actually, uh,

51
00:02:42,540 --> 00:02:44,780
the eminent Caltech professor Carver Mead,

52
00:02:44,780 --> 00:02:48,540
who was a close colleague of
Gordon Moore. Uh, he also had a,

53
00:02:48,580 --> 00:02:52,780
a good knack for predict in the
future. Um, so for instance, you know,

54
00:02:52,780 --> 00:02:53,460
very early on when,

55
00:02:53,460 --> 00:02:56,540
when many thought miniaturization would
actually make chip performance worse,

56
00:02:56,920 --> 00:02:59,860
he did some clever physics and
material science to show that in fact,

57
00:02:59,930 --> 00:03:02,820
miniaturizing ships would allow them
to get faster and more efficient.

58
00:03:02,820 --> 00:03:03,980
And that turned out to be the case.

59
00:03:04,480 --> 00:03:08,260
So I think it's actually Carver
Meat who's credited as the, um, uh,

60
00:03:08,260 --> 00:03:12,020
the one who actually referred to
Moore's prediction as Moore's Law. Uh,

61
00:03:12,020 --> 00:03:15,820
and that raised, raised its profile
and highlighted its significance. Um,

62
00:03:16,360 --> 00:03:18,540
and you knows partly through
that pro popularization,

63
00:03:18,560 --> 00:03:22,540
we start to see actually Moore's Law
becoming a rallying call for the industry.

64
00:03:22,920 --> 00:03:26,700
Uh, a bit of a target for, uh,
chip companies to aspire to, uh,

65
00:03:26,700 --> 00:03:28,660
as well as just an observation
of what's happening. So it,

66
00:03:28,760 --> 00:03:31,820
it developed a little bit of a
felt self-fulfilling nature, um,

67
00:03:31,820 --> 00:03:35,140
in that the companies themselves were
actually trying to, to, to, to reach it.

68
00:03:36,320 --> 00:03:39,540
Um, but you asked about whether,
whether it's dead. Um, it's,

69
00:03:39,540 --> 00:03:41,940
it's been amazing how long it's lasted,
right? So right at the beginning,

70
00:03:42,740 --> 00:03:44,940
I think people were talking
about, you know, 10 years, uh,

71
00:03:44,940 --> 00:03:46,420
perhaps it could last
for as long as 10 years.

72
00:03:46,990 --> 00:03:50,820
We've seen that that doubling trend over
every two years has ca actually carried

73
00:03:50,820 --> 00:03:55,620
on, uh, from the early 1970s,
uh, to, to the, the 2020s.

74
00:03:56,160 --> 00:03:57,460
So we've seen, um, you know,

75
00:03:57,460 --> 00:04:00,900
the numbers of transitions on ship
that go from somewhere around 5,000 in,

76
00:04:00,900 --> 00:04:05,820
in the 1970s to more like 50
billion in the 2020s. Through that,

77
00:04:06,250 --> 00:04:08,340
that success of doubling every two years,

78
00:04:08,970 --> 00:04:11,020
it's really mind blowing
that improvement curve could,

79
00:04:11,020 --> 00:04:14,820
could stay at that expon
level for 50 years. Um,

80
00:04:15,880 --> 00:04:19,300
uh, so I think we, we are
starting to see it level off. Um,

81
00:04:19,520 --> 00:04:23,020
but it's been amazing, uh, what it's
achieved, uh, over the last 50 years.

82
00:04:23,600 --> 00:04:26,740
And actually that leveling off potentially
from a UK perspective is really

83
00:04:26,900 --> 00:04:27,660
interesting. Um,

84
00:04:27,660 --> 00:04:31,100
because that means perhaps there are new
technologies that need to be looked at

85
00:04:31,220 --> 00:04:33,980
that could, um, could help
to continue the curve,

86
00:04:34,170 --> 00:04:38,380
help continue that transformation. And
perhaps that's a chance for the UK to,

87
00:04:38,600 --> 00:04:41,340
um, uh, to really articulate the,

88
00:04:41,340 --> 00:04:43,260
the role it wants to play in
this industry moving forward.

89
00:04:43,520 --> 00:04:46,980
The UK government have, uh,
to some extent, at least,

90
00:04:47,770 --> 00:04:49,060
said what they would like to do.

91
00:04:49,400 --> 00:04:54,140
You know, it's clear how transformational
this this industry is. Um, uh,

92
00:04:54,140 --> 00:04:58,620
you know, access to chips is now a matter
of critical economic importance. Uh,

93
00:04:58,620 --> 00:05:01,020
it's not just something that's
important for consumer electronics.

94
00:05:01,030 --> 00:05:04,740
Every industry from medical devices to
defense and security to aerospace and

95
00:05:04,740 --> 00:05:08,300
automotive have critical reliance
on access to cutting edge chips.

96
00:05:08,320 --> 00:05:11,300
And I think we saw during the
pandemic that, uh, you know,

97
00:05:11,300 --> 00:05:14,820
a worldwide chip shortage actually
caused the automotive supply chain issues

98
00:05:14,820 --> 00:05:19,460
that led to more than a 25% slump in
auto production and some big backlogs.

99
00:05:19,460 --> 00:05:21,700
They're only just getting unwound. Uh,

100
00:05:21,720 --> 00:05:23,220
and it's worth noting
that chips themselves.

101
00:05:23,240 --> 00:05:25,220
So now one of the world's biggest markets,

102
00:05:25,410 --> 00:05:30,180
roughly half a trillion
dollar industry, um, and, uh,

103
00:05:30,180 --> 00:05:34,340
semiconductor chips and other world's
fourth most traded products after crude

104
00:05:34,340 --> 00:05:37,940
oil, refined oil and cars,
it's chips. So, uh, you know,

105
00:05:37,940 --> 00:05:40,260
it's not surprising that, that sometimes
people refer to them as like the,

106
00:05:40,260 --> 00:05:43,500
the new oil. They are very nearly as
traded and very nearly as vital, uh,

107
00:05:43,500 --> 00:05:45,540
for other worlds major industries.

108
00:05:45,600 --> 00:05:49,940
So it's not really surprising
that governments are taking
notice. Um, you know,

109
00:05:50,240 --> 00:05:52,780
as, as you and others
will know, the us, China,

110
00:05:52,880 --> 00:05:57,060
and Europe are slugging it out in a super
high stakes battle of supremacy where,

111
00:05:57,280 --> 00:06:00,060
um, you know, building a new
fabrication facilities often in the,

112
00:06:00,060 --> 00:06:00,900
the billions of dollars,

113
00:06:01,460 --> 00:06:06,340
I think tsm C'S latest plant in Arizona
triples their overall investment in the

114
00:06:06,340 --> 00:06:11,300
state of 40 billion. So it's, it's
really eye watering stakes. Um, I,

115
00:06:11,380 --> 00:06:12,940
I guess for the US
government, for instance,

116
00:06:13,220 --> 00:06:16,780
recently committed to more than 50 billion
to support industry and innovation in

117
00:06:16,780 --> 00:06:21,220
its chips act, um, last year. So, uh,
you mentioned the UK sort of setting out,

118
00:06:21,240 --> 00:06:25,220
its still, um, the, the context
is that right? The UK is a,

119
00:06:25,620 --> 00:06:28,940
a mid-sized economy. It can't invest,
uh, at the levels of the us, China,

120
00:06:28,940 --> 00:06:32,620
Europe, but it has a lot of strengths in
chip design and packaging and some next

121
00:06:32,620 --> 00:06:37,580
generation technologies like, uh, compound
semiconductors, 2D materials, but it,

122
00:06:37,580 --> 00:06:40,980
it can't just go toe to toe with
these larger economies. So it had, uh,

123
00:06:41,020 --> 00:06:45,700
a real challenge. Um, and, uh, I guess,

124
00:06:46,160 --> 00:06:50,260
uh, it's worth just talking about
the, um, the context of ARM as well.

125
00:06:50,300 --> 00:06:52,860
I think that was, that was
really important in the,
the genesis of the strategy.

126
00:06:53,120 --> 00:06:57,020
So, um, I guess the UK's flagship design,

127
00:06:57,080 --> 00:06:59,700
the chip design company arm,
which came out of the, um,

128
00:07:00,200 --> 00:07:03,180
the nineties investment in
computing around Cambridge, uh,

129
00:07:03,180 --> 00:07:06,460
some of you might remember the Acorn
Edes had a bit of a su suspect with a

130
00:07:06,460 --> 00:07:09,100
child. So Acorn essentially
developed into Arm, uh,

131
00:07:09,100 --> 00:07:13,060
became one of the world's leading chip
design specialists with their chips in

132
00:07:13,060 --> 00:07:17,100
95% of the world's smartphones,
for instance. And in, uh, in 2014,

133
00:07:17,100 --> 00:07:21,380
they were purchased by, uh, the Japanese
tech and venture capital company, uh,

134
00:07:21,380 --> 00:07:22,213
SoftBank.

135
00:07:22,330 --> 00:07:23,660
Just a quick aside from me,

136
00:07:23,660 --> 00:07:27,620
because that mention of Acorn computers
took me on a trip down memory lane,

137
00:07:28,000 --> 00:07:32,980
and I discovered that the computer that
I was a particularly proud owner of in

138
00:07:33,260 --> 00:07:35,460
1983, the Acorn Electron,

139
00:07:35,680 --> 00:07:38,620
was the biggest selling
computer at the time in the uk,

140
00:07:38,760 --> 00:07:42,900
but it only sold between 200 and 250,000.

141
00:07:43,160 --> 00:07:47,620
And it just made me wonder is that
anybody listening to this podcast who also

142
00:07:47,720 --> 00:07:52,300
had an acorn electron, if so, perhaps
you could tweet us at Physics world.

143
00:07:52,680 --> 00:07:55,300
But anyway, back to Louis Barson.

144
00:07:55,400 --> 00:07:57,700
And this really illustrates the
dilemma that we have here in the uk.

145
00:07:58,000 --> 00:08:02,100
So arm sold for I think 32
billion, which is, you know,

146
00:08:02,100 --> 00:08:05,620
a lot of money and is clearly, uh,
clearly a success by many measures. But,

147
00:08:05,720 --> 00:08:06,300
you know, some,

148
00:08:06,300 --> 00:08:09,580
some felt and still feel that it may not
have valued its national significance

149
00:08:09,580 --> 00:08:14,380
appropriately and may have actually
weakened the, the domestic ecosystem. Um,

150
00:08:14,520 --> 00:08:16,580
so this, this was a big
debate, uh, and after this,

151
00:08:16,580 --> 00:08:20,060
the government ran a big review of its
strategy around critical technologies,

152
00:08:20,560 --> 00:08:24,260
um, which led to, uh, a new
more strategic approach to, to,

153
00:08:24,360 --> 00:08:28,100
to national critical technologies and
to the commitment to a semiconductor

154
00:08:28,500 --> 00:08:32,500
strategy. Um, so this felt like a big
opportunity for the UK to start to think,

155
00:08:32,840 --> 00:08:33,220
uh, you know,

156
00:08:33,220 --> 00:08:36,180
a bit more strategically about the role
he wants to play and to carve out a bit

157
00:08:36,180 --> 00:08:41,100
of a niche. Um, so that's
where the IOP recently ran,

158
00:08:41,320 --> 00:08:43,860
uh, an impact project
in partnership with, uh,

159
00:08:43,860 --> 00:08:45,980
the Royal Academy of Engineering to,

160
00:08:46,000 --> 00:08:48,620
to support the UK government in thinking
about the position they wanted to take,

161
00:08:48,800 --> 00:08:52,180
uh, towards this, you know,
critical technology. Uh,

162
00:08:52,240 --> 00:08:55,540
and we got together an expert round
table who produced, I think some really,

163
00:08:55,540 --> 00:08:59,660
you know, insightful recommendations
that helped inform the evidence base. Um,

164
00:09:00,680 --> 00:09:04,220
and, uh, you know, when the strategy
came out earlier this year, uh,

165
00:09:04,220 --> 00:09:07,500
we gave it a cautious welcome. And I
think that was firstly because, you know,

166
00:09:07,500 --> 00:09:10,260
it was really important to
have the strategy itself, uh,

167
00:09:10,440 --> 00:09:13,140
for the industry to hear the
strategic significance that, you know,

168
00:09:13,140 --> 00:09:16,540
the government places on this technology
for the world to see the value replaced

169
00:09:16,680 --> 00:09:20,540
in the industry and, and the plans that
we have. Second, cuz it felt, you know,

170
00:09:20,540 --> 00:09:22,980
really well structured, it was, uh,

171
00:09:22,980 --> 00:09:26,380
it was good to have the focus on growth
in the domestic industry as well as,

172
00:09:26,380 --> 00:09:28,620
you know, a recognition
of the importance of, uh,

173
00:09:29,180 --> 00:09:32,460
security of supply of chips for the
industrial sectors that are big users from

174
00:09:32,460 --> 00:09:35,660
healthcare to water. Uh, and,
you know, looking at the, uh,

175
00:09:35,660 --> 00:09:37,980
the national security dimensions, uh,

176
00:09:37,980 --> 00:09:41,100
so not just from the perspective
of security of, for instance,

177
00:09:41,300 --> 00:09:42,980
wireless infrastructure,
you know, 5G infrastructure,

178
00:09:42,980 --> 00:09:45,700
that's been such a big thing, but
also, you know, the vital role of,

179
00:09:45,700 --> 00:09:49,740
of chips in, in defense and
security. Um, so it seemed,

180
00:09:49,760 --> 00:09:52,580
it seemed well structured and it seemed
to recognize this narrative of this

181
00:09:52,580 --> 00:09:57,340
being a chance for us to, uh,
to position, to, to dive in. Uh,

182
00:09:57,360 --> 00:09:58,340
and third, uh,

183
00:09:58,400 --> 00:10:01,700
cuz it seemed to take on board the I
o P round table recommendations, uh,

184
00:10:01,700 --> 00:10:04,940
really, really well, uh, including,
uh, a recommendation around, uh,

185
00:10:04,940 --> 00:10:07,860
having in national infrastructure
initiative and, uh, you know,

186
00:10:07,860 --> 00:10:12,420
the joined up skills focus. So,
um, so that was all good. Uh,

187
00:10:12,440 --> 00:10:15,180
should also say, I guess it was, you
know, it was a relatively modest strategy.

188
00:10:15,300 --> 00:10:19,820
I guess the, the scale of the strategy
was, was 1 billion over 10 years. Um,

189
00:10:20,240 --> 00:10:21,660
and we need to recognize in the, in the,

190
00:10:21,660 --> 00:10:24,580
in the context of worldwide semiconductor
investment, that's not huge,

191
00:10:25,320 --> 00:10:29,340
but it felt like a really
important starting point
and statement of intent. So,

192
00:10:29,680 --> 00:10:33,620
um, you know, really I think this
gives, gives the UK the chance to, um,

193
00:10:33,680 --> 00:10:37,180
be using the ambition and the momentum
and the strategy to prepare to,

194
00:10:37,180 --> 00:10:38,860
to build on some of our strengths and,

195
00:10:38,860 --> 00:10:42,100
and develop the semiconductor technologies
of tomorrow so that when the, uh,

196
00:10:42,100 --> 00:10:44,140
inflection point for, uh, you know,

197
00:10:44,140 --> 00:10:47,300
Moore's law slowing for silicon
versus other technologies comes,

198
00:10:47,880 --> 00:10:50,460
we are ready to join the race for
the headstart and that we've got the,

199
00:10:50,480 --> 00:10:53,780
the stomach to invest at the scales
that will be needed when the time comes.

200
00:10:54,080 --> 00:10:56,020
Th this 1 billion pounds over 10 years,

201
00:10:56,360 --> 00:11:00,940
is that the sort of timescale where we'll
need to be investing more before those

202
00:11:00,960 --> 00:11:01,900
10 years are out?

203
00:11:02,340 --> 00:11:05,420
I think, I think almost certainly is the,
uh, is the answer, the honest answer.

204
00:11:05,980 --> 00:11:10,700
I mean, this is a starting point and,
you know, over, over the next decade, uh,

205
00:11:10,710 --> 00:11:14,780
we're gonna see quite a lot of,
um, uh, of development, I think,

206
00:11:14,780 --> 00:11:17,780
in the semiconductor industry. So, um,

207
00:11:18,140 --> 00:11:20,500
I think we're almost certainly gonna
be hearing more about what the,

208
00:11:20,560 --> 00:11:23,620
the government thinks about
semiconductors and for really this,

209
00:11:23,620 --> 00:11:26,700
this strategic direction to
just be a starting point. Um,

210
00:11:26,700 --> 00:11:29,260
and it's really good news. I think
they've established a uk, uh,

211
00:11:29,260 --> 00:11:32,900
industrial advisory council to help guide
the development and the evolution of

212
00:11:32,900 --> 00:11:34,940
the strategy. Uh, I know, um,

213
00:11:34,940 --> 00:11:37,820
the people I've spoken to and
the team are very open to, uh,

214
00:11:37,880 --> 00:11:39,500
the idea that actually, you know,

215
00:11:39,500 --> 00:11:42,700
we may need to think again and actually
inject more ambition of the strategy at,

216
00:11:42,700 --> 00:11:44,980
at the moment when it becomes
clear, you know, what the,

217
00:11:44,980 --> 00:11:46,420
the next step should be for the uk.

218
00:11:47,520 --> 00:11:52,220
Should the physics community be interested
in this and should everybody else be

219
00:11:52,220 --> 00:11:53,053
interested in this?

220
00:11:53,520 --> 00:11:56,220
Uh, yes and yes, I think
is the answer, <laugh>,

221
00:11:56,450 --> 00:12:00,740
what Moore's role really shows is how
central cutting edge physics and material

222
00:12:00,740 --> 00:12:05,100
science is to, uh, the digital revolution.

223
00:12:05,760 --> 00:12:07,940
So it shows that, uh, through reducing,

224
00:12:08,130 --> 00:12:11,380
through clever physics and material
science through miniaturizing chips to

225
00:12:11,380 --> 00:12:14,940
increasing the number of transistors,
increasing the number of, you know,

226
00:12:14,940 --> 00:12:19,020
ones and zeros you can represent
on a, on a chip. Um, you, uh,

227
00:12:19,300 --> 00:12:23,700
increase computing power and you reduce
the cost of computing technology.

228
00:12:24,520 --> 00:12:28,460
Um, so that really shows that, you
know, it's actually physics and, uh,

229
00:12:28,460 --> 00:12:32,660
a material science right at the heart
of the digital revolution. Um, uh,

230
00:12:32,660 --> 00:12:35,260
Moore's law really charts the
contribution, I would say, of,

231
00:12:35,260 --> 00:12:39,500
of physics and material science to,
um, to, to the digital, technological,

232
00:12:39,500 --> 00:12:43,900
technological revolution.
And, um, I think, you know,

233
00:12:43,900 --> 00:12:46,700
beyond the physics community,
obviously, uh, everyone, uh,

234
00:12:46,840 --> 00:12:51,420
it should be interested because, uh,
this is, this is transforming, um, the,

235
00:12:51,420 --> 00:12:52,700
the way in which we, we live and work,

236
00:12:52,940 --> 00:12:56,740
everyone now has a smartphone in
their pocket. Uh, in the 1965 when,

237
00:12:56,740 --> 00:12:59,220
when Gordon Moore coined
the term, uh, nobody, uh,

238
00:12:59,220 --> 00:13:01,700
had a smartphone in the pocket that was
not even in the realms of something that

239
00:13:01,700 --> 00:13:05,620
could be thought about. Um, uh,
cuz transistors, you know, you'd,

240
00:13:05,620 --> 00:13:08,740
you'd hold them in your hand and they'd
be, uh, uh, you know, a valve or a,

241
00:13:08,840 --> 00:13:13,660
in a, a, you know, in a light bulb
shaped, uh, uh, uh, apparatus. So,

242
00:13:14,120 --> 00:13:18,460
um, I think, you know, if, if,
if you care about the revolution,

243
00:13:18,460 --> 00:13:20,780
if you care about the direction of
transformation in sectors, you should,

244
00:13:20,780 --> 00:13:25,060
you should care about, uh, uh,
Moore law and semiconductors. Um,

245
00:13:25,060 --> 00:13:27,220
and that's true from a, uh,

246
00:13:27,430 --> 00:13:29,820
those in the physics community who care
about these things, but also, you know,

247
00:13:29,820 --> 00:13:32,140
those in, in wider, uh, society
who care about these things.

248
00:13:32,530 --> 00:13:36,940
Whether Moore's law is dead on life
support or otherwise, the problem is,

249
00:13:36,960 --> 00:13:40,660
whilst making the transistor smaller
means that the computers are faster,

250
00:13:40,990 --> 00:13:45,820
there is a limit to how small
and how many you can fit

251
00:13:46,120 --> 00:13:46,953
on a chip.

252
00:13:46,960 --> 00:13:48,340
In, in simple terms, uh, you know,

253
00:13:48,340 --> 00:13:51,700
transistor is a switch that can be
turned on or off, so it can represent,

254
00:13:51,700 --> 00:13:54,820
you know, the one or zero of,
of binary code, which is the,

255
00:13:55,040 --> 00:13:59,380
the basic language that can be to speak.
So, uh, the more transistors you have,

256
00:13:59,600 --> 00:14:01,860
the more ones and zeros
you can represent, uh,

257
00:14:02,000 --> 00:14:06,470
and the more complex calculations
you can do more quickly. Um,

258
00:14:06,850 --> 00:14:09,870
so, so shrinking those transistors means
you get more on a chip and each chip

259
00:14:09,870 --> 00:14:14,110
becomes able to do, to do more.
Uh, but you're absolutely right.

260
00:14:14,110 --> 00:14:17,270
We're starting to get, uh, the,
the, the individual elements on a,

261
00:14:17,270 --> 00:14:20,470
on a chip are starting to get
so small. Um, you know, we get,

262
00:14:20,470 --> 00:14:22,550
we're sub 10 nanometer now. I think the,

263
00:14:22,550 --> 00:14:27,190
the latest process is ar around three
nanometers, um, which is, you know,

264
00:14:27,190 --> 00:14:30,790
getting to the, uh, the scale
of individual atoms. Um,

265
00:14:30,790 --> 00:14:33,790
you start to see quantum effects
coming into play that you don't,

266
00:14:33,790 --> 00:14:37,150
things that we don't see happening in
everyday life happen in chips because

267
00:14:37,180 --> 00:14:41,910
they're so small. Um, and, uh,
we are definitely beginning to,

268
00:14:42,010 --> 00:14:44,070
to, uh, reach, uh,

269
00:14:44,290 --> 00:14:48,110
the limits because I suppose if this
continues on for another, uh, 50 years,

270
00:14:48,110 --> 00:14:50,950
we're gonna be, you know,
individual elements would
have to be smaller than, uh,

271
00:14:51,020 --> 00:14:55,020
individual atoms, which, uh, is
clearly, clearly not feasible. Um,

272
00:14:55,400 --> 00:14:58,660
but there are interesting technologies
that are starting to extend the

273
00:14:58,660 --> 00:15:01,420
boundaries of what that could
mean. So, um, for instance,

274
00:15:01,720 --> 00:15:05,980
3D chips that use more complex chip
architectures to mean that, uh,

275
00:15:06,320 --> 00:15:10,980
you can squeeze, uh, more, more life
out of Moore's law, uh, are starting to,

276
00:15:11,040 --> 00:15:15,860
to become more popular. Um,
and I mentioned a couple
of, uh, potential sort of,

277
00:15:16,120 --> 00:15:19,420
um, UK strength opportunities
previously, but, uh,

278
00:15:19,660 --> 00:15:23,220
compound semiconductors are a great
example of, um, a new material science,

279
00:15:23,220 --> 00:15:27,380
physics technology that, um, um, is
using multiple elements, not, um,

280
00:15:28,280 --> 00:15:31,500
uh, not just the traditional
silicon to create substrates for,

281
00:15:31,640 --> 00:15:34,900
for chips that could be more
efficient and effective. Um,

282
00:15:35,040 --> 00:15:39,060
2D materials such as graphene and
others, um, have some potential if,

283
00:15:39,080 --> 00:15:41,380
if we can manufacture them in,
in ways that give them, uh,

284
00:15:41,380 --> 00:15:43,580
semiconductor characteristics. Um,

285
00:15:44,280 --> 00:15:46,980
and there's also interesting
experiments around using, um,

286
00:15:46,980 --> 00:15:49,380
photonics or using lights to, uh,

287
00:15:49,380 --> 00:15:53,900
to do computing in ways that could be
more efficient and effective. So for me,

288
00:15:53,900 --> 00:15:56,500
it's exciting that Moore's law is
starting to slow because we're,

289
00:15:56,500 --> 00:15:57,700
we're reaching the limits of that,

290
00:15:57,700 --> 00:15:59,980
that way of looking at
semiconductors and starting to,

291
00:16:00,570 --> 00:16:03,300
it's starting to mean we need to look
at other ways of doing it, which, uh,

292
00:16:03,300 --> 00:16:04,500
to me is very exciting. And to,

293
00:16:04,540 --> 00:16:06,980
I know to others in the physics and and
engineering community is very exciting

294
00:16:07,000 --> 00:16:07,833
as well.

295
00:16:08,680 --> 00:16:10,940
One such person is Thomas Verda.

296
00:16:11,500 --> 00:16:15,580
Delma. I'm a researcher
at NEC Labs America, uh,

297
00:16:15,580 --> 00:16:16,900
based in Princeton, New Jersey.

298
00:16:17,600 --> 00:16:22,380
And my research field is on the
topic of optical computing or sort of

299
00:16:22,450 --> 00:16:27,180
next generation computer hardware
with using optics optical

300
00:16:27,620 --> 00:16:29,340
computers. Uh,

301
00:16:30,210 --> 00:16:34,060
they rely on the physics of, um,

302
00:16:34,190 --> 00:16:38,220
light waves in order to
perform some computations.

303
00:16:38,440 --> 00:16:42,960
So traditional electronic
computers are based on,

304
00:16:43,260 --> 00:16:44,093
um,

305
00:16:44,540 --> 00:16:48,750
transer gate and information
is encoded into a voltage

306
00:16:49,860 --> 00:16:53,750
that if it's above a certain threshold,
then there's a one, if it's below,

307
00:16:53,750 --> 00:16:57,270
it's a zero. So, very binary,
very efficient way of, uh,

308
00:16:57,280 --> 00:17:01,590
connecting the device to binary logic. Uh,

309
00:17:01,730 --> 00:17:06,310
on electronics in optics, what you
try to do is do the same thing,

310
00:17:07,290 --> 00:17:07,610
uh,

311
00:17:07,610 --> 00:17:12,510
but instead of storing information voltage
you store in some property of light,

312
00:17:12,820 --> 00:17:16,070
like it's amplitude or it's
color, or it's polarization.

313
00:17:16,890 --> 00:17:21,430
And so the challenge here is
that there is no equivalent

314
00:17:21,930 --> 00:17:25,630
to a transistor in optics, right?

315
00:17:26,530 --> 00:17:27,550
So what I,

316
00:17:27,550 --> 00:17:32,150
what I've researched when I did my
PhD at Princeton University was,

317
00:17:32,250 --> 00:17:36,630
how do we think about a computer where
we cannot have an optical transistor,

318
00:17:36,880 --> 00:17:41,350
right? And in order to take
it full advantage of, um,

319
00:17:42,330 --> 00:17:44,000
light properties for computing,

320
00:17:44,630 --> 00:17:49,420
what we needed to do is to use linear
computations with optics and non-linear

321
00:17:49,420 --> 00:17:52,340
computations with some el
opto electronic device.

322
00:17:53,280 --> 00:17:57,900
And the architecture that
most resembled what these

323
00:17:57,950 --> 00:18:02,220
advantages were neuro networks,
coincidentally, right?

324
00:18:02,220 --> 00:18:06,660
Which were on vogue, uh, since 2009,

325
00:18:07,200 --> 00:18:11,580
um, with, with machine
learning. So optical computing,

326
00:18:11,580 --> 00:18:14,300
you'll hear from different researchers,
they come in multiple flavors.

327
00:18:14,380 --> 00:18:19,380
A lot of people focus on the linear
algebra part of it by optimizing, uh,

328
00:18:19,380 --> 00:18:20,020
sort of matrix,

329
00:18:20,020 --> 00:18:23,580
vector multiplication and reducing
the energy consumption of that.

330
00:18:24,320 --> 00:18:29,020
So creating like, uh, accelerator
chips, um, that focus on,

331
00:18:29,160 --> 00:18:30,780
on this aspect of computation.

332
00:18:31,450 --> 00:18:36,180
Some others would look into
image processing for convolution

333
00:18:36,420 --> 00:18:39,100
operators. And that
there, you wanna do, uh,

334
00:18:39,100 --> 00:18:41,860
you wanna process as many images
as possible, as fast as possible,

335
00:18:41,860 --> 00:18:46,260
things like that. But in all the
situations, what I see is they,

336
00:18:46,260 --> 00:18:49,460
they're gonna act as, um,
accelerator processors.

337
00:18:49,460 --> 00:18:52,780
They need to work in
conjunction with, um, um,

338
00:18:52,780 --> 00:18:54,980
state-of-the-art electronics processors.

339
00:18:56,120 --> 00:19:00,900
So you shouldn't think of optical
computing as an evolution or an iteration

340
00:19:00,960 --> 00:19:04,020
of the, the history of computers.

341
00:19:04,320 --> 00:19:08,620
You should think of a new
technology that will enable, um, um,

342
00:19:09,040 --> 00:19:12,420
you know, accelerator functions in, uh,

343
00:19:12,560 --> 00:19:16,340
in for multiple tasks that society
will need in the future. That's,

344
00:19:16,340 --> 00:19:17,420
that's my opinion at least.

345
00:19:17,800 --> 00:19:20,620
But we'll still need
other types of computers.

346
00:19:22,010 --> 00:19:23,860
Well, well, there's certainly a need,

347
00:19:24,400 --> 00:19:29,020
but to me it's unclear
what's gonna be, um, if, if,

348
00:19:29,080 --> 00:19:31,780
if ever there's gonna be, uh,

349
00:19:31,860 --> 00:19:35,740
a new paradigm that will replace
the current, uh, sort of, uh,

350
00:19:35,890 --> 00:19:38,860
digital electronic, uh, computers.

351
00:19:40,010 --> 00:19:43,630
So people talk about, uh,
quantum computing, for example,

352
00:19:45,010 --> 00:19:49,110
um, or, uh, using more advanced, um,

353
00:19:49,110 --> 00:19:52,830
physics like spintronics and
things like that. So, um,

354
00:19:53,090 --> 00:19:55,270
but so far as I know, as I've seen,

355
00:19:55,550 --> 00:20:00,230
I don't think none of those technologies
will replace fully the kinds of, uh,

356
00:20:00,280 --> 00:20:02,190
algorithms that current computers can run.

357
00:20:02,740 --> 00:20:04,990
Okay? But does that mean that, I mean,

358
00:20:04,990 --> 00:20:08,190
should we care really if Moore's law ends?

359
00:20:08,850 --> 00:20:13,310
Yes. Yes. Uh, it, it has a lot
of consequences in my opinion.

360
00:20:13,770 --> 00:20:18,590
Um, the, uh, the reason why
we should care is because

361
00:20:20,130 --> 00:20:25,070
now we are entering a new period
of, um, machine intelligence.

362
00:20:26,530 --> 00:20:31,150
And so the amount of
computation demand required by

363
00:20:31,180 --> 00:20:36,090
society is now going to scale, uh,

364
00:20:36,230 --> 00:20:40,370
beyond the individual
human demands, right?

365
00:20:40,390 --> 00:20:44,650
So now we have machine to machine
sort of communication processes.

366
00:20:44,740 --> 00:20:46,250
We'll have automated systems,

367
00:20:46,820 --> 00:20:50,940
we'll have self-driving
cars and things like that.

368
00:20:51,680 --> 00:20:56,450
So machines, we can build more, many
more machines than the human population.

369
00:20:56,470 --> 00:20:59,410
So there's no upper upper
limit on that. So as,

370
00:21:00,310 --> 00:21:05,290
as the men for computation for this,
uh, AI models, for example, with the,

371
00:21:05,430 --> 00:21:08,330
uh, Chad g p t, as they increase,

372
00:21:09,100 --> 00:21:13,890
we're gonna need more computing power.
And with the end of Moore's law, what,

373
00:21:14,070 --> 00:21:17,770
you know, what we were used
to in the past was, okay,

374
00:21:17,770 --> 00:21:22,250
we just wait two years and we
gain an, uh, uh, an immediate,

375
00:21:23,310 --> 00:21:27,250
you know, two x efficiency after two
years, right? Because of more slow.

376
00:21:27,510 --> 00:21:30,770
Now that's not true anymore.
If you need more computing,

377
00:21:30,770 --> 00:21:35,370
you're gonna have to spend,
um, more money on chips, pay,

378
00:21:35,470 --> 00:21:38,970
pay more for transistors, and you
gotta, it's gonna consume more, uh,

379
00:21:39,610 --> 00:21:40,610
electricity, right?

380
00:21:41,310 --> 00:21:46,170
So we're entering this new
regime from abundance to,

381
00:21:46,230 --> 00:21:49,850
to scarcity, right? It's
gonna become like, um,

382
00:21:50,650 --> 00:21:55,290
a commodity like oil, the chips as you,
as you know, as you probably heard on,

383
00:21:56,430 --> 00:22:00,370
uh, on the news. Yeah. So,
so I think in that context,

384
00:22:00,590 --> 00:22:04,610
that's why more Moore's law
ending is, is important.

385
00:22:05,190 --> 00:22:09,570
You are pretty certain from what you're
saying, that Moore's law is dead.

386
00:22:09,950 --> 00:22:14,810
If it's not dead, it certainly has
slowed down. And so that's, uh,

387
00:22:14,810 --> 00:22:19,800
very visible from the, um, the,

388
00:22:19,900 --> 00:22:21,520
the sort of,

389
00:22:21,520 --> 00:22:25,800
the number of transistors on every
chip as that they evolve has,

390
00:22:26,020 --> 00:22:30,520
has not been, um, following on
the same trajectory as it has, uh,

391
00:22:30,520 --> 00:22:34,200
as it had been since the
1970s. So I think so,

392
00:22:34,420 --> 00:22:38,280
but more importantly than more
slow, there is another, um,

393
00:22:39,190 --> 00:22:42,760
scaling law, so to speak, that's
called, that I pay more attention to,

394
00:22:43,370 --> 00:22:47,160
which is called the Denars
law. So the Denars law,

395
00:22:48,030 --> 00:22:49,440
it's more like a scaling law,

396
00:22:49,500 --> 00:22:53,360
is the sense that as the
dimensions of a device go down,

397
00:22:53,660 --> 00:22:55,000
so those power consumption,

398
00:22:55,980 --> 00:23:00,840
so smaller devices within the
dielectrics and shorter channels, uh, um,

399
00:23:00,950 --> 00:23:05,440
they improve the, um,
the frequency, the cost,

400
00:23:06,100 --> 00:23:10,720
and the power consumption of the
device. So it, it sort of had this, uh,

401
00:23:10,750 --> 00:23:15,560
virtual cycle of more slow
in parallel going on where

402
00:23:15,780 --> 00:23:19,960
the chips as you, you packed
more transistors on chip
by making them smaller,

403
00:23:20,390 --> 00:23:23,480
they also became more efficient. You see?

404
00:23:23,940 --> 00:23:27,600
And that stopped around 2005, and
that's when you saw your, you know,

405
00:23:27,600 --> 00:23:32,320
your computer CPUs, if you paid attention,
the frequency they operated at. Um,

406
00:23:33,180 --> 00:23:37,580
and they had been growing like
doubling every couple of years,

407
00:23:38,250 --> 00:23:39,340
400 megahertz,

408
00:23:39,340 --> 00:23:44,300
800 megahertz until they
stopped at about three to four

409
00:23:44,490 --> 00:23:49,460
gigahertz, right? And then since then,
instead of improving the frequency, uh,

410
00:23:49,460 --> 00:23:53,340
chip manufacturers started
adding more cores, so to speak.

411
00:23:54,000 --> 00:23:58,470
So how many cores does your
Intel CPU has, right? One core,

412
00:23:58,530 --> 00:24:02,550
two cores, four, eight, right?
Well, at the same clock frequency.

413
00:24:03,130 --> 00:24:08,010
So instead of, um, reducing the
size and using a single core,

414
00:24:08,320 --> 00:24:11,810
they are increasing the size of the
chip using more transducers that way.

415
00:24:12,430 --> 00:24:17,050
But more cores, you have more
power consumption, right? And that,

416
00:24:17,070 --> 00:24:22,050
that's the, the direct effect of the
end of the des then they're scaling low,

417
00:24:22,620 --> 00:24:23,453
right?

418
00:24:23,750 --> 00:24:26,970
So that's something that is
also interesting to think
about beyond the more,

419
00:24:26,970 --> 00:24:27,803
more slow.

420
00:24:28,070 --> 00:24:32,810
Do, do. How do you feel about the future?
Because if you look at the news, um,

421
00:24:33,160 --> 00:24:36,410
reports, there's great
concern about ai, for example,

422
00:24:36,920 --> 00:24:41,270
that maybe there's great
concern about, um, less,

423
00:24:41,380 --> 00:24:44,430
with less noise around it,
this concern about, um,

424
00:24:44,900 --> 00:24:47,590
transistors becoming a commodity
than the same way as oil.

425
00:24:48,050 --> 00:24:49,070
What's your feeling on.

426
00:24:49,070 --> 00:24:50,510
It? I'm a hardware researcher.

427
00:24:50,710 --> 00:24:55,070
I usually concern myself with thinking
about the problems related to,

428
00:24:55,650 --> 00:24:59,160
uh, uh, power consumption.

429
00:24:59,620 --> 00:25:04,120
How can we actually support
this demand? How can we, uh,

430
00:25:04,150 --> 00:25:08,720
improve the efficiency of
computing so that it's, uh, um,

431
00:25:08,940 --> 00:25:13,940
it doesn't contribute even more
to, to the, uh, to climate change?

432
00:25:14,640 --> 00:25:19,460
So, but yes, it, it concerns me
on a, you know, citizen level,

433
00:25:19,880 --> 00:25:22,300
uh, the, uh, development
of the, the, this,

434
00:25:22,360 --> 00:25:26,980
the high speed development of AI models
and how they're actually becoming more

435
00:25:26,980 --> 00:25:30,980
integrated in society. I'm not, um, I'm,

436
00:25:31,040 --> 00:25:32,860
I'm an optimist though
on that, on that front.

437
00:25:33,340 --> 00:25:35,850
I do think that on a personal opinion,

438
00:25:37,000 --> 00:25:40,980
that the, um, the, uh,

439
00:25:40,980 --> 00:25:45,580
the AI models that we have seen in

440
00:25:45,640 --> 00:25:49,340
recent years, in particular, the,
uh, the G P T three and four,

441
00:25:49,850 --> 00:25:53,520
they are getting close to, um,

442
00:25:55,940 --> 00:26:00,700
to, they're getting close to hitting
the limitations of hardware, right?

443
00:26:00,700 --> 00:26:03,860
They're already too expensive to
train on the millions of dollars.

444
00:26:04,040 --> 00:26:07,700
So the question is, if they
want to increase, does that,

445
00:26:07,730 --> 00:26:09,460
that model by tenfold,

446
00:26:10,080 --> 00:26:13,460
are they prepared to spend
10 times more money, right?

447
00:26:13,800 --> 00:26:18,700
So I think that's gonna be a real
bottleneck in the, in the short term.

448
00:26:19,680 --> 00:26:24,300
And so hardware engineers like me
are trying to catch up to that trend,

449
00:26:24,430 --> 00:26:28,940
right? Um, not not just me, but
of course the whole industry. And,

450
00:26:29,240 --> 00:26:32,100
um, so if we don't catch up,

451
00:26:33,200 --> 00:26:37,980
and if that trend continues
to increase, then, um,

452
00:26:39,490 --> 00:26:41,720
right now people don't
pay attention to this,

453
00:26:41,900 --> 00:26:46,420
but data centers already consumed
3% of the global electricity.

454
00:26:47,400 --> 00:26:51,460
So if you multiply that by
10, that becomes 30%, right?

455
00:26:52,240 --> 00:26:56,780
And, uh, 3% is tolerable, 30% isn't. Okay.

456
00:26:56,960 --> 00:27:00,220
So between now and then, then we have to

457
00:27:02,000 --> 00:27:04,220
really solve this problem
one way or another.

458
00:27:05,120 --> 00:27:07,580
That's mainly what I
concern myself with so far.

459
00:27:08,450 --> 00:27:11,180
When is this time when we're
gonna have to solve this problem?

460
00:27:11,440 --> 00:27:15,820
Um, it, it depends. It depends on
the, uh, appetite for, you know,

461
00:27:16,650 --> 00:27:21,540
machine learning models to keep
increasing, right? So in recent past,

462
00:27:21,610 --> 00:27:25,700
because it was unbounded, the,
uh, training models were doubling,

463
00:27:26,480 --> 00:27:29,460
uh, in size in, in a
timeframe of less than a year,

464
00:27:29,580 --> 00:27:33,900
I think every year in model doubled.
But, so if you continue that,

465
00:27:34,700 --> 00:27:39,300
a factor of 10 is just three years
away or four years away, right? Um,

466
00:27:39,930 --> 00:27:42,670
but it's not gonna continue like
that, at least not in the short term.

467
00:27:43,250 --> 00:27:47,930
So I'm thinking, I'm thinking
a decade. Uh, that's our,

468
00:27:48,470 --> 00:27:52,210
that's what I have in my mind
in a decade. We should have, um,

469
00:27:52,720 --> 00:27:56,330
more efficient accelerators for, uh,

470
00:27:56,600 --> 00:27:59,650
this kind of computing
that machines require. Um,

471
00:28:00,630 --> 00:28:04,890
and also working in parallel with that,
we have a lot of innovations from the,

472
00:28:04,910 --> 00:28:09,470
uh, software, uh, from
the software side with,

473
00:28:09,770 --> 00:28:11,630
uh, better mathematics,

474
00:28:12,290 --> 00:28:15,950
better ways of representing
information digitally,

475
00:28:16,290 --> 00:28:20,470
better neur network models. So
let's not forget that the, uh,

476
00:28:20,830 --> 00:28:25,710
transformer model is only a few
years old, and, you know, what's,

477
00:28:26,620 --> 00:28:30,550
that could be a new model that
will power the next generation, uh,

478
00:28:30,660 --> 00:28:35,230
sort of AI model in which it's gonna
be way more efficient, for example.

479
00:28:35,390 --> 00:28:37,270
I could see that coming as well.

480
00:28:38,460 --> 00:28:41,750
Yeah. What would you like to
see happen? In an ideal world?

481
00:28:43,590 --> 00:28:48,310
I think computers, um, I,

482
00:28:48,390 --> 00:28:51,510
I kind of partake Steve
Jobs vision for this.

483
00:28:51,580 --> 00:28:55,630
It's supposed to be a
tool to empower humans and

484
00:28:56,280 --> 00:28:59,630
human communication,
human uh, productivity.

485
00:29:00,730 --> 00:29:05,630
And so I do like the sort of the,

486
00:29:05,810 --> 00:29:09,470
um, the direction that
it has taken so far.

487
00:29:09,500 --> 00:29:13,470
This AI models are helping,
um, a lot of people,

488
00:29:13,470 --> 00:29:16,430
particularly knowledge workers, right?

489
00:29:16,810 --> 00:29:21,240
So I would like that to move towards,

490
00:29:22,020 --> 00:29:24,400
um, the, um, cyber physical space.

491
00:29:25,300 --> 00:29:30,120
So instead of having
your AI help you draft a

492
00:29:30,200 --> 00:29:33,520
better text or par your
code or things like that,

493
00:29:34,150 --> 00:29:38,320
it'd be nice if you could have AI help
you with your, uh, physical tasks.

494
00:29:38,690 --> 00:29:42,420
Maybe you are fixing your car and you can

495
00:29:43,620 --> 00:29:47,580
diagnose it for you, and it can
help you figure out which, you know,

496
00:29:47,790 --> 00:29:51,980
which knobs to, to turn, which screws
to tighten, things like that, right?

497
00:29:53,040 --> 00:29:57,920
Um, so more, more cyber physical
systems. I think that's, uh,

498
00:29:57,940 --> 00:30:00,480
that's what I would like, uh, to see.

499
00:30:00,990 --> 00:30:05,960
Another situation is, uh, think about, um,

500
00:30:07,450 --> 00:30:12,360
human machine interaction, right?
Being more, having a higher bandwidth,

501
00:30:13,260 --> 00:30:17,760
uh, of communication between yourself
and the machine, rather than right now,

502
00:30:17,760 --> 00:30:22,280
we, we can speak to it, we can type, can
we think of something better than that?

503
00:30:22,300 --> 00:30:26,640
So that's something that, uh,
that company, uh, Elon Musk's, uh,

504
00:30:26,640 --> 00:30:29,400
Neuralink is working towards
the pain machine interface.

505
00:30:29,540 --> 00:30:33,920
But if you ignore all that,
then um, you could have,

506
00:30:35,740 --> 00:30:37,920
for example, in the
context of remote surgery,

507
00:30:38,100 --> 00:30:42,320
you could have a specialist
in a particular kind of
surgery and a very advanced

508
00:30:42,500 --> 00:30:46,360
robots with, um, with a very, um, um,

509
00:30:46,750 --> 00:30:49,560
precise movements and precise
instruments, for example.

510
00:30:50,060 --> 00:30:54,470
And you could have these two things
sitting hundreds of kilometers apart.

511
00:30:55,250 --> 00:30:59,380
So you'll be interesting if that,

512
00:31:00,200 --> 00:31:02,460
uh, could happen in my lifetime, right?

513
00:31:02,480 --> 00:31:06,780
So an ability for a surgeon to perform
remote surgery far away, right?

514
00:31:06,780 --> 00:31:11,180
That would enable a lot, um, a lot of
progress in, in medicine, in my opinion.

515
00:31:11,920 --> 00:31:15,900
And I know for a fact that
in order for that to happen,

516
00:31:16,120 --> 00:31:18,480
you're gonna need, uh,

517
00:31:18,480 --> 00:31:22,040
better computers and faster
computers running on both sides.

518
00:31:23,270 --> 00:31:27,600
Like just higher bandwidth communication
is not enough because of a latency

519
00:31:27,600 --> 00:31:31,800
constraint. You need to, um, so just,

520
00:31:31,800 --> 00:31:36,200
so just to, to give you a
sense, let's say it takes, uh,

521
00:31:36,200 --> 00:31:40,800
20 milliseconds for any information to
go from one side of the globe to another,

522
00:31:41,660 --> 00:31:46,600
but humans require much less
than five milliseconds feedback

523
00:31:46,710 --> 00:31:51,360
time in order for you to, uh, really
enhance your fine motor skills.

524
00:31:52,020 --> 00:31:53,680
So that's an impossible thing, right?

525
00:31:54,100 --> 00:31:57,720
Unless you had a lot of enough
computing power to predict the future,

526
00:31:57,900 --> 00:32:00,640
not far ahead, but a few
milliseconds ahead on both sides,

527
00:32:00,940 --> 00:32:05,040
in which case you could enable
this sort of technology. So yes,

528
00:32:05,520 --> 00:32:09,040
embedded processors, embedded systems,
cyber, cyber, physical systems,

529
00:32:09,340 --> 00:32:11,680
so to speak. I think, uh,

530
00:32:11,680 --> 00:32:16,520
what excites me in the future sounds
fascinating. Yeah. Another point of,

531
00:32:17,020 --> 00:32:21,860
uh, concern, right? Is coming back to

532
00:32:23,380 --> 00:32:27,960
the commoditization of, uh,
computer processors, uh,

533
00:32:28,790 --> 00:32:33,560
because it's a, it is becoming
a resourceful commodity,

534
00:32:35,070 --> 00:32:39,110
then it, this issue is actually
becoming geopolitical, right?

535
00:32:40,400 --> 00:32:45,100
Um, you can see the U us, uh,

536
00:32:45,100 --> 00:32:49,820
passing the CHIPS act recently,
which is supposed to bring a lot of,

537
00:32:49,900 --> 00:32:53,100
a lot more manufacturing back to,
to the United States, for example,

538
00:32:53,650 --> 00:32:56,660
semiconductor manufacturing.
That is. And, uh,

539
00:32:56,660 --> 00:33:01,550
if you look at the supply
chains of how these chips

540
00:33:01,550 --> 00:33:05,430
are made, they are very, uh,
let's say delicate, right?

541
00:33:05,730 --> 00:33:09,830
It needs a lot of cooperation from
multiple companies in multiple

542
00:33:10,620 --> 00:33:14,350
different countries. And
there is a lot of, uh,

543
00:33:14,350 --> 00:33:17,750
different equipment
that are, uh, you know,

544
00:33:18,950 --> 00:33:21,990
provided by companies with a monopoly.

545
00:33:23,330 --> 00:33:28,230
And so, so it's now that, uh,

546
00:33:28,260 --> 00:33:30,670
it's becoming a more scarce resource, I,

547
00:33:30,790 --> 00:33:34,510
I worry a little bit about what are
the geopolitical consequences of that?

548
00:33:34,650 --> 00:33:36,310
Are we gonna see big, uh,

549
00:33:37,090 --> 00:33:41,550
big crisis as it happened during
covid with, um, the microchip, uh,

550
00:33:41,790 --> 00:33:45,190
shortage in, uh, in car
manufacturing, for example?

551
00:33:45,330 --> 00:33:47,910
So you had all the other
pieces of your car ready to go,

552
00:33:48,330 --> 00:33:50,910
but then because of a few chips
here and there, which are unique,

553
00:33:51,010 --> 00:33:53,030
and it has to come from
a particular vendor,

554
00:33:53,450 --> 00:33:56,190
you had to delay your whole
production, for example, right?

555
00:33:56,770 --> 00:34:00,550
So is there more of that in the horizon?
That's something that I, I think about.

556
00:34:01,290 --> 00:34:03,630
And also the, the other
thing is investments.

557
00:34:04,350 --> 00:34:08,870
I don't see adequate and
sufficient investments yet, uh, on

558
00:34:10,500 --> 00:34:12,980
non-standard, uh, or, um,

559
00:34:13,490 --> 00:34:17,380
semiconductor industry for
computing, right? I see. Of course,

560
00:34:17,380 --> 00:34:22,100
there's a lot of investments for the
semiconductor, uh, manufacturing for,

561
00:34:22,320 --> 00:34:26,300
um, electronics. But for example, in
photonics, it's a, it's a lot less,

562
00:34:26,320 --> 00:34:31,180
of course. But I do believe that
it's a technology that is going to,

563
00:34:32,120 --> 00:34:36,740
uh, be part of much of this feature
that I mentioned. Um, this, uh,

564
00:34:36,940 --> 00:34:39,220
advanced processors, this cyber
physical systems will need,

565
00:34:39,450 --> 00:34:44,380
will need optical elements in it. That's
something that will happen inevitably.

566
00:34:44,800 --> 00:34:47,460
So I would like to see a little bit
more investment on that, on that front.

567
00:34:53,120 --> 00:34:54,140
Here's Anson Ho,

568
00:34:54,460 --> 00:34:58,540
a physics graduate who now works at the
interface of AI and other disciplines

569
00:34:59,070 --> 00:35:00,460
among other deep questions.

570
00:35:00,890 --> 00:35:05,180
He's interested in the efficiency of
computer processing and its implications

571
00:35:05,640 --> 00:35:07,580
for climate change mitigation. I.

572
00:35:07,580 --> 00:35:10,900
Am an AI forecasting researcher at op. Uh,

573
00:35:10,900 --> 00:35:14,780
essentially that means that I try to use
tools from a bunch of different fields

574
00:35:14,970 --> 00:35:17,820
like, um, economics and
physics and deep learning,

575
00:35:17,840 --> 00:35:21,340
and try to help us to understand
what's going on in, um,

576
00:35:21,340 --> 00:35:23,940
with future AI developments, how
that's going to impact the world.

577
00:35:24,280 --> 00:35:26,700
And we use this information
to try to communicate to say,

578
00:35:26,700 --> 00:35:30,940
policymakers and other AI researchers
who want to ensure that the impacts of AI

579
00:35:30,940 --> 00:35:32,180
are going to be beneficial. What's.

580
00:35:32,180 --> 00:35:33,260
It looking like though, to you guys?

581
00:35:33,340 --> 00:35:37,900
I would say I'm somewhere on the side
where I'm quite concerned about like a lot

582
00:35:37,900 --> 00:35:41,420
of the possible, um,
misuse aspects of ai. Um,

583
00:35:41,420 --> 00:35:44,100
this is originally one of the ways in
which I got into it in the first place.

584
00:35:44,480 --> 00:35:46,900
Um, so I'm actually quite sympathetic
to like, view is that, uh,

585
00:35:46,900 --> 00:35:51,300
we really need to focus on, uh,
possible tail risks, um, of ai. Um,

586
00:35:51,700 --> 00:35:54,740
although I don't think I go all the
way to thinking that, oh, this is, um,

587
00:35:54,980 --> 00:35:57,900
a situation where we're absolutely doomed
and there's nothing we can do about

588
00:35:57,900 --> 00:36:02,020
it. We're going to be all be dead by
the end of the decade. Um, so I'd say,

589
00:36:03,230 --> 00:36:03,590
um,

590
00:36:03,590 --> 00:36:06,410
the most important thing for us right
now is to try to gain more clarity about

591
00:36:06,410 --> 00:36:08,730
these questions, and that's really
what I'm hoping to do. Fair enough.

592
00:36:08,760 --> 00:36:11,730
Well, how does MO'S law fit
into ai? Or is it the other way.

593
00:36:11,730 --> 00:36:15,250
Around? I think the framing
here is to take one of, um,

594
00:36:15,250 --> 00:36:19,930
thinking about AI developments in terms
of a key input. So we could first ask,

595
00:36:19,930 --> 00:36:21,890
what are the main things
that drive AI progress?

596
00:36:22,190 --> 00:36:23,410
And there are three main ingredients.

597
00:36:23,410 --> 00:36:27,810
People often talk about a thing called
the AI triad, um, which are, uh,

598
00:36:27,840 --> 00:36:31,050
just a thing like a framework with
three main components, compute,

599
00:36:31,140 --> 00:36:34,650
which is a total amount of computation.
It could be, um, hardware, uh,

600
00:36:34,650 --> 00:36:38,330
it could be like the amount of spending
on hardware. And there's also, um, data.

601
00:36:38,750 --> 00:36:41,810
People often talk about big, big data,
for example, there's like so much, um,

602
00:36:41,810 --> 00:36:46,130
text on the Internets we can use to
feed into deep neural networks. And, uh,

603
00:36:46,130 --> 00:36:49,210
the third aspect is algorithms. Like what
kinds of neural networks are we using?

604
00:36:49,990 --> 00:36:50,730
Um,

605
00:36:50,730 --> 00:36:55,290
I think right now it seems to be the
case that the most important thing is

606
00:36:55,530 --> 00:36:58,530
probably compute. Arguably, uh,
algorithms are also quite important,

607
00:36:58,630 --> 00:37:01,530
but maybe I'll get to that later.
But the most important thing, uh,

608
00:37:01,530 --> 00:37:04,810
seems to be compute. If you look
at the trends historically, um,

609
00:37:04,810 --> 00:37:08,970
we find that the between,
um, say the start of, um,

610
00:37:08,970 --> 00:37:10,650
machine learning or a start of, um,

611
00:37:10,770 --> 00:37:15,010
artificial intelligence around 1956
with like the Dartmouth conference, um,

612
00:37:15,960 --> 00:37:18,770
over until the start of
deep learning around 2010,

613
00:37:18,910 --> 00:37:22,290
and like 2012 was like the development
of Alex Snaps, which is really, really,

614
00:37:22,290 --> 00:37:23,930
really a famous paper. Um,

615
00:37:24,390 --> 00:37:29,050
the trend in the amount of computation
used to train these like top AI systems

616
00:37:29,630 --> 00:37:32,930
has followed something similar to like
a rate of progress as Moore's law,

617
00:37:32,930 --> 00:37:34,410
which is around a
doubling every two years.

618
00:37:35,710 --> 00:37:37,410
And this seems to
suggest to us that, okay,

619
00:37:38,210 --> 00:37:41,170
historically we've seen a lot
of progress, um, until 2012.

620
00:37:41,170 --> 00:37:43,570
That is in deep learning, or sorry,

621
00:37:43,570 --> 00:37:47,290
not deep learning in machine learning
due to improvements in processors,

622
00:37:47,370 --> 00:37:49,170
improvements in hardware, um,

623
00:37:49,170 --> 00:37:52,690
just these kinds of things like slowly
pushing forward. I say slowly, but two,

624
00:37:52,760 --> 00:37:55,410
like doubling every two years
is still pretty fast. Um,

625
00:37:55,470 --> 00:37:58,730
but the reason why I say slowly is that
after deep learning things really took

626
00:37:58,730 --> 00:38:01,690
off After the, uh, 2012, um,

627
00:38:01,690 --> 00:38:05,010
people realized that one of the
ways in which you get much, much,

628
00:38:05,010 --> 00:38:07,850
much better performance is really
to just scale things up by a lot.

629
00:38:08,590 --> 00:38:13,370
We use a lot more GPUs. We,
um, invest a lot more. We also,

630
00:38:13,670 --> 00:38:15,450
um, really just, uh,

631
00:38:15,450 --> 00:38:20,330
have seen like a massive growth in the
competition since then. So for example,

632
00:38:20,640 --> 00:38:25,130
between 2012 and, uh,
around 2018, we've seen, um,

633
00:38:25,410 --> 00:38:28,250
a doubling time of more like six
months in the training compute.

634
00:38:28,710 --> 00:38:31,850
And since then we've seen a little
bit of a slowing around, um,

635
00:38:32,470 --> 00:38:36,380
I'd say around like 2017 or so, uh,
to something more like 11 months.

636
00:38:36,840 --> 00:38:39,220
But these trends just
indicate to us that, uh,

637
00:38:39,220 --> 00:38:43,180
there was something happening initially
in the first era where we're going from,

638
00:38:43,400 --> 00:38:45,060
um, 1956 to thousand 12,

639
00:38:45,200 --> 00:38:48,300
mostly happening through like
hardware improvements afterwards.

640
00:38:48,680 --> 00:38:52,380
The big change was due to a massive
increase in investments. And, uh,

641
00:38:52,380 --> 00:38:54,820
we're seeing much more competition
being used to train systems.

642
00:38:55,040 --> 00:38:56,380
And after that we're kind of hit,

643
00:38:56,390 --> 00:38:59,580
we're going to start to see some like
bottlenecks where it's just hard to

644
00:38:59,940 --> 00:39:03,220
continue to increase investments, but
it's not, it's not really clear like, um,

645
00:39:03,220 --> 00:39:05,860
how long trends are going to last and
like what kinds of things are going to

646
00:39:05,860 --> 00:39:06,640
come into place.

647
00:39:06,640 --> 00:39:08,980
And hopefully we can try and get a better
understanding of that by looking into

648
00:39:08,980 --> 00:39:12,460
like the bottlenecks of different,
um, that could be in place.

649
00:39:12,460 --> 00:39:15,540
It could be economic bottlenecks, it
could be physical bottlenecks in hardware.

650
00:39:15,760 --> 00:39:19,700
So those physical bottlenecks are sort
of, I mean, they're ins surpassing,

651
00:39:19,700 --> 00:39:22,660
aren't they? If you can't
fit more things on a chip,

652
00:39:22,660 --> 00:39:24,420
then you can't fit more things on a chip.

653
00:39:25,330 --> 00:39:27,890
I guess there are a number of ways around
this. It is true. That is like, um,

654
00:39:28,120 --> 00:39:32,930
well, with Moore's law, um, I guess
we could have problems with say,

655
00:39:33,040 --> 00:39:35,010
when transitions become too small, uh,

656
00:39:35,010 --> 00:39:38,930
they don't operate as well because of
things like quantum tunneling. Um, also,

657
00:39:39,670 --> 00:39:43,220
um, yeah, there's just like
problems with, um, noise,

658
00:39:43,220 --> 00:39:46,380
like thermal noise that makes
them very unreliable, I would say.

659
00:39:46,380 --> 00:39:50,260
Like the possible ways
around them are to, uh, one,

660
00:39:50,460 --> 00:39:52,860
continue with the current paradigm and
come up with better solutions that can

661
00:39:52,860 --> 00:39:54,620
fit into the current paradigm. Um,

662
00:39:54,620 --> 00:39:58,700
so you could use different types of
transistors. Um, for most of, um,

663
00:39:59,080 --> 00:40:03,060
I'd say the developments
from 1980 or so until

664
00:40:03,320 --> 00:40:06,900
2010, the transistors that
were being used were MOSFETs.

665
00:40:06,900 --> 00:40:10,300
So these are metal oxide semiconductor
field effect transistors. Uh,

666
00:40:10,300 --> 00:40:13,460
more recently we've seen like a new type
of transistors that are becoming more

667
00:40:13,460 --> 00:40:17,340
and more popular called Fin FETs. And
these help to improve upon like these, um,

668
00:40:17,540 --> 00:40:22,540
previous, um, uh, bottlenecks in, uh,
microprocessors that occur due to, um,

669
00:40:22,800 --> 00:40:25,140
static power dissipation, which can occur.

670
00:40:25,560 --> 00:40:29,380
So this is power dissipation
that is present even when
we're not doing switching

671
00:40:29,380 --> 00:40:32,180
operations, like doing
logic operations. Um,

672
00:40:32,840 --> 00:40:34,940
so this is one of the
kinds of things we can do.

673
00:40:34,940 --> 00:40:38,100
We can try to improve the existing
paradigm, but even still, um,

674
00:40:38,100 --> 00:40:39,220
there's so far you can push this.

675
00:40:39,220 --> 00:40:43,660
Maybe you're saying the other other way
around this is to try and come up with

676
00:40:43,660 --> 00:40:47,460
alternative computing paradigms. So right
now most of the stuff we're doing is,

677
00:40:47,760 --> 00:40:52,020
um, using, um, irreversible operations.

678
00:40:52,400 --> 00:40:55,340
And we're using things that
are based on, um, you know,

679
00:40:55,340 --> 00:40:56,700
semiconductors like silicon.

680
00:40:57,240 --> 00:41:00,420
And we're not really thinking so
much about things like, um, uh,

681
00:41:00,820 --> 00:41:02,460
obstacle competing, which could
also be based on like silicon,

682
00:41:02,560 --> 00:41:06,100
but we're using like a different, um,
type of computation. We're using lights.

683
00:41:06,760 --> 00:41:08,900
Um, and this has like a,

684
00:41:09,170 --> 00:41:12,140
like a range of benefits that could
really drastically increase the energy

685
00:41:12,140 --> 00:41:15,380
efficiency of the processors. It
could also make things run faster.

686
00:41:16,000 --> 00:41:18,860
And it's especially good if we're doing
things like matrix multiply operations

687
00:41:18,860 --> 00:41:21,820
that are, you know, really
quite fixed and, um,

688
00:41:21,880 --> 00:41:24,420
we can just repeat the same operation
lots and lots of times. And in fact,

689
00:41:24,420 --> 00:41:26,060
this is what we do for deep learning,

690
00:41:26,230 --> 00:41:28,780
which just means that perhaps
this is like a really,

691
00:41:28,780 --> 00:41:30,940
really good paradigm that people
might want suspicion into,

692
00:41:30,940 --> 00:41:32,780
into the future if we want to
get really, really big gains.

693
00:41:33,800 --> 00:41:37,980
But so does that mean that
e essentially Moore's law is

694
00:41:38,740 --> 00:41:39,980
possibly gonna take off again?

695
00:41:40,170 --> 00:41:42,500
Yeah, I guess there's some
like differences here. Uh,

696
00:41:42,500 --> 00:41:46,100
because what exactly do we mean by
Moore's law? Um, I think Moore's Law, um,

697
00:41:46,100 --> 00:41:48,820
people often say like frame in
terms of like transistors. Um,

698
00:41:48,900 --> 00:41:52,740
I think the original paper frames it in
terms of this term called complexity,

699
00:41:53,200 --> 00:41:54,820
uh, which could refer to, um,

700
00:41:54,820 --> 00:41:58,580
components on the microprocessors that
aren't necessarily transistors. Um,

701
00:41:59,590 --> 00:42:03,610
I'd say from my understanding, this
is not really my field, but uh,

702
00:42:04,620 --> 00:42:07,800
things like obstacle transistors
have really quite struggled. Um,

703
00:42:08,860 --> 00:42:12,000
so maybe like the way in which we're
doing the computations could be quite

704
00:42:12,000 --> 00:42:15,240
different. Remember we were
thinking more like a level of, um,

705
00:42:16,220 --> 00:42:19,030
different kinds of circuits. So I
think they're might be called like, um,

706
00:42:19,230 --> 00:42:20,830
multiply accumulates. Um,

707
00:42:20,830 --> 00:42:24,870
we're just using different kinds of
obstacle components, um, like mock zender,

708
00:42:24,950 --> 00:42:28,150
interferometers, uh, I won't get into
that, but, um, essentially we're allowing,

709
00:42:28,250 --> 00:42:31,510
uh, we're using different kinds of
components that aren't necessarily, uh,

710
00:42:31,510 --> 00:42:33,270
transistors. And so maybe there
would be just something different.

711
00:42:33,290 --> 00:42:36,710
I'm not sure what exactly the right analog
is for Moore's Law in that scenario.

712
00:42:37,190 --> 00:42:39,590
I do think that if you're looking
at like a more general thing,

713
00:42:39,680 --> 00:42:41,950
we're not just saying like,
transistors, we could still see,

714
00:42:42,010 --> 00:42:45,950
and I do expect to see something like an
improvement in scaling, uh, over time.

715
00:42:46,570 --> 00:42:49,190
And in fact, I think
probably quite a lot of, um,

716
00:42:49,190 --> 00:42:52,910
developments historically in a whole
bunch of different technologies has been

717
00:42:52,910 --> 00:42:57,150
just due to us like, you know, cranking,
like, um, using more resources,

718
00:42:57,330 --> 00:43:00,830
slowly pushing the frontiers and seeing
how far we can push things until we're

719
00:43:00,830 --> 00:43:03,270
able to achieve something that we want
to get to. And in this case it could be,

720
00:43:03,450 --> 00:43:04,390
um, building really,

721
00:43:04,390 --> 00:43:07,670
really powerful artificial intelligence
that we'll be able to automate a lot of

722
00:43:07,670 --> 00:43:11,750
jobs. I don't know. Uh, and
essentially the question then is like,

723
00:43:11,930 --> 00:43:14,670
how much will we need, um,
to get to these levels?

724
00:43:15,020 --> 00:43:15,590
Yeah,

725
00:43:15,590 --> 00:43:20,110
I wonder whether it's sort of six or
eight months out from the deadline and

726
00:43:20,230 --> 00:43:23,350
everyone's suddenly like, oh, quick,
we better work on this so that we,

727
00:43:23,610 --> 00:43:26,670
we actually hit the deadline
and keep Moore's law going.

728
00:43:27,020 --> 00:43:29,710
I've certainly heard,
uh, stories of how, um,

729
00:43:30,340 --> 00:43:34,430
Moore's law is a bit of a
self-fulfilling prophecy. Um, and uh,

730
00:43:35,050 --> 00:43:38,790
I'd say I'd suspect that there is
like such a dynamic in place. Um,

731
00:43:38,790 --> 00:43:42,030
there are pretty strong like, um, um,
as far as I'm aware, there already,

732
00:43:42,030 --> 00:43:46,670
already like reports by the, um, R R
D S, um, that they publish every year,

733
00:43:46,700 --> 00:43:48,270
just saying like, what's,
what's the current states?

734
00:43:48,450 --> 00:43:51,670
How can we try to continue
like, making progress? And, uh,

735
00:43:51,820 --> 00:43:55,430
just because it's like such a big part
of like the, um, the conscious like of,

736
00:43:55,430 --> 00:43:57,270
uh, people who are
working in this area, um,

737
00:43:57,350 --> 00:44:00,030
I think it makes quite a bit of sense
that there would be such a dynamic for

738
00:44:00,030 --> 00:44:03,230
people are just trying to reinforce
like the existing trend. Um,

739
00:44:03,430 --> 00:44:07,470
I guess there's a bit more to say to
this actually, um, which is kind of, um,

740
00:44:08,010 --> 00:44:10,390
that's why what people want
to continue to, you know,

741
00:44:10,390 --> 00:44:11,390
scale in this particular way.

742
00:44:12,010 --> 00:44:14,670
Why would you want to follow that? Why
would you want to make that happen?

743
00:44:14,930 --> 00:44:18,390
I'm going to draw like a little bit
from, um, a paper by, um, cyber hooker.

744
00:44:18,710 --> 00:44:20,350
I think that's called
the hardware lottery.

745
00:44:20,890 --> 00:44:24,990
And essentially this points
out that, um, improvements. Um,

746
00:44:25,380 --> 00:44:30,090
well imagine that you are a researcher and

747
00:44:30,090 --> 00:44:34,700
you're trying to come up with the next
best hardware and, uh, you want to, um,

748
00:44:34,880 --> 00:44:39,220
you know, earner profits perhaps.
So what are you going to do? Well,

749
00:44:39,520 --> 00:44:42,820
one option is to try and come
up with something totally novel.

750
00:44:43,160 --> 00:44:45,140
You want to be like the next, uh, genius.

751
00:44:45,160 --> 00:44:49,100
You want to come up with a totally new
design and just beat everybody else.

752
00:44:49,730 --> 00:44:54,020
Well, unfortunately, it's quite hard
to do this. Um, uh, you, first of all,

753
00:44:54,020 --> 00:44:55,540
you need to be able to come up
with the idea in the first place,

754
00:44:55,640 --> 00:44:57,060
but even if you come up with the idea,

755
00:44:57,210 --> 00:44:59,500
it's not necessarily easy for
you to be able to implement it.

756
00:45:00,040 --> 00:45:03,460
You're trying to beat a, a
system where there's, um,

757
00:45:03,680 --> 00:45:07,060
or like a supply chain where
people have been, uh, optimizing,

758
00:45:07,060 --> 00:45:10,060
which people have been optimizing
for quite a long time. Um, there is,

759
00:45:10,060 --> 00:45:12,700
there's lots and lots of money being
poured into improving certain kinds of

760
00:45:12,700 --> 00:45:14,820
micro microprocessors with
certain architectures.

761
00:45:15,360 --> 00:45:18,780
And trying to switch out of that is
actually quite difficult. So in fact,

762
00:45:18,780 --> 00:45:20,980
there's like a strong incentive
for you to say, actually,

763
00:45:20,980 --> 00:45:23,740
why don't we just continue using this
particular recipe that we know already

764
00:45:23,740 --> 00:45:26,980
works and we can just, you know,
continue transistor scaling,

765
00:45:26,980 --> 00:45:30,780
meaning we make all the
components smaller. We have
perhaps like come up with,

766
00:45:31,080 --> 00:45:35,620
um, new techniques for, um, doing
lithography at smaller scales. And, um,

767
00:45:35,920 --> 00:45:36,260
you know,

768
00:45:36,260 --> 00:45:39,460
if it's like a try to test a recipe and
we know it's probably going to continue

769
00:45:39,460 --> 00:45:43,860
for some time, why don't we continue
doing it? And I'd say that's what's, um,

770
00:45:43,860 --> 00:45:46,820
happening or that's probably
good dynamic that's, uh,

771
00:45:46,820 --> 00:45:51,380
helped sustain Moore's law historically
and why we haven't like switched as

772
00:45:51,380 --> 00:45:54,420
quickly to some of these
alternative paradigms that
I mentioned with optical and

773
00:45:54,420 --> 00:45:55,300
reversible computing.

774
00:45:55,690 --> 00:45:58,900
With quantum computing, is that
going to take over condition, uh,

775
00:45:58,900 --> 00:45:59,580
traditional computing?

776
00:45:59,580 --> 00:46:04,420
Is it going to replace it in a way that
Moore's law is just a thing of the past

777
00:46:05,040 --> 00:46:09,140
or a quantum computer's always
gonna be slightly outside of the.

778
00:46:09,140 --> 00:46:12,340
Norm? Uh, quantum computing
thus far has been, um,

779
00:46:12,560 --> 00:46:13,780
not had that much of an influence.

780
00:46:13,840 --> 00:46:18,020
And it seems difficult because of maybe
three Bolton X one is just that, um,

781
00:46:18,400 --> 00:46:21,340
it seems hard to come up with the right
algorithms that we can apply to the

782
00:46:21,340 --> 00:46:24,820
right situations and we need to find
these right situations in the first place.

783
00:46:24,820 --> 00:46:28,140
And it's not necessarily easy,
it seems like thus far that, um,

784
00:46:28,140 --> 00:46:32,260
these tend to work in, uh, certain
specific domains like cryptography.

785
00:46:32,650 --> 00:46:35,060
Like a second difficulty is that, um,

786
00:46:35,060 --> 00:46:36,460
even if you can identify
what the domain is,

787
00:46:36,460 --> 00:46:39,060
it's not necessarily easy to come up with
the right algorithm that allows you to

788
00:46:39,060 --> 00:46:41,980
get like the final products that you
have. I think for example, in um,

789
00:46:42,060 --> 00:46:42,500
shores algorithm,

790
00:46:42,500 --> 00:46:45,140
you have like this process of like
cancellation of a lot of different things

791
00:46:45,210 --> 00:46:49,420
that, um, is, that gives you like the
right results, uh, at the end. And, uh,

792
00:46:49,420 --> 00:46:51,940
the third aspect to this
is probably just that, um,

793
00:46:51,940 --> 00:46:54,620
you need to also build the hardware and
make sure that the hardware is reliable.

794
00:46:54,620 --> 00:46:57,180
You don't have, um, problems where, um,

795
00:46:57,180 --> 00:47:00,660
like some perturbations from noise
cause like the states to, um,

796
00:47:00,660 --> 00:47:03,180
essentially get messed up and
you're not able to do like the, uh,

797
00:47:03,180 --> 00:47:06,060
quantum computations that you want
to do. So my understanding is just,

798
00:47:06,060 --> 00:47:07,620
it's quite difficult for us at present.

799
00:47:07,920 --> 00:47:09,580
So can you tell me a bit more about the,

800
00:47:09,640 --> 00:47:14,460
the sort of processing side of things
and how, how that's looking? We.

801
00:47:14,460 --> 00:47:18,860
Want to understand, uh, what the
bottlenecks are to, um, using,

802
00:47:18,960 --> 00:47:22,300
to getting really, really
powerful processors and
being able to use these for,

803
00:47:22,760 --> 00:47:26,740
um, very exp computationally
expensive tasks. So for example, um,

804
00:47:26,880 --> 00:47:30,940
AI training runs, um, in order to
train a deep neural network, uh,

805
00:47:31,120 --> 00:47:35,980
it typically costs like a, you need
like many, many GPUs for like the ta um,

806
00:47:36,040 --> 00:47:38,780
AI systems of today. I
think GPT four, for example,

807
00:47:39,420 --> 00:47:42,020
probably costs on the order
of 10 to depart of 25, uh,

808
00:47:42,140 --> 00:47:45,780
floating point operations over the entire
course of training, which is really,

809
00:47:45,780 --> 00:47:48,820
really high, I guess. Uh, my
computer certainly doesn't use, uh,

810
00:47:49,180 --> 00:47:51,820
I haven't used that much. I
don't, oh, I think I haven't.

811
00:47:52,480 --> 00:47:56,380
How are we going to be able to
continue to push, um, further if, um,

812
00:47:56,380 --> 00:47:58,100
the demands for competition
are just so high?

813
00:47:58,240 --> 00:48:00,500
How far can we continue to push
things within the current paradigm?

814
00:48:00,880 --> 00:48:04,140
And these are important questions I
think for, um, from two dimensions.

815
00:48:04,760 --> 00:48:06,540
One dimension is that, uh,

816
00:48:06,760 --> 00:48:11,620
if the energy costs associated
with computation are
increasing so much that they

817
00:48:11,620 --> 00:48:15,020
could in, you know, increasingly have
like a cause problem, say with, uh,

818
00:48:15,020 --> 00:48:18,660
environmental problems or with in fact
like melting the GPU that you're trying

819
00:48:18,660 --> 00:48:22,260
to use, that would be quite, um,
uh, problematic situation I think.

820
00:48:22,640 --> 00:48:25,380
So we're worried about power
density, we're worried about, um,

821
00:48:25,610 --> 00:48:27,540
even if you could have
really good cooling systems,

822
00:48:27,840 --> 00:48:32,260
what's going to be just
like an unsustainable amount
of, um, computation, um,

823
00:48:32,260 --> 00:48:35,580
that you would, uh, try to do but
would just caused like too much harm.

824
00:48:36,120 --> 00:48:39,900
And the other aspect is that let's
say we want to continue improving,

825
00:48:40,450 --> 00:48:43,980
what does this, uh, upper bound
to, uh, harbor efficiency,

826
00:48:43,980 --> 00:48:47,980
tell us about how quick progress
is going to be in the future. Well,

827
00:48:47,980 --> 00:48:50,140
let's say we want to
determine how many, um,

828
00:48:50,140 --> 00:48:53,260
closing forward operations we can
perform for like a, the largest training,

829
00:48:53,480 --> 00:48:57,060
ai training run using like
existing processors of today. Um,

830
00:48:57,240 --> 00:49:00,860
the reason why we might care about this
is that, um, AI training runs depend,

831
00:49:00,860 --> 00:49:04,420
or AI performance depends on these
like scaling laws. So it's kind of,

832
00:49:04,420 --> 00:49:06,980
if you're, if you know some
statistical physics then um,

833
00:49:06,980 --> 00:49:10,620
you often have heard of
things like, um, scalings and,

834
00:49:10,680 --> 00:49:14,180
and like phase transitions. There's like
a similar thing that's inspired, uh,

835
00:49:14,180 --> 00:49:15,980
by these kinds of like, um, uh,

836
00:49:15,980 --> 00:49:19,060
relationships and statistical physics
that people have found with deep neural

837
00:49:19,060 --> 00:49:22,260
networks. Essentially,
when you increase, um,

838
00:49:22,480 --> 00:49:26,020
things like the total amount of training
data or the size of the model or like

839
00:49:26,020 --> 00:49:27,780
the total amount of training
computation for these models,

840
00:49:28,080 --> 00:49:30,580
we find that the performance
using some metric,

841
00:49:30,580 --> 00:49:35,100
which is typically like on some benchmark
like the loss or like the accuracy for

842
00:49:35,380 --> 00:49:38,060
classification, it tends to, it
kind of scales quite smoothly.

843
00:49:38,440 --> 00:49:41,260
And we can therefore use this to try
to make some kind of forecast into the

844
00:49:41,260 --> 00:49:43,260
future of, um, uh,

845
00:49:43,400 --> 00:49:48,060
how powerful ass is going to be based
on the forecast of how much data we'll

846
00:49:48,060 --> 00:49:51,260
have and how much computation
we'll have. If we then say, well,

847
00:49:51,780 --> 00:49:53,820
I might expect that there's
going to be like a really,

848
00:49:53,820 --> 00:49:54,980
really large amount of computation,

849
00:49:54,980 --> 00:49:57,100
say like tentative hour
40 focusing operations,

850
00:49:57,310 --> 00:50:00,180
let's just say that we believe
this for the sake of arguments,

851
00:50:00,920 --> 00:50:04,140
is it going to be feasible
for us to do this? Um,

852
00:50:05,000 --> 00:50:09,950
turns out it's, uh, going to be quite
difficult, uh, given existing, um,

853
00:50:09,950 --> 00:50:14,150
microprocessors. Um, so, uh,
project that, um, some, uh,

854
00:50:14,150 --> 00:50:16,430
me and my colleagues have
been working on recently, uh,

855
00:50:16,570 --> 00:50:20,190
we roughly estimate that we're probably
going to hit some bounds on the order

856
00:50:20,330 --> 00:50:21,190
of, uh,

857
00:50:21,190 --> 00:50:25,390
performing 10 to the power of 16 floating
point operations per juul of energy

858
00:50:25,390 --> 00:50:28,500
dissipated, which is about like
three orders of magnitude. Uh,

859
00:50:28,920 --> 00:50:32,380
I'd say three or four orders of
magnitude higher, uh, than existing, uh,

860
00:50:32,380 --> 00:50:36,900
microprocessors. And this just means
that, well, if we're going at, uh,

861
00:50:36,900 --> 00:50:37,740
current rates of progress,

862
00:50:38,260 --> 00:50:40,940
I believe we're doubling on the order
of like two to three years for energy

863
00:50:40,940 --> 00:50:43,380
efficiency like floating
front operations per juul. Uh,

864
00:50:43,380 --> 00:50:46,660
we're going to hit these
limits like pretty quickly. Um,

865
00:50:46,840 --> 00:50:49,780
or before we hit these limits,
progress is going to have to slow.

866
00:50:51,420 --> 00:50:54,040
And if this is the case,
it just means that, uh,

867
00:50:54,040 --> 00:50:58,520
if we want to do this like tentative
power 40, uh, flop training run, uh,

868
00:50:59,090 --> 00:51:00,240
we're going to, uh,

869
00:51:01,220 --> 00:51:05,040
if you look at the total amount of
jus of energy that are released, um,

870
00:51:05,040 --> 00:51:07,320
really it just means that
we're probably going to, uh,

871
00:51:07,480 --> 00:51:09,880
emits on the order of like the same
amount of energy that's emitted from the

872
00:51:09,880 --> 00:51:12,240
earth every single year, which I
believe is on the order of tentative,

873
00:51:12,240 --> 00:51:14,760
however of 24 Jules. Um,

874
00:51:15,300 --> 00:51:19,960
and that's about four orders of magnitude
higher than, uh, the amount of, uh,

875
00:51:19,980 --> 00:51:24,080
energy that's released through like fossil
fuels every year, say think in 2021.

876
00:51:24,340 --> 00:51:27,640
Wow. Yeah. Uh, so this just means, well,

877
00:51:27,640 --> 00:51:32,080
there are some well restrictions on what
we can do from these energy efficiency

878
00:51:32,080 --> 00:51:33,880
limits, but of course you might ask, well,

879
00:51:34,050 --> 00:51:37,400
maybe we don't even need to get to
these like tentative power 40 like, uh,

880
00:51:37,470 --> 00:51:39,880
flop training runs. These are, that's
just a really, really large number.

881
00:51:40,100 --> 00:51:42,280
And in fact, I tend to agree, uh,

882
00:51:42,280 --> 00:51:45,120
the reason why you might believe that
tentative power 14 might be relevant in

883
00:51:45,120 --> 00:51:47,680
the first place is that
you might think, well,

884
00:51:48,060 --> 00:51:51,520
let me just try to guess how much
computation you'll need to build like a

885
00:51:51,520 --> 00:51:54,440
really, really powerful AI system
by looking at evolutionary history.

886
00:51:54,700 --> 00:51:58,520
If you look at evolutionary history and
you count like all of the floating front

887
00:51:58,520 --> 00:52:01,000
operations and quotation
marks, <laugh>, uh,

888
00:52:01,790 --> 00:52:06,200
they were like done in the neurons of
organisms throughout this period. Um,

889
00:52:06,460 --> 00:52:09,120
we find that it just take, it takes
like a really, really large amount.

890
00:52:09,120 --> 00:52:12,640
It's on the order of like 10 to the 40
something, uh, floating point operations.

891
00:52:12,940 --> 00:52:14,560
And so this just suggests that, um,

892
00:52:14,620 --> 00:52:18,400
if it is the case that we should be
anchoring our estimates of the compute

893
00:52:18,400 --> 00:52:20,920
requirements from evolution, um,

894
00:52:21,230 --> 00:52:24,840
that we're going to have some problems
with heating if we're using existing

895
00:52:24,840 --> 00:52:28,440
microprocessors. And perhaps this could
mean that there's going to be like a,

896
00:52:28,580 --> 00:52:29,320
in the future,

897
00:52:29,320 --> 00:52:32,280
a fairly strong incentive to switch
towards alternative paradigms,

898
00:52:32,280 --> 00:52:35,520
which are orders of magnitude more
efficient. And on the economic side,

899
00:52:35,780 --> 00:52:39,520
and first I'm going to like try to
explain this in the context of, um,

900
00:52:39,520 --> 00:52:42,040
my previous economic paper, which
is quite famous. It's called, um,

901
00:52:42,040 --> 00:52:43,320
our Ideas is Getting Harder to Find.

902
00:52:43,820 --> 00:52:48,440
The idea here is that as you make more
and more progress, um, in like, uh,

903
00:52:48,440 --> 00:52:53,440
doing some kind of r and d in some
domain, um, because most of like the,

904
00:52:53,660 --> 00:52:56,840
uh, initial earlier ideas are
getting fished out, uh, you know,

905
00:52:56,840 --> 00:52:59,200
you're picking all the
lowest hanging fruits, um,

906
00:52:59,200 --> 00:53:00,960
further progress becomes
harder and harder.

907
00:53:01,760 --> 00:53:04,620
You have to go and like climb
higher in the tree, I suppose, uh,

908
00:53:04,620 --> 00:53:08,970
to get like the fruits. So how large
is this effect is the question,

909
00:53:09,200 --> 00:53:13,050
does this mean that like, uh, researchers
are becoming less and less, uh,

910
00:53:13,050 --> 00:53:13,750
productive,

911
00:53:13,750 --> 00:53:17,610
are able to come up with fewer ideas
and make less progress over time? Um,

912
00:53:17,830 --> 00:53:22,010
in fact they find, uh, the paper
finds that, um, this is the case, uh,

913
00:53:22,010 --> 00:53:26,380
for quite a wide range of, uh, fields.
It's true across the aggregate economy,

914
00:53:26,800 --> 00:53:28,660
uh, in some like frontier
economies that they look at.

915
00:53:28,810 --> 00:53:32,820
It's also true within individual
sectors, um, including, um,

916
00:53:32,900 --> 00:53:36,050
I think semiconductor r and d. Uh,

917
00:53:36,050 --> 00:53:38,130
they also look at things like
Moore's Law in their paper.

918
00:53:38,670 --> 00:53:41,630
And why is this relevant? Well,

919
00:53:41,630 --> 00:53:44,630
it tells us that even without thinking
about the energy efficiency limits,

920
00:53:44,970 --> 00:53:47,870
we get these like kinds of like
diminishing returns to, uh,

921
00:53:47,870 --> 00:53:49,550
as we get doing more and more research,

922
00:53:49,550 --> 00:53:51,550
it becomes harder to
make further progress.

923
00:53:52,130 --> 00:53:55,150
Now when we bring in the energy
efficiency limits, it becomes much, much,

924
00:53:55,150 --> 00:53:56,710
much harder as we get
really, really close.

925
00:53:57,170 --> 00:54:00,630
The ideas are just going to dry up like
much quicker if we're assuming that

926
00:54:00,630 --> 00:54:01,990
we're continuing within the same paradigm.

927
00:54:02,610 --> 00:54:06,070
And this means that we're going to have
to massively increase the amounts of

928
00:54:06,230 --> 00:54:09,510
research inputs in order to be able to
make further progress at say the same

929
00:54:09,510 --> 00:54:10,670
rate. It's becomes,

930
00:54:10,940 --> 00:54:15,590
Moore's law in a sense becomes much less
sustainable and it becomes harder to,

931
00:54:15,690 --> 00:54:15,970
um,

932
00:54:15,970 --> 00:54:19,830
do that unless we're able to have a lot
more researchers that are able to come

933
00:54:19,830 --> 00:54:23,230
up with new ideas. And typically,
I think historically, um,

934
00:54:24,370 --> 00:54:28,060
some of the best economic theories
that we have tell us that, um,

935
00:54:28,340 --> 00:54:31,660
a big reason for this is like an
increase in population or the increase in

936
00:54:31,660 --> 00:54:34,980
number of scientists which have
been able to like compensate, uh,

937
00:54:34,980 --> 00:54:38,540
for these difficulties. Although I will
say that this is a little disused. Um,

938
00:54:40,150 --> 00:54:44,200
yeah, so that's the second way,
uh, in which, uh, we have um,

939
00:54:44,550 --> 00:54:48,480
some kind of an influence here. And the
way we could, one way we could try to,

940
00:54:48,860 --> 00:54:53,040
uh, get around these bottlenecks is to
bring artificial intelligence into the

941
00:54:53,040 --> 00:54:56,280
picture although human populations,
well, I'm not sure how, um,

942
00:54:56,990 --> 00:54:58,800
much we're going to be able to, um,

943
00:54:59,400 --> 00:55:03,200
increase like human population at the
same rates as say, like in, uh, the 1960s,

944
00:55:04,060 --> 00:55:08,200
uh, especially when, say like fertility
rates. And, um, some of the, uh,

945
00:55:08,400 --> 00:55:10,600
frontier economies right
now are, are, say, in Spain,

946
00:55:10,640 --> 00:55:14,440
I think it's down to like 1.4 in South
Korea, it's around 1.4 as well. Um,

947
00:55:15,540 --> 00:55:19,200
it just means that, um, we're, it's going
to be harder to get enough humans to,

948
00:55:19,460 --> 00:55:19,710
um,

949
00:55:19,710 --> 00:55:22,880
have like these ideas to do research and
to continue pushing things at the same

950
00:55:22,880 --> 00:55:26,600
rates. But with artificial intelligence
in the picture, maybe AI can help us,

951
00:55:27,060 --> 00:55:27,300
um,

952
00:55:27,300 --> 00:55:30,920
do like research and come up with new
ideas or to augment the productivity of

953
00:55:30,920 --> 00:55:35,480
these researchers such that we can
sustain, um, uh, meets this like increase,

954
00:55:35,780 --> 00:55:36,040
uh,

955
00:55:36,040 --> 00:55:39,040
for some time and push things a little
bit further until we get to the limits.

956
00:55:39,300 --> 00:55:42,600
And then maybe then we will still have
to switch to alternative paradigms. Yeah.

957
00:55:42,950 --> 00:55:45,720
Okay. Can artificial
intelligence come up with ideas?

958
00:55:46,160 --> 00:55:49,280
I tend to think that it can, like,
I don't, um, I would say like,

959
00:55:49,310 --> 00:55:50,840
it's not entirely clear, like, uh,

960
00:55:50,840 --> 00:55:52,800
there are cer there's
certainly some uncertainty. Uh,

961
00:55:52,800 --> 00:55:55,520
and I think we should be thinking
about this using some kind of like a

962
00:55:55,520 --> 00:55:56,520
probability distribution.

963
00:55:56,520 --> 00:55:59,600
Like we have some guesses and perhaps
like in the using some kind of, um,

964
00:55:59,600 --> 00:56:02,440
Bayesian statistics approach,
um, we have some, uh,

965
00:56:02,440 --> 00:56:05,960
initial guess like a prior distribution.
And then based on like our evidence,

966
00:56:06,140 --> 00:56:08,520
uh, that we might have, like
we might see that's, you know,

967
00:56:08,900 --> 00:56:12,520
GPT four was actually surprisingly good
at some cost that I didn't expect it to

968
00:56:12,520 --> 00:56:13,353
be good at.

969
00:56:13,420 --> 00:56:16,720
And then I might update my beliefs
towards the direction of thinking that

970
00:56:17,200 --> 00:56:20,560
actually it could be the case that it's,
uh, pretty intelligent. Like it can,

971
00:56:20,700 --> 00:56:24,380
it has, it's able to do some kinds of
common sense reasoning. Um, for example,

972
00:56:24,780 --> 00:56:29,580
I think, um, uh, <inaudible>, uh,
like, um, one of the famous like, uh,

973
00:56:29,780 --> 00:56:33,460
founders of like deep learning,
uh, made this claim, um, that, uh,

974
00:56:33,760 --> 00:56:36,140
if you ask g PT four that,

975
00:56:36,140 --> 00:56:39,380
or if you ask like a really
powerful language model
that is something like G P T

976
00:56:39,380 --> 00:56:42,140
four, if you put like a book on
the table and you push the table,

977
00:56:42,370 --> 00:56:47,250
will the book move along with the
table? Um, uh, he predicted that, uh,

978
00:56:47,430 --> 00:56:49,930
GVT four wouldn't be able to guess
that the book might move along with the

979
00:56:49,930 --> 00:56:52,810
table. Uh, but I think this is not true.
Actually, if you just asked gvt four,

980
00:56:53,070 --> 00:56:56,610
uh, it actually some has like
this kind of like, um, you know,

981
00:56:56,610 --> 00:56:59,250
some kind of a picture. I don't
know if it's like an actual picture,

982
00:56:59,430 --> 00:57:02,210
but it's able to answer this kind of
question with some degree of accuracy.

983
00:57:02,510 --> 00:57:03,690
So I think these kinds of things,

984
00:57:03,690 --> 00:57:06,730
like if we actually give it a test
and we see that it's able to do, um,

985
00:57:06,730 --> 00:57:07,770
solve these kinds of problems,

986
00:57:08,210 --> 00:57:11,810
I think it will be able to do similar
kinds of things. Um, and in fact,

987
00:57:12,050 --> 00:57:16,650
I think I would guess personally that
if you're able to scale to really,

988
00:57:16,650 --> 00:57:20,050
really large scales with massive
networks like on 10 to the 50 parameters,

989
00:57:20,260 --> 00:57:23,010
which is way larger than
anything we have today, uh,

990
00:57:23,010 --> 00:57:27,290
by many orders of magnitude, um, I
think we could do that. The question is,

991
00:57:27,440 --> 00:57:29,530
well, in practice we'll be able to do it.

992
00:57:29,910 --> 00:57:32,930
And I would guess that there are
probably more efficient ways of, um,

993
00:57:33,130 --> 00:57:37,450
training our artificial intelligence than
what we are currently doing today. Uh,

994
00:57:37,790 --> 00:57:40,170
and you know, I think in
practice, if you think, uh,

995
00:57:40,170 --> 00:57:44,010
from like an algorithmic perspective,
um, even really, really, um,

996
00:57:44,900 --> 00:57:48,260
I guess like not particularly intelligent
algorithms are able to perform really

997
00:57:48,260 --> 00:57:52,340
complicated tasks just by using a large
amount of computation to compensate for

998
00:57:52,340 --> 00:57:55,660
it. So I think there's probably
something similar here, but I, of course,

999
00:57:55,700 --> 00:57:56,740
I can't be certain. Yeah.

1000
00:57:56,880 --> 00:58:01,340
Who would you like to know
about all these issues

1001
00:58:01,610 --> 00:58:02,540
that are forthcoming?

1002
00:58:02,860 --> 00:58:06,420
I guess I'd be most interested
in like, um, people who are, uh,

1003
00:58:06,420 --> 00:58:10,620
maybe were in like a similar position
to me, I guess. Uh, so I only recently,

1004
00:58:10,880 --> 00:58:15,140
uh, got into, um, AI forecasting and
thinking about the future of ai. Uh,

1005
00:58:15,180 --> 00:58:19,380
I actually finished my undergrad at,
um, university of St. Andrews, um,

1006
00:58:19,450 --> 00:58:23,860
back in 2021. Um, and I was just
wondering what am, oh my God,

1007
00:58:23,860 --> 00:58:24,780
what am I gonna do with my life?

1008
00:58:25,200 --> 00:58:28,020
But I also wanted to really
just like figure out what, um,

1009
00:58:28,020 --> 00:58:32,020
things I could do that were impactful
and were like big questions to me, uh,

1010
00:58:32,220 --> 00:58:35,940
I think could have like an influence on
like policy, like it affects like, um,

1011
00:58:35,940 --> 00:58:39,820
the rates of, um, progress in from
like an economic perspective. And, um,

1012
00:58:39,820 --> 00:58:42,460
it has, um, influences for, um,

1013
00:58:42,480 --> 00:58:46,180
people who want to decide like how to
govern AI such that if the progress is too

1014
00:58:46,180 --> 00:58:47,660
fast, like there's going to be problems.

1015
00:58:47,660 --> 00:58:50,180
So we want to make sure that
we can adapt to these changes.

1016
00:58:50,570 --> 00:58:53,740
Like these are questions that help inform
us about that. And I really feel like,

1017
00:58:53,960 --> 00:58:58,020
um, that's, you know, there are some
ways in which we can try to use like, um,

1018
00:58:58,020 --> 00:59:01,380
things like the skill sets of
physicists and solve really,

1019
00:59:01,380 --> 00:59:02,900
really important problems. I mean,

1020
00:59:02,900 --> 00:59:05,820
I think it's a bit presumptuous of me to
say like, I'm doing, like solving, uh,

1021
00:59:05,820 --> 00:59:08,900
important problems. Like I, I try to
do, so I don't know if I actually am,

1022
00:59:08,960 --> 00:59:11,500
but I think more people should be thinking
along the lines that they actually

1023
00:59:11,500 --> 00:59:15,740
can do it. Um, I think a lot of my
friends at, uh, uni and myself included,

1024
00:59:15,860 --> 00:59:19,300
I would say actually with, um, that
we were initially thinking that, well,

1025
00:59:19,300 --> 00:59:22,740
what can we even do? It seems really
hard. The only thing we can do is, um,

1026
00:59:22,740 --> 00:59:25,860
to maybe think a little about
about climate change, but, um,

1027
00:59:25,860 --> 00:59:28,660
and that's important <laugh>. Uh, but, uh,

1028
00:59:28,680 --> 00:59:32,300
are there other things that we can do?
There's so many problems out there that,

1029
00:59:32,440 --> 00:59:34,580
uh, could benefit so much
from people with, um,

1030
00:59:34,600 --> 00:59:36,500
who are mathematically minded or, uh,

1031
00:59:36,500 --> 00:59:38,060
have some kinds of
intuitions from physics.

1032
00:59:38,310 --> 00:59:42,020
There are so many problems out there.
Why don't we try to, um, do this more,

1033
00:59:42,300 --> 00:59:43,780
I guess this would be the, um,

1034
00:59:43,780 --> 00:59:46,980
like people who are interested in trying
to figure out how to impact the world

1035
00:59:46,980 --> 00:59:49,580
in a positive way and to really use, um,

1036
00:59:49,580 --> 00:59:52,700
maybe physics or maybe to
switch to another thing, to,

1037
00:59:52,880 --> 00:59:54,340
I'd like to talk to people who are, uh,

1038
00:59:54,560 --> 00:59:58,220
to really communicate with people who
are, um, interested in this kind of thing.

1039
01:00:03,950 --> 01:00:06,740
Louis Barson has a message
for you, the listeners.

1040
01:00:07,060 --> 01:00:09,420
I mean, I think, you know, if
they're physicists listening, uh,

1041
01:00:09,420 --> 01:00:12,300
and they don't know about semiconductor
industry, please do, you know,

1042
01:00:12,330 --> 01:00:13,820
read up on it. I think it's a,

1043
01:00:13,900 --> 01:00:18,140
a fantastic example of where physics
is right at the heart of, uh, you know,

1044
01:00:18,420 --> 01:00:22,340
economic and societal transformation.
Uh, and it's a fantastic story, um,

1045
01:00:22,650 --> 01:00:24,660
even if it isn't the area
you wanna go and work in, uh,

1046
01:00:24,820 --> 01:00:27,860
although of course it's a great area to
work as well. Uh, so please, please do,

1047
01:00:27,860 --> 01:00:31,620
uh, read up and if you haven't, uh,
already, uh, heard the story, and, uh,

1048
01:00:31,760 --> 01:00:36,260
if you do already know about it, um,
then, uh, please do participate in, uh,

1049
01:00:36,260 --> 01:00:39,780
the, the IOP's, uh, impact projects
because, uh, we really want your, uh,

1050
01:00:39,850 --> 01:00:43,380
your voices and your views to be,
be reflected in, uh, you know,

1051
01:00:43,380 --> 01:00:45,340
the areas where the i p is trying
to make a difference on the,

1052
01:00:45,340 --> 01:00:46,173
on the national stage.

1053
01:00:46,680 --> 01:00:48,500
If you'd like to know
more about Moores Law,

1054
01:00:48,720 --> 01:00:52,940
you can find an article on physics
world.com by James Mackenzie,

1055
01:00:53,200 --> 01:00:56,780
and we'll be back next month when we'll
be looking into the International year

1056
01:00:57,040 --> 01:00:59,780
of basic Sciences for
sustainable development.

1057
01:01:00,280 --> 01:01:01,980
And thank you very much for listening.

1058
01:01:06,530 --> 01:01:07,270
Physics.

1059
01:01:07,270 --> 01:01:07,620
World.