Artificial intelligence predicts future directions in quantum science

1
00:00:08,080 --> 00:00:11,039
Hello, and welcome to the Physics World weekly

2
00:00:11,039 --> 00:00:11,539
podcast.

3
00:00:11,919 --> 00:00:13,219
I'm Hamish Johnston.

4
00:00:13,855 --> 00:00:14,674
In this episode,

5
00:00:14,974 --> 00:00:16,675
I'm joined by two physicists

6
00:00:17,135 --> 00:00:18,114
who are exploring

7
00:00:18,494 --> 00:00:20,195
how artificial intelligence

8
00:00:21,054 --> 00:00:23,634
can be used to predict future directions

9
00:00:24,015 --> 00:00:26,035
in quantum science and technology.

10
00:00:27,029 --> 00:00:27,769
They are

11
00:00:28,070 --> 00:00:30,649
Mario Kran, who heads the artificial

12
00:00:31,189 --> 00:00:33,210
scientist lab at Germany's

13
00:00:33,670 --> 00:00:37,049
Max Planck Institute for the Science of Light,

14
00:00:37,429 --> 00:00:38,329
and Felix

15
00:00:38,804 --> 00:00:39,304
Frohnert,

16
00:00:39,765 --> 00:00:41,384
who is doing a PhD

17
00:00:41,844 --> 00:00:42,824
on the intersection

18
00:00:43,204 --> 00:00:46,024
of quantum physics and machine learning

19
00:00:46,405 --> 00:00:47,945
at Leiden University

20
00:00:48,405 --> 00:00:49,304
in The Netherlands.

21
00:00:50,164 --> 00:00:52,104
Earlier this year, Felix,

22
00:00:52,460 --> 00:00:54,000
Mario, and colleagues

23
00:00:54,460 --> 00:00:56,719
published a paper called discovering

24
00:00:57,659 --> 00:01:01,039
emergent connections in quantum physics research

25
00:01:01,500 --> 00:01:04,079
via dynamic word embeddings.

26
00:01:04,984 --> 00:01:07,385
And they join me today to chat about

27
00:01:07,385 --> 00:01:08,125
that work

28
00:01:08,424 --> 00:01:12,125
in which they analyzed more than 66,000

29
00:01:12,505 --> 00:01:13,005
abstracts

30
00:01:13,385 --> 00:01:14,924
from the quantum literature

31
00:01:15,305 --> 00:01:16,924
to see if they could predict

32
00:01:17,305 --> 00:01:19,325
future trends in the field.

33
00:01:27,450 --> 00:01:29,870
Hi, Mario and Felix. Welcome to the podcast.

34
00:01:30,729 --> 00:01:33,530
Hi, Mitch. Very nice, to see you. Glad

35
00:01:33,530 --> 00:01:34,349
to be here.

36
00:01:34,834 --> 00:01:35,655
So, Mario,

37
00:01:36,274 --> 00:01:38,274
can we start off by sort of delving

38
00:01:38,274 --> 00:01:40,055
into the history of this?

39
00:01:40,914 --> 00:01:43,155
Apparently, in the past, people have tried to

40
00:01:43,155 --> 00:01:43,655
use,

41
00:01:44,515 --> 00:01:46,935
data from the scientific literature

42
00:01:47,369 --> 00:01:49,149
to predict the evolution

43
00:01:49,930 --> 00:01:53,049
of a research field. Can you talk a

44
00:01:53,049 --> 00:01:55,930
bit about some of that early work and,

45
00:01:56,329 --> 00:01:58,030
and how successful it was?

46
00:01:58,969 --> 00:01:59,469
Yes.

47
00:02:00,444 --> 00:02:02,284
There have been a lot of, work in

48
00:02:02,284 --> 00:02:04,865
the past where people try to use this

49
00:02:04,924 --> 00:02:07,905
enormous amount of data and see how

50
00:02:08,525 --> 00:02:11,985
researchers work and how researchers could work. So

51
00:02:12,189 --> 00:02:14,770
looking at the strategy of how,

52
00:02:15,150 --> 00:02:17,250
research is done. This has interestingly,

53
00:02:17,709 --> 00:02:19,409
mainly been done by computational

54
00:02:19,789 --> 00:02:20,289
sociologists

55
00:02:21,069 --> 00:02:21,729
who look

56
00:02:22,189 --> 00:02:24,530
at science as a whole and analyze

57
00:02:25,069 --> 00:02:25,569
what

58
00:02:25,995 --> 00:02:28,235
strategies that people use and how we could

59
00:02:28,235 --> 00:02:29,775
accelerate those strategies.

60
00:02:30,555 --> 00:02:32,955
And maybe one of the most exciting work

61
00:02:32,955 --> 00:02:34,655
here in this field that at least,

62
00:02:35,514 --> 00:02:37,835
motivated me to look into this was a

63
00:02:37,835 --> 00:02:38,655
paper from

64
00:02:39,010 --> 00:02:40,550
02/2015,

65
00:02:41,330 --> 00:02:42,230
led by,

66
00:02:43,090 --> 00:02:43,590
Chicago

67
00:02:44,050 --> 00:02:45,189
computational sociologist,

68
00:02:45,490 --> 00:02:46,389
James Evans.

69
00:02:46,770 --> 00:02:49,430
And that paper, it was published in PNAS.

70
00:02:49,569 --> 00:02:51,590
It was called choosing experiments

71
00:02:52,215 --> 00:02:52,955
to accelerate

72
00:02:53,254 --> 00:02:54,474
collective discovery.

73
00:02:55,094 --> 00:02:58,075
And, therefore, the first time, I've seen how

74
00:02:58,135 --> 00:03:01,275
people use millions of scientific papers and patents

75
00:03:02,855 --> 00:03:06,474
to analyze what other concrete strategies that humans

76
00:03:06,694 --> 00:03:07,169
did,

77
00:03:08,689 --> 00:03:11,669
showing that those strategies are at a collective

78
00:03:11,810 --> 00:03:13,349
scale actually quite,

79
00:03:15,009 --> 00:03:15,509
improveable,

80
00:03:15,810 --> 00:03:17,969
let's say, like that. And then they did

81
00:03:17,969 --> 00:03:19,810
a lot of computer simulation to come up

82
00:03:19,810 --> 00:03:21,750
with potential different strategies.

83
00:03:22,375 --> 00:03:24,375
And they have pointed out that if you

84
00:03:24,375 --> 00:03:27,575
would follow if the collective would follow different

85
00:03:27,575 --> 00:03:29,594
strategies, you could theoretically,

86
00:03:30,455 --> 00:03:31,754
come to much faster,

87
00:03:33,094 --> 00:03:34,395
progress in science.

88
00:03:34,879 --> 00:03:36,800
But this was in 02/2015,

89
00:03:36,800 --> 00:03:37,280
and,

90
00:03:37,760 --> 00:03:38,740
this motivated

91
00:03:39,040 --> 00:03:40,479
me at the time that was still during

92
00:03:40,479 --> 00:03:43,199
my PhD to also try the same thing

93
00:03:43,199 --> 00:03:45,520
in my own field in quantum physics. And

94
00:03:45,520 --> 00:03:46,560
then we also,

95
00:03:47,944 --> 00:03:51,004
tried to use hundreds of thousands of papers

96
00:03:51,465 --> 00:03:52,284
to predict,

97
00:03:54,264 --> 00:03:56,504
what scientists will do in the field of

98
00:03:56,504 --> 00:03:58,584
quantum physics. And then we use the same

99
00:03:58,584 --> 00:04:02,310
techniques that James Evans, teams have used. Knowledge

100
00:04:02,310 --> 00:04:04,389
graphs where you have concepts that are the

101
00:04:04,389 --> 00:04:05,849
notes and edges

102
00:04:06,150 --> 00:04:07,930
that are formed when two concepts

103
00:04:08,549 --> 00:04:09,209
are connected,

104
00:04:11,030 --> 00:04:13,269
by a single paper. So you see somehow

105
00:04:13,269 --> 00:04:13,930
the evolution

106
00:04:14,229 --> 00:04:14,729
of,

107
00:04:15,764 --> 00:04:17,064
of what scientists

108
00:04:17,845 --> 00:04:18,824
did in the past.

109
00:04:20,164 --> 00:04:22,004
And that a number of,

110
00:04:22,564 --> 00:04:24,644
there are a number of follow-up papers came.

111
00:04:24,644 --> 00:04:26,725
So this one was published in 2020 in

112
00:04:26,725 --> 00:04:29,285
PNAS, and then we did the AI competition,

113
00:04:29,285 --> 00:04:32,009
but all was based on the same idea

114
00:04:32,009 --> 00:04:33,709
of using this knowledge graphs.

115
00:04:34,329 --> 00:04:34,649
And then,

116
00:04:36,569 --> 00:04:37,149
I met,

117
00:04:37,529 --> 00:04:38,029
Felix,

118
00:04:39,370 --> 00:04:40,110
and Evert,

119
00:04:41,209 --> 00:04:43,529
and they had a very different idea of

120
00:04:43,529 --> 00:04:44,589
how one could

121
00:04:44,970 --> 00:04:45,470
integrate

122
00:04:45,914 --> 00:04:48,414
scientific knowledge and especially the dynamics,

123
00:04:48,794 --> 00:04:51,354
and that is what this specific paper is

124
00:04:51,354 --> 00:04:51,854
about.

125
00:04:52,714 --> 00:04:55,354
I see. Okay. And and I I suppose

126
00:04:55,354 --> 00:04:57,034
it's well, is it obvious? I mean, it

127
00:04:57,034 --> 00:04:58,634
makes sense to me that you've got a

128
00:04:58,634 --> 00:05:01,589
huge amount of data, and that's where machine

129
00:05:01,589 --> 00:05:03,129
learning, artificial intelligence

130
00:05:03,830 --> 00:05:05,830
can really help. And and I think in

131
00:05:05,830 --> 00:05:07,689
the paper that you've published recently,

132
00:05:08,149 --> 00:05:10,310
you've looked at how machine learning can be

133
00:05:10,310 --> 00:05:11,449
used to look for

134
00:05:11,830 --> 00:05:14,250
emergent connections between subfields

135
00:05:14,985 --> 00:05:18,204
in quantum physics with the ultimate goal of

136
00:05:18,584 --> 00:05:19,084
forecasting

137
00:05:19,384 --> 00:05:21,245
the future direction of research.

138
00:05:22,264 --> 00:05:25,785
Can you explain what a subfield is and

139
00:05:25,785 --> 00:05:27,324
and perhaps give an example

140
00:05:27,785 --> 00:05:28,764
of an emergent

141
00:05:29,199 --> 00:05:30,879
connection. I think may maybe a lot of

142
00:05:30,879 --> 00:05:32,879
our listeners would know what a subfield is,

143
00:05:32,879 --> 00:05:33,379
but

144
00:05:33,680 --> 00:05:35,300
the emergent connection bit

145
00:05:35,680 --> 00:05:37,779
might be something new to them.

146
00:05:38,720 --> 00:05:41,379
Felix, could you, could you address that?

147
00:05:42,560 --> 00:05:45,014
Yeah. Sure. Of course. So for us,

148
00:05:45,655 --> 00:05:46,875
subfields in

149
00:05:47,254 --> 00:05:50,235
quantum physics within kind of this broader category

150
00:05:50,295 --> 00:05:51,495
would be something like,

151
00:05:51,895 --> 00:05:55,254
single photon quantum optics. Different sublet would be

152
00:05:55,254 --> 00:05:56,634
gravitational wave physics

153
00:05:56,935 --> 00:05:59,495
or topics related to, I don't know, fault

154
00:05:59,495 --> 00:06:00,955
tolerant quantum algorithms.

155
00:06:01,470 --> 00:06:03,729
All of these would fall under the umbrella

156
00:06:03,789 --> 00:06:06,930
term of quantum physics, but in of itself

157
00:06:06,990 --> 00:06:09,629
deal with very kinds of different questions and

158
00:06:09,629 --> 00:06:10,129
techniques.

159
00:06:10,509 --> 00:06:13,310
While quantum optics might focus on questions like

160
00:06:13,310 --> 00:06:14,449
how to generate,

161
00:06:14,975 --> 00:06:16,834
interfere, or detect single photons,

162
00:06:17,214 --> 00:06:19,394
topics like fault tolerant quantum algorithms

163
00:06:19,694 --> 00:06:22,274
might deal with questions about error correction

164
00:06:22,574 --> 00:06:23,794
or resource estimation,

165
00:06:24,095 --> 00:06:26,735
which, of course, fall under this umbrella term

166
00:06:26,735 --> 00:06:29,714
of, quantum physics but are conceptually quite

167
00:06:30,689 --> 00:06:32,529
distinct. But what we have seen in the

168
00:06:32,529 --> 00:06:33,029
past

169
00:06:33,330 --> 00:06:36,370
is that despite these differences, there can be

170
00:06:36,370 --> 00:06:37,590
fruitful overlaps

171
00:06:38,050 --> 00:06:39,029
between different

172
00:06:39,410 --> 00:06:39,910
subfields

173
00:06:40,529 --> 00:06:41,910
in general in science.

174
00:06:42,449 --> 00:06:45,430
One example that is quite prominent

175
00:06:45,764 --> 00:06:46,504
in the,

176
00:06:47,125 --> 00:06:49,845
kind of physics community has been, I think,

177
00:06:49,845 --> 00:06:52,805
the application of, machine learning to the study

178
00:06:52,805 --> 00:06:53,544
of quantum

179
00:06:53,845 --> 00:06:54,664
phase transitions,

180
00:06:55,044 --> 00:06:56,564
which is a connection that has only been

181
00:06:56,564 --> 00:06:57,944
made in around,

182
00:06:58,725 --> 00:06:59,625
2017,

183
00:07:00,004 --> 00:07:00,360
I think.

184
00:07:00,919 --> 00:07:02,699
Before that, these two

185
00:07:03,319 --> 00:07:03,819
fields

186
00:07:04,199 --> 00:07:07,959
were kind of completely distant. Machine learning at

187
00:07:07,959 --> 00:07:09,719
that time was mostly used for things like

188
00:07:09,719 --> 00:07:11,500
image classification or regression,

189
00:07:11,879 --> 00:07:14,535
and quantum phase transitions were studied with kind

190
00:07:14,535 --> 00:07:15,595
of very

191
00:07:16,055 --> 00:07:17,115
physics specific

192
00:07:17,574 --> 00:07:18,074
tools.

193
00:07:18,375 --> 00:07:20,634
But then in around, 2017,

194
00:07:20,935 --> 00:07:23,814
researchers realized that you could use measurements of,

195
00:07:24,294 --> 00:07:26,694
quantum systems to kind of treat them as

196
00:07:26,694 --> 00:07:29,750
structured data similar to images, making it possible

197
00:07:29,810 --> 00:07:32,129
to kind of combine these two techniques or

198
00:07:32,129 --> 00:07:35,649
apply machine learning methods from computer vision to

199
00:07:35,649 --> 00:07:36,550
physics problems.

200
00:07:36,930 --> 00:07:40,229
And this kind of quite simple connection but

201
00:07:40,370 --> 00:07:41,349
powerful idea,

202
00:07:41,685 --> 00:07:43,865
Combining these two distinct ideas

203
00:07:44,725 --> 00:07:47,585
has led to many fruitful results, basically. And

204
00:07:47,605 --> 00:07:48,264
the goal

205
00:07:48,845 --> 00:07:49,345
for

206
00:07:49,925 --> 00:07:51,384
this research project,

207
00:07:51,764 --> 00:07:53,285
Mario and I did, is to try to

208
00:07:53,285 --> 00:07:55,285
build or kind of improve a data driven

209
00:07:55,285 --> 00:07:58,470
method methods that helps to assess or even

210
00:07:58,470 --> 00:07:58,970
forecast

211
00:07:59,350 --> 00:08:01,850
ideas of kind of that type, basically.

212
00:08:02,870 --> 00:08:04,629
I see. And so, I mean, one thing

213
00:08:04,629 --> 00:08:05,129
that

214
00:08:05,589 --> 00:08:07,509
I can think of in, you know, in

215
00:08:07,509 --> 00:08:10,245
in experimental physics, for example, And I'd like

216
00:08:10,245 --> 00:08:12,004
to see, I mean, have I got the

217
00:08:12,004 --> 00:08:14,405
right idea here when I'm thinking about emergent

218
00:08:14,405 --> 00:08:14,905
connections?

219
00:08:15,525 --> 00:08:17,465
You know, for for example, you've got superconducting

220
00:08:17,764 --> 00:08:18,264
circuits.

221
00:08:18,725 --> 00:08:20,504
And, you know, in the past, superconducting

222
00:08:20,884 --> 00:08:22,985
circuits were were used to make

223
00:08:24,069 --> 00:08:24,810
very sensitive

224
00:08:25,189 --> 00:08:28,389
detectors of magnetic fields, weren't they, squids, which

225
00:08:28,389 --> 00:08:30,790
were used in medical physics and and and

226
00:08:30,790 --> 00:08:32,330
other research. But now

227
00:08:32,710 --> 00:08:35,610
superconducting circuits are used as qubits

228
00:08:35,924 --> 00:08:36,664
in quantum

229
00:08:37,044 --> 00:08:38,024
computing. So

230
00:08:38,404 --> 00:08:39,845
is that the sort of thing that you're

231
00:08:39,845 --> 00:08:43,245
looking at? You know, the the use of

232
00:08:43,245 --> 00:08:44,585
of of one quantum

233
00:08:45,284 --> 00:08:48,404
technology suddenly or well, probably not suddenly, but

234
00:08:48,404 --> 00:08:48,904
slowly

235
00:08:49,919 --> 00:08:50,419
becomes,

236
00:08:51,279 --> 00:08:54,399
useful in another area of physics? So moving

237
00:08:54,399 --> 00:08:54,899
from

238
00:08:55,759 --> 00:08:56,259
sensing

239
00:08:56,639 --> 00:08:59,440
to quantum computing. Is that, I mean, are

240
00:08:59,440 --> 00:09:01,360
those the sort of things that you're trying

241
00:09:01,360 --> 00:09:03,840
to predict in the future, or am I

242
00:09:03,840 --> 00:09:04,820
getting that wrong?

243
00:09:06,014 --> 00:09:08,835
Yes. This would be one of the examples.

244
00:09:09,134 --> 00:09:10,355
There are a number of other

245
00:09:10,735 --> 00:09:11,875
examples that are,

246
00:09:13,134 --> 00:09:16,014
where you would, as a human, potentially not

247
00:09:16,014 --> 00:09:17,554
see the connection,

248
00:09:18,120 --> 00:09:19,340
But just by having

249
00:09:20,360 --> 00:09:22,759
access to hundreds of thousands or millions of

250
00:09:22,759 --> 00:09:25,879
papers, the machine sees that there is some

251
00:09:25,879 --> 00:09:27,019
underlying connection.

252
00:09:27,399 --> 00:09:28,539
And then it can,

253
00:09:29,240 --> 00:09:32,059
say, okay. Very likely, this will be connected

254
00:09:32,120 --> 00:09:35,274
because I've seen this underlying connection before, and

255
00:09:35,274 --> 00:09:36,735
that always leads to connections.

256
00:09:38,075 --> 00:09:38,575
Yeah.

257
00:09:40,154 --> 00:09:42,795
I see. And, I mean, obviously, you know,

258
00:09:42,795 --> 00:09:44,955
being able to predict the future is very

259
00:09:44,955 --> 00:09:45,455
important.

260
00:09:46,990 --> 00:09:49,309
So who, I mean, who would benefit from

261
00:09:49,309 --> 00:09:51,389
this? I mean, it are you thinking of

262
00:09:51,389 --> 00:09:54,289
scientists who are trying to plan their career,

263
00:09:54,509 --> 00:09:55,950
or are you thinking of,

264
00:09:56,830 --> 00:09:57,889
research funders,

265
00:09:58,830 --> 00:10:00,924
looking at, you know, new areas to put

266
00:10:00,924 --> 00:10:02,225
money into or companies

267
00:10:02,924 --> 00:10:05,245
trying to work out what, you know, new

268
00:10:05,245 --> 00:10:08,125
technologies will emerge in the next little while.

269
00:10:08,125 --> 00:10:09,904
I mean, it sounds like it could benefit

270
00:10:10,524 --> 00:10:11,825
a lot of people.

271
00:10:13,840 --> 00:10:15,279
So I think for both,

272
00:10:15,840 --> 00:10:18,799
Mario and me, the motivation in the beginning

273
00:10:18,799 --> 00:10:21,220
was actually just a bit of, self interest

274
00:10:22,320 --> 00:10:25,120
at first. Because as researchers, I think one

275
00:10:25,120 --> 00:10:27,414
of the hardest tasks is to consistently come

276
00:10:27,414 --> 00:10:29,514
up with good and fruitful

277
00:10:29,975 --> 00:10:32,634
research ideas, either for ourselves

278
00:10:33,095 --> 00:10:34,475
or for our students.

279
00:10:34,934 --> 00:10:37,095
Because in order to kind of come up

280
00:10:37,095 --> 00:10:39,700
with many good research ideas, you need to

281
00:10:39,700 --> 00:10:42,019
have a mix of, like, good intuition about

282
00:10:42,019 --> 00:10:44,659
the topic, some experience, and sometimes even just

283
00:10:44,659 --> 00:10:46,019
a bit of luck to come up with,

284
00:10:46,339 --> 00:10:47,080
good ideas.

285
00:10:47,459 --> 00:10:49,379
And the hope for both of us was

286
00:10:49,379 --> 00:10:51,139
to kind of use such a model as

287
00:10:51,139 --> 00:10:52,360
sort of like an artificial

288
00:10:52,820 --> 00:10:54,695
muse in the sense that we can have

289
00:10:54,695 --> 00:10:57,414
a data driven method that helps us to

290
00:10:57,414 --> 00:11:00,695
uncover or assess combination of research topics that

291
00:11:00,695 --> 00:11:03,095
no one has considered before, where there's, like,

292
00:11:03,095 --> 00:11:06,074
no benchmark of, combining these two things

293
00:11:07,254 --> 00:11:07,754
together.

294
00:11:08,240 --> 00:11:10,579
So who would benefit from that? Basically,

295
00:11:10,879 --> 00:11:13,539
anyone working in science who regularly needs to

296
00:11:13,600 --> 00:11:16,259
generate new research ideas or kind of project,

297
00:11:17,440 --> 00:11:18,259
project topics.

298
00:11:18,959 --> 00:11:21,139
Yeah. Good benefit from this tool, I think.

299
00:11:22,125 --> 00:11:25,184
I think the biggest benefit could come from

300
00:11:25,485 --> 00:11:25,985
collaborations,

301
00:11:27,804 --> 00:11:28,785
suggesting collaborations

302
00:11:29,325 --> 00:11:29,825
for

303
00:11:30,205 --> 00:11:32,465
researchers that are not in the same field,

304
00:11:32,764 --> 00:11:35,730
potentially that are in very far away fields.

305
00:11:35,970 --> 00:11:37,429
For instance, let's say I have

306
00:11:37,889 --> 00:11:38,870
a friend who works

307
00:11:39,570 --> 00:11:42,070
in astrophysics. I know nothing about astrophysics.

308
00:11:42,850 --> 00:11:44,389
Now I want to work together.

309
00:11:44,850 --> 00:11:45,509
I would

310
00:11:45,889 --> 00:11:48,709
potentially not even know where to start.

311
00:11:50,154 --> 00:11:52,735
I have my techniques. They have their techniques.

312
00:11:53,914 --> 00:11:54,735
I have no idea,

313
00:11:55,434 --> 00:11:57,514
what is their connection. But the machine has

314
00:11:57,514 --> 00:12:00,075
seen hundreds of thousands or millions of papers,

315
00:12:00,075 --> 00:12:02,335
and they could see some underlying

316
00:12:03,070 --> 00:12:04,450
connection that might,

317
00:12:05,389 --> 00:12:07,070
be possible in the future that has not

318
00:12:07,070 --> 00:12:07,809
been possible,

319
00:12:08,829 --> 00:12:11,149
or that has not been done so far.

320
00:12:11,149 --> 00:12:12,990
So I see a lot of potential for

321
00:12:12,990 --> 00:12:13,490
those

322
00:12:13,950 --> 00:12:15,089
very surprising,

323
00:12:15,470 --> 00:12:16,450
very distant

324
00:12:18,225 --> 00:12:18,725
collaborations.

325
00:12:19,105 --> 00:12:21,044
That would be really great.

326
00:12:21,985 --> 00:12:24,464
I see. And I wanted to ask you

327
00:12:24,464 --> 00:12:24,865
about,

328
00:12:25,345 --> 00:12:27,444
the data that you use to train

329
00:12:27,745 --> 00:12:28,644
your system.

330
00:12:29,024 --> 00:12:31,024
I mean, I'm guessing you that you're using

331
00:12:31,024 --> 00:12:31,764
the titles

332
00:12:32,360 --> 00:12:35,740
of, peer reviewed papers. You're using the abstracts.

333
00:12:35,879 --> 00:12:37,820
Are you also looking at,

334
00:12:39,079 --> 00:12:41,100
references that are made in papers

335
00:12:41,559 --> 00:12:45,000
in order to to make connections? What what

336
00:12:45,000 --> 00:12:46,539
sort of input do you have?

337
00:12:48,004 --> 00:12:50,084
It's actually quite quite simple. So we use

338
00:12:50,084 --> 00:12:54,084
publicly available preprints of quantum physics papers from

339
00:12:54,084 --> 00:12:54,985
the archive.

340
00:12:56,004 --> 00:12:57,225
In general, researchers

341
00:12:57,605 --> 00:12:59,705
submitted their manuscripts to,

342
00:13:00,164 --> 00:13:03,705
websites like the archive before formal, formal publication,

343
00:13:04,139 --> 00:13:06,879
which makes the consent quite easily accessible,

344
00:13:07,580 --> 00:13:10,300
for us. And in particular, we focus on

345
00:13:10,300 --> 00:13:14,000
the abstracts of each preprint in this quantum

346
00:13:14,139 --> 00:13:14,639
physics,

347
00:13:15,580 --> 00:13:16,080
subsection.

348
00:13:16,725 --> 00:13:19,384
The idea behind that is that the abstract

349
00:13:19,764 --> 00:13:20,264
provides,

350
00:13:21,044 --> 00:13:23,684
provides a condensed summary of the kind of

351
00:13:23,684 --> 00:13:26,644
core ideas of the paper, which typically also

352
00:13:26,644 --> 00:13:29,125
includes kind of one or two sentences about

353
00:13:29,125 --> 00:13:29,865
the background,

354
00:13:30,325 --> 00:13:32,579
some information about the main results, and then

355
00:13:32,579 --> 00:13:35,139
also some information about the the impact,

356
00:13:35,779 --> 00:13:36,600
maybe even,

357
00:13:37,059 --> 00:13:38,039
about that paper.

358
00:13:38,579 --> 00:13:41,079
And only focusing on the abstract

359
00:13:41,379 --> 00:13:43,220
and not, for instance, on the kind of

360
00:13:43,220 --> 00:13:46,339
main figures in the paper simplifies the data

361
00:13:46,339 --> 00:13:49,404
quite a bit because, well, typically, in abstracts,

362
00:13:49,404 --> 00:13:50,625
you don't need to deal with,

363
00:13:51,165 --> 00:13:53,504
formulas or citations or references.

364
00:13:54,205 --> 00:13:56,524
And it also kind of, takes away all

365
00:13:56,524 --> 00:13:59,404
the guesswork with things like, figures, for for

366
00:13:59,404 --> 00:14:02,610
instance. Because what abstracts are typically text based,

367
00:14:03,470 --> 00:14:04,449
which means it's,

368
00:14:05,070 --> 00:14:07,730
much easier to kind of extract the information

369
00:14:07,949 --> 00:14:09,889
from that for the machine learning model.

370
00:14:11,070 --> 00:14:13,709
I see. And and is it possible to

371
00:14:13,709 --> 00:14:15,649
explain in simple terms

372
00:14:16,774 --> 00:14:19,254
how the the machine learning is used to

373
00:14:19,254 --> 00:14:21,174
process these data? I mean, you know, sort

374
00:14:21,174 --> 00:14:23,414
of keep in mind that you're you're speaking

375
00:14:23,414 --> 00:14:24,154
to physicists

376
00:14:25,014 --> 00:14:25,834
rather than

377
00:14:26,134 --> 00:14:28,154
computer scientists. Is there

378
00:14:28,559 --> 00:14:30,899
is there an easy way to describe

379
00:14:31,440 --> 00:14:33,779
how it's done? Or may maybe there's not

380
00:14:34,240 --> 00:14:36,320
an easy way. But, I mean, you know,

381
00:14:36,320 --> 00:14:38,000
how would you explain it to a colleague

382
00:14:38,000 --> 00:14:40,980
who's a physicist rather than a computer scientist?

383
00:14:42,504 --> 00:14:46,044
For sure. So in the machine learning pipeline,

384
00:14:46,184 --> 00:14:49,004
there are three main steps. There is a

385
00:14:49,144 --> 00:14:52,584
data collection phase, there is the words embedding

386
00:14:52,584 --> 00:14:55,485
phase, and a training and prediction phase.

387
00:14:55,820 --> 00:14:58,539
The first step, data collection, kind of connects

388
00:14:58,539 --> 00:15:00,639
to the previous question you asked,

389
00:15:01,740 --> 00:15:04,299
where we kind of take a bunch of,

390
00:15:04,779 --> 00:15:08,080
quantum physics publications, look at the abstract,

391
00:15:08,634 --> 00:15:10,894
and then from each abstract, identify

392
00:15:11,274 --> 00:15:14,235
key scientific concepts we are interested in tracking

393
00:15:14,235 --> 00:15:17,054
over time, which in our case are, well,

394
00:15:17,355 --> 00:15:19,615
quantum physics concepts, things like,

395
00:15:19,995 --> 00:15:20,495
entanglement,

396
00:15:21,115 --> 00:15:23,534
quantum circuits, or phase transition.

397
00:15:24,610 --> 00:15:25,590
The second step,

398
00:15:26,049 --> 00:15:26,549
the

399
00:15:27,330 --> 00:15:30,129
word embedding is then the kind of key

400
00:15:30,129 --> 00:15:34,070
idea of this paper. Because in order to

401
00:15:34,210 --> 00:15:35,830
use machine learning to

402
00:15:36,769 --> 00:15:37,269
connect

403
00:15:37,730 --> 00:15:38,549
these concepts,

404
00:15:39,004 --> 00:15:40,784
You need to be able to translate

405
00:15:41,164 --> 00:15:42,384
these words

406
00:15:42,764 --> 00:15:45,804
into numbers that a machine learning model can

407
00:15:45,804 --> 00:15:49,084
understand and process that somehow encodes their meaning

408
00:15:49,084 --> 00:15:49,584
and

409
00:15:49,964 --> 00:15:50,464
usage.

410
00:15:51,004 --> 00:15:54,460
And, yeah, kind of simple ideas like just

411
00:15:54,460 --> 00:15:57,019
giving each word you're interested in a unique

412
00:15:57,019 --> 00:15:59,740
number, of course, wouldn't really work because this

413
00:15:59,740 --> 00:16:02,160
would throw away all the contextual information

414
00:16:02,540 --> 00:16:05,565
about how similar or dissimilar certain words are

415
00:16:05,725 --> 00:16:07,105
or how they are used,

416
00:16:07,804 --> 00:16:10,524
in the context of these abstracts. So what

417
00:16:10,524 --> 00:16:13,404
we do instead is a technique called word

418
00:16:13,404 --> 00:16:16,845
embeddings, which basically turns each word into the

419
00:16:16,845 --> 00:16:17,345
abstract,

420
00:16:17,820 --> 00:16:18,320
into

421
00:16:18,779 --> 00:16:19,679
a high dimensional

422
00:16:20,460 --> 00:16:23,820
vector, which somehow encodes its meaning or how

423
00:16:23,820 --> 00:16:25,200
it is used.

424
00:16:25,740 --> 00:16:28,940
And then you kind of use these,

425
00:16:29,740 --> 00:16:32,460
vectors for the machine learning model. I tried

426
00:16:32,460 --> 00:16:34,355
to come up with a simple example to

427
00:16:34,355 --> 00:16:35,654
kind of highlight

428
00:16:36,115 --> 00:16:38,154
how this could look like. And for this,

429
00:16:38,154 --> 00:16:39,815
I have, like, two sentences.

430
00:16:40,674 --> 00:16:42,294
So the sun is hot

431
00:16:42,674 --> 00:16:45,794
and the moon is cold, which you could

432
00:16:45,794 --> 00:16:48,455
try to encode with such a word embedding.

433
00:16:48,620 --> 00:16:50,620
And in the end, what you would look

434
00:16:50,620 --> 00:16:52,000
for in the embeddings

435
00:16:52,379 --> 00:16:54,459
are then kind of a representation of how

436
00:16:54,459 --> 00:16:56,220
these individual words are used in these two

437
00:16:56,220 --> 00:16:58,779
sentences. So for instance, on some kind of

438
00:16:58,779 --> 00:17:01,179
axis in the embedding, the words sun and

439
00:17:01,179 --> 00:17:02,325
moon might

440
00:17:02,804 --> 00:17:05,285
share one dimension because they're used as the

441
00:17:05,285 --> 00:17:06,904
subjects in the sentences.

442
00:17:07,285 --> 00:17:09,865
The words hot and cold might then lie

443
00:17:09,924 --> 00:17:12,805
might then lie on different axis because they

444
00:17:12,805 --> 00:17:14,505
kind of represent the temperature

445
00:17:15,285 --> 00:17:15,990
of an object.

446
00:17:17,029 --> 00:17:19,269
And the idea is then that somehow the

447
00:17:19,269 --> 00:17:20,650
embedding also encodes

448
00:17:21,990 --> 00:17:24,150
how these different words relate to each other.

449
00:17:24,150 --> 00:17:26,150
So for instance, if you would compute the

450
00:17:26,150 --> 00:17:27,769
overlap between the vector

451
00:17:28,085 --> 00:17:30,984
of the word sun and hot, it should,

452
00:17:31,444 --> 00:17:33,865
if the embedding is well trained, be larger

453
00:17:34,085 --> 00:17:36,964
as then the embedding between sun and cold

454
00:17:36,964 --> 00:17:38,884
because sun and hot are kind of more

455
00:17:38,884 --> 00:17:39,865
closely aligned.

456
00:17:40,325 --> 00:17:42,005
Yeah. This is kind of the the general

457
00:17:42,005 --> 00:17:44,390
idea of how we want to encode the

458
00:17:44,529 --> 00:17:46,309
underlying information about the physics

459
00:17:46,690 --> 00:17:47,589
into these

460
00:17:48,130 --> 00:17:48,630
vectors.

461
00:17:49,250 --> 00:17:50,950
And then the only thing that this,

462
00:17:52,769 --> 00:17:54,769
machine learning model that we then kind of

463
00:17:54,769 --> 00:17:56,210
train to do the predictions in the end

464
00:17:56,210 --> 00:17:58,115
does is look at,

465
00:17:58,974 --> 00:18:01,455
two of these vectors and then try to

466
00:18:01,455 --> 00:18:03,474
predict whether these two vectors

467
00:18:03,775 --> 00:18:05,634
will appear in the same abstract

468
00:18:05,934 --> 00:18:06,674
in a given,

469
00:18:07,855 --> 00:18:08,595
time frame.

470
00:18:09,150 --> 00:18:10,910
What we do in the paper is we

471
00:18:10,910 --> 00:18:11,410
define

472
00:18:11,710 --> 00:18:14,049
a training window, so for instance, from

473
00:18:14,430 --> 00:18:16,670
2000 to 02/2010,

474
00:18:16,670 --> 00:18:19,170
and look at which concept pairs were not

475
00:18:19,390 --> 00:18:22,515
linked yet in this tracking window. And what

476
00:18:22,515 --> 00:18:24,595
the model, tries to do is then to

477
00:18:24,595 --> 00:18:28,195
predict which of these unconnected pairs are likely

478
00:18:28,195 --> 00:18:30,695
to appear together in future abstracts

479
00:18:31,075 --> 00:18:33,795
within a time window of three years, for

480
00:18:33,795 --> 00:18:34,295
instance.

481
00:18:34,789 --> 00:18:36,789
And then the idea is that if you

482
00:18:36,789 --> 00:18:38,710
want some kind of use this machine learning

483
00:18:38,710 --> 00:18:41,029
model for inference to make kind of further

484
00:18:41,029 --> 00:18:43,769
predictions, you basically just shift the time window

485
00:18:44,069 --> 00:18:47,190
from kind of 2,000 to 2,010. You could

486
00:18:47,190 --> 00:18:49,829
also use it to predict from 2,010 to

487
00:18:49,829 --> 00:18:50,684
2,020

488
00:18:51,164 --> 00:18:53,404
or even, further. That's kind of the the

489
00:18:53,404 --> 00:18:54,305
general idea.

490
00:18:54,684 --> 00:18:55,985
I would like to mention

491
00:18:56,445 --> 00:18:56,945
that

492
00:18:57,245 --> 00:18:59,505
what Felix just explained is the

493
00:18:59,884 --> 00:19:00,785
main difference

494
00:19:01,164 --> 00:19:02,924
from our work to what has been done

495
00:19:02,924 --> 00:19:03,424
before.

496
00:19:03,839 --> 00:19:05,519
So before I close what I mentioned in

497
00:19:05,519 --> 00:19:07,299
the beginning, the computational sociologists

498
00:19:08,000 --> 00:19:10,339
and also in my own previous

499
00:19:10,720 --> 00:19:13,519
work, we have built up knowledge graphs. And

500
00:19:13,519 --> 00:19:14,339
knowledge graphs,

501
00:19:15,039 --> 00:19:18,559
have concepts that are the vertices of the

502
00:19:18,559 --> 00:19:20,875
graph, and edges are drawn when,

503
00:19:21,974 --> 00:19:24,234
two concepts are mentioned in the paper.

504
00:19:25,335 --> 00:19:25,835
Now

505
00:19:26,294 --> 00:19:28,154
when I just use the concepts,

506
00:19:28,774 --> 00:19:31,595
I throw away everything about the

507
00:19:32,054 --> 00:19:33,269
context of that word.

508
00:19:34,150 --> 00:19:35,930
That's exactly what Felix mentioned.

509
00:19:36,309 --> 00:19:38,009
Basically, I just define

510
00:19:38,309 --> 00:19:40,390
the word as a number and remove all

511
00:19:40,390 --> 00:19:41,529
of the other information.

512
00:19:42,070 --> 00:19:44,710
But now with the technique that, Felix just

513
00:19:44,710 --> 00:19:47,244
explained that we published in this paper, we

514
00:19:47,244 --> 00:19:49,585
get much more context of the words

515
00:19:50,044 --> 00:19:52,065
by using this more modern,

516
00:19:52,605 --> 00:19:55,565
word embedding instead of knowledge graphs. That comes

517
00:19:55,565 --> 00:19:57,265
with extra technical,

518
00:19:58,044 --> 00:20:00,144
challenges that we had to,

519
00:20:00,525 --> 00:20:01,025
solve,

520
00:20:01,380 --> 00:20:03,720
But then you can get much more context

521
00:20:04,099 --> 00:20:06,680
of the word, of the phrases itself,

522
00:20:07,059 --> 00:20:07,880
and, hopefully,

523
00:20:08,180 --> 00:20:09,799
then get better predictions.

524
00:20:11,059 --> 00:20:14,339
I see. And you use data that went

525
00:20:14,339 --> 00:20:16,259
back to 1994.

526
00:20:16,259 --> 00:20:18,055
So that's, you know, we're looking more

527
00:20:18,595 --> 00:20:20,454
than thirty years ago.

528
00:20:21,954 --> 00:20:23,634
I mean, if you sort of run your

529
00:20:23,634 --> 00:20:25,335
system using older data,

530
00:20:25,795 --> 00:20:28,375
do do you find that it predicts things

531
00:20:28,835 --> 00:20:30,214
that actually happened

532
00:20:30,789 --> 00:20:32,410
a decade or two later?

533
00:20:33,990 --> 00:20:36,970
The short answer to that would be yes.

534
00:20:37,509 --> 00:20:38,410
Well, congratulations.

535
00:20:40,630 --> 00:20:42,250
And there are basically

536
00:20:42,549 --> 00:20:45,190
two different types of predictions our model can

537
00:20:45,190 --> 00:20:48,044
make. Our model can make validation predictions

538
00:20:48,345 --> 00:20:49,404
where we can still

539
00:20:49,704 --> 00:20:52,044
check the outcome and truly

540
00:20:52,345 --> 00:20:53,085
new predictions

541
00:20:53,384 --> 00:20:56,444
where we don't entirely know the outcome yet.

542
00:20:56,585 --> 00:20:58,424
As you mentioned, our model was trained on

543
00:20:58,424 --> 00:21:01,259
data from nineteen nineteen four till,

544
00:21:01,660 --> 00:21:02,559
2017,

545
00:21:02,859 --> 00:21:04,859
in our case, where the model was kind

546
00:21:04,859 --> 00:21:06,640
of used to predict,

547
00:21:07,900 --> 00:21:09,980
contact connections within the next three years, so

548
00:21:09,980 --> 00:21:11,279
until 2019.

549
00:21:11,820 --> 00:21:14,320
And the validation predictions we did

550
00:21:14,904 --> 00:21:15,644
were then

551
00:21:16,025 --> 00:21:18,184
by, kind of shift the time window, as

552
00:21:18,184 --> 00:21:20,744
I explained for the, previous question, where we

553
00:21:20,744 --> 00:21:21,644
tried to predict

554
00:21:22,904 --> 00:21:24,684
or could yeah. Validation predictions,

555
00:21:25,464 --> 00:21:26,904
from 2020

556
00:21:26,904 --> 00:21:28,664
till 2023

557
00:21:28,664 --> 00:21:31,220
where there still is real world data that

558
00:21:31,220 --> 00:21:32,599
we can test against.

559
00:21:33,139 --> 00:21:35,480
And our model managed to kind of discover

560
00:21:36,019 --> 00:21:39,139
a couple of emergent research directions there, which

561
00:21:39,139 --> 00:21:42,765
includes ideas like using tensor network methods to

562
00:21:42,765 --> 00:21:45,724
simulate local quantum circuits, so quantum circuits with

563
00:21:45,724 --> 00:21:47,105
only minimal entanglement,

564
00:21:47,724 --> 00:21:50,384
or ideas of using machine learning to optimize

565
00:21:50,445 --> 00:21:51,984
the structure of,

566
00:21:53,325 --> 00:21:54,144
quantum circuits.

567
00:21:54,460 --> 00:21:56,539
And these kind of, two examples and a

568
00:21:56,539 --> 00:21:58,779
few more are things we discuss in the

569
00:21:58,779 --> 00:22:00,320
paper in detail, basically.

570
00:22:01,820 --> 00:22:03,680
The question about truly

571
00:22:04,059 --> 00:22:04,720
new predictions

572
00:22:05,340 --> 00:22:07,440
is slightly more complicated,

573
00:22:08,615 --> 00:22:09,434
I would say.

574
00:22:10,774 --> 00:22:11,274
One

575
00:22:11,654 --> 00:22:12,154
idea

576
00:22:12,615 --> 00:22:14,534
that I need to discuss for further first

577
00:22:14,534 --> 00:22:15,914
is the idea of,

578
00:22:16,454 --> 00:22:16,954
calibration

579
00:22:17,335 --> 00:22:18,294
of the,

580
00:22:18,855 --> 00:22:20,554
prediction model. Because

581
00:22:21,095 --> 00:22:23,390
in the end, what you are generally interested

582
00:22:23,390 --> 00:22:25,569
in is to have kind of a

583
00:22:25,869 --> 00:22:27,809
small subsets of predictions

584
00:22:28,109 --> 00:22:30,930
where the model is super subbed that,

585
00:22:31,630 --> 00:22:32,369
these predictions

586
00:22:32,990 --> 00:22:33,650
are actually,

587
00:22:34,509 --> 00:22:36,190
correct. So one thing that you need to

588
00:22:36,190 --> 00:22:37,569
check first is

589
00:22:37,964 --> 00:22:39,184
is the confidence

590
00:22:39,884 --> 00:22:42,525
of your machine learning model aligns to the

591
00:22:42,525 --> 00:22:43,025
probability

592
00:22:43,404 --> 00:22:46,625
of being right or false in this prediction?

593
00:22:46,684 --> 00:22:48,044
This is kind of one section in the

594
00:22:48,044 --> 00:22:49,105
paper where we analyze,

595
00:22:49,724 --> 00:22:51,265
like, how our model performs,

596
00:22:52,044 --> 00:22:54,279
in this aspect. And it turns out that

597
00:22:54,279 --> 00:22:56,519
this actually works kind of well. So if

598
00:22:56,519 --> 00:22:58,200
we look at the output probability of the

599
00:22:58,200 --> 00:23:00,519
model, this is aligned quite well with the

600
00:23:00,519 --> 00:23:03,259
probability of the model being false or correct.

601
00:23:03,559 --> 00:23:05,420
So what you can do then is basically

602
00:23:06,359 --> 00:23:06,859
predict

603
00:23:07,615 --> 00:23:09,934
many, many concepts, like, tens of thousands of

604
00:23:09,934 --> 00:23:10,434
concepts,

605
00:23:10,734 --> 00:23:14,255
and then rank them by their probability of

606
00:23:14,255 --> 00:23:15,554
actually occurring

607
00:23:15,934 --> 00:23:16,674
in the future.

608
00:23:17,375 --> 00:23:20,170
And we we did that and found a

609
00:23:20,170 --> 00:23:21,710
subset of predictions that have

610
00:23:22,009 --> 00:23:23,630
over 99.9%

611
00:23:24,009 --> 00:23:24,509
probability

612
00:23:25,289 --> 00:23:27,930
of occurring. And this was very interesting because

613
00:23:27,930 --> 00:23:29,230
most of these concepts

614
00:23:29,529 --> 00:23:32,730
seem to be related to single photon quantum

615
00:23:32,730 --> 00:23:33,230
optics.

616
00:23:33,609 --> 00:23:36,474
So things that are related to concepts like

617
00:23:36,474 --> 00:23:38,734
photon detector and spatial modes

618
00:23:39,035 --> 00:23:41,295
or concepts like polarization entanglements

619
00:23:41,674 --> 00:23:42,174
and,

620
00:23:42,554 --> 00:23:43,375
single photon,

621
00:23:44,075 --> 00:23:45,214
single photon source.

622
00:23:46,234 --> 00:23:48,234
And this was actually something I had a

623
00:23:48,234 --> 00:23:51,130
chat about with with a quantum opt optics

624
00:23:51,130 --> 00:23:53,930
professor from Leiden University, where I'm doing my

625
00:23:53,930 --> 00:23:54,430
PhD.

626
00:23:54,890 --> 00:23:57,690
And all of these predictions actually somehow seem

627
00:23:57,690 --> 00:23:58,269
to make

628
00:23:58,730 --> 00:24:00,430
sense, but are also already

629
00:24:01,130 --> 00:24:03,230
known. So the catch with that is basically

630
00:24:03,724 --> 00:24:05,184
that some of the publications

631
00:24:06,445 --> 00:24:08,924
that were in the kind of subset of

632
00:24:08,924 --> 00:24:09,904
quantum physics,

633
00:24:10,525 --> 00:24:11,744
abstracts that we used

634
00:24:12,285 --> 00:24:14,765
mentioned these concepts. But, of course, we did

635
00:24:14,765 --> 00:24:15,424
not specifically

636
00:24:15,805 --> 00:24:18,865
train our model on optics papers,

637
00:24:19,259 --> 00:24:22,400
which means it's kind of solve these connections,

638
00:24:22,539 --> 00:24:23,440
make these connections,

639
00:24:23,820 --> 00:24:26,220
but it never saw the actual kind of

640
00:24:26,220 --> 00:24:26,720
papers

641
00:24:27,180 --> 00:24:29,580
where these concepts are explicitly mentioned. So there

642
00:24:29,580 --> 00:24:30,240
are kind of

643
00:24:30,779 --> 00:24:33,644
two different interpretations I can make on that.

644
00:24:33,724 --> 00:24:36,205
Kind of one optimistic interpretation would be that

645
00:24:36,205 --> 00:24:38,464
our machine learning model kind of has discovered

646
00:24:38,605 --> 00:24:41,325
the concept of single photon quantum optics on

647
00:24:41,325 --> 00:24:41,984
its own,

648
00:24:42,285 --> 00:24:44,384
and the kind of slightly more conservative,

649
00:24:45,164 --> 00:24:47,460
interpretation would be that in order to get

650
00:24:47,940 --> 00:24:49,880
very meaningful predictions in the future,

651
00:24:50,659 --> 00:24:51,960
one would need to train

652
00:24:52,579 --> 00:24:54,759
our model in kind of all physics applications

653
00:24:54,980 --> 00:24:56,679
to rule out these

654
00:24:57,059 --> 00:25:00,039
kind of trivial or simple predictions

655
00:25:00,445 --> 00:25:02,205
that might have not occurred together in the

656
00:25:02,205 --> 00:25:03,825
training set, but

657
00:25:04,445 --> 00:25:06,205
might be, like, somewhere out there on the

658
00:25:06,205 --> 00:25:07,105
Internet, basically.

659
00:25:08,285 --> 00:25:10,765
I see. And is that is that something

660
00:25:10,765 --> 00:25:11,585
that's possible,

661
00:25:12,605 --> 00:25:14,465
you know, to expand the training

662
00:25:14,920 --> 00:25:16,380
to include all

663
00:25:16,920 --> 00:25:19,640
physics papers? I mean, I mean, first of

664
00:25:19,640 --> 00:25:21,420
all, you know, would you have the computing

665
00:25:21,960 --> 00:25:24,440
resources to do that, and and would that

666
00:25:24,440 --> 00:25:25,180
be useful,

667
00:25:26,359 --> 00:25:27,980
in terms of making predictions?

668
00:25:28,954 --> 00:25:30,894
Maybe you wanna speak about that, Mario?

669
00:25:31,434 --> 00:25:31,934
Yes.

670
00:25:32,714 --> 00:25:33,615
So this is,

671
00:25:34,075 --> 00:25:37,755
definitely possible. Actually, we had, different status where

672
00:25:37,755 --> 00:25:38,894
we not only used,

673
00:25:39,275 --> 00:25:40,950
quantum physics as

674
00:25:41,509 --> 00:25:42,970
input data, but also,

675
00:25:45,109 --> 00:25:48,230
entire physics. Or in one study, we even

676
00:25:48,230 --> 00:25:49,690
used all published

677
00:25:50,149 --> 00:25:50,649
papers,

678
00:25:51,509 --> 00:25:52,329
that exist.

679
00:25:53,525 --> 00:25:55,944
I just looked it up 58,000,000

680
00:25:56,244 --> 00:25:59,224
research papers. So that is possible to take

681
00:25:59,365 --> 00:26:00,505
much, much larger,

682
00:26:02,964 --> 00:26:03,464
subsets

683
00:26:03,845 --> 00:26:04,325
of,

684
00:26:04,884 --> 00:26:05,339
science.

685
00:26:06,220 --> 00:26:07,200
Of course, then

686
00:26:07,579 --> 00:26:09,119
using this data will become

687
00:26:09,500 --> 00:26:10,000
significantly

688
00:26:10,460 --> 00:26:12,000
more computationally expensive

689
00:26:12,460 --> 00:26:15,339
so that the process will be more expensive.

690
00:26:15,339 --> 00:26:15,839
But,

691
00:26:16,380 --> 00:26:18,799
if we really want to go into predicting

692
00:26:18,940 --> 00:26:19,440
novel

693
00:26:20,194 --> 00:26:22,375
research directions, then we have to do this.

694
00:26:22,515 --> 00:26:24,115
So I think in our paper, we showed

695
00:26:24,115 --> 00:26:25,654
it as this huge potential.

696
00:26:26,275 --> 00:26:27,494
And now

697
00:26:27,795 --> 00:26:28,855
we could go into,

698
00:26:29,634 --> 00:26:32,355
thinking about using it for whole physics or

699
00:26:32,355 --> 00:26:35,320
for whole natural science or just whatever

700
00:26:35,779 --> 00:26:36,759
for all science.

701
00:26:37,859 --> 00:26:39,700
Right. Okay. And is that, I mean, is

702
00:26:39,700 --> 00:26:41,400
that something that you and your

703
00:26:41,700 --> 00:26:44,420
collaborators would be interested in? Or do or

704
00:26:44,420 --> 00:26:45,539
do you think that,

705
00:26:46,580 --> 00:26:49,434
that, you know, may maybe maybe people working

706
00:26:49,654 --> 00:26:52,235
or studying other fields, people who study

707
00:26:53,095 --> 00:26:54,154
how biology

708
00:26:54,455 --> 00:26:57,815
evolves, for example, would, would join with you?

709
00:26:57,815 --> 00:26:59,460
I mean, do do do you see this?

710
00:26:59,940 --> 00:27:01,399
I mean, is this a new field

711
00:27:01,700 --> 00:27:03,700
or a growing field that,

712
00:27:04,339 --> 00:27:06,200
that you think you've made a significant

713
00:27:06,500 --> 00:27:07,720
contribution to,

714
00:27:08,500 --> 00:27:10,740
and, you know, other people will follow on

715
00:27:10,740 --> 00:27:11,480
from you?

716
00:27:12,464 --> 00:27:12,964
There

717
00:27:13,664 --> 00:27:15,265
there are, there are a lot of groups

718
00:27:15,265 --> 00:27:16,244
that look into

719
00:27:17,825 --> 00:27:20,805
this direction or related direction. It's a huge,

720
00:27:21,744 --> 00:27:22,644
field to,

721
00:27:23,744 --> 00:27:25,049
to try to come up with

722
00:27:25,690 --> 00:27:29,610
computer inspired new ideas in a, data driven

723
00:27:29,610 --> 00:27:30,110
way.

724
00:27:30,809 --> 00:27:32,970
We show one method how that is done.

725
00:27:32,970 --> 00:27:34,350
There are many other methods.

726
00:27:35,450 --> 00:27:36,830
Now the question is,

727
00:27:37,369 --> 00:27:37,869
is

728
00:27:38,455 --> 00:27:40,555
predicting what scientists will do,

729
00:27:41,654 --> 00:27:43,815
is this the one thing you need to

730
00:27:43,815 --> 00:27:46,075
discover new ideas or are there other things?

731
00:27:46,535 --> 00:27:48,855
For instance, you can think about, can I

732
00:27:48,855 --> 00:27:49,355
predict

733
00:27:49,735 --> 00:27:50,235
impactful

734
00:27:50,775 --> 00:27:51,914
research directions?

735
00:27:52,940 --> 00:27:54,779
And just a few months ago, we have

736
00:27:54,779 --> 00:27:57,519
also published in machine learning science and technology

737
00:27:57,579 --> 00:27:57,980
and,

738
00:27:58,380 --> 00:27:58,880
IOP,

739
00:27:59,980 --> 00:28:00,480
journal,

740
00:28:00,859 --> 00:28:02,240
a paper where we showed,

741
00:28:02,779 --> 00:28:04,539
that was, led by,

742
00:28:05,065 --> 00:28:07,785
my former post doc, Shumei Gu, who is

743
00:28:07,785 --> 00:28:08,924
also coauthor

744
00:28:09,225 --> 00:28:10,684
of the paper with Felix,

745
00:28:11,705 --> 00:28:14,445
that we showed that you can actually predict

746
00:28:14,505 --> 00:28:15,005
also,

747
00:28:15,865 --> 00:28:17,644
impactful research directions.

748
00:28:18,220 --> 00:28:20,240
So let's say you have two concepts

749
00:28:20,940 --> 00:28:23,600
that have never been started before together.

750
00:28:24,700 --> 00:28:26,000
Now the question is,

751
00:28:27,019 --> 00:28:28,160
will there be papers

752
00:28:28,460 --> 00:28:31,740
that started it? And the next question is,

753
00:28:31,740 --> 00:28:33,759
will those paper that studied this

754
00:28:34,195 --> 00:28:36,595
topics that have never been studied before, will

755
00:28:36,595 --> 00:28:37,734
they be impactful?

756
00:28:38,115 --> 00:28:38,775
Will they,

757
00:28:39,474 --> 00:28:40,775
create a lot of citations?

758
00:28:41,234 --> 00:28:42,615
And very surprisingly

759
00:28:42,914 --> 00:28:45,015
to us, this is also predictable.

760
00:28:46,410 --> 00:28:48,970
So now we can also predict in some

761
00:28:48,970 --> 00:28:52,430
way what future research directions could be impactful.

762
00:28:54,009 --> 00:28:56,170
Of course, what Felix has done, what is

763
00:28:56,170 --> 00:28:58,730
new, and what could, could be done and

764
00:28:58,730 --> 00:29:00,730
what might be done. And then I think

765
00:29:00,730 --> 00:29:03,644
the final question is what research directions

766
00:29:03,945 --> 00:29:05,005
are interesting

767
00:29:05,384 --> 00:29:06,205
for humans?

768
00:29:09,305 --> 00:29:10,045
For that,

769
00:29:11,224 --> 00:29:12,285
one could do,

770
00:29:12,904 --> 00:29:14,205
large scale evaluations,

771
00:29:15,384 --> 00:29:16,285
using humans.

772
00:29:16,599 --> 00:29:17,419
We have started,

773
00:29:18,039 --> 00:29:20,359
doing this, and I think there are thousand

774
00:29:20,359 --> 00:29:21,899
different things one can do.

775
00:29:23,159 --> 00:29:24,779
One thing I've seen

776
00:29:25,079 --> 00:29:25,899
in this whole,

777
00:29:27,240 --> 00:29:30,105
this whole range of work is the humans

778
00:29:30,404 --> 00:29:33,144
are exceptionally good in coming up with ideas,

779
00:29:33,284 --> 00:29:35,444
and it's really not clear where those ideas

780
00:29:35,444 --> 00:29:36,184
come from.

781
00:29:36,484 --> 00:29:38,404
I think there's a lot of a lot

782
00:29:38,404 --> 00:29:39,944
of things that we need to understand.

783
00:29:40,325 --> 00:29:42,644
From the human perspective, why are human create

784
00:29:42,644 --> 00:29:45,150
scientists? Where are those ideas coming from? That

785
00:29:45,150 --> 00:29:47,089
we can artificially recreate this.

786
00:29:48,029 --> 00:29:48,690
I see.

787
00:29:48,990 --> 00:29:51,409
And and what about, you know, this specific

788
00:29:51,470 --> 00:29:51,970
project,

789
00:29:52,669 --> 00:29:56,190
that that you, Felix, and, Mario are working

790
00:29:56,190 --> 00:29:56,595
on?

791
00:29:57,394 --> 00:29:59,474
Are you are are you following up this

792
00:29:59,474 --> 00:30:01,894
research? Are are you planning on improving

793
00:30:02,515 --> 00:30:03,734
your machine learning

794
00:30:04,275 --> 00:30:04,775
technique,

795
00:30:06,194 --> 00:30:08,515
or maybe applying it to more data? I

796
00:30:08,515 --> 00:30:10,515
suppose well, you have applied it to more

797
00:30:10,515 --> 00:30:12,779
data. What what what's next for you? Or

798
00:30:12,779 --> 00:30:14,640
are you moving on to something else?

799
00:30:15,580 --> 00:30:17,580
So there are a couple of kind of,

800
00:30:18,380 --> 00:30:20,940
obvious follow-up research directions that,

801
00:30:21,420 --> 00:30:22,559
one could pursue.

802
00:30:23,100 --> 00:30:23,660
There are,

803
00:30:24,955 --> 00:30:27,035
a couple of ideas about how to improve

804
00:30:27,035 --> 00:30:29,515
the embedding that we came up with,

805
00:30:30,394 --> 00:30:31,134
even further.

806
00:30:31,595 --> 00:30:33,695
One of these ideas would be that

807
00:30:34,075 --> 00:30:36,414
the kind of main application of our

808
00:30:37,035 --> 00:30:39,215
embedding was to kind of replace the information

809
00:30:39,275 --> 00:30:41,210
in the knowledge graph as Mavi

810
00:30:41,509 --> 00:30:42,809
mentioned earlier. But

811
00:30:43,109 --> 00:30:44,490
you don't necessarily

812
00:30:45,269 --> 00:30:46,869
need to view it in this kind of

813
00:30:46,869 --> 00:30:49,669
binary setting of either only using the knowledge

814
00:30:49,669 --> 00:30:51,990
graph or or embedding. You could also come

815
00:30:51,990 --> 00:30:54,315
up with some kind of combined representation of

816
00:30:54,535 --> 00:30:55,355
both that,

817
00:30:56,134 --> 00:30:58,535
takes kind of as much information as possible

818
00:30:58,535 --> 00:30:59,734
and then just kind of gives it to

819
00:30:59,734 --> 00:31:01,734
the machine learning model and lets it do

820
00:31:01,734 --> 00:31:02,394
its thing,

821
00:31:02,695 --> 00:31:03,195
basically.

822
00:31:04,055 --> 00:31:05,035
The other,

823
00:31:05,734 --> 00:31:07,595
obvious direction would be to,

824
00:31:08,134 --> 00:31:09,119
apply our methods

825
00:31:10,319 --> 00:31:11,619
to other fields.

826
00:31:12,319 --> 00:31:14,819
Like, on the archive alone, there are

827
00:31:15,119 --> 00:31:18,880
more than 15 other primary categories beyond quantum

828
00:31:18,880 --> 00:31:20,720
physics that we could try to use this

829
00:31:20,720 --> 00:31:22,579
method on where the data is basically

830
00:31:23,039 --> 00:31:25,285
publicly available and on the Internet ready to

831
00:31:25,285 --> 00:31:29,224
use. This includes topics like, mathematics, computer science,

832
00:31:29,525 --> 00:31:32,505
or economics or biology where we could also

833
00:31:32,724 --> 00:31:34,884
try our method on and see if we

834
00:31:34,884 --> 00:31:38,910
can find emergent research connections in these fields.

835
00:31:40,029 --> 00:31:41,630
And Mario, what about you? What are you

836
00:31:41,630 --> 00:31:43,250
looking forward to in the future?

837
00:31:44,269 --> 00:31:47,009
Yeah. So beyond things like predicting,

838
00:31:48,349 --> 00:31:50,529
more general properties like cetaceans

839
00:31:50,990 --> 00:31:51,490
and,

840
00:31:52,029 --> 00:31:53,730
maybe surprise and so on,

841
00:31:55,204 --> 00:31:55,704
one

842
00:31:56,164 --> 00:31:59,045
could one could probably make big steps into

843
00:31:59,045 --> 00:32:01,144
improving their AI models themselves

844
00:32:01,765 --> 00:32:03,464
by performing AI competitions.

845
00:32:04,244 --> 00:32:06,265
So I think the data that Felix,

846
00:32:07,285 --> 00:32:10,005
has is very well equipped for that, where

847
00:32:10,005 --> 00:32:11,990
you would have a pool of prize money

848
00:32:11,990 --> 00:32:15,669
and then a hidden dataset, and participants need

849
00:32:15,669 --> 00:32:16,809
to provide

850
00:32:17,509 --> 00:32:18,089
the best

851
00:32:18,470 --> 00:32:21,029
models that they can come up with to

852
00:32:21,029 --> 00:32:21,529
predict

853
00:32:22,875 --> 00:32:25,994
for predict, for instance, the future of, quantum

854
00:32:25,994 --> 00:32:26,494
physics.

855
00:32:27,755 --> 00:32:30,234
And then test in that case, we could

856
00:32:30,234 --> 00:32:30,734
compete,

857
00:32:31,595 --> 00:32:34,154
techniques like knowledge craft, the techniques that we

858
00:32:34,154 --> 00:32:36,095
have published in these papers,

859
00:32:37,670 --> 00:32:39,990
with many, many other techniques that other people

860
00:32:39,990 --> 00:32:42,549
come up with. And that is, I've seen

861
00:32:42,549 --> 00:32:44,490
this, now one time,

862
00:32:45,109 --> 00:32:47,829
very clearly that this is very useful. There

863
00:32:47,829 --> 00:32:50,410
are many, many diverse techniques that usually

864
00:32:50,789 --> 00:32:51,930
be in such competition,

865
00:32:52,375 --> 00:32:54,794
techniques that you alone would never think about.

866
00:32:55,894 --> 00:32:56,394
It's

867
00:32:57,494 --> 00:32:59,034
significantly more technical,

868
00:33:00,134 --> 00:33:01,194
ideas, significantly

869
00:33:01,654 --> 00:33:03,894
simpler ideas that you would not even believe

870
00:33:03,894 --> 00:33:05,034
that they might work.

871
00:33:05,740 --> 00:33:07,579
So that would be really great. And I

872
00:33:07,579 --> 00:33:08,240
think this

873
00:33:08,700 --> 00:33:10,460
type of data that we,

874
00:33:10,940 --> 00:33:13,579
prepared here would suit very well for such

875
00:33:13,579 --> 00:33:14,400
AI competitions.

876
00:33:15,419 --> 00:33:17,119
I see. And and Mario,

877
00:33:17,579 --> 00:33:19,839
you you mentioned the journal, Machine Learning

878
00:33:20,515 --> 00:33:21,575
Science and Technology.

879
00:33:23,315 --> 00:33:25,714
Can you and you're on the editorial board

880
00:33:25,714 --> 00:33:27,255
of that journal. Can

881
00:33:27,714 --> 00:33:29,154
you just give us a little bit of

882
00:33:29,154 --> 00:33:32,134
a flavor of what sort of physics related

883
00:33:32,674 --> 00:33:34,375
research the journal publishes?

884
00:33:35,690 --> 00:33:37,630
Yes. So this is

885
00:33:38,009 --> 00:33:40,329
probably one of the most important, if not

886
00:33:40,329 --> 00:33:41,150
the most important,

887
00:33:41,769 --> 00:33:42,990
journal in that

888
00:33:43,369 --> 00:33:44,509
field. Because

889
00:33:45,049 --> 00:33:46,430
machine learning research,

890
00:33:46,890 --> 00:33:49,789
getting it published in physics journals,

891
00:33:50,204 --> 00:33:50,944
can be

892
00:33:51,325 --> 00:33:52,944
quite challenging because,

893
00:33:53,644 --> 00:33:56,684
physicists are looking at different things that we

894
00:33:56,684 --> 00:33:58,704
might look at. Also publishing

895
00:33:59,484 --> 00:33:59,984
those,

896
00:34:01,005 --> 00:34:01,505
techniques

897
00:34:01,884 --> 00:34:02,865
in AI,

898
00:34:05,799 --> 00:34:08,140
conferences might also be very challenging because

899
00:34:08,920 --> 00:34:11,239
those people also look at very different things.

900
00:34:11,239 --> 00:34:13,719
So there was before this channel, there was

901
00:34:13,719 --> 00:34:15,719
really a gap where it's not clear how

902
00:34:15,719 --> 00:34:17,954
would you even publish such works. And now

903
00:34:17,954 --> 00:34:18,595
I think,

904
00:34:19,315 --> 00:34:20,454
MLST became

905
00:34:20,914 --> 00:34:21,414
the

906
00:34:21,875 --> 00:34:22,775
the two go

907
00:34:23,155 --> 00:34:23,655
address

908
00:34:23,954 --> 00:34:24,855
for exactly,

909
00:34:25,315 --> 00:34:26,135
such works.

910
00:34:26,675 --> 00:34:29,555
And then a lot of works on how

911
00:34:29,555 --> 00:34:30,179
you would,

912
00:34:31,059 --> 00:34:31,960
use new

913
00:34:32,260 --> 00:34:34,280
ideas for machine learning in

914
00:34:34,739 --> 00:34:36,839
physics and chemistry and biology.

915
00:34:37,300 --> 00:34:39,619
Our work is a little bit, outlier because

916
00:34:39,619 --> 00:34:41,699
it even goes a step further to kind

917
00:34:41,699 --> 00:34:43,844
of meet the science, but you see that

918
00:34:44,005 --> 00:34:46,344
MLC is, quite open in,

919
00:34:46,804 --> 00:34:47,304
disrespect.

920
00:34:48,085 --> 00:34:48,585
So,

921
00:34:49,045 --> 00:34:51,364
that's why I'm super happy that the channel

922
00:34:51,364 --> 00:34:53,844
exists and also super happy that they invited

923
00:34:53,844 --> 00:34:56,984
me to join the, editorial board.

924
00:34:57,969 --> 00:35:00,849
Well, that's great. Thanks. Thanks, Felix, and Mario

925
00:35:00,849 --> 00:35:02,710
as well. Thanks for coming on the podcast

926
00:35:02,769 --> 00:35:04,769
and talking about your research. It's,

927
00:35:05,250 --> 00:35:06,070
it's a fascinating

928
00:35:06,369 --> 00:35:08,469
application of, machine learning,

929
00:35:09,170 --> 00:35:10,630
and artificial intelligence.

930
00:35:10,930 --> 00:35:14,054
And, yeah, we hope to, to learn more

931
00:35:14,054 --> 00:35:16,054
about it from you and your colleagues. Thank

932
00:35:16,054 --> 00:35:16,554
you.

933
00:35:17,094 --> 00:35:19,355
Thank you very much. Thank you. Bye.

934
00:35:27,730 --> 00:35:30,930
That was Felix Frohnert of the University of

935
00:35:30,930 --> 00:35:34,469
Leiden and Mario Krenn of the Max Planck

936
00:35:34,530 --> 00:35:36,710
Institute for the Science of Light.

937
00:35:37,170 --> 00:35:39,510
Thanks to both of them for a fascinating

938
00:35:39,730 --> 00:35:40,230
discussion.

939
00:35:41,195 --> 00:35:44,155
You can find their open access paper in

940
00:35:44,155 --> 00:35:44,815
the journal,

941
00:35:45,114 --> 00:35:46,175
Machine Learning,

942
00:35:46,554 --> 00:35:47,775
Science and Technology,

943
00:35:48,394 --> 00:35:51,135
which can be found on the IOP Science

944
00:35:51,195 --> 00:35:51,695
website.

945
00:35:52,394 --> 00:35:53,775
Just look for the title,

946
00:35:54,279 --> 00:35:54,779
Discovering

947
00:35:55,239 --> 00:35:56,299
Emergent Connections

948
00:35:56,759 --> 00:35:58,859
in Quantum Physics Research

949
00:35:59,239 --> 00:36:01,739
via Dynamic Word Embeddings.

950
00:36:02,679 --> 00:36:04,519
I'm afraid that's all the time we have

951
00:36:04,519 --> 00:36:05,819
for this week's podcast.

952
00:36:06,454 --> 00:36:08,375
I'll sign off with a thanks to our

953
00:36:08,375 --> 00:36:12,235
producer, Fred Ailes, and an invitation for you

954
00:36:12,375 --> 00:36:13,994
to join us next week.