214 - Thermal Imagers with Martin Veit

WEBVTT

00:00:00.901 --> 00:00:02.782
<v Wojciech Wegrzynski>Hello everybody, welcome to the FireScience show.

00:00:02.782 --> 00:00:10.935
<v Wojciech Wegrzynski>In today's episode we will be talking about thermal cameras, and that's probably one of my most favorite devices that exist in the world.

00:00:10.935 --> 00:00:26.942
<v Wojciech Wegrzynski>I would claim to say that if you have a fire safety engineering friend and you'd like to give them an amazing gift that they will enjoy a lot and the budget is on a little higher end and thermal camera is your way to go there.

00:00:26.942 --> 00:00:42.652
<v Wojciech Wegrzynski>It's just fascinating to look through the world through the lens of thermal imaging, especially as soon as you start applying that into anything fire related and you see those harsh gradients between cold and hot surfaces, how stuff quickly cools or quickly heats up.

00:00:42.652 --> 00:00:47.912
<v Wojciech Wegrzynski>It really gives another dimension to the eye of a fire safety engineer.

00:00:47.912 --> 00:00:51.890
<v Wojciech Wegrzynski>But this episode is not just about fascination on thermal imaging.

00:00:51.890 --> 00:00:57.872
<v Wojciech Wegrzynski>It's more about its practical side and the practical side in the life and safety.

00:00:57.872 --> 00:01:17.992
<v Wojciech Wegrzynski>Thermal cameras are today a vital piece of equipment used by firefighters in the world to look further into the realm of fire when fighting fires, and we all understand how big advantage they can give to firefighters but also that a faulty thermal camera can be potentially life-threatening.

00:01:17.992 --> 00:01:40.036
<v Wojciech Wegrzynski>So in this episode I've invited Martin Veit from Zag Frisbee, and Martin has just finished his report for the Fire Protection Research Foundation in which he was looking into different metrics of quality for thermal cameras thermal cameras that are actually being used by firefighters in the United States.

00:01:40.036 --> 00:01:50.509
<v Wojciech Wegrzynski>In this episode we will dive a little bit deeper into the mathematical part of how to test different images and what does it mean to compare the image quality.

00:01:50.509 --> 00:02:13.492
<v Wojciech Wegrzynski>But before we reach that point, we have a long discussion about the thermal cameras how do they work, why measuring temperature with cameras is actually quite difficult and why just observing the temperature gradient or differences in a fixed location is not that hard, what you can do with them, how do we test them and how can we improve it for the future.

00:02:13.492 --> 00:02:16.606
<v Wojciech Wegrzynski>So let's not prolong this unnecessarily.

00:02:16.606 --> 00:02:19.233
<v Wojciech Wegrzynski>Let's be in danger and jump into the episode.

00:02:23.700 --> 00:02:25.268
<v Wojciech Wegrzynski>Welcome to the Firesize Show.

00:02:25.268 --> 00:02:28.801
<v Wojciech Wegrzynski>My name is Wojciech Wigrzyński and I will be your host.

00:02:28.801 --> 00:02:58.240
<v Wojciech Wegrzynski>The FireSense Show is into its third year of continued support from its sponsor, ofar Consultants, who are an independent, multi-award winning fire engineering consultancy with a reputation for delivering innovative, safety-driven solutions.

00:02:58.240 --> 00:03:12.044
<v Wojciech Wegrzynski>As the UK-leading independent fire risk consultancy, ofar's globally established team have developed a reputation for preeminent fire engineering expertise, with colleagues working across the world to help protect people, property and the plant.

00:03:12.044 --> 00:03:18.584
<v Wojciech Wegrzynski>Established in the UK in 2016 as a startup business by two highly experienced fire engineering consultants.

00:03:18.584 --> 00:03:28.149
<v Wojciech Wegrzynski>The business continues to grow at a phenomenal rate, with offices across the country in eight locations, from Edinburgh to Bath, and plans for future expansions.

00:03:28.149 --> 00:03:36.391
<v Wojciech Wegrzynski>If you're keen to find out more or join OFR Consultants during this exciting period of growth, visit their website at ofrconsultantscom.

00:03:37.681 --> 00:03:38.824
<v Wojciech Wegrzynski>And now back to the episode.

00:03:38.824 --> 00:03:46.271
<v Wojciech Wegrzynski>Hello everybody, I am joined today by Veit from from Frisbee at ZAG in Slovenia.

00:03:46.271 --> 00:03:48.085
<v Wojciech Wegrzynski>Hey, martin, good to have you in the podcast.

00:03:48.085 --> 00:03:51.406
<v Wojciech Wegrzynski>Hello, thanks for having me and thanks for joining me.

00:03:51.406 --> 00:04:07.765
<v Wojciech Wegrzynski>We have an interesting topic here to discuss and that is your very recent report for the Fire Protection Research Foundation at the NFPA, which is on measuring thermal image quality for fire service applications.

00:04:07.765 --> 00:04:28.791
<v Wojciech Wegrzynski>I have not had a thermal imaging episode yet, but I think it is quite an interesting topic that actually connects engineers, firefighters, practitioners Come on, everyone likes thermal cameras in fire safety, so perhaps let's start with your background and how you ended up in this project, right?

00:04:29.922 --> 00:04:40.971
<v Martin Veit>So for my background, I'm actually not a fire safety engineer, I'm a civil engineer and then I just stumbled into the field and I guess the first entry into the field was really when I was, I guess, cold called.

00:04:40.971 --> 00:04:53.851
<v Martin Veit>So a guy from my university, when he went into industry, recommended me to a partner in a company called Vibraxenernen, kenneth Håkvar Jensen, and then he asked if I wanted to work in the company as a student helper.

00:04:53.851 --> 00:05:08.324
<v Martin Veit>So I worked there for half a year, which was my first entry really into fire protection engineering, and I worked on multiple different things, so fire strategy, but also I worked a bit on documentation and programming and I was supposed to stay.

00:05:08.324 --> 00:05:11.516
<v Martin Veit>But then I was offered a PhD.

00:05:11.516 --> 00:05:16.942
<v Martin Veit>That didn't start immediately at Aalbe University, but I was supposed to start on that one Eventually.

00:05:16.942 --> 00:05:19.846
<v Martin Veit>It took a bit too long for me, it didn't go as planned.

00:05:19.846 --> 00:05:24.843
<v Martin Veit>But then Grunde Jomaas, my current supervisor, reached out to me on LinkedIn.

00:05:24.843 --> 00:05:27.086
<v Martin Veit>But then Grunde Jomaas, my current supervisor, reached out to me on LinkedIn.

00:05:27.086 --> 00:05:28.786
<v Martin Veit>I guess I liked LinkedIn posts.

00:05:28.927 --> 00:05:32.410
<v Martin Veit>And then he wrote to me in Danish, I believe on LinkedIn.

00:05:32.410 --> 00:05:35.372
<v Martin Veit>So I was very surprised and offered me a PhD in Slovenia.

00:05:35.372 --> 00:05:40.237
<v Martin Veit>So I thought let's go back to fire protection engineering and then take an adventure.

00:05:42.160 --> 00:05:42.800
<v Wojciech Wegrzynski>Go to Slovenia see what brings.

00:05:42.800 --> 00:05:46.146
<v Wojciech Wegrzynski>So there are actually positive things from Grundy's LinkedIn activity.

00:05:46.146 --> 00:05:46.827
<v Wojciech Wegrzynski>Good to know.

00:05:47.149 --> 00:05:49.994
<v Martin Veit>Absolutely Exactly.

00:05:50.173 --> 00:05:50.375
<v Wojciech Wegrzynski>Yeah.

00:05:50.375 --> 00:05:53.247
<v Wojciech Wegrzynski>And how did you end up studying thermal cameras?

00:05:53.829 --> 00:05:54.089
<v Martin Veit>Right.

00:05:54.089 --> 00:05:56.949
<v Martin Veit>So I didn't have a background in fire protection engineering.

00:05:56.949 --> 00:06:04.134
<v Martin Veit>So more or less the first half a year to a year was me figuring out what am I going to do in the PhD.

00:06:04.134 --> 00:06:10.867
<v Martin Veit>And I think I ended up after a meeting with Andrea Licardini, another one of my supervisors.

00:06:10.867 --> 00:06:27.966
<v Martin Veit>We had a meeting and then at some point we discussed okay, maybe it could be interesting to look into gas phase measurements, and so I started reading a lot, found some papers, and then I found a paper that used thermal imaging cameras to then characterize the flow field after pool fire, and so I looked into that a bit.

00:06:28.266 --> 00:06:36.552
<v Martin Veit>But then I stumbled into the fact that thermal imaging cameras normally have pretty low resolution, both spatially and also the frame rate.

00:06:36.552 --> 00:06:41.312
<v Martin Veit>So I thought, okay, how can we accommodate this problem, this issue, this?

00:06:41.312 --> 00:06:59.526
<v Martin Veit>So I looked into how to improve resolution of videos, so both the spatial and temporal resolution, and I recently presented a paper on that in Greece, using some machine learning to then enhance video footage, specifically of thermal videos.

00:06:59.526 --> 00:07:18.651
<v Martin Veit>And then at some point during this whole process, I think he received an email about something called student project initiatives from the FPRF, so the Fire Protection Research Foundation, and we looked into the projects that they had and one of them was specifically on measuring the thermal image quality.

00:07:18.651 --> 00:07:28.274
<v Martin Veit>So I think from the beginning to the end, from my background and then ending up to where I am now, has been a big coincidence, more or less, into this project also.

00:07:28.860 --> 00:07:30.447
<v Wojciech Wegrzynski>Welcome to World of Fire Science.

00:07:30.447 --> 00:07:32.362
<v Wojciech Wegrzynski>I guess it's a common story.

00:07:32.362 --> 00:07:43.649
<v Wojciech Wegrzynski>I guess nine out of ten people, I think so We'll have a similar life story about how they ended up in the current location where they are working or the topics that they are dealing with.

00:07:43.649 --> 00:07:45.406
<v Wojciech Wegrzynski>Good, so a nice coincidence.

00:07:45.406 --> 00:07:54.779
<v Wojciech Wegrzynski>I remember talking with you some time ago I think it was the conference in Slovenia about your ideas on measuring gas flow fields and using observation.

00:07:55.081 --> 00:08:02.334
<v Wojciech Wegrzynski>It's kind of a holy grail in fire science to be able to incorporate more of optical measurements.

00:08:02.334 --> 00:08:21.286
<v Wojciech Wegrzynski>They are fast, they are clean, they are easy Air quotes were shown while saying easy, because they're not very easy, but they look easy and it is my strong belief that you can get much more from recording fires and observing fires than you think you could get.

00:08:21.286 --> 00:08:31.509
<v Wojciech Wegrzynski>But the technology that you're investigating here is not specifically meant to support researchers in their experimental endeavors.

00:08:31.509 --> 00:08:35.049
<v Wojciech Wegrzynski>It's a technology used every day by firefighters, I presume.

00:08:35.049 --> 00:08:44.687
<v Wojciech Wegrzynski>Can you tell me more about the types of cameras and the technology and how it is currently overwhelmingly used in the firefighting profession?

00:08:45.129 --> 00:08:46.562
<v Martin Veit>Yes, absolutely so.

00:08:46.562 --> 00:08:48.899
<v Martin Veit>Firefighters use this technology.

00:08:48.899 --> 00:09:01.653
<v Martin Veit>So the thermal imaging camera or thermal imager, also called a TIC as a short name, and they use this for a lot of different of their duties when doing structural firefighting.

00:09:01.653 --> 00:09:17.110
<v Martin Veit>So going into a building and then trying to navigate the building because you might have a building with a fire, you have a lot of smoke can be very difficult to see with your eyes, and then trying to navigate the building, much less identify a person lying on the floor if you can't see anything.

00:09:17.110 --> 00:09:28.249
<v Martin Veit>So there was this nationwide study in the US I believe, where they looked into the efficacy of thermal imaging cameras and the way firefighters use them.

00:09:28.249 --> 00:09:40.754
<v Martin Veit>So what they found was that if a firefighter goes into a building with a lot of smoke, they I think it was 60% of the times they couldn't identify a person without the thermal imager.

00:09:40.754 --> 00:09:44.390
<v Martin Veit>But with the thermal imager they could identify the person 99% of the time.

00:09:44.390 --> 00:09:49.551
<v Martin Veit>Also, it reduces a lot of the time it takes to locate a person.

00:09:49.551 --> 00:09:51.120
<v Martin Veit>Also, navigate your way out of the building.

00:09:51.764 --> 00:09:55.599
<v Wojciech Wegrzynski>So is it something that you could call, now, a standard piece of equipment?

00:09:55.599 --> 00:09:57.583
<v Wojciech Wegrzynski>I wonder how it is around the world.

00:09:57.583 --> 00:10:03.083
<v Wojciech Wegrzynski>I think in Poland the fire brigades are very strongly equipped with thermal imaging cameras.

00:10:03.445 --> 00:10:03.664
<v Martin Veit>Right.

00:10:03.664 --> 00:10:04.908
<v Martin Veit>So that is also my belief.

00:10:04.908 --> 00:10:23.250
<v Martin Veit>So maybe in the 2000s, I think, when they started to route them out to firefighters and then before that, not so much because they were big, they were bulky, but then, when the technology then advanced, you can have a smaller thermal imaging camera, it's easier to use for the firefighter, and so on.

00:10:23.250 --> 00:10:43.668
<v Martin Veit>So for now at least, also in the US, which was the main focus of the project, because it was an American project, it seems to be very, very common in the fire service to use this thermal imaging, and also I talked to some people in Europe as well which, as you also say, very common to use because it's such an effective equipment.

00:10:44.630 --> 00:10:52.471
<v Wojciech Wegrzynski>I wonder when it's going to be equipped as a part of, you know, the helmet and the body kit itself.

00:10:52.471 --> 00:11:07.100
<v Wojciech Wegrzynski>I can imagine this kind of device to be integrated with the helmet and the visor in the helmet to actually present some augmented reality, thermal imaging, over what the firefighter directly sees, perhaps with some clever eye tracking.

00:11:07.100 --> 00:11:14.254
<v Wojciech Wegrzynski>I think every single component of technology that would be necessary to do such a thing already exists.

00:11:14.254 --> 00:11:18.652
<v Wojciech Wegrzynski>It's just about you know, scale and being able to deliver that.

00:11:18.652 --> 00:11:25.269
<v Wojciech Wegrzynski>So I'm absolutely convinced this is an element of infrastructure that will be a part of the future.

00:11:25.750 --> 00:11:33.581
<v Wojciech Wegrzynski>And I remember when I rejoined the ITB 15 years ago, we had this massive, bulky thermal camera.

00:11:33.581 --> 00:11:35.826
<v Wojciech Wegrzynski>It was like a beamer size, you know.

00:11:35.826 --> 00:11:47.241
<v Wojciech Wegrzynski>You had to have a whole bag with that, and now a thermal camera that's significantly, significantly better in technology is of the size of my, of my iphone.

00:11:47.241 --> 00:11:47.823
<v Wojciech Wegrzynski>That that's it.

00:11:47.823 --> 00:11:48.706
<v Wojciech Wegrzynski>That's that.

00:11:48.706 --> 00:11:51.596
<v Wojciech Wegrzynski>That's the progress in this technology you mentioned.

00:11:51.596 --> 00:11:53.484
<v Wojciech Wegrzynski>They have low resolution, low frame rate.

00:11:53.484 --> 00:11:58.964
<v Wojciech Wegrzynski>So how does that compare to devices that people are used to like cameras in their phones, if?

00:11:58.984 --> 00:12:05.931
<v Martin Veit>you have a standard iphone, not necessarily the newest one, but you'll have full hd images right, so you can have very high resolution.

00:12:05.931 --> 00:12:12.346
<v Martin Veit>If you take an image, you can easily discern details, you see a lot of colors, it's very clear, very visible.

00:12:12.346 --> 00:12:17.628
<v Martin Veit>But then for the thermal imaging camera it has a resolution much lower than that.

00:12:17.628 --> 00:12:33.373
<v Martin Veit>If you have like a, if you're going to have very expensive cameras that will have a high resolution, also high frame rate, but the ones that are more affordable, also to the fire service, will typically have a resolution of 240 times 320.

00:12:33.754 --> 00:12:42.380
<v Wojciech Wegrzynski>That's like the common one I think my first phone had a camera of 240 times 320 and it's a very low resolution actually.

00:12:42.380 --> 00:12:44.504
<v Wojciech Wegrzynski>Uh, what about frame rates?

00:12:44.504 --> 00:12:48.067
<v Wojciech Wegrzynski>Are we talking about like single image per second?

00:12:48.067 --> 00:12:50.370
<v Wojciech Wegrzynski>Are we talking something like a movie?

00:12:50.370 --> 00:12:51.552
<v Wojciech Wegrzynski>24, 30 hertz?

00:12:51.552 --> 00:12:53.835
<v Wojciech Wegrzynski>How do they operate in frame rate?

00:12:56.820 --> 00:12:57.163
<v Martin Veit>To a large degree.

00:12:57.163 --> 00:13:09.139
<v Martin Veit>It's actually pretty good for what you need to do if you want to operate a thermal imaging camera and then go into a building because you don't want it to lag too much or you have too much time in between individual frames because that might make it very difficult to navigate.

00:13:09.139 --> 00:13:23.827
<v Martin Veit>So the ones that are nfpa approved so the ones that are very common in the us, typically have a frame rate of 30 hertz or even as high as 60 hertz, which is pretty good for a thermal imaging camera and an fpa approved.

00:13:23.888 --> 00:13:25.231
<v Wojciech Wegrzynski>Is there a standard they refer to?

00:13:25.231 --> 00:13:28.009
<v Wojciech Wegrzynski>Is there something that defines what they should do?

00:13:28.009 --> 00:13:30.596
<v Wojciech Wegrzynski>Perhaps that's something we should refer the listeners to.

00:13:31.078 --> 00:13:50.889
<v Martin Veit>Right, so the standards that goes into certifying these thermal imaging cameras in the US, so the NFPA standard, that's the NFPA-1801, which is currently being consolidated into 1930, I believe it is which also collects a few other standards into one big standard.

00:13:50.889 --> 00:13:59.405
<v Martin Veit>But that's the standard that goes into testing and what this thermal imaging camera should be able to withstand.

00:13:59.446 --> 00:14:10.434
<v Wojciech Wegrzynski>And also to do and in terms of what they see, I know also in my thermal cameras there's this annoying parameter which is the temperature range which I have to set.

00:14:10.434 --> 00:14:22.155
<v Wojciech Wegrzynski>So those devices used by firefighters, do they operate in like whole spectrum of fire temperatures up to, like I don't know, 1200s, or they are narrowed down to a few hundred degrees?

00:14:22.155 --> 00:14:26.350
<v Wojciech Wegrzynski>What's the target temperature range on those devices?

00:14:26.940 --> 00:14:29.188
<v Martin Veit>So actually, the different temperature ranges.

00:14:29.188 --> 00:14:34.285
<v Martin Veit>So it sort of depends from camera to camera and the NFPA-approved ones.

00:14:34.285 --> 00:14:38.466
<v Martin Veit>But I believe there must be like a minimum range in the NFPA standard.

00:14:38.466 --> 00:14:49.701
<v Martin Veit>For example, some of them go from minus 20 up to 550, and then some goes from minus 40 up to 550, and then some goes from minus 40 up to 1100 degrees Celsius, so it can range.

00:14:49.701 --> 00:14:54.212
<v Martin Veit>There's a big range that they can visualize and observe.

00:14:54.860 --> 00:14:57.609
<v Wojciech Wegrzynski>But it's not like they cap at 100 degrees, right.

00:14:57.870 --> 00:14:58.149
<v Martin Veit>No, no, no.

00:14:59.240 --> 00:15:00.724
<v Wojciech Wegrzynski>And also to get utility.

00:15:00.724 --> 00:15:04.972
<v Wojciech Wegrzynski>You don't need it to be beyond the fire range, right, no right?

00:15:05.514 --> 00:15:07.979
<v Martin Veit>to get utility, you don't need it to be beyond the fire range, right?

00:15:07.979 --> 00:15:11.687
<v Martin Veit>No, right, I mean, you just need it to be able to operate within the expected conditions that you want to go into.

00:15:12.240 --> 00:15:16.692
<v Wojciech Wegrzynski>And, if you think about it, I guess the expected condition is not necessarily a fully flashed over fire.

00:15:16.692 --> 00:15:19.909
<v Wojciech Wegrzynski>Like what kind of information does a firefighter get from a camera?

00:15:19.909 --> 00:15:26.634
<v Wojciech Wegrzynski>Right, it's like if you have a fully flashed over fire in front of your face, you don't need to have a thermal camera to confirm that Exactly.

00:15:26.634 --> 00:15:32.873
<v Wojciech Wegrzynski>Okay, let's move on into how thermal cameras work.

00:15:32.873 --> 00:15:36.899
<v Wojciech Wegrzynski>So perhaps there's a bit of interesting technical knowledge.

00:15:36.899 --> 00:15:52.047
<v Wojciech Wegrzynski>So if you could briefly explain to me why my camera in a phone does not capture the temperatures and why this specific piece of equipment does, so we can go into some of the basics.

00:15:52.159 --> 00:15:54.426
<v Martin Veit>I think if you want to go into the diesel and stuff.

00:15:54.446 --> 00:15:56.230
<v Martin Veit>It might be like a full episode in itself.

00:15:56.230 --> 00:16:05.379
<v Martin Veit>But I guess the idea is that you have different spectrums, electromagnetic spectrum, different regions that you want to observe.

00:16:05.379 --> 00:16:15.071
<v Martin Veit>So you have the visible spectrum, which is 380 up to 780 nanometers, but then the infrared is also partitioned into different regions.

00:16:15.071 --> 00:16:25.254
<v Martin Veit>But for long wave infrared cameras the range is 7 microns or 7 micrometers up to 14 approximately.

00:16:25.254 --> 00:16:27.908
<v Martin Veit>So it's a very different range that they observe.

00:16:27.908 --> 00:16:33.769
<v Martin Veit>So we have larger wavelengths, so the individual detectors also needs to be bigger.

00:16:33.769 --> 00:16:40.607
<v Martin Veit>So if you have a standard digital camera, for example, it has a much higher resolution because also it's a different technology.

00:16:40.607 --> 00:16:58.986
<v Martin Veit>So there's also been a lot of research in that specific field, compared to thermal cameras, which is normally used by the military, for example, if you want to go into building inspections, fire service but there's a lot of more utility necessarily for this common person with a camera.

00:16:58.986 --> 00:17:09.627
<v Martin Veit>So the main thing is also the technology used, which is why you lower frame rates, lower resolution, but that's due to some of the physics involved.

00:17:11.102 --> 00:17:13.750
<v Wojciech Wegrzynski>How can the thermal camera distinguish between temperatures?

00:17:13.750 --> 00:17:14.151
<v Wojciech Wegrzynski>Actually?

00:17:14.151 --> 00:17:17.328
<v Wojciech Wegrzynski>How does it know a thing is at a low temperature?

00:17:17.328 --> 00:17:19.579
<v Wojciech Wegrzynski>How does it know that the thing is at a high temperature?

00:17:19.941 --> 00:17:52.111
<v Martin Veit>So thermal imaging cameras works on the principle that if you have any objects in front of you, it will emit a certain amount of radiation Right, and then if you have an object that's of higher temperature, it's going to emit more radiation, which is then detected by this thermal imaging camera, meaning that you can discern two different objects of two different temperatures based on the one based, because they emit different amounts of radiation that is then detected by the detector in the thermal imaging camera.

00:17:52.720 --> 00:17:59.193
<v Wojciech Wegrzynski>Okay, I can understand how we can observe walls and solid objects with thermal cameras.

00:17:59.193 --> 00:18:03.921
<v Wojciech Wegrzynski>What about gases like in observing flames and smoke, etc.

00:18:03.921 --> 00:18:06.141
<v Wojciech Wegrzynski>Because I know it's not a trivial thing, about gases like in observing flames and smoke, etc.

00:18:06.141 --> 00:18:08.162
<v Wojciech Wegrzynski>Because I know it's not a trivial thing.

00:18:08.162 --> 00:18:10.701
<v Wojciech Wegrzynski>So how much smoke do you see on a thermal camera?

00:18:10.701 --> 00:18:12.522
<v Wojciech Wegrzynski>How much flame do you see in a thermal camera?

00:18:12.522 --> 00:18:13.462
<v Wojciech Wegrzynski>What do you exactly see?

00:18:14.042 --> 00:18:15.663
<v Martin Veit>You have to be right, it's not trivial.

00:18:15.663 --> 00:18:23.664
<v Martin Veit>It's a lot easier to detect a wall, an object, something, especially if you want to have accurate gas phase temperatures off a flame.

00:18:23.664 --> 00:18:24.826
<v Martin Veit>Very difficult to do.

00:18:24.826 --> 00:18:57.526
<v Martin Veit>But I guess the main principle and why this thermal imaging camera is of utility to a firefighter is because that they operate in the range of 7 to 14 microns, because if you look at some of the combustion gases that is present during a fire, you know If you look at the absorbance, they have a large range where they have very low or no absorbance in that specific range, meaning that they're not going to obscure the image that you're trying to see.

00:18:57.609 --> 00:19:01.781
<v Martin Veit>So you can see true smoke because of the gases' absorbance.

00:19:01.781 --> 00:19:08.303
<v Martin Veit>So, for example, I used a thermal camera, for example, to make some videos of a small pool fire.

00:19:08.303 --> 00:19:13.833
<v Martin Veit>I can see that because it still has radiation, blackbody radiation, so I can still see the flame.

00:19:13.833 --> 00:19:40.803
<v Martin Veit>But if I want to visualize the combustion gases and it's not necessarily super, they don't have a lot of absorbance in the specific region or the spectrum of the longwave infrared camera then I can go down and use a midwave infrared camera so I can visualize the combustion gases more clearly, meaning that I can actually see it fairly clearly, even though it's not necessarily super visible to my eye.

00:19:40.803 --> 00:19:50.751
<v Martin Veit>So also flames, for example, I used a midwave infrared camera to then make videos of a pool fire and then you can also see.

00:19:50.751 --> 00:20:03.699
<v Martin Veit>So it was a visible flame, so it was a heptane, so it also has some soot, but then you can also very clearly see the flame and different flame structures and then also some of the combustion gases even above the flame.

00:20:04.299 --> 00:20:05.282
<v Wojciech Wegrzynski>Can you define mid-wave?

00:20:05.282 --> 00:20:06.693
<v Wojciech Wegrzynski>What spectrum would that be?

00:20:06.693 --> 00:20:08.377
<v Wojciech Wegrzynski>Is it below seven?

00:20:09.059 --> 00:20:12.799
<v Martin Veit>It is, so I guess it also depends on who you ask.

00:20:12.799 --> 00:20:15.239
<v Martin Veit>I've seen different things one to five or three to five.

00:20:15.239 --> 00:20:19.301
<v Martin Veit>I think the one that I would choose is probably three to five microns.

00:20:19.990 --> 00:20:35.981
<v Wojciech Wegrzynski>So in essence, the ones that would be used by the fire service let's say seven to 14 default range, basically the stuff that the fire produces in abundance CO, co2, is transparent to those radiations.

00:20:35.981 --> 00:20:48.499
<v Wojciech Wegrzynski>Therefore you can see better through them because you're not obscured by a large amount of radiation produced by those hot gases and at the same time it makes it difficult to observe those structures because basically they're transparent as well.

00:20:48.499 --> 00:20:53.762
<v Wojciech Wegrzynski>So it's kind of a blessing and an issue depends on on on how you look on the problem.

00:20:53.762 --> 00:21:00.182
<v Wojciech Wegrzynski>I I had an interesting episode on on observing through flames with matt hayler from from this.

00:21:00.182 --> 00:21:16.757
<v Wojciech Wegrzynski>So yeah, we've been talking about blue light technology, so they've also found a way to to go very near uv radiation, where UV radiation, where the flame is almost transparent to that particular blue light wavelength and you can see through flames and, yes, it actually works like that.

00:21:16.757 --> 00:21:22.423
<v Wojciech Wegrzynski>So, in essence, a very similar principle in here.

00:21:23.349 --> 00:21:25.837
<v Wojciech Wegrzynski>So how much you need to know to?

00:21:25.837 --> 00:21:29.820
<v Wojciech Wegrzynski>So let's imagine you're taking a picture with a thermal camera.

00:21:29.820 --> 00:21:31.856
<v Wojciech Wegrzynski>It gives you some output.

00:21:31.856 --> 00:21:33.276
<v Wojciech Wegrzynski>It's a simple device.

00:21:33.276 --> 00:21:37.361
<v Wojciech Wegrzynski>It looks on photons, accumulates them, gives you an outcome.

00:21:37.361 --> 00:21:39.758
<v Wojciech Wegrzynski>But what exactly?

00:21:39.758 --> 00:21:49.075
<v Wojciech Wegrzynski>You have to know about the object you're observing to understand the values that you are seeing on your screen, for example, the emissivity of the surface that you are seeing on your screen.

00:21:49.075 --> 00:21:51.853
<v Wojciech Wegrzynski>For example, the emissivity of the surface that you're observing with thermal camera.

00:21:51.853 --> 00:21:56.653
<v Wojciech Wegrzynski>Do you need to know the emissivity, the reflectance of the surface that you're measuring?

00:21:57.315 --> 00:22:04.780
<v Martin Veit>As you mentioned, there's a few different things that you need to know, but I guess this also depends on the purpose of what you want to do.

00:22:04.780 --> 00:22:07.413
<v Martin Veit>Do you want to have accurate gas-based measurements?

00:22:07.413 --> 00:22:12.455
<v Martin Veit>Do you want to have accurate measurements of the temperature of some solid object, which might be very valuable?

00:22:12.455 --> 00:22:27.255
<v Martin Veit>But if you don't need that, if you just want to, let's say, identify hotspots, identify heat losses of a building, for example, which is also a common use of thermal cameras, you can do that without getting all of the different specifications correct.

00:22:27.255 --> 00:22:33.298
<v Martin Veit>So, for example, the emissivity, distance to the object and so on, and also ambient temperature conditions.

00:22:33.298 --> 00:22:45.277
<v Martin Veit>If you need to have accurate measurements of some phenomena, let's say gas phase or sonded phase, then you need a bit more information on what to do correctly, for example the emissivity.

00:22:45.277 --> 00:22:54.698
<v Martin Veit>But as far as I know, the getting accurate gas phase measurements of a flame and then fully resolve it is very difficult.

00:22:54.698 --> 00:22:56.577
<v Martin Veit>It's not so straightforward.

00:22:57.059 --> 00:23:03.994
<v Wojciech Wegrzynski>Well, technically, if you take a camera and point it towards the flame, it will tell you like 600 degrees.

00:23:03.994 --> 00:23:08.776
<v Wojciech Wegrzynski>It's not a flame temperature of any way, so it must be incorrect.

00:23:08.776 --> 00:23:16.996
<v Wojciech Wegrzynski>But then I, a long time before COVID, we were doing some facades within pre-rule and we were observing.

00:23:16.996 --> 00:23:23.061
<v Wojciech Wegrzynski>We had a steel facade, one that you actually the next generation of which is sold in the summer school.

00:23:23.061 --> 00:23:28.836
<v Wojciech Wegrzynski>We had a prototype of that Long, long time ago and we had the steel plate.

00:23:28.930 --> 00:23:34.619
<v Wojciech Wegrzynski>There was a fire in the cavity in the steel plate and basically we were taking pictures of this facade.

00:23:34.619 --> 00:23:53.118
<v Wojciech Wegrzynski>It was giving us a very weird numbers and we had a black spray paint which was meant for chimneys, like literally black, very dark, very matte spray paint, and we were painting on that steel facade and the locations in which we painted it it's very black.

00:23:53.118 --> 00:24:01.038
<v Wojciech Wegrzynski>The temperature measurements in that location were like almost one degree accurate with the thermocouple measurement in the same location.

00:24:01.038 --> 00:24:18.086
<v Wojciech Wegrzynski>So because we, like you, could very well approximate emissivity of a matte black paint, which is very, very high, whereas for the steel surface that is undergoing heating and a lot of crystal formation transitions, it changes colors, it bends a little bit, it reflects less.

00:24:18.086 --> 00:24:18.569
<v Wojciech Wegrzynski>In this location.

00:24:18.569 --> 00:24:20.576
<v Wojciech Wegrzynski>It was a mess.

00:24:20.576 --> 00:24:27.604
<v Wojciech Wegrzynski>So that was a moment when I realized I really need good control over what I'm measuring.

00:24:27.604 --> 00:24:35.678
<v Wojciech Wegrzynski>I think the word measure in here is the key, because if you want to measure, you have to put in the effort to measure.

00:24:36.380 --> 00:24:50.175
<v Martin Veit>If you're just observing, here you go, you're welcome to to see the range I think it's a great distinction to have either measure or observe, because for a lot of different things that you want to do with a thermal imaging camera, observations are just fine, right.

00:24:50.175 --> 00:24:54.356
<v Martin Veit>You don't need to have very accurate temperatures, you just need to discern some difference.

00:24:54.356 --> 00:24:56.184
<v Martin Veit>Maybe it's hot, maybe it's cold.

00:24:56.184 --> 00:24:57.691
<v Martin Veit>If it's heat losses or whatever for building.

00:24:57.691 --> 00:25:07.393
<v Martin Veit>If you want to measure, you have to take a lot more care into what you're doing, control the emissaries and do one of the steps to get accurate temperatures if that's what you want to do.

00:25:07.393 --> 00:25:18.296
<v Martin Veit>But I think both observation and measurement has a place in fire safety science and what we used to do with it, especially if you're trying to do smoke and fire phenomena.

00:25:19.009 --> 00:25:21.098
<v Wojciech Wegrzynski>There is this thing called optical thickness.

00:25:21.098 --> 00:25:26.900
<v Wojciech Wegrzynski>Basically, the emissivity of a smoke layer is not just a physical function of the suit inside.

00:25:26.900 --> 00:25:33.816
<v Wojciech Wegrzynski>It's about how deep the layer is until you reach the optical thickness and then it looks black.

00:25:33.816 --> 00:25:53.256
<v Wojciech Wegrzynski>And if you don't know how thick the layer is, you cannot even tell how much smoke is in the layer, because you don't know if it's like very thin but very optically thick, or it's just a little smoke spread over 10 meters which will look to you in a similar way from the emissivity of a layer standpoint.

00:25:53.256 --> 00:25:56.521
<v Wojciech Wegrzynski>So there's a lot of cave-ins.

00:25:56.722 --> 00:26:13.509
<v Wojciech Wegrzynski>So in the end in fires I don't think the firefighters should trust the number they see on the camera, but I think they are very well advised to trust if something is hot or not in relationship to the surroundings.

00:26:13.509 --> 00:26:21.944
<v Wojciech Wegrzynski>If I had to give a guidance to my firefighting colleagues, who definitely are professionals in using those devices they train.

00:26:21.944 --> 00:26:40.540
<v Wojciech Wegrzynski>But my observation as a researcher using tools like that is that wow, so hard to get an accurate measurement, but it's, it's super easy to just, you know, get an overview absolutely yeah, and I think that's also valuable to know as a firefighter, like what to do with this should I trust the temperature or not?

00:26:40.840 --> 00:26:42.613
<v Martin Veit>because I also heard from a firefighter.

00:26:42.613 --> 00:26:48.452
<v Martin Veit>It was in europe, but he said that they didn't have so much money to get thermal imaging cameras.

00:26:48.452 --> 00:27:06.753
<v Martin Veit>So they had, I think, one good thermal imaging camera, which was for the chief, I believe and then they had cheaper thermal imaging cameras still very useful because, as you say, you can still observe information and get a lot of valuable information even though you don't have the highest resolution or you haven't, you don't have the highest frame rate at all.

00:27:06.753 --> 00:27:08.439
<v Martin Veit>Still are useful too, even though you don't have the highest frame rate at all.

00:27:08.439 --> 00:27:11.750
<v Martin Veit>Still are useful too, even though you don't have the newest of the new or the best of the best.

00:27:12.430 --> 00:27:16.622
<v Wojciech Wegrzynski>Are there big differences between particular cameras?

00:27:16.622 --> 00:27:22.819
<v Wojciech Wegrzynski>How sensitive are they to the radiation of hot objects or gases in fires?

00:27:22.819 --> 00:27:31.640
<v Wojciech Wegrzynski>Anything like any practical differences, or you can qualify all of them as, let's say, useful for fire and that's it.

00:27:31.640 --> 00:27:33.323
<v Wojciech Wegrzynski>No, no big differences among them.

00:27:34.044 --> 00:27:34.284
<v Martin Veit>Right.

00:27:34.284 --> 00:27:46.540
<v Martin Veit>Different cameras of course, made by different manufacturers, will also have differences in quality service.

00:27:46.540 --> 00:27:50.210
<v Martin Veit>The ones that are certified by the NFPA standard.

00:27:50.210 --> 00:28:05.604
<v Martin Veit>They primarily use two different detector technologies, so one called vanadium oxide and one called amorphous silicon, and I think at some point I believe VOX of vanadium oxide was more prevalent on the market.

00:28:05.604 --> 00:28:16.714
<v Martin Veit>But I think amorphous silicon as a detector technology also caught up and is also now part of the majority on the NFPA certified thermal imaging camera list.

00:28:16.714 --> 00:28:26.978
<v Martin Veit>So there's definitely different materials used and they also have different material properties that also make some slight changes for the thermal imaging camera.

00:28:26.978 --> 00:28:33.079
<v Martin Veit>But I think to a large degree thermal imaging cameras that are then certified by the NFPA.

00:28:33.079 --> 00:28:34.663
<v Martin Veit>They have to go through the same test.

00:28:34.663 --> 00:28:41.920
<v Martin Veit>So even though it's a difference, they take the technology.

00:28:41.920 --> 00:28:43.789
<v Martin Veit>It should also still be very much of a utility to the fire service.

00:28:44.250 --> 00:28:50.903
<v Wojciech Wegrzynski>And Hjulje, I guess, get to the point of the research grant and the research question that you were involved in.

00:28:50.903 --> 00:29:12.750
<v Wojciech Wegrzynski>How do you assess the quality of those cameras for fire applications in order to verify that when the new sensor comes to the market or a new product is introduced to the market and wants to get the certification, the certification?

00:29:12.750 --> 00:29:15.336
<v Wojciech Wegrzynski>So let's perhaps start with how those devices would be tested in accordance to NFPA or for the purpose of certification.

00:29:15.336 --> 00:29:20.315
<v Wojciech Wegrzynski>What exactly is being assessed when you certify a thermal camera?

00:29:20.315 --> 00:29:21.176
<v Wojciech Wegrzynski>What you're looking into?

00:29:21.597 --> 00:29:42.000
<v Martin Veit>So currently the NFPA standard that I mentioned before, the 1801, or the new one that it it's been consolidated into it goes through a bunch of tests, for durability, for example, and then the thermal image quality is then assessed with something called the image recognition test and image image recognition test.

00:29:42.000 --> 00:29:56.125
<v Martin Veit>Yes, so this is then based on some work from dr francine amon back in the early 2000s, where they tried to look into different things or different ways to quantify the thermal image quality.

00:29:56.931 --> 00:30:08.015
<v Martin Veit>And they looked into a bunch of different ones, for example effective temperature range, non-uniformity, spatial frequency response and the ones incorporated into the standard.

00:30:08.075 --> 00:30:25.644
<v Martin Veit>Now is, the main test would be this spatial resolution test or the image recognition test, and essentially you have a bit of math that you have to do so they use something called a contrast transfer function to estimate this spatial resolution.

00:30:25.644 --> 00:30:52.875
<v Martin Veit>So for this you need a blackbody target and then the test is actually a bit cumbersome because you have this thermal image that you want to certify and then to assess the quality of the spatial resolution, then you take an image of the display and then extract some region of interest and then do some math to then see okay, how well did it respond to the test.

00:30:52.875 --> 00:31:01.377
<v Martin Veit>So, speaking with the technical committee, what I was told was that, first off, the test is a bit tedious to perform.

00:31:01.377 --> 00:31:16.733
<v Martin Veit>It also has some consistency issues, so if you repeat it you might not get exactly the same value, also because you have this extra element of a camera taking an image off the display, and then also the image quality.

00:31:16.733 --> 00:31:22.964
<v Martin Veit>If you have a human and the test side to side, they might also not agree with the image quality.

00:31:23.349 --> 00:31:30.463
<v Martin Veit>So one might say okay if I said this left picture is better than the right one, the test might say otherwise.

00:31:30.463 --> 00:31:35.019
<v Martin Veit>They could still both pass, but they have some consistency issues at the moment.

00:31:35.019 --> 00:31:50.397
<v Martin Veit>So the whole idea with the project was to look into how can we accommodate or perhaps improve the current framework being performed to assess the thermal image quality, to make it more consistent and also to have it better aligned with what a human might score the image.

00:31:51.292 --> 00:31:57.421
<v Wojciech Wegrzynski>Yeah, but it's not like a relative comparison between the cameras that are certified.

00:31:57.421 --> 00:31:59.617
<v Wojciech Wegrzynski>It's more about meeting a threshold, right?

00:31:59.617 --> 00:32:07.162
<v Wojciech Wegrzynski>Exactly, yeah, Okay, and the thing that camera is shooting do I understand?

00:32:07.162 --> 00:32:10.760
<v Wojciech Wegrzynski>You called it the display, but it must be a hot object, right?

00:32:10.760 --> 00:32:12.978
<v Wojciech Wegrzynski>And it's controlled by the NFPA.

00:32:12.978 --> 00:32:15.999
<v Wojciech Wegrzynski>What pictures of what are you taking during the test?

00:32:16.769 --> 00:32:19.817
<v Martin Veit>So what I meant before was that the thermal imaging camera.

00:32:19.817 --> 00:32:31.096
<v Martin Veit>It takes an image, so this thermal image, of something called a stencil pattern, which is essentially just a target with a specific pattern, with a specific emissivity.

00:32:31.096 --> 00:32:36.676
<v Martin Veit>But the image that you take from the thermal imaging camera is not the one that is being assessed.

00:32:36.676 --> 00:32:47.537
<v Martin Veit>You have an extra step, meaning that you have a camera on the display of the thermal imaging camera that is then taking an image of that display and then from that you extract a region of interest.

00:32:47.537 --> 00:32:49.634
<v Martin Veit>So you can imagine there's also some things with reflections that display and then from that you extract a region of interest.

00:32:49.634 --> 00:32:53.605
<v Martin Veit>So you can imagine there's also some things with reflections that you have to take into account because you have the display.

00:32:53.605 --> 00:32:59.281
<v Martin Veit>You don't want extra reflections that could change the score of the ah, I understand.

00:32:59.342 --> 00:33:09.438
<v Wojciech Wegrzynski>So it's not just assessing the quality of the image taken by the camera, it's about assessing how you can view an object through the camera.

00:33:09.609 --> 00:33:33.003
<v Martin Veit>So it also includes the assessment of the viewfinder, of the screen of the camera, in a way, exactly, and I believe the argument for this was that you have a person also looking at the display, so they want to take it from the very image that you're trying to take all the way up to the display and then treat that as a black box and then trying to assess the quality of this black box that you're trying to take all the way up to the display and then treat that as a black box and then trying to assess the quality of this black box that you have.

00:33:34.411 --> 00:33:36.218
<v Wojciech Wegrzynski>Yeah, that makes sense.

00:33:36.218 --> 00:33:36.832
<v Wojciech Wegrzynski>That makes sense.

00:33:36.832 --> 00:33:57.743
<v Wojciech Wegrzynski>I was, I have not been thinking about this in such a way, but it is like testing a system in end-use conditions, something we need to do more in fire science, which you working at Frisbee, you're very, very well aware, because that's something that me and Grunder are battling for all the time.

00:33:57.743 --> 00:34:00.569
<v Wojciech Wegrzynski>Good, good, very, very interesting.

00:34:00.569 --> 00:34:19.282
<v Wojciech Wegrzynski>And the assessment is based on this image quality, the one that the test takes, and then apply some mathematical filters to extract some sort of outcomes of that, and then, based on that, it gives you a value, a number that indicates the performance of the device.

00:34:19.282 --> 00:34:20.143
<v Wojciech Wegrzynski>Do I get it correct?

00:34:21.471 --> 00:34:31.418
<v Martin Veit>So you more or less have this image and then you extract some breach of interest, apply some math we don't need to go into that and then in the end what you get is a single score.

00:34:31.418 --> 00:34:35.610
<v Martin Veit>So this spatial response, or spatial- resolution.

00:34:37.295 --> 00:34:49.938
<v Wojciech Wegrzynski>So what's the main challenge with this approach other than it's well, I guess being cumbersome and to some extent inconsistent is something that is very difficult for a testing method, so perhaps let's step over that.

00:34:49.938 --> 00:34:58.983
<v Wojciech Wegrzynski>What is your view on how this can be improved and how we could improve the consistency of this assessment?

00:34:59.784 --> 00:35:02.829
<v Martin Veit>Right, so that gets into the topic of the project models.

00:35:03.291 --> 00:35:12.485
<v Martin Veit>So I guess the main question was what do other people, other industries, currently do in order to assess image quality?

00:35:13.130 --> 00:35:35.063
<v Martin Veit>And then there's a huge field within image processing where you try to assess image quality, which is more or less the field that I then looked at and then saw okay, how can we extract some of this terminology and models and methods that they're currently using this terminology and models and methods that they're currently using and how can we apply that to thermal images and then try to use that for an image quality assessment?

00:35:35.063 --> 00:35:44.880
<v Martin Veit>So I looked into multiple different approaches, but before that, I think we need to get some terminology, some common terminology.

00:35:44.880 --> 00:36:14.865
<v Martin Veit>So when you have image quality assessments and you want to make sure that I want to have an assessment of this image and what I would then want to do is I want to either do something called a full reference image quality assessment, where I have an image that might have some distortions, let's say noise or blur or something and I want to compare that to an image of the same object, but with a very high quality.

00:36:14.865 --> 00:36:17.018
<v Martin Veit>That's full reference image quality assessment.

00:36:17.369 --> 00:36:19.237
<v Wojciech Wegrzynski>So I have an image that I then compare to something else.

00:36:20.090 --> 00:36:28.418
<v Martin Veit>We can't use that for thermal imaging cameras, because if you take an image, you don't necessarily have a reference to then compare with it's more used.

00:36:28.418 --> 00:36:40.199
<v Martin Veit>If you have, uh, for example, different algorithms to improve the resolution, then you want to perform some distortions and then see how can we improve the resolution of this poor image.

00:36:40.199 --> 00:36:41.101
<v Martin Veit>Then you can compare.

00:36:42.170 --> 00:36:52.802
<v Wojciech Wegrzynski>Unless you had a super high quality like order of magnitude quality better thermal camera that you could shoot the super cool reference picture, then you could probably do it.

00:36:52.802 --> 00:36:56.440
<v Wojciech Wegrzynski>But I guess it could be challenging, right.

00:36:57.471 --> 00:37:17.632
<v Martin Veit>So one of the other approaches is something called no reference image quality assessment, meaning that you take an image let's just say for now you take an image from your phone and we want to assess what is the quality of this compared to what a human might score it, so we can use some model that then tries to make a prediction of this.

00:37:17.632 --> 00:37:31.996
<v Martin Veit>So that's the two main approaches, and I looked into both also because in the end of the project I had like sort of like a, a case study where I want to use these full reference metrics to then compare the no reference metrics.

00:37:31.996 --> 00:37:36.152
<v Martin Veit>But let's skip ahead to the no reference image quality assessment.

00:37:36.152 --> 00:37:38.822
<v Martin Veit>So I looked into three main approaches.

00:37:38.822 --> 00:37:42.355
<v Martin Veit>So one of them is called natural scene statistics.

00:37:42.355 --> 00:37:47.675
<v Martin Veit>Then you have machine learning-based models and then you have something called saliency-based models.

00:37:47.675 --> 00:37:50.974
<v Martin Veit>So the first one, the natural scene statistics.

00:37:51.625 --> 00:37:58.655
<v Martin Veit>Essentially what it does is it looks at statistical distributions of the image that you're trying to assess.

00:37:58.655 --> 00:38:08.715
<v Martin Veit>So let's say you have an image, then you compute something called a wavelet coefficient or wavelet coefficients of the image and then you get some sort of distribution.

00:38:08.715 --> 00:38:18.713
<v Martin Veit>It's not so important now the math that you do, but what people noticed was if you have different distortions, for example blur, you have a noise.

00:38:18.713 --> 00:38:23.351
<v Martin Veit>It changes this distribution and that's essentially what the models are built on.

00:38:23.351 --> 00:38:26.925
<v Martin Veit>Changes this distribution and that's essentially what the models are built on.

00:38:28.666 --> 00:38:31.900
<v Martin Veit>So you can extract different features to then make a model that then computes some sort of image quality.

00:38:31.900 --> 00:38:40.858
<v Martin Veit>And the whole backbone of this is that you have data sets where people have scored individual images with different distortion circles.

00:38:40.858 --> 00:38:45.059
<v Martin Veit>It could be JPEG compression, you could have noise, you could have blur, you could have blur.

00:38:45.059 --> 00:38:49.786
<v Martin Veit>You could have a very wide variety of distortions that you have on images.

00:38:49.786 --> 00:38:56.992
<v Martin Veit>And then you have individual humans scoring the quality of this image and we can talk a bit about that later.

00:38:56.992 --> 00:39:07.590
<v Martin Veit>But essentially you could have, let's just say, 100 people scoring different images with different levels of severity of the different distortions, and then you have an image dataset.

00:39:07.590 --> 00:39:09.771
<v Martin Veit>In the end you can train models on this.

00:39:10.945 --> 00:39:14.110
<v Wojciech Wegrzynski>But it's like an ambiguous scale, like everyone comes up with their own.

00:39:14.110 --> 00:39:17.289
<v Wojciech Wegrzynski>For one person, the distortion could be nine, for one it could be three.

00:39:17.289 --> 00:39:21.713
<v Wojciech Wegrzynski>Right, but you solve that by a large sample of you.

00:39:21.733 --> 00:39:36.001
<v Martin Veit>You have a large sample and also what people normally do when they want to create one of these data sets is they have sort of like a calibration round just to sort of like base people in some scale that they can expect to be in.

00:39:36.706 --> 00:39:40.114
<v Martin Veit>So this image is this bad and this image is this good?

00:39:40.114 --> 00:39:41.518
<v Martin Veit>Approximately these scores.

00:39:41.518 --> 00:39:55.786
<v Martin Veit>And then you sort of like calibrate the participants and then they go through a lot of images, they score that, and then in the end you'll have a data set where you have different distortions, different images and then different severity of the distortions.

00:39:55.786 --> 00:40:03.391
<v Martin Veit>And then, once you have a lot of good data, you can then use that to train models on by train models, you mean machine learning.

00:40:04.186 --> 00:40:16.307
<v Martin Veit>Also machine learning, but also some of the natural scene statistics models use support vector machines, for example, to then, I guess, map these features to the scores.

00:40:16.307 --> 00:40:18.088
<v Martin Veit>But that leads us to the next one.

00:40:18.088 --> 00:40:25.251
<v Martin Veit>So machine learning, and it's more or less, I guess the most common one would be convolutional neural networks.

00:40:25.251 --> 00:40:42.380
<v Martin Veit>So you have a lot of different, I guess, machine learning architectures where people try to improve upon previous models and then once again try to see okay, if I have a score of this specific model, how do I compare it with the human score of this image?

00:40:42.380 --> 00:40:46.617
<v Martin Veit>And then what's the correlation between, if I run this through the data set?

00:40:46.617 --> 00:40:54.318
<v Martin Veit>So what you want to have is a high correlation between what the model predicts and also what a human would give this image.

00:40:54.318 --> 00:41:06.052
<v Martin Veit>Because what this does is that you now directly have something where I can say a human would score the image, this specific value, but we also have a model that can do this.

00:41:06.052 --> 00:41:12.007
<v Martin Veit>So now you actually have something that can predict the image quality of an image, of an arbitrary image.

00:41:12.007 --> 00:41:22.536
<v Martin Veit>So that's the whole idea, and also the plan would be to do this also on thermal images, because if you have a very high correlation with this model and then human scores.

00:41:22.536 --> 00:41:32.811
<v Martin Veit>What you can say is the model should score this pretty consistently and also, if you have this higher quality image, this should also be consistently higher than the lower image or the lower quality image.

00:41:32.811 --> 00:41:35.105
<v Martin Veit>So that's the two first approaches.

00:41:35.346 --> 00:41:45.715
<v Martin Veit>The last one is something called saliency, and this is a bit different because essentially what saliency models do is they predict where people look on images.

00:41:45.715 --> 00:41:57.016
<v Martin Veit>So say, you have an image with some people and you have some faces Based on eye tracker data, there's large data sets where people have participated.

00:41:57.016 --> 00:42:04.434
<v Martin Veit>Then if they look at a specific point of this image, then they get a point here, for example.

00:42:04.434 --> 00:42:12.855
<v Martin Veit>So the idea with these salience models is to predict where people look on images, but once again removing the participants.

00:42:12.855 --> 00:42:34.650
<v Martin Veit>So train a model and then you can say, okay, based on this model, trained on this data, with eye-trigger data, we predict that a person will look at this point in the image and the way people use this for image quality assessment, because it's not necessarily immediately trivial to see why that would blend into image quality assessment.

00:42:35.331 --> 00:42:44.998
<v Martin Veit>But what people saw was that if you have this saliency model and it creates a saliency map, which is essentially just probability of where people look on the image.

00:42:44.998 --> 00:42:55.451
<v Martin Veit>Then they can use the saliency map with another image quality assessment model to then boost the performance of the model by a few points.

00:42:55.451 --> 00:43:04.534
<v Martin Veit>So then in the end what you get is a higher correlation between the model that you trained with the saliency map and then the human score from the data set.

00:43:04.534 --> 00:43:09.836
<v Martin Veit>So those are some of the models that I looked into.

00:43:09.836 --> 00:43:11.431
<v Martin Veit>So these three main approaches.

00:43:12.704 --> 00:43:21.853
<v Wojciech Wegrzynski>I have so many more questions about thermal imaging, but let's close the report because I think we're narrowing to the conclusions and giving those three techniques.

00:43:21.853 --> 00:43:28.894
<v Wojciech Wegrzynski>What were your concluding recommendations to improving the quality of the assessment of those cameras?

00:43:29.677 --> 00:43:29.878
<v Martin Veit>Right.

00:43:29.878 --> 00:43:37.373
<v Martin Veit>So what I saw when I looked into the different models and also the datasets was that a lot of work in the visible spectrum.

00:43:37.373 --> 00:43:47.393
<v Martin Veit>But I've only saw maybe one study, I think, that created its own dataset and then trained a model on that dataset to then predict the thermal image quality.

00:43:47.393 --> 00:44:02.597
<v Martin Veit>So there is some work on it and it can be done even with some of the earlier models the natural scene statistics but at the moment there's a gap in the research because we don't actually have a thermal image quality assessment dataset.

00:44:02.597 --> 00:44:06.018
<v Martin Veit>We don't have a dataset with also it could be for firefighting, but also just in general we don't actually have a thermal image quality assessment data set.

00:44:06.018 --> 00:44:13.498
<v Martin Veit>We don't have a data set with also it could be for firefighting, but also just in general we don't have a data set with thermal images where people have scored the quality of this specific image.

00:44:14.105 --> 00:44:15.349
<v Wojciech Wegrzynski>What would constitute a data set?

00:44:15.349 --> 00:44:16.373
<v Wojciech Wegrzynski>What's a data set?

00:44:16.373 --> 00:44:16.885
<v Wojciech Wegrzynski>So?

00:44:16.985 --> 00:44:23.425
<v Martin Veit>imagine you put a bunch of people into a room and you have some images in the thermal spectrum.

00:44:23.425 --> 00:44:25.472
<v Martin Veit>So you have a lot of different thermal images.

00:44:25.472 --> 00:44:28.152
<v Martin Veit>Then you degrade that to some degree.

00:44:28.152 --> 00:44:39.934
<v Martin Veit>So let's say we add blur, we add noise, we add some JPEG compression, we ask people to give a specific score of all of these images and then in the end we aggregate that.

00:44:39.934 --> 00:44:53.960
<v Martin Veit>So you have one big data set, so all of the different images named and then also the corresponding scores from the human that would constitute the data set and what metrics make it a good data set versus a bad data set?

00:44:54.460 --> 00:45:02.282
<v Martin Veit>I mean you need to have a certain amount of people, but also it's a difficult question because a lot of the work was in the visible spectrum.

00:45:02.282 --> 00:45:09.943
<v Martin Veit>But you need a certain amount of representative distortions that you expect to see on thermal images in general.

00:45:09.943 --> 00:45:12.612
<v Martin Veit>You also need it, especially if you go.

00:45:12.612 --> 00:45:20.733
<v Martin Veit>If you want to apply this to the fire service and it should be relevant for firefighters then you also need to have some conditions.

00:45:20.733 --> 00:45:26.132
<v Martin Veit>That's representative of the scenarios that a firefighter will expect.

00:45:26.132 --> 00:45:28.588
<v Martin Veit>So it's in the visible spectrum.

00:45:28.668 --> 00:45:31.695
<v Martin Veit>People were sitting in front of monitors.

00:45:31.695 --> 00:45:33.099
<v Martin Veit>Those are calibrated.

00:45:33.099 --> 00:45:39.525
<v Martin Veit>You had an evenly illuminated environment, but that environment is not necessarily the same.

00:45:39.525 --> 00:45:53.681
<v Martin Veit>If you want to have a dataset that's also fitting for firefighters and the thermal image quality in the fire service, perhaps you need to use monitors that are representative of the thermal imaging cameras.

00:45:53.681 --> 00:45:56.875
<v Martin Veit>It may be three, three and a half inches.

00:45:56.875 --> 00:46:10.456
<v Martin Veit>So there's a lot of different things and also the distortions that you might expect is also different for infrared images, right, so you might have some radiation reflected off whatever surfaces and so on.

00:46:10.456 --> 00:46:25.137
<v Martin Veit>But in the report we try to give some recommendations on how to create a dataset and also what we should think about if it should be relevant for thermal images and also some of the challenges that you might run into in the end.

00:46:25.625 --> 00:46:26.427
<v Wojciech Wegrzynski>A follow-up question.

00:46:26.427 --> 00:46:26.829
<v Wojciech Wegrzynski>Is that?

00:46:26.829 --> 00:46:46.376
<v Wojciech Wegrzynski>Okay, you're talking here a lot about image quality in relationship to fire fighting, which is absolutely understandable if we're talking NFPA 1801 thermal cameras, but that's not an entire use of thermal cameras in fire safety and I would argue there are, for example, detecting a fire.

00:46:46.376 --> 00:46:55.353
<v Wojciech Wegrzynski>I think that is a very interesting use of thermal cameras which is currently not your mainstream way of detecting fires.

00:46:55.353 --> 00:46:57.713
<v Wojciech Wegrzynski>I had an episode on detection a few episodes ago.

00:46:57.713 --> 00:47:19.706
<v Wojciech Wegrzynski>I had a very interesting episode on waste fires with Ryan Fogelman, who is the globe's number one refurbisher of FLIR cameras, and the reason for that is Ryan is using them to detect fires in his devices in the fire over towers, so they are using thermal cameras to detect fires in the waste facilities.

00:47:19.706 --> 00:47:48.862
<v Wojciech Wegrzynski>If you've missed that episode, I recommend going there because we talk a lot about practical use of this technology in that episode and I think you know having the ability to stand in a standardized way, in a controlled manner, define the minimum characteristics of a thermal camera is also one of the barricades on the road towards using them as a reliable detection device.

00:47:49.385 --> 00:47:55.338
<v Wojciech Wegrzynski>And, what's interesting, you said that the resolution of your cameras in 320 to 140,.

00:47:55.338 --> 00:48:09.500
<v Wojciech Wegrzynski>You've classified this as low, and I agree if I want to look at an image, but if I'm trying to cover a field of view that spans over a building or something and I just want to know is there a fire in the room or is there not a fire in the room?

00:48:09.500 --> 00:48:14.956
<v Wojciech Wegrzynski>Perhaps a simple information any pixel of them is overheated over my fire temperature.

00:48:14.956 --> 00:48:19.876
<v Wojciech Wegrzynski>Maybe I can go with a camera of a resolution of, I don't know, 50 by 50, 20 by 20.

00:48:19.876 --> 00:48:21.351
<v Wojciech Wegrzynski>I don't know how low you can go.

00:48:21.351 --> 00:48:29.956
<v Wojciech Wegrzynski>I know people who go extremely low with the resolution and yet are capable of getting some sort of information through their system.

00:48:29.956 --> 00:48:56.166
<v Wojciech Wegrzynski>So do you think those metrics and those testing approaches perhaps minus the people assessing the data, because that's very you know, I observe the image type of assessment, but in terms of gathering data resolution, I think we'll also need to come up with tests like that to check the robustness of the future infrared fire detectors really Right?

00:48:56.206 --> 00:48:56.969
<v Martin Veit>And I completely agree.

00:48:56.969 --> 00:49:04.302
<v Martin Veit>I mean you don't need a full HD image in the infrared spectrum in order to do a lot of the tasks that we use.

00:49:04.302 --> 00:49:11.938
<v Martin Veit>Thermal imaging cameras for, absolutely, as you say, go lower, also depending on what you need to do with the camera.

00:49:11.938 --> 00:49:17.876
<v Martin Veit>So the current resolution is more than enough, I believe, to go into burning buildings.

00:49:17.876 --> 00:49:40.793
<v Martin Veit>I think that the question is not so much should we improve the quality of the cameras for this specific purpose, but more do we have a way that's actually robust, predictable, consistent to test the quality, because you don't want to have a mishap, for example, where you have to fail a camera because of the tests, because of the inherent inconsistencies in the test.

00:49:40.793 --> 00:49:43.474
<v Martin Veit>You just want to have a robust method.

00:49:44.264 --> 00:49:51.132
<v Martin Veit>I think what this report also opens up to is perhaps more interesting question is what else could we use such a dataset for?

00:49:51.385 --> 00:50:15.525
<v Martin Veit>Because there was a recently published report by NIST that also talks about AI and machine learning in the FHIR service, and perhaps it's not so much the quality that should be in the focus from this report, but maybe the fact that we don't have a dataset of thermal images of firefighting scenarios, because if you have that, you could apply that to a lot of different things.

00:50:15.525 --> 00:50:19.215
<v Martin Veit>You could apply machine learning directly on the thermal imaging camera.

00:50:19.215 --> 00:50:21.010
<v Martin Veit>You could try to detect humans.

00:50:21.010 --> 00:50:23.360
<v Martin Veit>You could improve split-second decisions.

00:50:23.360 --> 00:50:25.264
<v Martin Veit>You could try to detect humans, you could improve split-second decisions.

00:50:25.264 --> 00:50:46.976
<v Martin Veit>And if you have a data set, that will not only open up for this image quality assessment, which I believe is also valuable in itself, but perhaps more for research purposes and this consistency issues, but opening up the door to a lot of difference and other opportunities applying machine learning to who knows what right.

00:50:46.976 --> 00:50:50.231
<v Martin Veit>People are inventive and there's a lot of research on interesting things.

00:50:50.231 --> 00:50:53.494
<v Martin Veit>But in order to do research, we need good data.

00:50:54.505 --> 00:51:14.934
<v Wojciech Wegrzynski>Let's try to close with some final recommendations and just your final take to the reader of the report and to the NFP committees when they should head with that and what actions should be taken to improve the reliability of those testing methods.

00:51:15.447 --> 00:51:17.815
<v Martin Veit>I think the main takeaway from the report.

00:51:17.815 --> 00:51:24.954
<v Martin Veit>There's a lot of different recommendations and we look into different models to perform image quality assessments.

00:51:24.954 --> 00:51:27.262
<v Martin Veit>The main thing we need good data.

00:51:27.262 --> 00:52:03.411
<v Martin Veit>We need a data set where we can actually train the different models, see if this can be used, see if we can improve the consistency issues, see if we can improve the current framework to testing the image quality assessments, and then I think it will not only open up for image quality assessment, but I think it will also open up a lot of other doors and interesting research that could then, in the end, improve and help firefighters and the fire service, who put their lives at risk every single day in order to protect us as a redundancy in the fire safety strategy.

00:52:03.813 --> 00:52:08.235
<v Wojciech Wegrzynski>Perfect, a better, more robust, useful, open data sets.

00:52:08.235 --> 00:52:11.253
<v Wojciech Wegrzynski>I cannot ask for anything else in fire science.

00:52:11.253 --> 00:52:19.751
<v Wojciech Wegrzynski>Martin, thank you very much for coming to the Fire Science Show and discussing the testing methods for thermal cameras, important piece of technology.

00:52:19.751 --> 00:52:25.976
<v Wojciech Wegrzynski>I hope it was interesting for many of our firefighting friends and I'm sure it was interesting for fire engineers, our firefighting friends, and I'm sure it was interesting for fire engineers as well.

00:52:25.976 --> 00:52:26.958
<v Wojciech Wegrzynski>Cheers, mate, thank you.

00:52:26.958 --> 00:52:30.125
<v Wojciech Wegrzynski>Thank you for for having me and that's it.

00:52:30.125 --> 00:52:31.108
<v Wojciech Wegrzynski>Thank you for listening.

00:52:31.688 --> 00:52:36.949
<v Wojciech Wegrzynski>I must say after the episode I was still a little confused on how the thermal cameras actually work.

00:52:36.949 --> 00:52:42.398
<v Wojciech Wegrzynski>I'm perhaps it's just me and the way how I use cameras Personally.

00:52:42.398 --> 00:52:51.733
<v Wojciech Wegrzynski>I'm also a hobbyist astrophotographer and every time I image something, I assign the color to some sort of wavelength spectrum.

00:52:51.733 --> 00:52:56.012
<v Wojciech Wegrzynski>That's how color photography works, that's how astrophotography works.

00:52:56.012 --> 00:53:05.887
<v Wojciech Wegrzynski>So you have different elements that emit the different peaks of wavelengths and by measuring where the wavelengths hit, that's your color in thermal cameras.

00:53:05.887 --> 00:53:16.746
<v Wojciech Wegrzynski>It didn't make sense to me why, how the hell the camera can see the different temperatures, because you know the peak wavelength will come from different bodies at different temperature.

00:53:16.746 --> 00:53:19.612
<v Wojciech Wegrzynski>It's it's not an easy value to measure.

00:53:19.612 --> 00:53:24.951
<v Wojciech Wegrzynski>And and then I I was listening to this episode I finally understood.

00:53:24.951 --> 00:53:27.237
<v Wojciech Wegrzynski>It really is about the intensity of rotation.

00:53:27.237 --> 00:53:40.490
<v Wojciech Wegrzynski>So the optical pathway focuses the signal on the chip, whatever it is, and then different pixels of that chip respond to different temperature, different intensity of radiation.

00:53:40.490 --> 00:53:43.855
<v Wojciech Wegrzynski>Because the chip's properties, thermal properties, are very well known.

00:53:43.855 --> 00:53:54.117
<v Wojciech Wegrzynski>Different radiation intensity will heat up the pixels to different temperatures and therefore it can respond to the differences of observed temperatures.

00:53:54.117 --> 00:54:02.688
<v Wojciech Wegrzynski>And because infrared is basically electromagnetic wavelength, it gets transported and focused through the lenses almost like a normal light.

00:54:02.688 --> 00:54:04.802
<v Wojciech Wegrzynski>So I finally got it.

00:54:04.802 --> 00:54:09.806
<v Wojciech Wegrzynski>So it really responds to the changes of intensity of rotation, not some specific wavelength.

00:54:09.806 --> 00:54:16.469
<v Wojciech Wegrzynski>Perhaps that's why the earlier cameras had it very difficult to have high frame rates, because you need to heat up those pixels.

00:54:16.469 --> 00:54:18.349
<v Wojciech Wegrzynski>They have to be very responsive.

00:54:18.349 --> 00:54:20.112
<v Wojciech Wegrzynski>So, yeah, kind of makes sense.

00:54:20.112 --> 00:54:22.355
<v Wojciech Wegrzynski>Finally, I finally got it.

00:54:22.355 --> 00:54:24.570
<v Wojciech Wegrzynski>You can congratulate me in the emails.

00:54:24.570 --> 00:54:30.494
<v Wojciech Wegrzynski>It took me so long of fascination with thermal cameras to finally finally understand how they work.

00:54:30.494 --> 00:54:33.835
<v Wojciech Wegrzynski>Thank you, martin, for helping me capture that in there.

00:54:34.485 --> 00:55:03.364
<v Wojciech Wegrzynski>In this episode we've talked about some very fundamental things about the use of thermal cameras and some things that are highly complicated, the ones that Martin was looking into, the quality of image measurements, and I think they perhaps are not the most useful for everyday engineers, but for those who work with this field optical imaging who work with thermal cameras who work with certification, those things are fundamental.

00:55:03.364 --> 00:55:05.088
<v Wojciech Wegrzynski>Thermal cameras who work with certification, those things are fundamental.

00:55:05.088 --> 00:55:13.327
<v Wojciech Wegrzynski>Those things make or break your entire certification scheme around approving thermal cameras for use If you want to have a detection.

00:55:13.327 --> 00:55:19.259
<v Wojciech Wegrzynski>Those things make or break your capability to detect and filter out false alarms.

00:55:19.259 --> 00:55:26.438
<v Wojciech Wegrzynski>So indeed, I think there will be a lot of people who will be using the contents of this episode in their everyday work.

00:55:26.438 --> 00:55:31.336
<v Wojciech Wegrzynski>I actually got some ideas for my own experiments with visibility in smoke.

00:55:31.336 --> 00:55:32.907
<v Wojciech Wegrzynski>After what Martin said.

00:55:32.907 --> 00:55:43.514
<v Wojciech Wegrzynski>I'm changing the schedule and scope of my experiments a little bit to use some of the clever ideas he put forward in this episode, so I am very happy about that.

00:55:43.784 --> 00:55:45.898
<v Wojciech Wegrzynski>I hope you also found something nice for yourself in this episode.

00:55:45.898 --> 00:55:46.114
<v Wojciech Wegrzynski>So I am very happy about that.

00:55:46.114 --> 00:55:59.288
<v Wojciech Wegrzynski>I hope you also found something nice for yourself in this podcast episode, and I'm simply happy that I was able to give the fire science show to to a phd student, to someone who just finished their fire protection research foundation grant.

00:55:59.288 --> 00:56:12.882
<v Wojciech Wegrzynski>It's amazing that those foundations exist and and fund research like that, and I really want to highlight that it's really amazing that that foundations exist and fund research like that, and I really want to highlight that it's really amazing that there are many ways people can do research in science in the world of fire protection.

00:56:12.882 --> 00:56:18.034
<v Wojciech Wegrzynski>So that's great, and I am just happy to talk about fire science with you.

00:56:18.034 --> 00:56:22.893
<v Wojciech Wegrzynski>And well, next week I will be doing the same thing, so see you here again next Wednesday.

00:56:22.893 --> 00:56:23.744
<v Wojciech Wegrzynski>Cheers, bye.