Andy Thé, MathWorks
Cameras are everywhere, even in your phone. You might have a new idea for using your camera in an engineering and scientific application, but have no idea where to start. While image processing can seem like a black art, there are a few key workflows to learn that will get you started.
In this webinar we explore the fundamentals of image processing using MATLAB. Through several examples we will review typical workflows for:
Previous knowledge of MATLAB is not required.
About the Presenter: Andy The' holds a B.S. in Electrical Engineering from Georgia Institute of Technology and a B.A. in Business from Kennesaw State University. Before joining MathWorks, Andy spent 12 years as a field applications engineer focused on embedded processors at Texas Instruments, and 3 years as a product marketing manager for real-time software at IntervalZero.
Hello, and welcome to the Image Processing Made Easy webinar. My name is Andy Thé, and I'm a technical marketing manager here at MathWorks. Image and video processing is an extremely popular field, and you see it being integrated into a lot of different application spaces—anything from medical to robotics, automotive, machine vision. The list is pretty endless.
Unfortunately, with all this integration comes a lot of challenges. A lot of people are new to image processing, and therefore we get a lot of questions, such as: How do I deal with all these different file formats? How do I create and actually test my algorithms? How do I visualize and maybe find the cause of the problems? And then finally, how do I deal with large images and speed up the execution of my algorithms?
So in this webinar, we're going to go over the first four topics you see here. So the goal of this webinar, really, is we hope to equip you with some common strategies that you can apply to very typical image and video processing workflows. And with that, if you see this image below, you see this as just some simple toys and some candy. We're going to use this image throughout many of our examples here. And I'll have about three different major examples to walk you through.
But using this simple image, we're going to use several processing techniques. And by the end of it, you'll be able to see, on the right-hand side, how we can greatly simplify the image and really segment it and be able to do, maybe, some statistical analysis to get some details out of the image.
So the agenda for today is we're going to spend some time going over an introduction. We'll go through image enhancement. We'll do some image segmentation. And then we'll do image registration, and then we'll have a quick summary. Let's go over a typical image and video processing workflow.
Of course, in the beginning, you're going to want to acquire the data. That's what we call access. So this can come in the form of files, like JPEG images. Can come in a form of output from other software. And of course, from hardware, such as a camera. So that's very common, to capture live images and video if you want to process those on the fly.
Once you acquire your data, of course, you're going to want to explore it and also do some discovery, find out what you need to do to create that algorithm. So a lot of time is going to be spent here, actually, making your application and your algorithms. And finally, you're going to share it.
So you'll share with colleagues, either in the form of documentation. Maybe it's a form of libraries that they're going to consume. And it could be consumed for third-party software that you might use to design an end product, for example, a camcorder, or maybe some sort of automotive vision application.
So although I showed a single streamlined path, the reality is this is really an iterative path. And so once you acquire data, you're probably going to develop the algorithms, and then you're going to come back and acquire some more, increase your data sets to really solidify and validate your algorithms.
Once that's developed, you're going to share it, of course, get feedback. You're probably going to come back and iterate on it to refine your application. So this really needs to be automated. With MATLAB, you'll find out that MATLAB is not only language, but it's also a scripting language, so you'll be able to automate your entire process.
What we'll be using throughout all these demonstrations in addition to MATLAB is the Image Processing Toolbox. So there's a lot of functions. There's over 200 functions at your disposal. But at a high level, you'll be able to do things like displaying and exploring the images. We'll do some enhancement, some image analysis, morphological operators—which is shape-based processing, or shape-based filtering—image registration, geometric transformations, and region-based processing.
And in these demos, and what we'll do in this webinar, we'll cover many of these topics highlighted in blue. And so this brings us to our first topic. What is image enhancement? So simply, this is the process of adjusting an image so that the results are more suitable not only for display, but also for further processing. So as you see down in the camera man image, you can do such things as deep blurring, or you can do some filtering, otherwise known as pre-processing. That can be a preliminary step for you to do more enhanced processing or more advanced processing later.
So that takes us to our first demonstration, where we're actually going to take a Landsat image. So this is a satellite image of Paris. And you see on the left-hand side there's definitely some issues here. We see a lot of purple, not a lot of contrast. On the right-hand side is what we want to drive towards.
So the goal, really, is to import this map of Paris. We're going to correct for the poor contrast and the unbalanced colors and make the image, hopefully, a lot more usable. So here's MATLAB. And just so it's not overwhelming, I'm only going to cover the aspects of MATLAB, or features, as I come across them. So let's go ahead. Before I get into the image enhancement demonstration, I'm just going to show you some of the basic features.
So here, we have our toy and candy image right here. It's a JPEG file. I can import that into the environment, as simply I equals imread, and then this model complete, and then there. And so what you'll see is I created this variable I. I imported the image. And you can see here, it's inside of the workspace. So this is where all my variables are stored.
And you'll see it's about a 2,400-by-2,800 pixeled image, and there's three layers of it. And the reason why there's three layers is there's a red, a blue, and a green layer, so all three colors. And so we can view this image, just to verify it's there. If we click on it, you can go to Plots, and here are the different plotting tools you can use.
And so we can simply do an IM show. And here, we have the image. So here are the toys, as well as the candy. And you can see at a command line what was entered. So if you prefer to operate things at a command line, you have that option as well.
So another tool I'm going to bring up—let me close this window—is just using the IM tool function. So using that same variable, I'm going to click on IM tool. And you'll see here, it entered in IM tool with the variable I. And it looks like a similar window, except this is one of the nice apps that are within the Image Processing Toolbox.
And what that gives you access to is things like, let's say we want to measure distance. So I can click on this ruler and I can simply connect and draw these distance lines. So that's about 700 pixels long, that car, about 266 seven pixels wide. How big are these pieces of candy? So this type of pixel information and measurement is very helpful, and we'll play a little bit with this later as we do some, maybe, shape-based processing. So just to kind of give you a little foreshadowing of what we'll be doing later.
And also, you can do such things as some more detailed analysis. So here is the button to inspect pixel values. If I click on that, what you'll see—let me move this over—is this little crosshair appeared. And here is a zoom in of what that crosshair is pointing at. We actually see the pixel values on this little hex bug, and this is showing us the RGB values, or intensity values.
So I'm going to go ahead and zoom in here so you can see, more or less, what's going on. And you'll see, as I drag around these crosshairs, I can inspect the pixels of where the crosshairs are pointing. So here, you can see RGB values are very low because this piece of candy is close to, essentially, the color black.
So when you move to the desktop, here's closer to white. That's why the RGB values are a lot higher. And then the hex bug is a shaded turquoise. And here is, basically, the RGB values for that color. So again, this is just a really nice way of exploring images and really giving you some strategies that you may want to tackle later based on maybe some shape-based processing, and you see these colors here. We're going to actually be doing some color-based processing as well.
And so if we go to our first one, this is an example that you'll actually find inside of the product. And what we're dealing with here is actually multi-spectral data. So this is a Landsat image. And if I scroll down a little bit, you'll see how I'll be reading the imagine in. Because this is multi-band data, we're going to use a function, instead of imread, called multi-band read.
And to pull up help on anything you might want inside of MATLAB, you can hit F1, and simply, it'll bring up the Help window. And you can see this is to read banded interleague data from a binary file. So here, it tells you different parameters, a description, showing you the different bands here, a nice graphic. And then here is the syntax and telling you what are the different parameters, and even giving you an example. So a lot of different ways to get the help. One is just by clicking F1. And of course, you can click inside of the documentation, or search bar right here as well. Okay.
But let's go ahead and run this code. So here we are right here at the read. I'm going to just run, and here we'll say, run in advance the next layer. So here, the Advance button. So this is what's really nice about MATLAB as well, is I can run what's called sections of code. So every time you see these double percent signs, that denotes another section. A single percent sign is comments. So what's really nice about MATLAB is you can actually run code out of order based on just moving around these different sections.
So here we are inside of this section here. I'm going to go ahead and run and advance to the next one. And here we see an imshow of our Landsat image. So this is the image we showed earlier. Not really a great image. Everything looks kind of purple. It's kind of very difficult to work with. So let's go ahead and close that, and let's try to analyze what's going on with that image.
So I'm going to use an iamhist function. So what this is—I'll just open it up—it's just going to give me a histogram of the image data. And so let's go ahead and run this section, advance. And then we'll see here, this is showing us where the problem's coming from. So we see here, all of the data is clustered together right here, around 50. And so this is the red band. That's what's giving us pretty much poor contrast.
And to look at the data a little more detail, we're going to do something else here. We're going to actually run a scatter plot. So what this is is plotting the red, blue, and green in a three-dimensional plot for us. So here's all the data points. And you can see, it's not hard. This means, also, to see what the problem is. In all three planes, we see everything clustered along this line right here. So again, not really great. You kind of want to see some, as far as distribution here, to get some more dynamic range and to see different colors.
So let's go ahead and see how we can correct this. If we scroll down here, this is where we're going to try to do what's called a contrast stretch. And so this is using a function call imadjust, or image adjust. And you'll see here, hitting F1, I'm going to adjust the intensity values of the color map.
And so you simply pass it in here, and you'll scroll down. Here's just another example of it. I'm going to be passing in the true color, and then you'll see I'm just going to be doing a stretch limiter. And you'll see here what this stretch limit is, the function. It finds the limits to the contrast stretch of the image.
So we're going to go ahead and pass that in and we're going to run it. And you can see, instantly, we have a much nicer image. We have actually some shades as far as even dark, dark green, all the way to some shades of white. So this is a pretty nice image. And so we can actually kind of do some processing on this. We can really work with it.
But we can even take this a little further. If you look, although there's some good contrast, there still, as far as the colors, are quite muted. And let's see if we can actually make the colors a little more dynamic. So this was a great step forward just using a contrast stretch. Let's go ahead and check the histogram on why this actually worked. And so you see here—before, remember, everything was clustered in a narrow band around 50. Now, we see the histogram is pretty evenly distributed, and so that's why we're able to see so much more detail.
So to go one step further and maybe fix the color, we're going to do something, what's called a decorrelation stretch. So this is going to enhance the color separation across the highly correlated channels. Remember earlier that the scatter plot, everything was banded, or correlated, really tight across one line.
So with this decorrelation stretch right here—let's go ahead and run this and go to the next section—you'll see, again, not only do we have a nice bit of contrast difference, now we have a pretty dynamic range of color, everything from these nice dynamic greens to blue to shades of red. So we have a lot more tangible data that we can deal with. And really, you can use the image as is or continue down the path and do some more image processing.
So let's go ahead and take one last look at this data from a plotting perspective. And if we look here, here's the scatter plot. And you can see, instantly, is that we have a pretty good distribution of color, and that's why we have such a dynamic image.
So just to summarize our image enhancement example, you saw we were able to visualize not only the images, but we're able to look at histograms and scatter plots to help analyze the problems in the image. And then based on that, we're able to do some performance enhancement, like contrast stretch, as well as a decorrelation stretch. And so we, essentially, move from this poorly illustrated image here that was all purple to this nice dynamic image with lots of shades of color, as well as correct contrasts.
So this takes us now to our next subject. We're going to talk about, what is image segmentation? Image segmentation is simply the process of dividing an image into multiple parts. And typically, this is a foreground as well as a background. This process is typically used to help identify and isolate objects so that you can use other functions to extract meaningful information from an image.
So here below, you see a densely packed image here. And we use watershed segmentation to essentially isolate and start counting some of the objects here in this coined image. You can see we segmented it, and we have a nice binary image here that we can actually do analysis on to maybe even count how much money is here based on the size.
So let's go ahead and get into the demonstration here. So we're going to deal with our toy cars and candy and hex bug image. And we're going to segment and analyze the image. We're going to turn it into this binary image here of everything that we care about, and then we're going to extract some statistical information from it.
We'll go ahead and open up a script I wrote earlier. So here, you can see I'm doing exactly what you saw me do earlier. I'm going to read in the toys and candy image. Okay. And let's go ahead and dock that so it's a little easier to see. And I'm going to minimize this tool strip, just to make some more room here.
And so next thing I'm going to do is I'm going to convert it to grayscale. So let's go ahead and convert it. And you see here, I removed all the color. And if I look at my workspace over here, you can see I move from three layers to one. So I don't have a red, blue, green layer anymore, just a grayscale image with just single intensities for each pixel.
And then the last thing I'm going to do is an im2bw function. So this is a thresholding function. So this is probably one of the most simplest forms of segmentation, but it's also quite powerful. If we look at this function-- and let's go ahead and go our documentation, im2bw. I'm going to do it this way.
And here I am, inside of the Help. And if I look at this function, you can see this converts grayscale image to a binary image and uses a level. So this level value, essentially, sets the cutoff. So it's between 0 and 1 right here. And then wherever this cutoff is, everything above the cutoff is going to go white, to a value of 1, and everything below it is going to go to 0. And that's what makes the binary image.
And also, before I go, I'll just show you here, if you go to Documentation this way, this is another great way to navigate as far as to the function that you might care about; for example, image enhancement. Here we are, Contrast Adjustment. These are some of the functions we dealt with earlier. So again, and just another way to get to help and get some real guidance, as far as what you can actually do.
So let's go ahead and do this. And so I'm going to initially set this level of value to 0.5. I'm just at the halfway point. And we're going to do an im2bw and create this valuable ithresh. So let's go ahead and run this section. So I just hit Control Enter, just to run that section.
And you can see—let me go ahead and blow this up—here, we have our original image. Here we have our threshold image. So we did a pretty good job, you can see, just by putting a level of 4.5. We have some of the candies, some of them pretty well segmented. We have the cars. And then some of them, you can see, we're starting to lose it. So these lighter shades don't threshold as well. We didn't pick the right value. But at least we got the edges.
But there's a problem. You can see the candy. Most of them were segmented, except for you see these yellow ones. And you can probably guess why this is, and that's because the car, as well as these candies, the yellows are very close to the desktop here. And so when you have colors that are very close like this, the thresholding can be somewhat challenging and tricky. When you have high levels of contrast, basic thresholding is pretty simple to do. But again, when you have colors that are close, these might only vary by a couple different intensity levels. So that's where it can be very challenging.
So let's go ahead and think about this a little more. Before we do that, we can still experiment, though. So look at this level value. We can right-click on it and we can say, increment value and run section. Here's a little window that popped up. I can say, I want to increment that by 0.1, so the tenths place. And then let's go ahead and increment that.
And you can see, as I click on it, you see changes taking place. I'm now letting in some more black here. You can see that the problem is I get a lot more noise. However, some things are segmented pretty well. But I still can't get these yellow pieces of candy to show up any better. So there's definitely some issues. But the nice thing is I was able to iterate pretty quickly and see what was working and what isn't. So you see 0.6 was pretty good, but still, it wasn't good enough. I still can't get these yellow pieces of candy.
That actually brings us to a question. We really need to maybe take a step back and try to maybe think about, are there any unique attributes or characteristics about this image that maybe we can take to our advantage? So if you look at it, what are some things that jump out at you?
So one is, of course, the sizes of the objects. We've got the cars are pretty large in mass compared to the candy. The hex bugs are kind of in the middle. So we've got sizes we could work with. We can also do some image processing based on shape, right? We've got these very rectangular objects here with the toy cars and hex bugs, but the candy is actually round. So we could actually play a bit with shape-based processing, maybe with morphology.
And then finally, another thing we could look at is color. There's quite a bit of color being represented here. And so that is a pre-processing technique that many people overlook, but it's actually quite beneficial. So let's actually start with color. So if I come back to here—I actually wrote this function earlier, and this is called color spaces dot m.
If I open this up—let me go ahead and minimize that—you can see I'm going to just read my candy imagine in. But what this is going to do is we're going to print out each one of the color spaces. Remember, I said this was three layers here. Well, we're going to print out the red, the green, and the blue. And you can see, here's the plots I'm doing as well.
So let's go ahead and run this section here. You can see these easier. Here's my original image, but now, here's each one of the grayscale images from each one of the color planes. So I'm actually going to do pre-processing in color, and then I'm going to threshold each one of the different color planes.
So coming back to our Help here, you can actually see, if we go to Image Processing Toolbox, you can see there's a whole section on color. And you can deal with things like the ICC color profile, the LAP color spaces. You can even deal with HSV for the hue. So there's a lot of different ways you can actually do processing just based on color.
So let's go ahead and go back to MATLAB here. And so I'm going to open up another file. So this is similar. It's a combination of the color processing file, or script, I wrote earlier, as well as a thresholding one. Except here, we're just going to read in the image. We're going to do what we did before when we plotted out our three color spaces.
And then now, we're actually going to threshold on each one of the layers. So here—let's go ahead and open this up—you can see I did an IM to black and white, im2bw, on the red plane, the green plane, and the blue plane. And we pick the values that would get the objects isolated correctly without bringing in too much noise in the background. And what we did is we summed and added all the planes together. And so we have a correctly, as far as isolated image. I mean, there's still some noise in here, but we'll deal with that in a minute.
If you look back at our script here, you can see what happened. Essentially, again, thresholded each different color plane. Here is the red, the green, and the blue, and then I summed them all together. So I simply just did an and put them all together and then we plotted them out.
What I did with this next step here, if you looked here, this is great, but it's more intuitive to work with black as a background and white as the foreground. So I'm just going to complement the image. And so that's what's being done here. And I'm going to call a function called imfilled. So imfilled is a morphological function. So remember, I mentioned morphology is a study of shapes.
So this is a shape-base filter. And what this does is going to fill up the holes. So all these holes that you see in the image here are going to be filled up and made solid. So let's go ahead and do that. And you can see, it filled up all the holes very nicely, but we still have some random noise. We have these frayed edges. We have these little dots around. So we're going to deal with some more morphological operators, or shape-based filtering.
And before I get into that, let me actually show you the function, and we'll talk about what it's actually doing. If we look here, the next section right here uses a morphological operator called imopen. So this is going to open the image. And what all morphological operators tend to use is something called a structuring element.
So if I hit F1, you can see this structuring element is essentially the shape that you are going to select to help do your processing. And so you see this structuring element. You can pick things like a ball. You can pick diamond, disks, lines, octagons, and so on. We're going to deal with a disk.
So let's go ahead and run this. And so here, I'm going to do an opening. And you can see, if I come back to my image here, you can see it's cleaned up quite a bit. All the little dots are gone. All the frayed edges are cleaned up. And you've got a pretty well-segmented image right here. And you can see—let me go and dock this for a minute—I could have played with that structuring element a little more.
So let me go ahead and blow this up so you can see it a little better. So I can right-click again on the size of that disk. That's the radius. And you can say increment value section there. I can say 5. And now if I click, I can make that 30, 35, 40. And eventually, you'll see the size of that disk is actually going to get too big to fit inside of those pieces of candy, the Skittles.
So you'll see that 65, 70, 75. Now you see it couldn't fit in any of the candy, so it wouldn't redraw those pixels. And now, it can only fit in the hex bugs and the cars, so this is a way of I could deal with shape-based or pixel densities, as far as filtering, and I can keep incrementing until all the hex bugs are gone.
So you can see how morphological operators on a binary file is pretty powerful. You can really do some nice cleanup, as well as some shape-based filtering. So let's go ahead and move it back to what it was. I actually want all the pieces of candy, as well as hex bugs. I'm going to change this back to 25 and run that section. And so I have everything back.
And then the final step, what I'm going to do is I'm going to extract some details. So the function I'm going to use is called region props, or region properties. And this is a very powerful function. So hit F1. You can see what makes it so powerful is the fact that you can extract a lot of detail from a binary image.
And so you can get things, for example, such as area, how many pixels there are in a particular object. You can find things like the centroid, the centers, the eccentricity, the diameters, intensities of the actual pixels. So there's a lot of things you can pull from it, even the number of actual objects in the binary image. So a lot of neat statistics can be extracted with this region property function.
And so we're going to go ahead and run this. And then in the end here, the final step is we're actually going to draw some nice bounding boxes and label all of them, and also count how many objects there are. So you can see, I drew nice bounding boxes around all the objects for the users who can be easily identified. And also, we counted them, and there we saw that there 18 objects in the image.
And so you saw, through using this image of toys and cars and using a lot of the features in MATLAB, we were able to identify some unique attributes. So at first, we did some basic thresholding. I recommend that, as it's a very simple approach to segmenting an image. Well, you saw we had some issues with color here, especially these yellow pieces of candy here. They created problems because there wasn't enough contrast.
So what we did is some pre-processing, if you will. We did some color-based processing. We separated the color spaces into red, blue, and green. And we did now a thresholding on each one of the different planes. And as a result, we were able to sum them all together and get a complete segmented image of everything that we wanted to isolate.
And then finally, we did some region property functions to gather some statistics, to count the objects, and to surround them with nice bounding boxes for the user for easy identification. And then finally I kind of talked about morphological operators. That's a very powerful tool for pre- and post-processing, but also you saw how I can really segment an image based on the sizes of different structuring elements. So I can basically filter based on shapes.
So now, that brings us to some additional things, so other utilities and tools that can help you. You saw me play with imtool. There's a lot of fantastic apps that are at MATLAB Central. So this is our online community and where you see a lot of people contributing. For example, here are a lot of tools. You can use, for example, Segmentation tool, Morph tool to play with morphology, image registration app.
So these are fantastic utilities to not only help you explore, but also really converge on a solution. You can play with the different parameters. And a lot of these utilities actually output MATLAB code, so you'll be able to integrate it directly into your application. So I encourage you to take a look at these, and you'll find them on the File Exchange on MATLAB Central.
And so this brings us to our final topic, and that's image registration. So what is image registration? So simply, this is the process of aligning images from different data sets for either visual comparison or computational analysis. And you can see here from the images below, you can be using image registration for maps, maybe collected from multiple satellites. Maybe it's the same satellite. On the right, we have 3D medical imaging, where you're registering multiple images to give you a more comprehensive data so you can do some more processing as well.
So when it comes to the topic of automatic image registration, there's actually two major categories. One is intensity-based. This is exactly what it sounds. You actually are going to be aligning two images based on intensities. And in this case, it's maybe aligning the skull. You can see here, the intensity is very bright with maybe another image that has a high intensity there as well. And then in a map, that's exactly what's going on, too. It's aligning things like streets, houses, any sort of characteristics in the image where the intensities are going to line up.
The other type of registration is feature-based. So with feature-based, it's exactly what it sounds like. You're extracting different features; for example, corners. It could be edges. Could be particular unique features within it. And the difference between these two is intensity-based registration algorithm is found in the Image Processing Toolbox. The feature-based, because features is a big part of computer vision, is found in the Computer Vision System Toolbox. So the example I'm about to show you is the intensity-based automatic registration, and that's from the Image Processing Toolbox.
So in this demonstration, what I'm going to show you is how to work with, in this case, a webcam image and an infrared image. And the goal of this example is to register the two images together because we want to measure body temperature. And because the locations of the inner corners of the eye is the most accurate place to measure body temperature, we want to register these images so then I can run an algorithm on the webcam image to find out where the location of the eyes are, and then I'll be able to correlate on the infrared image that exact location to read the correct body temperature.
So let's go ahead and get back to MATLAB. And so here's our script here. And you'll see what I'm doing is I'm importing in the two images into my workspace. I'm going to be reading in the infrared image as the fixed image. So that's the one that's going to remain stationary. And then I'm going to read in the webcam image. So this is the one that's the moving image and the one that's going to try to move and line itself against the fixed image.
So let's go ahead and read those in. And then what I'm going to do quickly is imshowpair. So here, showing you the two images next to one another. Again, we want to register these two. So what we're going to do to configure that is we're going to call this function called imregconfig.
And so this is really the inputs. If I hit F1 you can see it's only expecting, really, one input argument. And you can see, here's the modality. So all that means, are your images captured from a single camera, or a single device? If so, then you use the input argument monomodal. If it's captured from different devices, like we are here, a webcam and an infrared camera, then we're going to input multi-modal. So that's our input argument, is multi-modal. You can go ahead and run this.
And then we're going to do our first try at registration. So we're going to use the imregister function. You can see here it's going to do intensity-based image registration. And the first thing, the transform type we're going to pick first, is simply translation. And so all that means, it's going to move the moving image in xy direction to see if it can align the two.
So let's go ahead and run that. And you can see here, this is our result. So here, we're doing an imshowpair in what's called false color. So we're overlaying them. And this kind of gives you, at least, some nice color to see what's going on. And you see it did a pretty decent job on the right-hand side, but there's definitely some issues. You see this ghosting here, and even the shoulders are misaligned.
Let's take another view of this. Let's do a blend instead of a false color. This gives us a little better viewpoint. So you can see here that there's actually scaling issues going on. So it aligned up properly on the right side here, but on the left-hand side, you can see one of the images is clearly larger than the other one.
So the simple translation, the default transform, is not going to work. So let's go ahead and change things. So we're going to change the transform type to Affine. And if you scroll down here, just to remind you what that is. So we use translation. That didn't work.
Affine is more advanced. It's going to use rotation, as well as going to change scale and some sheer as well. So let's go ahead and run that and see what happens. And so you see here, oh, it's gotten a little bit better. So we have not as much ghosting around the head, but we still have issues here around the neck and the ears. Even the shoulder seem to be off quite a bit. But you see that the image was rotated. So that part worked.
We're getting closer, but let's try something else. Let's see if we can try a different transform. We're going to try this similarity transform. And all that is is simply one additional change. It's going to deal with non-reflective similarity. But it's still going to do a translation, rotation, and scale, but it's not going to play with sheer. It's a little simpler.
So let's actually try this one. And here's the results. You can instantly see that this is actually a very good registration. So you see the head is aligned very well. You see the shoulders. You have a little bit of gap here, but to be honest, this is really what we're after, is we want the eyes aligned quite accurately. Because we want to detect, again, where the eyes are so we can accurately read the body temperature.
So the final step here is we're going to use a function from the Computer Vision System Toolbox, and that's a cascade object detector. And if you look here, if I hit F1, you see it's using the Viola-Jones algorithm, which is a great algorithm for anything dealing with faces. And not only detects where the faces are, but it helps to detect facial features, such as eyes, nose, mouths, upper bodies.
And you can see here, we're going to try to look for the pair of eyes. So if we go ahead and run that section, and then we're going to compute the temperature near the eyes. So this is where it's going to convert it. And what we're going to do here is we're going to embed the temperature and do a text inserter and write it on the screen so it's easy for the user to see. So if we go back to our image, here we have our nice registered image and we have a nice bounding box around the eyes. And you see here the temperature read is 98 degrees Fahrenheit, which is totally normal. And this person does not have a fever.
To summarize the demo, you saw how we acquired images, both infrared and standard webcam images. We visualize the data, both before and throughout our algorithm process. We used an automatic image registration to align the images. And then finally, once aligned, we use feature detection, using the Viola-Jones detector in the Computer Vision System Toolbox. And then finally, we found the relative temperature and we wrote that on the image for the user.
And for more information, here some great resources. First, we have a product page. We also have Steve's blog, where Steve Evans, one of our technical experts related to image processing, authors this blog. And there's a lot of great interesting topics there. And then books-wise, if you're new to image processing, Digital Image Processing Using MATLAB, which is also co-authored by Steve Evans, is a great resource to learn about the key concepts with image processing and how they are actually executed in MATLAB.
And then finally, a product trial where you can try out a lot of the examples you saw today, as well as experiment with your own algorithms. And that brings us to the end of our webinar. First of all, I'd just like to say thank you for the time, and hopefully you found this session helpful and that maybe you walked away with some new techniques or ideas that you can implement in your own image and video processing workflow. Thanks again.
Recorded: 15 Oct 2014