Artificial Intelligence (AI) is increasingly being used to solve automation issues. Self-learning algorithms are making it easier and easier to take specific but monotonous tasks off our hands. Despite the fact that you can use AI technologies for all kinds of problems, such as speech recognition and text recognition, it is not always the solution for every issue. At Pegamento, for example, we dealt with several cases where that occurred. We sat down with Computer Vision & AI Lead Thomas de Wolf to discuss why artificial intelligence is not always the best working solution for every issue.
For this story, we take the print industry. For example, an organization that prints personalized packaging, such as Tony’s Chocolonely where you can design your own wrapper. There are hundreds, maybe thousands, of different types of chocolate wrappers being made, each of which must be shipped in boxes to the right address. This occurs at many organizations within the print industry. While preparing orders, organizations in the print industry often manually check the products in and the picture (image label) on a box on a conveyor belt and then stick the correct order information on it. In a box, for example, there are chocolate bars with personalized wrappers for a specific customer, and of those they put the sticker on. And that for thousands of pieces a day.
Computer vision and Artificial Intelligence behind the scenes
‘It’s like a baby, who at some point learns that he can grasp something with his hands. He finds out it’s handy and learns from it.’
An automation issue for this may be that organizations in the print industry are looking for a way to automate manual checking. One application is a camera system that sees the box on the conveyor belt and recognizes the picture on the box. The system compares that picture to the database to then confirm, ”these are Tony’s Chocolonely’s milk chocolate bars and not the dark or hazelnut.” ”This is an issue where you don’t know in advance what will pass in front of the camera, but you want to start recognizing it,” Thomas says. ”For this kind of problem, we show the computer 500 examples of the milk, 500 of the dark and 500 of the hazelnut wrappers. Then we have three types of chocolate bars of which we have 500 images of each wrapper type on a white box in the database.”
With all those examples, the technology is going to train a learning algorithm. ”The software tries to recognize the milk label on the box, then it sees if it has gotten good at this,” Thomas explains. ”If it hasn’t, it changes all the little numbers and tries again. It’s like a baby, who at some point learns that he can grasp something with his hands. He finds out it’s convenient and learns from it. That’s how a learning algorithm works, too.”
Practically speaking, you give the computer many examples and instruct the software to train based on those examples. After a while, the network has learned to recognize the Tony’s milk chocolate label on the white box. Then when you show it a new picture of a box with a label on it, the system recognizes that it is the milk chocolate and sticks the correct order information on it.
In the image to the right, you can see an example of image recognition i.c.w. AI. The camera has learned to recognize a car.
Back to classic image recognition: feature matching
‘If new types of labels are constantly coming along, how can a computer tell that it is a label and not the box?’
Yet AI is not always the best solution to this automation issue. That’s because organizations in the print industry don’t have just three categories of labels, but rather lots of types in different sizes, figures, colors and sizes. Especially if you have a similar business to Tony’s Chocolonely, which is to be able to design wrappers, labels or packaging yourself. Like a personalized Pegamento chocolate bar wrapper. This means you might get a thousand categories. ”A typical problem where you don’t want to reach for a learning network, but you have to look specifically at what makes all those labels different,” Thomas points out. ”Training a network that knows what a label is, but doesn’t know what the milk chocolate is, doesn’t work either. Because when new types of labels come along again, the computer can start to see the label as the box. For such an issue, you have to fall back on classical image recognition: feature matching.”
Feature matching
”First of all, we write the software so that it recognizes that anything that is not background must be the white box. And everything that is not white is then the label. We then extract the outline of a label. We cut that piece out and then you get to the feature matching part.”
In a nutshell, feature matching means that an algorithm looks at specific pixels of an image to start comparing them with other images in the database. For example, it grabs a small piece of the wrapper of the Pegamento chocolate bar from the label and compares that pixel group with all the other images in the database. ”Suppose you have a design of a Pegamento wrapper, Nike wrapper and IKEA wrapper, it’s going to look to see if that pixel group appears anywhere. He does that for about 1,000 pixel groups,” Thomas says while using an image to show what it looks like visually. ”He looks to see which image is the best match. That way he can tell, without ever having seen anything beforehand, that something looks alike. And that’s the trick behind feature matching. That’s how the computer knows what order information should be on the box.”
And so organizations in the print industry no longer have to manually check what is in and on the box to put the correct order information on it. This allows an organization to create a higher production process with the same amount of manpower.
For all companies doing anything with custom printing and printing
Another example Thomas mentions during the conversation is photo gifts on which you can have a photo printed. That kind of organization prints photos on products after which they go into the oven by 400. Afterwards, it is often difficult to identify which product belongs to which order. You only see a picture of a family on the product. Based on that image, they have to manually retrieve the order that belongs to it. ”You could also use a camera with feature matching for something like this,” Thomas points out. ”You put the product with the image in front of the camera and the computer looks for a match based on all those pixel groups. Easy as that! With this technique, you can rule out so specifically what it is based on images that it actually becomes as simple as scanning a barcode.”
AI is not the best solution in this case either, because feature matching works better in this issue because each photo is different. Just goes to show that artificial intelligence is not the right solution for all automation issues. Fortunately, there are plenty of other techniques that can be the solution. And together with you, we look for the technique that fits your problem.
Free: image recognition for beginners
Totally up to date on the possibilities of image recognition anno 2022? Download your free white paper here. We will also tell you without obligation whether image recognition is applicable within your organization.
