A Turtle Is a Rifle And a Rifle Is a Helicopter

Wed, Dec 27, 2017

A team of security researchers has developed an algorithm with which they can fool neural network based image classifiers, such as Google’s Cloud Vision, with a remarkable success rate of > 95 %. They can actively change the classification result by generating an image which looks like A but gets classified as B. In their publication it is shown how they successfully fooled several neural network image classifiers to think that a picture showing a couple assault rifles actually shows a helicopter. A picture of a guy on a snowboard and a guy on skis is classified as a dog.

In a nutshell they take a reverse approach and start with an image that shows the adversarial (fake) object, for example a helicopter. This image of course gets classified as such. Then this image is modified over several iterations to look different, for example like a couple assault rifles, while still retaining its classification as a helicopter. In the end they show that they can pretty much make anything be labeled as something completely different with a success rate of more than 95 %. The combinations are staggeringly confusing:

  • a cat gets labeled as an airplane
  • an airplane gets labeled as a deer
  • a deer gets labeled as a truck
  • a lionfish gets labeled as eggnog (this could really hurt ^^)

One thing these images have in common is that they sort of lose their (fake) touch if they get transformed, for example rotated by more than 30 degrees. Also we are talking images so who cares about someone being able to fool a neural network into thinking a perfectly aligned 2D image of a rifle is a helicopter?

Well, they did not stop there as their newest publication deals with robust adversarial examples. In reality this culminates in them printing a 3D model of a turtle which is classified as a rifle regardless of background or rotational angle. Oh, and they made a baseball look to a neural network like an espresso. Check this screenshot of page 8 of the most recent paper:

adversarial thumb

Essentially they went the same route as before, modifying the texture to look like A but be classified as B. Albeit with more tweaks so it keeps up the sharade even when rotated.

I think this is remarkable, especially in light of neural network based image classifiers being used more and more not just in everyday life but also for security purposes. So if you can reliably make a security system think that an assault rifle is actually a helicopter, a teddy bear, or a cup of espresso, just by painting or printing a certain texture on it, you pretty much won the game there.

This also tells us it still pays off having a human brain, for now at least.