Can a computer be sure and still be wrong?

Have you ever watched a phone camera draw a little box around a smiling face? Some computers can look at a picture and say what is in it. They can even tell you how sure they are. So can a computer be really sure and still be wrong?

Let's picture one computer program with a special job. You show it a photo of an animal, and it tells you the animal's name. We can follow this program and see how it works.

First, the people who build the program show it thousands of photos of dogs and cats. Each photo comes with its right name, dog or cat. This is called training, which means learning from lots of photos that already have the right names.

During training, the computer looks for what the dog photos share, like a long nose and a big mouth. It does this so it can spot a dog later in a photo it has never seen. The cat photos teach it round faces and short noses in the same way.

One day, a girl snaps a photo of a fox in her garden and shows it to the program. In all its training, the computer has never seen a single fox. The fox has a long nose, a big mouth and soft reddish fur. What do you think the computer will say?

It says dog! The computer can only pick from the names it learned, so it picks the closest one. A fox looks more like a dog than a cat, so dog wins.

Next to its answer, the program shows a number called its confidence, which means how sure the computer says it is. Here the confidence is ninety-five out of a hundred. That number grows bigger when a new photo looks much more like the photos of one name than the photos of the other.

The fox's long nose and big mouth match the dogs from training very closely. So the computer finds a strong match, and its confidence comes out high. To the computer, this fox simply looks like a dog.

A high confidence only tells how much more a photo looks like the dog photos than the cat photos. It does not tell whether the answer is true. The computer cannot step outside and check the real animal, and it cannot notice that foxes were missing from its training.

Training and confidence. Because people train a computer with only some examples, its confidence just shows how much something new looks like them, so it can be very sure and still be wrong.

Now your own clever mind can rest, warm and cozy, until tomorrow's new questions.