How old are they? A new website guesses the ages of famous faces in our collection—with revealing results

This past week, everyone’s favorite online distraction has been How-old.net, which quickly guesses the age and gender of people in images you submit. It was created by Microsoft engineers to showcase the algorithms they’ve built, like face detection software. But it raises at least as many questions about our expectations of aging and gender, and how they’ve changed over time, as it does machine learning. Because it’s spookily accurate—except when it’s not.

We uploaded a dozen or so faces from the MIA’s paintings collection, going back to the Renaissance. Here’s what happened.

Screen Shot 2015-05-06 at 11.15.50 AM

Poor Sarah Allen. Her portraitist was not one for idealization—she was painted exactly as she apparently was in real life: a sturdy, masculine-looking wife of a Boston businessman. Indeed, such realism may have enabled Microsoft to guess her age perfectly: She was 34 at the time this was painted around 1763. However, she was reportedly not a man.

Screen Shot 2015-05-06 at 11.40.02 AM

How does the algorithm determine gender? If it has anything to do with a smooth or whiskered face, that may explain why Microsoft added insult to injury for this unfortunate boy—he was dead when this was painted as a memorial in 1856—by calling him a girl. That said, they got the gender right on the equally young child in the lead image (Portrait of a Girl), but pulled a Benjamin Button on her, fast-forwarding her age right to retirement.

Screen Shot 2015-05-06 at 11.06.38 AM

She’s been dead a long time, about 3,000 years. But Lady Teshat shouldn’t look a day over 15 or 16. She was the first mummy to be X-rayed, back in 1918, and was only the second to get in a CT scanner. The X-ray determined that was she was only 15 or 16 when she died, but perhaps she packed in 35 years worth of living: Mysteriously, there was a second skull in her coffin, between her legs, and romantics have speculated that she was buried with her lover.

Screen Shot 2015-05-06 at 11.43.00 AM

It’s one of the more squirm-inducing images on the MIA’s walls, the German painter Otto Dix living up to his reputed claim that he would “either become notorious or famous,” and succeeding at both. Dix is best known for his unflinchingly grotesque scenes from the trenches of World War I, and although this portrait is intended to show a girl innocently shameless in her natural state, there may be something dark about her gaunt stare that caused Microsoft to guess she was 74.

Young Girls in a boat

Two pretty young women in a boat. Why does one appear seven years older? She doesn’t have quite the baby face of her friend, or maybe brunettes are just more mature.

Mother Child

Oh boy. Or girl, as Microsoft sees it. Maybe they know something we don’t. Maybe God is a woman. But that wasn’t the artist’s notion when he painted this Madonna and Child in the 1300s. Nor did he intend Jesus’s mother, Mary, to look like a guy. But our ideals of beauty have changed since the early Renaissance and perhaps that dissonance is mucking up the digital machinery. That said, this was a natural-looking image for the time—Mary is not glamming it up, she’s a mother who recently gave birth in a stable on the road while a king plotted to kill her child, and she looks it. If that was your story, you might not look yourself either.