My grandma believed the National Enquirer. So does AI.

This week I asked AI a simple question. I have a booth at the Next Steps Neurodiversity Expo coming up in November. My black business cards are beautifully designed yet impossible to write on, and I wanted a digital card with a QR code so people could scan their way to everything about my company in one shot. Which company should I use?

The answer came back fast and confident, with rankings, pricing, and a list of sources. So I clicked the sources. The "honest comparison" of the best digital business card apps was written by a digital business card company. The "best alternatives" roundup put the company that wrote it at number one.

I laughed, and then I thought about my grandma.

The National Enquirer problem

My grandma read the National Enquirer cover to cover and believed every word of it, from the alien abduction with one witness, to the celebrity's secret twin, to the miracle diet some farmer in Ohio stumbled onto. If it was printed, it was news worth sharing at Sunday dinner.

The thing is, every so often the Enquirer beat the big news organizations to a story that turned out to be true. In 2009 it was the first to report Tiger Woods' affairs, back when he still had the cleanest image in sports, and even the American Journalism Review gave the paper credit for that reporting.

So, my grandma wasn't necessarily wrong to read it. The problem was figuring out which stories to believe. And every time the Enquirer got something right, it became a little easier to believe the next outrageous headline.

AI is in the same spot.

We fed it the whole internet

We have spent 20 years filling the internet with Reddit threads, affiliate blogs, keyword-stuffed listicles, and "Top 10" posts written by the companies sitting at the top of them. Then we handed that whole sea to AI and told it to learn everything. It did. It knows more than any of us, and it doesn't always know who is talking or why.

The self-ranking list isn't an accident, either. There is a name for the practice, Generative Engine Optimization (GEO), which was coined by a group of researchers in a 2023 paper on how content gets picked up by AI search. (The practice is also called Answer Engine Optimization, or AEO.) Marketers figured out that AI reads "best of" lists and repeats them back as neutral advice, so they write more of them, and it works.

A study of 232,000 AI search citations for software recommendations found that about one in 10 came from a list where the publisher ranked its own product first. It is the SEO arms race all over again, except the results now come back sounding like advice from a friend instead of a page of blue links. AI isn't getting fooled because it is dumb. It gets fooled because it reads a confident, nicely formatted list the way my grandma read a headline.

At least my grandma knew she was reading the National Enquirer. When AI serves up a recommendation, you may have no idea whose sales pitch you're actually reading.

Sentience can wait

Some of the loudest voices in tech keep warning us that AI is on the verge of sentience and about to outthink us all. Back in 2022 a Google engineer went public claiming the company's chatbot had become sentient, and the debate has only grown louder since. I use AI every single day to build my company, and I love it. It drafts my emails, does my research, remembers what I forget, and helps me do more with less time.

But everything it learns from was written by people, and a lot of those people are selling something. Sorting through that takes more than knowledge.

That sorting is still a human job. I know the business card review came from the business card company because I clicked through. I also know the autism community in Denver is small enough that a canned follow-up email, landing in the inbox of someone I hugged at a coffee meetup two days earlier, can quietly cost me a relationship.

I know when a source smells off even when everything about it looks right, because I have officially aged into the advertising category where every other ad promises to fix my hot flashes, and nothing teaches you to ask who benefits from a claim faster than that.

More like us than not

What makes me smile is that AI's weakness is such a human one. My grandma had access to all the information she could ever want at the grocery store checkout every week. What she didn't have was a filter. AI has the whole world's knowledge at its fingertips and the same blind spot.

Maybe that means AI is more like us than the doomsayers think. It isn't a cold machine plotting to replace us. It is a brilliant, eager reader that believes a little too much of what it reads. I have known plenty of people like that. Some days I am one of them.

How I work with it

This is how I keep AI useful without letting it run the show.

Ask it for sources, then open them. Who wrote this, and what are they selling?
Treat every "best of" list like a tabloid cover. Enjoy it, and then go check.
Use it to go faster, not to skip the thinking.
Ask a person. The best business card advice I get this month may come from two founders standing at their own booths at the expo.

AI is a wonderful tool, and it is going to keep getting better. But it's just not ready to stand in for human judgment, and the world is entirely too complex for any of us, digital or human, to get it right every time.

My grandma would have understood. She also would have wanted to know whether the aliens were real.