How Accurate is Google’s AI Overview?

We’ve all seen Google’s AI Overview.  It has been available in the UK since August 2024.  Whenever you now do a Google search, it is the first response that is offered to you, appearing at the top of the screen, and after a brief delay––searching; generating; little blue star––which gives the suggestion that Google has taken your query onboard with all due seriousness and has spent a quantifiable amount of its busy time duly thinking before giving its considered answer.  Or perhaps I am reading too much into this human/machine interaction?

Google AI Overview’s description of itself states:

“Google AI Overviews are a feature in Google Search that provides AI-generated summaries of information, aiming to offer quick and comprehensive answers to user queries.  These overviews appear at the top of search results when Google’s systems determine that a generative response would be particularly helpful.  They are designed to quickly summarize information from a range of sources, including web content and Google’s Knowledge Graph.”

But just how accurate is the answer that Google’s AI Overview provides?  I decided to do a test to find out.

What I needed was a subject that I know a Mastermind-amount of knowledge about, so that when I asked Google’s AI Overview a question about it, the chances were that I would already know the correct answer.  But what subject?  I struggled for a topic, until it suddenly hit me.  A subject that I knew everything about?  More about that anyone else on the planet, in fact.  The subject?  Well, me, of course.

My first question was quite simple: Who is Simon Turner-Tree?  Frankly, I was fascinated to find out.

This was Google AI Overview’s response:

“Simon Turner-Tree is a writer and artist known for his observational essays and artistic practice, which often explores themes of nature, urban landscapes, and everyday life.  He’s particularly drawn to the details of the natural world and the textures and patterns found in both nature and man-made structures, like graveyards and street ironwork.”

I quite liked the sound of this Simon Turner-Tree.  It didn’t sound much like me––observational essays, perhaps; artistic practice, not––but it sounded like someone I might like to be.  It felt a bit like reading a horoscope.  A description that could be made to fit where you wanted it to, and not where you didn’t.

I decided to ask Google the same question but, this time, I put my name in inverted commas to narrow the potential scope of the search: Who is “Simon Turner-Tree”?

This was Google AI Overview’s response:

“Simon Turner-Tree is not a widely known figure.  The provided search results indicate he is an author, likely of books on personal reflections and observations about daily life, as suggested by the title of his book “This Pedestrian Life”.  He focuses on the absurdities and irritations of everyday living, exploring topics like shredding, keychains, Secret Santa, and even COVID-19 facemasks.”

“Not a widely known figure.”  Start with an insult, why don’t you?  Rather rude, Google AI Overview.  However, I am forced to concede the rest of the information is largely accurate, and nice of them to include a plug for my book, This Pedestrian LifeOne of my books.

The trouble is, I am not so sure that I like this description of myself.  He sounds a bit of a pedant.  Despite its dubious accuracy, I preferred the bloke described by my first search.  The chap who is doing artistic things in graveyards with street ironwork, rather than the one writing about keychains.  But, who is this unknown Simon Turner-Tree.  A Simon Turner-Tree of the future, or one from an alternative digital universe who will never live at all?

© Simon Turner-Tree

Simon Turner-Tree discovers he has multiple digital lives.

One comment

Leave a comment