Visual Intelligence first impressions leave me excited for the future

I know, it’s just a developer beta, and if anyone from Apple asks then my Visual Intelligence first impressions are purely offered in the context of its future potential. But I have to say that trying it leaves me excited for that future.

The actual Apple Intelligence part is currently relatively limited. In most cases, you get a choice between asking ChatGPT to describe to you what you are seeing and a Google search …

The actual Apple Intelligence parts

From what we can tell, Apple Intelligence performs a few functions directly:

• Summarising text seen in the scene
• Getting information on businesses, from Apple Maps
• Recognizing dates and times in text, and opening Calendar for that slot

But for object recognition, the feature currently appears to exclusively use ChatGPT and Google. In my limited experience so far, Google is by far the most impressive.

Image recognition using Google and ChatGPT

I tend to research most of my non-trivial purchases rather extensively before pressing the button. I read reviews, ask advice from friends, and generally interrogate Google to the nth degree.

So if I see something in use somewhere and want to check it out, I need specifics: telling me that I’m looking at a compact bean-to-cup coffee machine or similar wouldn’t really cut it.

So how does Visual Intelligence v0.1 do? I tested it on a few things around my home.

First up was my Sage Combi Wave 3-in-1, a combined microwave, oven, and air-fryer. This is sold in the US under the Breville brand. Visual Intelligence immediately identified it (first match, top right). It did show a competing product too, but the rest of the top hits were all correct.

Next up, my Sage Barista Touch coffee maker. VI was a little less certain here, suggesting it might be either that or the far more expensive Oracle Touch model from the same brand, but it doesn’t take much to determine which of its two suggestions is correct. (While it seems to throw a third model into the mix, that’s a classified ad from a site with a mislabelled photo.)

Aarke Carbonater II sparking water maker? No problem.

It struggled a little more with a Reidel O Wine Tumbler, but that’s forgiveable as there are a bunch of imitators. It mostly only showed the original and direct copies.

Moving onto the living room, it identified this as an Eames Lounge Chair, and since it’s a millimeter-accurate replica, I’ll forgive it for not being able to distinguish it from the real thing.

I tried it on some original art by lesser-known (but not unknown) artists, without success. However, when it came to a print, it had no problem identifying it.

With my Naim Mu-so, it assumed it was the second-generation device rather than the first, but given that they are visually identical and almost every web reference is about that model, that’s entirely understandable.

I tried it on a bunch of other tech around my home:

• Original HomePod
• Echo Dot
• Philips Hue Switch
• Netatmo Smart Doorbell

It was right about all of these.

This is a great start, and there’s much more to come

My first taste of AI-driven image-recognition tech was when I tried Ray-Ban Meta glasses, though it did require some careful phrasing and/or follow-up questions to get the most from them.

Visual Intelligence already seems better than this, and it’s that pace of development which most excites me. With multiple companies competing in this field, I think we’re going to see a huge amount of progress made in a very short time.

Sure, there will be embarrassing mistakes, as with all AI tech, but the very nature of this technology is that it learns from every interaction.

I don’t think it will be that long before we can do things like point an iPhone at a random product we see in a store or on the street and instantly get an average star rating and a decent AI summary of the reviews. Usually it’s Apple telling us the company can’t wait to see what we do with its new tech, but in this case it’s very much the other way around.

If you have access (which took a couple of weeks in my case), what are your own first impressions of Visual Intelligence? Please share your thoughts in the comments, and check out Fernando’s video below.

Photo: 9to5Mac