On the latest episode of the HomeKit Insider Podcast, Dan Wroclawski from Consumer Reports joins to break down the week’s news, as well as talk about AI smart fridges, how smart devices are tested, and more.
Dan Wrocawski is this week’s guest host who serves as Senior Home & Appliances Writer at Consumer Reports. The nonprofit tests several types of smart home devices, including their security and privacy.
AppleInsider has a storied history with the publication, stemming from an observed anti-Apple bias with its MacBook Pro and HomePod recommendations in 2017.
After some back and forth, AppleInsider was invited to visit the Consumer Reports facilities in New York. We walked through their data-driven testing procedures that at the time, didn’t value design or cohesive ecosystem elements.
Eight years later, now on amicable terms, we get into the work that Consumer Reports does after the news, including how it decides what devices to test, how it measures their security, and how it helps inform consumers about smart home platforms and standards.
Plus, we have some fun talking about Samsung’s latest fridge. The new fridge touts AI-capabilities that may not be as accurate as it hoped in tracking your food.
The full interview is transcribed below and edited for clarity.
In the news, Eufy launched its latest smart home vacuum. The Omni E28 is a Matter-enabled robotic vacuum but the top has a removable deep clean wet vac for cleaning your carpets and upholstery.
Govee also announced its new outdoor “branch” lights. These path lights are available only in the EU but have full color control and IP66 resistance rating with Matter support.
Bird Buddy has launched its newest Kickstarter, raising money to fund the production of its Petal camera and WonderBlocks habitat. It also has its own optional local hub for processing, instead of relying on cloud services.
Finally, we get into Find My-enabled wallets, includes ESR’s Boost Wallet that supports up to 5 cards and 3 bills. Despite so many options out there, the your host’s ideal wallet has yet to be released.
Links from the HomeKit Insider podcast
Interview with Dan Wroclawski from Consumer Reports
Andrew O’Hara: Let’s talk about Consumer Reports and all the stuff you get into. I find this stuff super fascinating.
So tell me a little bit, just in general, about how you got to Consumer Reports. And yeah, let’s start talking about your testing process for smart home devices.
Dan Wroclawski: I’ve been with CR now for seven and a half years, which is kind of mind-blowing. But it’s just a great place to work. It’s really good if you want to have a family, and it’s nice working for a mission-driven organization— not just one solely focused on making money.
We do a lot of advocacy work. When we find problems with products, we go to the manufacturer and try to get them to fix them. It’s just a really rewarding place to be.
For CR, I cover a bunch of different things. Smart home is one of them, but I also do refrigerators, home security— and I just randomly got dash cams, of all things. So I’ll be diving into that world soon.
Andrew O’Hara: I was going to say— fridges, smart fridges cool. That’s smart home. Security systems, too.
Dan Wroclawski: Yeah, I still count that as smart home. And dash cams— I think the thought was, I’m the home security guy, so that’s like security-adjacent. But yeah, I do a lot of fun stuff.
In terms of smart home, we can’t test everything under the sun because we already test thousands of products every year, and the market is so much bigger than that. So we really have to pick and choose efficiently.
We focus on things like smart thermostats, smart locks, security cameras, video doorbells, robotic vacuums, smoke and CO detectors. We also do smart TVs, streaming players, and soundbars— though that falls more under our TV team.
Andrew O’Hara: I have many questions. Here’s a general one: you have all these categories, but how do you determine which specific models to focus on? Take robotic vacuums— there are a million.
iRobot alone has released nine new ones this year, and it’s only May. Then there’s Roborock, Ecovacs, SwitchBot and that’s not even counting infomercial brands like Shark. So how do you narrow it down?
Dan Wroclawski: Great question. We actually have a whole team of market analysts whose job is to assess the marketplace for different product categories. They decide, based on retail placement and sales data, what brands and models the majority of people are buying. That’s what we try to focus on.
We can’t cover everything — it’s just not possible. The testing takes a lot of time and money. But we try to get a sample that represents what people will actually see when they go shopping— whether it’s on Amazon, at Best Buy, Walmart, or Target.
For robotic vacuums, that means covering the latest iRobot models, Roborock, Shark, and some of the others that dominate shelf space and customer interest.
And even beyond that— we don’t do this for robot vacuums, but I can tell you about refrigerators— we conduct national surveys and member surveys. We use that data to create brand-level ratings based on reliability and satisfaction.
Andrew O’Hara: So even if someone’s looking at a fridge that isn’t directly reviewed, they can still use your data to make a decision?
Dan Wroclawski: Exactly. Like with refrigerators, we have maybe 250 to 350 fridges currently in our ratings— don’t quote me on that— but that’s still just a fraction of what’s on the market. The brand-level scores help fill in the gaps.
For example, if LG scores well for top-freezer fridges, and you’re considering an LG that we haven’t reviewed, at least you know the brand has a strong track record.
Andrew O’Hara: Okay, that makes sense. I want to come back to fridges later— especially Samsung’s AI fridge— but let’s look generally at smart home ecosystems. Some consumers don’t care, but for others, it totally changes their experience.
If you’re in Apple Home or Google Home, you’re going to ignore anything outside that vertical. Do you help guide users in that way? And how does that factor into your scoring, especially with Matter making things more complex?
Dan Wroclawski: It’s not a huge scoring factor, but we do account for it. For categories like thermostats, locks, and security cameras, we list which ecosystems the product supports— Amazon’s assistant, Google Home, Apple Home— on the review pages under “features and specs.”
Sometimes we’ll even note if a smart platform integration is a paid feature, which does happen and is annoying. That might impact the smart features score a bit.
But really, our core focus is still the mass-market consumer, and that audience hasn’t fully embraced smart home tech— yet. It’s improving, and Matter should help, but adoption has been slow.
We do allow filtering by ecosystem, though. So you can go to the ratings page for smart locks and filter for “Apple Home” compatibility, for example, and just see the models that work within your ecosystem.
It’s something we want to improve on, because with Matter, everything is becoming more connected. It’s definitely on our radar.
Andrew O’Hara: True. And I feel like you have more responsibility than I do. My audience— whether it’s HomeKit Insider or AppleInsider— already cares about HomeKit and Matter. It’s easy for me to narrow the focus.
Andrew: Everything I cover is basically going to be that, but for you, like you said, it’s just that wide audience. Most of them don’t know what Matter is, but I feel like it’s your reviews they’re going to read that are going to have to explain to them why they should know what it is and why it should be important and what it means.
I mean, you know, from listening to the show, you’ve talked so many times that Matter has kind of a branding problem — like, what is it? Who knows!
You go into the store and things say “Matter” on the box, but no one knows what that means, and they’re doing very little to improve that. So I feel like right now the only way customers know is from reading reviews and kind of pushing that news out there.
Dan: Yeah, I mean, it’s a big problem. We do try to — I don’t want to say promote — but educate our readers on all these different smart home integrations. We generally note them in all the reviews. We have buying guides for all these different product categories, and we’ll talk about these features in those guides.
And yeah, we do try to educate our readers. I have a big, long Matter guide explaining all the nitty-gritty — what Matter is, what it works with, what devices it supports, and all that useful stuff. Because yeah, it’s a lot, and it’s a lot for the average consumer to take in.
Andrew: For sure. One of the big things people are always curious about — fortunately, HomeKit stuff is very locked down — but when you get into like Amazon, the certification process is a little more lax. How do you quantify and test things like security and vulnerabilities there?
I mean, when we’re looking at locks and some of those, you’ve got to do something. So how do you do security?
Dan: Yeah, so we have a dedicated team that solely does privacy and security testing on a bunch of products. Right now, it’s mainly security cameras and doorbells, robotic vacuums, smart TVs, streaming media players like Apple TV, and Wi-Fi routers.
We follow something called The Digital Standard, which is a testing standard that CR and a bunch of other advocacy groups helped create. There’s a privacy and a security portion to it.
For privacy, we look at all the documentation for a product and a brand — privacy policies, terms of service, other documentation — and collect it all. We hunt it down because it’s not always easy to find. And we do the reading that everyone is told to do and never does.
We use that to understand: What are they doing with your data? Are they selling it? Are they using it to improve their products? All that kind of stuff. We then turn that into a data privacy score.
For security, we test if the product is using encryption. We do penetration testing to look for vulnerabilities. We’ve actually found some. There was a pretty egregious no-name video doorbell last year being sold on Amazon and Walmart that was very easy to hack.
If there’s a vulnerability, we try to work with the brand to fix it. We note that in articles or reviews to make people aware. Most devices we find are actually fairly secure — that’s good. It’s privacy that’s quite lacking, even with some Apple Home products. Privacy is okay but could be better.
So yeah, it’s a great resource, and I don’t think anyone else really does that kind of work.
Andrew: On the Apple Home side of things for privacy — like with Eve, for example, since we talk about them a lot — every Apple Home product, you can use it solely with the Apple Home app. How much do those privacy concerns still come into play if you’re not using the brand’s app? Like, if you just scan and go?
Dan: That’s a really fair point. That’s a great strategy to avoid privacy and security problems. HomeKit is very secure and private, which is great. If you solely use it through Apple Home and don’t have to download the app — which sometimes you do, just to get the code, especially with Matter — that’s a more secure and private route.
If the code is on the device and you can just use it directly with Apple Home, that’s better.
Andrew: How do you feel as someone who tests all of these things about cameras on robotic vacuum cleaners? I have mixed feelings. I like the idea of a roving security guard — we had an attempted break-in years ago, so we’re very security conscious now — but I also don’t love the idea of a robot roaming and recording everything.
Dan: I’m not a huge fan either. I’m not a huge fan of cameras in the home in general unless they have privacy covers. I do need them — I have little kids and need to know what happened when something goes wrong.
But I go only with models that have privacy covers so I can turn them off when I don’t want them recording.
Andrew: When I tested the Roborock S8, they ditched the camera — then brought it back. I missed it! It would take a photo when it encountered something unusual, and that was helpful. And I liked using it to find the dog or monitor the vacuum remotely.
My wife is wary about cameras, so we don’t have them in bedrooms, etc. But I liked that Roborock requires you to manually enable the camera — holding buttons on the bot, setting a passcode. It can’t be remotely accessed, unlike some other brands. I feel like that’s progress.
Dan: That’s smart implementation. There are good use cases. Object detection is one. iRobot used to talk about detecting animal droppings and navigating around them. That’s probably harder to do with LiDAR than with vision-based AI.
Andrew: Let’s wrap up and talk about fridges. I’m curious about your take. On one hand, adding AI and cameras seems silly for a major appliance — it’s expensive, should last years, and could get outdated fast.
But I’ve been in the grocery store and loved being able to look inside via the Samsung app. Plus, my daughter redecorates the fridge screen monthly with a collage of photos — it’s fun!
So I’m torn. There’s privacy, unnecessary cost, short software lifespan… what do you think?
Dan: It’s tough. They do have some fun and useful features. Even using the screen as a kitchen TV is nice. But yes, fridges are expected to last at least a decade, and Samsung has already stopped providing updates to some Family Hub models that were only 5-6 years old. That’s only half the fridge’s life.
So I’d caution people. Think about whether you really want that. It likely won’t be supported for the fridge’s lifespan.
The internal cameras are helpful — like when you’re grocery shopping. I tested one of the Bespoke AI fridges that can ID food and log expiration dates. It worked about 50% of the time. It struggled with packaged items, but did okay with clear containers — like a container of strawberries.
Andrew: What about the crisper drawers? Are there cameras inside?
Dan: No, but it can see through clear drawers — just the front. It depends on the model. One has three side cameras. The newer one has an overhead camera that records as you put items in. But the angle isn’t great — I found myself having to “show” the item to the camera as I placed it inside. No one will do that.
It also doesn’t see what’s on the door shelves or off to the sides. It’s a cool idea, and I think it will work one day. But it needs more cameras and a lot of improvement.
Andrew: I have one of those older models. Probably lost support already! It’s supposed to ID items but often gets them wrong — like telling me I put in tamales, grape tomatoes, and dog food… when I didn’t. But man, I want that to work.
Imagine just asking the fridge, “What should I make?” and it tells you based on what’s inside. That’d be awesome.
Dan: Yeah, that’s the dream. It could cut down on food waste and save money. That would be really useful. But it’ll take more cameras, more smarts, and it doesn’t even address the freezer yet!
Andrew: Exactly. Also, these things need better processors. They cheap out on it because no one knows what chip is inside a fridge. Then new features can’t be supported. Mine freezes sometimes just trying to animate the screen shake when I close the door.
If you’re paying $2,000+, spend a bit more on a decent processor. Especially if it’s supposed to last 10 years.
Dan: I agree. They’re underpowered. And margins are tight, so they try to cut costs where they can. But that shortchanges consumers long term. They need to think bigger.
Andrew: Well, I feel like we could keep chatting forever, but we should wrap up. Anything you want to leave us with, Dan?
Dan: Just that Consumer Reports is a great resource — whether you’re looking for home appliances, tech, baby gear, or cars. I’m really trying to make it a great smart home resource with reviews and content for everyone. So check it out. And don’t forget the promo code at cr.org/homekitinsider — it’s a great deal, and you’ll probably make back your money in product savings.
Andrew: Awesome. Thanks so much for hanging out, Dan. Come back anytime!
Those interested in sponsoring the show can reach out to us at advertising@appleinsider.com
Keep up with everything Apple in the weekly AppleInsider Podcast. Just say, “Hey, Siri,” to your HomePod mini and ask it, and our latest HomeKit Insider episode too. If you want an ad-free main AppleInsider Podcast experience, you can support the AppleInsider podcast by subscribing for $5 per month through Apple’s Podcasts app, or via Patreon if you prefer any other podcast player.
