They are using Google reviews as proxy for customer satisfaction. I really can't take rest of the article seriously after that, knowing anyone can leave a review at Google and there's no guarantee they own the phone.
Sure, 99% satisfaction rate is pretty much impossible, but if all Apple asked was "are you satisfied with iPhone 14" with "yes" & "not at all" options and only surveyed iPhone 14 owners, I can easily see the number being 99%.
I'm quoted in the post. We're using Google reviews as one of several imperfect comparison points to Apple's 99% claim. You're absolutely right about a key downside to Google reviews. The limited information 451 provides on their survey methodology shows that they aren't asking about satisfaction as a yes-or-no style question.
If you want to go really deep on online review manipulation, I'm co-author of this research paper on ratings inflation in online marketplaces: https://pubsonline.informs.org/doi/abs/10.1287/mksc.2022.135.... If you get in touch, I can send you more papers on fake reviews that I like.
America’s favorite holiday, Thanksgiving, enjoys 82% popularity according to a 2023 survey by Statista. Memorial Day comes in second at 77%.
The three most popular movies of all time on IMDB are The Shawshank Redemption, The Godfather and The Dark Knight. They have ratings of 9.2, 9.2 and 9.0 respectively.
These examples are so orange and apples it’s baffling the authors even came up with them.
You compare the most advanced personal pocket computer of all time with a patriotic holiday (assuming patriotism as an absolute value) or a movie about inmates escaping a prison.
Like, really?!
Better would be to ask everybody who watched Shawshank Redemption: ‘were you satisfied with the movie?’. A 9.2 on imdb is not equal to 92% satisfaction due to granularity in the answer options
I really can't take rest of the article seriously after that, knowing anyone can leave a review at Google and there's no guarantee they own the phone.
but if all Apple asked was "are you satisfied with iPhone 14" with "yes" & "not at all" options and only surveyed iPhone 14 owners, I can easily see the number being 99%.
"and only surveyed iPhone 14 owners"
451 Research's own statement on their survey mentions _nothing_ about owning the phone:
The findings are based on a December 14th – January 22nd survey of 3,568 primarily North American consumers from 451 Research’s user insight service, Voice of the Connected User Landscape and its Leading Indicator panel of 25,000 accredited business and technology professionals whose application for inclusion identifies them as having a high wallet share being used for personal technologies, and a high readiness to try new products and services. The service captures consumer and business spending through weekly tracking studies, delivering a continuous view of user perceptions and purchase activity as new products and services enter the market.
And also, how valid would you consider a "satisfied" rating from an iPhone 14 owner who'd never owned anything but an iPhone? There's value in their ongoing satisfaction, but relatively speaking what does it mean? It could just as easily be interpreted as "not annoyed enough to change ecosystems".
And also, how valid would you consider a "satisfied" rating from an iPhone 14 owner who'd never owned anything but an iPhone?
Why wouldn’t it be valid? If somebody says they’re satisfied with something, take them at their word. There’s no shortage of people who complain about anything and everything, especially if they paid money for the thing in question, and especially if they paid a lot of money.
Apple is a single vendor that makes a very small number of models and earned $205.49 billion on iPhones alone in 2022. This excludes a lot of their “services” revenue of which I’m convinced the Lion’s share and then some is also basically just iPhone revenue that wouldn’t exist without specifically the iPhones tied to them. You don’t get anywhere near that for one company on one hardware product division in one year without some seriously high customer satisfaction and repeat buyers.
Yeah, especially when you consider that Apple likely uses NetPromoter as its primary metric for customer satisfaction. NetPromoter only sends out surveys to verified purchasers and the question is a 1-10 scale for satisfaction. Anything 8-10 is “satisfactory”. This is not only plausible but likely and is why the rating is so high. There wouldn’t be very many anomalous responses of people not understanding the question because the question is literally “Would you recommend the iPhone (whatever model) to friends and family?”. That’s not a hard question to understand.
Apple likely uses NetPromoter as its primary metric for customer satisfaction
The article is entirely about a completely different firm, 451 Research, who actually is not and does not use NetPromoter. They state that they use a pool of "high wallet spend early adopters and technical people".
(For added irony, 451 Research says it's own customers are 98%+ satisfied with them...)
Apple may use NetPromoter internally, but on the earnings calls they are citing satisfaction surveys from 451 Research.
In 2018, they shared some of the response categories from the survey. They are customer satisfaction questions "Very Satisfied, Somewhat satisfied." Not recommendation questions.
Hey - I'm the author of the post. Thanks for reading.
I included the Google reviews as a secondary example since it is public information. Google reviews are actually more positive on the iPhone than the third party surveys, which found satisfaction rates in the low 80s for 5G iPhones.
A sample limited to accredited business and technology professionals who consider themselves early adopters and spend a lot on gadgets is, of course, not representative of iPhone purchasers generally.
1. Granurality: as OC mentioned, w/ an overall satisfaction "yes or no" method can reach there.
2. Bias: Pretty sure they did the research before releasing the product to get feedback from a loyal subset base/target customers.
Why would a study include non-Apple users if they have never used or refuse to use Apple products? Wouldn't this be biased if they are included to be "representative"?
I know people who don't use Apple products yet are vocal critics...humans can be highly ignorant creatures
They mention in TFA that even surveying anomalies (people accidentally answering negatively) can’t be represented by a 99% figure.
That said, if I had a survey that asked a couple times in different ways if someone was satisfied with the iPhone and they gave conflicting answers, I’d probably simply throw their answer out for being nonsensical. Maybe that’s what’s going on here: they’re teasing out survey anomalies.
I'm the post author. I suspect aside from conducting extremely unrepresentative surveys, they are also aggressively filtering out respondents. I didn't include that since it is purely speculative on my part, but I'd guess that is part of what is happening.
I'd note that other seemingly innocuous things could have an impact too. For instance, if they surveyed only people who've owned their phone for 90 days or whatnot, you would eliminate the vast majority of people who didn't like it and returned it from the sample.
99% is an extraordinary claim, which is why we think Apple should be more transparent about their methodology.
Thanks for weighing in and your decisions in writing the article make sense to me.
I think it’s important (and probably simply ethical) for people and companies who make such extraordinary claims to explain how they obtained them, but having taken a stats class in college and doing a lot of growth-related work in my professional career I know how frustrating it is to get useful data, especially out of surveys.
iPhone has been around for 15 years so there almost isn't a person on this planet who doesn't know (a) what it is, (b) what the competition is and (c) whether it's a right fit for them.
So if you're surveying existing users you're talking to people who knew before they purchased the iPhone that it was going to be a suitable fit for them. So of course the survey is going to be highly biased in favour of a positive rating.
It seems very obvious to me that a 99% satisfaction rating for an electronics product is extremely unlikely, even given the factors that you noted. Not impossible like the other poster is indicating, obviously it's impossible to prove that claim, but very unlikely.
If you reflect on it and genuinely believe that 99% seems realistic and even "very likely", well, we'll have to agree to disagree.
People buy things all the time that are not a perfect fit, just the best they are able to get at that time. Or they are forced into it for other reason.
99% is not possible. Nothing on Earth has a 99% satisfaction rating - not even breathing.
Do you think that (even if that was the survey) it is an honest survey? Do you not care about the richest company in the world being so blatantly dishonest?
Comments
They are using Google reviews as proxy for customer satisfaction. I really can't take rest of the article seriously after that, knowing anyone can leave a review at Google and there's no guarantee they own the phone.
Sure, 99% satisfaction rate is pretty much impossible, but if all Apple asked was "are you satisfied with iPhone 14" with "yes" & "not at all" options and only surveyed iPhone 14 owners, I can easily see the number being 99%.
I'm quoted in the post. We're using Google reviews as one of several imperfect comparison points to Apple's 99% claim. You're absolutely right about a key downside to Google reviews. The limited information 451 provides on their survey methodology shows that they aren't asking about satisfaction as a yes-or-no style question.
If you want to go really deep on online review manipulation, I'm co-author of this research paper on ratings inflation in online marketplaces: https://pubsonline.informs.org/doi/abs/10.1287/mksc.2022.135.... If you get in touch, I can send you more papers on fake reviews that I like.
These examples are so orange and apples it’s baffling the authors even came up with them.
You compare the most advanced personal pocket computer of all time with a patriotic holiday (assuming patriotism as an absolute value) or a movie about inmates escaping a prison. Like, really?!
That's not what's being compared. It's the rating those things received compared to the rating of the phone.
I think it's fair to include the ratings of the highest rated [whatever] to compare to the ratings of another [whatever].
The author is saying "even people's favorite things don't get such high ratings."
Better would be to ask everybody who watched Shawshank Redemption: ‘were you satisfied with the movie?’. A 9.2 on imdb is not equal to 92% satisfaction due to granularity in the answer options
Also satisfaction doesn’t mean you’d rate something a 10/10.
It’s a completely different methodology!!! Are they comparable just because they express the final value as a percentage?
Quoted by your employee who wrote the post? At your request? To generate traffic for your site?
"and only surveyed iPhone 14 owners"
451 Research's own statement on their survey mentions _nothing_ about owning the phone:
And also, how valid would you consider a "satisfied" rating from an iPhone 14 owner who'd never owned anything but an iPhone? There's value in their ongoing satisfaction, but relatively speaking what does it mean? It could just as easily be interpreted as "not annoyed enough to change ecosystems".
Why wouldn’t it be valid? If somebody says they’re satisfied with something, take them at their word. There’s no shortage of people who complain about anything and everything, especially if they paid money for the thing in question, and especially if they paid a lot of money.
Apple is a single vendor that makes a very small number of models and earned $205.49 billion on iPhones alone in 2022. This excludes a lot of their “services” revenue of which I’m convinced the Lion’s share and then some is also basically just iPhone revenue that wouldn’t exist without specifically the iPhones tied to them. You don’t get anywhere near that for one company on one hardware product division in one year without some seriously high customer satisfaction and repeat buyers.
Very? It’s a subjective position no matter what but they know their own satisfaction level better than you do, certainly.
Yeah, especially when you consider that Apple likely uses NetPromoter as its primary metric for customer satisfaction. NetPromoter only sends out surveys to verified purchasers and the question is a 1-10 scale for satisfaction. Anything 8-10 is “satisfactory”. This is not only plausible but likely and is why the rating is so high. There wouldn’t be very many anomalous responses of people not understanding the question because the question is literally “Would you recommend the iPhone (whatever model) to friends and family?”. That’s not a hard question to understand.
This is not a seriously written article.
The article is entirely about a completely different firm, 451 Research, who actually is not and does not use NetPromoter. They state that they use a pool of "high wallet spend early adopters and technical people".
(For added irony, 451 Research says it's own customers are 98%+ satisfied with them...)
Hey - I'm the post author.
Apple may use NetPromoter internally, but on the earnings calls they are citing satisfaction surveys from 451 Research.
In 2018, they shared some of the response categories from the survey. They are customer satisfaction questions "Very Satisfied, Somewhat satisfied." Not recommendation questions.
Hey - I'm the author of the post. Thanks for reading.
I included the Google reviews as a secondary example since it is public information. Google reviews are actually more positive on the iPhone than the third party surveys, which found satisfaction rates in the low 80s for 5G iPhones.
1. Granurality: as OC mentioned, w/ an overall satisfaction "yes or no" method can reach there.
2. Bias: Pretty sure they did the research before releasing the product to get feedback from a loyal subset base/target customers.
Why would a study include non-Apple users if they have never used or refuse to use Apple products? Wouldn't this be biased if they are included to be "representative"?
I know people who don't use Apple products yet are vocal critics...humans can be highly ignorant creatures
They mention in TFA that even surveying anomalies (people accidentally answering negatively) can’t be represented by a 99% figure.
That said, if I had a survey that asked a couple times in different ways if someone was satisfied with the iPhone and they gave conflicting answers, I’d probably simply throw their answer out for being nonsensical. Maybe that’s what’s going on here: they’re teasing out survey anomalies.
I'm the post author. I suspect aside from conducting extremely unrepresentative surveys, they are also aggressively filtering out respondents. I didn't include that since it is purely speculative on my part, but I'd guess that is part of what is happening.
I'd note that other seemingly innocuous things could have an impact too. For instance, if they surveyed only people who've owned their phone for 90 days or whatnot, you would eliminate the vast majority of people who didn't like it and returned it from the sample.
99% is an extraordinary claim, which is why we think Apple should be more transparent about their methodology.
Thanks for weighing in and your decisions in writing the article make sense to me.
I think it’s important (and probably simply ethical) for people and companies who make such extraordinary claims to explain how they obtained them, but having taken a stats class in college and doing a lot of growth-related work in my professional career I know how frustrating it is to get useful data, especially out of surveys.
Do you mean you can't take Apple seriously? The article is calling out this issue, not engaging in it.
No, it can't. It's utterly impossible for it to be 99% unless it's fake data.
No, it's not. Unbelievably unlikely? For sure.
No it's very likely in fact.
iPhone has been around for 15 years so there almost isn't a person on this planet who doesn't know (a) what it is, (b) what the competition is and (c) whether it's a right fit for them.
So if you're surveying existing users you're talking to people who knew before they purchased the iPhone that it was going to be a suitable fit for them. So of course the survey is going to be highly biased in favour of a positive rating.
It seems very obvious to me that a 99% satisfaction rating for an electronics product is extremely unlikely, even given the factors that you noted. Not impossible like the other poster is indicating, obviously it's impossible to prove that claim, but very unlikely.
If you reflect on it and genuinely believe that 99% seems realistic and even "very likely", well, we'll have to agree to disagree.
People buy things all the time that are not a perfect fit, just the best they are able to get at that time. Or they are forced into it for other reason.
99% is not possible. Nothing on Earth has a 99% satisfaction rating - not even breathing.
Do you think that (even if that was the survey) it is an honest survey? Do you not care about the richest company in the world being so blatantly dishonest?