No CSI Award for J.D. Power
Another customer satisfaction award season has come and gone in the marine industry, and it seems time for someone to comment on the emperor’s clothes — to question whether the presence of J.D. Power in our industry is more destructive than constructive.
No one argues with the value of survey information, and regular customer feedback is no doubt among the most valuable forms of market research any manufacturer or service provider can commission. In this respect, the emergence and growth of the National Marine Manufacturers Association CSI Survey Program is a laudable step forward, giving even small boat companies access to a professionally designed survey program at a reasonable price.
But does the value of new-owner surveys diminish when the results are used to pit one manufacturer against another, or to publicly declare one winner in a category and thereby create another obstacle that every other manufacturer has to sell through?
What started in autos with the J.D. Power model of naming names and issuing trophies has now spread to industries as diverse as health care, telecommunications and travel. The J.D. Power trophy is now seen so often and in so many places that American consumers are becoming numb to its presence or anything special about its winners. The fact that the U.S. marine industry has seemingly welcomed J.D. Power with open arms is regrettable.
Should one manufacturer get to claim they have been judged the best when their margin of “victory” over another manufacturer was one point on a thousand-point scale? Should the formula for the trophy award be a complicated (blended) mean of several questions with different weights assigned to each one? Should any winners be declared when the sampling margins of error are large enough to drive an express cruiser through? Should boat companies be publicly admonished with the dreaded two-circle (among the rest) rating when their results are within the margin of error from other companies rated above them? And should award winners be able to claim broad, company-wide superiority when the award they won was for a specific boat type or engine technology?
What has been the overall net gain of J.D. Power being in our industry for the past five years, providing data largely duplicative of the NMMA and internal CSI survey programs? There have no doubt been significant quality gains that translate into fewer problems and happier boaters. This, however, is a testament to thousands of dedicated professionals in engineering, manufacturing, quality control, and management, who are doing their job better than ever at boat and engine companies around the world. This is also a testament to an industry that has stepped up to certification, and to using internal survey results for continued improvement.
Have overall boat sales improved now that we live in a J.D. Power-measured world? We know the answer to that question is, unfortunately, no. We seem to be stuck around the same 300,000-unit mark that we were at when J.D. Power first entered our industry. While we are still searching for the net gain for the industry as a whole, the gain for J.D. Power is clear. They have been paid handsomely by boat and engine manufacturers over the past five years, money that, in the alternative, could significantly enhance Grow Boating, or some other effort to try to grow the pie instead of fighting over who gets what slice.
As a market research professional who has surveyed boaters for more than 20 years, I know how most traditional market research companies work. We tend to have long-term, strategic partner relationships, with a vested interest in our client’s overall success. We try to provide solid information with statistically significant sample sizes, and we succeed or fail on the value of that information alone.
By contrast, J.D. Power has little vested interest in individual boat or engine manufacturers doing better. As long as there is a winner in each category each year willing to pay for the trophy publicity rights, their business model succeeds. Today’s winners can become tomorrow’s also-rans, and it makes no difference to the business model. Their annual studies give the average boat company feedback on 150 customers, a sample size so small with such a wide margin of error that comparisons (both internal and external) can be meaningless. For any boat company, this sample size also makes the real value of survey information, drilling down to results by model or dealer, impossible.
J.D. Power certainly has the right to survey boat owners using a methodology of their choosing, and then to sell those results at a price they see fit. And marine OEMs certainly have the right to buy this study, making a value judgment about the information they receive for the paid price. But once study results go public with named winners and losers, we have a right to make a value judgment about the fairness of the process and its effect on our industry. We face a constant battle to grow boating, and to fight for our share of recreational spending. We need to think long and hard about a process that keeps providing ammunition that we use to fire at each other.
