More than a year ago, a company called the American Productivity & Quality Center, or APQC, asked Warranty Week to help publicize a survey that would delve into after-sales services.

The "Service After Sales" survey ended up asking 27 questions on a variety of topics that touched upon everything from customer credit to call centers. Several questions asked specifically about product warranties or extended warranties.

This was what APQC calls a "Media Survey," which is essentially a shortened version of the longer form surveys that the company is known for. The advantage is that surveys can be completed and analyzed faster. The disadvantage is there isn't much depth to the responses -- no explanations or details behind the simple yes or no answers.

Speed vs. Depth

Cathy Hill, APQC's director of global alliances, said it's a simple tradeoff between depth and speed. "When we went to this shorter version, we wanted to stimulate conversation out there, and to put some questions out at a high level, so people could answer them fairly fast, and we could have some statistics that would be talking points that might lead to other discussions," she said.

Rachele Williams, APQC's Service After Sales practice leader, said that while this brief format resulted in numerous talking points, it didn't return a whole lot of explanation or reasoning behind the responses. "People answered and didn't provide any context around it," she said. "They just gave us their actual data."

Although the survey was posted on the APQC Web site from October 2006 until August 2007, only 20 companies completed it. "Normally, we get a lot more responses than this," Hill said. But the problem wasn't so much the number of questions as it was the kind of expertise required. "One of the things I heard back from a lot of the people who participated was it was a little bit more difficult because it wasn't a clear process owner. So it was multiple people having to respond to different questions. And even though the survey was quite short, it still might have required several bodies to do it. And that always reduces the number of responses," she said.

Williams said that some organizations that responded did in fact have one person in charge of service after sales. For them, it was much easier to fill out the survey, because one person could conceivably answer all 27 questions. But most of the others had to bring in different people for different sections of the survey. "That was one of our big lessons learned."

Simple Responses to Complex Questions

Williams said the low number of respondents was a little disappointing. "But the numbers that you see there have been statistically validated at any rate. So we feel confident in them," she said. But while the answers that came back told a lot about "what" and "how many," they didn't have a lot to say about "why." And it wasn't always possible to make sense of how a given company's answer to one question related to its answer to other questions.

"Unfortunately, with the number of respondents being so low, we didn't have the data to do cross-tabulations like we would do with our other surveys to generate findings," Williams said. "So I think these survey results are at a higher level than what we usually see. But I think they can generate some good high-level metrics data that people are very interested in." And hopefully, at some future date that will spur them to do an even deeper dive into these questions within their own organizations, she suggested.

With such a low sample size, the danger is that any one company's responses take on tremendous importance. And if one respondent gives an absurd response, for instance saying that it takes 120 days to ship replacement parts, it skews the answer tremendously. But Williams said APQC is always on the lookout for answers that lie far outside the norm. "We would have taken out any hugely significant statistical outliers on all of the data. Anything that was above or below two standard deviations on the data, we would have eliminated," she said.

Respondents came from a variety of industries. In the chart below, we've converted the percentages into actual numbers, because with only 19 responses to this particular question, it's hardly useful to say that 15.8% were in retail when three out of 19 is just as accurate and much more concrete.

The most respondents came from the industrial products industry, followed by retail. Then there was a three-way tie between high tech, automotive, and electronics. There was one aerospace company, one service company, and three that didn't fit into any of these industry classifications. None of the respondents were identified by name.

Geographically, all but one respondent came from North or South America. The only exception was a company that was based in the Asia-Pacific region. However, it wasn't clear if the respondent was located in the Asia-Pacific region, or if their employer was merely headquartered there.

Seven companies reported total revenues below $100 million, while four reported revenues above $1 billion. The rest reported total revenues between $100 million and $1 billion.

Similarly, seven companies reported having less than 150 employees, while six reported more than 2,500. The rest were reported between 150 and 2,500 employees.

One question had 16 separate parts. Or more precisely, it asked respondents to disclose 16 different metrics for their company, ranging from the average number of days it takes to ship a replacement part to the average percentage of parts on back order at any given time.

Ranked From Lowest to Highest

These answers were collected and ranked from lowest to highest. Usually, the lowest answers were the best, or more precisely, lower numbers were more desirable than higher numbers. In some instances, an answer of 0 days or 0 dollars or 0 percent was the best answer of all.

Williams said the APQC analysts then looked for three benchmarks within each metric. The benchmark for "Top Performers" was the company at the 25th percentile, meaning that one-quarter of the responses were above and three-quarters of the responses were below that figure. The median performer was at the 50th percentile -- the statistical middle, at which half the respondents were above and half were below. And the "Bottom Performers" benchmark was the company at the 75th percentile.

For instance, 19 companies provided a figure for the percentage of total units returned for reasons other than product damage or failure. Top performers were those that reported a figure of 1.0% and under. Bottom performers were those that reported 30% and over. The median figure was 6.2%.

So that means four companies were under 1.0% while four companies were above 30%. The middle company was at 6.2%, and nine companies were above this figure while nine were below it. In this case, because there were exactly 19 responses, we can also assume that four companies were between 1.0% and 6.2%, while four companies were between 6.2% and 30%. The three companies whose data became the benchmarks separate four groups of four companies, adding up to precisely 19 data points.

The percentage of repair incidents identified as "no fault found" was similarly varied among respondents. The top performing quarter were those with a figure of 0.9% and under. The bottom performing quarter were those with a figure of 20.8% and over. The median was 5.0%.

Warranty Claims Rate

Warranty costs as a percentage of total product revenue came in at 0.1% and under for the top performers and 2.7% and above for the bottom performers. The median was 1.2%. But there were only 14 responses for this particular part of the survey question.

Of course, Warranty Week has tracked this figure for 800 companies on a quarterly basis since early 2003. At the end of 2006, the company at the 25th percentile had a claims rate of 0.4%. The company at the 75th percentile had a claims rate of 1.5%. The company in the exact middle had a claims rate of 0.8%.

However, the weighted average, which is mathematically different from the median because it divides all 800 companies' warranty claims by all 800 companies' product sales. That figure was 1.7% at the end of 2006. Over the past five years, the weighted average for claims as a percent of product sales has remained in a range of 1.6% to 1.8%.

Of course, that's the advantage of working with larger sample sizes over longer periods of time. With the small sample size, half the respondents reported claims rates between 0.1% and 2.7%. With the large sample size, half the companies were between 0.4% and 1.5%. But then again, we also know that the largest populations within the larger sample size come from industries such as medical and telecom equipment that aren't known for high claims rates. We also know that the highest sales totals come from industries such as passenger cars and personal computers that are known for their high warranty costs.

Two other metrics showed a very high degree of variation, perhaps also due in part to the small sample size. The "fully-loaded contact center cost per customer contact" was found to be $18.50 for the top performers, and $78.70 for the bottom performers. The median was $38.00.

Extended Warranty Data

The figure for extended warranty and service contract revenue as a percentage of product revenue was found to be 0.0% for what were called the top performers, and 7.3% for the bottom performers. We'd suggest that although extended warranties may have a bad reputation in some circles, revenue from these sources is actually a good thing. So let's reverse the polarity of that metric and say that the top performers were at 7.3% and above, while the bottom performers were at 0%. The number of respondents was 12 and the median was 0.3%.

Further down the list, a question was asked, "Does your business entity recover credits for returned defective parts under warranty from suppliers?" Eighteen people responded, and surprisingly, only five said no. Perhaps there needed to be a second question asking about the success rate of these supplier recovery efforts? For while most companies may make the attempt, we'd suggest that there is far less than a 100% success rate.

In another question that required a simple yes/no answer, APQC asked respondents what value-added after-sales services their companies offered. Respondents were offered a list of 14 possibilities.

Among the leading yes responses: delivery/installation/setup (55%); actual installation, repair, or maintenance service (50%); product upgrade, refresh, or customization (50%); and financing/credit (45%).

Extended warranties came in at 40% yes. Premium service contracts (i.e. 24x7 technical support) came in at 35% yes. In another question, APQC asked respondents whether their company offered either aftermarket services or extended warranty sales, and a yes answer came back from 12 out of the 19, a positive percentage of 63.2%. However, keep in mind that the latter question had an either/or format, which probably accounts for the different outcome.

For all of these after-sales services, respondents were asked what their companies do if their channel partners offer similar services. Six said they don't use channel partners. Five said their channel partners don't sell similar services. Three said they will not offer services that their channel partners offer. And two said they will not do so within a given channel partner's territory.

Two questions received unanimous yes answers. The first was "Does your business entity use a returns material authorization (RMA) system?" The second was "Does your business entity provide value-added service offerings?"

Next Steps?

So what happens next? "It depends on the interest level," Hill said. "If people want to learn more and it has created discussion, then we're all for putting out something more detailed. Our surveys are funded up front, mostly by professional service firms. So if there is interest, we'll look for partners and see if we can put something out there." This particular survey was sponsored by IBM Global Business Services, she noted.

Williams added that APQC also gets customized research requests from some of its members. And that current membership list includes Ford, HP, IBM, Caterpillar, Deere, United Technologies, Boeing, Cisco Systems, Sun Microsystems, and Motorola, which are some of the largest warranty providers around.

"And this information around warranty and returns is very popular," she added. "People are very interested in it. Customer service is also a huge area of interest. So we constantly go back and mine our data to see what we can share with a larger group. It all depends on the needs of our members and others that approach us for benchmarking information."