Two questions for now, one of which I've asked elsewhere:

1 (new): you often include an asterisk when a verdict is based on less than three years of information. Why not include an additional note when the verdict is based on no actual data, but instead on the manufacturer's track record? (generally new Honda and Toyota models, which aren't always above average once the data roll in)

2 (old): the verdicts for the Fit, Yaris, GM SUVs, and others are based on incorrectly filled in surveys, since the data range should have excluded most or even all of those owned at the time of the survey. Isn't it risky to use such data, since there are two possibilities, neither of them good:

Group A. the respondent didn't read the directions (that include an instruction to only report repairs up to 3/31/06)

Group B. the respondent read the instructions, so while they reported owning the car did not report repairs that it required, since these occurred after 3/31

Further, aren't people in Group A likely to report any repairs they reported this year again next year, if they happen to read the instructions?

Jose

In the reliable list, the 4-cylinder and V6 Highlander were combined to report as a single model (both would have been on the most reliable list) and fell below the Highlander Hybrid and 4Runner.  Since the Hybrid was new in 2006, it was reported as a separate model.  The reliability comparison has more details that listed out all the different engine variants.

Gary

Each years survey is new. This year we actually asked for 10 years.

Michael  

1)  Thanks for the suggestion. Actually, in most cases that's pretty clear, but I will pass it on.
2)  We used the 2007 model year information because it was available. As stated in the earlier reply  "Even with the short time on the road and the fact that these vehicles were outside of the survey period, we feel that the information is of interest to our subscribers and therefore we presented it. For some models, such as the Honda Fit, Toyota Yaris, and Toyota FJ Cruiser, we would have predicted the same rating based on the track record of the manufacturer."
3)  The CR survey is new each year (data is not cumulated), so if there is a limited overlap it is not a critical problem.

Thanks, Jerry.

I hadn't thought through the implications of #3. I suppose that one way to think of it is that you're giving a preview of next year's results, so it's not really an issue if the same problems get reported twice.

But if you're not going to enforce the date range, and want to have data on early intro models, why have it? Why not just say "during the past year?" Sure, it's much less precise, but this is the amount of precision in the resulting data given the policy of including data that clearly falls outside the date range.

Michael

That's a very good way of putting it. It is a preview. The time period is necessary for clarity, and normally subscribers follow the survey instructions with few problems. Possibly, this year, the extra early introduction of 2007 models was a factor.

I wouldn't be so sure they follow the instructions. I've done enough of this myself to know how hard it is to get many people to read them.

I don't see how this year is unusual. There are always a few early intro vehicles, and CR each year provides reliability predictions for a few of them the next November. Since this was the first year I'd filled out the survey, I'd never realized that this required relying on data from people who didn't read, misread, or ignored the instructions. Again, I suspect you'd be better off bringing the instructions further in line with how respondents actually use the form.

Just seems an odd situation to be in, relying on people who don't read/misread/ignore the instructions to provide all of the data for some models.

Can't say I agree with offering predictions based on a manufacturer's track record, either--both Honda and Toyota have produced "merely" average vehicles in recent years--but that's another issue that has no doubt already been discussed to death.

Michael,

I think the biggest problem with releasing the data for cars just introduced at survey time is that it's already proved possibly too limited to make an accurate prediction. The Ridgeline has already proved that point. Though it still ended up solidly Average in the current survey, from the responses received in last year's survey it was rated MBTA, shown going way to the right on the bar chart (I can't seem to find my April '06 print edition to tell you just how much). So it may not be fair to say that the Average given to the '07 Camry 4 is equal to the Average given to the SC Grand Prix, even though to look at them on the chart, they'd seem to be about the same and even though both show to be based on just one year's data because the Pontiacs in the survey would include many more examples with higher miles and would have a much higher average age.

Combined with what owners are saying in their online reviews, it also makes the BTA for the transmission on the '07 Camry V6 seem a little more suspect.

Online reviews are anecdotal. With millions of people on the web, problems can seem much more common than they actually are.

Also remember that the survey lets respondents decide whether a problem is serious enough to be reported. Maybe some Camry owners with the transmission issue decided it wasn't serious, since the fix is just a computer reflash.

I have the 2006 Honda issue. It lacks the bar charts, but does state the the Ridgeline has a higher predicted reliability score than any other Honda.

Bottom line is that flawed methods yield messy data that even a large sample size cannot entirely compensate for. My favorite from the 2006 survey:  the V8 scored over 60 points worse than the V6 in the Chrysler 300, but scored about 20 points better than the V6 in the Dodge Magnum. Same engines, same assembly line, results 80 POINTS APART. Based on these results, someone interested in reliability should get the V6 in the Chrysler but the V8 in the Dodge. Which makes absolutely no sense.

Bottom line is that the results simply are not nearly as precise as most people think they are, or as CR pretends they are. The system-level dots can represent differences in repair rates as small as a single percentage point while we have unexplained, and perhaps unexplainable, 80-point swings in overall reliability scores, which should be more robust than system-level reliability scores.

Could you explain where the points are itemized? I don't see them on the road test charts. Thanks.

The points of the road test scores are never itemized. This is one of my personal criticisms, it would be nice to see how the overall scores are calculated, ideally in each instance. Everyone but CR does this. In contrast, CR uses a "secret formula."

The point swings I was refering to with the big Chryslers are in terms of the difference from the average. Looking at the predicted reliability bar graphs in last year's New Car Preview, we can estimate the following relative to the average car:

300 V6: +8

300 V8: -53

Magnum V6: -18

Magnum V8: +2

Hence my statement, "the V8 scored over 60 points worse than the V6 in the 300, but scored about 20 points better than the V6 in the Magnum."

It's not possible to tell if the same is true this year. While the V8 remains much less reliable than the V6 in the 300, results for the Magnum are no longer broken up by powertrain. The also related Dodge Charger has similar scores for both engines.

Whatever factor is responsible, it's of concern because it clearly has a huge impact, and likely affects many other results, just not as obviously.

These cars are made in the same plant and use the same engines, and their designs are nearly identical except that one is a sedan and one is a wagon.

Michael,

Again, you raise good points and there are so many aberrations that have led me to question CR's methodology for some time. For example, last year in the 2006 New Car Preview, the Mercury Mariner had a predicted reliability of much greater than average (it was based on just one year 2005). The Ford Escape and Mazda Tribute, the rebadged cousins of the Mariner, had average predicted reliability based on 3 years results. However, the 2005 results for the Escape and Tribute, unlike the Mariners, was that of average reliability. The 2003 and 2004 Escape/ Tribute had above average reliability according to the 2006 NCP.

The same thing has happened this year with the Pontiac Torrent and Chevrolet Equinox, both of which are built at the Ingersoll Plant in Ontario, Canada. The Torrent based on one year (2006) has better than average predicted reliability while the Equinox has worst  than average reliability. CR writes that the Torrent's reliability is above average "because the Torrent went on sale one year later." Nevermind that just looking at the Equinox's reliability verdict for 2006, it was average compared to the Torrent's better than average score.

Plus, looking how a particular model years verdict changes brings up that question of how people taking the survey are responding to CR's questions (again it's up to the reader to decide what is serious) and whether they are responding to issues that should and should not be recorded (for example recall issues).

The 2004 Saturn Vue, for example had much worst than average reliability in the 05 and 06 New Car Previews, but for the 07 NCP it has average reliability. The 05 had average reliability in the 06 NCP and now has better than average reliability in this years NCP. Going by this logic, anyone seeking to purchase a 2004 Saturn Vue, just from reliability point would be OK to do so, unlike in previous years and the 05, in particular, would certainly be one to seek out.

By the way, I agree that CR should come forward and report how a car got it's score, by showing how many points a vehicle received in a certain category and the weight that category carries overall in the scoring. I don't know why they don't do this (although I have some theories). Showing this, I believe, would help the consumer because they could look at the scoring of a certain category and decide for themselves, whether CR put to much/ less emphasis for what they like to see in a car.

Jeff

I am just wondering, what are the significant problem areas for these following cars?

BMW 5-Series V8
Buick Rainier V8
Cadillac STS V8
Chevrolet Impala V8
Ford F-250 Turbodiesel
Mercedes-Benz E-Class V8
Volvo V70 AWD

Jeff,

I agree with your comments and also those from Michael and now we can add the new VW Passat to the mix.  The 2006 2.0T 4 cyl version has the dubious distinction of being CR's worst reliability rated family car while the same identical car with the V6 is on the recommended list with above average reliability.  To anyone who understands cars, equipment and manufacturing this makes absolutely no sense whatsoever.  This can only point to people reading or interpreting the surveys incorrectly.     

If examples like this don't make people stand back and question CR's survey results I don't know what it will take. 

Bob

 

When will the reliability histories of discontinued models such as the Hyundai XG and the Cadillac DeVille be updated?    

Bob,

I can understand your frustration. But, it goes beyond their methodology. CR, despite gathering information in a flawed way, also overstates the differences in the reliability of most of these vehicles. If you look at the average rate of trouble spots where CR explains how to understand the reliability history of a vehicle, there are 15 categories. 4 of the categories, engine, cooling, transmission and drive system are weighted more heavily in the calculation of reliability than the other categories. Problems here are expensive and can leave you stranded, so it is understandable that CR puts more weight on these areas.

2006 models, for example, had 3 categories, engine, cooling and exhaust where the average problem rate is less than 1%. 9 categories, including the transmission and drive system had an average problem rate of 1%. 3 categories, audio system, body integrity and electrical had an average problem rate of 2%. 

The 4 most weighted categories in CR's reliability survey for 2006 models, the average problem rate is 1% or less than 1%. If you were to check the vehicle profiles in the 2007 NCP, every vehicle that had data for the 2006 model year had average or better than average recorded problem rates in the engine, cooling, transmission and drive system with the exception of these vehicles:

Lincoln Navigator (drive), Mercedes Benz's CLK, CLS, E-Class, M-Class (transmission) and Pontiac Solstice (drive). Most of the 2006 cars had a better or much better than average bubble in these categories.

Going back to 2004,  the first year used in predicting a vehicle's reliability, the average problem rate for these 4 categories is 1% for cooling and engine, 2% transmission and 3% for drive system. Even here, most cars do pretty well in these categories. Newer cars, in the grand scheme of things are not leaving people stranded. Most issues are nuisance issues.

You may wonder why CR is reluctant, if not obdurate, in admitting the issues inherent in their methodology and statistical differences between most cars are not as great as they may look, particular in the most significant areas, engine, cooling, transmission and drive system. I have said it before, that CR may be a nonprofit organization, but it still has a monetary interest to sell magazines and subscriptions so the bills can be paid. 

For a car to get a recommendation, it has test decently, have pretty good crash results and have at least average reliability. Testing and crash results, most cars today will meet. It is the reliability category that keeps most cars from being recommended. People turn to CR to see which cars are recommended. CR releases their reliability results to the papers every year at this time (free advertisement). People in turn go out and buy the New Car Preview.

Some people have wondered why car companies don't dispute the findings of CR more often. I think the Washington Post's Warren Brown, one of the few who consistently questions CR, stated it best when he wrote that in the "world of public relations, car manufacturers are David and Consumer Reports is Goliath." If you don't believe this ask Bose who won a libel suit against CR in the early 80s (which was later overturned by the Supreme Court), but lost the public relations battle among the public and in particular CR readers, who viewed the Bose company as bullies who attacked an organization who protected consumers. A company hardly stands a chance in a battle against an organization that has a reputation for protecting consumer interests.

Jeff  

 

Jerry

We are currently working on the reliability data for used cars, and they will be updated sometime in December, including the models you mention  Actually, some used models did pick up the new chart. (There was some kind of logic to match the chart and model name, but it does not always work).

The problem with the 6AT transmission does NOT just require a reflash. Owners with build dates as late as last month are reporting valve bodies or entire transmissions needing replacement.

Andrew

In the future CRO redesign, we have plans to show all the charts related to a model. Right now we match to the model tested, or to the one for which we have the most data.

Will there be any "notice" given when the discontinued cars reliability is updated?

I have great interest in the 2004 DeVille and would greatly appreciate any advance info you   could give on this model.

Thanks.