Jump to content

Blind testing with speakers (general public)


deepak

Recommended Posts

I thought this was an interesting read over at an AV forum: Best Sounding Speakers According to Scientific Tests - Audio Video Revolution Forum

Random and blind testing with members of the general public choosing their favorite speakers without knowing anything about them (ie cost or looks).

I was asked by Wes to post some results and ratings from a scientific study I'm a part of.

What we've done:

2,000 "Test Subjects," had their hearing checked and passed. We decided they had "above average to good," hearing for their age and sex.

Then we tested different parts of the audio chain: cables, amp, pre-pro, speakers, and CD players. Only 8 people could be tested at a time due to the limited "sweet spot," for sound listening for the tests. For speakers our reference rig consisted of a complete Arcam FMJ setup: amp, cd player, pre-pro. The wiring was all mid-priced Cardas. They listened to a track of Beethoven, Pink Floyd, Waylon Jennings, and Public Enemy (just so we could affect the political race, and get the phrase "corrupt like a senator," ingrained in their brains). 5 identical listening rooms were used. Each had 2 sets of speakers properly set up. They'd listen to one track and move to another room. Speakers would randomly be switched from "A," and "B" for example in room 1. So say a PSB and a Def Tech were in that room, one was A and one was B, this could be switched without the listener knowing we had changed designation for a different track.

The test was completely blind. The test subjects were never told what speaker they were listening to, nor could they see them.

We tested hundreds of different speakers at first (pretty much every brand you could think of). After many tests some speakers were removed because it was apparent they were "disliked," from results. Basically in one day 40 people would test 40 different speakers. We did this by price point. The first tests were for $0-$1000 pair of speakers. They weren't asked to "Rate," just pick the best 5 from a group (of 10). It was never the same 10 vs the same 10 more than per 40 people. We mixed it all up. Still we were able to identify speakers that were "preferred," to "disliked." Interestingly this process of elimination only resulted in nearly 30% of the speakers priced under $1,000 to be eliminated. The entire process was repeated multiple times until we had valid data showing the 10 most preferred speakers. Then those 10 speakers were "rated," by the test subjects.

Then we took the top 5 rated speakers under $1000 and added them to the speakers under $2,000 and repeated the tests with the speakers priced $1000-$2000. We again reached a point where we had the top 10 speakers. These were rated and the top 5 were then compared to $2001 to $5000 speakers. Ten speakers were preferred, then rated. Then $5001 to $10,000. Then $10,000+.

These tests are not finished. So my posting results should be taken as a work in progress. So here are some incredibly interesting findings:

The Top Rated Speaker for under $1,000:

Magnepan MMG, $550 direct from Magnepan. This speaker was the overwhelming choice in the category.

The Top Rated Speaker for under $2,000:

Magnepan 1.6 for $1,800 from Magnepan Dealers. Interesting note: The Magnepan MG12 and MMG (the top under $1,000) also made the top 5 highest rated under $2,000 pair of speakers. The Dali Ikon 6 and Martin Logan Source also made the top 5 rated. (Maggie 1.6, Martin Logan Source, Maggie MG12, Dali Ikon 6, and Maggie MMG in that order for those wondering).

The Top Rated Speaker for under $5,000:

Martin Logan Vantage, Magnepan 3.6, Anthony Gallo Reference 3.1, Def Tech Mythos Super Towers, Martin Logan Purity

The Top Rated Speaker for under $10,000:

Martin Logan Vantage, Martin Logan Summit (technically more expensive than the Vantage)

Top Rated Speaker for over $10,000:

Martin Logan Vantage

We're not done with our testing. We're about 60% finished, so the numbers could change. We got most of our gear from donations, the rest we bought or have on loan (the 7 speakers we have over $10,000 for instance). We had to skip certain models and generally chose the most expensive of a line or 2nd highest in a line.

Speaker Brands we tested: Bose, Klipsch, Polk Audio, Martin Logan, Magnepan, Def Tech, PSB, Mirage, Wilson, Anthony Gallo, Sony, KEF, Canton, and Krell just to mention some of them. Like I said we tried to test as many cheap speakers as possible and removed complete lines that didn't test well. So take Bose for an example. They tested horribly, and by the time we eliminated their 101s and 201s, we only tested one of the Life series. Polk Audio, Klipsch, and many others suffered the same fate, with us just picking one of their line from a price range.

I'll let everyone here know when we're finished, because we'll be selling the gear.

Edited by deepak
Link to comment
Share on other sites

  • Replies 98
  • Created
  • Last Reply

Top Posters In This Topic

Doug, I'll make you a deal for my MG12s, if you're interested. I'll bullshit customs however you please.

I'd have to sell the quads for a chance of that, and at that money I could probably get ESLs in fair repair. I'm just spouting cos of my positive paypal balance. Thanks for the offer though.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.