Zaph's Challenge

[cotdt]

I like this idea very much! This is very interesting and I'm glad Zaph is doing it. That said, I admit that after trying a DSP-based distortion-adding approach on myself, where I failed to tell the difference between the original signal and one with 3% added 2nd harmonic, I'm currently in Earl Geddes' camp but am willing to change my mind.

[kevinp.]

I like this idea very much! This is very interesting and I'm glad Zaph is doing it. That said, I admit that after trying a DSP-based distortion-adding approach on myself, where I failed to tell the difference between the original signal and one with 3% added 2nd harmonic, I'm currently in Earl Geddes' camp but am willing to change my mind.

Isn't non-linear distortion the same as odd-order? While 3% 2nd order might be hard to distinguish, how about 3% 5th order?

[Dennis H]

Isn't non-linear distortion the same as odd-order? While 3% 2nd order might be hard to distinguish, how about 3% 5th order?

Non-linear distortion is any frequencies created that aren't in the original so 2nd and 5th are both non-linear but it takes much more 2nd to be objectionable.

Here's Olson's data replotted. What he did was attempt to measure the distortion our ear produces. He figured that distortion from the sound system less than that wouldn't be too noticeable. At 90dB, the ear is distorting nearly 30% 2nd order but only about .1% 5th order according to his research.

In another thread, Earl Geddes challenged Olson's data and couldn't imagine how he could have measured it. I had to go back and look; here's how he and earlier researchers did it: "The data was derived by using the understood phenomenon of hearing beating when two notes are impressed on the ear. An auxiliary tone of a frequency near the fundamental test tones’ harmonic is used and its level raised until beating is just audible. This level is related to the ears natural aural harmonic creation." So, you play a 1k pure tone and then add in a tone a bit off the 2k 2nd harmonic and turn it up until you start hearing beats in the 2k tone -- that gives you an idea of how much 2nd harmonic the ear is producing.



Edit: I just realized that the guy who replotted Olson's data did some extrapolation. Olson's data only extends down to 10dB SPL which is about the lower limit of hearing for most people. Everything above the diagonal red line in the pic below is in Olson's original data.

[AJINFLA]

You can read all about it in this masters thesis written a couple of years before Geddes's paper.

A New Methodology For Audio Frequency Power Amplifier Testing Based On Psychoacoustic Data That Better Correlates With Sound Quality

Oh...no...Dennis, not Cheever's thesis :banghead: :B
That paper was not without controversy, to say the least :W
Here is (a nice) Scott Frankland analysis


cheers,

AJ

[Saurav]

At 90dB, the ear is distorting nearly 30% 2nd order but only about .1% 5th order according to his research.

Wow. I knew that we were more sensitive to higher order distortion, but I had no idea the difference was that much.

Was this all done with a 1kHz base tone? I.e. the results may be different depending on the frequency region where the distortion lands?

[Dennis H]

Oh...no...Dennis, not Cheever's thesis :banghead: :B
That paper was not without controversy, to say the least :W
Here is (a nice) Scott Frankland analysis


cheers,

AJ

Nah AJ, I don't buy Cheever's conclusions or his math. If I'd been his masters advisor, I would have flunked him because he thinks -10dB SPL is 10% distortion when it's really 30%. I just used him as a handy resource for Olson's original research. I live in a small town and don't have access to:

Olsen, Harry F. “Music, Physics and Engineering”
Dover Publications, Inc. N.Y., 2nd ed. : 1967.

Wow. I knew that we were more sensitive to higher order distortion, but I had no idea the difference was that much. Was this all done with a 1kHz base tone? I.e. the results may be different depending on the frequency region where the distortion lands?

I don't know what the frequency was for that graph. There are some other graphs in the paper showing it varies with frequency. I wish I could find Olson's book.

Edit: I just realized I'm spelling Olsen wrong. Blame my mom, her name was Olson when she was a girl.

Edit again: just found the book on Amazon (whoda thunk it) and it is spelled Olson -- another strike against Cheever's bibliography.

[Amphiprion]

31.623%, to be a little more precise

I'll have to go read that. Amazing that a prof would let something like that slip by.

[AJINFLA]

Yeah, he misspelled the rather controversial Matti Otala as well :W
Hey, I drank in college too :B
Kinda reminds me of Mark K referring to Toole as O'Toole until I had to ask him if he'd watched too much Lawrence of Arabia as a kid..

cheers,

AJ

[benchtester]

Here is an article on Zaph's web site. It provides some good background.

http://www.zaphaudio.com/nonlinear.html

[fbov]

To be fair, this link is good background on Zaph's view, the one he's developed after years of investigation and based on his listening experiences. It nicely frames thsi thread.

At the same time, it is quite certain that test methodology determines what differences can be discerned. If Zaph's thesis is correct, no short-term testing will ever show the true perceptual magnitude of non-linear distortion effects. Geddes test methodology can only show what it investigates, and so any conclusions about long-term effects would be flawed by virtue of exceeding the experimental design.

I've not read Geddes extensively, but what writings I have read is not complimentary. His logic is flawed, and his ego and intelligence are sufficient to convince many. Fortunately for us, logic serves the truth; when you "baffle with BS," your audience is limited. Sadly, the audience is large enough to keep many such practitioners in business...

Have fun,
Frank

[Hank]

Mark, Scott came to my concert Sunday and said he wants to have an audio/music get-together at his house soon. This might be something we could do. Hey, he and I have no street creds, so we could balance Jonathan and Igor and you. I don't design, I just slap veneer onto MDF.

[Rick Craig]

Here is an article on Zaph's web site. It provides some good background.

http://www.zaphaudio.com/nonlinear.html

Some poor assumptions and questionable comments as well.

[benchtester]

Some poor assumptions and questionable comments as well.

Nice drive-by. :roll:

Care to be more specific?

[brianpowers27]

What now?

It could just be me but I don't see that Geddes' paper was conclusive. I believe that the paper actually suggests that distortion has some connection to perception. I was hophing the sample size would be a little larger. I felt that the confidence of the statistics was severely limited. I am also wondering if headphones are the correct media for this test. WHile it has been shown that headphones improve the perceptibility of certain types of distortion it could mask other effects of distortion.(Such as the effect on stereo imaging due to free air cancellation.)

It would be nice to do distortion testing vs perception based upon a different set of rules. A test that goes by frequency and accounts for the ear's self harmonics would take this field one step closer to an answer that satisfies everyone.

[Rick Craig]

Nice drive-by. :roll:

Care to be more specific?

"For those who don't know what non-linear distortion is, this article is probably not for you".

>>> Not a good way to start off, a little condescending don't you think?

"Many of the papers written in the last 30 years are garbage. Most of those papers are not subjected to peer review the way they should be, and are full of speculation, inaccuracies and outright errors. The AES library has gotten like the patent office - crap gets in easily. Many of the best papers are from the 70's and older."

>>> There are many good AES papers from Keele,Olive,Toole, Gander, etc. that have benefited all of us. I think John's level of accountability is much lower than anyone in this group of people as well as Linkwitz or Klippel.

"What goes wrong with listening tests?

Quite to the point, most listening tests are not long enough. Listening fatigue can take a long time to set in, sometimes more than an hour. Once it does set in, your brain will have learned what listening fatigue sounds like for that particular speaker. Then with each subsequent listening session, the listening fatigue sets in faster as your memory of that speaker's sound returns. If it's bad enough, it will get to a point where you can't stand it and you avoid listening altogether.

"It's a sad fact that many commercial high distortion speakers are better sellers because of the initial short term preference for higher distortion. It either leads to buyer's remorse a couple weeks down the road, or just a mental fight to continue accepting the initial first impression.

In the DIY community, many designs are released far too early, before the designer has a chance to let the sound sink in. No designer should post a design without listening to it a lot, maybe even a month. Much longer if no non-linear distortion data is involved in the design, and the ears are the only tools.

As I've mentioned in the "evaluation myths" article, first impressions are generally wrong. This does not bode well for selecting a speaker on a showroom floor, picking a winner at a DIY event or getting valuable results from a controlled test that is far too short to have any real meaning."

>>> How do you make the conclusion that a speaker is popular because people like distortion? That's a ridiculous assumption. I agree that it takes some time to properly evaluate a speaker but after a few minutes with the right recordings it's not difficult to sort out the bad from the good. And most of the time that's due much more to frequency response errors than distortion. The hard part is sorting out the "better" from the "best" and that takes some time to do.

The premise that you can't trust your ears and need to listen longer if the speaker design wasn't based on distortion tests is nonsense.

[benchtester]

Rick Craig,

Thanks for your elaboration. I found you comments interesting.

John does express his opinions with a great amount of conviction, as do you. Personally, I don't quibble with some exaggeration if it helps to get the point across.

I do agree that a great deal of speaker evaluation can be done in a short amount of time. Also, much of this is based on frequency response. I do think the point of this project is to address sound quality issues beyond FR errors.

It appears to me that you and John agree that "separating the best from the better may take some time." He is attributing much of this to non-linear distortion, perhaps you don't.

I have perhaps a third perspective: Recent I built some Scanspeak speakers, the frequency response was close to some of my other speakers. But within one minute, I knew they had a clarity superior any other speaker I owned and most speakers that I have ever listened to. My hypothesis is that this is due to lower non-linear distortion. Admittedly, I have not controlled the conditions to the degree necessary to be conclusive.

My "startling proposition" is that non-linear distortion is audible; and I go so far as to claim that can be perceived quickly in many cases. I also think that with education, the perception of non-linear distortion will be much more obvious.

[fbov]

Actually, Rick, you can't trust your ears, and extended exposure frequently alters perception. Neither depend on a speaker design philosophy. Both are characteristics of human aural perception, which is the basis for the hypothesis under test.

HAve fun,
Frank

[brianpowers27]

I spent some time reading over the 2007 alma presentation at the GedLee site. I don't know how this thread has been sidetracked to the point where we are suggesting that this report totally denounces the audibility of non-linear distortion. The report states that the non linear distortion alone is not highly audible, in all cases.

The test is based on the 2003 paper(http://www.gedlee.com/downloads/Distortion_AES_II.pdf) Geddes does attempt to prove that his Gm does describe a perceived quality level. He draws a high statistical correlation, using this metric. I could be wrong. This Gm metric is based upon several factors, including non-linear distortion.

I am sure there is a long history regarding Geddes and Zaph so I wont pretend to know anything regarding this relationship. I can only read The Geddes literature.

Is it possible that they are both right?

These were the guiding principles behind the metric.

It should be more sensitive to higher order nonlinearities than lower order ones. • It should be weighted towards greater values for nonlinearities at lower signal levels. • It must be immune to changes in offset and gain (first order slope) since these are inaudible effects.

[Dennis H]

I don't know how this thread has been sidetracked to the point where we are suggesting that this report totally denounces the audibility of non-linear distortion.

I don't think most of us are suggesting that, just the opposite. If you read the thread, I quoted the same part of Geddes's paper you did. The only denouncing going on is Geddes denouncing Zaph's experiment over at diyaudio before Zaph has even built the speakers. He seems rather quick to ridicule any research he hasn't done himself.

[stangbat]

I am sure there is a long history regarding Geddes and Zaph so I wont pretend to know anything regarding this relationship.

I'm no expert on it, but from what I've seen, Zaph has opinions and isn't afraid to state them. Dr. Geddes has opinions and isn't afraid to state them. Dr. Geddes then states his opinion again. And again. And in case you didn't get it the first time, he'll tell you again. And when he is done, he's actually not, because by now you've probably forgot that he has an opinion.

[brianpowers27]

I don't think most of us are suggesting that, just the opposite. If you read the thread, I quoted the same part of Geddes's paper you did. The only denouncing going on is Geddes denouncing Zaph's experiment over at diyaudio before Zaph has even built the speakers. He seems rather quick to ridicule any research he hasn't done himself.

It has been a truly long day. I shall have to re-read this post in its entirety later.

Anyhow, it would be a fun test to participate in. It is hard to say what will come of this.

P.S. I wonder how much different Geddes' testing would have been if he could have exceeded 80db. I suppose part of the reason the ears linearity changes in its perception of harmonics and frequency has a lot to do with compression.

[Dennis H]

P.S. I wonder how much different Geddes' testing would have been if he could have exceeded 80db. I suppose part of the reason the ears linearity changes in its perception of harmonics and frequency has a lot to do with compression.

I find Olson's research from 40-50 years ago more useful than Geddes's although, of course, Geddes rejects it. I don't think he had even seen it before he published his paper and, in another thread here, he accused Olson of faking the data. A differential equation isn't very useful if you're looking at a distortion graph and trying to decide if that 5th harmonic will sound bad. I'm going to order Olson's book so I can get the raw data rather than having it interpreted in the Cheever paper.

[benchtester]

P.S. I wonder how much different Geddes' testing would have been if he could have exceeded 80db. I suppose part of the reason the ears linearity changes in its perception of harmonics and frequency has a lot to do with compression.

Speaking of compression (on topic, trust me):

I am a bit obsessed with "low level detail" as a quality of merit for speakers. I suspect that non-linear distortion masks very low level signals much like a noise floor would. However, I am fairly ignorant about evaluating speakers for low level performance. I have seen power response graphs from 1 watt up to high levels used to evaluate compression at high levels. Has there been much work going down to 10 micro watts? (I am surprised by the number, but this would take a 85 dB/watt speaker down to 35 dB. Still definitely audible. Check my math, its been a long day for me too.) I am concerned that speakers may vary in their ability to reproduce low level sounds. Since this would be linear distortion, it would not show up on HD and TIM charts.

Is this effect (I'm searching for a name: negative compression, expansion, low-level compression?) real, measurable and would it be a confounding :twisted: factor in the non-linear distortion listening test?

[brianpowers27]

Speaking of compression (on topic, trust me):

I am a bit obsessed with "low level detail" as a quality of merit for speakers. I suspect that non-linear distortion masks very low level signals much like a noise floor would.

Is this effect (I'm searching for a name: negative compression, expansion, low-level compression?) real, measurable and would it be a confounding :twisted: factor in the non-linear distortion listening test?

The graph that was posted from Olson's data starts at 50db. I would assume that this is a low enough level to reveal much of the information you are asking for. At some point I would imagine that the blood pumping through your heart would be it's own noise floor.

The whole response curve of human hearing seems non-linear in itself. Low levels are not any different.

[Dave Bullet]

Interesting comparison but difficult to pull off. It would also be best to test them blind; otherwise, personal biases can affect the outcome.

Rick is absolutely correct. Whether or not we'd like to admit it, listening to the more expensive speaker (you'll know because the drivers are flashier and Zaph will select people he can trust which will also be people who have a clue), knowing it comes from Zaph etc... will bias the listener.

I hope specifies exactly the control scenario to all listeners. That is how testing should be conducted - especially the use of double blind testing. That would eliminate the human factor.

Then there is a whole host of other discrepancies to standardise:
- listening material
- source components
- listening environment
- listening volume
- age of listener (consistency since hearing deteriorates with age)

and no doubt others.

I'm not bagging the idea, but as Rick said difficult to pull off. There will always be naysayers that will pick holes in any method or approach.

Nonetheless - it will help or contribute to the field of study that is distortion.

[cotdt]

You guys are not getting the point. Zaph's goal is not to prove anything, he wants to give people the tools in order to come to their own opinion regarding the audibility of nonlinear distortion. You can nitpick on the technicalities of rigorous blind testing and at the end never get anywhere. Have fun searching the Planet for qualified listeners who are completely unbiased and have no opinion on the subject or any related psychoacoustics subjects whatsoever.

[Chris7]

You guys are not getting the point. Zaph's goal is not to prove anything, he wants to give people the tools in order to come to their own opinion regarding the audibility of nonlinear distortion. You can nitpick on the technicalities of rigorous blind testing and at the end never get anywhere. Have fun searching the Planet for qualified listeners who are completely unbiased and have no opinion on the subject or any related psychoacoustics subjects whatsoever.

Yes, in the end it's all about hobbyists. I think this test has the potential to make a difference to people's mindsets in the hobbyist community. I know the test a few years back with two identical speakers but with different capacitors helped convince me that capacitors weren't as audible as I thought they were, even if people quibbled about the exact details of that particular test too.

[benchtester]

The graph that was posted from Olson's data starts at 50db. I would assume that this is a low enough level to reveal much of the information you are asking for. At some point I would imagine that the blood pumping through your heart would be it's own noise floor.

50 dB is pretty loud - I would compare it to a loud computer fan. You are correct about heart beat being audible and don't forget blood flow in the ears; but this is low on the SPL scale.

Here are some values from Wikipedia

TV (set at home level) at 1 m approx. 60 dB
Normal talking at 1 m 40 – 60 dB
Very calm room 20 – 30 dB
Leaves rustling, calm breathing 10 dB
Auditory threshold at 1 kHz 0 dB

I once did a lot of work with pumps in a sound room, it was easy to hear if they were running or not at 38 to 42 dB at 1 meter. My meter only measured down to 36 dB, my breathing and heart beat were not an issue.

A big part of my fascination with audio is that it works as good as it does. I am pretty sure that good speakers go well below 35 dB which implies they are responding to millivolt or even sub-millivolt signals.

I guess I'll have to make some low level measurements. If I get real ambitious, I'll try to do some harmonic distortion measurements. On the plus side, I should be able to do this without disturbing the wife.

[Johnloudb]

"What goes wrong with listening tests?

Quite to the point, most listening tests are not long enough. Listening fatigue can take a long time to set in, sometimes more than an hour. Once it does set in, your brain will have learned what listening fatigue sounds like for that particular speaker. Then with each subsequent listening session, the listening fatigue sets in faster as your memory of that speaker's sound returns. If it's bad enough, it will get to a point where you can't stand it and you avoid listening altogether.

"It's a sad fact that many commercial high distortion speakers are better sellers because of the initial short term preference for higher distortion. It either leads to buyer's remorse a couple weeks down the road, or just a mental fight to continue accepting the initial first impression.

In the DIY community, many designs are released far too early, before the designer has a chance to let the sound sink in. No designer should post a design without listening to it a lot, maybe even a month. Much longer if no non-linear distortion data is involved in the design, and the ears are the only tools.

As I've mentioned in the "evaluation myths" article, first impressions are generally wrong. This does not bode well for selecting a speaker on a showroom floor, picking a winner at a DIY event or getting valuable results from a controlled test that is far too short to have any real meaning."

>>> How do you make the conclusion that a speaker is popular because people like distortion? That's a ridiculous assumption. I agree that it takes some time to properly evaluate a speaker but after a few minutes with the right recordings it's not difficult to sort out the bad from the good. And most of the time that's due much more to frequency response errors than distortion. The hard part is sorting out the "better" from the "best" and that takes some time to do.

The premise that you can't trust your ears and need to listen longer if the speaker design wasn't based on distortion tests is nonsense.

Rick,

What John is describing here is a positive feedback loop. When you have aversion to a sound it limits the loudness and length of time you can listen to it. When you listen to the point of ear fatigue or so long than it causes you distress, you can't listen as long the next time you hear it. The brain turns up the gain for this sound. This is the Limbic and Autonomic Nervous System response to a sound you have aversion to.

It could be caused by distortion or frequency response problem or a combination of the two. It could even be that the listener is used to a "softer" sounding speaker.

Just thought I should point that out. Though this info is probably not much help in speaker design.

[brianpowers27]

50 dB is pretty loud - I would compare it to a loud computer fan. You are correct about heart beat being audible and don't forget blood flow in the ears; but this is low on the SPL scale.

A big part of my fascination with audio is that it works as good as it does. I am pretty sure that good speakers go well below 35 dB which implies they are responding to millivolt or even sub-millivolt signals.

I guess I'll have to make some low level measurements. If I get real ambitious, I'll try to do some harmonic distortion measurements. On the plus side, I should be able to do this without disturbing the wife.

I guess I didn't realize how loud 50db is. I suppose that there might be more inertia to overcome in some speakers. This would also cause a level of non-linear distortion.

On Zaph's test. It sounds like fun and shouldn't be taken as absolutely scientific. I think that it opens the door to good discussion!

[Dave Bullet]

You guys are not getting the point. Zaph's goal is not to prove anything, he wants to give people the tools in order to come to their own opinion regarding the audibility of nonlinear distortion. You can nitpick on the technicalities of rigorous blind testing and at the end never get anywhere. Have fun searching the Planet for qualified listeners who are completely unbiased and have no opinion on the subject or any related psychoacoustics subjects whatsoever.

You nor I (unless John has told you) has any idea whether John wants to prove anything or not!

My suggestions were to help John cover off the nature of the testing (which I think is a good idea). Science is about rigour, repetition and allowing repudiation. If an element of science exists here, then double blind testing and other suggestions are perfectly valid.

Mate - I want to see the result of this, and I'm guessing we all do. It's a great idea... and this discussion will offer John thoughts about how to conduct it and to make it better.... why do you think he even bothered to announce on his blog? If he didn't want a thread like this, then he wouldn't have announced it publicly before running it.... think about it.

[cotdt]

Dave, we all want scientific rigor but here it is impossible. What would double blind testing prove here, when it is not possible to isolate all the variables? We have two speakers with different nonlinear distortion profiles, but within nonlinear distortion there is harmonic distortion and there is IM distortion. Maybe only IM distortion matters, maybe harmonic distortion also matters, and this experiment will not tell us. For example, Seas Excel and Scanspeak Revelator both have low amounts of distortion, and you can give it a similar response curve, yet they will sound completely different. There are just too many variables that are hard to isolate, and we will never acheive scientific rigor.

Instead of trying to prove anything in Science, Zaph's test is useful for Engineering purposes. It will teach us the Subjective effects of nonlinear distortion, to aid our DIY community in designing speakers. Those who are after rigorous scientific proof should ignore this thread and start their own experiment and new thread, because this simply is not about that.

[Undefinition]

If you're open minded enough, it doesn't matter if it's scientific enough or not. I look at it like trying two different brands of food. Even if my eyes might know which is which, I can still let my taste buds give me information. So it's not a completely blind test... how often do people taste food blindfolded? Plus, no one can taste food for you.
It's not a perfect scientific test, but at least it's something--it makes for an interesting firsthand experience if you haven't ever tried it. Myself, I have heard the difference with my own ears many times when it comes to using drivers with low nonlinear distortion vs. ones with more nonlinear distortion. In my experience, it's not exactly subtle--it's not like evaluating cables or caps (which I would probably get nowhere with).

In other words, no amount of statistics or data can compare with firsthand exposure. I dunno; maybe it's the teacher in me. I know that students really need to "experience" a concept for themselves; play with it, try it out in their own ways. They will have a much deeper understanding than if I just lecture and show them data.

[Dennis H]

The graph that was posted from Olson's data starts at 50db. I would assume that this is a low enough level to reveal much of the information you are asking for. At some point I would imagine that the blood pumping through your heart would be it's own noise floor.

The whole response curve of human hearing seems non-linear in itself. Low levels are not any different.

50dB is the fundamental tone which is pretty dang quiet -- certainly a lower level than any of us would listen to music. The distortion products go down to 10dB, about the lower level of human hearing. Here's Olson's original graph.