DIY Bench Testing Commercial components Update and planning

[JonMarsh]

This is just a "utility" thread to appraise folks that mostly hang out in Mission Possible about bench testing of commercial components that may or may not be of interest, posted in "Audio Hideout" and "Digital Audio".

A few components have been tested and more are planned for the near future- this may be of interest to some of you. I'm focussing more on measured characteristics with some occasional discussion of correlation with what we hear, and why something likely measures the way it does- good or bad.


Links to existing test reports:


AURALiC Taurus Pre

http://www.htguide.com/forum/showthread.php?43139-Benching-the-AURALiC-Taurus-Pre

Benchmark AHB2 Power Amplifier

http://www.htguide.com/forum/showthread.php?43141-Benchmarking-the-Benchmark-AHB2-Amplifier

Cambridge Audio 851D Digital Preamp/DAC

http://www.htguide.com/forum/showthread.php?43193-Cambridge-Audio-851D-excels-at-quot-budget-quot-price-point

Cambridge Audio 851E Analog Preamp

http://www.htguide.com/forum/showthread.php?43208-Benching-the-Cambridge-Audio-851E-Analog-Preamp

Planned test reports in progress (not necessarily in this order!):


AURALiC Vega DAC

NAD M22 Class D Power Amp

Cambridge Audio 651W Power Amp

Hypex NCORE 400 DIY Class D modules

TotalDAC-D1 Dual

Brainstorm DCD-8 Clock system

Muted MC3+ Clock system



I'm particularly interested in puzzling out the correlation between audible performance and measured performance, especially the degree to which time domain issues like digital filters or jitter or sampling methodologies count versus "Classic" standards of THD, Noise, and IMD. A typical "vexing" area is comparing DACs which use fundamentally different technologies and convert all inputs to one format of signal (DSD or PCM) for sampling and conversion. Some DACs which are highly regarded by some as regards audible performance, (Playback Designs MPS-5, for example, or PS Audio DirectStream DAC) but which have some aspects of measurements that don't really jive with that assessment, such as barely mustering 16 bit equivalent noise performance for a so called high resolution device.

There are units out there that I'd like to measure and examine, such as the Schiit Yggdrasil DAC, but I can't afford to buy every interesting piece of gear on the market- so hopefully now and then we'll come across someone willing to loan me their piece for a few days to bench test. I may have a sample of the PS Audio Directsream DAC to look at in the near future, but it's not a done deal yet.

Comments, requests, and discussions are all welcome...

[---k---]

John,
Thanks for pointing that out. I just skimmed through them all. Well done. I really enjoy reading the technical and science part. JA's testing was always my favorite part of Sterophile when I subscribed. I'm going to have to make a point of checking into the Audio Hideout from time to time.

One thing, I noted several times you made reference to the fact that you could post graphs from gear costing 4x as much but with much worse performance. I would love to see that! In fact, I think a post that went through each graph explaining what we should be looking for and showed good results, average results and bad results side-by-side would be an amazing resource. Half the time, I don't know what I'm supposed to be seeing in these graphs. I hope that you could possibly find some time for this.

... and what about this top secret classified Class D forum?

[JonMarsh]

Those are some good suggestions, Ryan.

First, I will plan to do sometime soon an "educational" section talking about the why's and wherefores of each measurement, and try to put some perspective on just what the graphs mean and what to look for and pay attention to.

Second, I will go back to some of those existing write-ups and post specific comparisons, with a graph excerpted from the "costs 4x" category component and the DUT.

Third, I'll use this thread as a "tracker" so that when I update a review post, or add another test bench review, you can find out here.

If there's interest, we could change this to a sticky thread.

All of this will take time to do, because it's sort of a secondary activity to things like... "this top secret classified Class D forum". Those primary activities are what has driven equipment acquisition and more wide spread testing... and they are behind schedule! :W

In the course of things, though, if from time to time I can identify components that seem to have an unusually promising value proposition (like that 851D "Digital Preamp") then I want to be sure HT Guide members know about that in a reasonable fashion.

And lastly, glad to see another fan of JA and what he does; though I value the other guys at Stereophile, too, of course, nonetheless my personal inclinations and sympathies run with John Atkinson, for sure. It's not really my intention to explicitly emulate him; I think of myself more as a creator and tinkerer than a reviewer, but having good awareness of what others have labored at is very useful background information. And one could do far, far worse than emulating his efforts- guys like him and Richard C. Heyser of Audio magazine have definitely made a difference in this industry.

[ergo]

Great work and interesting reading for sure. I'm also always amazed how much quality gear you have... Estonia being a very small country/market I would have hard time finding a HiFi dealer who would have such list of quality component selection
AP-s new analyzers also seems to be a much easier to use compared to the older 2700 series units. I've toyed with those a bit but there is enough of a learning curve and much less flexibility as far as I can tell.

For class-D it would actually be interesting to also see one plot with the high frequency (beyond audio band) crap visible. Otherwise these measurements are always made with special LP filters and this is not compared... most probably there is a great difference in how much HF noise those amps spit out.

[wkhanna]

If there's interest, we could change this to a sticky thread.

i would like to see a dedicated forum for this.
this is powerful stuff.

[---k---]

Agreed.


I think a tutorial would be an amazing resource.

[JonMarsh]

Great work and interesting reading for sure. I'm also always amazed how much quality gear you have... Estonia being a very small country/market I would have hard time finding a HiFi dealer who would have such list of quality component selection
AP-s new analyzers also seems to be a much easier to use compared to the older 2700 series units. I've toyed with those a bit but there is enough of a learning curve and much less flexibility as far as I can tell.

For class-D it would actually be interesting to also see one plot with the high frequency (beyond audio band) crap visible. Otherwise these measurements are always made with special LP filters and this is not compared... most probably there is a great difference in how much HF noise those amps spit out.

I had my own passive HF filters for class D work with the HP8903 I've had for years, but now I also have an AP AUX0025, which is their box for that application. Are you familiar with the documentation on that unit, and it's response characteristic? Without some type of filter, the analyzer sees the residual carrier as a big lump of noise (typically 300mV P-P) and makes any other measurement meaningless. At the typical carrier frequency, drivers and networks are so inductive there really isn't any influence by this carrier. But the Analyzer in the APx555 is very wideband, up to 1MHz working bandwidth, so it takes some care to deal with that.

Now, something that is important is just how good is the output filter design of a given Class D amp? There are quite a few that have what I can only call bad designs; with not just a sine wave at the carrier frequency, but quite a bit of spike feed through. This results in the sort of situation that JA has reported on some class D amplifier reviews where turning on the amplifier blanks out the FM reception in the room. This should NOT occur with a properly designed Class D amplifier, and it is something I'll use my Teledyne-LeCroy scope to document. (The bandwidth of the scope function in the AP isn't up to high signal fidelity at 300MHz!)

I agree with you about the difference in usability and functionality of the old and new testers- the APx500 series can be very powerful and fast, often using just a couple of sweep acquisitions to acquire quite a bit of data, and is not destructive of the data between subsequent "measurements". This is called Sequence mode.

OTOH, I do find myself often using what is called "bench mode" on the APx555, which is more like how the older series operate; it's a good way to experiment and suss out a setup for a pariticular test, then put together a sequence mode project for doing that test quickly and consistently, which is especially important in an evaluation sequence for R&D. Overall, I think the software on the new series is much easier to use and has a less confusing interface, which is why I decided to pop for a new one, rather than one of the fairly good deals I could find on eBay for a SYS2722 at about half price. Mind you, that's half price for a unit that is 10 years old! That's definitely a case of holding value!

[JonMarsh]

i would like to see a dedicated forum for this.
this is powerful stuff.

If there is support for that, let's float the idea with Lex and the admins.

[sdl2112]

If there is support for that, let's float the idea with Lex and the admins.

I would like to see a dedicated forum. Jon...I honestly don't know how you have the time. :E

[wkhanna]

be careful what you ask for.....:W

[PewterTA]

Bill, I think this means you need to get a Yggy...... no doubt about it now... then burn it in and have Jon test it!

I mean after all... I have a new Mutec MC-3+ USB and they are sending me their new Master Reference Clock too.

Also... I have a Bricasti M1 that I could lend him for some testing.

Wow, I have been away from the forums (or any forums for that matter). I need to get back on the ball and get some more free time!

-Dan

Would be nice to create a "database" of sorts of various equipment all tested the same way to see how they all differ!

[wkhanna]

you did not tell anyone (me!) you got the M1.
must have been a V recent acquisition.
anxious to hear one on our systems, after having heard them for so many hours at Capital Audiofest.

of course, you realize it must to be sent to Jon now!

AFA the Yiggy......
i want one, but i gotta finish the Nat P upgrade first.
then sell my Gungir....
unless.......we all decided to pitch in to get one.......then one of us could 'Buy Out' the others latter?

[PewterTA]

I just got it, it's in transit from Canadia (Toronto) as I type this... so most likely it'll be here by the end of the week. The guy checked with Brian to verify that he would support the unit if sold to me and Brian said since it was registered to him and is valid that is not a problem.... So since the price was right (also a big positive), I decided I wanted to jump on it. It's not the Special Edition, that's an upgrade, but not a big deal to get done later.

I'd be willing to pitch in on the Yggy and get bought out later!!!! Then we could have a head to head on the Bricasti M1, NAD M51, and any other DACs we can get together! Throw in my stacked Mutec MC-3+ USB & MC-3 for dual reclocking and iFI Micro USB3.0 (reclocking as well)... I think we can send the gear some pretty good signals!

I also got my OCXO Motherboard clock installed in the media PC (upgrade as well to the sound you know from my system) and I have a new USB 3.0 Card to get... so we have some well deserved upgrades that need to be vetted with the audio group!

[augerpro]

Any interest in testing a miniDSP 4x10 HD?

[JonMarsh]

It could be done- I've already got some DAC stuff planned. I have one of the smaller miniDSP's somewhere around here, but I'm not sure where... Are you interested in how it does just as regards I/O quality?

[augerpro]

Yep, haven't seen much testing of them so far.

[JonMarsh]

Well, I'll keep an eye out for mine, I've seen it recently, but I'm short on time to mount a dedicated search in the near future (the last one took 3 hours, in order to sell some drivers to another forum member). Otherwise, if it doesn't turn up soon, I could measure yours specifically. UPS between here and Denver isn't much - if you ship it to me, I'll pay for shipping it back.

[augerpro]

Sounds good, I'll let you know when I get some days free to ship it

[---k---]

With the popularity of the MiniDSP, that testing could be interesting. Depending on how it measures up, I'd be interested in commentary on "good enough for subwoofer usage" and "good enough for active speakers".

It's too bad this isn't your full time job. If it were, I'm sure we could get you a Behringer unit to compare to.

[JonMarsh]

I have a DCX2496 somewhere in storage-

The thing about all of these units is they're both ADC and DAC, plus a DSP, and trying to keep the cost down. There is a price performance point where I think they may make sense, from EQ and getting rid of low cost passive crossover components. Not sure exactly where that is...

[olu78]

As always Jon, I find your observations highly captivating. I have a 'virtually brand new ' Harman /Kardon AVR8000, and it has spent the last 13 years packaged in a box, in the corner. Though its Crystal DSP, and AD1852 Dacs are considered outdated in most circles, I have long wondered how it stacks up sonically to today's upper echelon AVR's. I do find your analysis aided by the AP, a new perspective previously unavailable to us forum members. For that, and your steadfast contributions, my sincerest
gratitude.

[JonMarsh]

Anything that helps, I'm glad...

Most AV components use Delta-Sigma DAC chips which encode/upsample PCB to low bit count high bit rate DSD. Many feel that a lot is lost in the process, and hence we have a resurgence in popularity of R2R ladder DACs, from price points as varied as the Schitt Yggdrasil to the TotalDAC-D1 series.

Others continue to work on DSD approaches, but have found they have to employ custom algorithms in large FPGA's to get to the performance level they want. That performance level doesn't necessarily measure as well as one would hope but with the right data interface, (I2S) the sound can reportedly be very good.

http://www.stereophile.com/content/ps-audio-perfectwave-directstream-da-processor#VdWXGlxOM9orfdME.97

A classic issue with the noise shaping used in multi-bit Delta-Sigma DACs is high frequency noise:



1kHz @ -90 dB, 16bit data- 24bit version not shown, due to noise.





50Hz Sine FFT, at 0dBFS, with updated firmware (original distortion was higher)

Now, for comparison, let's look at that inexpensive Cambridge 851D DAC:



Cambridge 851D, at 0dBFS, 50Hz


Lower noise floor (compare level of baseline of FFT sweep) and lower distortion.


(I have a friend very interested in that path, though, and we've sourced a Signature Rendu for him, and he's still working on finding the right Audiogon deal on a used PS Audio DSD. If/when we round up a DSD for him, I'll be testing it myself; he's bidding on one right now with the Yale firmware revision.)

So, in my experience, HT AVR's work pretty good for what we're expecting in sound reproduction for movies in spite of their limitations- but I think it's partly a matter of expectations. Movie sound mastering is pretty sophisticated these days, and BluRays come with very high resolution PCM track options.

I don't think you'll really know how good the PCM sound is from a typical disk, though, unless you setup your Blu-Ray player to output discrete PCM on HDMI (must have the decoders built in to the player; many don't) and then pick off the audio channels, say use a 2.1 mix and run the HDMI through an NAD M51 (that's what I originally bought my M51 for, was video) and listen to that- THEN you'll have an idea of how good movie sound mastering is these days.

I expect there must be some AVRs that do use that level of quality processing- in fact, the built in Audio discrete outputs for the current Oppo players is in that league with Sabre DACs. well, they're sort of in that league, because if you compare their audio playback on a stereo disk source with an AURALiC Vega DAC, it's not at the same level of quality- it's not that easy to do a truly first class job with the SABER DACs, from what I read in many sources.

Of course, if there is an AVR like that, I can't likely afford it! :W :B