Actual measured counter performance with slow drifting input frequency
All frequency counters provide data on their accuracy but at least for
some this data is measured using a stable input, often locked to the
counter reference
When testing with a slow drifting input frequency from just below to
just above 10 MHz the actual performance could be rather different.
I'm searching for performance measurement data of frequency counters.
Not the Phasestation/Timepod or DMTD type of measurement devices.
Or maybe someone is willing to do a measurement using a good counter.
My setup was a 500 s sweep around 10 MHz with a span of 0.02 Hz measured
at a 1 s interval.
I'm interested in both the ADEV measurement and the linear residue of
the phase after removing the global linear frequency trend
Erik.
Erik Kaashoek via time-nuts writes:
All frequency counters provide data on their accuracy but at least for
some this data is measured using a stable input, often locked to the
counter reference
When testing with a slow drifting input frequency from just below to
just above 10 MHz the actual performance could be rather different.
It's not the frequency, it's the phase between the input
signal and the counter's reference frequency.
If you have a HP3336 try this:
Feed the same reference signal to the counter and the HP3336
Have the HP3336 output the same frequency as the reference.
Set the counter to measure period.
Vary the HP3336 output phase through 360 degrees and plot
period vs. angle.
A more modern sig-gen probably also work, but the amplitude stability
is a very important specification.
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
Hoi Erik,
On Sun, 9 Mar 2025 09:30:37 +0100
Erik Kaashoek via time-nuts time-nuts@lists.febo.com wrote:
Not the Phasestation/Timepod or DMTD type of measurement devices.
Or maybe someone is willing to do a measurement using a good counter.
My setup was a 500 s sweep around 10 MHz with a span of 0.02 Hz measured
at a 1 s interval.
While I have not done this type of measurement exactly,
I have measured good OCXOs to judge their stability with
a CNT-90 (Option 30). I used to do 1s gate time measurement
and then average over 100 measurement. During those 100s,
I would generally see fluctuations in the order of 1-2mHz p-p
and a stdev in the range of 200-300µHz
Note: I was measuring OCXO against OCXO, so whether the limiting
factor was the instrument or the OCXO was not clear.
Now, for what you are trying, this is actually not easy.
You are trying to measure a frequency sweep that goes
from -1e-9 to +1e-9. To get a decently precise reading,
the counter needs to be able to measure below 1e-10.
Or in different words: at a gate time of 1s, the counter
needs to be able to resolve better than 100ps. So you need
at least something like an CNT-91, CNT-104 or a SR620.
The next issue is, that you need a low-noise, high stability
frequency reference. Yes, both. High stability alone is not
good enough, as the integrated phase noise leads to quite
a bit of timing jitter. So, the frequency source needs also
a low phase noise. As the circuitry used in counters is pretty
wide band, the phase noise needs to be low at broad band,
ie at large offsets (the white phase noise shoulder is what
dominates jitter when integrating).
That's the reason why people use DMTD systems, be it analog
or their modern digital equivalents, for this kind of measurements.
Simplified, DMTD systems average over multiple zero crossings,
which allows to reduce the measurement bandwidth. Reduced
bandwith means, that less of the noise, both from the DUT and
the reference, is seen, thus reducing the jitter/noise/error at
the measurement rate.
Coming back to the counters. If the counter is the limiting element
and if the counter is well designed, then the error you will see
will be uncorrelated in time (i.e. white) and approximately Gauss
distributed (approximately, because in reality there are always
something that shape the distribution a bit).
If the reference is the limit, the error you will be seeing will
have correlation in time. You might or might not get a Gauss
distribution, depending on the exact factors that cause the drift
in the reference at these tau. Of course, the same goes for
whatever your DUT is. It will have instabilities. Thus you will
have a hard time figuring out whether you are measuring your
DUT against your reference or your reference against your DUT.
And, last but not least, at this precision level, you really
want to have phase stable cables (i.e. not PTFE core cables,
or operate everything above 35°C and at a stable temperature)
and replace all BNC connectors with good screwed connectors
(SMA, TNC, N,...)
Depending on what your goal is, I really would consider building
a small analog DMTD system. Jürg Kögel did a nice circuit back
in 2018 that has a self-noise limit at 1e-14 @ 1s with 10MHz -> 1Hz.
But, I am not sure, whether he ever shared it on the mailinglist.
Alternatively, go a similar way as Claudio Calosso did with
the tracking DDS and have a DDS create the offset frequency
to mix it down, then sample it with a slow (thus cheap) precision
ADC.[1,2]
Attila Kinali
[1] "6/12-channel Synchronous Digital Phasemeter for
Ultrastable Signal Characterization and Use", by
Massimo Caligaris, Costanzo Giovanni and Calosso Claudio, 2015
[2] "Frequency Stability Measurement of Cryogenic Sapphire
Oscillators With a Multichannel Tracking DDS and the
Two-Sample Covariance", by Calosso et al., 2019.
--
The driving force behind research is the question: "Why?"
There are things we don't understand and things we always
wonder about. And that's why we do research.
-- Kobayashi Makoto
You are completely right.
The phase between the input signal and the reference signal determines
the amount of pulling and its easy to measure with a signal generator
that can vary the output phase in small steps.
I've done that and it helped me understand the extent of the coupling.
And here are some measurements done by Tom that demonstrate this
pulling: http://leapsecond.com/pages/53132/
But next to the coupling there is in some counters a problem when the
input signal is harmonically related to the reference signal due to the
"resolution enhancement" technique they are using.
As far as I understood the resolution enhancement, or maybe more
correctly called linear regression of the multiple measurements, has no
or little data to work with if the input signal is harmonically related
to the reference frequency.
So I tried to create a test case that tests both pulling , visible as a
phase deviation in some form, and a difference in measurement accuracy,
visible as a variation of the noise level over the sweep.
The span and sweep time has been chosen such that you get at least two
full phase rotations versus the 10 MHz reference. And the sweep is also
wide enough in relation to the gate time to possibly see a change of
noise level depending frequency difference that determines the amount
of usable data that gets into of the resolution enhancement
This can be clearly seen in this sweep where the frequency noise
increases when the sweep gets close to 10 MHz and when at 10 MHz the
noise disappears
http://athome.kaashoek.com/time-nuts/Freq_error.PNG
I do not claim to be an expert, I only tried to define a simple test
which hopefully will show relevant performance data of a phase/frequency
measurement device.
If there is a better simple test I would be very much interested.
And I understand there are many, not so simple tests, one can do to
understand the performance of a counter but I can't find those tests
done in a way that makes it possible to compare phase/frequency
measurement devices.
There is no standardized test setup and reporting for phase pulling.
There is no standard test and reporting that will quickly show how much
performance deteriorates when coming closer to a harmonic relation
between input signal and some (reference) frequency.
And I did not even mention this should be done both for signals of high
frequency (10 MHz) but also for low frequency inputs (1 Hz) so
understand the behavior of the interpolator used in the device.
20 ps resolution may sound nice but if there is 200 ps RMS noise its
basically useless.
Erik.
On 9-3-2025 21:24, Poul-Henning Kamp wrote:
Erik Kaashoek via time-nuts writes:
All frequency counters provide data on their accuracy but at least for
some this data is measured using a stable input, often locked to the
counter reference
When testing with a slow drifting input frequency from just below to
just above 10 MHz the actual performance could be rather different.
It's not the frequency, it's the phase between the input
signal and the counter's reference frequency.
If you have a HP3336 try this:
Feed the same reference signal to the counter and the HP3336
Have the HP3336 output the same frequency as the reference.
Set the counter to measure period.
Vary the HP3336 output phase through 360 degrees and plot
period vs. angle.
A more modern sig-gen probably also work, but the amplitude stability
is a very important specification.
And here are some measurements done by Tom that demonstrate this pulling:
http://leapsecond.com/pages/53132/
But next to the coupling there is in some counters a problem
when the input signal is harmonically related to the reference signal
due to the "resolution enhancement" technique they are using.
The explanation from hp, including a SCPI command to report the state:
http://leapsecond.com/pages/53132/53132-reduced-resolution.gif
I've run into the harmonic effect you speak of. I wasn't looking for it,
but DUT was an extremely stable and slowly warming up oscillator so I
got a frequency sweep for free. If you're impatient, just jump to the
final plot.
http://leapsecond.com/pages/racal/stdev.htm
http://leapsecond.com/pages/racal/
As far as I understood the resolution enhancement,
or maybe more correctly called linear regression of the multiple
measurements,
has no or little data to work with
if the input signal is harmonically related to the reference frequency.
In hp's case they lose only a factor of ten, which is far from "no or
little data". But your case may be worse due to your pure digital, large
quantization, small aperture design. Most hp counters use analog
interpolators and this actually helps because there's free noise in the
low bits. In fact there's at least one vintage hp counter that
deliberately added clock noise to solve the problem. Pretty clever.
Someone else can post the op/svc manual or hp journal article links.
/tvb
The HP5345 is one example of such a counter.
Bruce
On 10/03/2025 11:06 NZDT Tom Van Baak via time-nuts time-nuts@lists.febo.com wrote:
And here are some measurements done by Tom that demonstrate this pulling:
http://leapsecond.com/pages/53132/But next to the coupling there is in some counters a problem
when the input signal is harmonically related to the reference signal
due to the "resolution enhancement" technique they are using.
The explanation from hp, including a SCPI command to report the state:
http://leapsecond.com/pages/53132/53132-reduced-resolution.gif
I've run into the harmonic effect you speak of. I wasn't looking for it,
but DUT was an extremely stable and slowly warming up oscillator so I
got a frequency sweep for free. If you're impatient, just jump to the
final plot.
http://leapsecond.com/pages/racal/stdev.htm
http://leapsecond.com/pages/racal/
As far as I understood the resolution enhancement,
or maybe more correctly called linear regression of the multiple
measurements,
has no or little data to work with
if the input signal is harmonically related to the reference frequency.
In hp's case they lose only a factor of ten, which is far from "no or
little data". But your case may be worse due to your pure digital, large
quantization, small aperture design. Most hp counters use analog
interpolators and this actually helps because there's free noise in the
low bits. In fact there's at least one vintage hp counter that
deliberately added clock noise to solve the problem. Pretty clever.
Someone else can post the op/svc manual or hp journal article links.
/tvb
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
Hi,
On 3/9/25 21:57, Erik Kaashoek via time-nuts wrote:
You are completely right.
The phase between the input signal and the reference signal determines
the amount of pulling and its easy to measure with a signal generator
that can vary the output phase in small steps.
I've done that and it helped me understand the extent of the coupling.
And here are some measurements done by Tom that demonstrate this
pulling: http://leapsecond.com/pages/53132/
Beware, there is "pulling" and "pulling".
The pulling we have on counters is phase-pulling. Depending on where on
a counters time-base cycle an input occurs, it will experience a shift
from ideal time-stamping location. While sweeping over different
phase-relationships you experience different values, and on average they
avarage out, but if your frequency measure time base is unfortunate for
the fractional frequency difference, the start and stop value will shift
the apparent frequency estimation in a systematic manor. If you use any
of the averaging methods available, this systematic can be significantly
reduced. For multiple signals coming into the counter, whenever they
occur near each other in time, the signal integrity also skews the
timing, on top of the effect from timebase. This is apparent both when
measuring two frequencies at the same time or using the counter for Time
Interval measurements. The phase-pulling effect comes from both signal
integrity issue within the counter but also in how details of
interpolator non-linearities express themselves. The core problem is the
quantization of time is bound to create a pattern, just as a
phase-accumulator does.
For oscillators we often speak about frequency pulling. That would for a
counter mean that the time-base oscillator is pulled by the input
frequency. While this would be possible, I've not seen any real evidence
of this, while I've seen it claimed, not considering the phase-pulling
effect. To verify the frequency pulling, use the 10 MHz reference output
of the counter and measure that with a separate counter to see if it
deviates. That way you can separate the phase-pulling effect from the
frequency-pulling effect.
I know that there is an art in specifying the performance on a counter,
and different vendors go about doing this in very different ways. Some
is overspecing their counters whlie others are underspecing their counters.
But next to the coupling there is in some counters a problem when the
input signal is harmonically related to the reference signal due to
the "resolution enhancement" technique they are using.
As far as I understood the resolution enhancement, or maybe more
correctly called linear regression of the multiple measurements, has
no or little data to work with if the input signal is harmonically
related to the reference frequency.
First, not all "resolution enhancements" is linear regression. The
HP53131/132 counters use an moving average approach. To be more precise,
it uses average of start and stop events for frequency estimation.
Linear regression / Least Square fit counters adds another component
getting improved performance in it's frequency estimation. I would
designate these as Pi, Delta and Omega type frequency estimators to be
inline with IEEE Std 1139. What we did there is we made the language
stricter to talk about frequency estimation. Depending on how the
estimation enhancement is done, it has different ability to suppress the
systematic noise of quantization and also that of random noise. We see
this in the ADEV, MDEV and PDEV plots respectively that is the matching
responses. Naturally there is more details in this so the actual
uncertainty numbers is very hard to predict and there is no real
guidance available.
However, I somewhat disagree on the notion of harmonically related.
Mostly because I think it is a misnomer. Yes, there is a repeating
pattern to it and yes you see among other things even integers. However
you also see various fractional relationships in the pattern too, and
this pattern is related to the phase accumulator pattern too. At some
point this pattern is smoothed out by the random walk of the oscillators
and noise. I would not use harmonics though as that has a very specific
connotation which does not help fully here.
So I tried to create a test case that tests both pulling , visible as
a phase deviation in some form, and a difference in measurement
accuracy, visible as a variation of the noise level over the sweep.
The span and sweep time has been chosen such that you get at least
two full phase rotations versus the 10 MHz reference. And the sweep is
also wide enough in relation to the gate time to possibly see a change
of noise level depending frequency difference that determines the
amount of usable data that gets into of the resolution enhancement
This can be clearly seen in this sweep where the frequency noise
increases when the sweep gets close to 10 MHz and when at 10 MHz the
noise disappears
http://athome.kaashoek.com/time-nuts/Freq_error.PNG
I do not claim to be an expert, I only tried to define a simple test
which hopefully will show relevant performance data of a
phase/frequency measurement device.
If there is a better simple test I would be very much interested.
And I understand there are many, not so simple tests, one can do to
understand the performance of a counter but I can't find those tests
done in a way that makes it possible to compare phase/frequency
measurement devices.
There is no standardized test setup and reporting for phase pulling.
There is no standard test and reporting that will quickly show how
much performance deteriorates when coming closer to a harmonic
relation between input signal and some (reference) frequency.
One test that's been used for a long time is to setup an oscillator with
an intended frequency offset so you sweep over all the
phase-relationships. It works well and have been used for a long time to
test this performance by many vendors. The downside of that test is that
it cannot establish the 0-phase part, but you get the error pattern
swept over.
You can setup pulsers and/or programmable delay-chains to sweep for
exact delays in ways that you would get the true shift of the error
pattern. This is extremely useful for calibrations.
These methods is not detailed in any of the standards I know, but is
known in the business. You can also find traces in patents.
And I did not even mention this should be done both for signals of
high frequency (10 MHz) but also for low frequency inputs (1 Hz) so
understand the behavior of the interpolator used in the device.
20 ps resolution may sound nice but if there is 200 ps RMS noise its
basically useless.
Not so fast. As you average, noise being significantly over your
resolution (i.e. quanitzation) actually helps to average out your
quantization steps and improve your resolution, if you can afford to
measure multiple events to average over. This is just being dithering
and both systematic and random components can help you in doing this. I
presented some data on that in one of the most unreadable articles I've
produced, but the effect is there. The HP5328A/B counters used this.
Detailing the resolution, uncertainty etc. is a very complex little
field. There be dragons, but at the same time, some really good things
have already been learned.
Cheers,
Magnus
Many thanks for the excellent replies
Back to basic measurements. Of course I may do stupid things so feedback
is most welcome.
4 counters where used A,B,C and D
I removed the counter details to ensure a focus on what the measurements
tell us.
Counters A,B and C are general purpose counters using zero crossing
detection, 1 s gate time, reciprocal counting and possible resolution
enhancement.
Counter D works as Attila mentioned at the end of his reply. An offset
signal is used to down mix both inputs (test signal and reference) after
which both down mixed signals are sampled and in the digital domain
converted to an I/Q signal to calculate the phase using all available
samples thus greatly reducing the noise.
First test is done with 500 s period 10 Hz sweep around 10 MHz to see if
anything is happening that makes further experiments relevant.
The sweep is done using a Digital Signal Generator that can not do a
true sweep but uses about 1000 steps (or 1024?) for its 500 s sweep.
Of course this step wise sweeping could cause all kind of unwanted
effects so the impact of the steps has to be investigated
http://athome.kaashoek.com/time-nuts/Sweep%2010%20Hz.png
A lot is happening for counter A and also to some extent with counter B
when close to 10 MHz
Counter D was not used in this sweep but it has been used to confirm the
exact nature of the stepping of the generator
Next step is to zoom in using a reduced span of 1 Hz
http://athome.kaashoek.com/time-nuts/Sweep%201%20Hz.png
Still a lot is happening with counter A and the "increased noise around
10 MHz" for counter B is better visible
Reduced frequency/step has changed the spiky pattern for counter A and
its now clearly visible that whatever weird is happening it is reduced
when close to 10 MHz. As the size of the peaks is comparable to when
sweeping with 10 Hz the steps of the signal generator may not be the
primary driver for the peaks.
Counter D runs at 0.1 s gate time to catch the generator steps and is
just able to see these steps
Next step is to zoom in even further to a span of 0.1 Hz
http://athome.kaashoek.com/time-nuts/Sweep%200.1%20Hz.png
Counter A still shows some weird peaks that could be phase pulling
Counter B shows many more but slightly smaller peaks.
To show the measurement variations a HDEV (not ADEV because of the
frequency drift) was created
http://athome.kaashoek.com/time-nuts/Sweep%200.1%20Hz%20HDEV.png
Counter A and B perform similar in the HDEV. counter C is roughly a
factor 10 better and as expected counter D is much better.
To test if there is phase pulling a signal with 0.01 Hz offset from 10
MHz was created and the frequency was measured during 500 s to check if
any pattern repeats every 100 s
http://athome.kaashoek.com/time-nuts/+0.01%20Hz.png
Counter A is stable except for a rather large deviation every 100 s
Counter B is noisy
Counter C is better and D as expected much better
This is clearly visible in the ADEV
http://athome.kaashoek.com/time-nuts/+0.01%20Hz%20ADEV.png
From the ADEV you can also see that that counters B,C and D are
actually measuring phase which is converted to frequency in Timelab if
needed.
Counters B, C and to some extent D do have a dip in the ADEV at 100 s so
there must be some kind of repeating pattern with 100 s period for these
counters
Please let me know if what I am doing makes sense and if you have
suggestions for what next to do.
Erik.
Hi Eric,
So, what is apparent for counter C is that there is a systematic noise
in there, which causes the ripple in the ADEV. Through selection of a
tau at the end of that, your tau filters out this systematic and you
access the underlying random noise precision.
The ADEV for counter A have the tell-tale of having systematic noise as
it does not follow the expected slope. In comparison the B counter has
some systematics, but not much to fully dominate.
If you can, rather than sweeping frequency, just offset the 10 MHz
frequency with +/- 2 mHz to sweep all phase-relationships in 500 ns. The
RMS of that is a good judge of which is the best linearity counter.
Questions:
Can you disclose the single-shot resolution of these four solutions?
Can you disclose the datasheet "spec" for the counters?
Can I assume that you used none frequency estimation improvement
processing on any of these counters?
Cheers,
Magnus
On 3/10/25 15:07, Erik Kaashoek wrote:
Many thanks for the excellent replies
Back to basic measurements. Of course I may do stupid things so
feedback is most welcome.
4 counters where used A,B,C and D
I removed the counter details to ensure a focus on what the
measurements tell us.
Counters A,B and C are general purpose counters using zero crossing
detection, 1 s gate time, reciprocal counting and possible resolution
enhancement.
Counter D works as Attila mentioned at the end of his reply. An offset
signal is used to down mix both inputs (test signal and reference)
after which both down mixed signals are sampled and in the digital
domain converted to an I/Q signal to calculate the phase using all
available samples thus greatly reducing the noise.
First test is done with 500 s period 10 Hz sweep around 10 MHz to see
if anything is happening that makes further experiments relevant.
The sweep is done using a Digital Signal Generator that can not do a
true sweep but uses about 1000 steps (or 1024?) for its 500 s sweep.
Of course this step wise sweeping could cause all kind of unwanted
effects so the impact of the steps has to be investigated
http://athome.kaashoek.com/time-nuts/Sweep%2010%20Hz.png
A lot is happening for counter A and also to some extent with counter
B when close to 10 MHz
Counter D was not used in this sweep but it has been used to confirm
the exact nature of the stepping of the generator
Next step is to zoom in using a reduced span of 1 Hz
http://athome.kaashoek.com/time-nuts/Sweep%201%20Hz.png
Still a lot is happening with counter A and the "increased noise
around 10 MHz" for counter B is better visible
Reduced frequency/step has changed the spiky pattern for counter A and
its now clearly visible that whatever weird is happening it is reduced
when close to 10 MHz. As the size of the peaks is comparable to when
sweeping with 10 Hz the steps of the signal generator may not be the
primary driver for the peaks.
Counter D runs at 0.1 s gate time to catch the generator steps and is
just able to see these steps
Next step is to zoom in even further to a span of 0.1 Hz
http://athome.kaashoek.com/time-nuts/Sweep%200.1%20Hz.png
Counter A still shows some weird peaks that could be phase pulling
Counter B shows many more but slightly smaller peaks.
To show the measurement variations a HDEV (not ADEV because of the
frequency drift) was created
http://athome.kaashoek.com/time-nuts/Sweep%200.1%20Hz%20HDEV.png
Counter A and B perform similar in the HDEV. counter C is roughly a
factor 10 better and as expected counter D is much better.
To test if there is phase pulling a signal with 0.01 Hz offset from 10
MHz was created and the frequency was measured during 500 s to check
if any pattern repeats every 100 s
http://athome.kaashoek.com/time-nuts/+0.01%20Hz.png
Counter A is stable except for a rather large deviation every 100 s
Counter B is noisy
Counter C is better and D as expected much better
This is clearly visible in the ADEV
http://athome.kaashoek.com/time-nuts/+0.01%20Hz%20ADEV.png
From the ADEV you can also see that that counters B,C and D are
actually measuring phase which is converted to frequency in Timelab if
needed.
Counters B, C and to some extent D do have a dip in the ADEV at 100 s
so there must be some kind of repeating pattern with 100 s period for
these counters
Please let me know if what I am doing makes sense and if you have
suggestions for what next to do.
Erik.
Bruce wrote:
The HP5345 is one example of such a counter.
Thanks, that's the one.
Magnus wrote:
However, I somewhat disagree on the notion of harmonically related.
Mostly because I think it is a misnomer.
If you find a better word or phrase please share it. You recall the hp
53132 manual uses the words integer and fraction, analogous to harmonic
and sub harmonic:
http://leapsecond.com/pages/53132/53132-reduced-resolution.gif
The hp 5345 manual (see above) is even more explicit, using the words
"preventing a harmonic relationship" or "cannot be harmonically related":
http://leapsecond.com/pages/53132/5345-noise-generator-4-117.png
http://leapsecond.com/pages/53132/5345-noise-generator-4-126.png
/tvb
