time-nuts@lists.febo.com

Discussion of precise time and frequency measurement

View all threads

Re: 3-Cornered Hat fails with negative variances, is there anything I can to to improve?

A(
AC0XU (Jim)
Mon, Oct 17, 2022 12:06 AM

Erik-

Measuring simultaneously just guarantees that the measurements between the pairs A-B, B-C, A-C are consistent (so you might as well make only two measurements). For example, if you measure A-B and A-C this way, you can combine the measurements to get an estimate of B-C stability that is independent of A's stability. This does not strike me as particularly useful however. Why not just measure B-C directly? Regardless - each phase, frequency or time measurement has associated measurement error and uncertainty.

Suppose one of the time/frequency references A is much more stable than the other two. B and C Then the Tri-Cornered Hat estimates for B and C will be reasonably accurate. However, any attempt to estimate stability of A  with Tri-Cornered hat is doomed. You can easily get negative variance estimates.

Suppose A and B are much more stable than C. Then the Tri-Cornered Hat estimate of C's stability may be more accurate than if you used A-C or B-C alone, because in a statistical sense you are averaging together two stable references to synthesize an even more stable one. The accuracy of the A-C relative stability will depend on how similar A and B are.

If all three references have about the same stability, then the Tri-Cornered Hat estimates would usually be not terrible. The estimate uncertainties depend on the measurement uncertainties.

The above statistical ideas assume that the references have perfectly stable phase plus a random instability, and that that random instabilities of the three references are statistically independent. Otherwise, Tri-Cornered Hat doesn't work well at all. For example, if your three references are proximal, then temperature fluctuations will cause correlated frequency perturbations. I have noticed this effect with OCXOs and rubidium oscillators when the ceiling fan in my lab cycles on and off.

In the second paper I showed that Groslambert estimates are not generally better than Tri-Cornered Hat or the N-reference equivalent thereof. And that is one conclusion from all this - the more references, the better.

Jim

At 01:39 PM 10/16/2022, Erik Kaashoek wrote:

Jim
I'm not able to access the paper as I'm not an IEEE member.
And the math probably will be way over my head anyway.
Do I understand correctly that for testing one should next to something like the ADEV of the counter noise, also measure the histogram to check if the noise is gaussian and the PSD to check for hidden frequencies in the noise?

If the three phase relations are measured simultaneously and the 3 equations solved to calculate A, B and C. Would calculating the ADEV of A,B and C not eliminate the noise of the measurement device?
Keep in mind it's been a very long time since I studied statistics.
Erik

On Sun, Oct 16, 2022, 20:30 AC0XU (Jim) via time-nuts <mailto:time-nuts@lists.febo.comtime-nuts@lists.febo.com> wrote:
I published a couple of papers on Tri-Cornered Hat and Grosslambert.

https://ieeexplore.ieee.org/document/9234829https://ieeexplore.ieee.org/document/9234829

https://www.ion.org/publications/abstract.cfm?articleID=17805

My position is that this is a statistical problem, and it is best to analyze the results statistically. In particular, you can estimate the distributions of the measurements and the computed ADEV estimates. You never end up with unphysical results because even if the simple tri-cornered hat formula produces a negative result for some time scale, the estimate includes an uncertainty interval that includes positive values. Unfortunately, the usual ADEV apps don't perform this analysis. Some day I hope to publish one but that hasn't happened yet. The algorithm I use is described in the first paper.

Sadly, no manufacturer of stable timing devices makes much of an effort to capture the statistical properties of their devices. The most that most manufacturers do is post ADEV values at a few time scales. I haven't seen any datasheets with confidence intervals for ADEV. Also, manufacturers can be cavalier with temperature sensitivity, which can be huge even for OCXOs and  "atomic clock on a chip" devices. So you have the right idea - testing is super important.


time-nuts mailing list -- mailto:time-nuts@lists.febo.comtime-nuts@lists.febo.com
To unsubscribe send an email to mailto:time-nuts-leave@lists.febo.comtime-nuts-leave@lists.febo.com

Erik- Measuring simultaneously just guarantees that the measurements between the pairs A-B, B-C, A-C are consistent (so you might as well make only two measurements). For example, if you measure A-B and A-C this way, you can combine the measurements to get an estimate of B-C stability that is independent of A's stability. This does not strike me as particularly useful however. Why not just measure B-C directly? Regardless - each phase, frequency or time measurement has associated measurement error and uncertainty. Suppose one of the time/frequency references A is much more stable than the other two. B and C Then the Tri-Cornered Hat estimates for B and C will be reasonably accurate. However, any attempt to estimate stability of A with Tri-Cornered hat is doomed. You can easily get negative variance estimates. Suppose A and B are much more stable than C. Then the Tri-Cornered Hat estimate of C's stability may be more accurate than if you used A-C or B-C alone, because in a statistical sense you are averaging together two stable references to synthesize an even more stable one. The accuracy of the A-C relative stability will depend on how similar A and B are. If all three references have about the same stability, then the Tri-Cornered Hat estimates would usually be not terrible. The estimate uncertainties depend on the measurement uncertainties. The above statistical ideas assume that the references have perfectly stable phase plus a random instability, and that that random instabilities of the three references are statistically independent. Otherwise, Tri-Cornered Hat doesn't work well at all. For example, if your three references are proximal, then temperature fluctuations will cause correlated frequency perturbations. I have noticed this effect with OCXOs and rubidium oscillators when the ceiling fan in my lab cycles on and off. In the second paper I showed that Groslambert estimates are not generally better than Tri-Cornered Hat or the N-reference equivalent thereof. And that is one conclusion from all this - the more references, the better. Jim At 01:39 PM 10/16/2022, Erik Kaashoek wrote: >Jim >I'm not able to access the paper as I'm not an IEEE member. >And the math probably will be way over my head anyway. >Do I understand correctly that for testing one should next to something like the ADEV of the counter noise, also measure the histogram to check if the noise is gaussian and the PSD to check for hidden frequencies in the noise? > >If the three phase relations are measured simultaneously and the 3 equations solved to calculate A, B and C. Would calculating the ADEV of A,B and C not eliminate the noise of the measurement device? >Keep in mind it's been a very long time since I studied statistics. >Erik > >On Sun, Oct 16, 2022, 20:30 AC0XU (Jim) via time-nuts <<mailto:time-nuts@lists.febo.com>time-nuts@lists.febo.com> wrote: >I published a couple of papers on Tri-Cornered Hat and Grosslambert. > ><https://ieeexplore.ieee.org/document/9234829>https://ieeexplore.ieee.org/document/9234829 > >https://www.ion.org/publications/abstract.cfm?articleID=17805 > >My position is that this is a statistical problem, and it is best to analyze the results statistically. In particular, you can estimate the distributions of the measurements and the computed ADEV estimates. You never end up with unphysical results because even if the simple tri-cornered hat formula produces a negative result for some time scale, the estimate includes an uncertainty interval that includes positive values. Unfortunately, the usual ADEV apps don't perform this analysis. Some day I hope to publish one but that hasn't happened yet. The algorithm I use is described in the first paper. > >Sadly, no manufacturer of stable timing devices makes much of an effort to capture the statistical properties of their devices. The most that most manufacturers do is post ADEV values at a few time scales. I haven't seen any datasheets with confidence intervals for ADEV. Also, manufacturers can be cavalier with temperature sensitivity, which can be huge even for OCXOs and "atomic clock on a chip" devices. So you have the right idea - testing is super important. > > > >_______________________________________________ >time-nuts mailing list -- <mailto:time-nuts@lists.febo.com>time-nuts@lists.febo.com >To unsubscribe send an email to <mailto:time-nuts-leave@lists.febo.com>time-nuts-leave@lists.febo.com
BK
Bob kb8tq
Mon, Oct 17, 2022 1:07 AM

Hi

While temperature is typically used as the “evil” issue, just about any common
mode systematic can be a problem. Atmospheric pressure can be an issue. So
can humidity ( … yikes ….). Line voltage just might get into the mix.

The typical approach when a manufacturer goes to measure this stuff is to go back
to basics. To the best of your ability, you eliminate this or that environmental issue.
That might involve a pretty fancy temperature chamber. It could involve pressure
stabilization. Highly regulated supplies with zero influence from the line voltage are
pretty much a given.

Down in the typical DIY basement, doing all that “stuff” may not be very easy …..

Bob

On Oct 16, 2022, at 8:06 PM, AC0XU (Jim) via time-nuts time-nuts@lists.febo.com wrote:

Erik-

Measuring simultaneously just guarantees that the measurements between the pairs A-B, B-C, A-C are consistent (so you might as well make only two measurements). For example, if you measure A-B and A-C this way, you can combine the measurements to get an estimate of B-C stability that is independent of A's stability. This does not strike me as particularly useful however. Why not just measure B-C directly? Regardless - each phase, frequency or time measurement has associated measurement error and uncertainty.

Suppose one of the time/frequency references A is much more stable than the other two. B and C Then the Tri-Cornered Hat estimates for B and C will be reasonably accurate. However, any attempt to estimate stability of A  with Tri-Cornered hat is doomed. You can easily get negative variance estimates.

Suppose A and B are much more stable than C. Then the Tri-Cornered Hat estimate of C's stability may be more accurate than if you used A-C or B-C alone, because in a statistical sense you are averaging together two stable references to synthesize an even more stable one. The accuracy of the A-C relative stability will depend on how similar A and B are.

If all three references have about the same stability, then the Tri-Cornered Hat estimates would usually be not terrible. The estimate uncertainties depend on the measurement uncertainties.

The above statistical ideas assume that the references have perfectly stable phase plus a random instability, and that that random instabilities of the three references are statistically independent. Otherwise, Tri-Cornered Hat doesn't work well at all. For example, if your three references are proximal, then temperature fluctuations will cause correlated frequency perturbations. I have noticed this effect with OCXOs and rubidium oscillators when the ceiling fan in my lab cycles on and off.

In the second paper I showed that Groslambert estimates are not generally better than Tri-Cornered Hat or the N-reference equivalent thereof. And that is one conclusion from all this - the more references, the better.

Jim

At 01:39 PM 10/16/2022, Erik Kaashoek wrote:

Jim
I'm not able to access the paper as I'm not an IEEE member.
And the math probably will be way over my head anyway.
Do I understand correctly that for testing one should next to something like the ADEV of the counter noise, also measure the histogram to check if the noise is gaussian and the PSD to check for hidden frequencies in the noise?

If the three phase relations are measured simultaneously and the 3 equations solved to calculate A, B and C. Would calculating the ADEV of A,B and C not eliminate the noise of the measurement device?
Keep in mind it's been a very long time since I studied statistics.
Erik

On Sun, Oct 16, 2022, 20:30 AC0XU (Jim) via time-nuts <mailto:time-nuts@lists.febo.comtime-nuts@lists.febo.com> wrote:
I published a couple of papers on Tri-Cornered Hat and Grosslambert.

https://ieeexplore.ieee.org/document/9234829https://ieeexplore.ieee.org/document/9234829

https://www.ion.org/publications/abstract.cfm?articleID=17805

My position is that this is a statistical problem, and it is best to analyze the results statistically. In particular, you can estimate the distributions of the measurements and the computed ADEV estimates. You never end up with unphysical results because even if the simple tri-cornered hat formula produces a negative result for some time scale, the estimate includes an uncertainty interval that includes positive values. Unfortunately, the usual ADEV apps don't perform this analysis. Some day I hope to publish one but that hasn't happened yet. The algorithm I use is described in the first paper.

Sadly, no manufacturer of stable timing devices makes much of an effort to capture the statistical properties of their devices. The most that most manufacturers do is post ADEV values at a few time scales. I haven't seen any datasheets with confidence intervals for ADEV. Also, manufacturers can be cavalier with temperature sensitivity, which can be huge even for OCXOs and  "atomic clock on a chip" devices. So you have the right idea - testing is super important.


time-nuts mailing list -- mailto:time-nuts@lists.febo.comtime-nuts@lists.febo.com
To unsubscribe send an email to mailto:time-nuts-leave@lists.febo.comtime-nuts-leave@lists.febo.com


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com

Hi While temperature is typically used as the “evil” issue, just about any common mode systematic can be a problem. Atmospheric pressure can be an issue. So can humidity ( … yikes ….). Line voltage just *might* get into the mix. The typical approach when a manufacturer goes to measure this stuff is to go back to basics. To the best of your ability, you eliminate this or that environmental issue. That might involve a pretty fancy temperature chamber. It could involve pressure stabilization. Highly regulated supplies with zero influence from the line voltage are pretty much a given. Down in the typical DIY basement, doing all that “stuff” may not be very easy ….. Bob > On Oct 16, 2022, at 8:06 PM, AC0XU (Jim) via time-nuts <time-nuts@lists.febo.com> wrote: > > Erik- > > Measuring simultaneously just guarantees that the measurements between the pairs A-B, B-C, A-C are consistent (so you might as well make only two measurements). For example, if you measure A-B and A-C this way, you can combine the measurements to get an estimate of B-C stability that is independent of A's stability. This does not strike me as particularly useful however. Why not just measure B-C directly? Regardless - each phase, frequency or time measurement has associated measurement error and uncertainty. > > Suppose one of the time/frequency references A is much more stable than the other two. B and C Then the Tri-Cornered Hat estimates for B and C will be reasonably accurate. However, any attempt to estimate stability of A with Tri-Cornered hat is doomed. You can easily get negative variance estimates. > > Suppose A and B are much more stable than C. Then the Tri-Cornered Hat estimate of C's stability may be more accurate than if you used A-C or B-C alone, because in a statistical sense you are averaging together two stable references to synthesize an even more stable one. The accuracy of the A-C relative stability will depend on how similar A and B are. > > If all three references have about the same stability, then the Tri-Cornered Hat estimates would usually be not terrible. The estimate uncertainties depend on the measurement uncertainties. > > The above statistical ideas assume that the references have perfectly stable phase plus a random instability, and that that random instabilities of the three references are statistically independent. Otherwise, Tri-Cornered Hat doesn't work well at all. For example, if your three references are proximal, then temperature fluctuations will cause correlated frequency perturbations. I have noticed this effect with OCXOs and rubidium oscillators when the ceiling fan in my lab cycles on and off. > > In the second paper I showed that Groslambert estimates are not generally better than Tri-Cornered Hat or the N-reference equivalent thereof. And that is one conclusion from all this - the more references, the better. > > Jim > > > At 01:39 PM 10/16/2022, Erik Kaashoek wrote: >> Jim >> I'm not able to access the paper as I'm not an IEEE member. >> And the math probably will be way over my head anyway. >> Do I understand correctly that for testing one should next to something like the ADEV of the counter noise, also measure the histogram to check if the noise is gaussian and the PSD to check for hidden frequencies in the noise? >> >> If the three phase relations are measured simultaneously and the 3 equations solved to calculate A, B and C. Would calculating the ADEV of A,B and C not eliminate the noise of the measurement device? >> Keep in mind it's been a very long time since I studied statistics. >> Erik >> >> On Sun, Oct 16, 2022, 20:30 AC0XU (Jim) via time-nuts <<mailto:time-nuts@lists.febo.com>time-nuts@lists.febo.com> wrote: >> I published a couple of papers on Tri-Cornered Hat and Grosslambert. >> >> <https://ieeexplore.ieee.org/document/9234829>https://ieeexplore.ieee.org/document/9234829 >> >> https://www.ion.org/publications/abstract.cfm?articleID=17805 >> >> My position is that this is a statistical problem, and it is best to analyze the results statistically. In particular, you can estimate the distributions of the measurements and the computed ADEV estimates. You never end up with unphysical results because even if the simple tri-cornered hat formula produces a negative result for some time scale, the estimate includes an uncertainty interval that includes positive values. Unfortunately, the usual ADEV apps don't perform this analysis. Some day I hope to publish one but that hasn't happened yet. The algorithm I use is described in the first paper. >> >> Sadly, no manufacturer of stable timing devices makes much of an effort to capture the statistical properties of their devices. The most that most manufacturers do is post ADEV values at a few time scales. I haven't seen any datasheets with confidence intervals for ADEV. Also, manufacturers can be cavalier with temperature sensitivity, which can be huge even for OCXOs and "atomic clock on a chip" devices. So you have the right idea - testing is super important. >> >> >> >> _______________________________________________ >> time-nuts mailing list -- <mailto:time-nuts@lists.febo.com>time-nuts@lists.febo.com >> To unsubscribe send an email to <mailto:time-nuts-leave@lists.febo.com>time-nuts-leave@lists.febo.com > _______________________________________________ > time-nuts mailing list -- time-nuts@lists.febo.com > To unsubscribe send an email to time-nuts-leave@lists.febo.com