time-nuts@lists.febo.com

Discussion of precise time and frequency measurement

View all threads

Long Wave Radio-Frequency standard testing

GC
Gilles Clement
Sat, Jan 16, 2021 10:24 AM

Hi Paul,

I guess I would ask what the goal of the effort might be.
It seems like a good answer. One thing I was amazed buy was that back in
the 1960s HP wrote some papers on using WWVB. What they did was check the
offset at distance everyday about the same time. This offered quite a bit
more accuracy and avoided all of the night propagation effects. Essentially
from day to day the path is about the same.

Interesting, do you know where I could find such papers ?

If your system is stable enough could you simply put it in holdover at
night?

I don’t think so. It is quite sensitive and when uncoupled the VCO drifts significantly.

The other question I had is how do you then use the locked crystal
oscillator at 5.184 Mhz?
Does it somehow divide down to a useful number?

5184000 =2e9 × 3e4 × 5e3 which leads to 200 possible divisors
Among which : 1000 ; 2000 ; 4000 ; 6000 ; 8000 ; 9000 ; 162k ; 192k ….
I am using the PPS to lock other oscillators
One could use another OCXO frequency (typical 10MHz) and phase lock
on an intermediate common divisor such as 2000Hz or 4000Hz etc...
I am currently looking at this option but it is more
complex and not sure it would provide better performances.

Nice project and good luck. Looking forward to hearing more.
Regards
Paul
WB8TSL

Thank you,
Best,
Gilles.

Hi Paul, > I guess I would ask what the goal of the effort might be. > It seems like a good answer. One thing I was amazed buy was that back in > the 1960s HP wrote some papers on using WWVB. What they did was check the > offset at distance everyday about the same time. This offered quite a bit > more accuracy and avoided all of the night propagation effects. Essentially > from day to day the path is about the same. Interesting, do you know where I could find such papers ? > If your system is stable enough could you simply put it in holdover at > night? I don’t think so. It is quite sensitive and when uncoupled the VCO drifts significantly. > The other question I had is how do you then use the locked crystal > oscillator at 5.184 Mhz? > Does it somehow divide down to a useful number? 5184000 =2e9 × 3e4 × 5e3 which leads to 200 possible divisors Among which : 1000 ; 2000 ; 4000 ; 6000 ; 8000 ; 9000 ; 162k ; 192k …. I am using the PPS to lock other oscillators One could use another OCXO frequency (typical 10MHz) and phase lock on an intermediate common divisor such as 2000Hz or 4000Hz etc... I am currently looking at this option but it is more complex and not sure it would provide better performances. > Nice project and good luck. Looking forward to hearing more. > Regards > Paul > WB8TSL Thank you, Best, Gilles.
JP
JF PICARD
Sat, Jan 16, 2021 11:31 AM

HP journal
On Saturday, January 16, 2021, 12:29:24 PM GMT+1, Gilles Clement clemgill@gmail.com wrote:

Hi Paul,

I guess I would ask what the goal of the effort might be.
It seems like a good answer. One thing I was amazed buy was that back in
the 1960s HP wrote some papers on using WWVB. What they did was check the
offset at distance everyday about the same time. This offered quite a bit
more accuracy and avoided all of the night propagation effects. Essentially
from day to day the path is about the same.

Interesting, do you know where I could find such papers ?

If your system is stable enough could you simply put it in holdover at
night?

I don’t think so. It is quite sensitive and when uncoupled the VCO drifts significantly.

The other question I had is how do you then use the locked crystal
oscillator at 5.184 Mhz?
Does it somehow divide down to a useful number?

5184000 =2e9 × 3e4 × 5e3 which leads to 200 possible divisors
Among which : 1000 ; 2000 ; 4000 ; 6000 ; 8000 ; 9000 ; 162k ; 192k ….
I am using the PPS to lock other oscillators
One could use another OCXO frequency (typical 10MHz) and phase lock
on an intermediate common divisor such as 2000Hz or 4000Hz etc...
I am currently looking at this option but it is more
complex and not sure it would provide better performances. 

Nice project and good luck. Looking forward to hearing more.
Regards
Paul
WB8TSL

Thank you,
Best,
Gilles.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.

HP journal On Saturday, January 16, 2021, 12:29:24 PM GMT+1, Gilles Clement <clemgill@gmail.com> wrote: Hi Paul, > I guess I would ask what the goal of the effort might be. > It seems like a good answer. One thing I was amazed buy was that back in > the 1960s HP wrote some papers on using WWVB. What they did was check the > offset at distance everyday about the same time. This offered quite a bit > more accuracy and avoided all of the night propagation effects. Essentially > from day to day the path is about the same. Interesting, do you know where I could find such papers ? > If your system is stable enough could you simply put it in holdover at > night? I don’t think so. It is quite sensitive and when uncoupled the VCO drifts significantly. > The other question I had is how do you then use the locked crystal > oscillator at 5.184 Mhz? > Does it somehow divide down to a useful number? 5184000 =2e9 × 3e4 × 5e3 which leads to 200 possible divisors Among which : 1000 ; 2000 ; 4000 ; 6000 ; 8000 ; 9000 ; 162k ; 192k …. I am using the PPS to lock other oscillators One could use another OCXO frequency (typical 10MHz) and phase lock on an intermediate common divisor such as 2000Hz or 4000Hz etc... I am currently looking at this option but it is more complex and not sure it would provide better performances.  > Nice project and good luck. Looking forward to hearing more. > Regards > Paul > WB8TSL Thank you, Best, Gilles. _______________________________________________ time-nuts mailing list -- time-nuts@lists.febo.com To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com and follow the instructions there.
GC
Gilles Clement
Mon, Jan 18, 2021 6:33 PM

Hi

Very cool !!!

The red trace is obviously the one to focus on. Some sort of digital loop that
only operates under the “known good” conditions would seem to make sense.

Thanks for sharing

Bob

Hi,
I tried something with the idea to consider night records fluctuations as « outliers » as compared to day records.
Indeed the 3 days record mean value is flat and the histogram quite gaussian.
So I processed the 3 days record (green trace) with Stable32’s « Check Function »,
while removing outliers with decreasing values of the Sigma Factor. The graph below shows the outcome.
The graph with Sigma=0.8 (blue trace) connects rather well with the 1Day record (red trace).
Would this be a workable approach ?
Best,
Gilles.

> Hi > > Very cool !!! > > The red trace is obviously the one to focus on. Some sort of digital loop that > only operates under the “known good” conditions would seem to make sense. > > Thanks for sharing > > Bob Hi, I tried something with the idea to consider night records fluctuations as « outliers » as compared to day records. Indeed the 3 days record mean value is flat and the histogram quite gaussian. So I processed the 3 days record (green trace) with Stable32’s « Check Function », while removing outliers with decreasing values of the Sigma Factor. The graph below shows the outcome. The graph with Sigma=0.8 (blue trace) connects rather well with the 1Day record (red trace). Would this be a workable approach ? Best, Gilles.
BK
Bob kb8tq
Mon, Jan 18, 2021 9:06 PM

Hi

As you throw away samples that are far off the mean, you reduce the sample
rate ( or at least create gaps in the record). Dealing with that could be difficult.

Bob

On Jan 18, 2021, at 1:33 PM, Gilles Clement clemgill@gmail.com wrote:

Hi

Very cool !!!

The red trace is obviously the one to focus on. Some sort of digital loop that
only operates under the “known good” conditions would seem to make sense.

Thanks for sharing

Bob

Hi,
I tried something with the idea to consider night records fluctuations as « outliers » as compared to day records.
Indeed the 3 days record mean value is flat and the histogram quite gaussian.
So I processed the 3 days record (green trace) with Stable32’s « Check Function »,
while removing outliers with decreasing values of the Sigma Factor. The graph below shows the outcome.
The graph with Sigma=0.8 (blue trace) connects rather well with the 1Day record (red trace).
Would this be a workable approach ?
Best,
Gilles.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.

Hi As you throw away samples that are far off the mean, you reduce the sample rate ( or at least create gaps in the record). Dealing with that could be difficult. Bob > On Jan 18, 2021, at 1:33 PM, Gilles Clement <clemgill@gmail.com> wrote: > >> Hi >> >> Very cool !!! >> >> The red trace is obviously the one to focus on. Some sort of digital loop that >> only operates under the “known good” conditions would seem to make sense. >> >> Thanks for sharing >> >> Bob > > Hi, > I tried something with the idea to consider night records fluctuations as « outliers » as compared to day records. > Indeed the 3 days record mean value is flat and the histogram quite gaussian. > So I processed the 3 days record (green trace) with Stable32’s « Check Function », > while removing outliers with decreasing values of the Sigma Factor. The graph below shows the outcome. > The graph with Sigma=0.8 (blue trace) connects rather well with the 1Day record (red trace). > Would this be a workable approach ? > Best, > Gilles. > > > > > > _______________________________________________ > time-nuts mailing list -- time-nuts@lists.febo.com > To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com > and follow the instructions there.
GC
Gilles Clement
Tue, Jan 19, 2021 7:15 AM

Hi,
Yes outliers removal creates gap in Stable32.
The « fill »  function can fills gaps with interpolated values.
It does not change much the graphs, except in the low Tau area (see attached).
Do you know a discussion of impact of outliers removal ?
Gilles.

Le 18 janv. 2021 à 22:06, Bob kb8tq kb8tq@n1k.org a écrit :

Hi

As you throw away samples that are far off the mean, you reduce the sample
rate ( or at least create gaps in the record). Dealing with that could be difficult.

Bob

On Jan 18, 2021, at 1:33 PM, Gilles Clement clemgill@gmail.com wrote:

Hi

Very cool !!!

The red trace is obviously the one to focus on. Some sort of digital loop that
only operates under the “known good” conditions would seem to make sense.

Thanks for sharing

Bob

Hi,
I tried something with the idea to consider night records fluctuations as « outliers » as compared to day records.
Indeed the 3 days record mean value is flat and the histogram quite gaussian.
So I processed the 3 days record (green trace) with Stable32’s « Check Function »,
while removing outliers with decreasing values of the Sigma Factor. The graph below shows the outcome.
The graph with Sigma=0.8 (blue trace) connects rather well with the 1Day record (red trace).
Would this be a workable approach ?
Best,
Gilles.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.

Hi, Yes outliers removal creates gap in Stable32. The « fill » function can fills gaps with interpolated values. It does not change much the graphs, except in the low Tau area (see attached). Do you know a discussion of impact of outliers removal ? Gilles. > Le 18 janv. 2021 à 22:06, Bob kb8tq <kb8tq@n1k.org> a écrit : > > Hi > > As you throw away samples that are far off the mean, you reduce the sample > rate ( or at least create gaps in the record). Dealing with that could be difficult. > > Bob > >> On Jan 18, 2021, at 1:33 PM, Gilles Clement <clemgill@gmail.com> wrote: >> >>> Hi >>> >>> Very cool !!! >>> >>> The red trace is obviously the one to focus on. Some sort of digital loop that >>> only operates under the “known good” conditions would seem to make sense. >>> >>> Thanks for sharing >>> >>> Bob >> >> Hi, >> I tried something with the idea to consider night records fluctuations as « outliers » as compared to day records. >> Indeed the 3 days record mean value is flat and the histogram quite gaussian. >> So I processed the 3 days record (green trace) with Stable32’s « Check Function », >> while removing outliers with decreasing values of the Sigma Factor. The graph below shows the outcome. >> The graph with Sigma=0.8 (blue trace) connects rather well with the 1Day record (red trace). >> Would this be a workable approach ? >> Best, >> Gilles. >> >> >> >> >> >> _______________________________________________ >> time-nuts mailing list -- time-nuts@lists.febo.com >> To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com >> and follow the instructions there. > > > _______________________________________________ > time-nuts mailing list -- time-nuts@lists.febo.com > To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com > and follow the instructions there.
BK
Bob kb8tq
Tue, Jan 19, 2021 1:49 PM

Hi

The normal approach to filling a gap is to put in a point that is the average
of the two adjacent points. The assumption is that this is a “safe” value that
will not blow up the result. That’s probably ok if it is done rarely. The risk is
that you are running a filter process (averaging is a low pass filter).

If you pull out a lot of outliers and replace them, you are doing a lot of filtering.
Since you are measuring noise, filtering is very likely to improve the result.
The question becomes - how representative is the result after a lot of this or
that has been done?

Obviously the answer to all this depends on what you are trying to do. If you
are running a control loop and the output improves, that’s fine. If you are
trying to provide an accurate measure of noise …. maybe not so much :)

Bob

On Jan 19, 2021, at 2:15 AM, Gilles Clement clemgill@gmail.com wrote:

Hi,
Yes outliers removal creates gap in Stable32.
The « fill »  function can fills gaps with interpolated values.
It does not change much the graphs, except in the low Tau area (see attached).
Do you know a discussion of impact of outliers removal ?
Gilles.

Le 18 janv. 2021 à 22:06, Bob kb8tq kb8tq@n1k.org a écrit :

Hi

As you throw away samples that are far off the mean, you reduce the sample
rate ( or at least create gaps in the record). Dealing with that could be difficult.

Bob

On Jan 18, 2021, at 1:33 PM, Gilles Clement clemgill@gmail.com wrote:

Hi

Very cool !!!

The red trace is obviously the one to focus on. Some sort of digital loop that
only operates under the “known good” conditions would seem to make sense.

Thanks for sharing

Bob

Hi,
I tried something with the idea to consider night records fluctuations as « outliers » as compared to day records.
Indeed the 3 days record mean value is flat and the histogram quite gaussian.
So I processed the 3 days record (green trace) with Stable32’s « Check Function »,
while removing outliers with decreasing values of the Sigma Factor. The graph below shows the outcome.
The graph with Sigma=0.8 (blue trace) connects rather well with the 1Day record (red trace).
Would this be a workable approach ?
Best,
Gilles.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.

Hi The normal approach to filling a gap is to put in a point that is the average of the two adjacent points. The assumption is that this is a “safe” value that will not blow up the result. That’s probably ok if it is done rarely. The risk is that you are running a filter process (averaging is a low pass filter). If you pull out a *lot* of outliers and replace them, you are doing a lot of filtering. Since you are measuring noise, filtering is very likely to improve the result. The question becomes - how representative is the result after a lot of this or that has been done? Obviously the answer to all this depends on what you are trying to do. If you are running a control loop and the output improves, that’s fine. If you are trying to provide an accurate measure of noise …. maybe not so much :) Bob > On Jan 19, 2021, at 2:15 AM, Gilles Clement <clemgill@gmail.com> wrote: > > Hi, > Yes outliers removal creates gap in Stable32. > The « fill » function can fills gaps with interpolated values. > It does not change much the graphs, except in the low Tau area (see attached). > Do you know a discussion of impact of outliers removal ? > Gilles. > > > >> Le 18 janv. 2021 à 22:06, Bob kb8tq <kb8tq@n1k.org> a écrit : >> >> Hi >> >> As you throw away samples that are far off the mean, you reduce the sample >> rate ( or at least create gaps in the record). Dealing with that could be difficult. >> >> Bob >> >>> On Jan 18, 2021, at 1:33 PM, Gilles Clement <clemgill@gmail.com> wrote: >>> >>>> Hi >>>> >>>> Very cool !!! >>>> >>>> The red trace is obviously the one to focus on. Some sort of digital loop that >>>> only operates under the “known good” conditions would seem to make sense. >>>> >>>> Thanks for sharing >>>> >>>> Bob >>> >>> Hi, >>> I tried something with the idea to consider night records fluctuations as « outliers » as compared to day records. >>> Indeed the 3 days record mean value is flat and the histogram quite gaussian. >>> So I processed the 3 days record (green trace) with Stable32’s « Check Function », >>> while removing outliers with decreasing values of the Sigma Factor. The graph below shows the outcome. >>> The graph with Sigma=0.8 (blue trace) connects rather well with the 1Day record (red trace). >>> Would this be a workable approach ? >>> Best, >>> Gilles. >>> >>> >>> >>> >>> >>> _______________________________________________ >>> time-nuts mailing list -- time-nuts@lists.febo.com >>> To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com >>> and follow the instructions there. >> >> >> _______________________________________________ >> time-nuts mailing list -- time-nuts@lists.febo.com >> To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com >> and follow the instructions there. > > _______________________________________________ > time-nuts mailing list -- time-nuts@lists.febo.com > To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com > and follow the instructions there.
DD
Dave Daniel
Tue, Jan 19, 2021 6:37 PM

Or one can replace those values with zero. That eliminates them; averaging then proceeds without those values altering the most probable correct average.

DaveD

On Jan 19, 2021, at 08:49, Bob kb8tq kb8tq@n1k.org wrote:

Hi

The normal approach to filling a gap is to put in a point that is the average
of the two adjacent points. The assumption is that this is a “safe” value that
will not blow up the result. That’s probably ok if it is done rarely. The risk is
that you are running a filter process (averaging is a low pass filter).

If you pull out a lot of outliers and replace them, you are doing a lot of filtering.
Since you are measuring noise, filtering is very likely to improve the result.
The question becomes - how representative is the result after a lot of this or
that has been done?

Obviously the answer to all this depends on what you are trying to do. If you
are running a control loop and the output improves, that’s fine. If you are
trying to provide an accurate measure of noise …. maybe not so much :)

Bob

On Jan 19, 2021, at 2:15 AM, Gilles Clement clemgill@gmail.com wrote:

Hi,
Yes outliers removal creates gap in Stable32.
The « fill »  function can fills gaps with interpolated values.
It does not change much the graphs, except in the low Tau area (see attached).
Do you know a discussion of impact of outliers removal ?
Gilles.

Le 18 janv. 2021 à 22:06, Bob kb8tq kb8tq@n1k.org a écrit :

Hi

As you throw away samples that are far off the mean, you reduce the sample
rate ( or at least create gaps in the record). Dealing with that could be difficult.

Bob

On Jan 18, 2021, at 1:33 PM, Gilles Clement clemgill@gmail.com wrote:

Hi

Very cool !!!

The red trace is obviously the one to focus on. Some sort of digital loop that
only operates under the “known good” conditions would seem to make sense.

Thanks for sharing

Bob

Hi,
I tried something with the idea to consider night records fluctuations as « outliers » as compared to day records.
Indeed the 3 days record mean value is flat and the histogram quite gaussian.
So I processed the 3 days record (green trace) with Stable32’s « Check Function »,
while removing outliers with decreasing values of the Sigma Factor. The graph below shows the outcome.
The graph with Sigma=0.8 (blue trace) connects rather well with the 1Day record (red trace).
Would this be a workable approach ?
Best,
Gilles.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.

Or one can replace those values with zero. That eliminates them; averaging then proceeds without those values altering the most probable correct average. DaveD > On Jan 19, 2021, at 08:49, Bob kb8tq <kb8tq@n1k.org> wrote: > > Hi > > The normal approach to filling a gap is to put in a point that is the average > of the two adjacent points. The assumption is that this is a “safe” value that > will not blow up the result. That’s probably ok if it is done rarely. The risk is > that you are running a filter process (averaging is a low pass filter). > > If you pull out a *lot* of outliers and replace them, you are doing a lot of filtering. > Since you are measuring noise, filtering is very likely to improve the result. > The question becomes - how representative is the result after a lot of this or > that has been done? > > Obviously the answer to all this depends on what you are trying to do. If you > are running a control loop and the output improves, that’s fine. If you are > trying to provide an accurate measure of noise …. maybe not so much :) > > Bob > >> On Jan 19, 2021, at 2:15 AM, Gilles Clement <clemgill@gmail.com> wrote: >> >> Hi, >> Yes outliers removal creates gap in Stable32. >> The « fill » function can fills gaps with interpolated values. >> It does not change much the graphs, except in the low Tau area (see attached). >> Do you know a discussion of impact of outliers removal ? >> Gilles. >> >> >> >>> Le 18 janv. 2021 à 22:06, Bob kb8tq <kb8tq@n1k.org> a écrit : >>> >>> Hi >>> >>> As you throw away samples that are far off the mean, you reduce the sample >>> rate ( or at least create gaps in the record). Dealing with that could be difficult. >>> >>> Bob >>> >>>>> On Jan 18, 2021, at 1:33 PM, Gilles Clement <clemgill@gmail.com> wrote: >>>>> >>>>> Hi >>>>> >>>>> Very cool !!! >>>>> >>>>> The red trace is obviously the one to focus on. Some sort of digital loop that >>>>> only operates under the “known good” conditions would seem to make sense. >>>>> >>>>> Thanks for sharing >>>>> >>>>> Bob >>>> >>>> Hi, >>>> I tried something with the idea to consider night records fluctuations as « outliers » as compared to day records. >>>> Indeed the 3 days record mean value is flat and the histogram quite gaussian. >>>> So I processed the 3 days record (green trace) with Stable32’s « Check Function », >>>> while removing outliers with decreasing values of the Sigma Factor. The graph below shows the outcome. >>>> The graph with Sigma=0.8 (blue trace) connects rather well with the 1Day record (red trace). >>>> Would this be a workable approach ? >>>> Best, >>>> Gilles. >>>> >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> time-nuts mailing list -- time-nuts@lists.febo.com >>>> To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com >>>> and follow the instructions there. >>> >>> >>> _______________________________________________ >>> time-nuts mailing list -- time-nuts@lists.febo.com >>> To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com >>> and follow the instructions there. >> >> _______________________________________________ >> time-nuts mailing list -- time-nuts@lists.febo.com >> To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com >> and follow the instructions there. > > > _______________________________________________ > time-nuts mailing list -- time-nuts@lists.febo.com > To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com > and follow the instructions there.
BK
Bob kb8tq
Tue, Jan 19, 2021 9:27 PM

Hi

Assuming the goal is a normal ADEV or xDEV sort of calculation:

If you replace the raw phase values with zero that can mess things up

0 seconds +20 ns
1 seconds +22 ns
2 seconds +23 ns
3 seconds +25 ns
4 seconds +27ns
5 seconds +29 ns

If you “loose” one of those 20 to 30 ns values and replace it with zero, you have significantly
changed the data set.

Even if you are looking at deltas, zero stuffing would be problematic with that
(contrived) phase data set.

1 seconds +2
2 seconds +1
3 seconds +2
4 seconds +2
5 seconds +2

If the objective is something like a PLL then “hold at the last value” is the only practical
answer to the question. You don’t have the next value and you need to stuff something
into the control loop computation.

Bob

On Jan 19, 2021, at 1:37 PM, Dave Daniel kc0wjn@gmail.com wrote:

Or one can replace those values with zero. That eliminates them; averaging then proceeds without those values altering the most probable correct average.

DaveD

On Jan 19, 2021, at 08:49, Bob kb8tq kb8tq@n1k.org wrote:

Hi

The normal approach to filling a gap is to put in a point that is the average
of the two adjacent points. The assumption is that this is a “safe” value that
will not blow up the result. That’s probably ok if it is done rarely. The risk is
that you are running a filter process (averaging is a low pass filter).

If you pull out a lot of outliers and replace them, you are doing a lot of filtering.
Since you are measuring noise, filtering is very likely to improve the result.
The question becomes - how representative is the result after a lot of this or
that has been done?

Obviously the answer to all this depends on what you are trying to do. If you
are running a control loop and the output improves, that’s fine. If you are
trying to provide an accurate measure of noise …. maybe not so much :)

Bob

On Jan 19, 2021, at 2:15 AM, Gilles Clement clemgill@gmail.com wrote:

Hi,
Yes outliers removal creates gap in Stable32.
The « fill »  function can fills gaps with interpolated values.
It does not change much the graphs, except in the low Tau area (see attached).
Do you know a discussion of impact of outliers removal ?
Gilles.

Le 18 janv. 2021 à 22:06, Bob kb8tq kb8tq@n1k.org a écrit :

Hi

As you throw away samples that are far off the mean, you reduce the sample
rate ( or at least create gaps in the record). Dealing with that could be difficult.

Bob

On Jan 18, 2021, at 1:33 PM, Gilles Clement clemgill@gmail.com wrote:

Hi

Very cool !!!

The red trace is obviously the one to focus on. Some sort of digital loop that
only operates under the “known good” conditions would seem to make sense.

Thanks for sharing

Bob

Hi,
I tried something with the idea to consider night records fluctuations as « outliers » as compared to day records.
Indeed the 3 days record mean value is flat and the histogram quite gaussian.
So I processed the 3 days record (green trace) with Stable32’s « Check Function »,
while removing outliers with decreasing values of the Sigma Factor. The graph below shows the outcome.
The graph with Sigma=0.8 (blue trace) connects rather well with the 1Day record (red trace).
Would this be a workable approach ?
Best,
Gilles.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.

Hi Assuming the goal is a normal ADEV or xDEV sort of calculation: If you replace the raw phase values with zero that can mess things up 0 seconds +20 ns 1 seconds +22 ns 2 seconds +23 ns 3 seconds +25 ns 4 seconds +27ns 5 seconds +29 ns If you “loose” one of those 20 to 30 ns values and replace it with zero, you have significantly changed the data set. Even if you are looking at deltas, zero stuffing would be problematic with that (contrived) phase data set. 1 seconds +2 2 seconds +1 3 seconds +2 4 seconds +2 5 seconds +2 If the objective is something like a PLL then “hold at the last value” is the only practical answer to the question. You don’t *have* the next value and you need to stuff something into the control loop computation. Bob > On Jan 19, 2021, at 1:37 PM, Dave Daniel <kc0wjn@gmail.com> wrote: > > Or one can replace those values with zero. That eliminates them; averaging then proceeds without those values altering the most probable correct average. > > DaveD > >> On Jan 19, 2021, at 08:49, Bob kb8tq <kb8tq@n1k.org> wrote: >> >> Hi >> >> The normal approach to filling a gap is to put in a point that is the average >> of the two adjacent points. The assumption is that this is a “safe” value that >> will not blow up the result. That’s probably ok if it is done rarely. The risk is >> that you are running a filter process (averaging is a low pass filter). >> >> If you pull out a *lot* of outliers and replace them, you are doing a lot of filtering. >> Since you are measuring noise, filtering is very likely to improve the result. >> The question becomes - how representative is the result after a lot of this or >> that has been done? >> >> Obviously the answer to all this depends on what you are trying to do. If you >> are running a control loop and the output improves, that’s fine. If you are >> trying to provide an accurate measure of noise …. maybe not so much :) >> >> Bob >> >>> On Jan 19, 2021, at 2:15 AM, Gilles Clement <clemgill@gmail.com> wrote: >>> >>> Hi, >>> Yes outliers removal creates gap in Stable32. >>> The « fill » function can fills gaps with interpolated values. >>> It does not change much the graphs, except in the low Tau area (see attached). >>> Do you know a discussion of impact of outliers removal ? >>> Gilles. >>> >>> >>> >>>> Le 18 janv. 2021 à 22:06, Bob kb8tq <kb8tq@n1k.org> a écrit : >>>> >>>> Hi >>>> >>>> As you throw away samples that are far off the mean, you reduce the sample >>>> rate ( or at least create gaps in the record). Dealing with that could be difficult. >>>> >>>> Bob >>>> >>>>>> On Jan 18, 2021, at 1:33 PM, Gilles Clement <clemgill@gmail.com> wrote: >>>>>> >>>>>> Hi >>>>>> >>>>>> Very cool !!! >>>>>> >>>>>> The red trace is obviously the one to focus on. Some sort of digital loop that >>>>>> only operates under the “known good” conditions would seem to make sense. >>>>>> >>>>>> Thanks for sharing >>>>>> >>>>>> Bob >>>>> >>>>> Hi, >>>>> I tried something with the idea to consider night records fluctuations as « outliers » as compared to day records. >>>>> Indeed the 3 days record mean value is flat and the histogram quite gaussian. >>>>> So I processed the 3 days record (green trace) with Stable32’s « Check Function », >>>>> while removing outliers with decreasing values of the Sigma Factor. The graph below shows the outcome. >>>>> The graph with Sigma=0.8 (blue trace) connects rather well with the 1Day record (red trace). >>>>> Would this be a workable approach ? >>>>> Best, >>>>> Gilles. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> time-nuts mailing list -- time-nuts@lists.febo.com >>>>> To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com >>>>> and follow the instructions there. >>>> >>>> >>>> _______________________________________________ >>>> time-nuts mailing list -- time-nuts@lists.febo.com >>>> To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com >>>> and follow the instructions there. >>> >>> _______________________________________________ >>> time-nuts mailing list -- time-nuts@lists.febo.com >>> To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com >>> and follow the instructions there. >> >> >> _______________________________________________ >> time-nuts mailing list -- time-nuts@lists.febo.com >> To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com >> and follow the instructions there. > > _______________________________________________ > time-nuts mailing list -- time-nuts@lists.febo.com > To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com > and follow the instructions there.
MW
Michael Wouters
Tue, Jan 19, 2021 10:19 PM

Hello Gilles

There's a reasonable way of treating data with gaps in it:

https://iopscience.iop.org/article/10.1088/0026-1394/45/6/S19

Essentially, any averaging interval with missing data is dropped from
the ADEV summation.
This reduces the number of intervals averaged over and increases the
uncertainty but is better than faking data.

Stable32 provides an  implementation of the algorithm. See p18 of the manual.

Cheers
Michael

On Tue, Jan 19, 2021 at 6:18 PM Gilles Clement clemgill@gmail.com wrote:

Hi,
Yes outliers removal creates gap in Stable32.
The « fill »  function can fills gaps with interpolated values.
It does not change much the graphs, except in the low Tau area (see attached).
Do you know a discussion of impact of outliers removal ?
Gilles.

Le 18 janv. 2021 à 22:06, Bob kb8tq kb8tq@n1k.org a écrit :

Hi

As you throw away samples that are far off the mean, you reduce the sample
rate ( or at least create gaps in the record). Dealing with that could be difficult.

Bob

On Jan 18, 2021, at 1:33 PM, Gilles Clement clemgill@gmail.com wrote:

Hi

Very cool !!!

The red trace is obviously the one to focus on. Some sort of digital loop that
only operates under the “known good” conditions would seem to make sense.

Thanks for sharing

Bob

Hi,
I tried something with the idea to consider night records fluctuations as « outliers » as compared to day records.
Indeed the 3 days record mean value is flat and the histogram quite gaussian.
So I processed the 3 days record (green trace) with Stable32’s « Check Function »,
while removing outliers with decreasing values of the Sigma Factor. The graph below shows the outcome.
The graph with Sigma=0.8 (blue trace) connects rather well with the 1Day record (red trace).
Would this be a workable approach ?
Best,
Gilles.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.

Hello Gilles There's a reasonable way of treating data with gaps in it: https://iopscience.iop.org/article/10.1088/0026-1394/45/6/S19 Essentially, any averaging interval with missing data is dropped from the ADEV summation. This reduces the number of intervals averaged over and increases the uncertainty but is better than faking data. Stable32 provides an implementation of the algorithm. See p18 of the manual. Cheers Michael On Tue, Jan 19, 2021 at 6:18 PM Gilles Clement <clemgill@gmail.com> wrote: > > Hi, > Yes outliers removal creates gap in Stable32. > The « fill » function can fills gaps with interpolated values. > It does not change much the graphs, except in the low Tau area (see attached). > Do you know a discussion of impact of outliers removal ? > Gilles. > > > > > Le 18 janv. 2021 à 22:06, Bob kb8tq <kb8tq@n1k.org> a écrit : > > > > Hi > > > > As you throw away samples that are far off the mean, you reduce the sample > > rate ( or at least create gaps in the record). Dealing with that could be difficult. > > > > Bob > > > >> On Jan 18, 2021, at 1:33 PM, Gilles Clement <clemgill@gmail.com> wrote: > >> > >>> Hi > >>> > >>> Very cool !!! > >>> > >>> The red trace is obviously the one to focus on. Some sort of digital loop that > >>> only operates under the “known good” conditions would seem to make sense. > >>> > >>> Thanks for sharing > >>> > >>> Bob > >> > >> Hi, > >> I tried something with the idea to consider night records fluctuations as « outliers » as compared to day records. > >> Indeed the 3 days record mean value is flat and the histogram quite gaussian. > >> So I processed the 3 days record (green trace) with Stable32’s « Check Function », > >> while removing outliers with decreasing values of the Sigma Factor. The graph below shows the outcome. > >> The graph with Sigma=0.8 (blue trace) connects rather well with the 1Day record (red trace). > >> Would this be a workable approach ? > >> Best, > >> Gilles. > >> > >> > >> > >> > >> > >> _______________________________________________ > >> time-nuts mailing list -- time-nuts@lists.febo.com > >> To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com > >> and follow the instructions there. > > > > > > _______________________________________________ > > time-nuts mailing list -- time-nuts@lists.febo.com > > To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com > > and follow the instructions there. > > _______________________________________________ > time-nuts mailing list -- time-nuts@lists.febo.com > To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com > and follow the instructions there.
DD
Dave Daniel
Tue, Jan 19, 2021 10:19 PM

Answers inline

On Jan 19, 2021, at 16:27, Bob kb8tq kb8tq@n1k.org wrote:

Hi

Assuming the goal is a normal ADEV or xDEV sort of calculation:

If you replace the raw phase values with zero that can mess things up

0 seconds +20 ns
1 seconds +22 ns
2 seconds +23 ns
3 seconds +25 ns
4 seconds +27ns
5 seconds +29 ns

If you “loose” one of those 20 to 30 ns values and replace it with zero, you have significantly
changed the data set.

Ok

Even if you are looking at deltas,

Nope. It doesn’t work for deltas.

zero stuffing would be problematic with that
(contrived) phase data set.

1 seconds +2
2 seconds +1
3 seconds +2
4 seconds +2
5 seconds +2

If the objective is something like a PLL then “hold at the last value” is the only practical
answer to the question. You don’t have the next value and you need to stuff something
into the control loop computation.

For a control loop, certainly. For just generating the waveform after DAC using interploation, it works well.

So, my suggestion doesn’t cover all use cases, and I learned something. That makes it a good day.

DaveD

Bob

On Jan 19, 2021, at 1:37 PM, Dave Daniel kc0wjn@gmail.com wrote:

Or one can replace those values with zero. That eliminates them; averaging then proceeds without those values altering the most probable correct average.

DaveD

On Jan 19, 2021, at 08:49, Bob kb8tq kb8tq@n1k.org wrote:

Hi

The normal approach to filling a gap is to put in a point that is the average
of the two adjacent points. The assumption is that this is a “safe” value that
will not blow up the result. That’s probably ok if it is done rarely. The risk is
that you are running a filter process (averaging is a low pass filter).

If you pull out a lot of outliers and replace them, you are doing a lot of filtering.
Since you are measuring noise, filtering is very likely to improve the result.
The question becomes - how representative is the result after a lot of this or
that has been done?

Obviously the answer to all this depends on what you are trying to do. If you
are running a control loop and the output improves, that’s fine. If you are
trying to provide an accurate measure of noise …. maybe not so much :)

Bob

On Jan 19, 2021, at 2:15 AM, Gilles Clement clemgill@gmail.com wrote:

Hi,
Yes outliers removal creates gap in Stable32.
The « fill »  function can fills gaps with interpolated values.
It does not change much the graphs, except in the low Tau area (see attached).
Do you know a discussion of impact of outliers removal ?
Gilles.

Le 18 janv. 2021 à 22:06, Bob kb8tq kb8tq@n1k.org a écrit :

Hi

As you throw away samples that are far off the mean, you reduce the sample
rate ( or at least create gaps in the record). Dealing with that could be difficult.

Bob

On Jan 18, 2021, at 1:33 PM, Gilles Clement clemgill@gmail.com wrote:

Hi

Very cool !!!

The red trace is obviously the one to focus on. Some sort of digital loop that
only operates under the “known good” conditions would seem to make sense.

Thanks for sharing

Bob

Hi,
I tried something with the idea to consider night records fluctuations as « outliers » as compared to day records.
Indeed the 3 days record mean value is flat and the histogram quite gaussian.
So I processed the 3 days record (green trace) with Stable32’s « Check Function »,
while removing outliers with decreasing values of the Sigma Factor. The graph below shows the outcome.
The graph with Sigma=0.8 (blue trace) connects rather well with the 1Day record (red trace).
Would this be a workable approach ?
Best,
Gilles.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.


time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.

Answers inline > On Jan 19, 2021, at 16:27, Bob kb8tq <kb8tq@n1k.org> wrote: > > Hi > > Assuming the goal is a normal ADEV or xDEV sort of calculation: > > If you replace the raw phase values with zero that can mess things up > > 0 seconds +20 ns > 1 seconds +22 ns > 2 seconds +23 ns > 3 seconds +25 ns > 4 seconds +27ns > 5 seconds +29 ns > > If you “loose” one of those 20 to 30 ns values and replace it with zero, you have significantly > changed the data set. Ok > > Even if you are looking at deltas, Nope. It doesn’t work for deltas. > zero stuffing would be problematic with that > (contrived) phase data set. > > 1 seconds +2 > 2 seconds +1 > 3 seconds +2 > 4 seconds +2 > 5 seconds +2 > > If the objective is something like a PLL then “hold at the last value” is the only practical > answer to the question. You don’t *have* the next value and you need to stuff something > into the control loop computation. For a control loop, certainly. For just generating the waveform after DAC using interploation, it works well. So, my suggestion doesn’t cover all use cases, and I learned something. That makes it a good day. DaveD > > Bob > > >> On Jan 19, 2021, at 1:37 PM, Dave Daniel <kc0wjn@gmail.com> wrote: >> >> Or one can replace those values with zero. That eliminates them; averaging then proceeds without those values altering the most probable correct average. >> >> DaveD >> >>> On Jan 19, 2021, at 08:49, Bob kb8tq <kb8tq@n1k.org> wrote: >>> >>> Hi >>> >>> The normal approach to filling a gap is to put in a point that is the average >>> of the two adjacent points. The assumption is that this is a “safe” value that >>> will not blow up the result. That’s probably ok if it is done rarely. The risk is >>> that you are running a filter process (averaging is a low pass filter). >>> >>> If you pull out a *lot* of outliers and replace them, you are doing a lot of filtering. >>> Since you are measuring noise, filtering is very likely to improve the result. >>> The question becomes - how representative is the result after a lot of this or >>> that has been done? >>> >>> Obviously the answer to all this depends on what you are trying to do. If you >>> are running a control loop and the output improves, that’s fine. If you are >>> trying to provide an accurate measure of noise …. maybe not so much :) >>> >>> Bob >>> >>>> On Jan 19, 2021, at 2:15 AM, Gilles Clement <clemgill@gmail.com> wrote: >>>> >>>> Hi, >>>> Yes outliers removal creates gap in Stable32. >>>> The « fill » function can fills gaps with interpolated values. >>>> It does not change much the graphs, except in the low Tau area (see attached). >>>> Do you know a discussion of impact of outliers removal ? >>>> Gilles. >>>> >>>> >>>> >>>>> Le 18 janv. 2021 à 22:06, Bob kb8tq <kb8tq@n1k.org> a écrit : >>>>> >>>>> Hi >>>>> >>>>> As you throw away samples that are far off the mean, you reduce the sample >>>>> rate ( or at least create gaps in the record). Dealing with that could be difficult. >>>>> >>>>> Bob >>>>> >>>>>>> On Jan 18, 2021, at 1:33 PM, Gilles Clement <clemgill@gmail.com> wrote: >>>>>>> >>>>>>> Hi >>>>>>> >>>>>>> Very cool !!! >>>>>>> >>>>>>> The red trace is obviously the one to focus on. Some sort of digital loop that >>>>>>> only operates under the “known good” conditions would seem to make sense. >>>>>>> >>>>>>> Thanks for sharing >>>>>>> >>>>>>> Bob >>>>>> >>>>>> Hi, >>>>>> I tried something with the idea to consider night records fluctuations as « outliers » as compared to day records. >>>>>> Indeed the 3 days record mean value is flat and the histogram quite gaussian. >>>>>> So I processed the 3 days record (green trace) with Stable32’s « Check Function », >>>>>> while removing outliers with decreasing values of the Sigma Factor. The graph below shows the outcome. >>>>>> The graph with Sigma=0.8 (blue trace) connects rather well with the 1Day record (red trace). >>>>>> Would this be a workable approach ? >>>>>> Best, >>>>>> Gilles. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> time-nuts mailing list -- time-nuts@lists.febo.com >>>>>> To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com >>>>>> and follow the instructions there. >>>>> >>>>> >>>>> _______________________________________________ >>>>> time-nuts mailing list -- time-nuts@lists.febo.com >>>>> To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com >>>>> and follow the instructions there. >>>> >>>> _______________________________________________ >>>> time-nuts mailing list -- time-nuts@lists.febo.com >>>> To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com >>>> and follow the instructions there. >>> >>> >>> _______________________________________________ >>> time-nuts mailing list -- time-nuts@lists.febo.com >>> To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com >>> and follow the instructions there. >> >> _______________________________________________ >> time-nuts mailing list -- time-nuts@lists.febo.com >> To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com >> and follow the instructions there. > > > _______________________________________________ > time-nuts mailing list -- time-nuts@lists.febo.com > To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com > and follow the instructions there.