Magnus,

I am aware that you know a lot about these things. Nevertheless I
believe you are starting a most dangerous discussion in the sense that
you put some terms into question of which I believed that they have well
been established. For that reason let me test where we agree and where
not:

Mr. Allan decided that for his new statistical measure the summation
shall run over

square(y(i+1)-y(i))

for frequency data and over

square(x(i+2)-2*x(i+1)+(xi))

for phase data. Both in contrast to the standard deviation where the
summation runs over squares of distances from the mean. This new
variance was called "Allan variance" and its square root "Allan
deviation" to honor Mr. Allan for his work. This variance/deviation has
a certain "overlapping aspect" since a single y(i) or x(i) appears in
multiple terms of the summation. Agreed?

Both terms require that the elements with subsequent indices are spaced
apart at the "Tau" for wich the computation shall be done. Considered a
number of phase measurements spaced 1 s apart then the computation will
run over

square(x(i+2)-2*x(i+1)+(xi))

for Tau = 1 s. If you are going to compute for Tau = 2 s from the SAME
data set you will have to use the "original" samples

square(x(5)-2*x(3)+x(1))

for the first summand and

square(x(7)-2*x(5)+x(3))

for the second summand and

square(x(9)-2*x(7)+x(5))

for the third summand and so on. All indices are incremented by two
between neighbour summands because the next summand is 2 s (or two
original samples) apart from the current summand. Agreed?

As we notice the summation leaves out a number of summands where the
elements are also spaced 2 s apart, for example

square(x(6)-2*x(4)+x(2))

or

square(x(8)-2*x(6)+x(4))

If we use these additional terms in the summation the number of summands
increases a lot and improves the confidence interval of the estimation,
even though the added summands are NOT completely statistical
independend from the original ones and therefore this measure shall be
clearly distincted from the original Allan variance/deviation. The
summation over the original terms plus the added terms delivers the
"Overlapping Allan variance/deviation" in conjunction with a suitable
normation factor. Agreed?

Best regards
Ulrich

Ulrich,

Ulrich Bangert skrev:

I have only recently seen the OADEV being used where as I have seen
countless articles on calculations of these without encountering them,
so from my standpoint OADEV is not well established, which is why I
raised the question in order to "shake the tree" to see what fruits that
I have missed.

Yes, yes....

Actually, what you describe is the estimator formulas rather than
definition. This is also targeting the fine point that I am trying to
make. It's not about the basic definition, but accepted convention to
denote the estimators.

Yes, yes...

Disagree. The estimator formulation that is classically used includes
these "missed" tau0 steps that you claim that OAVAR/OADEV includes. This
is my point. Somewhere along the line the established ADEV estimator
became the OADEV estimator and another estimator took the ADEV place.
This is what I oppose without a more detailed look at things.

I agree that it changes the statistical properties in terms of
confidence interval, but it also change the frequency dependence. The
analysis on frequency dependency needs to be redone as I suspect they do
not always agree.

Cheers,
Magnus

Magnus,

I still do not understand the fine point! A estimator might have this
property and that property and may perform this task good and another
task bad, but at the basics we have a formula and if the formula is new
or different from prior art then the thing needs an name of its own. In
this sense the summation over square(y(i+1)-y(i)) is called the base of
the "Allan variance/deviation" just for historical reasons. So the name
is "Allen deviation" and it is defined by its formula.

ADEV estimator

The OAVAR/OADEV has this name of its own BECAUSE it includes the
summands that are missed by the original AVAR/ADEV so its needs an name
of its own.

OADEV estimator

If you had said: "The currently established estimator for oscillator
stability is the OADEV estimator" I would have perfectly agreed.
However, ADEV does already point to a different thing, so to say "Today
we call ADEV what was formerly called OADEV and what was formerly called
ADEV now is also called different" is not excused with a certain
sloppiness in language but simply wrong use of terms. Exactly this is
the point why I said that the discussion is dangerous. This is not a
change in paradigm this is a case of inaccurate use of scientifical
terms.

Best regards
Ulrich

Ulrich Bangert wrote:

Ulrich. Magnus

Perhaps the situation is best summarised in /NIST special Publication
1065/ (can be downloaded as 2220.pdf
http://tf.nist.gov/timefreq/general/pdf/2220.pdf from NIST) wherein it
is stated that ADEV, AVAR are often taken to mean the overlapped form of
the Allan deviation at least in the US.

The attached figure graphically illustrates the difference between the
ordinary and the overlapped versions.

Bruce

Ulrich,

Ulrich Bangert skrev:

This part we agree on, however, you fail to see that what I try to point
out is that you seems to have the wrong reference to start with. What I
am trying to say is that it seems that ADEV is being used to identify a
different estimator than I have in my old material, including the
articles collected in NIST TN1337, for instance "Time and Frequency
(Time-Domain) Characterization, Estimation, and Prediction of Precision
Clocks and Oscillators" by David W. Allan.
http://tf.nist.gov/timefreq/general/tn1337/Tn121.pdf
See page 4 and formulas 8 and 9. These are overlapping.

A further reference would be the IEEE standard found in

http://tf.nist.gov/timefreq/general/tn1337/Tn139.pdf

This is also overlapping (from page 2):

                         N-2m
                          ___
  2              1       \                        2

sigma  (tau) = -----------  >  (x    - 2x    + x )
y                  2  /___  i+2m    i+m    i
2(N-2m)tau  i = 1

a non-interleaved variant would have to be written as (assuming that m
divides N):

           N
           - - 2
           m
            ___
  m        \                              2

------------  >  (x      - 2x      + x  )
2  /___  (i+2)m    (i+1)m    im
2(N-2m)tau    i = 1

and these obviously isn't the same, the later form skips over samples
not being a multiple of m.

Also, it is still overlapping in the sense that samples is being re-used.

I deeply disagree, see my reference to early papers (I agree not
original). Also, the standardised form is overlapping.

This is the reason for me to react.

Well, that part was never what we disagreed on IMHO.

Well, if we were doing a shift in interpretation I fully agree with you,
but what I reacted on was due to a shift in interpretation as I
experienced it and when looking at the old reference material (altought
I have not had the time for an extensive search that I would feel
confident with). The issue was that I detected the dangerous shift and I
wanted to bring it up to bring it back on tracks, or at least learn
something useful.

I really kindly ask you or anyone else to bring forward articles
describing the non-overlapping ADEV and help plotting out the issues.
What has become standardised (and thus assumed accepted) as the ADEV
estimator is overlappping unless you can point out that I have made a
very deep misunderstanding of all those papers, in which case I would be
happy to be corrected.

Cheers,
Magnus

Bruce Griffiths skrev:

This is an excellent contribution to the discussion as it details the
"Original Allan variance" and "Overlapping Allan variance" and also has
a special information box detailing that AVAR and ADEV used mainly
(notice standards and much of the reference material) for the
overlapping. It also shows the inability for the 2-sample variance
(Original Allan variance) to create smooth and good curves.

That the 2-sample variance existed and was the basis for Allan variance
was know, but the overlapping formulation of the established AVAR and
ADEV terms estimators is motivated and accepted for its improved
precission was also known to me indirectly, in the sense that I saw they
where not directly equivalent but saying the same thing, then I forgot
about it until this mysterious OADEV/OAVAR came up recently.

There is however a huge difference between the 2-sample variance being
the original Allan variance and being AVAR. Here I tend to rely on
standards set by IEEE and ITU-T as they establish a defined relationship
and I suspect some wisedom was applied in the process. Which estimator
is we best serviced by to get defined as AVAR? They chose the
overlapping one and it has also been the most used in theoretical
analysis to the best of my knowledge.

The plots given in the NIST SP 1065 figure 8 is very descriptive on the
difference.

In the end, I think we must realize that there is a distinction between the
2
sigma  (tau)
y

and

AVAR(tau)

The former is the Original Allan Variance where as the later is the
Overlapping Allan Variance. It is easy to confuse the two. Maybe this is
the fundamental problem and not the definitions themselfs.

Cheers,
Magnus

Magnus,

the paper http://tf.nist.gov/timefreq/general/tn1337/Tn121.pdf is
thought-provoking. Not that I would simply say that you are right, but
because I dont't understand some things.

No discussion about that, simply correct.

However the note to figure 8 as well as the note to figure 9 cover the
non-overlapping case. Indeed formulas (8) and (10) are overlapping and
to me it is a bit kind of magic where they come from in regard to thise
two notes.

Do you agree to the fact that the ADEV for Tau = 2 s should be the same,
regardless if computed from 1 s spaced phase data or from 2 s spaced
phase data?

Best regards
Ulrich

Hi Tom,

Tom Van Baak skrev:

This at least explains the origin of the OADEV name. Many thanks.

If you look at the NIST paper that Bruce gave a link for, it makes very
clear that the overlapping Allan deviation/variance should be used and
that the original Allan deviation/variance should only be used "when
necessary". The original form allows for some useful reductions in
computation as a form of "quick" processing could be made with factor 2
tau steps.

I will look at it again, as I recall I have looked at it earlier.

Cheers,
Magnus