On Samstag, 12. September 2020 23:53:39 CEST Bob kb8tq wrote:

That is interesting, why wouldn't they use the information if it's available?
Watching the temperature gives you a prediction of what is going to happen,
while watching the phase only shows what has already happened. And knowing
what's about to happen is always very favorable in control theory.

Hi

The point is that there is no need to compensate the part while it is locked.
It meets all the specs required of it by virtue of the lock to GPS. The big issue
on these devices ( = cell tower GPSDO’s) is holdover. That’s where all the
magic comes in.

Indeed, trying to compensate and learn at the same time generates even
more issues to deal with. The temperature effect is not a simple linear / first
order sort of thing ….

Bob

But doesn't the compensation help generate a feed-forward term, a way to
make the integrator better managed ?

On Sun, Sep 13, 2020 at 2:31 PM Bob kb8tq kb8tq@n1k.org wrote:

Hi

Not done that way in this case. Keep in mind that the “stock” configuration only runs a loop time
constant in the 10’s of seconds.

On the TBolt, the main source of temperature error is the DAC. Tossing a bit more “stuff” at that
part of the design would have been higher on the list than playing with compensation. It met the
spec so why spend the money …..

Bob

Bob kb8tq writes:

Data point:

20 years ago, I needed to buy a batch of a dozen OCXO's for hand-built
NTP servers for the new danish ATC network.

This is where the Soekris 4501 trick comes from, and I wanted a
frequency I could feed directly to the Soekris if at all possible.

I ended up "buying into" an existing production run of IsoTemp
OCXO-0131's at 32.768 MHz spec'ed for "some telco application".

I attach a 16 year long plot of the frequency error for one of
the servers on my side of the firewall-of-doom.

(There were a network misconfiguration, so the data from middle of
2011 to end of 2015 is not valid - long story, mostly about fibers
and backhoes)

It took this one five years to "settle in", but for the last decade
it has consistently drifted 2.2e-9/year = 7e-16/second.

The software PLL in these NTP servers did model drift and we tested
that, unplugging the GPS antenna on one server.

After 18 weeks in hold-over it had still not drifted out of spec.

In the 19th week, they added two 3.5" disks to a server mounted in
the same rack, literally opening the rack-door, checking the dymo
to make sure they had the right server, pulling out the spacers,
stuck in the disks, clsoed the rack-door and left the room.

The plot for that server clearly shows a change in OCXO-behaviour
and 121 hours later, the NTP finally wandered out of spec.

Other tests rules out thermal causes, so our conclusion is that
either the vibration from the intervention or from the rotation
of the two new disks did it.

My personal conclusion is that high-order PLL's are fun, but they
require patience on a timescale of years and you need not bother
unless you have a very benign environment.

Poul-Henning

--
Poul-Henning Kamp      | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG        | TCP/IP since RFC 956
FreeBSD committer      | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

On Sonntag, 13. September 2020 15:05:36 CEST Bob kb8tq wrote:

It depends. For short time constants, yes, likely the control loop is able to
follow the temperature-induced drift of the OCXO. But you might want the TC to
be as long as possible. Naturally, the control loop becomes very slow to
react. By following the temperature, you have an additional input that allows
the controller to act more quickly to a changing environment. Effectively this
will lead to higher stability of the output.

Maybe, but a linear approximation is probably better than nothing as it still
reduces the amount of change that the control loop has to counteract.

If you look at the attached screenshot - there's roughly 5500 seconds of data
from my GPSDO. At about 2500 seconds the temperature compensation was engaged.
It's just a linear correction factor for the DAC output. The factor was
computed using a linear regression fit to find the correlation between the
temperature and the OCXO drift. In the DAC chart the blue trace is the actual,
corrected DAC output, the orange trace is the uncorrected DAC computed from
the TIC input. The green trace is the OCXO drift. Without the temperature
compensation, the TIC takes a dive down to almost -15ns. While the temperature
continues its decline at an approximately constant rate, after the
compensation was engaged the TIC excursions are smaller and therefore the
stability of the output higher.

Now, of course the picture gets somewhat muddy if you keep computing the
compensation factor while the compensation is active, but what you still get
is a residual factor, which you can maybe use to refine the compensation
subsequently. The correlation factor (r_xy) will tell if there's still
something to be done.

Matthias Welwarsky writes:

The word you are looking for here is not "Temperature Coefficient"
but "Thermal Impedance" (more on this below).

Only if you first spend months and years measuring all the parameters
and time constants of the multiphysics model you use, well enough
to make useful predictions with it.

No, it is usually worse.

A major part of the trouble is the complex hysteresis-effects when
the temperture changes direction: The components which warm fast
also cools fast.

For example:  When the temp goes up you will likely find that your
DAC warms faster than the XTAL, but when the temp goes down it also
cools faster than the XTAL.

That means the temperature difference between the DAC and XTAL depends
on the temperature rising or dropping, so you have to model the tempco
and temperature of them individually.

This is nowhere near enough data to show anything.  Collect a full
week with/without and we can talk.

In the end, the proof is in your allan-variance, if it improves, you got
something, if it does not, you wasted your time.

The easiest and cheapest way for you to get better results, is to increase
the thermal impedance between the surroundings and your GPSDO.

That will make the temperature change slower, which also means it
changes less and therefore your hysteresis effects also get smaller.

Note that "thermal impedance" is not the same as "thermal insulation":

Thermal insulation materials have high thermal resistance and low
thermal mass, and wrapping your GPSDO in that would just make it
run hot.

Think of it is a thermal RC filter with a huge resistor and a small
capacitor.

We want the a low to moderate resistor, so the GPSDO can still dump
its heat, with a huge capacitor to filter out the changes in
temperature.

Apart from the entire "get electronics wet" thing, water would have
been near perfect.

Table-top granite (about 2cm thick) is really great, but not very accessible.

Metals almost conduct heat too well, but a box of 1-5cm thick iron
plates works great, but pay attention to the weight.

For most of us, concrete is the way to go:

Get three cinderblocks of the kind that looks like a 'H' with two
horizontal bars.

Put the first cinderblock down on its side.

Put the next cinderblock on top of it in normal orientation.

Put your OCXO into the cavity.

Hack notches in the edge of the cinderblock for the cables.

Put the third cinderblock on top, also on its side.

You have now increased the thermal impedance by almost two orders
of magnitude, and your PLL will be boooooored.

If need be, you can make the central cavity more air-tight by
"sealing" between the cinderblocks with a layer of cloth or
tissue-paper.

--
Poul-Henning Kamp      | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG        | TCP/IP since RFC 956
FreeBSD committer      | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

On Mon, Sep 14, 2020 at 10:36 AM Poul-Henning Kamp phk@phk.freebsd.dk wrote:

There are alternatives to water...
https://www.3m.co.uk/3M/en_GB/novec-uk/applications/immersion-cooling/

Cheers

Arne

In 1973 I moved for TI to Dallas and had a 20 foot hole drilled to place my Sulzer One alternate.. Today I monitor my lab closely to better understand what to do. The monitor is placed on the top of the HP5065A  which stands on its sides.Attached monitor data that shows when I had a 3 hour power outage including no AC but also pressure now with a storm 120 miles west.Bert Kehren  In a message dated 9/14/2020 5:36:48 AM Eastern Standard Time, phk@phk.freebsd.dk writes: 

Matthias Welwarsky writes:

The word you are looking for here is not "Temperature Coefficient"
but "Thermal Impedance" (more on this below).

Only if you first spend months and years measuring all the parameters
and time constants of the multiphysics model you use, well enough
to make useful predictions with it.

No, it is usually worse.

A major part of the trouble is the complex hysteresis-effects when
the temperture changes direction: The components which warm fast
also cools fast.

For example:  When the temp goes up you will likely find that your
DAC warms faster than the XTAL, but when the temp goes down it also
cools faster than the XTAL.

That means the temperature difference between the DAC and XTAL depends
on the temperature rising or dropping, so you have to model the tempco
and temperature of them individually.

This is nowhere near enough data to show anything.  Collect a full
week with/without and we can talk.

In the end, the proof is in your allan-variance, if it improves, you got
something, if it does not, you wasted your time.

The easiest and cheapest way for you to get better results, is to increase
the thermal impedance between the surroundings and your GPSDO.

That will make the temperature change slower, which also means it
changes less and therefore your hysteresis effects also get smaller.

Note that "thermal impedance" is not the same as "thermal insulation":

Thermal insulation materials have high thermal resistance and low
thermal mass, and wrapping your GPSDO in that would just make it
run hot.

Think of it is a thermal RC filter with a huge resistor and a small
capacitor.

We want the a low to moderate resistor, so the GPSDO can still dump
its heat, with a huge capacitor to filter out the changes in
temperature.

Apart from the entire "get electronics wet" thing, water would have
been near perfect.

Table-top granite (about 2cm thick) is really great, but not very accessible.

Metals almost conduct heat too well, but a box of 1-5cm thick iron
plates works great, but pay attention to the weight.

For most of us, concrete is the way to go:

    Get three cinderblocks of the kind that looks like a 'H' with two
    horizontal bars.

    Put the first cinderblock down on its side.

    Put the next cinderblock on top of it in normal orientation.

    Put your OCXO into the cavity.

    Hack notches in the edge of the cinderblock for the cables.

    Put the third cinderblock on top, also on its side.

You have now increased the thermal impedance by almost two orders
of magnitude, and your PLL will be boooooored.

If need be, you can make the central cavity more air-tight by
"sealing" between the cinderblocks with a layer of cloth or
tissue-paper.

--
Poul-Henning Kamp      | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG        | TCP/IP since RFC 956
FreeBSD committer      | BSD since 4.3-tahoe   
Never attribute to malice what can adequately be explained by incompetence.

time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.

Hi

At that point the part meets all relevant specifications. The customer is happy
( = it meets spec). Any further work on the part simply increases the cost of the
device. Why would you do this if it already is fully spec compliant ?

If you spend a few years digging into this and trying various approaches, be sure
to apply the same firmware to multiple devices. If it all is a “learning” process. How
do you suggest training? The “worst case” will always be an excursion outside the
normal variation the part sees…..

If the data is collected in a formal fashion (via temperature test) and then programmed
into the device, how stable is it long term? How is it impacted by air flow and unit
orientation? ( hint: a lot ….).

Not cheap or easy …..

Bob