weird Raspberry PPS+GPS NTP server behaviour
Hi!
For several weeks I'm seeing strange behavior of my Raspberry Pi based NTP
server with Uputronics GPS PPS expansion board (Ublox MAX-M8Q chip).
Several weeks ago the mean daily jitter was below 300 ns, offset didn't
exceed ±1 µs, provided no restart or configuration change was made. Now the
offset often wanders even below 5 µs, steady change, and then comes back.
You can see it at the screenshot from my Grafana server (attached). As you
can see, the number of satellites visible and TDOP value should not be the
cause of it.
My current ntp.conf is as follows (I use ntpsec and gpsd):
driftfile /var/lib/ntp/ntp.drift
leapfile /var/lib/ntp/leap-seconds.list
statsdir /var/log/ntpstats/
statistics loopstats peerstats
filegen loopstats file loopstats type day enable
filegen peerstats file peerstats type day enable
restrict default limited kod nomodify nopeer noquery
restrict source nomodify noquery
restrict 127.0.0.1
restrict 192.168.0.0 mask 255.255.0.0
tos mindist 0.002
refclock shm unit 1 minpoll 4 maxpoll 4 flag4 1 prefer refid PPS
refclock shm unit 0 minpoll 4 maxpoll 4 time1 0.057984 refid GPS
#server nts.time.nl nts
#server nts.ntp.se:4443 nts
server tempus1.gum.gov.pl prefer
server tempus2.gum.gov.pl
server ntp.task.gda.pl
server ntp.nask.pl
server MY_GPSDO iburst
Before that I thought it was because I used NTP pool, as some time ago
similar behavior was mentioned here on the list, but, as you can see, I do
not use it anymore and the same strange behavior remained.
Any idea what could be the cause of this? Thanks in advance.
Adam
Hi!
For several weeks I'm seeing strange behavior of my Raspberry Pi based NTP
server with Uputronics GPS PPS expansion board (Ublox MAX-M8Q chip).
Several weeks ago the mean daily jitter was below 300 ns, offset didn't
exceed ±1 µs, provided no restart or configuration change was made. Now the
offset often wanders even below 5 µs, steady change, and then comes back.
You can see it at the screenshot from my Grafana server (attached). As you
can see, the number of satellites visible and TDOP value should not be the
cause of it.
My current ntp.conf is as follows (I use ntpsec and gpsd):
driftfile /var/lib/ntp/ntp.drift
leapfile /var/lib/ntp/leap-seconds.list
statsdir /var/log/ntpstats/
statistics loopstats peerstats
filegen loopstats file loopstats type day enable
filegen peerstats file peerstats type day enable
restrict default limited kod nomodify nopeer noquery
restrict source nomodify noquery
restrict 127.0.0.1
restrict 192.168.0.0 mask 255.255.0.0
tos mindist 0.002
refclock shm unit 1 minpoll 4 maxpoll 4 flag4 1 prefer refid PPS
refclock shm unit 0 minpoll 4 maxpoll 4 time1 0.057984 refid GPS
#server nts.time.nl nts
#server nts.ntp.se:4443 nts
server tempus1.gum.gov.pl prefer
server tempus2.gum.gov.pl
server ntp.task.gda.pl
server ntp.nask.pl
server MY_GPSDO iburst
Before that I thought it was because I used NTP pool, as some time ago
similar behavior was mentioned here on the list, but, as you can see, I do
not use it anymore and the same strange behavior remained.
Any idea what could be the cause of this? Thanks in advance.
Adam
---===
What are you doing to that poor RPi to drive its CPU up to 66 C! Rather
hot!
That's the first thing I would investigate if the CPU load is really so low
(9%). Comparing, my oldest RPi server is a lowly RasPi 1B, and it runs at
2% CPU, ~43 C CPU (averaged over a day). It lives in an unheated cupboard
on an outside wall.
Cheers,
David
SatSignal Software - Quality software for you
Web: http://www.satsignal.eu
Email: david-taylor@blueyonder.co.uk
Twitter: @gm8arv
On Mon, Mar 2, 2020 at 5:23 PM David J Taylor via time-nuts <
time-nuts@lists.febo.com> wrote:
Hi!
For several weeks I'm seeing strange behavior of my Raspberry Pi based NTP
server with Uputronics GPS PPS expansion board (Ublox MAX-M8Q chip).
Several weeks ago the mean daily jitter was below 300 ns, offset didn't
exceed ±1 µs, provided no restart or configuration change was made. Now the
offset often wanders even below 5 µs, steady change, and then comes back.
You can see it at the screenshot from my Grafana server (attached). As you
can see, the number of satellites visible and TDOP value should not be the
cause of it.
My current ntp.conf is as follows (I use ntpsec and gpsd):
driftfile /var/lib/ntp/ntp.drift
leapfile /var/lib/ntp/leap-seconds.list
statsdir /var/log/ntpstats/
statistics loopstats peerstats
filegen loopstats file loopstats type day enable
filegen peerstats file peerstats type day enable
restrict default limited kod nomodify nopeer noquery
restrict source nomodify noquery
restrict 127.0.0.1
restrict 192.168.0.0 mask 255.255.0.0
tos mindist 0.002
refclock shm unit 1 minpoll 4 maxpoll 4 flag4 1 prefer refid PPS
refclock shm unit 0 minpoll 4 maxpoll 4 time1 0.057984 refid GPS
#server nts.time.nl nts
#server nts.ntp.se:4443 nts
server tempus1.gum.gov.pl prefer
server tempus2.gum.gov.pl
server ntp.task.gda.pl
server ntp.nask.pl
server MY_GPSDO iburst
Before that I thought it was because I used NTP pool, as some time ago
similar behavior was mentioned here on the list, but, as you can see, I do
not use it anymore and the same strange behavior remained.
Any idea what could be the cause of this? Thanks in advance.
Adam
---===
What are you doing to that poor RPi to drive its CPU up to 66 C! Rather
hot!
That's the first thing I would investigate if the CPU load is really so
low
(9%). Comparing, my oldest RPi server is a lowly RasPi 1B, and it runs at
2% CPU, ~43 C CPU (averaged over a day). It lives in an unheated cupboard
on an outside wall.
Cheers,
David
It's ~65°C on purpose. I'm using ntpheat (
https://blog.ntpsec.org/2017/02/01/heat-it-up.html) to keep stable
temperature no matter what it does. It turns my raspberry into kind of OCXO
:)
Anyway, ntpheat runs there for months, so I guess it's not the culprit.
Cheers,
Adam
It's ~65°C on purpose. I'm using ntpheat (
https://blog.ntpsec.org/2017/02/01/heat-it-up.html) to keep stable
temperature no matter what it does. It turns my raspberry into kind of OCXO
:)
Anyway, ntpheat runs there for months, so I guess it's not the culprit.
Cheers,
Adam
---=======
Agreed, if it was running before then it's not the culprit this time.
I didn't know about that program and I'd like to try it here. A neat idea.
Is it available stand-alone - ready to run? I could compile it here given
the source, but not if it has dozens of dependencies on NTPsec, which I
don't use.
Temperature is certainly the prime contributor to instability here (or
temperature variations produced by CPU load variations).
Cheers,
David
SatSignal Software - Quality software for you
Web: http://www.satsignal.eu
Email: david-taylor@blueyonder.co.uk
Twitter: @gm8arv
On Mon, Mar 2, 2020 at 7:58 PM David J Taylor via time-nuts <
time-nuts@lists.febo.com> wrote:
It's ~65°C on purpose. I'm using ntpheat (
https://blog.ntpsec.org/2017/02/01/heat-it-up.html) to keep stable
temperature no matter what it does. It turns my raspberry into kind of OCXO
:)
Anyway, ntpheat runs there for months, so I guess it's not the culprit.
Cheers,
Adam
---=======
Agreed, if it was running before then it's not the culprit this time.
I didn't know about that program and I'd like to try it here. A neat
idea.
Is it available stand-alone - ready to run? I could compile it here given
the source, but not if it has dozens of dependencies on NTPsec, which I
don't use.
Temperature is certainly the prime contributor to instability here (or
temperature variations produced by CPU load variations).
Cheers,
David
The source code is here:
https://github.com/ntpsec/ntpsec/blob/master/contrib/ntpheat
It imports ntp.util. But from my rudimentary knowledge of Python I see that
it only uses it to get version number. So I think removing lines 24-29 and
51-53 should make it work without ntpsec.
Cheers,
Adam
The source code is here:
https://github.com/ntpsec/ntpsec/blob/master/contrib/ntpheat
It imports ntp.util. But from my rudimentary knowledge of Python I see that
it only uses it to get version number. So I think removing lines 24-29 and
51-53 should make it work without ntpsec.
Cheers,
Adam
Thanks, Adam and Hal. I will have a play. I need to decide which of my
flock to use, as some already sit in a fairly stable environment, and others
are doing real work which may be affected by running the CPU at 100%.
OK, there's a non-critical one rather nearer to a CH radiator than I would
like, so I've started it there (no error messages after the edits you
suggested). I ran it as background and logged out. I guess it doesn't need
any privilege.
https://www.satsignal.eu/mrtg/performance_raspi-12.php
Thanks for drawing my attention to this.
Cheers,
David
SatSignal Software - Quality software for you
Web: http://www.satsignal.eu
Email: david-taylor@blueyonder.co.uk
Twitter: @gm8arv
Thanks, Adam and Hal. I will have a play. I need to decide which of my
flock to use, as some already sit in a fairly stable environment, and others
are doing real work which may be affected by running the CPU at 100%.
OK, there's a non-critical one rather nearer to a CH radiator than I would
like, so I've started it there (no error messages after the edits you
suggested). I ran it as background and logged out. I guess it doesn't need
any privilege.
https://www.satsignal.eu/mrtg/performance_raspi-12.php
Thanks for drawing my attention to this.
You can also keep the GPU busy. That'll work as well with the bonus that
the main cpu stays mostly idle (iirc) for other tasks.
https://vanheusden.com/ntpheat.cpp
requires https://github.com/mn416/QPULib
From: folkert
You can also keep the GPU busy. That'll work as well with the bonus that
the main cpu stays mostly idle (iirc) for other tasks.
https://vanheusden.com/ntpheat.cpp
requires https://github.com/mn416/QPULib
---==============
That's neat! Thanks!
Cheers,
David
SatSignal Software - Quality software for you
Web: http://www.satsignal.eu
Email: david-taylor@blueyonder.co.uk
Twitter: @gm8arv
Folks,
Thanks for all the help. I've put up a simple Web page showing the program
and its results here:
https://www.satsignal.eu/ntp/Raspberry-Pi-ntpheat.html
A good gain on that Raspberry Pi in a centrally heated room and nearer the
radiator than it should be.
Cheers,
David
SatSignal Software - Quality software for you
Web: http://www.satsignal.eu
Email: david-taylor@blueyonder.co.uk
Twitter: @gm8arv