sync computer clock ticks
This has probably been asked many times before, but i am unable to
find any documentation to point me in the right direction. And yes i
am a newbie with clocks...
I'd like to synchronize the clock ticks on 16 different computers
running linux, down to the 1usec area. Is this even possible from a
computer hardware standpoint?
If so, can someone point me towards some documentation that explains
how to wire up 16 computers to a reference clock source and override
the computer internal clocks?
I found the Symetricom website, but it seems light on details on
exactly how a solution like this all connects together and they're
products are likely outside my budget.
Ideally, I'd like to use the serial port on each of these 16 servers
to connect to some device to get the synchronization signal from a
reference. Even if this was possible, i don't think it would get me
to 1usec though.
Thanks
- Michael
On Thu, Jan 03, 2008 at 10:14:01PM -0500, Michael Di Domenico wrote:
I'd like to synchronize the clock ticks on 16 different computers
running linux, down to the 1usec area. Is this even possible from a
computer hardware standpoint?
Ideally, I'd like to use the serial port on each of these 16 servers
to connect to some device to get the synchronization signal from a
reference. Even if this was possible, i don't think it would get me
to 1usec though.
I've had ntp sync of the microsecond/nanosecond time of day
counter on Linux in the 1 us area using serial port timing with custom
mods to the kernel (years ago, I think this is standard now)...
This of course DOES NOT sync the actual clock tick interrupts,
it only syncs the microclock stuff...
The only way I know to sync clock interrupts is to use a common
clock oscillator and even then the actual interrupt times will not be
synced... unless one does some hardware modifications to the motherboards
But it certainly IS possible to get multiple machines to see the
same time of day to with around a microsecond or two...
--
Dave Emery N1PRE/AE, die@dieconsulting.com DIE Consulting, Weston, Mass 02493
"An empty zombie mind with a forlorn barely readable weatherbeaten
'For Rent' sign still vainly flapping outside on the weed encrusted pole - in
celebration of what could have been, but wasn't and is not to be now either."
Michael Di Domenico wrote:
This has probably been asked many times before, but i am unable to
find any documentation to point me in the right direction. And yes i
am a newbie with clocks...
I'd like to synchronize the clock ticks on 16 different computers
running linux, down to the 1usec area. Is this even possible from a
computer hardware standpoint?
If so, can someone point me towards some documentation that explains
how to wire up 16 computers to a reference clock source and override
the computer internal clocks?
I found the Symetricom website, but it seems light on details on
exactly how a solution like this all connects together and they're
products are likely outside my budget.
Ideally, I'd like to use the serial port on each of these 16 servers
to connect to some device to get the synchronization signal from a
reference. Even if this was possible, i don't think it would get me
to 1usec though.
Thanks
- Michael
Michael
Google ntp.
It is unlikely that you will synchronise the computer clocks to within
1us with ntp unless the computer hardware and operating system is
suitably modified.
This can be done cheaply (one machine needs to be setup as a dedicated
stratum 1 ntp server synchronised to a good time source like a M12M gps
timing receiver or equivalent device).
NTP broadcasts the required info as LAN packets.
If you really need that precisionyou may need to use ptp with suitable
hardware (National Instruments and others have suitable boards)
NIST has a webpage on PTP.
Bruce
I've used NTP for years, but i was under the impression that it is not
able to synchonize the clocks below 1 or 2 seconds. Can you point me
towards a specific article that speaks about the configuration
parameters to get ntp sync'd that low?
On Jan 3, 2008 10:25 PM, Bruce Griffiths bruce.griffiths@xtra.co.nz wrote:
Michael Di Domenico wrote:
This has probably been asked many times before, but i am unable to
find any documentation to point me in the right direction. And yes i
am a newbie with clocks...
I'd like to synchronize the clock ticks on 16 different computers
running linux, down to the 1usec area. Is this even possible from a
computer hardware standpoint?
If so, can someone point me towards some documentation that explains
how to wire up 16 computers to a reference clock source and override
the computer internal clocks?
I found the Symetricom website, but it seems light on details on
exactly how a solution like this all connects together and they're
products are likely outside my budget.
Ideally, I'd like to use the serial port on each of these 16 servers
to connect to some device to get the synchronization signal from a
reference. Even if this was possible, i don't think it would get me
to 1usec though.
Thanks
- Michael
Michael
Google ntp.
It is unlikely that you will synchronise the computer clocks to within
1us with ntp unless the computer hardware and operating system is
suitably modified.
This can be done cheaply (one machine needs to be setup as a dedicated
stratum 1 ntp server synchronised to a good time source like a M12M gps
timing receiver or equivalent device).
NTP broadcasts the required info as LAN packets.
If you really need that precisionyou may need to use ptp with suitable
hardware (National Instruments and others have suitable boards)
NIST has a webpage on PTP.
Bruce
time-nuts mailing list -- time-nuts@febo.com
To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
and follow the instructions there.
Michael Di Domenico wrote:
I've used NTP for years, but i was under the impression that it is not
able to synchonize the clocks below 1 or 2 seconds. Can you point me
towards a specific article that speaks about the configuration
parameters to get ntp sync'd that low?
Hi Michael --
PC hardware and operating systems can put some limits on NTP
performance, though with some attention to detail getting ~1 microsecond
with an attached hardware reference clock is fairly straightforward. On
a lightly loaded LAN, machines can be sync'd to a server on the same LAN
to a few hundred microseconds.
Challenges may be (a) in the Windows world, earlier versions (prior to
some SP level of XP) have inherent limitations on timekeeping accuracy,
and (b) Linux does not directly support high accuracy timekeeping with
hardware clocks that provide a PPS signal. For various (mainly
political, it seems) reasons you need to do a kernel patch to enable use
of the PPS signal.
I have a bunch of statistics showing NTP performance at
http://www.febo.com/time-freq/ntp/stats/index.html. There, you can see
how a group of very high performance stratum 1 servers (with directly
connected reference clocks) look across the local LAN, as well as how
some local stratum 2 and external stratum 2 servers look. There's also
a link there to a page that describes what is probably the most accurate
hardware available for an NTP server (and which costs less than $300 to
implement).
John
Michael:
What NTP software are you using? Many are intended just to keep the clock
within a second, and don't work very hard at it.
The best is to get a port of the "real" NTP from the source at
http://www.ntp.org/ They have ports for most major OS's. You should be
able to maintain better than 10 milliseconds to UTC once it has settled for
a while.
Also be sure to check out the NTP server pool project at
http://www.pool.ntp.org/. It's an effort to spread out the load on major
NTP servers.
Keith
Keith Payea
Bryant Labs
kpayea@bryantlabs.net
www.bryantlabs.net
(707) 566-8935
-----Original Message-----
From: time-nuts-bounces@febo.com [mailto:time-nuts-bounces@febo.com] On
Behalf Of Michael Di Domenico
Sent: Friday, January 04, 2008 6:00 AM
To: Discussion of precise time and frequency measurement
Subject: Re: [time-nuts] sync computer clock ticks
I've used NTP for years, but i was under the impression that it is not
able to synchonize the clocks below 1 or 2 seconds. Can you point me
towards a specific article that speaks about the configuration
parameters to get ntp sync'd that low?
On Jan 3, 2008 10:25 PM, Bruce Griffiths bruce.griffiths@xtra.co.nz wrote:
Michael Di Domenico wrote:
This has probably been asked many times before, but i am unable to
find any documentation to point me in the right direction. And yes i
am a newbie with clocks...
I'd like to synchronize the clock ticks on 16 different computers
running linux, down to the 1usec area. Is this even possible from a
computer hardware standpoint?
If so, can someone point me towards some documentation that explains
how to wire up 16 computers to a reference clock source and override
the computer internal clocks?
I found the Symetricom website, but it seems light on details on
exactly how a solution like this all connects together and they're
products are likely outside my budget.
Ideally, I'd like to use the serial port on each of these 16 servers
to connect to some device to get the synchronization signal from a
reference. Even if this was possible, i don't think it would get me
to 1usec though.
Thanks
- Michael
Michael
Google ntp.
It is unlikely that you will synchronise the computer clocks to within
1us with ntp unless the computer hardware and operating system is
suitably modified.
This can be done cheaply (one machine needs to be setup as a dedicated
stratum 1 ntp server synchronised to a good time source like a M12M gps
timing receiver or equivalent device).
NTP broadcasts the required info as LAN packets.
If you really need that precisionyou may need to use ptp with suitable
hardware (National Instruments and others have suitable boards)
NIST has a webpage on PTP.
Bruce
time-nuts mailing list -- time-nuts@febo.com
To unsubscribe, go to
and follow the instructions there.
time-nuts mailing list -- time-nuts@febo.com
To unsubscribe, go to
https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
and follow the instructions there.
In message: 97a7c7ed0801040600x5d2781cfl40e31dd70731492c@mail.gmail.com
"Michael Di Domenico" mdidomenico@gmail.com writes:
: I've used NTP for years, but i was under the impression that it is not
: able to synchonize the clocks below 1 or 2 seconds. Can you point me
: towards a specific article that speaks about the configuration
: parameters to get ntp sync'd that low?
I don't know about other people, but out of the box I get sub 20ms
synchronization. Right now, it is sitting at about 12.5ms:
remote refid st t when poll reach delay offset jitter
---============
*xxxxx.yyyyyy.co 10.7.1.1 2 u 965 1024 377 64.176 -12.570 0.508
My config file is just a simple:
server 1.2.3.4 # xxxxx.yyyyyyy.com
driftfile /mod/etc/ntp.drift
restrict default notrust nomodify
restrict 127.0.0.1
restrict 1.2.3.4
restrict 10.0.0.6
restrict 10.0.0.0 mask 255.255.0.0 notrust
On Jan 4, 2008 12:03 PM, M. Warner Losh imp@bsdimp.com wrote:
I don't know about other people, but out of the box I get sub 20ms
synchronization. Right now, it is sitting at about 12.5ms:
remote refid st t when poll reach delay offset jitter
---============
*xxxxx.yyyyyy.co 10.7.1.1 2 u 965 1024 377 64.176 -12.570 0.508
That's a 12 ms offset from UTC, as far as I understand the original
poster only needs <= 1 microsecond synchronization on his local
network.
-Michael
In message: 25630a120801040918x2203daebs7eb0de5f00ffc56c@mail.gmail.com
"michael taylor" mctylr@gmail.com writes:
: On Jan 4, 2008 12:03 PM, M. Warner Losh imp@bsdimp.com wrote:
: >
: > I don't know about other people, but out of the box I get sub 20ms
: > synchronization. Right now, it is sitting at about 12.5ms:
: >
: > remote refid st t when poll reach delay offset jitter
: >
---============
: > *xxxxx.yyyyyy.co 10.7.1.1 2 u 965 1024 377 64.176 -12.570 0.508
: >
:
: That's a 12 ms offset from UTC, as far as I understand the original
: poster only needs <= 1 microsecond synchronization on his local
: network.
I have no experience with gige networks, but the best I've been able
to do on 100baseT networks is 50us. I'm unsure what <1us
synchronization really means, since that's starting to get below the
level of system calls on fast machines.
Warner
Hi Michael,
I've used NTP for years, but i was under the impression that it is not
able to synchonize the clocks below 1 or 2 seconds. Can you point me
towards a specific article that speaks about the configuration
parameters to get ntp sync'd that low?
at work, I use it to sync all servers and workstations to 4 stratum 1
NTP-servers. I get ~100us offset usually. Without special hardware I'd
say that 1usec is not possible.
Server, Gig-Ethernet, 4 hops (>=Gigabit) to ntp:
*ntp2-rz.rrze.un .GPS. 1 u 758 1024 377 1.225 -0.179 0.068
(HP DL380-G3, Dual 3.2 GHz P4, no powersaving)
-> Temperature fluctuates +/- 3 Deg-Celsius every 20sec due to HVAC
Client, 100Mbit, 5 hops (last 4 are better than Gigabit):
*ntp2-rz.rrze.un .GPS. 1 u 987 1024 377 1.577 -0.111 0.091
(P4-Celeron, 2.4 GHz, currently throttled to 300 MHz)
-> Temperature currently stable and possibly very cool, have not
been in my office for a week :-)
My main problem is probably that ntp switches between three equally good
servers quite frequently (it just did).
Chris
I forgot two things!
(1) It's all Linux, with stock kernel.org kernels, see below.
(2) see comment about the application below.
Christian Vogel schrieb:
at work, I use it to sync all servers and workstations to 4 stratum 1
NTP-servers. I get ~100us offset usually. Without special hardware I'd
say that 1usec is not possible.
Server, Gig-Ethernet, 4 hops (>=Gigabit) to ntp:
*ntp2-rz.rrze.un .GPS. 1 u 758 1024 377 1.225 -0.179 0.068
(HP DL380-G3, Dual 3.2 GHz P4, no powersaving)
-> Temperature fluctuates +/- 3 Deg-Celsius every 20sec due to HVAC
Kernel 2.6.21.3 SMP
Client, 100Mbit, 5 hops (last 4 are better than Gigabit):
*ntp2-rz.rrze.un .GPS. 1 u 987 1024 377 1.577 -0.111 0.091
(P4-Celeron, 2.4 GHz, currently throttled to 300 MHz)
-> Temperature currently stable and possibly very cool, have not
been in my office for a week :-)
Kernel 2.6.22.1 PREEMP
And: We should not forget the original poster's question about if 1 usec
precision can be achieved? I'd say that the most important thing about
this question is, what application the precision is needed for.
For example, do you want to timestamp interrupts to synchronize
machinery? Or just note down the timestamps of bittorrent downloads very
precisely? Just synchronizing an otherwise idling computer is probably
much easier than a machine that is doing a lot of additional work that
can mess up the timekeeping by clogging the processor or just creating
varying stress on the power-supply lines.
Scatterbrained,
Chris
In message: 477E816D.5010803@vogel.cx
Christian Vogel vogelchr@vogel.cx writes:
: For example, do you want to timestamp interrupts to synchronize
: machinery? Or just note down the timestamps of bittorrent downloads very
: precisely? Just synchronizing an otherwise idling computer is probably
: much easier than a machine that is doing a lot of additional work that
: can mess up the timekeeping by clogging the processor or just creating
: varying stress on the power-supply lines.
That's the real question. Like I said before, when you are down to
the microsecond level, what good is that? The jitter from system
calls and such is going to be high enough to make that not completely
useful. And if you are trying to do things to hardware with software
that requires that level of synchronization, you aren't going to get
it without timestamping done in hardware of some mutually observable
event (pps, packets on the network, etc).
Warner
M. Warner Losh wrote:
In message: 477E816D.5010803@vogel.cx
Christian Vogel vogelchr@vogel.cx writes:
: For example, do you want to timestamp interrupts to synchronize
: machinery? Or just note down the timestamps of bittorrent downloads very
: precisely? Just synchronizing an otherwise idling computer is probably
: much easier than a machine that is doing a lot of additional work that
: can mess up the timekeeping by clogging the processor or just creating
: varying stress on the power-supply lines.
That's the real question. Like I said before, when you are down to
the microsecond level, what good is that? The jitter from system
calls and such is going to be high enough to make that not completely
useful. And if you are trying to do things to hardware with software
that requires that level of synchronization, you aren't going to get
it without timestamping done in hardware of some mutually observable
event (pps, packets on the network, etc).
Warner
PTP achieves submicrosecond synchronisation over small LANs without
network switches.
It uses hardware time stamping of LAN packets.
Someone has even implemented it in software (with degraded performance).
See:
http://ieee1588.nist.gov/
Bruce
In message: 477E8D67.1010505@xtra.co.nz
Bruce Griffiths bruce.griffiths@xtra.co.nz writes:
: M. Warner Losh wrote:
: > In message: 477E816D.5010803@vogel.cx
: > Christian Vogel vogelchr@vogel.cx writes:
: > : For example, do you want to timestamp interrupts to synchronize
: > : machinery? Or just note down the timestamps of bittorrent downloads very
: > : precisely? Just synchronizing an otherwise idling computer is probably
: > : much easier than a machine that is doing a lot of additional work that
: > : can mess up the timekeeping by clogging the processor or just creating
: > : varying stress on the power-supply lines.
: >
: > That's the real question. Like I said before, when you are down to
: > the microsecond level, what good is that? The jitter from system
: > calls and such is going to be high enough to make that not completely
: > useful. And if you are trying to do things to hardware with software
: > that requires that level of synchronization, you aren't going to get
: > it without timestamping done in hardware of some mutually observable
: > event (pps, packets on the network, etc).
: >
: > Warner
: >
: PTP achieves submicrosecond synchronisation over small LANs without
: network switches.
: It uses hardware time stamping of LAN packets.
Right. This is hardware time stamping of mutually observable events
(the two way time exchange).
One can measure down to nanosecond levels with custom hardware with
1588. Sam Stein presented a paper at 2006 PITI that talked about an
average of 2.5ns with a standard deviation of 0.9ns in measuring clock
differences.
: Someone has even implemented it in software (with degraded performance).
: See:
: http://ieee1588.nist.gov/
Right, and the performance here is not much better than ntp, at least
in the tests that I've done. The jitter is slightly better.
Warner
On Fri, Jan 04, 2008 at 10:28:33AM -0700, M. Warner Losh wrote:
I have no experience with gige networks, but the best I've been able
to do on 100baseT networks is 50us. I'm unsure what <1us
synchronization really means, since that's starting to get below the
level of system calls on fast machines.
My experience in the late 90s with syncing Linux was based on
using serial port 1 PPS interrupts with custom code in the kernel to
optimize the serial port modem interrupt path. Time stamps of this
interrupt versus the hardware based microclock (based on an incrementing
counter in the CPU) were used to drive the NTP PLL which computed the
frequency error and time offset for the kernel PLL.
I found the microclock readings of time of day on the 1 PPS were
usually within 1 us (there were a few late ones due to interrupt off
intervals of course). This was on fairly slow server class hardware
(Pentium Pro at 200 MHz).
On ethernet I would not expect that level of accuracy due to the
inherent CSMA/CD algorithm jitter... plus of course whatever the bridge
in the star hub introduces...
Obviously once one gets into the microsecond area one gets into
the world of software relativity where what one means by time of day
depends on where one is looking. What I was using to judge performance
was what an interrupt level call to microclock returned... but inside a
larger software system this will obviously be skewed by the various
calling delays to get to the kernel...
--
Dave Emery N1PRE/AE, die@dieconsulting.com DIE Consulting, Weston, Mass 02493
"An empty zombie mind with a forlorn barely readable weatherbeaten
'For Rent' sign still vainly flapping outside on the weed encrusted pole - in
celebration of what could have been, but wasn't and is not to be now either."
For example, do you want to timestamp interrupts to synchronize
machinery? Or just note down the timestamps of bittorrent downloads very
precisely? Just synchronizing an otherwise idling computer is probably
much easier than a machine that is doing a lot of additional work that
can mess up the timekeeping by clogging the processor or just creating
varying stress on the power-supply lines.
The company i work for makes high performance interconnect solutions
for super computers. i'm working on a demo where we're sending data
around a ring and i wanted to timestamp all the packets as it hits
each node of the ring. the servers could be under some stress when
this is running.
i had a feeling what i wanted to do isn't possible, but i figured i'd
ask some experts before i give up.
Michael Di Domenico wrote:
For example, do you want to timestamp interrupts to synchronize
machinery? Or just note down the timestamps of bittorrent downloads very
precisely? Just synchronizing an otherwise idling computer is probably
much easier than a machine that is doing a lot of additional work that
can mess up the timekeeping by clogging the processor or just creating
varying stress on the power-supply lines.
The company i work for makes high performance interconnect solutions
for super computers. i'm working on a demo where we're sending data
around a ring and i wanted to timestamp all the packets as it hits
each node of the ring. the servers could be under some stress when
this is running.
i had a feeling what i wanted to do isn't possible, but i figured i'd
ask some experts before i give up.
Unless you use packet time stamping cards as used in network
characterisation.
Errors of around 120ns or so are typical, see:
http://nistboulder.net/Presentations/FCSEFTF2003(final).pdf
http://nistboulder.net/Presentations/FCSEFTF2003%28final%29.pdf
Suitable PCI cards are available.
Bruce