Network interface cards that support timestamping
Hey All,
I'm curious as to what the collective experience is amongst this group when it comes to feeding a timestamp signal into a NIC, either for PTP purposes or for as a normal PPS refclock. The Intel i210 PCIe network card has a few software defined pins that can be used for this purpose, and the chrony documentation has some information about it:
https://chrony.tuxfamily.org/examples.html#_server_using_reference_clock_on_nic
I imagine most of us here use traditional RS232 serial ports to get this signal into an x86 computer, usually using the DCD pin. I've found some implementations of RS232 on PCs don't implement all of the additional signaling pins, so sometimes DCD is flat out missing. In other cases - especially with smaller embedded-type boards only UARTs with Rx/Tx are available to the user. (I'm fiddling with using GPIO pins on a few such systems, but it's quite a struggle.) So, there is a need for getting PPS signals into systems.
There are also highly specialized network interface cards, such as this one: https://www.ebay.com/itm/404117321209
... which does explicitly support PTP timestamping, but it's not clear to me if it could pull in a "standard" PPS signal to be made available to chrony or NTPsec. I haven't found much documentation on the NT20E2 yet, unfortunately, and it may simply not exist for hobbyists.
Does anyone here know if any other network cards that support software defined pins, like the i210 does, or any other methods for getting a PPS signal to be fed into a PC, and of course interpreted correctly?
Thanks!
John
Most common way of getting 1PPS into a computer without a dedicated 1PPS
interface is to drive a RS-232 port’s Carrier Detect pin (pin 1 on DB-9)
with the 1PPS signal
On Mon, Jan 30, 2023 at 3:19 PM John Miller via time-nuts <
time-nuts@lists.febo.com> wrote:
Hey All,
I'm curious as to what the collective experience is amongst this group
when it comes to feeding a timestamp signal into a NIC, either for PTP
purposes or for as a normal PPS refclock. The Intel i210 PCIe network card
has a few software defined pins that can be used for this purpose, and the
chrony documentation has some information about it:
https://chrony.tuxfamily.org/examples.html#_server_using_reference_clock_on_nic
I imagine most of us here use traditional RS232 serial ports to get this
signal into an x86 computer, usually using the DCD pin. I've found some
implementations of RS232 on PCs don't implement all of the additional
signaling pins, so sometimes DCD is flat out missing. In other cases -
especially with smaller embedded-type boards only UARTs with Rx/Tx are
available to the user. (I'm fiddling with using GPIO pins on a few such
systems, but it's quite a struggle.) So, there is a need for getting PPS
signals into systems.
There are also highly specialized network interface cards, such as this
one: https://www.ebay.com/itm/404117321209
... which does explicitly support PTP timestamping, but it's not clear to
me if it could pull in a "standard" PPS signal to be made available to
chrony or NTPsec. I haven't found much documentation on the NT20E2 yet,
unfortunately, and it may simply not exist for hobbyists.
Does anyone here know if any other network cards that support software
defined pins, like the i210 does, or any other methods for getting a PPS
signal to be fed into a PC, and of course interpreted correctly?
Thanks!
John
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
John Miller via time-nuts writes:
Does anyone here know if any other network cards that support
software defined pins, like the i210 does, or any other methods for
getting a PPS signal to be fed into a PC, and of course interpreted
correctly?
We used Intel '599 chip based cards in the ELT adaptive optics prototype
compute cluster, and it has both hardware timestamping of packets and
a couple of external signals.
Unfortunately the boards we used had not wired those external signals
to anything reachable by me.
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
Hey John,
as far as I'm aware, there are a few Ethernet ICs which do have hardware
pins for PTP purposes. The Broadcom BCM54210PE comes to mind as used on
the Raspberry Pi CM4 (not the regular Raspberry Pi 4, the Compute
module, they use different Ethernet PHYs). I believe the Intel i225 also
has dedicated Pins for PTP, but can't find the datasheet I once obtained
to make sure. IF these pins however are accessible on the typical NIC
(as in plugin card, not just the silicon chip itself) depends very much
on who did the design. The abovementioned Raspberry pi CM4 has two
dedicated pins available, directly wired to the appropriate pins of the
PHY chip, but driver support is not entirely there yet last I heard.
For generic off-the-shelf i210/i225 cards, I have yet to come across one
which has documented hardware pins for PTP purposes, but the sample size
is rather small (four or five different cards in total...).
Judging from a "grep -r TX_HARDWARE ./*" in Linux kernel tree there seem
to be a few drivers which support hardware timestamping of some sort, so
my first guess would be that those coming up there could potentially
have pins which can be used for timestamping purposes. There might be
more as not everyone is publicly documenting their chips in a way that's
actually useful, but at least it's a start.
HTH,
Florian
On 30.01.23 19:15, John Miller via time-nuts wrote:
Hey All,
I'm curious as to what the collective experience is amongst this group when it comes to feeding a timestamp signal into a NIC, either for PTP purposes or for as a normal PPS refclock. The Intel i210 PCIe network card has a few software defined pins that can be used for this purpose, and the chrony documentation has some information about it:
https://chrony.tuxfamily.org/examples.html#_server_using_reference_clock_on_nic
I imagine most of us here use traditional RS232 serial ports to get this signal into an x86 computer, usually using the DCD pin. I've found some implementations of RS232 on PCs don't implement all of the additional signaling pins, so sometimes DCD is flat out missing. In other cases - especially with smaller embedded-type boards only UARTs with Rx/Tx are available to the user. (I'm fiddling with using GPIO pins on a few such systems, but it's quite a struggle.) So, there is a need for getting PPS signals into systems.
There are also highly specialized network interface cards, such as this one: https://www.ebay.com/itm/404117321209
... which does explicitly support PTP timestamping, but it's not clear to me if it could pull in a "standard" PPS signal to be made available to chrony or NTPsec. I haven't found much documentation on the NT20E2 yet, unfortunately, and it may simply not exist for hobbyists.
Does anyone here know if any other network cards that support software defined pins, like the i210 does, or any other methods for getting a PPS signal to be fed into a PC, and of course interpreted correctly?
Thanks!
John
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
Hi
There are PTP specific cards that bring out various PPS pins. They are pretty
far from what I’d call a “normal” interface card. They show up on eBay from time
to time. It’s not clear what sort of a tangle one gets in finding drivers for those cards.
Bob
On Jan 30, 2023, at 4:40 PM, usenet--- via time-nuts time-nuts@lists.febo.com wrote:
Hey John,
as far as I'm aware, there are a few Ethernet ICs which do have hardware pins for PTP purposes. The Broadcom BCM54210PE comes to mind as used on the Raspberry Pi CM4 (not the regular Raspberry Pi 4, the Compute module, they use different Ethernet PHYs). I believe the Intel i225 also has dedicated Pins for PTP, but can't find the datasheet I once obtained to make sure. IF these pins however are accessible on the typical NIC (as in plugin card, not just the silicon chip itself) depends very much on who did the design. The abovementioned Raspberry pi CM4 has two dedicated pins available, directly wired to the appropriate pins of the PHY chip, but driver support is not entirely there yet last I heard.
For generic off-the-shelf i210/i225 cards, I have yet to come across one which has documented hardware pins for PTP purposes, but the sample size is rather small (four or five different cards in total...).
Judging from a "grep -r TX_HARDWARE ./*" in Linux kernel tree there seem to be a few drivers which support hardware timestamping of some sort, so my first guess would be that those coming up there could potentially have pins which can be used for timestamping purposes. There might be more as not everyone is publicly documenting their chips in a way that's actually useful, but at least it's a start.
HTH,
Florian
On 30.01.23 19:15, John Miller via time-nuts wrote:
Hey All,
I'm curious as to what the collective experience is amongst this group when it comes to feeding a timestamp signal into a NIC, either for PTP purposes or for as a normal PPS refclock. The Intel i210 PCIe network card has a few software defined pins that can be used for this purpose, and the chrony documentation has some information about it:
https://chrony.tuxfamily.org/examples.html#_server_using_reference_clock_on_nic
I imagine most of us here use traditional RS232 serial ports to get this signal into an x86 computer, usually using the DCD pin. I've found some implementations of RS232 on PCs don't implement all of the additional signaling pins, so sometimes DCD is flat out missing. In other cases - especially with smaller embedded-type boards only UARTs with Rx/Tx are available to the user. (I'm fiddling with using GPIO pins on a few such systems, but it's quite a struggle.) So, there is a need for getting PPS signals into systems.
There are also highly specialized network interface cards, such as this one: https://www.ebay.com/itm/404117321209
... which does explicitly support PTP timestamping, but it's not clear to me if it could pull in a "standard" PPS signal to be made available to chrony or NTPsec. I haven't found much documentation on the NT20E2 yet, unfortunately, and it may simply not exist for hobbyists.
Does anyone here know if any other network cards that support software defined pins, like the i210 does, or any other methods for getting a PPS signal to be fed into a PC, and of course interpreted correctly?
Thanks!
John
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
Poul-Henning Kamp via time-nuts writes:
We used Intel '599 chip based cards in the ELT adaptive optics prototype
compute cluster, and it has both hardware timestamping of packets and
a couple of external signals.
I should probably mention a couple of things we learned the hard
way, even though I dont know if this applies to just the '599 chip:
-
The timestamping facility is very obviously a "bolt-on" and as such
the documentation was severely lacking. -
The timestamping counter runs at the RX-clock rate.
That means:
a) You want something 10GE plugged into the port for max resolution.
b) If you unplug it or it resets or reboots, your timescale jumps.
c) You timescale is the Xtal at the /far/ end of the cable.
This can probably all be mitigated with a loop-back, but then you cannot
timestamp packets.
- To timestamp packets, they have to look "sufficiently" like certain PTP
packets. This was not documented then, could be now.
Poul-Henning
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
On 1/30/23 12:28 PM, S McGrath via time-nuts wrote:
Most common way of getting 1PPS into a computer without a dedicated 1PPS
interface is to drive a RS-232 port’s Carrier Detect pin (pin 1 on DB-9)
with the 1PPS signal
back in the day when that tied directly to an interrupt with consistent
latency, that works pretty well.
These days, though, there's often USB hosts in the way, or some other
intermediate interfaces that increases the uncertainty in timing of
"when software does something" in response to "external event"
Driving Pin1 only works with a ‘real’ serial port on a dedicated IRQ. a
USB interface will add unpredictable latency
On Mon, Jan 30, 2023 at 6:16 PM Lux, Jim via time-nuts <
time-nuts@lists.febo.com> wrote:
On 1/30/23 12:28 PM, S McGrath via time-nuts wrote:
Most common way of getting 1PPS into a computer without a dedicated 1PPS
interface is to drive a RS-232 port’s Carrier Detect pin (pin 1 on DB-9)
with the 1PPS signal
back in the day when that tied directly to an interrupt with consistent
latency, that works pretty well.
These days, though, there's often USB hosts in the way, or some other
intermediate interfaces that increases the uncertainty in timing of
"when software does something" in response to "external event"
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
Even if you have a real serial port, still common in embedded SoCs that run
Linux if not on PCs, and even if it still has a CD line to generate an
interrupt, there are still issues.
Modern CPUs are complex, and while the serial port might even be on the
same chip as the CPU, they are connected through an internal network like
an AXI bus that adds latency and jitter. A PCI-E network card generating
an interrupt still needs to send a MSI packet over the PCI-E network bus to
the PCI-E controller, which then needs to send it on the CPU through some
other kind of internal network.
But that's not even the bad part. Once the CPU gets an interrupt, there is
still a lot of stuff that needs to happen in software before a timestamp is
generated. The CPU must context switch, new cachline loads from RAM for
data and code, the base kernel interrupt dispatching, etc.
With tests I did a while back with GPS PPS on an embedded Linux board, this
was the largest source of error.
Despite hardware timestampers being common on microcontrollers, SoCs that
can run Linux never seem to have them.
I wonder if one could make a useful USB timestamper with a common USB
microcontroller board. These have hardware counters that can timestamp a
PPS. It would be easy to send these timestamps over USB. Not sending
events about a PPS, but sending the actual timestamp of the PPS. USB has,
comparatively, terrible timing jitter, so timestamping the arrival of a USB
packet is of course no good.
With a bit of thought, we might think this is a step back. We have GPS
with an accurate clock that sends NMEA timestamps to us over an inaccurate
UART. Now we have a microcontroller with a semi-accurate clock sending
timestamps to us over an even more inaccurate USB interface. Same problem,
but worse.
But there is a difference! Typically the clock in an USB connected
microcontroller can use USB clock recovery. It's disciplined to the 1kHz
USB SOF signal from the host computer. This lets them meet the USB timing
spec with a cheap internal resonator.
So the clock in the microcontroller that gives us timestamps is in fact
tracking our clock. Does that make a difference? We can infer that the
error in the timestamps we get reflects a matching error in the host clock
we are trying to discipline with the PPS.
On Mon, Jan 30, 2023 at 4:01 PM S McGrath via time-nuts <
time-nuts@lists.febo.com> wrote:
Driving Pin1 only works with a ‘real’ serial port on a dedicated IRQ. a
USB interface will add unpredictable latency
On Mon, Jan 30, 2023 at 6:16 PM Lux, Jim via time-nuts <
time-nuts@lists.febo.com> wrote:
On 1/30/23 12:28 PM, S McGrath via time-nuts wrote:
Most common way of getting 1PPS into a computer without a dedicated
1PPS
interface is to drive a RS-232 port’s Carrier Detect pin (pin 1 on
DB-9)
with the 1PPS signal
back in the day when that tied directly to an interrupt with consistent
latency, that works pretty well.
These days, though, there's often USB hosts in the way, or some other
intermediate interfaces that increases the uncertainty in timing of
"when software does something" in response to "external event"
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
All your points are good ones, My point was this is the common lowest
cost approach, better approaches exist and they range from tens of dollars
to thousands depending on the accuracy needed.
Driving the CD line with a ‘real’ serial port is good enough for ‘standard’
NTP timing applications, its not good enough for a ‘time nuts’
implementation
Scott
On Mon, Jan 30, 2023 at 8:15 PM Trent Piepho tpiepho@gmail.com wrote:
Even if you have a real serial port, still common in embedded SoCs that
run Linux if not on PCs, and even if it still has a CD line to generate an
interrupt, there are still issues.
Modern CPUs are complex, and while the serial port might even be on the
same chip as the CPU, they are connected through an internal network like
an AXI bus that adds latency and jitter. A PCI-E network card generating
an interrupt still needs to send a MSI packet over the PCI-E network bus to
the PCI-E controller, which then needs to send it on the CPU through some
other kind of internal network.
But that's not even the bad part. Once the CPU gets an interrupt, there
is still a lot of stuff that needs to happen in software before a timestamp
is generated. The CPU must context switch, new cachline loads from RAM for
data and code, the base kernel interrupt dispatching, etc.
With tests I did a while back with GPS PPS on an embedded Linux board,
this was the largest source of error.
Despite hardware timestampers being common on microcontrollers, SoCs that
can run Linux never seem to have them.
I wonder if one could make a useful USB timestamper with a common USB
microcontroller board. These have hardware counters that can timestamp a
PPS. It would be easy to send these timestamps over USB. Not sending
events about a PPS, but sending the actual timestamp of the PPS. USB has,
comparatively, terrible timing jitter, so timestamping the arrival of a USB
packet is of course no good.
With a bit of thought, we might think this is a step back. We have GPS
with an accurate clock that sends NMEA timestamps to us over an inaccurate
UART. Now we have a microcontroller with a semi-accurate clock sending
timestamps to us over an even more inaccurate USB interface. Same problem,
but worse.
But there is a difference! Typically the clock in an USB connected
microcontroller can use USB clock recovery. It's disciplined to the 1kHz
USB SOF signal from the host computer. This lets them meet the USB timing
spec with a cheap internal resonator.
So the clock in the microcontroller that gives us timestamps is in fact
tracking our clock. Does that make a difference? We can infer that the
error in the timestamps we get reflects a matching error in the host clock
we are trying to discipline with the PPS.
On Mon, Jan 30, 2023 at 4:01 PM S McGrath via time-nuts <
time-nuts@lists.febo.com> wrote:
Driving Pin1 only works with a ‘real’ serial port on a dedicated IRQ. a
USB interface will add unpredictable latency
On Mon, Jan 30, 2023 at 6:16 PM Lux, Jim via time-nuts <
time-nuts@lists.febo.com> wrote:
On 1/30/23 12:28 PM, S McGrath via time-nuts wrote:
Most common way of getting 1PPS into a computer without a dedicated
1PPS
interface is to drive a RS-232 port’s Carrier Detect pin (pin 1 on
DB-9)
with the 1PPS signal
back in the day when that tied directly to an interrupt with consistent
latency, that works pretty well.
These days, though, there's often USB hosts in the way, or some other
intermediate interfaces that increases the uncertainty in timing of
"when software does something" in response to "external event"
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
Hi
Yet again, back to: what is the goal?
For NTP, you have a range of numbers To a great degree they are limited by the network
nonsense that may (or may not) be going on. A 12ps NTP logging process likely does not
do much for a network that has 12 ms of nonsense going on.
PTP may / will / might / should fix some of that stuff. Unfortunately there is a “maybe” that
qualifies all of that.
Welcome to the real world, all sorts of bad stuff happens.
Bob
On Jan 30, 2023, at 6:18 PM, S McGrath via time-nuts time-nuts@lists.febo.com wrote:
Driving Pin1 only works with a ‘real’ serial port on a dedicated IRQ. a
USB interface will add unpredictable latency
On Mon, Jan 30, 2023 at 6:16 PM Lux, Jim via time-nuts <
time-nuts@lists.febo.com> wrote:
On 1/30/23 12:28 PM, S McGrath via time-nuts wrote:
Most common way of getting 1PPS into a computer without a dedicated 1PPS
interface is to drive a RS-232 port’s Carrier Detect pin (pin 1 on DB-9)
with the 1PPS signal
back in the day when that tied directly to an interrupt with consistent
latency, that works pretty well.
These days, though, there's often USB hosts in the way, or some other
intermediate interfaces that increases the uncertainty in timing of
"when software does something" in response to "external event"
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
On 1/30/23 5:31 PM, S McGrath via time-nuts wrote:
All your points are good ones, My point was this is the common lowest
cost approach, better approaches exist and they range from tens of dollars
to thousands depending on the accuracy needed.
Driving the CD line with a ‘real’ serial port is good enough for ‘standard’
NTP timing applications, its not good enough for a ‘time nuts’
implementation
I daresay that finding a motherboard with a "real" serial port is pretty
challenging. And even if you plugged in a serial expansion card of some
sort, modern mobos probably aren't going to have a clean deterministic
IRQ behavior.
The idea of a (potentially inexpensive) standalone box that can do it is
intriguing. But we started with networking - So what is being discussed
is something like a USB Ethernet dongle with time stamping.
John-
I brought this up once before. White Rabbit research has shown to do sync
well you need BIDI fiber SFPs. Two wavelengths going opposite directions
down the same single mode fiber. See:
https://ohwr.org/project/white-rabbit/wikis/SFP
The new future of nics that do this type of timing sync are DPU's.
https://www.nvidia.com/content/dam/en-zz/Solutions/gtcf21/networking/data-processing-unit/gtc-fall-21-networking-overall-dpu-technical-overview-firefly.pdf
I'm really interested in precision PTP over Ethernet over fiber.
Kinda related... Meta is looking at timebeat and I recently posted about
the new cards they're selling with MEMS DCOCXO SiT5721 for holdover.
Seems the future of Ethernet timing sync time stamping is developing well
right now. Fiber is the way to go though.
Bill
On Mon, Jan 30, 2023, 9:18 PM Lux, Jim via time-nuts <
time-nuts@lists.febo.com> wrote:
On 1/30/23 5:31 PM, S McGrath via time-nuts wrote:
All your points are good ones, My point was this is the common lowest
cost approach, better approaches exist and they range from tens of
dollars
to thousands depending on the accuracy needed.
Driving the CD line with a ‘real’ serial port is good enough for
‘standard’
NTP timing applications, its not good enough for a ‘time nuts’
implementation
I daresay that finding a motherboard with a "real" serial port is pretty
challenging. And even if you plugged in a serial expansion card of some
sort, modern mobos probably aren't going to have a clean deterministic
IRQ behavior.
The idea of a (potentially inexpensive) standalone box that can do it is
intriguing. But we started with networking - So what is being discussed
is something like a USB Ethernet dongle with time stamping.
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
It’s too bad the SiT5721 is over $135 in 25 quantity orders. That prices it above a used but not used up rubidium.
Get Outlook for iOShttps://aka.ms/o0ukef
From: Bill Notfaded via time-nuts time-nuts@lists.febo.com
Sent: Tuesday, January 31, 2023 5:21:04 AM
To: Discussion of precise time and frequency measurement time-nuts@lists.febo.com
Cc: Bill Notfaded notfaded1@gmail.com
Subject: [time-nuts] Re: Network interface cards that support timestamping
John-
I brought this up once before. White Rabbit research has shown to do sync
well you need BIDI fiber SFPs. Two wavelengths going opposite directions
down the same single mode fiber. See:
https://ohwr.org/project/white-rabbit/wikis/SFP
The new future of nics that do this type of timing sync are DPU's.
https://www.nvidia.com/content/dam/en-zz/Solutions/gtcf21/networking/data-processing-unit/gtc-fall-21-networking-overall-dpu-technical-overview-firefly.pdf
I'm really interested in precision PTP over Ethernet over fiber.
Kinda related... Meta is looking at timebeat and I recently posted about
the new cards they're selling with MEMS DCOCXO SiT5721 for holdover.
Seems the future of Ethernet timing sync time stamping is developing well
right now. Fiber is the way to go though.
Bill
On Mon, Jan 30, 2023, 9:18 PM Lux, Jim via time-nuts <
time-nuts@lists.febo.com> wrote:
On 1/30/23 5:31 PM, S McGrath via time-nuts wrote:
All your points are good ones, My point was this is the common lowest
cost approach, better approaches exist and they range from tens of
dollars
to thousands depending on the accuracy needed.
Driving the CD line with a ‘real’ serial port is good enough for
‘standard’
NTP timing applications, its not good enough for a ‘time nuts’
implementation
I daresay that finding a motherboard with a "real" serial port is pretty
challenging. And even if you plugged in a serial expansion card of some
sort, modern mobos probably aren't going to have a clean deterministic
IRQ behavior.
The idea of a (potentially inexpensive) standalone box that can do it is
intriguing. But we started with networking - So what is being discussed
is something like a USB Ethernet dongle with time stamping.
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
Despite hardware timestampers being common on microcontrollers, SoCs that
can run Linux never seem to have them.
BeagleBone Black does run Linux and has timer capture hardware, though it's
a bit 'slow' it's still available. Dan Drown (originally) also wrote a PPS
module targeting it, though it's been unmaintained (
https://github.com/thinkfat/pps-gmtimer was the closest fork to working for
me, but it still required some fiddling around with DeviceTree etc. ). It
tries to pull the somewhat clever trick of switching the system's clock
source to the capture timer, and then copying the capture value directly
into the PPS report, which in theory should be about as good a capture as
you could get - no polling the kernel timestamp or the input pin. I've been
running this for a while with a GPSDO as PPS source, and while the average
offset reported by chrony is single-digit ns, there are occasional spikes
to 100s of ns.
The NIC also does have hardware timestamping functionality for PTP.
On Mon, 30 Jan 2023 at 18:26, Trent Piepho via time-nuts <
time-nuts@lists.febo.com> wrote:
Even if you have a real serial port, still common in embedded SoCs that run
Linux if not on PCs, and even if it still has a CD line to generate an
interrupt, there are still issues.
Modern CPUs are complex, and while the serial port might even be on the
same chip as the CPU, they are connected through an internal network like
an AXI bus that adds latency and jitter. A PCI-E network card generating
an interrupt still needs to send a MSI packet over the PCI-E network bus to
the PCI-E controller, which then needs to send it on the CPU through some
other kind of internal network.
But that's not even the bad part. Once the CPU gets an interrupt, there is
still a lot of stuff that needs to happen in software before a timestamp is
generated. The CPU must context switch, new cachline loads from RAM for
data and code, the base kernel interrupt dispatching, etc.
With tests I did a while back with GPS PPS on an embedded Linux board, this
was the largest source of error.
Despite hardware timestampers being common on microcontrollers, SoCs that
can run Linux never seem to have them.
I wonder if one could make a useful USB timestamper with a common USB
microcontroller board. These have hardware counters that can timestamp a
PPS. It would be easy to send these timestamps over USB. Not sending
events about a PPS, but sending the actual timestamp of the PPS. USB has,
comparatively, terrible timing jitter, so timestamping the arrival of a USB
packet is of course no good.
With a bit of thought, we might think this is a step back. We have GPS
with an accurate clock that sends NMEA timestamps to us over an inaccurate
UART. Now we have a microcontroller with a semi-accurate clock sending
timestamps to us over an even more inaccurate USB interface. Same problem,
but worse.
But there is a difference! Typically the clock in an USB connected
microcontroller can use USB clock recovery. It's disciplined to the 1kHz
USB SOF signal from the host computer. This lets them meet the USB timing
spec with a cheap internal resonator.
So the clock in the microcontroller that gives us timestamps is in fact
tracking our clock. Does that make a difference? We can infer that the
error in the timestamps we get reflects a matching error in the host clock
we are trying to discipline with the PPS.
On Mon, Jan 30, 2023 at 4:01 PM S McGrath via time-nuts <
time-nuts@lists.febo.com> wrote:
Driving Pin1 only works with a ‘real’ serial port on a dedicated IRQ. a
USB interface will add unpredictable latency
On Mon, Jan 30, 2023 at 6:16 PM Lux, Jim via time-nuts <
time-nuts@lists.febo.com> wrote:
On 1/30/23 12:28 PM, S McGrath via time-nuts wrote:
Most common way of getting 1PPS into a computer without a dedicated
1PPS
interface is to drive a RS-232 port’s Carrier Detect pin (pin 1 on
DB-9)
with the 1PPS signal
back in the day when that tied directly to an interrupt with consistent
latency, that works pretty well.
These days, though, there's often USB hosts in the way, or some other
intermediate interfaces that increases the uncertainty in timing of
"when software does something" in response to "external event"
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
I get the impression from a cursory search that the card might take either a PPS or a 10 MHz, and that to get it to function as a PTP server that it probably needs a specific FPGA image and/or driver
Get Outlook for iOShttps://aka.ms/o0ukef
From: John Miller via time-nuts time-nuts@lists.febo.com
Sent: Monday, January 30, 2023 12:15:27 PM
To: Discussion of precise time and frequency measurement time-nuts@lists.febo.com
Cc: John Miller john@millerjs.org
Subject: [time-nuts] Network interface cards that support timestamping
Hey All,
I'm curious as to what the collective experience is amongst this group when it comes to feeding a timestamp signal into a NIC, either for PTP purposes or for as a normal PPS refclock. The Intel i210 PCIe network card has a few software defined pins that can be used for this purpose, and the chrony documentation has some information about it:
https://chrony.tuxfamily.org/examples.html#_server_using_reference_clock_on_nic
I imagine most of us here use traditional RS232 serial ports to get this signal into an x86 computer, usually using the DCD pin. I've found some implementations of RS232 on PCs don't implement all of the additional signaling pins, so sometimes DCD is flat out missing. In other cases - especially with smaller embedded-type boards only UARTs with Rx/Tx are available to the user. (I'm fiddling with using GPIO pins on a few such systems, but it's quite a struggle.) So, there is a need for getting PPS signals into systems.
There are also highly specialized network interface cards, such as this one: https://www.ebay.com/itm/404117321209
... which does explicitly support PTP timestamping, but it's not clear to me if it could pull in a "standard" PPS signal to be made available to chrony or NTPsec. I haven't found much documentation on the NT20E2 yet, unfortunately, and it may simply not exist for hobbyists.
Does anyone here know if any other network cards that support software defined pins, like the i210 does, or any other methods for getting a PPS signal to be fed into a PC, and of course interpreted correctly?
Thanks!
John
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
On 1/30/23 10:15 AM, John Miller via time-nuts wrote:
Hey All,
I'm curious as to what the collective experience is amongst this group when it comes to feeding a timestamp signal into a NIC, either for PTP purposes or for as a normal PPS refclock. The Intel i210 PCIe network card has a few software defined pins that can be used for this purpose, and the chrony documentation has some information about it:
https://chrony.tuxfamily.org/examples.html#_server_using_reference_clock_on_nic
I imagine most of us here use traditional RS232 serial ports to get this signal into an x86 computer, usually using the DCD pin. I've found some implementations of RS232 on PCs don't implement all of the additional signaling pins, so sometimes DCD is flat out missing. In other cases - especially with smaller embedded-type boards only UARTs with Rx/Tx are available to the user. (I'm fiddling with using GPIO pins on a few such systems, but it's quite a struggle.) So, there is a need for getting PPS signals into systems.
There are also highly specialized network interface cards, such as this one: https://www.ebay.com/itm/404117321209
... which does explicitly support PTP timestamping, but it's not clear to me if it could pull in a "standard" PPS signal to be made available to chrony or NTPsec. I haven't found much documentation on the NT20E2 yet, unfortunately, and it may simply not exist for hobbyists.
Does anyone here know if any other network cards that support software defined pins, like the i210 does, or any other methods for getting a PPS signal to be fed into a PC, and of course interpreted correctly?
A friend pointed me at this thread, I just finished hooking up 2 i211s to two separate u-blox
receivers to rather good effect. Specifically, I have an APU2 and hooked up the receivers as
described at [1]. Note that it doesn't actually matter which of the four pins you hook it up to, any
can be configured to interrupt (though you need the patch at [2] for recent kernels).
The serials on the same board come in via a Nuvoton NCT5104D LPC <-> 4xCOM port chip (which I assume
is probably not uncommon for many machines these days). The same GPS receivers were hooked up via
that via DCD pins, which were properly interrupt'ing but had jitter on the order of 10-25us. Chrony
would often report a bit less std dev across a few samples, but the two receivers would wander
around the current time and often by off in different directions. With the receivers moved over to
the i211s chrony now reports std dev between 5 and 100ns, though given they're always off in the
same direction on a similar order of magnitude I think that's the system clock being off more than
interrupt latency.
You can find graphs at [3], though excuse the lack of history, I re-soldered a few hours ago and am
still testing it so dropping data since last boot.
While the i211 theoretically supports both interrupting for the Software-Defined-Pins and packet
timestamping for PTP (and NTP) purposes, it is entirely unclear to me if chrony uses the NIC clock
to respond to NTP. chrony happily accepts the hwtimestamping option for the NICs and correctly reads
from the NIC's PPS pin as a PHC clock, but there doesn't seem to be any config to hook the two up
somehow.
The testptp utility in the kernel source tree tells me that it takes 8000-9000ns to read the NIC's
clock, and chrony pretty reliably reports the +/- on a sample from the NIC PPS as +/- 1100-1200ns so
it lines up that there's around 1000ns of jitter on the interrupt.
Matt
[1] https://blog.dan.drown.org/apu2-ntp-server-2/
[2] https://lists.osuosl.org/pipermail/intel-wired-lan/Week-of-Mon-20230130/031970.html
[3] https://noc.as397444.net/ntpgraphs.day/
Hi Matt,
This is excellent information, thank you - after reading through all the responses in the thread (wow, this is a wild topic!) and trawling around online a bit, I think that using the SDP pins on the i210 to feed a PPS signal into an x86 PC is what I want to do right now.
I have read Dan Drown's blog post on using the APU2 to achieve this, and while it's more or less exactly what I want to replicate, for now, there are a few gaps I need to fill in - and you may be able to help me do so. I don't have a PC Engines APU2 board yet, but I suspect I'll order one once I have the software side of things ironed out, because I really like their form factor.
What I'm using right now is a Portwell NANO-6050[1], and it has i210 and i218 NICs. The SDP pins on the i210 aren't broken out into nice pads like the APU2 has, but with some magnet wire and a steady hand I was able to connect directly to one of the pins. Where I am struggling right now is ironing out the software configuration side of things. I'm not interested at all in PTP yet, right now the focus is purely on feeding the PPS into chrony via the SDP pins. The GNSS I'm using is a Sparkfun uBlox NEO-M9N GNSS receiver in its default configuration, connected to the computer over USB. PPS pin connected to the i210's SDP0. PPS pulse is 3v3 for 100ms.
I have tried using refclock strings from chrony's examples page[2] as a jumping-off point, but it doesn't make much sense to me. 16Hz PPS with a rate of 16 - why not use 1Hz with a default rate of 1? The Dan Drown blog post makes a little bit more sense, but I think something is missing from the 'relevant chrony.conf' example - notably, it includes a single refclock line with the "pps" option, which according to the chrony.conf docs[4]: "Another time source is needed to complete samples from the refclock." and that example isn't included. Furthermore, his "width" parameter is set to 0.7 milliseconds, which seems tremendously short to me. If the PPS pulses are 100ms (which I understand to be fairly standard) why wouldn't the width be configured as such? I may be misunderstanding this parameter.
Either way, am not able to get chrony to lock up to a PHC refclock.[5] This is what my chrony.conf looks like: https://paste.millerjs.org/ajoxigiquc.txt
I have not been able to use the testptp tool on this system yet - building it on another machine and running the copied-over executable fails, citing too old a version of glibc, and when I try and build it locally it fails for reasons that are unknown to me (I am very much not a C developer). I may have to install a newer OS in order to try this out.
I am going to continue to fiddle with this - I think getting testptp working such that I can verify that I'm actually getting a pulse on SDP0 is the most important thing to confirm right now. I would greatly appreciate any help or feedback!
Thanks,
John
KC1QLN
[1] https://portwell.com/products/detail.php?CUSTCHAR1=NANO-6050
[2] https://chrony.tuxfamily.org/examples.html#_server_using_reference_clock_on_nic
[3] https://blog.dan.drown.org/apu2-ntp-server-2/
[4] https://chrony.tuxfamily.org/doc/4.3/chrony.conf.html
[5] https://paste.millerjs.org/edebutayoz.txt
On Jan 31, 2023, at 9:43 PM, Matt Corallo timenuts-list@mattcorallo.com wrote:
On 1/30/23 10:15 AM, John Miller via time-nuts wrote:
Hey All,
I'm curious as to what the collective experience is amongst this group when it comes to feeding a timestamp signal into a NIC, either for PTP purposes or for as a normal PPS refclock. The Intel i210 PCIe network card has a few software defined pins that can be used for this purpose, and the chrony documentation has some information about it:
https://chrony.tuxfamily.org/examples.html#_server_using_reference_clock_on_nic
I imagine most of us here use traditional RS232 serial ports to get this signal into an x86 computer, usually using the DCD pin. I've found some implementations of RS232 on PCs don't implement all of the additional signaling pins, so sometimes DCD is flat out missing. In other cases - especially with smaller embedded-type boards only UARTs with Rx/Tx are available to the user. (I'm fiddling with using GPIO pins on a few such systems, but it's quite a struggle.) So, there is a need for getting PPS signals into systems.
There are also highly specialized network interface cards, such as this one: https://www.ebay.com/itm/404117321209
... which does explicitly support PTP timestamping, but it's not clear to me if it could pull in a "standard" PPS signal to be made available to chrony or NTPsec. I haven't found much documentation on the NT20E2 yet, unfortunately, and it may simply not exist for hobbyists.
Does anyone here know if any other network cards that support software defined pins, like the i210 does, or any other methods for getting a PPS signal to be fed into a PC, and of course interpreted correctly?
A friend pointed me at this thread, I just finished hooking up 2 i211s to two separate u-blox receivers to rather good effect. Specifically, I have an APU2 and hooked up the receivers as described at [1]. Note that it doesn't actually matter which of the four pins you hook it up to, any can be configured to interrupt (though you need the patch at [2] for recent kernels).
The serials on the same board come in via a Nuvoton NCT5104D LPC <-> 4xCOM port chip (which I assume is probably not uncommon for many machines these days). The same GPS receivers were hooked up via that via DCD pins, which were properly interrupt'ing but had jitter on the order of 10-25us. Chrony would often report a bit less std dev across a few samples, but the two receivers would wander around the current time and often by off in different directions. With the receivers moved over to the i211s chrony now reports std dev between 5 and 100ns, though given they're always off in the same direction on a similar order of magnitude I think that's the system clock being off more than interrupt latency.
You can find graphs at [3], though excuse the lack of history, I re-soldered a few hours ago and am still testing it so dropping data since last boot.
While the i211 theoretically supports both interrupting for the Software-Defined-Pins and packet timestamping for PTP (and NTP) purposes, it is entirely unclear to me if chrony uses the NIC clock to respond to NTP. chrony happily accepts the hwtimestamping option for the NICs and correctly reads from the NIC's PPS pin as a PHC clock, but there doesn't seem to be any config to hook the two up somehow.
The testptp utility in the kernel source tree tells me that it takes 8000-9000ns to read the NIC's clock, and chrony pretty reliably reports the +/- on a sample from the NIC PPS as +/- 1100-1200ns so it lines up that there's around 1000ns of jitter on the interrupt.
Matt
[1] https://blog.dan.drown.org/apu2-ntp-server-2/
[2] https://lists.osuosl.org/pipermail/intel-wired-lan/Week-of-Mon-20230130/031970.html
[3] https://noc.as397444.net/ntpgraphs.day/
Hi
If accuracy off of a GPS module is the goal then one needs to start at the module.
All the low cost modules put out their PPS based on an internal free running clock.
It is off from “correct” by some number of NS each time. For historical reasons it
gets called sawtooth error ( = the error chart often looks like a sawtooth).
There are a number of modules that will tell you what the error is in one string
or another. It might have a range of +/- 20 ns. It could be down around +/-2 depending
on the make and model of the unit.
The error is calculated once a second and is provided relative to a pps output.
If you adjust the output rate, working out just what point the “new” error comes
in can be problematic. Most folks just go with 1 pps and move on.
With some care, one can indeed move these edges into a fairly typical PC. Drivers
for this or that are a mixed bag. I’d stick with NTPSec if you want drivers that work.
What typically happens is the pulses are used to come up with an estimate of the
computer’s internal clock. That clock is used to stamp this or that. It’s far more
efficient to do it this way. The data from the external clock is used often enough
to keep the estimate reasonably accurate. Put another way, your timestamp is
always removed from that clock edge by a bit. Faster external clock rates
don’t really do much with this approach.
PTP does this all a bit differently. The above applies to the sorts of programs you
are currently looking at.
Bob
On Feb 1, 2023, at 1:38 PM, John Miller via time-nuts time-nuts@lists.febo.com wrote:
Hi Matt,
This is excellent information, thank you - after reading through all the responses in the thread (wow, this is a wild topic!) and trawling around online a bit, I think that using the SDP pins on the i210 to feed a PPS signal into an x86 PC is what I want to do right now.
I have read Dan Drown's blog post on using the APU2 to achieve this, and while it's more or less exactly what I want to replicate, for now, there are a few gaps I need to fill in - and you may be able to help me do so. I don't have a PC Engines APU2 board yet, but I suspect I'll order one once I have the software side of things ironed out, because I really like their form factor.
What I'm using right now is a Portwell NANO-6050[1], and it has i210 and i218 NICs. The SDP pins on the i210 aren't broken out into nice pads like the APU2 has, but with some magnet wire and a steady hand I was able to connect directly to one of the pins. Where I am struggling right now is ironing out the software configuration side of things. I'm not interested at all in PTP yet, right now the focus is purely on feeding the PPS into chrony via the SDP pins. The GNSS I'm using is a Sparkfun uBlox NEO-M9N GNSS receiver in its default configuration, connected to the computer over USB. PPS pin connected to the i210's SDP0. PPS pulse is 3v3 for 100ms.
I have tried using refclock strings from chrony's examples page[2] as a jumping-off point, but it doesn't make much sense to me. 16Hz PPS with a rate of 16 - why not use 1Hz with a default rate of 1? The Dan Drown blog post makes a little bit more sense, but I think something is missing from the 'relevant chrony.conf' example - notably, it includes a single refclock line with the "pps" option, which according to the chrony.conf docs[4]: "Another time source is needed to complete samples from the refclock." and that example isn't included. Furthermore, his "width" parameter is set to 0.7 milliseconds, which seems tremendously short to me. If the PPS pulses are 100ms (which I understand to be fairly standard) why wouldn't the width be configured as such? I may be misunderstanding this parameter.
Either way, am not able to get chrony to lock up to a PHC refclock.[5] This is what my chrony.conf looks like: https://paste.millerjs.org/ajoxigiquc.txt
I have not been able to use the testptp tool on this system yet - building it on another machine and running the copied-over executable fails, citing too old a version of glibc, and when I try and build it locally it fails for reasons that are unknown to me (I am very much not a C developer). I may have to install a newer OS in order to try this out.
I am going to continue to fiddle with this - I think getting testptp working such that I can verify that I'm actually getting a pulse on SDP0 is the most important thing to confirm right now. I would greatly appreciate any help or feedback!
Thanks,
John
KC1QLN
[1] https://portwell.com/products/detail.php?CUSTCHAR1=NANO-6050
[2] https://chrony.tuxfamily.org/examples.html#_server_using_reference_clock_on_nic
[3] https://blog.dan.drown.org/apu2-ntp-server-2/
[4] https://chrony.tuxfamily.org/doc/4.3/chrony.conf.html
[5] https://paste.millerjs.org/edebutayoz.txt
On Jan 31, 2023, at 9:43 PM, Matt Corallo timenuts-list@mattcorallo.com wrote:
On 1/30/23 10:15 AM, John Miller via time-nuts wrote:
Hey All,
I'm curious as to what the collective experience is amongst this group when it comes to feeding a timestamp signal into a NIC, either for PTP purposes or for as a normal PPS refclock. The Intel i210 PCIe network card has a few software defined pins that can be used for this purpose, and the chrony documentation has some information about it:
https://chrony.tuxfamily.org/examples.html#_server_using_reference_clock_on_nic
I imagine most of us here use traditional RS232 serial ports to get this signal into an x86 computer, usually using the DCD pin. I've found some implementations of RS232 on PCs don't implement all of the additional signaling pins, so sometimes DCD is flat out missing. In other cases - especially with smaller embedded-type boards only UARTs with Rx/Tx are available to the user. (I'm fiddling with using GPIO pins on a few such systems, but it's quite a struggle.) So, there is a need for getting PPS signals into systems.
There are also highly specialized network interface cards, such as this one: https://www.ebay.com/itm/404117321209
... which does explicitly support PTP timestamping, but it's not clear to me if it could pull in a "standard" PPS signal to be made available to chrony or NTPsec. I haven't found much documentation on the NT20E2 yet, unfortunately, and it may simply not exist for hobbyists.
Does anyone here know if any other network cards that support software defined pins, like the i210 does, or any other methods for getting a PPS signal to be fed into a PC, and of course interpreted correctly?
A friend pointed me at this thread, I just finished hooking up 2 i211s to two separate u-blox receivers to rather good effect. Specifically, I have an APU2 and hooked up the receivers as described at [1]. Note that it doesn't actually matter which of the four pins you hook it up to, any can be configured to interrupt (though you need the patch at [2] for recent kernels).
The serials on the same board come in via a Nuvoton NCT5104D LPC <-> 4xCOM port chip (which I assume is probably not uncommon for many machines these days). The same GPS receivers were hooked up via that via DCD pins, which were properly interrupt'ing but had jitter on the order of 10-25us. Chrony would often report a bit less std dev across a few samples, but the two receivers would wander around the current time and often by off in different directions. With the receivers moved over to the i211s chrony now reports std dev between 5 and 100ns, though given they're always off in the same direction on a similar order of magnitude I think that's the system clock being off more than interrupt latency.
You can find graphs at [3], though excuse the lack of history, I re-soldered a few hours ago and am still testing it so dropping data since last boot.
While the i211 theoretically supports both interrupting for the Software-Defined-Pins and packet timestamping for PTP (and NTP) purposes, it is entirely unclear to me if chrony uses the NIC clock to respond to NTP. chrony happily accepts the hwtimestamping option for the NICs and correctly reads from the NIC's PPS pin as a PHC clock, but there doesn't seem to be any config to hook the two up somehow.
The testptp utility in the kernel source tree tells me that it takes 8000-9000ns to read the NIC's clock, and chrony pretty reliably reports the +/- on a sample from the NIC PPS as +/- 1100-1200ns so it lines up that there's around 1000ns of jitter on the interrupt.
Matt
[1] https://blog.dan.drown.org/apu2-ntp-server-2/
[2] https://lists.osuosl.org/pipermail/intel-wired-lan/Week-of-Mon-20230130/031970.html
[3] https://noc.as397444.net/ntpgraphs.day/
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
On 2/1/23 10:38 AM, John Miller via time-nuts wrote:
Hi Matt,
What I'm using right now is a Portwell NANO-6050[1], and it has i210 and i218 NICs. The SDP pins on the i210 aren't broken out into nice pads like the APU2 has, but with some magnet wire and a steady hand I was able to connect directly to one of the pins. Where I am struggling right now is ironing out the software configuration side of things. I'm not interested at all in PTP yet, right now the focus is purely on feeding the PPS into chrony via the SDP pins. The GNSS I'm using is a Sparkfun uBlox NEO-M9N GNSS receiver in its default configuration, connected to the computer over USB. PPS pin connected to the i210's SDP0. PPS pulse is 3v3 for 100ms.
I have tried using refclock strings from chrony's examples page[2] as a jumping-off point, but it doesn't make much sense to me.
16Hz PPS with a rate of 16 - why not use 1Hz with a default rate of 1?
More samples can hide at least some of the noise. As Bob points out in his reply there's a limit,
though, of course, the NEO-M9N datasheet says its time-pulse accuracy is 30ns RMS/60ns 99%. If your
interrupt jitter is less than that, you're not gonna hide much :).
The Dan Drown blog post makes a little bit more sense, but I think something is missing from the 'relevant chrony.conf' example - notably, it includes a single refclock line with the "pps" option, which according to the chrony.conf docs[4]: "Another time source is needed to complete samples from the refclock." and that example isn't included.
A PPS input is just a pulse. chrony has no idea what time that pulse was meant to indicate, only
that it indicated a specific time. You have to somehow hook that pulse up to something else via the
"lock" option. Probably you want to run gpsd and tell chrony about the gpsd input via a "refclock
SHM...noselect" line. There should be examples of how to do this elsewhere
Furthermore, his "width" parameter is set to 0.7 milliseconds, which seems tremendously short to me. If the PPS pulses are 100ms (which I understand to be fairly standard) why wouldn't the width be configured as such? I may be misunderstanding this parameter.
Yea, not sure, that does seem super small. If chrony knows the time (via the lock'ed refclock) with
more accuracy than 0.7/2ms then its fine, but if you're locking to an NMEA input via gpsd you're
gonna have way more variance than that. Ideally your pulse is half the width of your pulse rate, so
500ms if you're doing 1s PPS.
Either way, am not able to get chrony to lock up to a PHC refclock.[5] This is what my chrony.conf looks like: https://paste.millerjs.org/ajoxigiquc.txt
I have not been able to use the testptp tool on this system yet - building it on another machine and running the copied-over executable fails, citing too old a version of glibc, and when I try and build it locally it fails for reasons that are unknown to me (I am very much not a C developer). I may have to install a newer OS in order to try this out.
I am going to continue to fiddle with this - I think getting testptp working such that I can verify that I'm actually getting a pulse on SDP0 is the most important thing to confirm right now. I would greatly appreciate any help or feedback!
chrony has been a bit...fidgety with me, the testptp utility will show at least whether your input
is pulsing and being read or not.
Matt
Hi
A 10 or 100 microsecond wide 1 pps signal is more common than a 10 to 100 millisecond
wide pulse.
Why?
Back in the day, they coupled this stuff via a transformer. The frequency response issues
made a narrower pulse more appealing. These days it is not uncommon to try to drive a
50 ohm load to some sort of logic level. The less time you have to do that, the happier
your driver IC (and regulator) are likely to be.
One example of this would be the Trimble TBolt. It has a 10 us wide output pulse. Another
would be a uBlox LEA-6T (and the others in that lineup) with a default 10us wide output.
Bob
On Feb 1, 2023, at 5:02 PM, Matt Corallo via time-nuts time-nuts@lists.febo.com wrote:
On 2/1/23 10:38 AM, John Miller via time-nuts wrote:
Hi Matt,
What I'm using right now is a Portwell NANO-6050[1], and it has i210 and i218 NICs. The SDP pins on the i210 aren't broken out into nice pads like the APU2 has, but with some magnet wire and a steady hand I was able to connect directly to one of the pins. Where I am struggling right now is ironing out the software configuration side of things. I'm not interested at all in PTP yet, right now the focus is purely on feeding the PPS into chrony via the SDP pins. The GNSS I'm using is a Sparkfun uBlox NEO-M9N GNSS receiver in its default configuration, connected to the computer over USB. PPS pin connected to the i210's SDP0. PPS pulse is 3v3 for 100ms.
I have tried using refclock strings from chrony's examples page[2] as a jumping-off point, but it doesn't make much sense to me.
16Hz PPS with a rate of 16 - why not use 1Hz with a default rate of 1?
More samples can hide at least some of the noise. As Bob points out in his reply there's a limit, though, of course, the NEO-M9N datasheet says its time-pulse accuracy is 30ns RMS/60ns 99%. If your interrupt jitter is less than that, you're not gonna hide much :).
The Dan Drown blog post makes a little bit more sense, but I think something is missing from the 'relevant chrony.conf' example - notably, it includes a single refclock line with the "pps" option, which according to the chrony.conf docs[4]: "Another time source is needed to complete samples from the refclock." and that example isn't included.
A PPS input is just a pulse. chrony has no idea what time that pulse was meant to indicate, only that it indicated a specific time. You have to somehow hook that pulse up to something else via the "lock" option. Probably you want to run gpsd and tell chrony about the gpsd input via a "refclock SHM...noselect" line. There should be examples of how to do this elsewhere
Furthermore, his "width" parameter is set to 0.7 milliseconds, which seems tremendously short to me. If the PPS pulses are 100ms (which I understand to be fairly standard) why wouldn't the width be configured as such? I may be misunderstanding this parameter.
Yea, not sure, that does seem super small. If chrony knows the time (via the lock'ed refclock) with more accuracy than 0.7/2ms then its fine, but if you're locking to an NMEA input via gpsd you're gonna have way more variance than that. Ideally your pulse is half the width of your pulse rate, so 500ms if you're doing 1s PPS.
Either way, am not able to get chrony to lock up to a PHC refclock.[5] This is what my chrony.conf looks like: https://paste.millerjs.org/ajoxigiquc.txt
I have not been able to use the testptp tool on this system yet - building it on another machine and running the copied-over executable fails, citing too old a version of glibc, and when I try and build it locally it fails for reasons that are unknown to me (I am very much not a C developer). I may have to install a newer OS in order to try this out.
I am going to continue to fiddle with this - I think getting testptp working such that I can verify that I'm actually getting a pulse on SDP0 is the most important thing to confirm right now. I would greatly appreciate any help or feedback!
chrony has been a bit...fidgety with me, the testptp utility will show at least whether your input is pulsing and being read or not.
Matt
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
Makes sense.
Sadly, as far as I can tell, the i211 will mark both the rising and falling edge of the pulse, with
no distinction between the two. While you'd expect chrony to be able to differentiate based on the
time between them, the manual for chrony.conf seems to indicate otherwise, saying:
width width
This option specifies the width of the pulses (in seconds). It is used to filter PPS samples when
the driver provides samples for both rising and falling edges. Note that it reduces the maximum
allowed error of the time source which completes the PPS samples. If the duty cycle is
configurable, 50% should be preferred in order to maximise the allowed error.
which, maybe incorrectly, implied to me that there was no smarts for highly asymmetric
pulse-to-not-pulsing time.
Matt
On 2/1/23 4:15 PM, Bob Camp wrote:
Hi
A 10 or 100 microsecond wide 1 pps signal is more common than a 10 to 100 millisecond
wide pulse.
Why?
Back in the day, they coupled this stuff via a transformer. The frequency response issues
made a narrower pulse more appealing. These days it is not uncommon to try to drive a
50 ohm load to some sort of logic level. The less time you have to do that, the happier
your driver IC (and regulator) are likely to be.
One example of this would be the Trimble TBolt. It has a 10 us wide output pulse. Another
would be a uBlox LEA-6T (and the others in that lineup) with a default 10us wide output.
Bob
On Feb 1, 2023, at 5:02 PM, Matt Corallo via time-nuts time-nuts@lists.febo.com wrote:
On 2/1/23 10:38 AM, John Miller via time-nuts wrote:
Hi Matt,
What I'm using right now is a Portwell NANO-6050[1], and it has i210 and i218 NICs. The SDP pins on the i210 aren't broken out into nice pads like the APU2 has, but with some magnet wire and a steady hand I was able to connect directly to one of the pins. Where I am struggling right now is ironing out the software configuration side of things. I'm not interested at all in PTP yet, right now the focus is purely on feeding the PPS into chrony via the SDP pins. The GNSS I'm using is a Sparkfun uBlox NEO-M9N GNSS receiver in its default configuration, connected to the computer over USB. PPS pin connected to the i210's SDP0. PPS pulse is 3v3 for 100ms.
I have tried using refclock strings from chrony's examples page[2] as a jumping-off point, but it doesn't make much sense to me.16Hz PPS with a rate of 16 - why not use 1Hz with a default rate of 1?
More samples can hide at least some of the noise. As Bob points out in his reply there's a limit, though, of course, the NEO-M9N datasheet says its time-pulse accuracy is 30ns RMS/60ns 99%. If your interrupt jitter is less than that, you're not gonna hide much :).
The Dan Drown blog post makes a little bit more sense, but I think something is missing from the 'relevant chrony.conf' example - notably, it includes a single refclock line with the "pps" option, which according to the chrony.conf docs[4]: "Another time source is needed to complete samples from the refclock." and that example isn't included.
A PPS input is just a pulse. chrony has no idea what time that pulse was meant to indicate, only that it indicated a specific time. You have to somehow hook that pulse up to something else via the "lock" option. Probably you want to run gpsd and tell chrony about the gpsd input via a "refclock SHM...noselect" line. There should be examples of how to do this elsewhere
Furthermore, his "width" parameter is set to 0.7 milliseconds, which seems tremendously short to me. If the PPS pulses are 100ms (which I understand to be fairly standard) why wouldn't the width be configured as such? I may be misunderstanding this parameter.
Yea, not sure, that does seem super small. If chrony knows the time (via the lock'ed refclock) with more accuracy than 0.7/2ms then its fine, but if you're locking to an NMEA input via gpsd you're gonna have way more variance than that. Ideally your pulse is half the width of your pulse rate, so 500ms if you're doing 1s PPS.
Either way, am not able to get chrony to lock up to a PHC refclock.[5] This is what my chrony.conf looks like: https://paste.millerjs.org/ajoxigiquc.txt
I have not been able to use the testptp tool on this system yet - building it on another machine and running the copied-over executable fails, citing too old a version of glibc, and when I try and build it locally it fails for reasons that are unknown to me (I am very much not a C developer). I may have to install a newer OS in order to try this out.
I am going to continue to fiddle with this - I think getting testptp working such that I can verify that I'm actually getting a pulse on SDP0 is the most important thing to confirm right now. I would greatly appreciate any help or feedback!chrony has been a bit...fidgety with me, the testptp utility will show at least whether your input is pulsing and being read or not.
Matt
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
On Thu, Feb 2, 2023 at 1:49 AM John Miller via time-nuts
time-nuts@lists.febo.com wrote:
Either way, am not able to get chrony to lock up to a PHC refclock.[5] This is what my chrony.conf looks like: https://paste.millerjs.org/ajoxigiquc.txt
Your PHC refclock line is:
refclock PHC /dev/ptp0:nocrossts:pin=0 width 0.09 poll 2 refid PHC pps
The fix is easy. Just use the extpps option:
refclock PHC /dev/ptp0:extpps width 0.1 poll 2 pps
I am not sure why you are using "width 0.09": 100ms corresponds to
"width 0.1". You don't need pin=0 option (it's the default) nor the
nocrossts option (i210-T1 doesn't support it)
I just tried this config on an i210-T1 card connected to a u-blox M8T
and it works fine.
I would then use the "lock GPS" option (instead of the "pps" option)
to link the PHC refclock driver pulses to the time-of-day from the
GPS.
James
Dear John,
On Wednesday, 1 February 2023 19:38:18 CET John Miller via time-nuts wrote:
Hi Matt,
This is excellent information, thank you - after reading through all the
responses in the thread (wow, this is a wild topic!) and trawling around
online a bit, I think that using the SDP pins on the i210 to feed a PPS
signal into an x86 PC is what I want to do right now.
I have read Dan Drown's blog post on using the APU2 to achieve this, and
while it's more or less exactly what I want to replicate, for now, there
are a few gaps I need to fill in - and you may be able to help me do so. I
don't have a PC Engines APU2 board yet, but I suspect I'll order one once I
have the software side of things ironed out, because I really like their
form factor.
I'm currently testing an APU6b4 unit and made a small adaptor PCB that gets
clicked into the Mini-PCIE slot and touches to the main pcb via some pogo-pins:
No soldering on the main apu board. If you are interested, I can send you
the gerber-files and BOM. It should also fit the apu3-series (and possibly
yours, too) and allows convenient SMA-access through the front panel to
the SDP pins of two of the NICs.
What I'm using right now is a Portwell NANO-6050[1], and it has i210 and
i218 NICs. The SDP pins on the i210 aren't broken out into nice pads like
the APU2 has, but with some magnet wire and a steady hand I was able to
connect directly to one of the pins. Where I am struggling right now is
ironing out the software configuration side of things. I'm not interested
at all in PTP yet, right now the focus is purely on feeding the PPS into
chrony via the SDP pins.
The GNSS I'm using is a Sparkfun uBlox NEO-M9N
GNSS receiver in its default configuration, connected to the computer over
USB. PPS pin connected to the i210's SDP0. PPS pulse is 3v3 for 100ms.
That should work fine.
I have tried using refclock strings from chrony's examples page[2] as a
jumping-off point, but it doesn't make much sense to me.
To me it seems that chrony is just using the timestamps on the PHC of that
network interface and not actually locking the PHC to the PPS input. As I
wanted to verify that the lock is working, and maybe later use the PHC also
for PTP, I also wanted the PHC to follow the external time, and running
both chrony and phc2sys to make the phc follow system time ended in chaos.
Therefore it seems that a two-step approach works better: First I make sure
that using ts2phc the PHC is locked to the 100ms-1PPS input on SDP0:
----------[contents of /etc/linuxptp/ts2phc.conf]-------------------------
run with ts2phc -c eth3 -s generic -l 7 -f /etc/linuxptp/ts2phc.conf -m -q
[global]
use_syslog 0
verbose 1
logging_level 6
ts2phc.pulsewidth 100000000
max_frequency 1000000
step_threshold 0.05
[eth3]
ts2phc.channel 0
ts2phc.extts_polarity both
ts2phc.pin_index 0
ts2phc.extts_correction -32
For this to start up properly, you need to make sure the clocks are roughly correct,
i.e. you need to set your system time (e.g. with ntpdate, ntp, chrony) and then
sync your PHC to system time by letting
Locking the PHC of eth3 (ptp3) to CLOCK_REALTIME
phc2sys -c eth3 -s CLOCK_REALTIME -O +37 -m -q
run for a few seconds.
If you also output the PHC timer on SDP1 by
testptp -d /dev/ptp3 -L1,2 -p 1000000000
you can see a falling edge right when SDP0 has a rising edge input
with only a few nanoseconds jitter.
(if anyone know how to invert the 1PPS output on i211 interfaces, let me know...)
Sidenote: The interfaces /sys/class/ptp/* seem to be broken with my kernel.
I can configure PPS output there, but every time I do that I get random
frequencies for ~10 seconds before the output actually starts...
When the PCH is locked to the 1PPS, I can have chrony use the PHC as clock input
with this chrony.conf line:
refclock PHC /dev/ptp3 tai refid PTP3 dpoll -4 poll -2
I do not use the "extpps" option here, as ts2phc is already reading and consuming
the time stamps. If there are two time-stamp readers, my chrony just hangs.
I have not been able to use the testptp tool on this system yet - building
it on another machine and running the copied-over executable fails, citing
too old a version of glibc, and when I try and build it locally it fails
for reasons that are unknown to me (I am very much not a C developer).
Maybe try making a static library: gcc -static testptp.c -o testptp
so that it has it's current glibc version linked in.
I am going to continue to fiddle with this - I think getting testptp working
such that I can verify that I'm actually getting a pulse on SDP0 is the
most important thing to confirm right now.
The intel cards always give timestamp events on both pulse edges. The pulse-width and
the roughly correct clock are necessary such that ts2phc can ignore the wrong pulse edge.
From the source code it seems that ts2phc only considers edges falling within ±pulsewidth/2
around the expected time valid and discards the rest. So for a 100ms-1PPS pulse you should
get the clock precise to 50ms before you start. As my pps is originally only µs long, I needed extra
hardware to stretch it.
The result:
watch -n 0.5 'chronyc -m "sources -v" "sourcestats -v" "selectdata -v"'
.-- Source mode '^' = server, '=' = peer, '#' = local clock.
/ .- Source state '*' = current best, '+' = combined, '-' = not combined,
| / 'x' = may be in error, '~' = too variable, '?' = unusable.
|| .- xxxx [ yyyy ] +/- zzzz
|| Reachability register (octal) -. | xxxx = adjusted offset,
|| Log2(Polling interval) --. | | yyyy = measured offset,
|| \ | | zzzz = estimated error.
|| | |
MS Name/IP address Stratum Poll Reach LastRx Last sample
---=============
#* PTP3 0 -2 377 0 +2ns[ +2ns] +/- 195ns
^- sth1-ts.nts.netnod.se 1 8 377 83 -449us[ -450us] +/- 5331us
^- sth2-ts.nts.netnod.se 1 8 377 213 -461us[ -463us] +/- 5322us
^? ptbtime1.ptb.de 1 8 377 21 -46us[ -46us] +/- 5489us
^? ptbtime2.ptb.de 1 7 377 19 -60us[ -60us] +/- 5505us
^- polarx5tr.time.internal 1 8 377 147 -41us[ -43us] +/- 12ms
.- Number of sample points in measurement set.
/ .- Number of residual runs with same sign.
| / .- Length of measurement set (time).
| | / .- Est. clock freq error (ppm).
| | | / .- Est. error in freq.
| | | | / .- Est. offset.
| | | | | | On the -.
| | | | | | samples.
| | | | | | |
Name/IP Address NP NR Span Frequency Freq Skew Offset Std Dev
---============
PTP3 9 7 2 +0.000 0.005 +0ns 2ns
sth1-ts.nts.netnod.se 25 11 31m +0.060 0.044 -483us 28us
sth2-ts.nts.netnod.se 22 10 29m +0.097 0.031 -419us 16us
ptbtime1.ptb.de 25 12 32m +0.071 0.031 -43us 23us
ptbtime2.ptb.de 21 5 23m +0.097 0.053 -5339ns 26us
polarx5tr.time.internal 25 14 30m +0.038 0.153 -41us 104us
.-- State: N - noselect, M - missing samples, d/D - large distance,
/ ~ - jittery, w/W - waits for others, T - not trusted,
| x - falseticker, P - not preferred, U - waits for update,
| S - stale, O - orphan, + - combined, * - best.
| Effective options ------. (N - noselect, P - prefer
| Configured options -. \ T - trust, R - require)
| Auth. enabled (Y/N) -. \ \ Offset interval --.
| | | | |
S Name/IP Address Auth COpts EOpts Last Score Interval Leap
---=====
- PTP3 N ----- --TR- 0 1.0 -194ns +199ns N
D sth1-ts.nts.netnod.se Y ----- --TR- 83 1.0 -5780us +4891us N
D sth2-ts.nts.netnod.se Y ----- --TR- 213 1.0 -5769us +4888us N
T ptbtime1.ptb.de N ----- ----- 20 1.0 -5515us +5458us N
T ptbtime2.ptb.de N ----- ----- 19 1.0 -5564us +5448us N
D polarx5tr.time.internal N --TR- --TR- 147 1.0 -12ms +11ms N
Cheers,
Jürgen
On 2/1/23 4:15 PM, Bob Camp via time-nuts wrote:
Hi
A 10 or 100 microsecond wide 1 pps signal is more common than a 10 to 100 millisecond
wide pulse.
Why?
Back in the day, they coupled this stuff via a transformer. The frequency response issues
made a narrower pulse more appealing. These days it is not uncommon to try to drive a
50 ohm load to some sort of logic level. The less time you have to do that, the happier
your driver IC (and regulator) are likely to be.
One example of this would be the Trimble TBolt. It has a 10 us wide output pulse. Another
would be a uBlox LEA-6T (and the others in that lineup) with a default 10us wide output.
Bob
Power dissipation, too. Fewer joules in a 10 microsecond pulse than in a
10 millisecond pulse.
On Fri, Feb 3, 2023 at 1:30 AM Matt Corallo via time-nuts
time-nuts@lists.febo.com wrote:
Makes sense.
Sadly, as far as I can tell, the i211 will mark both the rising and falling edge of the pulse, with
no distinction between the two. While you'd expect chrony to be able to differentiate based on the
time between them, the manual for chrony.conf seems to indicate otherwise, saying:
width width
This option specifies the width of the pulses (in seconds). It is used to filter PPS samples when
the driver provides samples for both rising and falling edges. Note that it reduces the maximum
allowed error of the time source which completes the PPS samples. If the duty cycle is
configurable, 50% should be preferred in order to maximise the allowed error.
which, maybe incorrectly, implied to me that there was no smarts for highly asymmetric
pulse-to-not-pulsing time.
I think your analysis is correct, since
https://chrony.tuxfamily.org/examples.html#_server_using_reference_clock_on_nic
says
"The offset option of the SHM 0 refclock compensates for the delay of
messages received on the USB port. It needs to be measured carefully,
e.g. against a known good NTP server. A wrong offset could cause the
server to be off by an integer multiple of 62.5 milliseconds (1/16s)."
I've been writing a program to sync the PHC from a GPS and it includes
the obvious smarts to distinguish the rising and falling edges based
on the time between. I have been testing with the i210 and it seems
to work fine. With these smarts, 50% duty cycle is the worst possible
configuration, since it prevents the smarts from working.
James
On Fri, Feb 3, 2023 at 1:57 AM Jürgen Appel via time-nuts
time-nuts@lists.febo.com wrote:
Hi Jürgen
I have been experimenting with PTP and the i210 quite a bit, so it's
interesting to compare notes.
I'm currently testing an APU6b4 unit and made a small adaptor PCB that gets
clicked into the Mini-PCIE slot and touches to the main pcb via some pogo-pins:
No soldering on the main apu board. If you are interested, I can send you
the gerber-files and BOM. It should also fit the apu3-series (and possibly
yours, too) and allows convenient SMA-access through the front panel to
the SDP pins of two of the NICs.
This sounds really interesting and looks like it would be a nice
option for a small-form factor NTP/PTP server. I would love to be able
to buy one of these. Are there companies that will take a gerber file
and a BOM and a modest amount of money, and send me a completed PCB?
(I am a software person, and know very little about the hardware side
of things.)
As I
wanted to verify that the lock is working, and maybe later use the PHC also
for PTP, I also wanted the PHC to follow the external time,
I also wanted this.
Therefore it seems that a two-step approach works better: First I make sure
that using ts2phc the PHC is locked to the 100ms-1PPS input on SDP0:
----------[contents of /etc/linuxptp/ts2phc.conf]-------------------------
run with ts2phc -c eth3 -s generic -l 7 -f /etc/linuxptp/ts2phc.conf -m -q
I have found it quite tricky to get this approach to startup robustly.
There's a dependency circularity:
- ts2phc -s generic depends on chrony to have set the system clock
- chrony with the PHC refclock depends on ts2phc to have set the PHC
I have tried a couple of approaches to deal with this.
phc2sys has a -E ntpshm option. With this option, instead of phc2sys
modifying the system clock directly, it feeds samples into the SHM
used by the chrony SHM refclock. This provides a bit more flexibility
over using the chrony PHC refclock directly: if phc2sys is not running
then chrony will ignore the SHM refclock. We can then startup in
stages:
- Run chrony with the SHM refclock; do not yet start phc2sys -E
ntpshm. Chrony will then set the system time using other NTP servers
or using the NMEA messages from a GPS receiver via gpsd if there is no
network connectivity (which would mean the time is very rough). - Run ts2phc -s generic. This will set the PHC very accurately, using
the less accurate system time from chrony. - Start phc2sys -E ntpshm. Now that the PHC is correct, we can let
chrony use the samples from the PHC.
Another approach is to use a pps option (not the extpps option) with
the PHC refclock. This will make chrony ignore the time-of-day
information in the PHC, and just use the PPS information.
For this to start up properly, you need to make sure the clocks are roughly correct,
i.e. you need to set your system time (e.g. with ntpdate, ntp, chrony) and then
sync your PHC to system time by letting
Locking the PHC of eth3 (ptp3) to CLOCK_REALTIME
phc2sys -c eth3 -s CLOCK_REALTIME -O +37 -m -q
run for a few seconds.
As far as I remember, ts2phc can handle the PHC being wrong. If you
want to do a one-shot setting of the PHC from the system clock, you
can do
phc_ctl eth0 "set;" adj 37
This works immediately, although it is not as accurate as phc2sys.
The intel cards always give timestamp events on both pulse edges. The pulse-width and
the roughly correct clock are necessary such that ts2phc can ignore the wrong pulse edge.
From the source code it seems that ts2phc only considers edges falling within ±pulsewidth/2
around the expected time valid and discards the rest. So for a 100ms-1PPS pulse you should
get the clock precise to 50ms before you start. As my pps is originally only µs long, I needed extra
hardware to stretch it.
I am writing code (which will be released as open source) that amongst
other things tries to deal better with receiving timestamp events for
both pulse edges. Does the i210/i211 generate two events for the
original µs pulse? In your use case, do you have a GPS receiver
providing time-of-day information or do you just have the PPS?
James
On Mon, Jan 30, 2023 at 8:19 PM John Miller via time-nuts
time-nuts@lists.febo.com wrote:
Hey All,
I'm curious as to what the collective experience is amongst this group when it comes to feeding a timestamp signal into a NIC, either for PTP purposes or for as a normal PPS refclock. The Intel i210 PCIe network card has a few software defined pins that can be used for this purpose, and the chrony documentation has some information about it:
https://chrony.tuxfamily.org/examples.html#_server_using_reference_clock_on_nic
I imagine most of us here use traditional RS232 serial ports to get this signal into an x86 computer, usually using the DCD pin. I've found some implementations of RS232 on PCs don't implement all of the additional signaling pins, so sometimes DCD is flat out missing. In other cases - especially with smaller embedded-type boards only UARTs with Rx/Tx are available to the user. (I'm fiddling with using GPIO pins on a few such systems, but it's quite a struggle.) So, there is a need for getting PPS signals into systems.
There are also highly specialized network interface cards, such as this one: https://www.ebay.com/itm/404117321209
... which does explicitly support PTP timestamping, but it's not clear to me if it could pull in a "standard" PPS signal to be made available to chrony or NTPsec. I haven't found much documentation on the NT20E2 yet, unfortunately, and it may simply not exist for hobbyists.
Does anyone here know if any other network cards that support software defined pins, like the i210 does, or any other methods for getting a PPS signal to be fed into a PC, and of course interpreted correctly?
Thanks!
John
Apologies for the delayed response.
In case not already seen
https://linuxptp.sourceforge.net/i210-rework/i210-rework.html
There are also some SFP+ (10G) Solarflare cards from a decade ago or
so (model 107224, but finding them could be fun) that have (on SMA
connectors) PPS in & out, which were there to get PPS in or out of
PTP.
Cheers
Thanks for the tip! I'll keep my eyes peeled. Looks like there are still quite a few out there - the more common designation for them is SolarFlare SFN6322F.
Regards,
John
Apologies for the delayed response.
In case not already seen
https://linuxptp.sourceforge.net/i210-rework/i210-rework.html
There are also some SFP+ (10G) Solarflare cards from a decade ago or
so (model 107224, but finding them could be fun) that have (on SMA
connectors) PPS in & out, which were there to get PPS in or out of
PTP.
Cheers
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
Hi Jürgen,
Finally getting around to sinking my teeth into doing i210 PPS stuff, so I am finally revisiting this. So, first off - thank you for the very informative post - lots of good information in here. I am very interested in seeing your design data for a mini PCIe board with pogo pins for the SDP interface, thank you for offering. I have an apu2e0 on order, and I plan to put a mini PCIe GNSS receiver in it - of course, the options for miniPCIe GNSS receivers are pretty bad, so I've designed my own and should get them in two weeks or so, but I am very interested in coupling one with your design for a very tidy solution that doesn't require soldering.
Compiling testptp with a static library worked, of course, and I'm a little bit disappointed in myself for not thinking of this - I'm pretty rusty at this, go figure. In any case, it seems to be working fine - most of the time. I can use it to list capabilities, list pin configuration, and set/get PTP time, no problem, and I can see that SDP0 is configured for "external time stamp". Sometimes when I try and run it to view timestamp events (e.g. ./testptp -e 10) I get a cryptic "device in use" type error but I'm not quite sure what is hanging it up.
In the following command:
ts2phc -c eth3 -s generic -l 7 -f /etc/linuxptp/ts2phc.conf -m -q
i assume the contents of ts2phc.conf are the "[global] ... " that you posted below it, is that correct?
Thanks very much for all your info, this is very exciting!
Regards,
John
On Feb 2, 2023, at 1:32 PM, Jürgen Appel via time-nuts time-nuts@lists.febo.com wrote:
Dear John,
On Wednesday, 1 February 2023 19:38:18 CET John Miller via time-nuts wrote:
Hi Matt,
This is excellent information, thank you - after reading through all the
responses in the thread (wow, this is a wild topic!) and trawling around
online a bit, I think that using the SDP pins on the i210 to feed a PPS
signal into an x86 PC is what I want to do right now.
I have read Dan Drown's blog post on using the APU2 to achieve this, and
while it's more or less exactly what I want to replicate, for now, there
are a few gaps I need to fill in - and you may be able to help me do so. I
don't have a PC Engines APU2 board yet, but I suspect I'll order one once I
have the software side of things ironed out, because I really like their
form factor.
I'm currently testing an APU6b4 unit and made a small adaptor PCB that gets
clicked into the Mini-PCIE slot and touches to the main pcb via some pogo-pins:
No soldering on the main apu board. If you are interested, I can send you
the gerber-files and BOM. It should also fit the apu3-series (and possibly
yours, too) and allows convenient SMA-access through the front panel to
the SDP pins of two of the NICs.
What I'm using right now is a Portwell NANO-6050[1], and it has i210 and
i218 NICs. The SDP pins on the i210 aren't broken out into nice pads like
the APU2 has, but with some magnet wire and a steady hand I was able to
connect directly to one of the pins. Where I am struggling right now is
ironing out the software configuration side of things. I'm not interested
at all in PTP yet, right now the focus is purely on feeding the PPS into
chrony via the SDP pins.
The GNSS I'm using is a Sparkfun uBlox NEO-M9N
GNSS receiver in its default configuration, connected to the computer over
USB. PPS pin connected to the i210's SDP0. PPS pulse is 3v3 for 100ms.
That should work fine.
I have tried using refclock strings from chrony's examples page[2] as a
jumping-off point, but it doesn't make much sense to me.
To me it seems that chrony is just using the timestamps on the PHC of that
network interface and not actually locking the PHC to the PPS input. As I
wanted to verify that the lock is working, and maybe later use the PHC also
for PTP, I also wanted the PHC to follow the external time, and running
both chrony and phc2sys to make the phc follow system time ended in chaos.
Therefore it seems that a two-step approach works better: First I make sure
that using ts2phc the PHC is locked to the 100ms-1PPS input on SDP0:
----------[contents of /etc/linuxptp/ts2phc.conf]-------------------------
run with ts2phc -c eth3 -s generic -l 7 -f /etc/linuxptp/ts2phc.conf -m -q
[global]
use_syslog 0
verbose 1
logging_level 6
ts2phc.pulsewidth 100000000
max_frequency 1000000
step_threshold 0.05
[eth3]
ts2phc.channel 0
ts2phc.extts_polarity both
ts2phc.pin_index 0
ts2phc.extts_correction -32
For this to start up properly, you need to make sure the clocks are roughly correct,
i.e. you need to set your system time (e.g. with ntpdate, ntp, chrony) and then
sync your PHC to system time by letting
Locking the PHC of eth3 (ptp3) to CLOCK_REALTIME
phc2sys -c eth3 -s CLOCK_REALTIME -O +37 -m -q
run for a few seconds.
If you also output the PHC timer on SDP1 by
testptp -d /dev/ptp3 -L1,2 -p 1000000000
you can see a falling edge right when SDP0 has a rising edge input
with only a few nanoseconds jitter.
(if anyone know how to invert the 1PPS output on i211 interfaces, let me know...)
Sidenote: The interfaces /sys/class/ptp/* seem to be broken with my kernel.
I can configure PPS output there, but every time I do that I get random
frequencies for ~10 seconds before the output actually starts...
When the PCH is locked to the 1PPS, I can have chrony use the PHC as clock input
with this chrony.conf line:
refclock PHC /dev/ptp3 tai refid PTP3 dpoll -4 poll -2
I do not use the "extpps" option here, as ts2phc is already reading and consuming
the time stamps. If there are two time-stamp readers, my chrony just hangs.
I have not been able to use the testptp tool on this system yet - building
it on another machine and running the copied-over executable fails, citing
too old a version of glibc, and when I try and build it locally it fails
for reasons that are unknown to me (I am very much not a C developer).
Maybe try making a static library: gcc -static testptp.c -o testptp
so that it has it's current glibc version linked in.
I am going to continue to fiddle with this - I think getting testptp working
such that I can verify that I'm actually getting a pulse on SDP0 is the
most important thing to confirm right now.
The intel cards always give timestamp events on both pulse edges. The pulse-width and
the roughly correct clock are necessary such that ts2phc can ignore the wrong pulse edge.
From the source code it seems that ts2phc only considers edges falling within ±pulsewidth/2
around the expected time valid and discards the rest. So for a 100ms-1PPS pulse you should
get the clock precise to 50ms before you start. As my pps is originally only µs long, I needed extra
hardware to stretch it.
The result:
watch -n 0.5 'chronyc -m "sources -v" "sourcestats -v" "selectdata -v"'
.-- Source mode '^' = server, '=' = peer, '#' = local clock.
/ .- Source state '*' = current best, '+' = combined, '-' = not combined,
| / 'x' = may be in error, '~' = too variable, '?' = unusable.
|| .- xxxx [ yyyy ] +/- zzzz
|| Reachability register (octal) -. | xxxx = adjusted offset,
|| Log2(Polling interval) --. | | yyyy = measured offset,
|| \ | | zzzz = estimated error.
|| | |
MS Name/IP address Stratum Poll Reach LastRx Last sample
---=============
#* PTP3 0 -2 377 0 +2ns[ +2ns] +/- 195ns
^- sth1-ts.nts.netnod.se http://sth1-ts.nts.netnod.se/ 1 8 377 83 -449us[ -450us] +/- 5331us
^- sth2-ts.nts.netnod.se http://sth2-ts.nts.netnod.se/ 1 8 377 213 -461us[ -463us] +/- 5322us
^? ptbtime1.ptb.de http://ptbtime1.ptb.de/ 1 8 377 21 -46us[ -46us] +/- 5489us
^? ptbtime2.ptb.de http://ptbtime2.ptb.de/ 1 7 377 19 -60us[ -60us] +/- 5505us
^- polarx5tr.time.internal 1 8 377 147 -41us[ -43us] +/- 12ms
.- Number of sample points in measurement set.
/ .- Number of residual runs with same sign.
| / .- Length of measurement set (time).
| | / .- Est. clock freq error (ppm).
| | | / .- Est. error in freq.
| | | | / .- Est. offset.
| | | | | | On the -.
| | | | | | samples.
| | | | | | |
Name/IP Address NP NR Span Frequency Freq Skew Offset Std Dev
---============
PTP3 9 7 2 +0.000 0.005 +0ns 2ns
sth1-ts.nts.netnod.se http://sth1-ts.nts.netnod.se/ 25 11 31m +0.060 0.044 -483us 28us
sth2-ts.nts.netnod.se http://sth2-ts.nts.netnod.se/ 22 10 29m +0.097 0.031 -419us 16us
ptbtime1.ptb.de http://ptbtime1.ptb.de/ 25 12 32m +0.071 0.031 -43us 23us
ptbtime2.ptb.de http://ptbtime2.ptb.de/ 21 5 23m +0.097 0.053 -5339ns 26us
polarx5tr.time.internal 25 14 30m +0.038 0.153 -41us 104us
.-- State: N - noselect, M - missing samples, d/D - large distance,
/ ~ - jittery, w/W - waits for others, T - not trusted,
| x - falseticker, P - not preferred, U - waits for update,
| S - stale, O - orphan, + - combined, * - best.
| Effective options ------. (N - noselect, P - prefer
| Configured options -. \ T - trust, R - require)
| Auth. enabled (Y/N) -. \ \ Offset interval --.
| | | | |
S Name/IP Address Auth COpts EOpts Last Score Interval Leap
---=====
- PTP3 N ----- --TR- 0 1.0 -194ns +199ns N
D sth1-ts.nts.netnod.se http://sth1-ts.nts.netnod.se/ Y ----- --TR- 83 1.0 -5780us +4891us N
D sth2-ts.nts.netnod.se http://sth2-ts.nts.netnod.se/ Y ----- --TR- 213 1.0 -5769us +4888us N
T ptbtime1.ptb.de http://ptbtime1.ptb.de/ N ----- ----- 20 1.0 -5515us +5458us N
T ptbtime2.ptb.de http://ptbtime2.ptb.de/ N ----- ----- 19 1.0 -5564us +5448us N
D polarx5tr.time.internal N --TR- --TR- 147 1.0 -12ms +11ms N
Cheers,
Jürgen
time-nuts mailing list -- time-nuts@lists.febo.com mailto:time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com mailto:time-nuts-leave@lists.febo.com