Dear time-nuts, On Tuesday, 14 February 2023 01:50:04 CET, John Miller <john@millerjs.org> wrote: > Hi Jürgen, > Finally getting around to sinking my teeth into doing i210 PPS stuff, so I > am finally revisiting this. Great, I'm happy to have some company in this project :-) > So, first off - thank you for the very > informative post - lots of good information in here. I am very interested > in seeing your design data for a mini PCIe board with pogo pins for the SDP > interface, thank you for offering. You got mail ;-) > Compiling testptp with a static library worked, of course, and I'm a little > bit disappointed in myself for not thinking of this - I'm pretty rusty at > this, go figure. In any case, it seems to be working fine - most of the > time. I can use it to list capabilities, list pin configuration, and > set/get PTP time, no problem, and I can see that SDP0 is configured for > "external time stamp". Sometimes when I try and run it to view timestamp > events (e.g. ./testptp -e 10) I get a cryptic "device in use" type error > but I'm not quite sure what is hanging it up. To me that happens when I don't have the pin configured for timestamping correctly, i.e. forgotten to run "-L x,1" or configured the wrong ptp-device. Note: testptp requires a '-d' switch whereas phc_ctl does not. > > ts2phc -c eth3 -s generic -l 7 -f /etc/linuxptp/ts2phc.conf -m -q > > i assume the contents of ts2phc.conf are the "[global] ... " that you posted > below it, is that correct? yes. As I am trying to learn the intricacies of the i211 chip and its PTP timing, maybe time-nuts find the following technical details useful: I did some experiments where I programmed the SDP1 to output 1pps and fed it back via cables of calibrated length into SDP0 of the same i211 chip for timestamping: # testptp -d /dev/ptp2 -i 0 -L 1,2 -p 1000000000 The i211-chip system timer is incremented by a fixed value each of its 125 MHz clock cycles. To set the increment fix to 8 ns, first the frequency-adjustment has to be set to zero: # phc_ctl eth2 freq 0 Then for an external cable delay of 12.8 ns (measured with VNA by reflection into the unpowered chip), my pulse ends up just between two detection clock cycles and I measure either a 24 ns or 32 ns delay for the detected edge. The jitter seems O.K.: Changing the cable delay length by 250 ps to either side leaves me firmly in the 24 ns or 32 ns-domain. # testptp -d /dev/ptp2 -i 0 -L 2,1 -e 10 set pin function okay external time stamp request okay event index 0 at 1677064396.000000032 event index 0 at 1677064396.500000032 event index 0 at 1677064397.000000024 event index 0 at 1677064397.500000032 event index 0 at 1677064398.000000032 event index 0 at 1677064398.500000024 event index 0 at 1677064399.000000032 event index 0 at 1677064399.500000032 event index 0 at 1677064400.000000024 event index 0 at 1677064400.500000032 That means, that there is a delay of at least a little bit more than two clock cycles between generation and time-tagging of the pulse (32ns - 12.8 ns = 19.2 ns). Unfortunately, the data sheet of the i211 is silent on where this comes from - whether the output pulse is produced late, or whether the input pulse is registered late or whether it's both, leaving three possibilities: 1) pulse produced within the right clock cycle but tagged 2 clock cycles after arrival 2) pulse produced 1 clock cycle late and tagged 1 clock cycle after arrival 3) pulse produced 2 clock cycles delayed and tagged at the right clock cycle. My best guess is 1) (assuming a 2ff-synchronizer within the i211-chip to sample the asynchronous input signals) which the i211 designers just forgot about to document in the datasheet, but we really cannot know, leaving us with this 2.5-cycle systematic offset uncertainty between the internal timer and the external signals. Since the behavior is stable and predictable, this is only significant if time stamps with nanosecond precision leave or enter the computer on other ways than via the i211 SDP pins (e.g. another brand network interface, a DAC/ADC card, GPIO-pins etc.). Using my little interface board, I can now also lock two i211 interfaces on the same APU-motherboard (i.e. each with independent 25MHz crystals), either via ethernet & ptp4l or by feeding the 1pps-output of one chip into the other and using ts2phc. Attached you find some scope traces of the 1pps outputs of master (CH2, teal) and slave (CH3, pink). While the two crystals clocking the i211 drift slowly with respect to each other, the software can decide to correct the 'frequency' of the slave. As it is not really possible to change the real frequency of the hardware oscillator (there is no VCO involved) the software instead changes the amount the chip's system time is incremented at each clock cycle, i.e. the output pulse pulse timing can only be stepped by 8 ns increments effectively, leaving an at least one clock cycle wide pink shaded area in the scope traces that fundamentally cannot be avoided with a 8 ns timing detection resolution even with a perfect lock. When connected directly via an Ethernet-cable the two interfaces lock nicely to each other via ptp4l (see ptp4l.png). When the master frequency tuning is set to 0, the slave is locked at an average offset within approximately -4 ns to +4 ns depending on the master clock-value modulo 8 ns. If the master 'frequency'-adjustment is unequal 0, the timing of the master 1pps is dithered with 8 ns steps, effectively doubling the jitter-variance between the 1pps pulses. When 1pps from eth3.SDP2 is fed into eth2.SDP3 (short connection with less than 1 ns propagation delay) and I lock via ts2phc, I get an 21.6 ns offset on average, agreeing roughly with the 19.2 ns-value obtained by time-tagging the pulses produced within the same chip. This average shift can be corrected for by the ts2phc.extts_correction parameter. > Thanks very much for all your info, this is very exciting! Indeed. As soon as I get some commercial PTP equipment, I'm curious to see whether the offsets agree... Cheers, Jürgen