time-nuts@lists.febo.com

Discussion of precise time and frequency measurement

View all threads

NTP jitter with Linux

JH
Javier Herrero
Sat, Apr 7, 2012 5:08 PM

El 07/04/2012 16:02, Jim Lux escribió:

On 4/7/12 4:47 AM, Javier Herrero wrote:

I'm very familiar with the LEON and RTEMS, having managed a software
development project with it for the last 3 or 4 years at work.

http://www.gaisler.com/ for LEON
http://www.rtems.org/ for RTEMS

I will have a look to RTEMS

And yes, there is a port (maybe two) of Linux for the LEON as well (A
few years ago, we loaded up the Snapgear port, but since we went
RTEMS, I haven't fooled with it).  You'd have to check the Gaisler.com
website.

I've done. The Snapgear port is quite old now, but the other port is
actively maintained and updated with current kernel.

You can drop a LEON core into a Virtex II in about a day, and judging
from the traffic on the LEON yahoo list (where the Gaisler folks hang
out), lots of people are doing things like multiple cores and things
on all manner of Xilinx eval boards.

And also for Altera (for example, the Terasic DE2-115 with a Cyclone IV)
and others. I've seen you int that list :) An I've seen implementations
for smaller FPGAs like the Spatarn 6LX25

RTEMS wise... It's pretty well supported by the community, it's open
source, it does all the stuff you want a RTOS to do.  it's NOT a
multitasking, dynamic loading OS like Linux.  That is it doesn't
support an MMU and process space isolation (although that might be
possible in newer versions.. there's a lot of configurability).  It's
basically a statically linked single task with threads.  They've got
RAM (and disk) file systems, IP stacks, a shell, YAFFS, etc.

Like all open source, there's quite a lot of interesting stuff
available (not from rtems.org, but others) that is 90% complete.
Somebody at Google Summer of Code or for their Masters decides to
implement something cool, and gets most of the way done, then wanders
away (the summer ended, they got their degree, the usual story).

But there's also a core of users who are serious and rigorous and
contribute back, so the main stuff in the distribution from Joel
Sherrill at OAR (who make RTEMS) is pretty rock solid.

I will learn more about RTEMS. For the application I've (and this links
directly to the message from Javier Serrano), the hardware platform is
one of the CERN Open Hardware ones, the SPEC. For the purpose and
interface needs, really an operating system is not required (no
filesystem, no TCP/IP needed, no multitasking, no framebuffer...), and
certainly a Linux would have a very large footprint without providing
any real help. And about the processor selection, the trade-off that
Javier exposes are the same I'm confronting. Both are open-sourced and
well supported, and in one side the LM32 is smaller, in the other the
LEON3 has more capabilities that can be implemented or not (like MMU or
FPU, and better multi-core support, although not currently needed in my
project). I probably will take the LEON3 road, but also because it is
more popular in my current field, but for now I usually do not need the
FT version since I'm more related with GSEs.

ESA has several rigorously verified flight qualified versions of RTEMS
(in Portugal and Austria, as I recall)

Yes, this is one of the reasons to gain experience in that road :) I
have some tendency to stay in Linux because I'm very familiarized with
it in the non-MMU implementations (for Blackfin) and also with MMU - and
I've found that for a small embedded system, to have the MMU is not so
important, even sometimes it is a drawback.

In any case we are running a bit OT (except considering that this
general discussion has timing applications, of course ;) ). Also I'm
happy to have found a time-nut colleage in other list, and probably I
will ask you some things about off-list in order to not increase noise,
if possible.

Best regards,

Javier

El 07/04/2012 16:02, Jim Lux escribió: > On 4/7/12 4:47 AM, Javier Herrero wrote: > > I'm very familiar with the LEON and RTEMS, having managed a software > development project with it for the last 3 or 4 years at work. > > http://www.gaisler.com/ for LEON > http://www.rtems.org/ for RTEMS I will have a look to RTEMS > > And yes, there is a port (maybe two) of Linux for the LEON as well (A > few years ago, we loaded up the Snapgear port, but since we went > RTEMS, I haven't fooled with it). You'd have to check the Gaisler.com > website. I've done. The Snapgear port is quite old now, but the other port is actively maintained and updated with current kernel. > > You can drop a LEON core into a Virtex II in about a day, and judging > from the traffic on the LEON yahoo list (where the Gaisler folks hang > out), lots of people are doing things like multiple cores and things > on all manner of Xilinx eval boards. And also for Altera (for example, the Terasic DE2-115 with a Cyclone IV) and others. I've seen you int that list :) An I've seen implementations for smaller FPGAs like the Spatarn 6LX25 > > > RTEMS wise... It's pretty well supported by the community, it's open > source, it does all the stuff you want a RTOS to do. it's NOT a > multitasking, dynamic loading OS like Linux. That is it doesn't > support an MMU and process space isolation (although that might be > possible in newer versions.. there's a lot of configurability). It's > basically a statically linked single task with threads. They've got > RAM (and disk) file systems, IP stacks, a shell, YAFFS, etc. > > Like all open source, there's quite a lot of interesting stuff > available (not from rtems.org, but others) that is 90% complete. > Somebody at Google Summer of Code or for their Masters decides to > implement something cool, and gets most of the way done, then wanders > away (the summer ended, they got their degree, the usual story). > > But there's also a core of users who are serious and rigorous and > contribute back, so the main stuff in the distribution from Joel > Sherrill at OAR (who make RTEMS) is pretty rock solid. I will learn more about RTEMS. For the application I've (and this links directly to the message from Javier Serrano), the hardware platform is one of the CERN Open Hardware ones, the SPEC. For the purpose and interface needs, really an operating system is not required (no filesystem, no TCP/IP needed, no multitasking, no framebuffer...), and certainly a Linux would have a very large footprint without providing any real help. And about the processor selection, the trade-off that Javier exposes are the same I'm confronting. Both are open-sourced and well supported, and in one side the LM32 is smaller, in the other the LEON3 has more capabilities that can be implemented or not (like MMU or FPU, and better multi-core support, although not currently needed in my project). I probably will take the LEON3 road, but also because it is more popular in my current field, but for now I usually do not need the FT version since I'm more related with GSEs. > > > ESA has several rigorously verified flight qualified versions of RTEMS > (in Portugal and Austria, as I recall) > Yes, this is one of the reasons to gain experience in that road :) I have some tendency to stay in Linux because I'm very familiarized with it in the non-MMU implementations (for Blackfin) and also with MMU - and I've found that for a small embedded system, to have the MMU is not so important, even sometimes it is a drawback. In any case we are running a bit OT (except considering that this general discussion has timing applications, of course ;) ). Also I'm happy to have found a time-nut colleage in other list, and probably I will ask you some things about off-list in order to not increase noise, if possible. Best regards, Javier
AR
Andrew Rodland
Sat, Apr 7, 2012 5:53 PM

Chris Albertson <albertson.chris@...> writes:

On Fri, Apr 6, 2012 at 9:41 PM, Andrew Rodland <andrew@...> wrote:

Another option would be building something on an FPGA. This would be a
considerable stretch for me, since I've never done FPGA work, but if I build
from the ground up, I can have very tight control over things that are

chosen

for me with a micro controller.

A compromise is to find a "soft core" for the FPGA.  This is a CPU
implemented in FPGA and then it runs software just like a "real" CPU.
This would let you move your micro controller based be sign over to
the FPGA quickly.  After that you can implement some specialized
peripherals that do time stamping

How does the performance of the Arduino based NTP compare with what
you could do with Linux on (say) and Atom or ARM processor?

Pretty well, I think. A PPS generated by my board shows an RMS absolute jitter
of 174ns compared to a Spectracom 9183, which is almost impossibly good
considering that all the timing is done off of a 2MHz clock.

As measured from the NTP side, the performance is diluted a lot by the fact that
the W5100 ethernet chip has unknown and unpredictable delays, but it still comes
back with a jitter of <20us (possibly better — the box I'm using to measure the
NTP is a Linux machine that isn't really a timing champ. Wish I still had my
net4801.)

Andrew

Chris Albertson <albertson.chris@...> writes: > > On Fri, Apr 6, 2012 at 9:41 PM, Andrew Rodland <andrew@...> wrote: > > > Another option would be building something on an FPGA. This would be a > > considerable stretch for me, since I've never done FPGA work, but if I build > > from the ground up, I can have *very* tight control over things that are chosen > > for me with a micro controller. > > A compromise is to find a "soft core" for the FPGA. This is a CPU > implemented in FPGA and then it runs software just like a "real" CPU. > This would let you move your micro controller based be sign over to > the FPGA quickly. After that you can implement some specialized > peripherals that do time stamping > > How does the performance of the Arduino based NTP compare with what > you could do with Linux on (say) and Atom or ARM processor? Pretty well, I think. A PPS generated by my board shows an RMS absolute jitter of 174ns compared to a Spectracom 9183, which is almost impossibly good considering that all the timing is done off of a 2MHz clock. As measured from the NTP side, the performance is diluted a lot by the fact that the W5100 ethernet chip has unknown and unpredictable delays, but it still comes back with a jitter of <20us (possibly better — the box I'm using to measure the NTP is a Linux machine that isn't really a timing champ. Wish I still had my net4801.) Andrew
JL
Jim Lux
Sat, Apr 7, 2012 10:21 PM

On 4/7/12 10:08 AM, Javier Herrero wrote:

El 07/04/2012 16:02, Jim Lux escribió:

RTEMS wise... It's pretty well supported by the community, it's open
source, it does all the stuff you want a RTOS to do. it's NOT a
multitasking, dynamic loading OS like Linux. That is it doesn't
support an MMU and process space isolation (although that might be
possible in newer versions.. there's a lot of configurability). It's
basically a statically linked single task with threads. They've got
RAM (and disk) file systems, IP stacks, a shell, YAFFS, etc.

But there's also a core of users who are serious and rigorous and
contribute back, so the main stuff in the distribution from Joel
Sherrill at OAR (who make RTEMS) is pretty rock solid.

I will learn more about RTEMS. For the application I've (and this links
directly to the message from Javier Serrano), the hardware platform is
one of the CERN Open Hardware ones, the SPEC. For the purpose and
interface needs, really an operating system is not required (no
filesystem, no TCP/IP needed, no multitasking, no framebuffer...), and
certainly a Linux would have a very large footprint without providing
any real help.

RTEMS might be just what you need.  Kernel, basic OS calls for
scheduling, queues, etc. It's nice when you decide you want threading to
not have to graft it into a "big loop no-OS" style program.

You can use native calls or POSIX style (I like POSIX, because I can
develop on Linux and just recompile for the RTEMS target).

There's all the usual GDB support as well.

And about the processor selection, the trade-off that

Javier exposes are the same I'm confronting. Both are open-sourced and
well supported, and in one side the LM32 is smaller, in the other the
LEON3 has more capabilities that can be implemented or not (like MMU or
FPU, and better multi-core support, although not currently needed in my
project). I probably will take the LEON3 road, but also because it is
more popular in my current field, but for now I usually do not need the
FT version since I'm more related with GSEs.

And that's good because the FT version costs money, but the regular old
LEON2 and LEON3 are free, and pretty bulletproof by now.

ESA has several rigorously verified flight qualified versions of RTEMS
(in Portugal and Austria, as I recall)

Yes, this is one of the reasons to gain experience in that road :) I
have some tendency to stay in Linux because I'm very familiarized with
it in the non-MMU implementations (for Blackfin) and also with MMU - and
I've found that for a small embedded system, to have the MMU is not so
important, even sometimes it is a drawback.

Device drivers are easy to write for RTEMS, and it has VERY fast ISRs.
That's probably one of the big advantages..

It's a small footprint, stripped down RTOS, but because you can work
with POSIX API calls, you can do most of your development in Linux
(particuarly things like calibration interfaces and computational stuff)
and then it ports very easily when you move it to RTEMS on the target.

On 4/7/12 10:08 AM, Javier Herrero wrote: > El 07/04/2012 16:02, Jim Lux escribió: >> >> RTEMS wise... It's pretty well supported by the community, it's open >> source, it does all the stuff you want a RTOS to do. it's NOT a >> multitasking, dynamic loading OS like Linux. That is it doesn't >> support an MMU and process space isolation (although that might be >> possible in newer versions.. there's a lot of configurability). It's >> basically a statically linked single task with threads. They've got >> RAM (and disk) file systems, IP stacks, a shell, YAFFS, etc. >> >> But there's also a core of users who are serious and rigorous and >> contribute back, so the main stuff in the distribution from Joel >> Sherrill at OAR (who make RTEMS) is pretty rock solid. > I will learn more about RTEMS. For the application I've (and this links > directly to the message from Javier Serrano), the hardware platform is > one of the CERN Open Hardware ones, the SPEC. For the purpose and > interface needs, really an operating system is not required (no > filesystem, no TCP/IP needed, no multitasking, no framebuffer...), and > certainly a Linux would have a very large footprint without providing > any real help. RTEMS might be just what you need. Kernel, basic OS calls for scheduling, queues, etc. It's nice when you decide you want threading to not have to graft it into a "big loop no-OS" style program. You can use native calls or POSIX style (I like POSIX, because I can develop on Linux and just recompile for the RTEMS target). There's all the usual GDB support as well. And about the processor selection, the trade-off that > Javier exposes are the same I'm confronting. Both are open-sourced and > well supported, and in one side the LM32 is smaller, in the other the > LEON3 has more capabilities that can be implemented or not (like MMU or > FPU, and better multi-core support, although not currently needed in my > project). I probably will take the LEON3 road, but also because it is > more popular in my current field, but for now I usually do not need the > FT version since I'm more related with GSEs. And that's good because the FT version costs money, but the regular old LEON2 and LEON3 are free, and pretty bulletproof by now. >> >> >> ESA has several rigorously verified flight qualified versions of RTEMS >> (in Portugal and Austria, as I recall) >> > Yes, this is one of the reasons to gain experience in that road :) I > have some tendency to stay in Linux because I'm very familiarized with > it in the non-MMU implementations (for Blackfin) and also with MMU - and > I've found that for a small embedded system, to have the MMU is not so > important, even sometimes it is a drawback. Device drivers are easy to write for RTEMS, and it has VERY fast ISRs. That's probably one of the big advantages.. It's a small footprint, stripped down RTOS, but because you can work with POSIX API calls, you can do most of your development in Linux (particuarly things like calibration interfaces and computational stuff) and then it ports very easily when you move it to RTEMS on the target.
JL
Jim Lux
Sat, Apr 7, 2012 10:29 PM

On 4/7/12 8:57 AM, Chris Albertson wrote:

If you are looking for free soft core CPUs for use in an FPGA then look here:
http://opencores.org/projects
Look under "processors" for many CPU cores.  They also have some
Eithernet controllers you'd need.

Like all things opencores/sourceforge/etc  you need to examine whats out
there...

We've used a SDRAM controller from opencores (and modified it for our
puproses) and it works pretty well.  Some other stuff, maybe not so
finished and ready for use.

It all depends on provenance....

To blow our horn a bit, we've got some useful building blocks available
for free.. We are targeting Xilinx Virtex II, but they're designed to be
pretty generic Verilog for any target.

If you need a 64 bit timer core with a bunch of latches and a
programmable pulse generator, let me know.  We've got one at JPL we're
happy to distribute (for free).

Goddard Space FLight Center has a variety of SpaceWire cores (in VHDL),
and we've got a Verilog wrapper for it at JPL.

Simple cores to do things like record samples from an ADC into a giant
SDRAM buffer or play back samples from SDRAM into a DAC, we've also got.
You want that digital oscilloscope or ARB with a 10s of MegaSample
buffer.. we've got it.

Gaisler has a lot of useful, well debugged, cores for free.. Ethernet,
RAM controlkers, various other peripherals.

On 4/7/12 8:57 AM, Chris Albertson wrote: > If you are looking for free soft core CPUs for use in an FPGA then look here: > http://opencores.org/projects > Look under "processors" for many CPU cores. They also have some > Eithernet controllers you'd need. > > Like all things opencores/sourceforge/etc you need to examine whats out there... We've used a SDRAM controller from opencores (and modified it for our puproses) and it works pretty well. Some other stuff, maybe not so finished and ready for use. It all depends on provenance.... To blow our horn a bit, we've got some useful building blocks available for free.. We are targeting Xilinx Virtex II, but they're designed to be pretty generic Verilog for any target. If you need a 64 bit timer core with a bunch of latches and a programmable pulse generator, let me know. We've got one at JPL we're happy to distribute (for free). Goddard Space FLight Center has a variety of SpaceWire cores (in VHDL), and we've got a Verilog wrapper for it at JPL. Simple cores to do things like record samples from an ADC into a giant SDRAM buffer or play back samples from SDRAM into a DAC, we've also got. You want that digital oscilloscope or ARB with a 10s of MegaSample buffer.. we've got it. Gaisler has a lot of useful, well debugged, cores for free.. Ethernet, RAM controlkers, various other peripherals.
JH
Javier Herrero
Sat, Apr 7, 2012 10:35 PM

El 08/04/2012 00:21, Jim Lux escribió:

On 4/7/12 10:08 AM, Javier Herrero wrote:

El 07/04/2012 16:02, Jim Lux escribió:

RTEMS might be just what you need.  Kernel, basic OS calls for
scheduling, queues, etc. It's nice when you decide you want threading
to not have to graft it into a "big loop no-OS" style program.

You can use native calls or POSIX style (I like POSIX, because I can
develop on Linux and just recompile for the RTEMS target).

There's all the usual GDB support as well.

Yes, it is starting to seem that would fit my needs for this project
very nicely.

Device drivers are easy to write for RTEMS, and it has VERY fast ISRs.
That's probably one of the big advantages..

I would say that one of the most important for me. Sometimes I've dealed
with Xenomai/Adeos and uClinux when Linux latencies and worse, latency
uncertainities, but things then start to be a bit complicated...

It's a small footprint, stripped down RTOS, but because you can work
with POSIX API calls, you can do most of your development in Linux
(particuarly things like calibration interfaces and computational
stuff) and then it ports very easily when you move it to RTEMS on the
target.

Looks very good for several of my usual applications, where bare metal
sometimes requires too much work, and Linux requires too much footprint
:) Also, to support POSIX style is a great advantage.

I've had a quick look around www.rtems.org, and I see that it is also
ported to Blackfin, so it will also fit my usual ADSP-BF532 based
platform. I hope that the learning curve will not be too steep :)

Thanks for the info. Best regards,

Javier

El 08/04/2012 00:21, Jim Lux escribió: > On 4/7/12 10:08 AM, Javier Herrero wrote: >> El 07/04/2012 16:02, Jim Lux escribió: > > > RTEMS might be just what you need. Kernel, basic OS calls for > scheduling, queues, etc. It's nice when you decide you want threading > to not have to graft it into a "big loop no-OS" style program. > > You can use native calls or POSIX style (I like POSIX, because I can > develop on Linux and just recompile for the RTEMS target). > > There's all the usual GDB support as well. > > Yes, it is starting to seem that would fit my needs for this project very nicely. > > > > Device drivers are easy to write for RTEMS, and it has VERY fast ISRs. > That's probably one of the big advantages.. I would say that one of the most important for me. Sometimes I've dealed with Xenomai/Adeos and uClinux when Linux latencies and worse, latency uncertainities, but things then start to be a bit complicated... > > It's a small footprint, stripped down RTOS, but because you can work > with POSIX API calls, you can do most of your development in Linux > (particuarly things like calibration interfaces and computational > stuff) and then it ports very easily when you move it to RTEMS on the > target. > Looks very good for several of my usual applications, where bare metal sometimes requires too much work, and Linux requires too much footprint :) Also, to support POSIX style is a great advantage. I've had a quick look around www.rtems.org, and I see that it is also ported to Blackfin, so it will also fit my usual ADSP-BF532 based platform. I hope that the learning curve will not be too steep :) Thanks for the info. Best regards, Javier
JH
Javier Herrero
Sat, Apr 7, 2012 10:46 PM

Hi, Jim,

I will ask off-list to reduce noise... :)

El 08/04/2012 00:29, Jim Lux escribió:

If you need a 64 bit timer core with a bunch of latches and a
programmable pulse generator, let me know.  We've got one at JPL we're
happy to distribute (for free).

I take good note. This is the kind of things that comes handy from time
to time :)

Goddard Space FLight Center has a variety of SpaceWire cores (in
VHDL), and we've got a Verilog wrapper for it at JPL.

GSF SpW cores are available for free? Or are even available for non-US?
One of the things that I plan to do sometime is an SpW implementation,
mainly for instrument EGSEs - probably will try in the near future if
one of my customers wins a project and contracts us the EGSE. We are
currently using SpW cards from Dynamic Engineering, but sincerely I'm
not too happy with their support. For internal use, we have a Star
Dundee SpW USB brick, but I find that SpW boards from Star Dundee are
expensive and only with 3 ports - and we usually need 4 (you know,
nominal and redundant I/Fs to both nominal and redundant instruments :) ).

There are some "free" SpW ESA cores. The fun part is that they are free
except for a 5000 EUR management fee...

Best regards,

Javier

--

Javier Herrero
Chief Technology Officer                  EMAIL: jherrero@hvsistemas.com
HV Sistemas S.L.                          PHONE:        +34 949 336 806
Los Charcones, 17                        FAX:          +34 949 336 792
19170 El Casar - Guadalajara - Spain      WEB: http://www.hvsistemas.com

Hi, Jim, I will ask off-list to reduce noise... :) El 08/04/2012 00:29, Jim Lux escribió: > > > > If you need a 64 bit timer core with a bunch of latches and a > programmable pulse generator, let me know. We've got one at JPL we're > happy to distribute (for free). > I take good note. This is the kind of things that comes handy from time to time :) > Goddard Space FLight Center has a variety of SpaceWire cores (in > VHDL), and we've got a Verilog wrapper for it at JPL. GSF SpW cores are available for free? Or are even available for non-US? One of the things that I plan to do sometime is an SpW implementation, mainly for instrument EGSEs - probably will try in the near future if one of my customers wins a project and contracts us the EGSE. We are currently using SpW cards from Dynamic Engineering, but sincerely I'm not too happy with their support. For internal use, we have a Star Dundee SpW USB brick, but I find that SpW boards from Star Dundee are expensive and only with 3 ports - and we usually need 4 (you know, nominal and redundant I/Fs to both nominal and redundant instruments :) ). There are some "free" SpW ESA cores. The fun part is that they are free except for a 5000 EUR management fee... Best regards, Javier -- ------------------------------------------------------------------------ Javier Herrero Chief Technology Officer EMAIL: jherrero@hvsistemas.com HV Sistemas S.L. PHONE: +34 949 336 806 Los Charcones, 17 FAX: +34 949 336 792 19170 El Casar - Guadalajara - Spain WEB: http://www.hvsistemas.com