JL
Jim Lux
Mon, Aug 6, 2012 3:34 PM
what would be useful is to have some sort of "plotting engine" that is a
canned webpage (or stored locally on the user/client computer) that can
ingest fairly raw data from a URL..
something, conceptually, like this:
<BODY>
*invocation of plotting engine*
data value 1
data value 2
data value 3
</BODY>
that way, a relatively dumb controller (think arduino-ish) could talk to
the instrument and build a web page on the fly without having to do much
formatting. The java/javascript/whathaveyou would do all the plotting
work on the client side (where, presumably, they have a display and some
computational horsepower to drive it)
A low end microcontroller has no problem serving readonly pages from
flash/SD, it just has a tough time doing graphics.
And, if you wanted the raw data, you serve up a page called "raw.html"
or something that just has the raw data.
what would be useful is to have some sort of "plotting engine" that is a
canned webpage (or stored locally on the user/client computer) that can
ingest fairly raw data from a URL..
something, conceptually, like this:
<BODY>
*invocation of plotting engine*
data value 1
data value 2
data value 3
</BODY>
that way, a relatively dumb controller (think arduino-ish) could talk to
the instrument and build a web page on the fly without having to do much
formatting. The java/javascript/whathaveyou would do all the plotting
work on the client side (where, presumably, they have a display and some
computational horsepower to drive it)
A low end microcontroller has no problem serving readonly pages from
flash/SD, it just has a tough time doing graphics.
And, if you wanted the raw data, you serve up a page called "raw.html"
or something that just has the raw data.
JA
John Ackermann N8UR
Mon, Aug 6, 2012 4:57 PM
I think there are now a couple of threads going on about this topic,
which I started by a clumsy attempt to use the "WIDTH" and "HEIGHT"
attributes in HTML. :-)
For what it's worth, I usually scale web graphics to no larger than 750
pixels horizontal or 550 pixels vertical. That goes back to the days of
lower resolution monitors, but still works well with the page layout I
use at febo.com.
The challenge in this case was that using the default settings in John's
TimeLab program, I'm getting plots that are about 1350 pixels wide and
(as PNGs) are ~130kB in size. When down-sampled, it can become
difficult to read the fine data. The best way to handle that, I think,
is to create a scaled version of the image and use that as a link to the
full-size version. Something like:
<A HREF="image.big"><IMG SRC="image.small"></A>
But that's extra work that I haven't gotten around to automating yet, so
I thought I'd try using the HTML size options:
<IMG SRC="image.big" WIDTH=50% HEIGHT=50%>
The viewer can then right-click on the image and via the "view image" or
similar menu open up the full-sized version for the fine detail. That
worked on my browser and monitor, but apparently not on some other
combinations. So, it's back to the drawing board.
Moving to the second thread on plot generation generally, apart from
TimeLab I do most of my data capture and analysis in Linux. I typically
break the two into separate pieces:
-
A single-purpose program (usually written in Perl because I make
slightly fewer errors with it than other languages) that talks via GPIB
or serial port and outputs a data file with typically MJD and phase or
frequency information.
-
A graphing tool that reads the data file. For this, I'm quite fond
of a program called "Grace" (http://plasma-gate.weizmann.ac.il/Grace/)
that provides a WYSIWIG graphics interface and saves plot information in
an ASCII format that's pretty easy to muck around with. Grace is
packaged with Debian-based Linux distributions; I don't know if there's
a Windows version available.
I've also done some automatic plot generation to go from data file to
regularly updated web page. This involves some fairly ugly text
processing taking advantage of Grace's batch mode, but the result is a
tool that will read the data file, do whatever statistics are desired,
combine with the Grace command file, run Grace in batch mode, and create
an output PNG file that's uploaded to the web. It's actually fairly
easy to do once you figure out the appropriate black magic...
John
On 8/6/2012 11:34 AM, Jim Lux wrote:
what would be useful is to have some sort of "plotting engine" that is a
canned webpage (or stored locally on the user/client computer) that can
ingest fairly raw data from a URL..
something, conceptually, like this:
<BODY>
*invocation of plotting engine*
data value 1
data value 2
data value 3
</BODY>
that way, a relatively dumb controller (think arduino-ish) could talk to
the instrument and build a web page on the fly without having to do much
formatting. The java/javascript/whathaveyou would do all the plotting
work on the client side (where, presumably, they have a display and some
computational horsepower to drive it)
A low end microcontroller has no problem serving readonly pages from
flash/SD, it just has a tough time doing graphics.
And, if you wanted the raw data, you serve up a page called "raw.html"
or something that just has the raw data.
time-nuts mailing list -- time-nuts@febo.com
To unsubscribe, go to
https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
and follow the instructions there.
I think there are now a couple of threads going on about this topic,
which I started by a clumsy attempt to use the "WIDTH" and "HEIGHT"
attributes in HTML. :-)
For what it's worth, I usually scale web graphics to no larger than 750
pixels horizontal or 550 pixels vertical. That goes back to the days of
lower resolution monitors, but still works well with the page layout I
use at febo.com.
The challenge in this case was that using the default settings in John's
TimeLab program, I'm getting plots that are about 1350 pixels wide and
(as PNGs) are ~130kB in size. When down-sampled, it can become
difficult to read the fine data. The best way to handle that, I think,
is to create a scaled version of the image and use that as a link to the
full-size version. Something like:
<A HREF="image.big"><IMG SRC="image.small"></A>
But that's extra work that I haven't gotten around to automating yet, so
I thought I'd try using the HTML size options:
<IMG SRC="image.big" WIDTH=50% HEIGHT=50%>
The viewer can then right-click on the image and via the "view image" or
similar menu open up the full-sized version for the fine detail. That
worked on my browser and monitor, but apparently not on some other
combinations. So, it's back to the drawing board.
Moving to the second thread on plot generation generally, apart from
TimeLab I do most of my data capture and analysis in Linux. I typically
break the two into separate pieces:
1. A single-purpose program (usually written in Perl because I make
slightly fewer errors with it than other languages) that talks via GPIB
or serial port and outputs a data file with typically MJD and phase or
frequency information.
2. A graphing tool that reads the data file. For this, I'm quite fond
of a program called "Grace" (http://plasma-gate.weizmann.ac.il/Grace/)
that provides a WYSIWIG graphics interface and saves plot information in
an ASCII format that's pretty easy to muck around with. Grace is
packaged with Debian-based Linux distributions; I don't know if there's
a Windows version available.
I've also done some automatic plot generation to go from data file to
regularly updated web page. This involves some fairly ugly text
processing taking advantage of Grace's batch mode, but the result is a
tool that will read the data file, do whatever statistics are desired,
combine with the Grace command file, run Grace in batch mode, and create
an output PNG file that's uploaded to the web. It's actually fairly
easy to do once you figure out the appropriate black magic...
John
----
On 8/6/2012 11:34 AM, Jim Lux wrote:
> what would be useful is to have some sort of "plotting engine" that is a
> canned webpage (or stored locally on the user/client computer) that can
> ingest fairly raw data from a URL..
>
> something, conceptually, like this:
>
>
> <BODY>
> *invocation of plotting engine*
>
> data value 1
> data value 2
> data value 3
>
> </BODY>
>
> that way, a relatively dumb controller (think arduino-ish) could talk to
> the instrument and build a web page on the fly without having to do much
> formatting. The java/javascript/whathaveyou would do all the plotting
> work on the client side (where, presumably, they have a display and some
> computational horsepower to drive it)
>
> A low end microcontroller has no problem serving readonly pages from
> flash/SD, it just has a tough time doing graphics.
>
>
> And, if you wanted the raw data, you serve up a page called "raw.html"
> or something that just has the raw data.
>
> _______________________________________________
> time-nuts mailing list -- time-nuts@febo.com
> To unsubscribe, go to
> https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
> and follow the instructions there.
PG
Pablo Garaizar Sagarminaga
Mon, Aug 6, 2012 5:22 PM
Hi there,
On Mon, 06 Aug 2012 12:57:00 -0400
John Ackermann N8UR jra@febo.com wrote:
<A HREF="image.big"><IMG SRC="image.small"></A>
But that's extra work that I haven't gotten around to automating yet,
so I thought I'd try using the HTML size options:
<IMG SRC="image.big" WIDTH=50% HEIGHT=50%>
It's easy to resize pictures automatically if you are a Linux user
using imagemagick's convert tool:
- Resize one pic:
convert pic.jpg -resize 50% small-pic.jpg
- Resize many pics in one directory:
for F in *jpg; do convert $F -resize 50% small-$F; done
Hope it helps O:-)
--
Pablo Garaizar Sagarminaga
Universidad de Deusto
Avda. de las Universidades 24
48007 Bilbao - Spain
Phone: +34-94-4139000 Ext 2512
Fax: +34-94-4139101
Hi there,
On Mon, 06 Aug 2012 12:57:00 -0400
John Ackermann N8UR <jra@febo.com> wrote:
> <A HREF="image.big"><IMG SRC="image.small"></A>
> But that's extra work that I haven't gotten around to automating yet,
> so I thought I'd try using the HTML size options:
> <IMG SRC="image.big" WIDTH=50% HEIGHT=50%>
It's easy to resize pictures automatically if you are a Linux user
using imagemagick's convert tool:
1) Resize one pic:
convert pic.jpg -resize 50% small-pic.jpg
2) Resize many pics in one directory:
for F in *jpg; do convert $F -resize 50% small-$F; done
Hope it helps O:-)
--
Pablo Garaizar Sagarminaga
Universidad de Deusto
Avda. de las Universidades 24
48007 Bilbao - Spain
Phone: +34-94-4139000 Ext 2512
Fax: +34-94-4139101
CA
Chris Albertson
Mon, Aug 6, 2012 5:43 PM
what would be useful is to have some sort of "plotting engine" that is a
canned webpage (or stored locally on the user/client computer) that can
ingest fairly raw data from a URL..
something, conceptually, like this:
<BODY>
*invocation of plotting engine*
data value 1
data value 2
data value 3
</BODY>
What you are saying is that the data needs to be rendered to a graphic
locally. I think the simplest way is to create vector based plots on the
server (I've used GNU Plot inside a CGI script) but there is a system to
pretty much what you are asking for. http://code.google.com/p/flot/
Chris Albertson
Redondo Beach, California
On Mon, Aug 6, 2012 at 8:34 AM, Jim Lux <jimlux@earthlink.net> wrote:
> what would be useful is to have some sort of "plotting engine" that is a
> canned webpage (or stored locally on the user/client computer) that can
> ingest fairly raw data from a URL..
>
> something, conceptually, like this:
>
>
> <BODY>
> *invocation of plotting engine*
>
> data value 1
> data value 2
> data value 3
>
> </BODY>
>
>
What you are saying is that the data needs to be rendered to a graphic
locally. I think the simplest way is to create vector based plots on the
server (I've used GNU Plot inside a CGI script) but there is a system to
pretty much what you are asking for. http://code.google.com/p/flot/
Chris Albertson
Redondo Beach, California
PG
Pablo Garaizar Sagarminaga
Mon, Aug 6, 2012 5:59 PM
What you are saying is that the data needs to be rendered to a graphic
locally. I think the simplest way is to create vector based plots
on the server (I've used GNU Plot inside a CGI script) but there is a
system to pretty much what you are asking for.
http://code.google.com/p/flot/
Indeed, I use flot in AllanJS [1], a JavaScript library to calculate
Allan deviation estimators based on previous work from this list (Tom
Van Baak, Magnus Danielson, William J. Riley, John Miles).
[1] http://txipi.github.com/AllanJS/
--
Pablo Garaizar Sagarminaga
Universidad de Deusto
Avda. de las Universidades 24
48007 Bilbao - Spain
Phone: +34-94-4139000 Ext 2512
Fax: +34-94-4139101
Hi there,
El Mon, 6 Aug 2012 10:43:50 -0700
Chris Albertson <albertson.chris@gmail.com> comentaba:
> What you are saying is that the data needs to be rendered to a graphic
> locally. I think the simplest way is to create vector based plots
> on the server (I've used GNU Plot inside a CGI script) but there is a
> system to pretty much what you are asking for.
> http://code.google.com/p/flot/
Indeed, I use flot in AllanJS [1], a JavaScript library to calculate
Allan deviation estimators based on previous work from this list (Tom
Van Baak, Magnus Danielson, William J. Riley, John Miles).
[1] http://txipi.github.com/AllanJS/
--
Pablo Garaizar Sagarminaga
Universidad de Deusto
Avda. de las Universidades 24
48007 Bilbao - Spain
Phone: +34-94-4139000 Ext 2512
Fax: +34-94-4139101
MD
Magnus Danielson
Mon, Aug 6, 2012 7:44 PM
Hi Pablo,
On 08/06/2012 07:59 PM, Pablo Garaizar Sagarminaga wrote:
What you are saying is that the data needs to be rendered to a graphic
locally. I think the simplest way is to create vector based plots
on the server (I've used GNU Plot inside a CGI script) but there is a
system to pretty much what you are asking for.
http://code.google.com/p/flot/
Indeed, I use flot in AllanJS [1], a JavaScript library to calculate
Allan deviation estimators based on previous work from this list (Tom
Van Baak, Magnus Danielson, William J. Riley, John Miles).
[1] http://txipi.github.com/AllanJS/
I feel honoured to be part of that list of names.
I haven't tried AllanJS yet, but Pablo has informed me of it's existence
before.
I've done my share of work implementing algorithms, but others are
better at getting a meaningful user interface done.
Cheers,
Magnus
Hi Pablo,
On 08/06/2012 07:59 PM, Pablo Garaizar Sagarminaga wrote:
> Hi there,
>
> El Mon, 6 Aug 2012 10:43:50 -0700
> Chris Albertson<albertson.chris@gmail.com> comentaba:
>
>> What you are saying is that the data needs to be rendered to a graphic
>> locally. I think the simplest way is to create vector based plots
>> on the server (I've used GNU Plot inside a CGI script) but there is a
>> system to pretty much what you are asking for.
>> http://code.google.com/p/flot/
>
> Indeed, I use flot in AllanJS [1], a JavaScript library to calculate
> Allan deviation estimators based on previous work from this list (Tom
> Van Baak, Magnus Danielson, William J. Riley, John Miles).
>
> [1] http://txipi.github.com/AllanJS/
>
I feel honoured to be part of that list of names.
I haven't tried AllanJS yet, but Pablo has informed me of it's existence
before.
I've done my share of work implementing algorithms, but others are
better at getting a meaningful user interface done.
Cheers,
Magnus
MS
Mike S
Tue, Aug 7, 2012 11:25 AM
On 8/6/2012 12:57 PM, John Ackermann N8UR wrote:
<IMG SRC="image.big" WIDTH=50% HEIGHT=50%>
The viewer can then right-click on the image and via the "view image" or
similar menu open up the full-sized version for the fine detail. That
worked on my browser and monitor, but apparently not on some other
combinations. So, it's back to the drawing board.
Try <IMG SRC="image.big" WIDTH="675" HEIGHT="506">
(assuming a 1350x1012 original). Specifying an exact size is preferred,
since it makes page rendering faster - the browser can do the layout
before having to download the image to determine the size.
On 8/6/2012 12:57 PM, John Ackermann N8UR wrote:
> <IMG SRC="image.big" WIDTH=50% HEIGHT=50%>
>
> The viewer can then right-click on the image and via the "view image" or
> similar menu open up the full-sized version for the fine detail. That
> worked on my browser and monitor, but apparently not on some other
> combinations. So, it's back to the drawing board.
Try <IMG SRC="image.big" WIDTH="675" HEIGHT="506">
(assuming a 1350x1012 original). Specifying an exact size is preferred,
since it makes page rendering faster - the browser can do the layout
before having to download the image to determine the size.
JL
Jim Lux
Tue, Aug 7, 2012 1:45 PM
On 8/6/12 10:43 AM, Chris Albertson wrote:
what would be useful is to have some sort of "plotting engine" that is a
canned webpage (or stored locally on the user/client computer) that can
ingest fairly raw data from a URL..
something, conceptually, like this:
<BODY>
*invocation of plotting engine*
data value 1
data value 2
data value 3
</BODY>
What you are saying is that the data needs to be rendered to a graphic
locally. I think the simplest way is to create vector based plots on the
server (I've used GNU Plot inside a CGI script) but there is a system to
pretty much what you are asking for. http://code.google.com/p/flot/
Somehow, I don't think running Gnuplot on a Arduino is feasible. ABout
all it can do is respond to the HTTP "Get" with the right page.
That's why I want to push the hard work of doing the rendering onto the
client (i.e. the browser). The comm link is fast, and an SD card on the
Arduino can hold tons of stuff, so serving a page containing 100kbytes
of Java or similar isn't a problem. But doing any computation is
probably not in the picture.
On 8/6/12 10:43 AM, Chris Albertson wrote:
> On Mon, Aug 6, 2012 at 8:34 AM, Jim Lux <jimlux@earthlink.net> wrote:
>
>> what would be useful is to have some sort of "plotting engine" that is a
>> canned webpage (or stored locally on the user/client computer) that can
>> ingest fairly raw data from a URL..
>>
>> something, conceptually, like this:
>>
>>
>> <BODY>
>> *invocation of plotting engine*
>>
>> data value 1
>> data value 2
>> data value 3
>>
>> </BODY>
>>
>>
> What you are saying is that the data needs to be rendered to a graphic
> locally. I think the simplest way is to create vector based plots on the
> server (I've used GNU Plot inside a CGI script) but there is a system to
> pretty much what you are asking for. http://code.google.com/p/flot/
>
Somehow, I don't think running Gnuplot on a Arduino is feasible. ABout
all it can do is respond to the HTTP "Get" with the right page.
That's why I want to push the hard work of doing the rendering onto the
client (i.e. the browser). The comm link is fast, and an SD card on the
Arduino can hold tons of stuff, so serving a page containing 100kbytes
of Java or similar isn't a problem. But doing any computation is
probably not in the picture.
S
shalimr9 (at) gmail.com
Tue, Aug 7, 2012 2:04 PM
Regarding plotting data files through a web page, some time ago I wrote a
small utility to do just that. We have a corona tester here at my workplace
that generates binary files with the corona data. These are not easily
plotted from a standard tool because of the proprietary format.
Instead of writing a Visual something program that would have to be
installed on all the machines that might need to display the data, I wrote
a php app that I loaded on the local Linux server.
I also copied it to my ko4bb.com site at http://www.ko4bb.com/graph
You can try it with this file (which you need to download to your local
hard drive first, so that you can upload it to the tool :)
http://www.ko4bb.com/graph/006.DAT
This software could be easily modified to print any standard file format
(comma delimited or else) and to adjust the width/height of the picture to
your liking. Save the picture size in a cookie so that you do not need to
enter it each time and you are done :)
The source code is there:
http://www.ko4bb.com/graph/index.php.txt
Didier KO4BB
On , John Ackermann N8UR jra@febo.com wrote:
I think there are now a couple of threads going on about this topic,
which I started by a clumsy attempt to use the "WIDTH" and "HEIGHT"
attributes in HTML. :-)
For what it's worth, I usually scale web graphics to no larger than 750
pixels horizontal or 550 pixels vertical. That goes back to the days of
lower resolution monitors, but still works well with the page layout I
use at febo.com.
The challenge in this case was that using the default settings in John's
TimeLab program, I'm getting plots that are about 1350 pixels wide and
(as PNGs) are ~130kB in size. When down-sampled, it can become difficult
to read the fine data. The best way to handle that, I think, is to create
a scaled version of the image and use that as a link to the full-size
version. Something like:
But that's extra work that I haven't gotten around to automating yet, so
I thought I'd try using the HTML size options:
The viewer can then right-click on the image and via the "view image" or
similar menu open up the full-sized version for the fine detail. That
worked on my browser and monitor, but apparently not on some other
combinations. So, it's back to the drawing board.
Moving to the second thread on plot generation generally, apart from
TimeLab I do most of my data capture and analysis in Linux. I typically
break the two into separate pieces:
- A single-purpose program (usually written in Perl because I make
slightly fewer errors with it than other languages) that talks via GPIB
or serial port and outputs a data file with typically MJD and phase or
frequency information.
- A graphing tool that reads the data file. For this, I'm quite fond of
a program called "Grace" (http://plasma-gate.weizmann.ac.il/Grace/) that
provides a WYSIWIG graphics interface and saves plot information in an
ASCII format that's pretty easy to muck around with. Grace is packaged
with Debian-based Linux distributions; I don't know if there's a Windows
version available.
I've also done some automatic plot generation to go from data file to
regularly updated web page. This involves some fairly ugly text
processing taking advantage of Grace's batch mode, but the result is a
tool that will read the data file, do whatever statistics are desired,
combine with the Grace command file, run Grace in batch mode, and create
an output PNG file that's uploaded to the web. It's actually fairly easy
to do once you figure out the appropriate black magic...
On 8/6/2012 11:34 AM, Jim Lux wrote:
what would be useful is to have some sort of "plotting engine" that is a
canned webpage (or stored locally on the user/client computer) that can
ingest fairly raw data from a URL..
something, conceptually, like this:
invocation of plotting engine
that way, a relatively dumb controller (think arduino-ish) could talk to
the instrument and build a web page on the fly without having to do much
formatting. The java/javascript/whathaveyou would do all the plotting
work on the client side (where, presumably, they have a display and some
computational horsepower to drive it)
A low end microcontroller has no problem serving readonly pages from
flash/SD, it just has a tough time doing graphics.
And, if you wanted the raw data, you serve up a page called "raw.html"
or something that just has the raw data.
and follow the instructions there.
and follow the instructions there.
Regarding plotting data files through a web page, some time ago I wrote a
small utility to do just that. We have a corona tester here at my workplace
that generates binary files with the corona data. These are not easily
plotted from a standard tool because of the proprietary format.
Instead of writing a Visual something program that would have to be
installed on all the machines that might need to display the data, I wrote
a php app that I loaded on the local Linux server.
I also copied it to my ko4bb.com site at http://www.ko4bb.com/graph
You can try it with this file (which you need to download to your local
hard drive first, so that you can upload it to the tool :)
http://www.ko4bb.com/graph/006.DAT
This software could be easily modified to print any standard file format
(comma delimited or else) and to adjust the width/height of the picture to
your liking. Save the picture size in a cookie so that you do not need to
enter it each time and you are done :)
The source code is there:
http://www.ko4bb.com/graph/index.php.txt
Didier KO4BB
On , John Ackermann N8UR <jra@febo.com> wrote:
> I think there are now a couple of threads going on about this topic,
> which I started by a clumsy attempt to use the "WIDTH" and "HEIGHT"
> attributes in HTML. :-)
> For what it's worth, I usually scale web graphics to no larger than 750
> pixels horizontal or 550 pixels vertical. That goes back to the days of
> lower resolution monitors, but still works well with the page layout I
> use at febo.com.
> The challenge in this case was that using the default settings in John's
> TimeLab program, I'm getting plots that are about 1350 pixels wide and
> (as PNGs) are ~130kB in size. When down-sampled, it can become difficult
> to read the fine data. The best way to handle that, I think, is to create
> a scaled version of the image and use that as a link to the full-size
> version. Something like:
> But that's extra work that I haven't gotten around to automating yet, so
> I thought I'd try using the HTML size options:
> The viewer can then right-click on the image and via the "view image" or
> similar menu open up the full-sized version for the fine detail. That
> worked on my browser and monitor, but apparently not on some other
> combinations. So, it's back to the drawing board.
> Moving to the second thread on plot generation generally, apart from
> TimeLab I do most of my data capture and analysis in Linux. I typically
> break the two into separate pieces:
> 1. A single-purpose program (usually written in Perl because I make
> slightly fewer errors with it than other languages) that talks via GPIB
> or serial port and outputs a data file with typically MJD and phase or
> frequency information.
> 2. A graphing tool that reads the data file. For this, I'm quite fond of
> a program called "Grace" (http://plasma-gate.weizmann.ac.il/Grace/) that
> provides a WYSIWIG graphics interface and saves plot information in an
> ASCII format that's pretty easy to muck around with. Grace is packaged
> with Debian-based Linux distributions; I don't know if there's a Windows
> version available.
> I've also done some automatic plot generation to go from data file to
> regularly updated web page. This involves some fairly ugly text
> processing taking advantage of Grace's batch mode, but the result is a
> tool that will read the data file, do whatever statistics are desired,
> combine with the Grace command file, run Grace in batch mode, and create
> an output PNG file that's uploaded to the web. It's actually fairly easy
> to do once you figure out the appropriate black magic...
> John
> ----
> On 8/6/2012 11:34 AM, Jim Lux wrote:
> what would be useful is to have some sort of "plotting engine" that is a
> canned webpage (or stored locally on the user/client computer) that can
> ingest fairly raw data from a URL..
> something, conceptually, like this:
> *invocation of plotting engine*
> data value 1
> data value 2
> data value 3
> that way, a relatively dumb controller (think arduino-ish) could talk to
> the instrument and build a web page on the fly without having to do much
> formatting. The java/javascript/whathaveyou would do all the plotting
> work on the client side (where, presumably, they have a display and some
> computational horsepower to drive it)
> A low end microcontroller has no problem serving readonly pages from
> flash/SD, it just has a tough time doing graphics.
> And, if you wanted the raw data, you serve up a page called "raw.html"
> or something that just has the raw data.
> _______________________________________________
> time-nuts mailing list -- time-nuts@febo.com
> To unsubscribe, go to
> https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
> and follow the instructions there.
> _______________________________________________
> time-nuts mailing list -- time-nuts@febo.com
> To unsubscribe, go to
> https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
> and follow the instructions there.
BB
Bob Bownes
Tue, Aug 7, 2012 2:15 PM
Regarding plotting data files through a web page, some time ago I wrote a
small utility to do just that. We have a corona tester here at my workplace
that generates binary files with the corona data. These are not easily
plotted from a standard tool because of the proprietary format.
Instead of writing a Visual something program that would have to be
installed on all the machines that might need to display the data, I wrote
a php app that I loaded on the local Linux server.
I also copied it to my ko4bb.com site at http://www.ko4bb.com/graph
You can try it with this file (which you need to download to your local
hard drive first, so that you can upload it to the tool :)
http://www.ko4bb.com/graph/**006.DAT http://www.ko4bb.com/graph/006.DAT
This software could be easily modified to print any standard file format
(comma delimited or else) and to adjust the width/height of the picture to
your liking. Save the picture size in a cookie so that you do not need to
enter it each time and you are done :)
The source code is there:
http://www.ko4bb.com/graph/**index.php.txthttp://www.ko4bb.com/graph/index.php.txt
Didier KO4BB
On , John Ackermann N8UR jra@febo.com wrote:
I think there are now a couple of threads going on about this topic,
which I started by a clumsy attempt to use the "WIDTH" and "HEIGHT"
attributes in HTML. :-)
For what it's worth, I usually scale web graphics to no larger than 750
pixels horizontal or 550 pixels vertical. That goes back to the days of
lower resolution monitors, but still works well with the page layout I use
at febo.com.
The challenge in this case was that using the default settings in John's
TimeLab program, I'm getting plots that are about 1350 pixels wide and (as
PNGs) are ~130kB in size. When down-sampled, it can become difficult to
read the fine data. The best way to handle that, I think, is to create a
scaled version of the image and use that as a link to the full-size
version. Something like:
But that's extra work that I haven't gotten around to automating yet, so
I thought I'd try using the HTML size options:
The viewer can then right-click on the image and via the "view image" or
similar menu open up the full-sized version for the fine detail. That
worked on my browser and monitor, but apparently not on some other
combinations. So, it's back to the drawing board.
Moving to the second thread on plot generation generally, apart from
TimeLab I do most of my data capture and analysis in Linux. I typically
break the two into separate pieces:
- A single-purpose program (usually written in Perl because I make
slightly fewer errors with it than other languages) that talks via GPIB or
serial port and outputs a data file with typically MJD and phase or
frequency information.
- A graphing tool that reads the data file. For this, I'm quite fond of
a program called "Grace" (http://plasma-gate.weizmann.**ac.il/Grace/http://plasma-gate.weizmann.ac.il/Grace/)
that provides a WYSIWIG graphics interface and saves plot information in an
ASCII format that's pretty easy to muck around with. Grace is packaged with
Debian-based Linux distributions; I don't know if there's a Windows version
available.
I've also done some automatic plot generation to go from data file to
regularly updated web page. This involves some fairly ugly text processing
taking advantage of Grace's batch mode, but the result is a tool that will
read the data file, do whatever statistics are desired, combine with the
Grace command file, run Grace in batch mode, and create an output PNG file
that's uploaded to the web. It's actually fairly easy to do once you figure
out the appropriate black magic...
On 8/6/2012 11:34 AM, Jim Lux wrote:
what would be useful is to have some sort of "plotting engine" that is a
canned webpage (or stored locally on the user/client computer) that can
ingest fairly raw data from a URL..
something, conceptually, like this:
invocation of plotting engine
that way, a relatively dumb controller (think arduino-ish) could talk to
the instrument and build a web page on the fly without having to do much
formatting. The java/javascript/whathaveyou would do all the plotting
work on the client side (where, presumably, they have a display and some
computational horsepower to drive it)
A low end microcontroller has no problem serving readonly pages from
flash/SD, it just has a tough time doing graphics.
And, if you wanted the raw data, you serve up a page called "raw.html"
or something that just has the raw data.
and follow the instructions there.
and follow the instructions there.
If you are looking to graph long running data and provide rolled up
summaries, the combination of MRTG and RRDtool is pretty hard to beat. Can
work with pretty much anything on the back end. And it's the industry
standard for network monitoring.
http://oss.oetiker.ch/mrtg/doc/mrtg.en.html
Or some examples:
http://www.switch.ch/network/operation/statistics/geant2.html
On Tue, Aug 7, 2012 at 10:04 AM, <shalimr9@gmail.com> wrote:
> Regarding plotting data files through a web page, some time ago I wrote a
> small utility to do just that. We have a corona tester here at my workplace
> that generates binary files with the corona data. These are not easily
> plotted from a standard tool because of the proprietary format.
>
> Instead of writing a Visual something program that would have to be
> installed on all the machines that might need to display the data, I wrote
> a php app that I loaded on the local Linux server.
> I also copied it to my ko4bb.com site at http://www.ko4bb.com/graph
>
> You can try it with this file (which you need to download to your local
> hard drive first, so that you can upload it to the tool :)
>
> http://www.ko4bb.com/graph/**006.DAT <http://www.ko4bb.com/graph/006.DAT>
>
> This software could be easily modified to print any standard file format
> (comma delimited or else) and to adjust the width/height of the picture to
> your liking. Save the picture size in a cookie so that you do not need to
> enter it each time and you are done :)
>
> The source code is there:
>
> http://www.ko4bb.com/graph/**index.php.txt<http://www.ko4bb.com/graph/index.php.txt>
>
> Didier KO4BB
>
>
> On , John Ackermann N8UR <jra@febo.com> wrote:
>
>> I think there are now a couple of threads going on about this topic,
>> which I started by a clumsy attempt to use the "WIDTH" and "HEIGHT"
>> attributes in HTML. :-)
>>
>
>
>
> For what it's worth, I usually scale web graphics to no larger than 750
>> pixels horizontal or 550 pixels vertical. That goes back to the days of
>> lower resolution monitors, but still works well with the page layout I use
>> at febo.com.
>>
>
>
>
> The challenge in this case was that using the default settings in John's
>> TimeLab program, I'm getting plots that are about 1350 pixels wide and (as
>> PNGs) are ~130kB in size. When down-sampled, it can become difficult to
>> read the fine data. The best way to handle that, I think, is to create a
>> scaled version of the image and use that as a link to the full-size
>> version. Something like:
>>
>
>
>
>
>
>
>
> But that's extra work that I haven't gotten around to automating yet, so
>> I thought I'd try using the HTML size options:
>>
>
>
>
>
>
>
>
> The viewer can then right-click on the image and via the "view image" or
>> similar menu open up the full-sized version for the fine detail. That
>> worked on my browser and monitor, but apparently not on some other
>> combinations. So, it's back to the drawing board.
>>
>
>
>
> Moving to the second thread on plot generation generally, apart from
>> TimeLab I do most of my data capture and analysis in Linux. I typically
>> break the two into separate pieces:
>>
>
>
>
> 1. A single-purpose program (usually written in Perl because I make
>> slightly fewer errors with it than other languages) that talks via GPIB or
>> serial port and outputs a data file with typically MJD and phase or
>> frequency information.
>>
>
>
>
> 2. A graphing tool that reads the data file. For this, I'm quite fond of
>> a program called "Grace" (http://plasma-gate.weizmann.**ac.il/Grace/<http://plasma-gate.weizmann.ac.il/Grace/>)
>> that provides a WYSIWIG graphics interface and saves plot information in an
>> ASCII format that's pretty easy to muck around with. Grace is packaged with
>> Debian-based Linux distributions; I don't know if there's a Windows version
>> available.
>>
>
>
>
> I've also done some automatic plot generation to go from data file to
>> regularly updated web page. This involves some fairly ugly text processing
>> taking advantage of Grace's batch mode, but the result is a tool that will
>> read the data file, do whatever statistics are desired, combine with the
>> Grace command file, run Grace in batch mode, and create an output PNG file
>> that's uploaded to the web. It's actually fairly easy to do once you figure
>> out the appropriate black magic...
>>
>
>
>
> John
>>
>
> ----
>>
>
>
>
> On 8/6/2012 11:34 AM, Jim Lux wrote:
>>
>
>
> what would be useful is to have some sort of "plotting engine" that is a
>>
>
> canned webpage (or stored locally on the user/client computer) that can
>>
>
> ingest fairly raw data from a URL..
>>
>
>
>
> something, conceptually, like this:
>>
>
>
>
>
>
>
>
> *invocation of plotting engine*
>>
>
>
>
> data value 1
>>
>
> data value 2
>>
>
> data value 3
>>
>
>
>
>
>
>
>
> that way, a relatively dumb controller (think arduino-ish) could talk to
>>
>
> the instrument and build a web page on the fly without having to do much
>>
>
> formatting. The java/javascript/whathaveyou would do all the plotting
>>
>
> work on the client side (where, presumably, they have a display and some
>>
>
> computational horsepower to drive it)
>>
>
>
>
> A low end microcontroller has no problem serving readonly pages from
>>
>
> flash/SD, it just has a tough time doing graphics.
>>
>
>
>
>
>
> And, if you wanted the raw data, you serve up a page called "raw.html"
>>
>
> or something that just has the raw data.
>>
>
>
>
> ______________________________**_________________
>>
>
> time-nuts mailing list -- time-nuts@febo.com
>>
>
> To unsubscribe, go to
>>
>
> https://www.febo.com/cgi-bin/**mailman/listinfo/time-nuts<https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts>
>>
>
> and follow the instructions there.
>>
>
>
>
>
>
>
> ______________________________**_________________
>>
>
> time-nuts mailing list -- time-nuts@febo.com
>>
>
> To unsubscribe, go to https://www.febo.com/cgi-bin/**
>> mailman/listinfo/time-nuts<https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts>
>>
>
> and follow the instructions there.
>>
>
>
> ______________________________**_________________
> time-nuts mailing list -- time-nuts@febo.com
> To unsubscribe, go to https://www.febo.com/cgi-bin/**
> mailman/listinfo/time-nuts<https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts>
> and follow the instructions there.
>
CA
Chris Albertson
Tue, Aug 7, 2012 3:56 PM
On 8/6/12 10:43 AM, Chris Albertson wrote:
On Mon, Aug 6, 2012 at 8:34 AM, Jim Lux jimlux@earthlink.net wrote:
what would be useful is to have some sort of "plotting engine" that is a
canned webpage (or stored locally on the user/client computer) that can
ingest fairly raw data from a URL..
something, conceptually, like this:
<BODY>
*invocation of plotting engine*
data value 1
data value 2
data value 3
</BODY>
What you are saying is that the data needs to be rendered to a graphic
locally. I think the simplest way is to create vector based plots on the
server (I've used GNU Plot inside a CGI script) but there is a system to
pretty much what you are asking for. http://code.google.com/p/flot/
Somehow, I don't think running Gnuplot on a Arduino is feasible. ABout
all it can do is respond to the HTTP "Get" with the right page.
That's why I want to push the hard work of doing the rendering onto the
client (i.e. the browser). The comm link is fast, and an SD card on the
Arduino can hold tons of stuff, so serving a page containing 100kbytes of
Java or similar isn't a problem. But doing any computation is probably not
in the picture.
So the web server is inside an Arduino? Yes can't run GNUplot there. Why
not have the Arduino produce either Postscript or PDF? It is very easy to
draw a graph in Postscript. Postscript is just ASCII text that contains
statements like "Draw a line from (2.34,6.65) to (3.45,78.4)" You can draw
using your application's natural units, then you apply a transform to map
those to output units like inches or cm. Later the web browser or the
printer turns those draw commands into a rasterised image fro display.
Postscript is very compact. A few KB would be more than enough for a
graph. The advantage is that the output resolution is "perfect" because
your lines are drawn using natural units with as many places after the
decimal point as you like. So your users can zoom in if they want to
1000X zoom and see details, not pixels. It wil fet on any screen
Also everyone already has the display engine. PDF is very much like
Postscript. but I think Postscrip in "EPS" form is easy to create inside an
Arduino,
--
Chris Albertson
Redondo Beach, California
On Tue, Aug 7, 2012 at 6:45 AM, Jim Lux <jimlux@earthlink.net> wrote:
> On 8/6/12 10:43 AM, Chris Albertson wrote:
>
>> On Mon, Aug 6, 2012 at 8:34 AM, Jim Lux <jimlux@earthlink.net> wrote:
>>
>> what would be useful is to have some sort of "plotting engine" that is a
>>> canned webpage (or stored locally on the user/client computer) that can
>>> ingest fairly raw data from a URL..
>>>
>>> something, conceptually, like this:
>>>
>>>
>>> <BODY>
>>> *invocation of plotting engine*
>>>
>>> data value 1
>>> data value 2
>>> data value 3
>>>
>>> </BODY>
>>>
>>>
>>> What you are saying is that the data needs to be rendered to a graphic
>> locally. I think the simplest way is to create vector based plots on the
>> server (I've used GNU Plot inside a CGI script) but there is a system to
>> pretty much what you are asking for. http://code.google.com/p/flot/
>>
>>
>
> Somehow, I don't think running Gnuplot on a Arduino is feasible. ABout
> all it can do is respond to the HTTP "Get" with the right page.
>
> That's why I want to push the hard work of doing the rendering onto the
> client (i.e. the browser). The comm link is fast, and an SD card on the
> Arduino can hold tons of stuff, so serving a page containing 100kbytes of
> Java or similar isn't a problem. But doing any computation is probably not
> in the picture.
>
>
So the web server is inside an Arduino? Yes can't run GNUplot there. Why
not have the Arduino produce either Postscript or PDF? It is very easy to
draw a graph in Postscript. Postscript is just ASCII text that contains
statements like "Draw a line from (2.34,6.65) to (3.45,78.4)" You can draw
using your application's natural units, then you apply a transform to map
those to output units like inches or cm. Later the web browser or the
printer turns those draw commands into a rasterised image fro display.
Postscript is very compact. A few KB would be more than enough for a
graph. The advantage is that the output resolution is "perfect" because
your lines are drawn using natural units with as many places after the
decimal point as you like. So your users can zoom in if they want to
1000X zoom and see details, not pixels. It wil fet on any screen
Also everyone already has the display engine. PDF is very much like
Postscript. but I think Postscrip in "EPS" form is easy to create inside an
Arduino,
--
Chris Albertson
Redondo Beach, California
MC
mike cook
Tue, Aug 7, 2012 5:57 PM
Le 07/08/2012 17:56, Chris Albertson a écrit :
On 8/6/12 10:43 AM, Chris Albertson wrote:
On Mon, Aug 6, 2012 at 8:34 AM, Jim Lux jimlux@earthlink.net wrote:
what would be useful is to have some sort of "plotting engine" that is a
canned webpage (or stored locally on the user/client computer) that can
ingest fairly raw data from a URL..
<snipped>
So the web server is inside an Arduino? Yes can't run GNUplot there. Why
not have the Arduino produce either Postscript or PDF? It is very easy to
draw a graph in Postscript.
<snipped>
Also everyone already has the display engine. PDF is very much like
Postscript. but I think Postscrip in "EPS" form is easy to create inside an
Arduino,
One good reason for doing as little as possible in a micro-controller is
that they are often used to collect raw data with as little latency as
possible. Doing web serving on a single slowish core in that case is not
a good idea. I have a bunch of Soekris doing data collection and if I
start to do compute intensive tasks on one, it skews the timing, so I
use a completely independent one to do the web serving, with the data
collectors providing access to data via nfs which does not seen to
impact the timing much even though I can't guarantee that requests are
satisfied in otherwise idle time . I like the idea of client side
graphing. It probably exists already somewhere though I haven't checked
details - RGraph pops up with google.
--
Les chiens aboient, et la caravane passe.
Le 07/08/2012 17:56, Chris Albertson a écrit :
> On Tue, Aug 7, 2012 at 6:45 AM, Jim Lux <jimlux@earthlink.net> wrote:
>
>> On 8/6/12 10:43 AM, Chris Albertson wrote:
>>
>>> On Mon, Aug 6, 2012 at 8:34 AM, Jim Lux <jimlux@earthlink.net> wrote:
>>>
>>> what would be useful is to have some sort of "plotting engine" that is a
>>>> canned webpage (or stored locally on the user/client computer) that can
>>>> ingest fairly raw data from a URL..
>>>> <snipped>
> So the web server is inside an Arduino? Yes can't run GNUplot there. Why
> not have the Arduino produce either Postscript or PDF? It is very easy to
> draw a graph in Postscript.
> <snipped>
> Also everyone already has the display engine. PDF is very much like
> Postscript. but I think Postscrip in "EPS" form is easy to create inside an
> Arduino,
>
One good reason for doing as little as possible in a micro-controller is
that they are often used to collect raw data with as little latency as
possible. Doing web serving on a single slowish core in that case is not
a good idea. I have a bunch of Soekris doing data collection and if I
start to do compute intensive tasks on one, it skews the timing, so I
use a completely independent one to do the web serving, with the data
collectors providing access to data via nfs which does not seen to
impact the timing much even though I can't guarantee that requests are
satisfied in otherwise idle time . I like the idea of client side
graphing. It probably exists already somewhere though I haven't checked
details - RGraph pops up with google.
--
Les chiens aboient, et la caravane passe.
CA
Chris Albertson
Tue, Aug 7, 2012 7:22 PM
One good reason for doing as little as possible in a micro-controller is
that they are often used to collect raw data with as little latency as
possible. Doing web serving on a single slowish core in that case is not a
good idea. I have a bunch of Soekris doing data collection and if I start
to do compute intensive tasks on one, it skews the timing, so I use a
completely independent one to do the web serving, with the data collectors
providing access to data via nfs which does not seen to impact the timing
much even though I can't guarantee that requests are satisfied in otherwise
idle time . I like the idea of client side graphing. It probably exists
already somewhere though I haven't checked details - RGraph pops up with
google.
You are right about i being easier to not mix real-time and backround tasks
on a small uP. But it can be done. The way to do it is to make the
real-time task interrup driven. You can use a timer that goes off (say) 50
ties per second and in the handler you do your real-tie data collection or
poll the hardware, read sensors or whatever. Then in the background task
you run the web server. This way there is never a chance of something
like network requests slowing down the data collection.. A simple two
task "multi-tasker" is easy. It gets harder when you have more tasks and
I'd fgo with some kind of OS even it that required a larger uP.
That said. Even if it is easy I'dstill put the web server on a "real"
computer and only pass data out of the arduino. But it could be done. I
build a web server (and an FTP server) into a CCD camera and there is no
performance issue at all. Even during an exposure while reading out the
CCD chip.
As for going the graphics in the client. That really is easy now that
every client computer has a PDF viewer. Simply make your graph a PDF
document.
Chris Albertson
Redondo Beach, California
One good reason for doing as little as possible in a micro-controller is
> that they are often used to collect raw data with as little latency as
> possible. Doing web serving on a single slowish core in that case is not a
> good idea. I have a bunch of Soekris doing data collection and if I start
> to do compute intensive tasks on one, it skews the timing, so I use a
> completely independent one to do the web serving, with the data collectors
> providing access to data via nfs which does not seen to impact the timing
> much even though I can't guarantee that requests are satisfied in otherwise
> idle time . I like the idea of client side graphing. It probably exists
> already somewhere though I haven't checked details - RGraph pops up with
> google.
>
>
You are right about i being easier to not mix real-time and backround tasks
on a small uP. But it can be done. The way to do it is to make the
real-time task interrup driven. You can use a timer that goes off (say) 50
ties per second and in the handler you do your real-tie data collection or
poll the hardware, read sensors or whatever. Then in the background task
you run the web server. This way there is never a chance of something
like network requests slowing down the data collection.. A simple two
task "multi-tasker" is easy. It gets harder when you have more tasks and
I'd fgo with some kind of OS even it that required a larger uP.
That said. Even if it is easy I'dstill put the web server on a "real"
computer and only pass data out of the arduino. But it could be done. I
build a web server (and an FTP server) into a CCD camera and there is no
performance issue at all. Even during an exposure while reading out the
CCD chip.
As for going the graphics in the client. That really is easy now that
every client computer has a PDF viewer. Simply make your graph a PDF
document.
Chris Albertson
Redondo Beach, California