ES
Eric Scace
Tue, May 28, 2019 2:38 PM
Hi fellow time nuts —
I’m looking for a sanity check or alternative suggestions for the problem and tentative solution described below.
Thanks for your thoughts.
— Eric
Problem:
In one of my day jobs, I am designing a global network of systems (using open-source software) that provide well-researched information about rights and licenses for musical works (e.g., songs, compositions).
Claiming rights, registering licenses (some of which are temporary), and time-stamping changes to data each exemplify cases where date/time must be included. In many cases the time order of events can be important — potentially changing who gets paid how much when music is performed or distributed.
The machines are scattered around the planet and the usual problems of time distribution exist. Furthermore, systems are operated independently. We assume occasional use of NTP to correct system clocks, but not a local GPS-provided time. The software development team is generally oblivious to the issues of time in distributed computer networks.
A grim picture — but, fortunately, this application does not require high precision time.
My tentative proposal:
-
To avoid burdening systems with multiple local time conversions, all date/time information throughout the system shall be UTC. Implications:
user apps will be responsible for converting from a human user’s local time to UTC
thus, user app developers will have to do this conversion correctly
-
Date/time stamps in the data shall be rounded to the nearest EVEN second by the system instances; e.g., to 2019 May 28 14:24:28 UTC. Implications:
user apps that submit claims or updates will have their claims/updates date/time-stamped by the receiving system node with this rounding method. Example: “John Smith claims that he and Jane Doe wrote these lyrics, making an equal contribution between them, on 2019 May 27 15:00:00 UTC. His claim was received by the system at 2019 May 28 14:26:50 UTC.” One or more blockchain ledgers record a hash of the musical work [the lyrics, in this example], the claim [who wrote the lyrics when], and which app/system registered the claim and when.
events occurring roughly within ±1 second of each other will be deemed to have occurred simultaneously. This is entirely adequate for this application.
competing leap second smearing methods employed by different operating systems and data center operators will be washed out of the time stamp by this rounding requirement.
— end —
Hi fellow time nuts —
I’m looking for a sanity check or alternative suggestions for the problem and tentative solution described below.
Thanks for your thoughts.
— Eric
Problem:
In one of my day jobs, I am designing a global network of systems (using open-source software) that provide well-researched information about rights and licenses for musical works (e.g., songs, compositions).
Claiming rights, registering licenses (some of which are temporary), and time-stamping changes to data each exemplify cases where date/time must be included. In many cases the time order of events can be important — potentially changing who gets paid how much when music is performed or distributed.
The machines are scattered around the planet and the usual problems of time distribution exist. Furthermore, systems are operated independently. We assume occasional use of NTP to correct system clocks, but not a local GPS-provided time. The software development team is generally oblivious to the issues of time in distributed computer networks.
A grim picture — but, fortunately, this application does not require high precision time.
My tentative proposal:
1. To avoid burdening systems with multiple local time conversions, all date/time information throughout the system shall be UTC. Implications:
user apps will be responsible for converting from a human user’s local time to UTC
thus, user app developers will have to do this conversion correctly
2. Date/time stamps in the data shall be rounded to the nearest EVEN second by the system instances; e.g., to 2019 May 28 14:24:28 UTC. Implications:
user apps that submit claims or updates will have their claims/updates date/time-stamped by the receiving system node with this rounding method. Example: “John Smith claims that he and Jane Doe wrote these lyrics, making an equal contribution between them, on 2019 May 27 15:00:00 UTC. His claim was received by the system at 2019 May 28 14:26:50 UTC.” One or more blockchain ledgers record a hash of the musical work [the lyrics, in this example], the claim [who wrote the lyrics when], and which app/system registered the claim and when.
events occurring roughly within ±1 second of each other will be deemed to have occurred simultaneously. This is entirely adequate for this application.
competing leap second smearing methods employed by different operating systems and data center operators will be washed out of the time stamp by this rounding requirement.
— end —
KM
K5ROE Mike
Tue, May 28, 2019 3:51 PM
If the data collected by your system could potentially be used in
litigation , I would reconsider your accuracy requirement, especially
the OKness of simultaneous transactions.
I assume that all nodes can write to the blockchain; how do you sanity
check if one node's clock is wildly off?
Since you don't control the operating system, but do control the
application code, you may want/have to maintain time in your application.
Since everybody is writing to the same chain, I don't see how you can
not use a mutual time reference among the nodes.
On 5/28/19 10:38 AM, Eric Scace wrote:
Hi fellow time nuts —
I’m looking for a sanity check or alternative suggestions for the problem and tentative solution described below.
Thanks for your thoughts.
— Eric
Problem:
In one of my day jobs, I am designing a global network of systems (using open-source software) that provide well-researched information about rights and licenses for musical works (e.g., songs, compositions).
Claiming rights, registering licenses (some of which are temporary), and time-stamping changes to data each exemplify cases where date/time must be included. In many cases the time order of events can be important — potentially changing who gets paid how much when music is performed or distributed.
The machines are scattered around the planet and the usual problems of time distribution exist. Furthermore, systems are operated independently. We assume occasional use of NTP to correct system clocks, but not a local GPS-provided time. The software development team is generally oblivious to the issues of time in distributed computer networks.
A grim picture — but, fortunately, this application does not require high precision time.
My tentative proposal:
1. To avoid burdening systems with multiple local time conversions, all date/time information throughout the system shall be UTC. Implications:
user apps will be responsible for converting from a human user’s local time to UTC
thus, user app developers will have to do this conversion correctly
2. Date/time stamps in the data shall be rounded to the nearest EVEN second by the system instances; e.g., to 2019 May 28 14:24:28 UTC. Implications:
user apps that submit claims or updates will have their claims/updates date/time-stamped by the receiving system node with this rounding method. Example: “John Smith claims that he and Jane Doe wrote these lyrics, making an equal contribution between them, on 2019 May 27 15:00:00 UTC. His claim was received by the system at 2019 May 28 14:26:50 UTC.” One or more blockchain ledgers record a hash of the musical work [the lyrics, in this example], the claim [who wrote the lyrics when], and which app/system registered the claim and when.
events occurring roughly within ±1 second of each other will be deemed to have occurred simultaneously. This is entirely adequate for this application.
competing leap second smearing methods employed by different operating systems and data center operators will be washed out of the time stamp by this rounding requirement.
— end —
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.
If the data collected by your system could potentially be used in
litigation , I would reconsider your accuracy requirement, especially
the OKness of simultaneous transactions.
I assume that all nodes can write to the blockchain; how do you sanity
check if one node's clock is wildly off?
Since you don't control the operating system, but do control the
application code, you may want/have to maintain time in your application.
Since everybody is writing to the same chain, I don't see how you can
not use a mutual time reference among the nodes.
On 5/28/19 10:38 AM, Eric Scace wrote:
> Hi fellow time nuts —
>
> I’m looking for a sanity check or alternative suggestions for the problem and tentative solution described below.
>
> Thanks for your thoughts.
>
> — Eric
>
> Problem:
>
> In one of my day jobs, I am designing a global network of systems (using open-source software) that provide well-researched information about rights and licenses for musical works (e.g., songs, compositions).
>
> Claiming rights, registering licenses (some of which are temporary), and time-stamping changes to data each exemplify cases where date/time must be included. In many cases the time order of events can be important — potentially changing who gets paid how much when music is performed or distributed.
>
> The machines are scattered around the planet and the usual problems of time distribution exist. Furthermore, systems are operated independently. We assume occasional use of NTP to correct system clocks, but not a local GPS-provided time. The software development team is generally oblivious to the issues of time in distributed computer networks.
>
> A grim picture — but, fortunately, this application does not require high precision time.
>
> My tentative proposal:
>
> 1. To avoid burdening systems with multiple local time conversions, all date/time information throughout the system shall be UTC. Implications:
> user apps will be responsible for converting from a human user’s local time to UTC
> thus, user app developers will have to do this conversion correctly
>
> 2. Date/time stamps in the data shall be rounded to the nearest EVEN second by the system instances; e.g., to 2019 May 28 14:24:28 UTC. Implications:
> user apps that submit claims or updates will have their claims/updates date/time-stamped by the receiving system node with this rounding method. Example: “John Smith claims that he and Jane Doe wrote these lyrics, making an equal contribution between them, on 2019 May 27 15:00:00 UTC. His claim was received by the system at 2019 May 28 14:26:50 UTC.” One or more blockchain ledgers record a hash of the musical work [the lyrics, in this example], the claim [who wrote the lyrics when], and which app/system registered the claim and when.
> events occurring roughly within ±1 second of each other will be deemed to have occurred simultaneously. This is entirely adequate for this application.
> competing leap second smearing methods employed by different operating systems and data center operators will be washed out of the time stamp by this rounding requirement.
>
> — end —
> _______________________________________________
> time-nuts mailing list -- time-nuts@lists.febo.com
> To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
> and follow the instructions there.
PS
paul swed
Tue, May 28, 2019 4:43 PM
Something to think about. Leap seconds.
But that said I suspect the reason for the reasonable time is that many
publishers wouldn't have access to a solid time source like GPS or some
other universal reference.
Really agree with Mike on the litigation aspect and the need for accurate
time. How do you stay clear of that issue. It will ultimately show up on
your door step.
Regards
Paul
WB8TSL
On Tue, May 28, 2019 at 12:00 PM K5ROE Mike K5ROE@roetto.org wrote:
If the data collected by your system could potentially be used in
litigation , I would reconsider your accuracy requirement, especially
the OKness of simultaneous transactions.
I assume that all nodes can write to the blockchain; how do you sanity
check if one node's clock is wildly off?
Since you don't control the operating system, but do control the
application code, you may want/have to maintain time in your application.
Since everybody is writing to the same chain, I don't see how you can
not use a mutual time reference among the nodes.
On 5/28/19 10:38 AM, Eric Scace wrote:
Hi fellow time nuts —
I’m looking for a sanity check or alternative suggestions for the
problem and tentative solution described below.
Thanks for your thoughts.
— Eric
Problem:
In one of my day jobs, I am designing a global network of systems
(using open-source software) that provide well-researched information about
rights and licenses for musical works (e.g., songs, compositions).
Claiming rights, registering licenses (some of which are temporary),
and time-stamping changes to data each exemplify cases where date/time must
be included. In many cases the time order of events can be important —
potentially changing who gets paid how much when music is performed or
distributed.
The machines are scattered around the planet and the usual problems
of time distribution exist. Furthermore, systems are operated
independently. We assume occasional use of NTP to correct system clocks,
but not a local GPS-provided time. The software development team is
generally oblivious to the issues of time in distributed computer networks.
A grim picture — but, fortunately, this application does not require
My tentative proposal:
1. To avoid burdening systems with multiple local time conversions,
all date/time information throughout the system shall be UTC. Implications:
user apps will be responsible for converting from a human user’s local
thus, user app developers will have to do this conversion correctly
2. Date/time stamps in the data shall be rounded to the nearest EVEN
second by the system instances; e.g., to 2019 May 28 14:24:28 UTC.
Implications:
user apps that submit claims or updates will have their claims/updates
date/time-stamped by the receiving system node with this rounding method.
Example: “John Smith claims that he and Jane Doe wrote these lyrics, making
an equal contribution between them, on 2019 May 27 15:00:00 UTC. His claim
was received by the system at 2019 May 28 14:26:50 UTC.” One or more
blockchain ledgers record a hash of the musical work [the lyrics, in this
example], the claim [who wrote the lyrics when], and which app/system
registered the claim and when.
events occurring roughly within ±1 second of each other will be deemed
to have occurred simultaneously. This is entirely adequate for this
application.
competing leap second smearing methods employed by different operating
systems and data center operators will be washed out of the time stamp by
this rounding requirement.
and follow the instructions there.
Something to think about. Leap seconds.
But that said I suspect the reason for the reasonable time is that many
publishers wouldn't have access to a solid time source like GPS or some
other universal reference.
Really agree with Mike on the litigation aspect and the need for accurate
time. How do you stay clear of that issue. It will ultimately show up on
your door step.
Regards
Paul
WB8TSL
On Tue, May 28, 2019 at 12:00 PM K5ROE Mike <K5ROE@roetto.org> wrote:
> If the data collected by your system could potentially be used in
> litigation , I would reconsider your accuracy requirement, especially
> the OKness of simultaneous transactions.
>
> I assume that all nodes can write to the blockchain; how do you sanity
> check if one node's clock is wildly off?
>
> Since you don't control the operating system, but do control the
> application code, you may want/have to maintain time in your application.
>
> Since everybody is writing to the same chain, I don't see how you can
> not use a mutual time reference among the nodes.
>
>
> On 5/28/19 10:38 AM, Eric Scace wrote:
> > Hi fellow time nuts —
> >
> > I’m looking for a sanity check or alternative suggestions for the
> problem and tentative solution described below.
> >
> > Thanks for your thoughts.
> >
> > — Eric
> >
> > Problem:
> >
> > In one of my day jobs, I am designing a global network of systems
> (using open-source software) that provide well-researched information about
> rights and licenses for musical works (e.g., songs, compositions).
> >
> > Claiming rights, registering licenses (some of which are temporary),
> and time-stamping changes to data each exemplify cases where date/time must
> be included. In many cases the time order of events can be important —
> potentially changing who gets paid how much when music is performed or
> distributed.
> >
> > The machines are scattered around the planet and the usual problems
> of time distribution exist. Furthermore, systems are operated
> independently. We assume occasional use of NTP to correct system clocks,
> but not a local GPS-provided time. The software development team is
> generally oblivious to the issues of time in distributed computer networks.
> >
> > A grim picture — but, fortunately, this application does not require
> high precision time.
> >
> > My tentative proposal:
> >
> > 1. To avoid burdening systems with multiple local time conversions,
> all date/time information throughout the system shall be UTC. Implications:
> > user apps will be responsible for converting from a human user’s local
> time to UTC
> > thus, user app developers will have to do this conversion correctly
> >
> > 2. Date/time stamps in the data shall be rounded to the nearest EVEN
> second by the system instances; e.g., to 2019 May 28 14:24:28 UTC.
> Implications:
> > user apps that submit claims or updates will have their claims/updates
> date/time-stamped by the receiving system node with this rounding method.
> Example: “John Smith claims that he and Jane Doe wrote these lyrics, making
> an equal contribution between them, on 2019 May 27 15:00:00 UTC. His claim
> was received by the system at 2019 May 28 14:26:50 UTC.” One or more
> blockchain ledgers record a hash of the musical work [the lyrics, in this
> example], the claim [who wrote the lyrics when], and which app/system
> registered the claim and when.
> > events occurring roughly within ±1 second of each other will be deemed
> to have occurred simultaneously. This is entirely adequate for this
> application.
> > competing leap second smearing methods employed by different operating
> systems and data center operators will be washed out of the time stamp by
> this rounding requirement.
> >
> > — end —
> > _______________________________________________
> > time-nuts mailing list -- time-nuts@lists.febo.com
> > To unsubscribe, go to
> http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
> > and follow the instructions there.
>
> _______________________________________________
> time-nuts mailing list -- time-nuts@lists.febo.com
> To unsubscribe, go to
> http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
> and follow the instructions there.
>
TV
Tom Van Baak
Tue, May 28, 2019 5:02 PM
- Date/time stamps in the data shall be rounded to the nearest EVEN
second by the system instances
That's a clever way to both mask accuracy & uncertainty and to avoid
leap seconds. Still, it smells like a hack, unfit for the 21st century.
But I feel your pain.
BTW, this is how mod times are stored in the FAT file system. [1] For
example, if a camera uses a SD card, you'll see that every photo has an
EVEN time stamp. In this case it wasn't to avoid leap seconds but rather
it allowed time-of-day to fit in 16 bits (back in the era when bytes
mattered). So there's ancient (any maybe even forensic or legal)
precedent for what you're doing.
I worry, though, about the next person tasked with improving your
design. You have fused two separate issues together: the issue of
timestamp accuracy / resolution / legal traceability, and the issue of
properly handling UTC (leap seconds). Are you sure there's no other
practical solution than to use EVEN seconds?
/tvb
[1] https://en.wikipedia.org/wiki/File_Allocation_Table
Hi Eric,
> 2. Date/time stamps in the data shall be rounded to the nearest EVEN
second by the system instances
That's a clever way to both mask accuracy & uncertainty and to avoid
leap seconds. Still, it smells like a hack, unfit for the 21st century.
But I feel your pain.
BTW, this is how mod times are stored in the FAT file system. [1] For
example, if a camera uses a SD card, you'll see that every photo has an
EVEN time stamp. In this case it wasn't to avoid leap seconds but rather
it allowed time-of-day to fit in 16 bits (back in the era when bytes
mattered). So there's ancient (any maybe even forensic or legal)
precedent for what you're doing.
I worry, though, about the next person tasked with improving your
design. You have fused two separate issues together: the issue of
timestamp accuracy / resolution / legal traceability, and the issue of
properly handling UTC (leap seconds). Are you sure there's no other
practical solution than to use EVEN seconds?
/tvb
[1] https://en.wikipedia.org/wiki/File_Allocation_Table
MC
Mike Cook
Tue, May 28, 2019 8:23 PM
Le 28 mai 2019 à 16:38, Eric Scace eric@scace.org a écrit :
Hi fellow time nuts —
I’m looking for a sanity check or alternative suggestions for the problem and tentative solution described below.
You have a situation where the application in any one system is unaware of the validity of the underlying OS time. So I think you have to have a mechanism where transactions are refused/re-routed if that validity can not be established. So I would say that some mechanism be put in place to check the local clock against the time reported by a majority , or a large enough sample, of the networks systems. Depending on the transaction rate that could be done on a per transaction, or reasonable interval.
Thanks for your thoughts.
— Eric
Problem:
In one of my day jobs, I am designing a global network of systems (using open-source software) that provide well-researched information about rights and licenses for musical works (e.g., songs, compositions).
Claiming rights, registering licenses (some of which are temporary), and time-stamping changes to data each exemplify cases where date/time must be included. In many cases the time order of events can be important — potentially changing who gets paid how much when music is performed or distributed.
The machines are scattered around the planet and the usual problems of time distribution exist. Furthermore, systems are operated independently. We assume occasional use of NTP to correct system clocks, but not a local GPS-provided time. The software development team is generally oblivious to the issues of time in distributed computer networks.
A grim picture — but, fortunately, this application does not require high precision time.
My tentative proposal:
-
To avoid burdening systems with multiple local time conversions, all date/time information throughout the system shall be UTC. Implications:
user apps will be responsible for converting from a human user’s local time to UTC
thus, user app developers will have to do this conversion correctly
-
Date/time stamps in the data shall be rounded to the nearest EVEN second by the system instances; e.g., to 2019 May 28 14:24:28 UTC. Implications:
user apps that submit claims or updates will have their claims/updates date/time-stamped by the receiving system node with this rounding method. Example: “John Smith claims that he and Jane Doe wrote these lyrics, making an equal contribution between them, on 2019 May 27 15:00:00 UTC. His claim was received by the system at 2019 May 28 14:26:50 UTC.” One or more blockchain ledgers record a hash of the musical work [the lyrics, in this example], the claim [who wrote the lyrics when], and which app/system registered the claim and when.
events occurring roughly within ±1 second of each other will be deemed to have occurred simultaneously. This is entirely adequate for this application.
competing leap second smearing methods employed by different operating systems and data center operators will be washed out of the time stamp by this rounding requirement.
— end —
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.
"Ceux qui sont prêts à abandonner une liberté essentielle pour obtenir une petite et provisoire sécurité, ne méritent ni liberté ni sécurité."
Benjimin Franklin
> Le 28 mai 2019 à 16:38, Eric Scace <eric@scace.org> a écrit :
>
> Hi fellow time nuts —
>
> I’m looking for a sanity check or alternative suggestions for the problem and tentative solution described below.
You have a situation where the application in any one system is unaware of the validity of the underlying OS time. So I think you have to have a mechanism where transactions are refused/re-routed if that validity can not be established. So I would say that some mechanism be put in place to check the local clock against the time reported by a majority , or a large enough sample, of the networks systems. Depending on the transaction rate that could be done on a per transaction, or reasonable interval.
>
> Thanks for your thoughts.
>
> — Eric
>
> Problem:
>
> In one of my day jobs, I am designing a global network of systems (using open-source software) that provide well-researched information about rights and licenses for musical works (e.g., songs, compositions).
>
> Claiming rights, registering licenses (some of which are temporary), and time-stamping changes to data each exemplify cases where date/time must be included. In many cases the time order of events can be important — potentially changing who gets paid how much when music is performed or distributed.
>
> The machines are scattered around the planet and the usual problems of time distribution exist. Furthermore, systems are operated independently. We assume occasional use of NTP to correct system clocks, but not a local GPS-provided time. The software development team is generally oblivious to the issues of time in distributed computer networks.
>
> A grim picture — but, fortunately, this application does not require high precision time.
>
> My tentative proposal:
>
> 1. To avoid burdening systems with multiple local time conversions, all date/time information throughout the system shall be UTC. Implications:
> user apps will be responsible for converting from a human user’s local time to UTC
> thus, user app developers will have to do this conversion correctly
>
> 2. Date/time stamps in the data shall be rounded to the nearest EVEN second by the system instances; e.g., to 2019 May 28 14:24:28 UTC. Implications:
> user apps that submit claims or updates will have their claims/updates date/time-stamped by the receiving system node with this rounding method. Example: “John Smith claims that he and Jane Doe wrote these lyrics, making an equal contribution between them, on 2019 May 27 15:00:00 UTC. His claim was received by the system at 2019 May 28 14:26:50 UTC.” One or more blockchain ledgers record a hash of the musical work [the lyrics, in this example], the claim [who wrote the lyrics when], and which app/system registered the claim and when.
> events occurring roughly within ±1 second of each other will be deemed to have occurred simultaneously. This is entirely adequate for this application.
> competing leap second smearing methods employed by different operating systems and data center operators will be washed out of the time stamp by this rounding requirement.
>
> — end —
> _______________________________________________
> time-nuts mailing list -- time-nuts@lists.febo.com
> To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
> and follow the instructions there.
"Ceux qui sont prêts à abandonner une liberté essentielle pour obtenir une petite et provisoire sécurité, ne méritent ni liberté ni sécurité."
Benjimin Franklin
