Transaction Posted Time

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Transaction Posted Time

John Ralls-2
Devs,

Bug 767824[1] has me thinking about this again. As I think everyone knows I want to change it from midnight local to 11:00AM UTC for the next version, but since fixing this bug also requires a scrub function at file read time to correct the (probably few) files with borked timestamps I'm thinking of doing it now.

To review, a timestamp of 11:00AM UTC won't change date with the timezone except in -12, the zone immediately to the east of the International Date Line, and +13 and +14. Places affected according to [2] include Adak, Midway, and Baker Islands (-12), Samoa and Tokelau (+13), and Kiribati (+14).

Comments?

Regards,
John Ralls

[1] https://bugzilla.gnome.org/show_bug.cgi?id=767824 <https://bugzilla.gnome.org/show_bug.cgi?id=767824>
[2] http://www.timeanddate.com/time/map/
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: Transaction Posted Time

Christian Stimming-4
Am Sonntag, 19. Juni 2016, 11:31:39 schrieb John Ralls:
> Bug 767824[1] has me thinking about this again. As I think everyone knows I
> want to change it from midnight local to 11:00AM UTC for the next version,
> but since fixing this bug also requires a scrub function at file read time
> to correct the (probably few) files with borked timestamps I'm thinking of
> doing it now.

Thanks for the info. In fact I didn't recognize your idea to change the posted
date's time-of-day.

Did you review bug https://bugzilla.gnome.org/show_bug.cgi?id=137017 for this?
My take is that the "posted date" should be changed into a data type that
doesn't have any time-of-day anymore. As long as this is not the case, we will
IMHO always run into troubles.

At the time bug 137017 was discussed in 2010, I decided to add the additional
KVP slot for posted date with data type GDate. Hence, since 2010 every
transaction has the regular posted-date timestamp (date plus time-of-day) plus
additionally a KVP slot with the posted-date (date only), where the kvp slot's
value will for sure contain the date value that was entered by the user,
regardless of the time zone the program was in at the time of entering.

When you have to think about a scrub function, this additional data field
might become handy.

However, some cases where the posted-date's time-of-day is chosen differently
are known:
- imported transactions that contained a time-of-day will have that time
written in here, such as those transactions that came from an OFX file that
was created by gnucash-on-android.
- book closing transactions contain some other time-of-day here
- other cases might exist, too.

Hence, unfortunately there are still multiple use cases of the time-of-day
part of the posted-date. Any scrub functions that tries to map this time-of-
day to another one, or also a final scrub function that tries to map this to a
fixed day-only data type, will have to take all those special cases into
account. Unfortunately.

Again, I would suggest to switch the posted-date data type to a date-only
(without time) as soon as possible, as discussed in bug 137017 and various
discussions throughout the years.

Regards,

Christian

_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: Transaction Posted Time

John Ralls-2

> On Jun 19, 2016, at 1:07 PM, Christian Stimming <[hidden email]> wrote:
>
> Am Sonntag, 19. Juni 2016, 11:31:39 schrieb John Ralls:
>> Bug 767824[1] has me thinking about this again. As I think everyone knows I
>> want to change it from midnight local to 11:00AM UTC for the next version,
>> but since fixing this bug also requires a scrub function at file read time
>> to correct the (probably few) files with borked timestamps I'm thinking of
>> doing it now.
>
> Thanks for the info. In fact I didn't recognize your idea to change the posted
> date's time-of-day.
>
> Did you review bug https://bugzilla.gnome.org/show_bug.cgi?id=137017 for this?
> My take is that the "posted date" should be changed into a data type that
> doesn't have any time-of-day anymore. As long as this is not the case, we will
> IMHO always run into troubles.
>
> At the time bug 137017 was discussed in 2010, I decided to add the additional
> KVP slot for posted date with data type GDate. Hence, since 2010 every
> transaction has the regular posted-date timestamp (date plus time-of-day) plus
> additionally a KVP slot with the posted-date (date only), where the kvp slot's
> value will for sure contain the date value that was entered by the user,
> regardless of the time zone the program was in at the time of entering.
>
> When you have to think about a scrub function, this additional data field
> might become handy.
>
> However, some cases where the posted-date's time-of-day is chosen differently
> are known:
> - imported transactions that contained a time-of-day will have that time
> written in here, such as those transactions that came from an OFX file that
> was created by gnucash-on-android.
> - book closing transactions contain some other time-of-day here
> - other cases might exist, too.
>
> Hence, unfortunately there are still multiple use cases of the time-of-day
> part of the posted-date. Any scrub functions that tries to map this time-of-
> day to another one, or also a final scrub function that tries to map this to a
> fixed day-only data type, will have to take all those special cases into
> account. Unfortunately.
>
> Again, I would suggest to switch the posted-date data type to a date-only
> (without time) as soon as possible, as discussed in bug 137017 and various
> discussions throughout the years.

Christian,

You and I both agree that a date-only posted date field is the best solution, but we've always gotten substantial push-back from users so we've never changed it. Even the ability to edit the time, mentioned in comment 28, comes up from time to time. So if we set that aside for now, what do you think of using 11:00AM UTC instead of 00:00 Local, in particular changing in the middle of a release series?

Using the transaction date slot for the scrub is an excellent idea, thanks for suggesting it, and thanks as well for reminding me that not all transactions have the timestamp adjusted to midnight local.

Regards,
John Ralls


_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: Transaction Posted Time

Aaron Laws
On Sun, Jun 19, 2016 at 6:15 PM, John Ralls <[hidden email]> wrote:

>
> > On Jun 19, 2016, at 1:07 PM, Christian Stimming <[hidden email]>
> wrote:
> >
> > Again, I would suggest to switch the posted-date data type to a date-only
> > (without time) as soon as possible, as discussed in bug 137017 and
> various
> > discussions throughout the years.
>
> Christian,
>
> You and I both agree that a date-only posted date field is the best
> solution, but we've always gotten substantial push-back from users so we've
> never changed it. Even the ability to edit the time, mentioned in comment
> 28, comes up from time to time. So if we set that aside for now, what do
> you think of using 11:00AM UTC instead of 00:00 Local, in particular
> changing in the middle of a release series?
>

Since there is no time-of-day reported, and it's not editable, no
time-of-day seems like the right representation. If we're dedicated to
storing time-of-day, we should be showing it and allowing it to be
editable. I'm guessing the argument for storing time of day, but not
showing it or allowing it to be edited is because 1) many users want time
of day, 2) no developers care enough to do the work to show and edit the
time of day, right?

Concerning changing to 1100 UTC, I take it that the current system has the
following flaw:
1) user A stores his file in Eastern Time (so that the transactions happen
at midnight in the morning of the transaction date), then later opens the
file in Central time and finds that the posting dates are off by one hour,
which is 2300 the previous day, so they appear to be off by a whole day?
In that case, moving to 1100 UTC (or anything UTC), and showing the date
UTC on the register has the effect of using a date-only representation.
Storing in 1100 UTC, and showing in local time on the register has the
problems you mentioned, but it seems a good deal better than having
problems between, e.g., EST and CST.
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: Transaction Posted Time

John Ralls-2

> On Jun 19, 2016, at 6:25 PM, Aaron Laws <[hidden email]> wrote:
>
> On Sun, Jun 19, 2016 at 6:15 PM, John Ralls <[hidden email] <mailto:[hidden email]>> wrote:
>
> > On Jun 19, 2016, at 1:07 PM, Christian Stimming <[hidden email] <mailto:[hidden email]>> wrote:
> >
> > Again, I would suggest to switch the posted-date data type to a date-only
> > (without time) as soon as possible, as discussed in bug 137017 and various
> > discussions throughout the years.
>
> Christian,
>
> You and I both agree that a date-only posted date field is the best solution, but we've always gotten substantial push-back from users so we've never changed it. Even the ability to edit the time, mentioned in comment 28, comes up from time to time. So if we set that aside for now, what do you think of using 11:00AM UTC instead of 00:00 Local, in particular changing in the middle of a release series?
>
> Since there is no time-of-day reported, and it's not editable, no time-of-day seems like the right representation. If we're dedicated to storing time-of-day, we should be showing it and allowing it to be editable. I'm guessing the argument for storing time of day, but not showing it or allowing it to be edited is because 1) many users want time of day, 2) no developers care enough to do the work to show and edit the time of day, right?
>
> Concerning changing to 1100 UTC, I take it that the current system has the following flaw:
> 1) user A stores his file in Eastern Time (so that the transactions happen at midnight in the morning of the transaction date), then later opens the file in Central time and finds that the posting dates are off by one hour, which is 2300 the previous day, so they appear to be off by a whole day?
> In that case, moving to 1100 UTC (or anything UTC), and showing the date UTC on the register has the effect of using a date-only representation. Storing in 1100 UTC, and showing in local time on the register has the problems you mentioned, but it seems a good deal better than having problems between, e.g., EST and CST.

Aaron,

Actually it's worse than that. Store a transaction in the summer, then look at it in the winter and it's the day before.

The reason for not changing it to date-only now is file compatibility. A newly stored file would have no time part, and that would fail to load in 2.6.12. If we change it now to scrub to 1100Z (sailor/flyer/military for 11:00AM UTC) then it will continue to load on older versions and will magically not flip dates even on those older versions for all but the few users in the time zones I mentioned at the start of the thread. This would also be a good time to change the input code to not blow up if there's no time part, but instead to set a timespec at 1100Z on the date indicated.

Regards,
John Ralls




_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: Transaction Posted Time

Aaron Laws
On Sun, Jun 19, 2016 at 11:12 PM, John Ralls <[hidden email]> wrote:

> Aaron,
>
> Actually it's worse than that. Store a transaction in the summer, then
> look at it in the winter and it's the day before.
>
> The reason for not changing it to date-only now is file compatibility. A
> newly stored file would have no time part, and that would fail to load in
> 2.6.12. If we change it now to scrub to 1100Z (sailor/flyer/military for
> 11:00AM UTC) then it will continue to load on older versions and will
> magically not flip dates even on those older versions for all but the few
> users in the time zones I mentioned at the start of the thread. This would
> also be a good time to change the input code to not blow up if there's no
> time part, but instead to set a timespec at 1100Z on the date indicated.
>
> Regards,
> John Ralls
>

All that sounds good to me; in particular, the backwards compatibility
story seems well-developed here.
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: Transaction Posted Time

Christian Stimming-4
In reply to this post by John Ralls-2
Dear John, yes, your proposal sounds like a good way to go. Thanks for explaining this and thanks for all the good work!

Regards, Christian

Am 20. Juni 2016 00:15:22 MESZ, schrieb John Ralls <[hidden email]>:

>
>> On Jun 19, 2016, at 1:07 PM, Christian Stimming
><[hidden email]> wrote:
>>
>> Am Sonntag, 19. Juni 2016, 11:31:39 schrieb John Ralls:
>>> Bug 767824[1] has me thinking about this again. As I think everyone
>knows I
>>> want to change it from midnight local to 11:00AM UTC for the next
>version,
>>> but since fixing this bug also requires a scrub function at file
>read time
>>> to correct the (probably few) files with borked timestamps I'm
>thinking of
>>> doing it now.
>>
>> Thanks for the info. In fact I didn't recognize your idea to change
>the posted
>> date's time-of-day.
>>
>> Did you review bug https://bugzilla.gnome.org/show_bug.cgi?id=137017
>for this?
>> My take is that the "posted date" should be changed into a data type
>that
>> doesn't have any time-of-day anymore. As long as this is not the
>case, we will
>> IMHO always run into troubles.
>>
>> At the time bug 137017 was discussed in 2010, I decided to add the
>additional
>> KVP slot for posted date with data type GDate. Hence, since 2010
>every
>> transaction has the regular posted-date timestamp (date plus
>time-of-day) plus
>> additionally a KVP slot with the posted-date (date only), where the
>kvp slot's
>> value will for sure contain the date value that was entered by the
>user,
>> regardless of the time zone the program was in at the time of
>entering.
>>
>> When you have to think about a scrub function, this additional data
>field
>> might become handy.
>>
>> However, some cases where the posted-date's time-of-day is chosen
>differently
>> are known:
>> - imported transactions that contained a time-of-day will have that
>time
>> written in here, such as those transactions that came from an OFX
>file that
>> was created by gnucash-on-android.
>> - book closing transactions contain some other time-of-day here
>> - other cases might exist, too.
>>
>> Hence, unfortunately there are still multiple use cases of the
>time-of-day
>> part of the posted-date. Any scrub functions that tries to map this
>time-of-
>> day to another one, or also a final scrub function that tries to map
>this to a
>> fixed day-only data type, will have to take all those special cases
>into
>> account. Unfortunately.
>>
>> Again, I would suggest to switch the posted-date data type to a
>date-only
>> (without time) as soon as possible, as discussed in bug 137017 and
>various
>> discussions throughout the years.
>
>Christian,
>
>You and I both agree that a date-only posted date field is the best
>solution, but we've always gotten substantial push-back from users so
>we've never changed it. Even the ability to edit the time, mentioned in
>comment 28, comes up from time to time. So if we set that aside for
>now, what do you think of using 11:00AM UTC instead of 00:00 Local, in
>particular changing in the middle of a release series?
>
>Using the transaction date slot for the scrub is an excellent idea,
>thanks for suggesting it, and thanks as well for reminding me that not
>all transactions have the timestamp adjusted to midnight local.
>
>Regards,
>John Ralls

--
Sent from mobile.
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: Transaction Posted Time

Geert Janssens-4
In reply to this post by John Ralls-2
On Sunday 19 June 2016 11:31:39 John Ralls wrote:

> Devs,
>
> Bug 767824[1] has me thinking about this again. As I think everyone
> knows I want to change it from midnight local to 11:00AM UTC for the
> next version, but since fixing this bug also requires a scrub
> function at file read time to correct the (probably few) files with
> borked timestamps I'm thinking of doing it now.
>
> To review, a timestamp of 11:00AM UTC won't change date with the
> timezone except in -12, the zone immediately to the east of the
> International Date Line, and +13 and +14. Places affected according
> to [2] include Adak, Midway, and Baker Islands (-12), Samoa and
> Tokelau (+13), and Kiribati (+14).
>
> Comments?
>
> Regards,
> John Ralls
>
> [1] https://bugzilla.gnome.org/show_bug.cgi?id=767824
> <https://bugzilla.gnome.org/show_bug.cgi?id=767824> [2]
> http://www.timeanddate.com/time/map/
> _______________________________________________
> gnucash-devel mailing list
> [hidden email]
> https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Hi John,

In addition to the time stamp use cases Christian already mentioned, I believe the time stamp
is also relevant for sorting entries on an invoice/bill ledger. This was implemented as a hack by
Christian IIRC such that to move an entry up or down the list on the same day, internally the
post dates of the two entries would be swapped. As the post times currently all differ, this
would effectively alter the sorting order. If all post times will be normalized this hack will no
longer work. That's one other case to keep in mind when dropping post times.

Other than that I'm fine with making the time change for the next release.

Regards,

Geert
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: Transaction Posted Time

John Ralls-2

> On Jun 20, 2016, at 11:45 AM, Geert Janssens <[hidden email]> wrote:
>
> On Sunday 19 June 2016 11:31:39 John Ralls wrote:
> > Devs,
> >
> > Bug 767824[1] has me thinking about this again. As I think everyone
> > knows I want to change it from midnight local to 11:00AM UTC for the
> > next version, but since fixing this bug also requires a scrub
> > function at file read time to correct the (probably few) files with
> > borked timestamps I'm thinking of doing it now.
> >
> > To review, a timestamp of 11:00AM UTC won't change date with the
> > timezone except in -12, the zone immediately to the east of the
> > International Date Line, and +13 and +14. Places affected according
> > to [2] include Adak, Midway, and Baker Islands (-12), Samoa and
> > Tokelau (+13), and Kiribati (+14).
> >
> > Comments?
> >
> > Regards,
> > John Ralls
> >
> > [1] https://bugzilla.gnome.org/show_bug.cgi?id=767824
> > <https://bugzilla.gnome.org/show_bug.cgi?id=767824> [2]
> > http://www.timeanddate.com/time/map/
> > _______________________________________________
> > gnucash-devel mailing list
> > [hidden email]
> > https://lists.gnucash.org/mailman/listinfo/gnucash-devel
>  
> Hi John,
>  
> In addition to the time stamp use cases Christian already mentioned, I believe the time stamp is also relevant for sorting entries on an invoice/bill ledger. This was implemented as a hack by Christian IIRC such that to move an entry up or down the list on the same day, internally the post dates of the two entries would be swapped. As the post times currently all differ, this would effectively alter the sorting order. If all post times will be normalized this hack will no longer work. That's one other case to keep in mind when dropping post times.
>  
> Other than that I'm fine with making the time change for the next release.

Geert,

Where is that code?

Regards,
John Ralls


_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: Transaction Posted Time

Geert Janssens-4
On Monday 20 June 2016 12:14:24 John Ralls wrote:

> > Hi John,
> >
> > In addition to the time stamp use cases Christian already mentioned,
> > I believe the time stamp is also relevant for sorting entries on an
> > invoice/bill ledger. This was implemented as a hack by Christian
> > IIRC such that to move an entry up or down the list on the same
> > day, internally the post dates of the two entries would be swapped.
> > As the post times currently all differ, this would effectively
> > alter the sorting order. If all post times will be normalized this
> > hack will no longer work. That's one other case to keep in mind
> > when dropping post times.
> >
> > Other than that I'm fine with making the time change for the next
> > release.
> Geert,
>
> Where is that code?
>
> Regards,
> John Ralls

https://github.com/Gnucash/gnucash/blob/master/src/business/business-ledger/gncEntryLedger.c#L975

Which is called from gnc_invoice_window_entryUpCB and gnc_invoice_window_entryUpCB in
https://github.com/Gnucash/gnucash/blob/master/src/business/business-gnome/dialog-invoice.c

These in turn get called by callbacks for gui buttons
(gnc_plugin_page_invoice_cmd_entryUp/Down). These callbacks can be found in
https://github.com/Gnucash/gnucash/blob/master/src/business/business-gnome/gnc-plugin-page-invoice.c

Regards,

Geert
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: Transaction Posted Time

John Ralls-2

> On Jun 20, 2016, at 12:36 PM, Geert Janssens <[hidden email]> wrote:
>
> On Monday 20 June 2016 12:14:24 John Ralls wrote:
> > > Hi John,
> > >
> > > In addition to the time stamp use cases Christian already mentioned,
> > > I believe the time stamp is also relevant for sorting entries on an
> > > invoice/bill ledger. This was implemented as a hack by Christian
> > > IIRC such that to move an entry up or down the list on the same
> > > day, internally the post dates of the two entries would be swapped.
> > > As the post times currently all differ, this would effectively
> > > alter the sorting order. If all post times will be normalized this
> > > hack will no longer work. That's one other case to keep in mind
> > > when dropping post times.
> > >
> > > Other than that I'm fine with making the time change for the next
> > > release.
> > Geert,
> >
> > Where is that code?
> >
> > Regards,
> > John Ralls
>  
> https://github.com/Gnucash/gnucash/blob/master/src/business/business-ledger/gncEntryLedger.c#L975
>  
> Which is called from gnc_invoice_window_entryUpCB and gnc_invoice_window_entryUpCB in
> https://github.com/Gnucash/gnucash/blob/master/src/business/business-gnome/dialog-invoice.c
>  
> These in turn get called by callbacks for gui buttons (gnc_plugin_page_invoice_cmd_entryUp/Down). These callbacks can be found in
> https://github.com/Gnucash/gnucash/blob/master/src/business/business-gnome/gnc-plugin-page-invoice.c

Geert,

No problem then. That adjusts a GncEntry's date_entered, not a Transaction's date_posted.

Regards,
John Ralls



_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: Transaction Posted Time

Geert Janssens-4
On Monday 20 June 2016 13:43:18 John Ralls wrote:

> > On Jun 20, 2016, at 12:36 PM, Geert Janssens
> > <[hidden email]> wrote:>
> > On Monday 20 June 2016 12:14:24 John Ralls wrote:
> > > > Hi John,
> > > >
> > > > In addition to the time stamp use cases Christian already
> > > > mentioned,
> > > > I believe the time stamp is also relevant for sorting entries on
> > > > an
> > > > invoice/bill ledger. This was implemented as a hack by Christian
> > > > IIRC such that to move an entry up or down the list on the same
> > > > day, internally the post dates of the two entries would be
> > > > swapped.
> > > > As the post times currently all differ, this would effectively
> > > > alter the sorting order. If all post times will be normalized
> > > > this
> > > > hack will no longer work. That's one other case to keep in mind
> > > > when dropping post times.
> > > >
> > > > Other than that I'm fine with making the time change for the
> > > > next
> > > > release.
> > >
> > > Geert,
> > >
> > > Where is that code?
> > >
> > > Regards,
> > > John Ralls
> >
> > https://github.com/Gnucash/gnucash/blob/master/src/business/business
> > -ledger/gncEntryLedger.c#L975
> >
> > Which is called from gnc_invoice_window_entryUpCB and
> > gnc_invoice_window_entryUpCB in
> > https://github.com/Gnucash/gnucash/blob/master/src/business/busines
> > s-gnome/dialog-invoice.c
> >
> > These in turn get called by callbacks for gui buttons
> > (gnc_plugin_page_invoice_cmd_entryUp/Down). These callbacks can be
> > found in
> > https://github.com/Gnucash/gnucash/blob/master/src/business/busines
> > s-gnome/gnc-plugin-page-invoice.c
> Geert,
>
> No problem then. That adjusts a GncEntry's date_entered, not a
> Transaction's date_posted.
>
> Regards,
> John Ralls

Good!

That will only make it a point of attention whenever we decide to drop timestamps
altogether from GnuCash.

Regards,

Geert
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel