[GNC] CSV import lags and burns CPU; can't report bug

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[GNC] CSV import lags and burns CPU; can't report bug

GnuCash - User mailing list
Hi.  I'm trying to import transactions from a very simple CSV file, using a
default account and a single transfer account in a CSV field.  I get to the
stage to go from "Transaction Information" where it says "On the following
page you will be able to associate each transaction to a category."  Even a
1-line CSV test file takes about 10 or 15 seconds to get to the next page.
With 160 transactions, the program stops responding, though it is burning
>30% CPU.  I assume it will finish eventually, but this process seems to be
very inefficient.

I can't seem to report a bug anymore.  I just installed GnuCash 3.5.  When
I go to Start > GnuCash > Report a GnuCash Bug (Online), this takes me to
bugzilla.gnome.org.  It says that most projects have migrated to GitLab,
but for GnuCash I should NOT go to GitLab.  When I try to log into
bugzilla, it says "Sorry, entering a bug into the product GnuCash has been
disabled."  However, the GitLab site linked from the BugZilla page does not
seem to have a GnuCash project.  Where should we report bugs these days?

Thanks.

Mark Hedges
_______________________________________________
gnucash-user mailing list
[hidden email]
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-----
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.
Reply | Threaded
Open this post in threaded view
|

Re: [GNC] CSV import lags and burns CPU; can't report bug

Adrien Monteleone-2
GnuCash’s Bugzilla database moved to bugs.gnucash.com several moons ago.

I can’t find any similar shortcut for bug filing on MacOS, so this is probably for the Windows version only. (I don’t ever recall seeing it with Linux versions either)

You should file a bug on that as well, since it needs to point to the correct location.

Regards,
Adrien

> On Apr 8, 2019, at 1:51 PM, Mark Hedges via gnucash-user <[hidden email]> wrote:
>
> Hi.  I'm trying to import transactions from a very simple CSV file, using a
> default account and a single transfer account in a CSV field.  I get to the
> stage to go from "Transaction Information" where it says "On the following
> page you will be able to associate each transaction to a category."  Even a
> 1-line CSV test file takes about 10 or 15 seconds to get to the next page.
> With 160 transactions, the program stops responding, though it is burning
>> 30% CPU.  I assume it will finish eventually, but this process seems to be
> very inefficient.
>
> I can't seem to report a bug anymore.  I just installed GnuCash 3.5.  When
> I go to Start > GnuCash > Report a GnuCash Bug (Online), this takes me to
> bugzilla.gnome.org.  It says that most projects have migrated to GitLab,
> but for GnuCash I should NOT go to GitLab.  When I try to log into
> bugzilla, it says "Sorry, entering a bug into the product GnuCash has been
> disabled."  However, the GitLab site linked from the BugZilla page does not
> seem to have a GnuCash project.  Where should we report bugs these days?
>
> Thanks.
>
> Mark Hedges

_______________________________________________
gnucash-user mailing list
[hidden email]
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-----
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.
Reply | Threaded
Open this post in threaded view
|

Re: [GNC] CSV import lags and burns CPU; can't report bug

John Ralls-2
Huh. Another corner of the Windows Installer I hadn't noticed before.

Fixed, will be in the next nightly build.

Regards,
John Ralls

> On Apr 8, 2019, at 12:57 PM, Adrien Monteleone <[hidden email]> wrote:
>
> GnuCash’s Bugzilla database moved to bugs.gnucash.com several moons ago.
>
> I can’t find any similar shortcut for bug filing on MacOS, so this is probably for the Windows version only. (I don’t ever recall seeing it with Linux versions either)
>
> You should file a bug on that as well, since it needs to point to the correct location.
>
> Regards,
> Adrien
>
>> On Apr 8, 2019, at 1:51 PM, Mark Hedges via gnucash-user <[hidden email]> wrote:
>>
>> Hi.  I'm trying to import transactions from a very simple CSV file, using a
>> default account and a single transfer account in a CSV field.  I get to the
>> stage to go from "Transaction Information" where it says "On the following
>> page you will be able to associate each transaction to a category."  Even a
>> 1-line CSV test file takes about 10 or 15 seconds to get to the next page.
>> With 160 transactions, the program stops responding, though it is burning
>>> 30% CPU.  I assume it will finish eventually, but this process seems to be
>> very inefficient.
>>
>> I can't seem to report a bug anymore.  I just installed GnuCash 3.5.  When
>> I go to Start > GnuCash > Report a GnuCash Bug (Online), this takes me to
>> bugzilla.gnome.org.  It says that most projects have migrated to GitLab,
>> but for GnuCash I should NOT go to GitLab.  When I try to log into
>> bugzilla, it says "Sorry, entering a bug into the product GnuCash has been
>> disabled."  However, the GitLab site linked from the BugZilla page does not
>> seem to have a GnuCash project.  Where should we report bugs these days?
>>
>> Thanks.
>>
>> Mark Hedges
>
> _______________________________________________
> gnucash-user mailing list
> [hidden email]
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> -----
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.

_______________________________________________
gnucash-user mailing list
[hidden email]
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-----
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.
Reply | Threaded
Open this post in threaded view
|

Re: [GNC] CSV import lags and burns CPU; can't report bug

GnuCash - User mailing list
It was super super slow... until it wasn't.  I tried the import over again
with the descriptions modified with an "x" so they don't stomp on
autocomplete for previously filed transactions.  It imported 160
transactions in about 2 seconds today.  So I don't know what was up.  Evil
gnomes?

Mark



On Mon, Apr 8, 2019 at 3:55 PM John Ralls <[hidden email]> wrote:

> Huh. Another corner of the Windows Installer I hadn't noticed before.
>
> Fixed, will be in the next nightly build.
>
> Regards,
> John Ralls
>
> > On Apr 8, 2019, at 12:57 PM, Adrien Monteleone <
> [hidden email]> wrote:
> >
> > GnuCash’s Bugzilla database moved to bugs.gnucash.com several moons ago.
> >
> > I can’t find any similar shortcut for bug filing on MacOS, so this is
> probably for the Windows version only. (I don’t ever recall seeing it with
> Linux versions either)
> >
> > You should file a bug on that as well, since it needs to point to the
> correct location.
> >
> > Regards,
> > Adrien
> >
> >> On Apr 8, 2019, at 1:51 PM, Mark Hedges via gnucash-user <
> [hidden email]> wrote:
> >>
> >> Hi.  I'm trying to import transactions from a very simple CSV file,
> using a
> >> default account and a single transfer account in a CSV field.  I get to
> the
> >> stage to go from "Transaction Information" where it says "On the
> following
> >> page you will be able to associate each transaction to a category."
> Even a
> >> 1-line CSV test file takes about 10 or 15 seconds to get to the next
> page.
> >> With 160 transactions, the program stops responding, though it is
> burning
> >>> 30% CPU.  I assume it will finish eventually, but this process seems
> to be
> >> very inefficient.
> >>
> >> I can't seem to report a bug anymore.  I just installed GnuCash 3.5.
> When
> >> I go to Start > GnuCash > Report a GnuCash Bug (Online), this takes me
> to
> >> bugzilla.gnome.org.  It says that most projects have migrated to
> GitLab,
> >> but for GnuCash I should NOT go to GitLab.  When I try to log into
> >> bugzilla, it says "Sorry, entering a bug into the product GnuCash has
> been
> >> disabled."  However, the GitLab site linked from the BugZilla page does
> not
> >> seem to have a GnuCash project.  Where should we report bugs these days?
> >>
> >> Thanks.
> >>
> >> Mark Hedges
> >
> > _______________________________________________
> > gnucash-user mailing list
> > [hidden email]
> > To update your subscription preferences or to unsubscribe:
> > https://lists.gnucash.org/mailman/listinfo/gnucash-user
> > If you are using Nabble or Gmane, please see
> https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> > -----
> > Please remember to CC this list on all your replies.
> > You can do this by using Reply-To-List or Reply-All.
>
> _______________________________________________
> gnucash-user mailing list
> [hidden email]
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> If you are using Nabble or Gmane, please see
> https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> -----
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
>
_______________________________________________
gnucash-user mailing list
[hidden email]
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-----
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.
Reply | Threaded
Open this post in threaded view
|

Re: [GNC] CSV import lags and burns CPU; can't report bug

cicko
Mark, please report issues with CSV importer as you run into them. Either
here or in Bugzilla.
One thing that's hindering my CSV import efforts is exactly the length of
time it takes to complete. I'm not sure what exactly is taking the time -
I'm suspecting it's the search for duplicates - but it would be interesting
to pinpoint. In which case, I'd file a request to have the ability to skip
that part during import.
Normally, the CSV records I'm generating don't need to be checked against
the existing transactions as they are completely new and I'd like to have
control over that decision, taking the full responsibility for the
correctness of data. I'm not downloading any statements from 3rd parties.
From the implementation standpoint, it should be as simple as adding a
checkbox on the import form, a bool parameter that is checked before the
duplicate match (or whatever else it is) operation.
I'm generating CSV files from transactions recorded on my mobile phone and
would really appreciate if the import was as quick as possible.
Your findings may also contribute to ironing out any other quirks with the
new CSV importer.



--
Sent from: http://gnucash.1415818.n4.nabble.com/GnuCash-User-f1415819.html
_______________________________________________
gnucash-user mailing list
[hidden email]
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-----
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.
Reply | Threaded
Open this post in threaded view
|

Re: [GNC] CSV import lags and burns CPU; can't report bug

Geert Janssens-4
In reply to this post by GnuCash - User mailing list
GnuCash 3 is using a different way to store import matcher heuristics
internally. The first time the csv importer is run, any heuristics in the old
format are converted to the newer format. Depending on how large this
heuristics database is, this can take a considerable amount of time. And
unfortunately there is no visual indication this process is running - an
unfortunate omission. The dev that implemented the conversion probably didn't
expect this to be an issue (and I wouldn't have either tbh).

So maybe this internal conversion is what you experienced. As it should only
run once (if you let it complete) subsequent imports should be faster.

Regards,

Geert

Op donderdag 11 april 2019 01:12:53 CEST schreef Mark Hedges via gnucash-user:

> It was super super slow... until it wasn't.  I tried the import over again
> with the descriptions modified with an "x" so they don't stomp on
> autocomplete for previously filed transactions.  It imported 160
> transactions in about 2 seconds today.  So I don't know what was up.  Evil
> gnomes?
>
> Mark
>
> On Mon, Apr 8, 2019 at 3:55 PM John Ralls <[hidden email]> wrote:
> > Huh. Another corner of the Windows Installer I hadn't noticed before.
> >
> > Fixed, will be in the next nightly build.
> >
> > Regards,
> > John Ralls
> >
> > > On Apr 8, 2019, at 12:57 PM, Adrien Monteleone <
> >
> > [hidden email]> wrote:
> > > GnuCash’s Bugzilla database moved to bugs.gnucash.com several moons ago.
> > >
> > > I can’t find any similar shortcut for bug filing on MacOS, so this is
> >
> > probably for the Windows version only. (I don’t ever recall seeing it with
> > Linux versions either)
> >
> > > You should file a bug on that as well, since it needs to point to the
> >
> > correct location.
> >
> > > Regards,
> > > Adrien
> > >
> > >> On Apr 8, 2019, at 1:51 PM, Mark Hedges via gnucash-user <
> >
> > [hidden email]> wrote:
> > >> Hi.  I'm trying to import transactions from a very simple CSV file,
> >
> > using a
> >
> > >> default account and a single transfer account in a CSV field.  I get to
> >
> > the
> >
> > >> stage to go from "Transaction Information" where it says "On the
> >
> > following
> >
> > >> page you will be able to associate each transaction to a category."
> >
> > Even a
> >
> > >> 1-line CSV test file takes about 10 or 15 seconds to get to the next
> >
> > page.
> >
> > >> With 160 transactions, the program stops responding, though it is
> >
> > burning
> >
> > >>> 30% CPU.  I assume it will finish eventually, but this process seems
> >
> > to be
> >
> > >> very inefficient.
> > >>
> > >> I can't seem to report a bug anymore.  I just installed GnuCash 3.5.
> >
> > When
> >
> > >> I go to Start > GnuCash > Report a GnuCash Bug (Online), this takes me
> >
> > to
> >
> > >> bugzilla.gnome.org.  It says that most projects have migrated to
> >
> > GitLab,
> >
> > >> but for GnuCash I should NOT go to GitLab.  When I try to log into
> > >> bugzilla, it says "Sorry, entering a bug into the product GnuCash has
> >
> > been
> >
> > >> disabled."  However, the GitLab site linked from the BugZilla page does
> >
> > not
> >
> > >> seem to have a GnuCash project.  Where should we report bugs these
> > >> days?
> > >>
> > >> Thanks.
> > >>
> > >> Mark Hedges
> > >
> > > _______________________________________________
> > > gnucash-user mailing list
> > > [hidden email]
> > > To update your subscription preferences or to unsubscribe:
> > > https://lists.gnucash.org/mailman/listinfo/gnucash-user
> > > If you are using Nabble or Gmane, please see
> >
> > https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> >
> > > -----
> > > Please remember to CC this list on all your replies.
> > > You can do this by using Reply-To-List or Reply-All.
> >
> > _______________________________________________
> > gnucash-user mailing list
> > [hidden email]
> > To update your subscription preferences or to unsubscribe:
> > https://lists.gnucash.org/mailman/listinfo/gnucash-user
> > If you are using Nabble or Gmane, please see
> > https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> > -----
> > Please remember to CC this list on all your replies.
> > You can do this by using Reply-To-List or Reply-All.
>
> _______________________________________________
> gnucash-user mailing list
> [hidden email]
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> If you are using Nabble or Gmane, please see
> https://wiki.gnucash.org/wiki/Mailing_Lists for more information. -----
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.




_______________________________________________
gnucash-user mailing list
[hidden email]
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-----
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.