Command line to dump data as plain text files

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Command line to dump data as plain text files

CY Lim
Hello all,

I cannot seem to find how to use command line to dump data from GnuCash
into a single or multiple plain text files (e.g. csv). Any suggestion will
be helpful.

If that cannot be done with command line, I am happy to use API provided in
C or Python to access the data. Some examples or documentations are
appreciated.

I want to avoid manual parsing of the files, unless what I need is not
doable with the 2 methods above.

Many thanks,
Chee
_______________________________________________
gnucash-user mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-----
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.
Reply | Threaded
Open this post in threaded view
|

Re: Command line to dump data as plain text files

John Ralls-2

> On Apr 13, 2017, at 7:52 AM, CY Lim <[hidden email]> wrote:
>
> Hello all,
>
> I cannot seem to find how to use command line to dump data from GnuCash
> into a single or multiple plain text files (e.g. csv). Any suggestion will
> be helpful.
>
> If that cannot be done with command line, I am happy to use API provided in
> C or Python to access the data. Some examples or documentations are
> appreciated.
>
> I want to avoid manual parsing of the files, unless what I need is not
> doable with the 2 methods above.


There is no export facility available from the command line, and only limited export (of transactions and splits to csv) from the GUI. You can of course get to any data you want from C and to most of it from Python, and there's also https://github.com/sdementen/piecash <https://github.com/sdementen/piecash>, a third-party framework for working with the XML file. But...

GnuCash uses by default a plain text file in XML, though it may be compressed with gzip. You can turn off compression in Preferences, on the General tab, or you can gunzip the file; GnuCash doesn't care about compression at load time. The not-default storage is in SQL with one of SQLite3, MySQL, and PostrgresQL. Most people who want to query their GnuCash data outside of GnuCash find that to be the easiest way.

Regards,
John Ralls

_______________________________________________
gnucash-user mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-----
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.
Reply | Threaded
Open this post in threaded view
|

Re: Command line to dump data as plain text files

David Carlson-4
Cy Lim,

Just in case you want to change you data file after you read it, try to do
it through the regular GnuCash program.  It is very easy to 'break' the
file and make some data unreadable or inconsistent if you use a text
editor, or, in some cases even with a database editor.

David C

On Sun, Apr 16, 2017 at 9:42 AM, John Ralls <[hidden email]> wrote:

>
> > On Apr 13, 2017, at 7:52 AM, CY Lim <[hidden email]> wrote:
> >
> > Hello all,
> >
> > I cannot seem to find how to use command line to dump data from GnuCash
> > into a single or multiple plain text files (e.g. csv). Any suggestion
> will
> > be helpful.
> >
> > If that cannot be done with command line, I am happy to use API provided
> in
> > C or Python to access the data. Some examples or documentations are
> > appreciated.
> >
> > I want to avoid manual parsing of the files, unless what I need is not
> > doable with the 2 methods above.
>
>
> There is no export facility available from the command line, and only
> limited export (of transactions and splits to csv) from the GUI. You can of
> course get to any data you want from C and to most of it from Python, and
> there's also https://github.com/sdementen/piecash <
> https://github.com/sdementen/piecash>, a third-party framework for
> working with the XML file. But...
>
> GnuCash uses by default a plain text file in XML, though it may be
> compressed with gzip. You can turn off compression in Preferences, on the
> General tab, or you can gunzip the file; GnuCash doesn't care about
> compression at load time. The not-default storage is in SQL with one of
> SQLite3, MySQL, and PostrgresQL. Most people who want to query their
> GnuCash data outside of GnuCash find that to be the easiest way.
>
> Regards,
> John Ralls
>
> _______________________________________________
> gnucash-user mailing list
> [hidden email]
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> -----
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
>
_______________________________________________
gnucash-user mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-----
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.
Reply | Threaded
Open this post in threaded view
|

Re: Command line to dump data as plain text files

Sébastien de Menten
In reply to this post by John Ralls-2
Hello Cheer

You can find an example of script that dumps the content of a gnucash book
in the Ledger format at
https://github.com/sdementen/piecash/blob/master/scripts/piecash_ledger.py.
Your gnucash book should be saved in a SQL format (not XML).

Kr

Sébastien

On Apr 16, 2017 16:51, "John Ralls" <[hidden email]> wrote:

>
> > On Apr 13, 2017, at 7:52 AM, CY Lim <[hidden email]> wrote:
> >
> > Hello all,
> >
> > I cannot seem to find how to use command line to dump data from GnuCash
> > into a single or multiple plain text files (e.g. csv). Any suggestion
> will
> > be helpful.
> >
> > If that cannot be done with command line, I am happy to use API provided
> in
> > C or Python to access the data. Some examples or documentations are
> > appreciated.
> >
> > I want to avoid manual parsing of the files, unless what I need is not
> > doable with the 2 methods above.
>
>
> There is no export facility available from the command line, and only
> limited export (of transactions and splits to csv) from the GUI. You can of
> course get to any data you want from C and to most of it from Python, and
> there's also https://github.com/sdementen/piecash <
> https://github.com/sdementen/piecash>, a third-party framework for
> working with the XML file. But...
>
> GnuCash uses by default a plain text file in XML, though it may be
> compressed with gzip. You can turn off compression in Preferences, on the
> General tab, or you can gunzip the file; GnuCash doesn't care about
> compression at load time. The not-default storage is in SQL with one of
> SQLite3, MySQL, and PostrgresQL. Most people who want to query their
> GnuCash data outside of GnuCash find that to be the easiest way.
>
> Regards,
> John Ralls
>
> _______________________________________________
> gnucash-user mailing list
> [hidden email]
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> -----
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
>
_______________________________________________
gnucash-user mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-----
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.
Reply | Threaded
Open this post in threaded view
|

Re: Command line to dump data as plain text files

Colin Law
In reply to this post by CY Lim
On 13 April 2017 at 15:52, CY Lim <[hidden email]> wrote:
> Hello all,
>
> I cannot seem to find how to use command line to dump data from GnuCash
> into a single or multiple plain text files (e.g. csv). Any suggestion will
> be helpful.

Can you explain why you want to dump to text files?  There may be
better ways to achieve what you want to do.

Colin
_______________________________________________
gnucash-user mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-----
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.
Reply | Threaded
Open this post in threaded view
|

Re: Command line to dump data as plain text files

CY Lim
Thank you all for the information. Piecash seems to be the best (and
easiest) solution for me to access the data within GnuCash.

I want to extract the data, so that I can setup an automated pipeline to
parse them and generate my own report in R and Python. This will be a
read-only process, so hopefully I will not run into difficulties and data
corruption.

Chee

On 16 April 2017 at 19:55, Colin Law <[hidden email]> wrote:

> On 13 April 2017 at 15:52, CY Lim <[hidden email]> wrote:
> > Hello all,
> >
> > I cannot seem to find how to use command line to dump data from GnuCash
> > into a single or multiple plain text files (e.g. csv). Any suggestion
> will
> > be helpful.
>
> Can you explain why you want to dump to text files?  There may be
> better ways to achieve what you want to do.
>
> Colin
>
_______________________________________________
gnucash-user mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-----
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.
Reply | Threaded
Open this post in threaded view
|

Re: Command line to dump data as plain text files

Christopher Lam
In reply to this post by John Ralls-2
Hi, I extended a lightweight Python3 library by jorgenschaefer which can
provide read-only access to xml files. It handles comporessed xml too.
https://github.com/christopherlam/gnucashxml

This also has .ledger output facility:
import gnucashxml
book = gnucashxml.from_filename("datafile.gnucash")
print(book.ledger)

On 16 April 2017 at 22:42, John Ralls <[hidden email]> wrote:

>
> > On Apr 13, 2017, at 7:52 AM, CY Lim <[hidden email]> wrote:
> >
> > Hello all,
> >
> > I cannot seem to find how to use command line to dump data from GnuCash
> > into a single or multiple plain text files (e.g. csv). Any suggestion
> will
> > be helpful.
> >
> > If that cannot be done with command line, I am happy to use API provided
> in
> > C or Python to access the data. Some examples or documentations are
> > appreciated.
> >
> > I want to avoid manual parsing of the files, unless what I need is not
> > doable with the 2 methods above.
>
>
> There is no export facility available from the command line, and only
> limited export (of transactions and splits to csv) from the GUI. You can of
> course get to any data you want from C and to most of it from Python, and
> there's also https://github.com/sdementen/piecash <
> https://github.com/sdementen/piecash>, a third-party framework for
> working with the XML file. But...
>
> GnuCash uses by default a plain text file in XML, though it may be
> compressed with gzip. You can turn off compression in Preferences, on the
> General tab, or you can gunzip the file; GnuCash doesn't care about
> compression at load time. The not-default storage is in SQL with one of
> SQLite3, MySQL, and PostrgresQL. Most people who want to query their
> GnuCash data outside of GnuCash find that to be the easiest way.
>
> Regards,
> John Ralls
>
> _______________________________________________
> gnucash-user mailing list
> [hidden email]
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> -----
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
>
_______________________________________________
gnucash-user mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-----
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.
Reply | Threaded
Open this post in threaded view
|

Re: Command line to dump data as plain text files

Plutocrat
In reply to this post by CY Lim
[hidden email] wrote on Sunday, 16 April, 2017 08:48 PM:
> I cannot seem to find how to use command line to dump data from GnuCash
> into a single or multiple plain text files (e.g. csv). Any suggestion will
> be helpful.

If you're using sqlite or mysql as the backend, you could use the database client (mysql or sqlite3) to extract and dump data with SQL queries? I guess Python would have tools for interacting with a database too.
If you're using the XML backend its a bit more complicated.

P.
_______________________________________________
gnucash-user mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-----
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.