Online banking

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Online banking

Paul Tirk
Hello!

I was thinking about adding online banking support for a bank which has
no HBCI (and is not supported by AqBanking). After some time browsing
the source of gnucash and AqBanking I realized that the online banking
functionality of gnucash is tightly coupled to AqBanking which in turn
has a restricted license and it actually doesn't look easy to add
different banks.

Are there plans on a clean online banking API in gnucash which could
enable developers to include different banking protocols? If not I would
be interested in contributing but I have no idea where to start. Could
somebody maybe give me a hint about this?

My idea would be: the current menu entries for online banking are fine,
it would just need a general user/account setup which could allow the
current AqBanking accounts as well as other "plugins/modules" which can
then be triggered as it is working right now with only AqBanking.

I hope I made myself clear and thanks in advance,

Paul
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: Online banking

Derek Atkins-3
Hi,

Yes, the GnuCash AqB backend is tied to AqB.  However...

AqBanking is licensed under the GPL, and already has multiple modules that
implement HBCI, OFX, OFX-DC, MT940, CSV, ...

So, why do you feel that you couldn't add a new module to that?

Do you know what online banking protocol your bank supports?  Or do you
have the mistaken impression that it's a website scraper?

-derek

On Sun, September 27, 2015 2:23 pm, Paul Tirk wrote:

> Hello!
>
> I was thinking about adding online banking support for a bank which has
> no HBCI (and is not supported by AqBanking). After some time browsing
> the source of gnucash and AqBanking I realized that the online banking
> functionality of gnucash is tightly coupled to AqBanking which in turn
> has a restricted license and it actually doesn't look easy to add
> different banks.
>
> Are there plans on a clean online banking API in gnucash which could
> enable developers to include different banking protocols? If not I would
> be interested in contributing but I have no idea where to start. Could
> somebody maybe give me a hint about this?
>
> My idea would be: the current menu entries for online banking are fine,
> it would just need a general user/account setup which could allow the
> current AqBanking accounts as well as other "plugins/modules" which can
> then be triggered as it is working right now with only AqBanking.
>
> I hope I made myself clear and thanks in advance,
>
> Paul
> _______________________________________________
> gnucash-devel mailing list
> [hidden email]
> https://lists.gnucash.org/mailman/listinfo/gnucash-devel
>


--
       Derek Atkins                 617-623-3745
       [hidden email]             www.ihtfp.com
       Computer and Internet Security Consultant

_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: Online banking

Paul Tirk
Hi,

no, I know that it's not a website scraper but in fact I want to program
one ;) And I assume the developer of AqBanking will not just add any set
of api-calls/https-requests for every other bank to his library.

So I was thinking about a generic interface in Gnucash to simplify
things because I also think it would be beneficial for future
development to not depend completely and only on AqBanking like it is
now because it seems that (at least in Europe) the trend is going away
from standards (except HBCI in Germany) towards bank-specific APIs (most
of them are https-requests sent).

Regards,

Paul



Am 27.09.2015 20:55, schrieb Derek Atkins:

> Hi,
>
> Yes, the GnuCash AqB backend is tied to AqB.  However...
>
> AqBanking is licensed under the GPL, and already has multiple modules
> that
> implement HBCI, OFX, OFX-DC, MT940, CSV, ...
>
> So, why do you feel that you couldn't add a new module to that?
>
> Do you know what online banking protocol your bank supports?  Or do you
> have the mistaken impression that it's a website scraper?
>
> -derek
>
> On Sun, September 27, 2015 2:23 pm, Paul Tirk wrote:
>> Hello!
>>
>> I was thinking about adding online banking support for a bank which
>> has
>> no HBCI (and is not supported by AqBanking). After some time browsing
>> the source of gnucash and AqBanking I realized that the online banking
>> functionality of gnucash is tightly coupled to AqBanking which in turn
>> has a restricted license and it actually doesn't look easy to add
>> different banks.
>>
>> Are there plans on a clean online banking API in gnucash which could
>> enable developers to include different banking protocols? If not I
>> would
>> be interested in contributing but I have no idea where to start. Could
>> somebody maybe give me a hint about this?
>>
>> My idea would be: the current menu entries for online banking are
>> fine,
>> it would just need a general user/account setup which could allow the
>> current AqBanking accounts as well as other "plugins/modules" which
>> can
>> then be triggered as it is working right now with only AqBanking.
>>
>> I hope I made myself clear and thanks in advance,
>>
>> Paul
>> _______________________________________________
>> gnucash-devel mailing list
>> [hidden email]
>> https://lists.gnucash.org/mailman/listinfo/gnucash-devel
>>
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: Online banking

John Ralls-2

> On Sep 27, 2015, at 3:24 PM, Paul Tirk <[hidden email]> wrote:
>
> Hi,
>
> no, I know that it's not a website scraper but in fact I want to program one ;) And I assume the developer of AqBanking will not just add any set of api-calls/https-requests for every other bank to his library.
>
> So I was thinking about a generic interface in Gnucash to simplify things because I also think it would be beneficial for future development to not depend completely and only on AqBanking like it is now because it seems that (at least in Europe) the trend is going away from standards (except HBCI in Germany) towards bank-specific APIs (most of them are https-requests sent).

Paul,

Take a look at the import/export API, src/import-export. You’ve probably also noticed that we already have a dependency on WebKitGtk that should work for retrieval and DOM parsing. Good luck.

Regards,
John Ralls


_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: Online banking

Christian Stimming-4
In reply to this post by Paul Tirk
Dear Paul,

I don't think it is possible or useful to think of "yet another generic API".
Instead, I would suggest the API of aqbanking is indeed the 4th or 5th
iteration on building an API from the application to online banking functions.
Also, I would strongly suggest against programming a website scraper - it will
give you very little gain for very large effort. If your bank doesn't have
anything else besides a website, in fact the easiest solution is to find a
different bank that has some standard protocol such as OFX. Sorry for that.

Regards,

Christian

Am Montag, 28. September 2015, 00:24:02 schrieb Paul Tirk:

> Hi,
>
> no, I know that it's not a website scraper but in fact I want to program
> one ;) And I assume the developer of AqBanking will not just add any set
> of api-calls/https-requests for every other bank to his library.
>
> So I was thinking about a generic interface in Gnucash to simplify
> things because I also think it would be beneficial for future
> development to not depend completely and only on AqBanking like it is
> now because it seems that (at least in Europe) the trend is going away
> from standards (except HBCI in Germany) towards bank-specific APIs (most
> of them are https-requests sent).
>
> Regards,
>
> Paul
>
> Am 27.09.2015 20:55, schrieb Derek Atkins:
> > Hi,
> >
> > Yes, the GnuCash AqB backend is tied to AqB.  However...
> >
> > AqBanking is licensed under the GPL, and already has multiple modules
> > that
> > implement HBCI, OFX, OFX-DC, MT940, CSV, ...
> >
> > So, why do you feel that you couldn't add a new module to that?
> >
> > Do you know what online banking protocol your bank supports?  Or do you
> > have the mistaken impression that it's a website scraper?
> >
> > -derek
> >
> > On Sun, September 27, 2015 2:23 pm, Paul Tirk wrote:
> >> Hello!
> >>
> >> I was thinking about adding online banking support for a bank which
> >> has
> >> no HBCI (and is not supported by AqBanking). After some time browsing
> >> the source of gnucash and AqBanking I realized that the online banking
> >> functionality of gnucash is tightly coupled to AqBanking which in turn
> >> has a restricted license and it actually doesn't look easy to add
> >> different banks.
> >>
> >> Are there plans on a clean online banking API in gnucash which could
> >> enable developers to include different banking protocols? If not I
> >> would
> >> be interested in contributing but I have no idea where to start. Could
> >> somebody maybe give me a hint about this?
> >>
> >> My idea would be: the current menu entries for online banking are
> >> fine,
> >> it would just need a general user/account setup which could allow the
> >> current AqBanking accounts as well as other "plugins/modules" which
> >> can
> >> then be triggered as it is working right now with only AqBanking.
> >>
> >> I hope I made myself clear and thanks in advance,
> >>
> >> Paul
> >> _______________________________________________
> >> gnucash-devel mailing list
> >> [hidden email]
> >> https://lists.gnucash.org/mailman/listinfo/gnucash-devel
>
> _______________________________________________
> gnucash-devel mailing list
> [hidden email]
> https://lists.gnucash.org/mailman/listinfo/gnucash-devel

_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: Online banking

Paul Tirk
Dear Christian,

I understand your point, but I am not actually trying to scrape a
webpage but rather using an existing API of the bank to return the
transactions in JSON format.

In a perfect world, there would be a global standard for everyone but
unfortunately in Europe banks and financial institutions seem not to be
interested to provide such a interface for the end user (except HBCI in
Germany, in Austria the discussion about this is already going on for at
least 10 years and it seems impossible to get closer to a solution).
Instead I have the impression that they are trying to abandon all
efforts on this and force the user/client to use their proprietary
programs/interfaces.

But as John pointed out, I will have a look at the import-export source
and create a plugin for that, maybe it can be useful to others later.

Thank you all for your suggestions.

Paul



Am 28.09.2015 08:40, schrieb Christian Stimming:

> Dear Paul,
>
> I don't think it is possible or useful to think of "yet another generic
> API".
> Instead, I would suggest the API of aqbanking is indeed the 4th or 5th
> iteration on building an API from the application to online banking
> functions.
> Also, I would strongly suggest against programming a website scraper -
> it will
> give you very little gain for very large effort. If your bank doesn't
> have
> anything else besides a website, in fact the easiest solution is to
> find a
> different bank that has some standard protocol such as OFX. Sorry for
> that.
>
> Regards,
>
> Christian
>
> Am Montag, 28. September 2015, 00:24:02 schrieb Paul Tirk:
>> Hi,
>>
>> no, I know that it's not a website scraper but in fact I want to
>> program
>> one ;) And I assume the developer of AqBanking will not just add any
>> set
>> of api-calls/https-requests for every other bank to his library.
>>
>> So I was thinking about a generic interface in Gnucash to simplify
>> things because I also think it would be beneficial for future
>> development to not depend completely and only on AqBanking like it is
>> now because it seems that (at least in Europe) the trend is going away
>> from standards (except HBCI in Germany) towards bank-specific APIs
>> (most
>> of them are https-requests sent).
>>
>> Regards,
>>
>> Paul
>>
>> Am 27.09.2015 20:55, schrieb Derek Atkins:
>> > Hi,
>> >
>> > Yes, the GnuCash AqB backend is tied to AqB.  However...
>> >
>> > AqBanking is licensed under the GPL, and already has multiple modules
>> > that
>> > implement HBCI, OFX, OFX-DC, MT940, CSV, ...
>> >
>> > So, why do you feel that you couldn't add a new module to that?
>> >
>> > Do you know what online banking protocol your bank supports?  Or do you
>> > have the mistaken impression that it's a website scraper?
>> >
>> > -derek
>> >
>> > On Sun, September 27, 2015 2:23 pm, Paul Tirk wrote:
>> >> Hello!
>> >>
>> >> I was thinking about adding online banking support for a bank which
>> >> has
>> >> no HBCI (and is not supported by AqBanking). After some time browsing
>> >> the source of gnucash and AqBanking I realized that the online banking
>> >> functionality of gnucash is tightly coupled to AqBanking which in turn
>> >> has a restricted license and it actually doesn't look easy to add
>> >> different banks.
>> >>
>> >> Are there plans on a clean online banking API in gnucash which could
>> >> enable developers to include different banking protocols? If not I
>> >> would
>> >> be interested in contributing but I have no idea where to start. Could
>> >> somebody maybe give me a hint about this?
>> >>
>> >> My idea would be: the current menu entries for online banking are
>> >> fine,
>> >> it would just need a general user/account setup which could allow the
>> >> current AqBanking accounts as well as other "plugins/modules" which
>> >> can
>> >> then be triggered as it is working right now with only AqBanking.
>> >>
>> >> I hope I made myself clear and thanks in advance,
>> >>
>> >> Paul
>> >> _______________________________________________
>> >> gnucash-devel mailing list
>> >> [hidden email]
>> >> https://lists.gnucash.org/mailman/listinfo/gnucash-devel
>>
>> _______________________________________________
>> gnucash-devel mailing list
>> [hidden email]
>> https://lists.gnucash.org/mailman/listinfo/gnucash-devel
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: Online banking

Bob Gustafson
In reply to this post by Christian Stimming-4
I would second that recommendation.

A web scraper solution is captive to any random changes that the bank
makes to its website - incurring a need to work on your parser before it
'works' again.

Changes to bank web sites seem to come at the worst possible moment in
terms of your own (developer) workload.

Even if a bank allows transaction downloads in various different formats
(MT940, CSV(??),CAMT,..) the standards of individual banks and
individual accounts can change over time (character set, whether to use
UE or u-umlaut, all upper case vs upper/lower). Misspelling of names and
other user-entered data is sometimes an unavoidable complication.

Data input is a complicated  game.

Have fun

Bob G

On 09/28/2015 01:40 AM, Christian Stimming wrote:

> Dear Paul,
>
> I don't think it is possible or useful to think of "yet another generic API".
> Instead, I would suggest the API of aqbanking is indeed the 4th or 5th
> iteration on building an API from the application to online banking functions.
> Also, I would strongly suggest against programming a website scraper - it will
> give you very little gain for very large effort. If your bank doesn't have
> anything else besides a website, in fact the easiest solution is to find a
> different bank that has some standard protocol such as OFX. Sorry for that.
>
> Regards,
>
> Christian
>
> Am Montag, 28. September 2015, 00:24:02 schrieb Paul Tirk:
>> Hi,
>>
>> no, I know that it's not a website scraper but in fact I want to program
>> one ;) And I assume the developer of AqBanking will not just add any set
>> of api-calls/https-requests for every other bank to his library.
>>
>> So I was thinking about a generic interface in Gnucash to simplify
>> things because I also think it would be beneficial for future
>> development to not depend completely and only on AqBanking like it is
>> now because it seems that (at least in Europe) the trend is going away
>> from standards (except HBCI in Germany) towards bank-specific APIs (most
>> of them are https-requests sent).
>>
>> Regards,
>>
>> Paul
>>
>> Am 27.09.2015 20:55, schrieb Derek Atkins:
>>> Hi,
>>>
>>> Yes, the GnuCash AqB backend is tied to AqB.  However...
>>>
>>> AqBanking is licensed under the GPL, and already has multiple modules
>>> that
>>> implement HBCI, OFX, OFX-DC, MT940, CSV, ...
>>>
>>> So, why do you feel that you couldn't add a new module to that?
>>>
>>> Do you know what online banking protocol your bank supports?  Or do you
>>> have the mistaken impression that it's a website scraper?
>>>
>>> -derek
>>>
>>> On Sun, September 27, 2015 2:23 pm, Paul Tirk wrote:
>>>> Hello!
>>>>
>>>> I was thinking about adding online banking support for a bank which
>>>> has
>>>> no HBCI (and is not supported by AqBanking). After some time browsing
>>>> the source of gnucash and AqBanking I realized that the online banking
>>>> functionality of gnucash is tightly coupled to AqBanking which in turn
>>>> has a restricted license and it actually doesn't look easy to add
>>>> different banks.
>>>>
>>>> Are there plans on a clean online banking API in gnucash which could
>>>> enable developers to include different banking protocols? If not I
>>>> would
>>>> be interested in contributing but I have no idea where to start. Could
>>>> somebody maybe give me a hint about this?
>>>>
>>>> My idea would be: the current menu entries for online banking are
>>>> fine,
>>>> it would just need a general user/account setup which could allow the
>>>> current AqBanking accounts as well as other "plugins/modules" which
>>>> can
>>>> then be triggered as it is working right now with only AqBanking.
>>>>
>>>> I hope I made myself clear and thanks in advance,
>>>>
>>>> Paul
>>>> _______________________________________________
>>>> gnucash-devel mailing list
>>>> [hidden email]
>>>> https://lists.gnucash.org/mailman/listinfo/gnucash-devel
>> _______________________________________________
>> gnucash-devel mailing list
>> [hidden email]
>> https://lists.gnucash.org/mailman/listinfo/gnucash-devel
> _______________________________________________
> gnucash-devel mailing list
> [hidden email]
> https://lists.gnucash.org/mailman/listinfo/gnucash-devel

_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: Online banking

Derek Atkins
In reply to this post by Paul Tirk
Paul,

Paul Tirk <[hidden email]> writes:

> Dear Christian,
>
> I understand your point, but I am not actually trying to scrape a
> webpage but rather using an existing API of the bank to return the
> transactions in JSON format.
>
> In a perfect world, there would be a global standard for everyone but
> unfortunately in Europe banks and financial institutions seem not to
> be interested to provide such a interface for the end user (except
> HBCI in Germany, in Austria the discussion about this is already going
> on for at least 10 years and it seems impossible to get closer to a
> solution). Instead I have the impression that they are trying to
> abandon all efforts on this and force the user/client to use their
> proprietary programs/interfaces.
>
> But as John pointed out, I will have a look at the import-export
> source and create a plugin for that, maybe it can be useful to others
> later.

I would recommend you either consider another AqB JSON plugin, or a
generic GnuCash JSON plugin.  I'm not sure which approach would be
easier to implement.  AqB does already have a bunch of code for account
management and network I/O, as well as the complete(ish) integration
into the GnuCash UI.  So, it might be easier to plug in there, where all
you really have to do is the data layer.

> Thank you all for your suggestions.
>
> Paul

-derek

> Am 28.09.2015 08:40, schrieb Christian Stimming:
>> Dear Paul,
>>
>> I don't think it is possible or useful to think of "yet another
>> generic API".
>> Instead, I would suggest the API of aqbanking is indeed the 4th or 5th
>> iteration on building an API from the application to online banking
>> functions.
>> Also, I would strongly suggest against programming a website scraper
>> -
>> it will
>> give you very little gain for very large effort. If your bank
>> doesn't have
>> anything else besides a website, in fact the easiest solution is to
>> find a
>> different bank that has some standard protocol such as OFX. Sorry
>> for that.
>>
>> Regards,
>>
>> Christian
>>
>> Am Montag, 28. September 2015, 00:24:02 schrieb Paul Tirk:
>>> Hi,
>>>
>>> no, I know that it's not a website scraper but in fact I want to
>>> program
>>> one ;) And I assume the developer of AqBanking will not just add
>>> any set
>>> of api-calls/https-requests for every other bank to his library.
>>>
>>> So I was thinking about a generic interface in Gnucash to simplify
>>> things because I also think it would be beneficial for future
>>> development to not depend completely and only on AqBanking like it is
>>> now because it seems that (at least in Europe) the trend is going away
>>> from standards (except HBCI in Germany) towards bank-specific APIs
>>> (most
>>> of them are https-requests sent).
>>>
>>> Regards,
>>>
>>> Paul
>>>
>>> Am 27.09.2015 20:55, schrieb Derek Atkins:
>>> > Hi,
>>> >
>>> > Yes, the GnuCash AqB backend is tied to AqB.  However...
>>> >
>>> > AqBanking is licensed under the GPL, and already has multiple modules
>>> > that
>>> > implement HBCI, OFX, OFX-DC, MT940, CSV, ...
>>> >
>>> > So, why do you feel that you couldn't add a new module to that?
>>> >
>>> > Do you know what online banking protocol your bank supports?  Or do you
>>> > have the mistaken impression that it's a website scraper?
>>> >
>>> > -derek
>>> >
>>> > On Sun, September 27, 2015 2:23 pm, Paul Tirk wrote:
>>> >> Hello!
>>> >>
>>> >> I was thinking about adding online banking support for a bank which
>>> >> has
>>> >> no HBCI (and is not supported by AqBanking). After some time browsing
>>> >> the source of gnucash and AqBanking I realized that the online banking
>>> >> functionality of gnucash is tightly coupled to AqBanking which in turn
>>> >> has a restricted license and it actually doesn't look easy to add
>>> >> different banks.
>>> >>
>>> >> Are there plans on a clean online banking API in gnucash which could
>>> >> enable developers to include different banking protocols? If not I
>>> >> would
>>> >> be interested in contributing but I have no idea where to start. Could
>>> >> somebody maybe give me a hint about this?
>>> >>
>>> >> My idea would be: the current menu entries for online banking are
>>> >> fine,
>>> >> it would just need a general user/account setup which could allow the
>>> >> current AqBanking accounts as well as other "plugins/modules" which
>>> >> can
>>> >> then be triggered as it is working right now with only AqBanking.
>>> >>
>>> >> I hope I made myself clear and thanks in advance,
>>> >>
>>> >> Paul
>>> >> _______________________________________________
>>> >> gnucash-devel mailing list
>>> >> [hidden email]
>>> >> https://lists.gnucash.org/mailman/listinfo/gnucash-devel
>>>
>>> _______________________________________________
>>> gnucash-devel mailing list
>>> [hidden email]
>>> https://lists.gnucash.org/mailman/listinfo/gnucash-devel
>
>

--
       Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
       Member, MIT Student Information Processing Board  (SIPB)
       URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
       [hidden email]                        PGP key available
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: Online banking

Paul Tirk
Hello,

Am 30.09.2015 17:04, schrieb Derek Atkins:

> Paul,
>
> I would recommend you either consider another AqB JSON plugin, or a
> generic GnuCash JSON plugin.  I'm not sure which approach would be
> easier to implement.  AqB does already have a bunch of code for account
> management and network I/O, as well as the complete(ish) integration
> into the GnuCash UI.  So, it might be easier to plug in there, where
> all
> you really have to do is the data layer.
>
> -derek

Thank you Derek for your suggestions, it indeed sounds reasonable to
plug into the AqBanking library
because of the already existing account handling and the integration
into the UI.

Thanks again to all for your helpful replies and keep up the good work!

Paul
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel