r20616-20630 (GncOwner)

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

r20616-20630 (GncOwner)

John Ralls-2
Geert,

Where are you trying to get to with these changes?

Regards,
John Ralls

_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: r20616-20630 (GncOwner)

Geert Janssens
On donderdag 12 mei 2011, John Ralls wrote:
> Geert,
>
> Where are you trying to get to with these changes?
>
> Regards,
> John Ralls

My larger goal is to get better integration of the business functions in the
GnuCash UI and logic. In my experience it just takes way too many clicks and
context switches to efficiently work with the business objects. So I'm looking
for ways to improve this.

These recent commits are a first step in that direction. They add overview
pages for business objects such as vendors or customers, just like there is an
overview page for accounts. The idea is that each of these overview pages get
their own toolbar buttons tailored to the type of object that is listed. For
now there's only add/edit vendor/customer/employee when such a page is open,
but I plan to add buttons for at least new invoice/bill/job. In my opinion
that would be a first important improvement to a business workflow where
invoices/bills are entered in batch. For bills, you could open the Vendor's
overview page, and start entering invoices with the click of a (toolbar)
button.
Another useful future enhancement I'm thinking of is adding some kind of a
balance column, which shows the amount each customer is still due or you still
owe each vendor, similar to the account's balance columns.

The whole concept still needs additional polish and refinement obviously. For
example, I'm still evaluating which columns would make sense to be visible by
default, which filter options to provide,...

There is an enhancement request I wrote last year that is mostly related to
this:
https://bugzilla.gnome.org/show_bug.cgi?id=635003

Geert
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: r20616-20630 (GncOwner)

John Ralls-2

On May 12, 2011, at 6:22 AM, Geert Janssens wrote:

> On donderdag 12 mei 2011, John Ralls wrote:
>> Geert,
>>
>> Where are you trying to get to with these changes?
>>
>> Regards,
>> John Ralls
>
> My larger goal is to get better integration of the business functions in the
> GnuCash UI and logic. In my experience it just takes way too many clicks and
> context switches to efficiently work with the business objects. So I'm looking
> for ways to improve this.
>
> These recent commits are a first step in that direction. They add overview
> pages for business objects such as vendors or customers, just like there is an
> overview page for accounts. The idea is that each of these overview pages get
> their own toolbar buttons tailored to the type of object that is listed. For
> now there's only add/edit vendor/customer/employee when such a page is open,
> but I plan to add buttons for at least new invoice/bill/job. In my opinion
> that would be a first important improvement to a business workflow where
> invoices/bills are entered in batch. For bills, you could open the Vendor's
> overview page, and start entering invoices with the click of a (toolbar)
> button.
> Another useful future enhancement I'm thinking of is adding some kind of a
> balance column, which shows the amount each customer is still due or you still
> owe each vendor, similar to the account's balance columns.
>
> The whole concept still needs additional polish and refinement obviously. For
> example, I'm still evaluating which columns would make sense to be visible by
> default, which filter options to provide,...
>
> There is an enhancement request I wrote last year that is mostly related to
> this:
> https://bugzilla.gnome.org/show_bug.cgi?id=635003

OK, sounds reasonable.

My concern is that the data model for business is a bit of a mess, with separate objects for the different flavors of "owner". (I don't even like using "owner" here, but "counterparty", which is technically correct, sounds rather ponderous.) There's no real difference between customers and vendors except the direction of the cash flow. The same is true of bills and invoices. Employees and payroll are quite different, and I rather doubt that Gnucash's handling of payroll is really adequate. I'd intended to rework the design as part of the transactional refactoring after getting unit test coverage in place, but ISTM you should consider it as part of this effort as it will make a big difference in how your overview pages work.

Regards,
John Ralls





_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: r20616-20630 (GncOwner)

Geert Janssens
On donderdag 12 mei 2011, John Ralls wrote:

> On May 12, 2011, at 6:22 AM, Geert Janssens wrote:
> > On donderdag 12 mei 2011, John Ralls wrote:
> >> Geert,
> >>
> >> Where are you trying to get to with these changes?
> >>
> >> Regards,
> >> John Ralls
> >
> > My larger goal is to get better integration of the business functions in
> > the GnuCash UI and logic. In my experience it just takes way too many
> > clicks and context switches to efficiently work with the business
> > objects. So I'm looking for ways to improve this.
> >
> > These recent commits are a first step in that direction. They add
> > overview pages for business objects such as vendors or customers, just
> > like there is an overview page for accounts. The idea is that each of
> > these overview pages get their own toolbar buttons tailored to the type
> > of object that is listed. For now there's only add/edit
> > vendor/customer/employee when such a page is open, but I plan to add
> > buttons for at least new invoice/bill/job. In my opinion that would be a
> > first important improvement to a business workflow where invoices/bills
> > are entered in batch. For bills, you could open the Vendor's overview
> > page, and start entering invoices with the click of a (toolbar) button.
> > Another useful future enhancement I'm thinking of is adding some kind of
> > a balance column, which shows the amount each customer is still due or
> > you still owe each vendor, similar to the account's balance columns.
> >
> > The whole concept still needs additional polish and refinement obviously.
> > For example, I'm still evaluating which columns would make sense to be
> > visible by default, which filter options to provide,...
> >
> > There is an enhancement request I wrote last year that is mostly related
> > to this:
> > https://bugzilla.gnome.org/show_bug.cgi?id=635003
>
> OK, sounds reasonable.
>
> My concern is that the data model for business is a bit of a mess, with
> separate objects for the different flavors of "owner". (I don't even like
> using "owner" here, but "counterparty", which is technically correct,
> sounds rather ponderous.)
Perhaps "businesspartner" or simply "partner" would be a useful alternative.

> There's no real difference between customers and
> vendors except the direction of the cash flow.
I suppose you are talking from an accounting point of view here. In GnuCash
implementation terms there is only one small difference: a customer can have a
shipping address in addition to the customer address.

Other than that I agree that there is so much in common that it could share
much of the code. In fact, some accounting applications don't make that
difference at all, like OpenERP for example. They only have business partners,
and one business partner can be a supplier, a customer or both. It's just a
flag.

> The same is true of bills
> and invoices.
Again I have noticed some differences at the implementation level, though
invoices and bills already share a lot of code. Invoices have discounts in the
invoice register and there's also something with jobs that plays a part there.
I'm sure Derek can give a much more detailed explanation than I can though.

> Employees and payroll are quite different, and I rather
> doubt that Gnucash's handling of payroll is really adequate.
As I came to understand, GnuCash doesn't handle payroll at all. The
functionality tied to employees is only for expense vouchers (what an employee
files when he paid business expences from his personal accounts) trip. So you
are quite right here.

> I'd intended
> to rework the design as part of the transactional refactoring after
> getting unit test coverage in place, but ISTM you should consider it as
> part of this effort as it will make a big difference in how your overview
> pages work.
>
I'm very interested in the improvements you suggest here. I agree this could
result in a much needed cleanup. Just like you, I would prefer to see this
happen after the unit test coverage though.

The unit test coverage is also the subject of the GSoC project you mentor
currently, right ? So I think it would be best to wait for that project to be
integrated in trunk. When unit test coverage is there, I think we may work
together on the transaction based refactoring for the business parts.

When creating the "owner" overview page, I deliberately tried to use the
existing engine objects as they are, partly so because I'm currently mainly
GUI focussed. I had to make some changes and additions, but I didn't want to
touch the structure. I understand that this means I'll have to update this
code once we start refactoring the business objects, but I think that's ok. I
gained some better insight again in some of the business parts and in the way
the gtk gui code is structured. That will surely be useful in the future
refactoring.

Geert
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: r20616-20630 (GncOwner)

Geert Janssens
In reply to this post by John Ralls-2
On donderdag 12 mei 2011, John Ralls wrote:
> I'd intended
> to rework the design as part of the transactional refactoring after
> getting unit test coverage in place, but ISTM you should consider it as
> part of this effort as it will make a big difference in how your overview
> pages work.

Let me zoom in on this part, to clear some things up for me. At this point  
"transactional refactoring" is a little fuzzy to me. I have no idea how you
vision this and what kind of work in would involve. Can you explain this a bit
better ? Perhaps you did somewhere already and I missed it. Just adding a link
to it would suffice.

I'm asking to see how/if I could potentially help.

As I suggested already in my previous mail, I think the unit testing should
take precedence anyway, so I guess the actual refactoring would move to a
later time frame. (I won't have much additional time left anyway in the coming
months).


Geert
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: r20616-20630 (GncOwner)

John Ralls-2

On May 12, 2011, at 11:09 AM, Geert Janssens wrote:

> On donderdag 12 mei 2011, John Ralls wrote:
>> I'd intended
>> to rework the design as part of the transactional refactoring after
>> getting unit test coverage in place, but ISTM you should consider it as
>> part of this effort as it will make a big difference in how your overview
>> pages work.
>
> Let me zoom in on this part, to clear some things up for me. At this point  
> "transactional refactoring" is a little fuzzy to me. I have no idea how you
> vision this and what kind of work in would involve. Can you explain this a bit
> better ? Perhaps you did somewhere already and I missed it. Just adding a link
> to it would suffice.
>
> I'm asking to see how/if I could potentially help.
>
> As I suggested already in my previous mail, I think the unit testing should
> take precedence anyway, so I guess the actual refactoring would move to a
> later time frame. (I won't have much additional time left anyway in the coming
> months).

And on May 12, 2011, at 9:44 AM, Geert Janssens wrote:

> On donderdag 12 mei 2011, John Ralls wrote:
>> On May 12, 2011, at 6:22 AM, Geert Janssens wrote:
>>> On donderdag 12 mei 2011, John Ralls wrote:
>>>> Geert,
>>>>
>>>> Where are you trying to get to with these changes?
>>>>
>>>> Regards,
>>>> John Ralls
>>>
>>> My larger goal is to get better integration of the business functions in
>>> the GnuCash UI and logic. In my experience it just takes way too many
>>> clicks and context switches to efficiently work with the business
>>> objects. So I'm looking for ways to improve this.
>>>
>>> These recent commits are a first step in that direction. They add
>>> overview pages for business objects such as vendors or customers, just
>>> like there is an overview page for accounts. The idea is that each of
>>> these overview pages get their own toolbar buttons tailored to the type
>>> of object that is listed. For now there's only add/edit
>>> vendor/customer/employee when such a page is open, but I plan to add
>>> buttons for at least new invoice/bill/job. In my opinion that would be a
>>> first important improvement to a business workflow where invoices/bills
>>> are entered in batch. For bills, you could open the Vendor's overview
>>> page, and start entering invoices with the click of a (toolbar) button.
>>> Another useful future enhancement I'm thinking of is adding some kind of
>>> a balance column, which shows the amount each customer is still due or
>>> you still owe each vendor, similar to the account's balance columns.
>>>
>>> The whole concept still needs additional polish and refinement obviously.
>>> For example, I'm still evaluating which columns would make sense to be
>>> visible by default, which filter options to provide,...
>>>
>>> There is an enhancement request I wrote last year that is mostly related
>>> to this:
>>> https://bugzilla.gnome.org/show_bug.cgi?id=635003
>>
>> OK, sounds reasonable.
>>
>> My concern is that the data model for business is a bit of a mess, with
>> separate objects for the different flavors of "owner". (I don't even like
>> using "owner" here, but "counterparty", which is technically correct,
>> sounds rather ponderous.)
> Perhaps "businesspartner" or simply "partner" would be a useful alternative.
>
>> There's no real difference between customers and
>> vendors except the direction of the cash flow.
> I suppose you are talking from an accounting point of view here. In GnuCash
> implementation terms there is only one small difference: a customer can have a
> shipping address in addition to the customer address.
>
> Other than that I agree that there is so much in common that it could share
> much of the code. In fact, some accounting applications don't make that
> difference at all, like OpenERP for example. They only have business partners,
> and one business partner can be a supplier, a customer or both. It's just a
> flag.
>
>> The same is true of bills
>> and invoices.
> Again I have noticed some differences at the implementation level, though
> invoices and bills already share a lot of code. Invoices have discounts in the
> invoice register and there's also something with jobs that plays a part there.
> I'm sure Derek can give a much more detailed explanation than I can though.
>
>> Employees and payroll are quite different, and I rather
>> doubt that Gnucash's handling of payroll is really adequate.
> As I came to understand, GnuCash doesn't handle payroll at all. The
> functionality tied to employees is only for expense vouchers (what an employee
> files when he paid business expences from his personal accounts) trip. So you
> are quite right here.
>
>> I'd intended
>> to rework the design as part of the transactional refactoring after
>> getting unit test coverage in place, but ISTM you should consider it as
>> part of this effort as it will make a big difference in how your overview
>> pages work.
>>
> I'm very interested in the improvements you suggest here. I agree this could
> result in a much needed cleanup. Just like you, I would prefer to see this
> happen after the unit test coverage though.
>
> The unit test coverage is also the subject of the GSoC project you mentor
> currently, right ? So I think it would be best to wait for that project to be
> integrated in trunk. When unit test coverage is there, I think we may work
> together on the transaction based refactoring for the business parts.
>
> When creating the "owner" overview page, I deliberately tried to use the
> existing engine objects as they are, partly so because I'm currently mainly
> GUI focussed. I had to make some changes and additions, but I didn't want to
> touch the structure. I understand that this means I'll have to update this
> code once we start refactoring the business objects, but I think that's ok. I
> gained some better insight again in some of the business parts and in the way
> the gtk gui code is structured. That will surely be useful in the future
> refactoring.

Geert,
I was actually thinking from the data model view rather than the accounting view, but ideally the data model closely reflects the problem domain, so they're not dissimilar.

The OpenERP approach (just having a flag) is what I had in mind. Bills and invoices should both have discounts. If a vendor offers a billing performance discount, that should be accounted for (and flagged, to make sure that you pay soon enough to take the discount) just as the discounts you offer to customers are.

But more broadly, Gnucash is supposed to be an object-oriented design, so that all of the common elements are provided by a superclass and any additional specializations (like the pointer to a shipping address for a customer) are provided by a subclass. (That single characteristic is the fundamental flaw of the KVP design: Because KVP data are added to objects outside of the class interface, inheritance and polymorphism are broken.) The goal of the transactional refactoring is that the GUI will be able to request from the backend only the objects it needs to display, to lock only the objects it is editing, to commit the changed edits as a logically atomic commit -- and to have the whole edit atomically rolled back if any part of the commit fails. In order for that to work, objects must have control over all of their member data and be able to serialize it to a standard interface that an arbitrary backend can store and retrieve. That was the original design goal stated for QO!
 F, BTW, but gnucash's implementation of QOF has destroyed that ability and gnucash's use of QOF is inconsistent.

Separately, we have a bunch of GUI projects going on: Gtk3, Qt, and Cocoa (well, somebody posted here that he wanted to write a Cocoa front end, Christian told him how wonderful that would be, but he hasn't been heard from since). For *that* to work out, we need to make sure that the MVC aspect of our design is solid with clear boundaries between the three aspects.

Those two goals motivate my refactoring plan: Make everything in QOF into a class hierarchy with separation of classes by functional and data responsibility and all in the same class system (meaning GSEALED GObject, C++, or Python -- each  of which has advantages, so which one we use is open for discussion, and that discussion can be put off until we have complete unit test coverage in QOF), then refactor Engine, Business Core, and Backend into derived classes from the QOF superclasses. That's the model part of MVC. After that (or in parallel if someone else decides to work on it) we need to ensure that all of the combining of those model objects for presentation to the user and for handling edits (For example, posting an invoice will need to read objects of the Customer, Invoice, Transaction, Split, Account, and Commodity classes, with a write lock on Invoice, Transaction, and Splits) is handled by another set of classes, so that the GUI and API can concern themselves only !
 with user interaction.
 

Having good test coverage is a prerequisite to refactoring.[1]. My GSoC student is going to set up testing for the most important bits of QOF, but all of Engine and Business Core needs to have unit tests, too, so there's plenty of work ahead.

Regards,
John Ralls

_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: r20616-20630 (GncOwner)

Phil Longstaff
On Thu, 2011-05-12 at 13:49 -0400, John Ralls wrote:

> Geert, I was actually thinking from the data model view rather than the
>  accounting view, but ideally the data model closely reflects the
>  problem domain, so they're not dissimilar.
>
[OpenERP - deleted]

> But more broadly, Gnucash is supposed to be an object-oriented design,
>  so that all of the common elements are provided by a superclass and
>  any additional specializations (like the pointer to a shipping address
>  for a customer) are provided by a subclass. (That single
>  characteristic is the fundamental flaw of the KVP design: Because KVP
>  data are added to objects outside of the class interface, inheritance
>  and polymorphism are broken.) The goal of the transactional
>  refactoring is that the GUI will be able to request from the backend
>  only the objects it needs to display, to lock only the objects it is
>  editing, to commit the changed edits as a logically atomic commit --
>  and to have the whole edit atomically rolled back if any part of the
>  commit fails. In order for that to work, objects must have control
>  over all of their member data and be able to serialize it to a
>  standard interface that an arbitrary backend can store and retrieve.
>  That was the original design goal stated for QO!  F, BTW, but
>  gnucash's implementation of QOF has destroyed that ability and
>  gnucash's use of QOF is inconsistent.

[ there are a bunch of GUIs - deleted ]

> Those two goals motivate my refactoring plan: Make everything in QOF
>  into a class hierarchy with separation of classes by functional and
>  data responsibility and all in the same class system (meaning GSEALED
>  GObject, C++, or Python -- each  of which has advantages, so which one
>  we use is open for discussion, and that discussion can be put off
>  until we have complete unit test coverage in QOF), then refactor
>  Engine, Business Core, and Backend into derived classes from the QOF
>  superclasses. That's the model part of MVC. After that (or in parallel
>  if someone else decides to work on it) we need to ensure that all of
>  the combining of those model objects for presentation to the user and
>  for handling edits (For example, posting an invoice will need to read
>  objects of the Customer, Invoice, Transaction, Split, Account, and
>  Commodity classes, with a write lock on Invoice, Transaction, and
>  Splits) is handled by another set of classes, so that the GUI and API
>  can concern themselves only !  with user interaction.  

This all sounds great and I'm wondering what the steps are.  One
original idea behind KVP is that modules can be added to gnucash and can
associate data with the core objects.  An example is a US tax data
module which associates an account with a line on a tax form.  Should
this tax line become part of the Account record in the db?  Might be OK
for US users, but what if a user from a different country needs more
information or different information.  Do we want Account, and
USTaxAccount and OtherCountryTaxAccount derived from it?  What about
other relationship types?

I think it would not be difficult to pull the info currently in slots
out of them and into better data structures which have their own tables.
They could be attached to the main core objects so that they load/save
at the same time but would be independent and in their own tables.  Does
this idea help with your refactoring?

Phil

_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: r20616-20630 (GncOwner)

John Ralls-2

On May 14, 2011, at 4:33 PM, Phil Longstaff wrote:

> On Thu, 2011-05-12 at 13:49 -0400, John Ralls wrote:
>
>> Geert, I was actually thinking from the data model view rather than the
>> accounting view, but ideally the data model closely reflects the
>> problem domain, so they're not dissimilar.
>>
> [OpenERP - deleted]
>
>> But more broadly, Gnucash is supposed to be an object-oriented design,
>> so that all of the common elements are provided by a superclass and
>> any additional specializations (like the pointer to a shipping address
>> for a customer) are provided by a subclass. (That single
>> characteristic is the fundamental flaw of the KVP design: Because KVP
>> data are added to objects outside of the class interface, inheritance
>> and polymorphism are broken.) The goal of the transactional
>> refactoring is that the GUI will be able to request from the backend
>> only the objects it needs to display, to lock only the objects it is
>> editing, to commit the changed edits as a logically atomic commit --
>> and to have the whole edit atomically rolled back if any part of the
>> commit fails. In order for that to work, objects must have control
>> over all of their member data and be able to serialize it to a
>> standard interface that an arbitrary backend can store and retrieve.
>> That was the original design goal stated for QO!  F, BTW, but
>> gnucash's implementation of QOF has destroyed that ability and
>> gnucash's use of QOF is inconsistent.
>
> [ there are a bunch of GUIs - deleted ]
>
>> Those two goals motivate my refactoring plan: Make everything in QOF
>> into a class hierarchy with separation of classes by functional and
>> data responsibility and all in the same class system (meaning GSEALED
>> GObject, C++, or Python -- each  of which has advantages, so which one
>> we use is open for discussion, and that discussion can be put off
>> until we have complete unit test coverage in QOF), then refactor
>> Engine, Business Core, and Backend into derived classes from the QOF
>> superclasses. That's the model part of MVC. After that (or in parallel
>> if someone else decides to work on it) we need to ensure that all of
>> the combining of those model objects for presentation to the user and
>> for handling edits (For example, posting an invoice will need to read
>> objects of the Customer, Invoice, Transaction, Split, Account, and
>> Commodity classes, with a write lock on Invoice, Transaction, and
>> Splits) is handled by another set of classes, so that the GUI and API
>> can concern themselves only !  with user interaction.  
>
> This all sounds great and I'm wondering what the steps are.  One
> original idea behind KVP is that modules can be added to gnucash and can
> associate data with the core objects.  An example is a US tax data
> module which associates an account with a line on a tax form.  Should
> this tax line become part of the Account record in the db?  Might be OK
> for US users, but what if a user from a different country needs more
> information or different information.  Do we want Account, and
> USTaxAccount and OtherCountryTaxAccount derived from it?  What about
> other relationship types?
>
> I think it would not be difficult to pull the info currently in slots
> out of them and into better data structures which have their own tables.
> They could be attached to the main core objects so that they load/save
> at the same time but would be independent and in their own tables.  Does
> this idea help with your refactoring?

The steps I have in mind are:
1. Get complete test coverage for QOF, Engine, Business Core, and Backend
2. Rewrite those libraries into a single, coherent class system (we'll discuss which one when the test-writing gets close to being done)
3. Refactor as necessary to get a good class hierarchy that reflects our acccounting model. This is the hard part, because we very likely keep different models in our heads, and we're going to have to work out how to express those models to each other and work out a common model that we all agree on, then document it so that we can refer to it as we refactor the code. This refactoring is very likely to involve moving parts of classes around, which is the reason for step 2: That's a real PITA if the classes aren't all written in the same "language".

There's more than one way to extend an object. Derivation is the one most people think of first. Another common pattern is a pointer to another class (hierarchy); depending on the semantics that you're trying to model, a pointer *from* another class can effectively extend a class, though that's likely to cause performance issues unless you also use signals (a common practice in Qt, Cocoa, and GObject, though not in mainstream C++ or Python). Note that registering a callback is a form of pointer-to-foreign-object. The important thing is that those pointers-to-foreign-objects are manipulated by class functions, not by foreign objects. That's where the KVP design breaks down: Instead of passing an object with a known interface to a class to store in that class's untyped "expansion data", KVP users are given a pointer to the expansion area and allowed to do whatever they like with it.

Now we can continue to use KVP *in the XML backend* as a way to allow users to open newer-versioned files with old versions of the program (though it's not a common thing to do and I think it's pretty risky). The backend storage logic just needs to be able to figure out which KVP slots go to which core objects or plugin objects. It doesn't belong in the SQL backend, where its recursive design slows everything down and prevents the backend from enforcing referential integrity.

I'm not at all sure that the plugin architecture gets us anything in return for the added complexity, though. AFAIK there aren't any plugins. The various libraries in Gnucash proper that are dloaded instead of being dynamically linked sure doesn't get us anything except longer load times and missed optimization opportunities.

The other important point is that XML datasets and SQL databases have very different structures. Those differences are going to have to be abstracted away in the backend interface, but I think that can be done with a bit of refinement of your design on the SQL side. I haven't studied the XML side enough to know how much work will be involved, but I know how to write an XML backend and an object serialization interface to get us there. The general architecture has serialization/deserialization functions in the core classes which pass a tagged structure to the backend, similar to the structs you hard-coded into the various entity read/write modules in the SQL backends. Each backend maintains a mapping of the structures it knows how to store. Depending on how we decide to handle compatibility between versions, the backend may also have a place like KVP to put data that it doesn't know what to do with or it can raise an exception. One effect of this is that the "scrub" functions!
  will move out of Engine into Backend.

Sorry, that's gotten really long-winded, and I might very well have added to the confusion. If I did, say so, and I'll try again in smaller chunks.

Regards,
John Ralls

_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

RE: r20616-20630 (GncOwner)

Nick Manville
In reply to this post by John Ralls-2
On the particular point about accounts vs tax lines i wonder if it would be better to have a seperate class of 'tax objects' which could reference accounts and/or individual transactions. And then these objects have US or or other territory subclasses. Otherwise, as you hinted at, we could end up with significant fragmentation and duplication of account data and functions and it is unlikely a tax rule will effect how an account interacts with it's transactions within the code base, merely how the account is reported at a different phase. An account could hold references to all the tax-objects of account sub-class that reference it with a maintenance function during any commit phase to keep them congruent.


-----Original Message-----
From: Phil Longstaff
Sent:  14/05/2011 21:33:27
Subject:  Re: r20616-20630 (GncOwner)

On Thu, 2011-05-12 at 13:49 -0400, John Ralls wrote:

> Geert, I was actually thinking from the data model view rather than the
>  accounting view, but ideally the data model closely reflects the
>  problem domain, so they're not dissimilar.
>
[OpenERP - deleted]

> But more broadly, Gnucash is supposed to be an object-oriented design,
>  so that all of the common elements are provided by a superclass and
>  any additional specializations (like the pointer to a shipping address
>  for a customer) are provided by a subclass. (That single
>  characteristic is the fundamental flaw of the KVP design: Because KVP
>  data are added to objects outside of the class interface, inheritance
>  and polymorphism are broken.) The goal of the transactional
>  refactoring is that the GUI will be able to request from the backend
>  only the objects it needs to display, to lock only the objects it is
>  editing, to commit the changed edits as a logically atomic commit --
>  and to have the whole edit atomically rolled back if any part of the
>  commit fails. In order for that to work, objects must have control
>  over all of their member data and be able to serialize it to a
>  standard interface that an arbitrary backend can store and retrieve.
>  That was the original design goal stated for QO!  F, BTW, but
>  gnucash's implementation of QOF has destroyed that ability and
>  gnucash's use of QOF is inconsistent.

[ there are a bunch of GUIs - deleted ]

> Those two goals motivate my refactoring plan: Make everything in QOF
>  into a class hierarchy with separation of classes by functional and
>  data responsibility and all in the same class system (meaning GSEALED
>  GObject, C++, or Python -- each  of which has advantages, so which one
>  we use is open for discussion, and that discussion can be put off
>  until we have complete unit test coverage in QOF), then refactor
>  Engine, Business Core, and Backend into derived classes from the QOF
>  superclasses. That's the model part of MVC. After that (or in parallel
>  if someone else decides to work on it) we need to ensure that all of
>  the combining of those model objects for presentation to the user and
>  for handling edits (For example, posting an invoice will need to read
>  objects of the Customer, Invoice, Transaction, Split, Account, and
>  Commodity classes, with a write lock on Invoice, Transaction, and
>  Splits) is handled by another set of classes, so that the GUI and API
>  can concern themselves only !  with user interaction.  

This all sounds great and I'm wondering what the steps are.  One
original idea behind KVP is that modules can be added to gnucash and can
associate data with the core objects.  An example is a US tax data
module which associates an account with a line on a tax form.  Should
this tax line become part of the Account record in the db?  Might be OK
for US users, but what if a user from a different country needs more
information or different information.  Do we want Account, and
USTaxAccount and OtherCountryTaxAccount derived from it?  What about
other relationship types?

I think it would not be difficult to pull the info currently in slots
out of them and into better data structures which have their own tables.
They could be attached to the main core objects so that they load/save
at the same time but would be independent and in their own tables.  Does
this idea help with your refactoring?

Phil

_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

RE: r20616-20630 (GncOwner)

Nick Manville
In reply to this post by John Ralls-2

The key, I think, is making the object class definitions strict enough to discourage casual contributors from tagging on a sub-class for a specific use case where either a more general amendment would be better or the data/function would be better kept within the module and classes they were concerned with and then referencing existing classes. Difficult to do in a volunteer project I know and also a potential performance hit. With a standardised set of class objects, though, pulling these back in at a later date would not be too difficult, I would have thought to allow progressive optimisation.

-----Original Message-----
From: Nicholas Manville
Sent:  15/05/2011 09:58:31
To: Phil Longstaff; devel gnucash
Subject:  RE: r20616-20630 (GncOwner)

On the particular point about accounts vs tax lines i wonder if it would be better to have a seperate class of 'tax objects' which could reference accounts and/or individual transactions. And then these objects have US or or other territory subclasses. Otherwise, as you hinted at, we could end up with significant fragmentation and duplication of account data and functions and it is unlikely a tax rule will effect how an account interacts with it's transactions within the code base, merely how the account is reported at a different phase. An account could hold references to all the tax-objects of account sub-class that reference it with a maintenance function during any commit phase to keep them congruent.


-----Original Message-----
From: Phil Longstaff
Sent:  14/05/2011 21:33:27
Subject:  Re: r20616-20630 (GncOwner)

On Thu, 2011-05-12 at 13:49 -0400, John Ralls wrote:

> Geert, I was actually thinking from the data model view rather than the
>  accounting view, but ideally the data model closely reflects the
>  problem domain, so they're not dissimilar.
>
[OpenERP - deleted]

> But more broadly, Gnucash is supposed to be an object-oriented design,
>  so that all of the common elements are provided by a superclass and
>  any additional specializations (like the pointer to a shipping address
>  for a customer) are provided by a subclass. (That single
>  characteristic is the fundamental flaw of the KVP design: Because KVP
>  data are added to objects outside of the class interface, inheritance
>  and polymorphism are broken.) The goal of the transactional
>  refactoring is that the GUI will be able to request from the backend
>  only the objects it needs to display, to lock only the objects it is
>  editing, to commit the changed edits as a logically atomic commit --
>  and to have the whole edit atomically rolled back if any part of the
>  commit fails. In order for that to work, objects must have control
>  over all of their member data and be able to serialize it to a
>  standard interface that an arbitrary backend can store and retrieve.
>  That was the original design goal stated for QO!  F, BTW, but
>  gnucash's implementation of QOF has destroyed that ability and
>  gnucash's use of QOF is inconsistent.

[ there are a bunch of GUIs - deleted ]

> Those two goals motivate my refactoring plan: Make everything in QOF
>  into a class hierarchy with separation of classes by functional and
>  data responsibility and all in the same class system (meaning GSEALED
>  GObject, C++, or Python -- each  of which has advantages, so which one
>  we use is open for discussion, and that discussion can be put off
>  until we have complete unit test coverage in QOF), then refactor
>  Engine, Business Core, and Backend into derived classes from the QOF
>  superclasses. That's the model part of MVC. After that (or in parallel
>  if someone else decides to work on it) we need to ensure that all of
>  the combining of those model objects for presentation to the user and
>  for handling edits (For example, posting an invoice will need to read
>  objects of the Customer, Invoice, Transaction, Split, Account, and
>  Commodity classes, with a write lock on Invoice, Transaction, and
>  Splits) is handled by another set of classes, so that the GUI and API
>  can concern themselves only !  with user interaction.  

This all sounds great and I'm wondering what the steps are.  One
original idea behind KVP is that modules can be added to gnucash and can
associate data with the core objects.  An example is a US tax data
module which associates an account with a line on a tax form.  Should
this tax line become part of the Account record in the db?  Might be OK
for US users, but what if a user from a different country needs more
information or different information.  Do we want Account, and
USTaxAccount and OtherCountryTaxAccount derived from it?  What about
other relationship types?

I think it would not be difficult to pull the info currently in slots
out of them and into better data structures which have their own tables.
They could be attached to the main core objects so that they load/save
at the same time but would be independent and in their own tables.  Does
this idea help with your refactoring?

Phil

_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: r20616-20630 (GncOwner)

Phil Longstaff
In reply to this post by John Ralls-2
On Sat, 2011-05-14 at 22:34 -0400, John Ralls wrote:

> On May 14, 2011, at 4:33 PM, Phil Longstaff wrote:
>
> > On Thu, 2011-05-12 at 13:49 -0400, John Ralls wrote:
> >
> >> Geert, I was actually thinking from the data model view rather than the
> >> accounting view, but ideally the data model closely reflects the
> >> problem domain, so they're not dissimilar.
> >>
> > [OpenERP - deleted]
> >
> >> But more broadly, Gnucash is supposed to be an object-oriented design,
> >> so that all of the common elements are provided by a superclass and
> >> any additional specializations (like the pointer to a shipping address
> >> for a customer) are provided by a subclass. (That single
> >> characteristic is the fundamental flaw of the KVP design: Because KVP
> >> data are added to objects outside of the class interface, inheritance
> >> and polymorphism are broken.) The goal of the transactional
> >> refactoring is that the GUI will be able to request from the backend
> >> only the objects it needs to display, to lock only the objects it is
> >> editing, to commit the changed edits as a logically atomic commit --
> >> and to have the whole edit atomically rolled back if any part of the
> >> commit fails. In order for that to work, objects must have control
> >> over all of their member data and be able to serialize it to a
> >> standard interface that an arbitrary backend can store and retrieve.
> >> That was the original design goal stated for QO!  F, BTW, but
> >> gnucash's implementation of QOF has destroyed that ability and
> >> gnucash's use of QOF is inconsistent.
> >
> > [ there are a bunch of GUIs - deleted ]
> >
> >> Those two goals motivate my refactoring plan: Make everything in QOF
> >> into a class hierarchy with separation of classes by functional and
> >> data responsibility and all in the same class system (meaning GSEALED
> >> GObject, C++, or Python -- each  of which has advantages, so which one
> >> we use is open for discussion, and that discussion can be put off
> >> until we have complete unit test coverage in QOF), then refactor
> >> Engine, Business Core, and Backend into derived classes from the QOF
> >> superclasses. That's the model part of MVC. After that (or in parallel
> >> if someone else decides to work on it) we need to ensure that all of
> >> the combining of those model objects for presentation to the user and
> >> for handling edits (For example, posting an invoice will need to read
> >> objects of the Customer, Invoice, Transaction, Split, Account, and
> >> Commodity classes, with a write lock on Invoice, Transaction, and
> >> Splits) is handled by another set of classes, so that the GUI and API
> >> can concern themselves only !  with user interaction.  
> >
> > This all sounds great and I'm wondering what the steps are.  One
> > original idea behind KVP is that modules can be added to gnucash and can
> > associate data with the core objects.  An example is a US tax data
> > module which associates an account with a line on a tax form.  Should
> > this tax line become part of the Account record in the db?  Might be OK
> > for US users, but what if a user from a different country needs more
> > information or different information.  Do we want Account, and
> > USTaxAccount and OtherCountryTaxAccount derived from it?  What about
> > other relationship types?
> >
> > I think it would not be difficult to pull the info currently in slots
> > out of them and into better data structures which have their own tables.
> > They could be attached to the main core objects so that they load/save
> > at the same time but would be independent and in their own tables.  Does
> > this idea help with your refactoring?
>
> The steps I have in mind are:
> 1. Get complete test coverage for QOF, Engine, Business Core, and Backend
> 2. Rewrite those libraries into a single, coherent class system (we'll discuss which one when the test-writing gets close to being done)
> 3. Refactor as necessary to get a good class hierarchy that reflects our acccounting model. This is the hard part, because we very likely keep different models in our heads, and we're going to have to work out how to express those models to each other and work out a common model that we all agree on, then document it so that we can refer to it as we refactor the code. This refactoring is very likely to involve moving parts of classes around, which is the reason for step 2: That's a real PITA if the classes aren't all written in the same "language".
>
> There's more than one way to extend an object. Derivation is the one most people think of first. Another common pattern is a pointer to another class (hierarchy); depending on the semantics that you're trying to model, a pointer *from* another class can effectively extend a class, though that's likely to cause performance issues unless you also use signals (a common practice in Qt, Cocoa, and GObject, though not in mainstream C++ or Python). Note that registering a callback is a form of pointer-to-foreign-object. The important thing is that those pointers-to-foreign-objects are manipulated by class functions, not by foreign objects. That's where the KVP design breaks down: Instead of passing an object with a known interface to a class to store in that class's untyped "expansion data", KVP users are given a pointer to the expansion area and allowed to do whatever they like with it.
>
> Now we can continue to use KVP *in the XML backend* as a way to allow users to open newer-versioned files with old versions of the program (though it's not a common thing to do and I think it's pretty risky). The backend storage logic just needs to be able to figure out which KVP slots go to which core objects or plugin objects. It doesn't belong in the SQL backend, where its recursive design slows everything down and prevents the backend from enforcing referential integrity.
>
> I'm not at all sure that the plugin architecture gets us anything in return for the added complexity, though. AFAIK there aren't any plugins. The various libraries in Gnucash proper that are dloaded instead of being dynamically linked sure doesn't get us anything except longer load times and missed optimization opportunities.
>
> The other important point is that XML datasets and SQL databases have very different structures. Those differences are going to have to be abstracted away in the backend interface, but I think that can be done with a bit of refinement of your design on the SQL side. I haven't studied the XML side enough to know how much work will be involved, but I know how to write an XML backend and an object serialization interface to get us there. The general architecture has serialization/deserialization functions in the core classes which pass a tagged structure to the backend, similar to the structs you hard-coded into the various entity read/write modules in the SQL backends. Each backend maintains a mapping of the structures it knows how to store. Depending on how we decide to handle compatibility between versions, the backend may also have a place like KVP to put data that it doesn't know what to do with or it can raise an exception. One effect of this is that the "scrub" functio!
 ns will move out of Engine into Backend.
>
> Sorry, that's gotten really long-winded, and I might very well have added to the confusion. If I did, say so, and I'll try again in smaller chunks.

I'm looking at what to work on next.  I have a few ideas:

1) I want to refactor the preferences system (I wrote an e-mail quite a
few months ago about this).  My thought was to have a preferences
interface similar to the java one (a hierarchy of nodes.  A node can
have child nodes similar to a directory structure.  A node can also have
leaf entries).  This common interface would provide access to general
preferences (currently in gconf on all platforms but could be in windows
registry on a windows system or in macos native config system on mac),
per-file preferences (currently KVP of the book which also has a
directory/leaf structure).  We could also add per-user-per-book
preferences.

2) Object extensions.  My idea here is that for each object class (Lot,
Account, ...), extension classes can be registered (for Account, could
be tax related, for example).  The extension interface would have
functions called when an object is created, destroyed, loaded, saved.
This would fix the "grab an object and stick stuff into its kvp" which
currently exists and formalize the interface connecting the extended
data to its main object.  The load/save could store info in kvp, or for
sql, could store in a new table indexed by main object guid.

3) Lots of report work - expand use of eguile/expand use of
css/javascript - formalize/normalize report engine.

I'm not sure what itch bothers me most right now.  I'm happy to let you
(John) take the lead on refactoring the main core.  I also see myself
more interested in the foundation of a software system rather than its
UI, so I'm not really interested in redoing UI.

Phil

_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: plugins (was: r20616-20630 (GncOwner)

Christian Stimming-4
In reply to this post by John Ralls-2
Am Sonntag, 15. Mai 2011 schrieb John Ralls:

> The steps I have in mind are:
> 1. Get complete test coverage for QOF, Engine, Business Core, and Backend
> 2. Rewrite those libraries into a single, coherent class system (we'll
> discuss which one when the test-writing gets close to being done) 3.
> Refactor as necessary to get a good class hierarchy that reflects our
> acccounting model. This is the hard part, because we very likely keep
> different models in our heads, and we're going to have to work out how to
> express those models to each other and work out a common model that we all
> agree on, then document it so that we can refer to it as we refactor the
> code.

Sounds very good. I'm all in :-)

> I'm not at all sure that the plugin architecture gets us anything in return
> for the added complexity, though. AFAIK there aren't any plugins. The
> various libraries in Gnucash proper that are dloaded instead of being
> dynamically linked sure doesn't get us anything except longer load times
> and missed optimization opportunities.

Are you asking whether gnucash should contain any of its functions as plugin
vs. no plugins at all? I agree (and I have stated before) that most of the
loadable modules of gnucash shouldn't be modules, but should instead belong
into one big binary and that's it. I have stated a similar goal for the
cutecash experiment.

However, after thinking about this again, one particular use case for plugins
is still around: Unusual dependencies from code parts that are interesting
only to a subset of the users. Specifically, the aqbanking code part is
interesting only to a small subset of users, and it requires a whole lot of
other dependencies (aqbanking, gwenhywfar and various sub-libraries of those).
Having gnucash's aqbanking module still around as a loadable module means the
packaging can more this into a separate package, so that users who don't want
the online banking features also don't need its dependencies.

Again, I'm all in for throwing out most of the current loadable modules. But
in some use cases they are still useful.

Regards,

Christian
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: r20616-20630 (GncOwner)

Alex Aycinena-2
In reply to this post by John Ralls-2
Phil,


> ---------- Forwarded message ----------
> From: Phil Longstaff <[hidden email]>
> To: devel gnucash <[hidden email]>
> Date: Sat, 14 May 2011 16:33:27 -0400
> Subject: Re: r20616-20630 (GncOwner)
> On Thu, 2011-05-12 at 13:49 -0400, John Ralls wrote:
>

snip

> This all sounds great and I'm wondering what the steps are.  One
> original idea behind KVP is that modules can be added to gnucash and can
> associate data with the core objects.  An example is a US tax data
> module which associates an account with a line on a tax form.  Should
> this tax line become part of the Account record in the db?  Might be OK
> for US users, but what if a user from a different country needs more
> information or different information.  Do we want Account, and
> USTaxAccount and OtherCountryTaxAccount derived from it?  What about
> other relationship types?
>

A different thought that has floated around deals with 'tagging'
accounts (and perhaps, individual transactions, too) with values that
correspond to one or more user-defined hierarchies (this has come up
with reference to having a capability sort of like 'classes' within
Quicken). This would be analogous to, and could share capabilities
with, the existing hierarchy used for accounts but would function in
parallel to it and would primarily drive reporting and searching
capabilities, although it might have an impact on data entry and
validation, too. I always thought that this approach could be
generalized enough to be part of the core and fit into the
restructuring/refactoring that is being talked about.

Then, using this capability, in addition to user-defined hierarchies,
there could be developer-defined hierarchies distributed with the
system in cases where someone stepped up to initially set it up and
maintain it. One of these would be a replacement for the existing
KVP-based US Income Tax system (and possibly, that for Germany if
someone volunteered to do it). Using the existing tax system(s) as
guides, other volunteers could set them up and maintain them for other
tax jurisdictions. Then we would have a system where users could use
none, one, or more tax reporting capabilities at the same time, as
there individual situation required. It would fit into a more
generalized capability and wouldn't be an exclusive, either-or,
location based system like we have now.

> I think it would not be difficult to pull the info currently in slots
> out of them and into better data structures which have their own tables.
> They could be attached to the main core objects so that they load/save
> at the same time but would be independent and in their own tables.  Does
> this idea help with your refactoring?
>
> Phil
>
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: r20616-20630 (GncOwner)

John Ralls-2
In reply to this post by Phil Longstaff

On May 15, 2011, at 10:19 AM, Phil Longstaff wrote:

> I'm looking at what to work on next.  I have a few ideas:
>
> 1) I want to refactor the preferences system (I wrote an e-mail quite a
> few months ago about this).  My thought was to have a preferences
> interface similar to the java one (a hierarchy of nodes.  A node can
> have child nodes similar to a directory structure.  A node can also have
> leaf entries).  This common interface would provide access to general
> preferences (currently in gconf on all platforms but could be in windows
> registry on a windows system or in macos native config system on mac),
> per-file preferences (currently KVP of the book which also has a
> directory/leaf structure).  We could also add per-user-per-book
> preferences.
>
> 2) Object extensions.  My idea here is that for each object class (Lot,
> Account, ...), extension classes can be registered (for Account, could
> be tax related, for example).  The extension interface would have
> functions called when an object is created, destroyed, loaded, saved.
> This would fix the "grab an object and stick stuff into its kvp" which
> currently exists and formalize the interface connecting the extended
> data to its main object.  The load/save could store info in kvp, or for
> sql, could store in a new table indexed by main object guid.
>
> 3) Lots of report work - expand use of eguile/expand use of
> css/javascript - formalize/normalize report engine.
>
> I'm not sure what itch bothers me most right now.  I'm happy to let you
> (John) take the lead on refactoring the main core.  I also see myself
> more interested in the foundation of a software system rather than its
> UI, so I'm not really interested in redoing UI.

I think (2) needs to wait until we've at least got a good handle on  what the post-refactoring structure is going to look like.

There are several ways to go with preferences, and which one we choose might depend on how tightly we want to stay integrated with Gnome. GLib now provides a generic dbus-like interface which platform devs are expected to write backends for. The most recent versions of GConf now talk to gdbus instead of dbus. GLib also provides a low-level config facility called gsettings, but it's not intended to replace GConf except in the simplest cases. Alternatives are to write a shim class that can interface to the configuration facility of whatever toolkit someone wants to use for a frontend or to just do our own configuration facility and forget about the toolkits in the interest of portablility... that mirrors the alternatives for choosing the class system for the backend... it would be a pain for you to go to spend the summer writing a GObject-based configuration facility only for us to decide in the fall to rewrite the core in Python in which you could have written the whole thing!
  in a week.

Then there's reports. I know that Scheme and Guile are popular with some devs, but I think that it's a serious barrier to both recruiting new devs and to users being able to write their own reports. Since we get a javascript interpreter for free with WebKit, and it's therefore pretty widely understood, I think rewriting the report system in javascript would be a major improvement for Gnucash. (I'm going to crawl into my foxhole now and await the inevitable incoming.)

Regards,
John Ralls

_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: plugins

John Ralls-2
In reply to this post by Christian Stimming-4

On May 15, 2011, at 12:04 PM, Christian Stimming wrote:

> Am Sonntag, 15. Mai 2011 schrieb John Ralls:
>
>> The steps I have in mind are:
>> 1. Get complete test coverage for QOF, Engine, Business Core, and Backend
>> 2. Rewrite those libraries into a single, coherent class system (we'll
>> discuss which one when the test-writing gets close to being done) 3.
>> Refactor as necessary to get a good class hierarchy that reflects our
>> acccounting model. This is the hard part, because we very likely keep
>> different models in our heads, and we're going to have to work out how to
>> express those models to each other and work out a common model that we all
>> agree on, then document it so that we can refer to it as we refactor the
>> code.
>
> Sounds very good. I'm all in :-)
>
>> I'm not at all sure that the plugin architecture gets us anything in return
>> for the added complexity, though. AFAIK there aren't any plugins. The
>> various libraries in Gnucash proper that are dloaded instead of being
>> dynamically linked sure doesn't get us anything except longer load times
>> and missed optimization opportunities.
>
> Are you asking whether gnucash should contain any of its functions as plugin
> vs. no plugins at all? I agree (and I have stated before) that most of the
> loadable modules of gnucash shouldn't be modules, but should instead belong
> into one big binary and that's it. I have stated a similar goal for the
> cutecash experiment.
>
> However, after thinking about this again, one particular use case for plugins
> is still around: Unusual dependencies from code parts that are interesting
> only to a subset of the users. Specifically, the aqbanking code part is
> interesting only to a small subset of users, and it requires a whole lot of
> other dependencies (aqbanking, gwenhywfar and various sub-libraries of those).
> Having gnucash's aqbanking module still around as a loadable module means the
> packaging can more this into a separate package, so that users who don't want
> the online banking features also don't need its dependencies.
>
> Again, I'm all in for throwing out most of the current loadable modules. But
> in some use cases they are still useful.

Hmm. I can see your point, but isn't there a lot of import-export functionality wrapped up in aqbanking? OTOH, if we could do a good plugin API, maybe people would come forward with more import/export plugins. Conversely, HCBI users wouldn't need to install the libofx plugin.

The use-case extends to backends, too.

So we agree that it would be nice to have. That doesn't answer the question about whether it's worth the extra complexity.

Regards,
John Ralls


_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: r20616-20630 (GncOwner)

Derek Atkins
In reply to this post by John Ralls-2
John Ralls <[hidden email]> writes:

> I'm not at all sure that the plugin architecture gets us anything in
> return for the added complexity, though. AFAIK there aren't any
> plugins. The various libraries in Gnucash proper that are dloaded
> instead of being dynamically linked sure doesn't get us anything
> except longer load times and missed optimization opportunities.

Technically the business features were designed to be a plugin.  When I
originally worked on that code a decade ago my idea was that packagers
could build a "gnucash" app and then supply a secondary
"gnucash-business" plugin that would supply all the business code, GUIs,
etc.   Obviously that never happened, but that WAS the original design.

-derek

--
       Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
       Member, MIT Student Information Processing Board  (SIPB)
       URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
       [hidden email]                        PGP key available
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: r20616-20630 (GncOwner)

John Ralls-2

On May 16, 2011, at 6:10 AM, Derek Atkins wrote:

> John Ralls <[hidden email]> writes:
>
>> I'm not at all sure that the plugin architecture gets us anything in
>> return for the added complexity, though. AFAIK there aren't any
>> plugins. The various libraries in Gnucash proper that are dloaded
>> instead of being dynamically linked sure doesn't get us anything
>> except longer load times and missed optimization opportunities.
>
> Technically the business features were designed to be a plugin.  When I
> originally worked on that code a decade ago my idea was that packagers
> could build a "gnucash" app and then supply a secondary
> "gnucash-business" plugin that would supply all the business code, GUIs,
> etc.   Obviously that never happened, but that WAS the original design.
>

Then why are the data objects in src/engine? Did they get moved there later?

Regardless, is there a good reason to keep (or restore) that architecture? It fails the test Christian advocated for aqbanking, because the business stuff has no special dependencies. It's also difficult to separate the data objects into an optional module and maintain referential integrity across the module interface.

Regards,
John Ralls


_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: r20616-20630 (GncOwner)

Phil Longstaff
Yes.  They were moved there later

 Phil
---------
I used to be a hypochondriac AND a kleptomaniac. So I took something for it.




________________________________
From: John Ralls <[hidden email]>
To: Derek Atkins <[hidden email]>
Cc: devel gnucash <[hidden email]>
Sent: Mon, May 16, 2011 10:11:07 AM
Subject: Re: r20616-20630 (GncOwner)


On May 16, 2011, at 6:10 AM, Derek Atkins wrote:

> John Ralls <[hidden email]> writes:
>
>> I'm not at all sure that the plugin architecture gets us anything in
>> return for the added complexity, though. AFAIK there aren't any
>> plugins. The various libraries in Gnucash proper that are dloaded
>> instead of being dynamically linked sure doesn't get us anything
>> except longer load times and missed optimization opportunities.
>
> Technically the business features were designed to be a plugin.  When I
> originally worked on that code a decade ago my idea was that packagers
> could build a "gnucash" app and then supply a secondary
> "gnucash-business" plugin that would supply all the business code, GUIs,
> etc.   Obviously that never happened, but that WAS the original design.
>

Then why are the data objects in src/engine? Did they get moved there later?

Regardless, is there a good reason to keep (or restore) that architecture? It
fails the test Christian advocated for aqbanking, because the business stuff has
no special dependencies. It's also difficult to separate the data objects into
an optional module and maintain referential integrity across the module
interface.

Regards,
John Ralls


_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Future GnuCash plans (was: Re: r20616-20630 (GncOwner))

Geert Janssens
In reply to this post by John Ralls-2
On zondag 15 mei 2011, John Ralls wrote:

> On May 14, 2011, at 4:33 PM, Phil Longstaff wrote:
> > On Thu, 2011-05-12 at 13:49 -0400, John Ralls wrote:
> >> Geert, I was actually thinking from the data model view rather than the
> >> accounting view, but ideally the data model closely reflects the
> >> problem domain, so they're not dissimilar.
> >
> > [OpenERP - deleted]
> >
> >> But more broadly, Gnucash is supposed to be an object-oriented design,
> >> so that all of the common elements are provided by a superclass and
> >> any additional specializations (like the pointer to a shipping address
> >> for a customer) are provided by a subclass. (That single
> >> characteristic is the fundamental flaw of the KVP design: Because KVP
> >> data are added to objects outside of the class interface, inheritance
> >> and polymorphism are broken.) The goal of the transactional
> >> refactoring is that the GUI will be able to request from the backend
> >> only the objects it needs to display, to lock only the objects it is
> >> editing, to commit the changed edits as a logically atomic commit --
> >> and to have the whole edit atomically rolled back if any part of the
> >> commit fails. In order for that to work, objects must have control
> >> over all of their member data and be able to serialize it to a
> >> standard interface that an arbitrary backend can store and retrieve.
> >> That was the original design goal stated for QO!  F, BTW, but
> >> gnucash's implementation of QOF has destroyed that ability and
> >> gnucash's use of QOF is inconsistent.
> >
> > [ there are a bunch of GUIs - deleted ]
> >
> >> Those two goals motivate my refactoring plan: Make everything in QOF
> >> into a class hierarchy with separation of classes by functional and
> >> data responsibility and all in the same class system (meaning GSEALED
> >> GObject, C++, or Python -- each  of which has advantages, so which one
> >> we use is open for discussion, and that discussion can be put off
> >> until we have complete unit test coverage in QOF), then refactor
> >> Engine, Business Core, and Backend into derived classes from the QOF
> >> superclasses. That's the model part of MVC. After that (or in parallel
> >> if someone else decides to work on it) we need to ensure that all of
> >> the combining of those model objects for presentation to the user and
> >> for handling edits (For example, posting an invoice will need to read
> >> objects of the Customer, Invoice, Transaction, Split, Account, and
> >> Commodity classes, with a write lock on Invoice, Transaction, and
> >> Splits) is handled by another set of classes, so that the GUI and API
> >> can concern themselves only !  with user interaction.
> >
> > This all sounds great and I'm wondering what the steps are.  One
> > original idea behind KVP is that modules can be added to gnucash and can
> > associate data with the core objects.  An example is a US tax data
> > module which associates an account with a line on a tax form.  Should
> > this tax line become part of the Account record in the db?  Might be OK
> > for US users, but what if a user from a different country needs more
> > information or different information.  Do we want Account, and
> > USTaxAccount and OtherCountryTaxAccount derived from it?  What about
> > other relationship types?
> >
> > I think it would not be difficult to pull the info currently in slots
> > out of them and into better data structures which have their own tables.
> > They could be attached to the main core objects so that they load/save
> > at the same time but would be independent and in their own tables.  Does
> > this idea help with your refactoring?
>
> The steps I have in mind are:
> 1. Get complete test coverage for QOF, Engine, Business Core, and Backend
> 2. Rewrite those libraries into a single, coherent class system (we'll
> discuss which one when the test-writing gets close to being done) 3.
> Refactor as necessary to get a good class hierarchy that reflects our
> acccounting model. This is the hard part, because we very likely keep
> different models in our heads, and we're going to have to work out how to
> express those models to each other and work out a common model that we all
> agree on, then document it so that we can refer to it as we refactor the
> code. This refactoring is very likely to involve moving parts of classes
> around, which is the reason for step 2: That's a real PITA if the classes
> aren't all written in the same "language".
>
This seems like a good plan to me. A huge one also. From how I understand it,
this plan is close to a complete rewrite of GnuCash' core. Just correct me if
I'm wrong.

> There's more than one way to extend an object....
I have no particular opinion on how this is done as I lack experience in that
area. So I'll agree with whatever the others decide here.

> I'm not at all sure that the plugin architecture gets us anything in return
> for the added complexity, though. AFAIK there aren't any plugins. The
> various libraries in Gnucash proper that are dloaded instead of being
> dynamically linked sure doesn't get us anything except longer load times
> and missed optimization opportunities.
>
I fully agree with Christian's opinion here (we discussed this in the past on
the list). Plugins could be useful for corner cases, but not for core
functionality. So we could keep the concept of a plugin, but in my opinion it
needs work. The current implementation for example prevents good integration
between the import and business features. Simple things like marking an
invoice paid (business plugin) by a payment on a bank statement (import
plugin) is not possible.

Geert
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|

Re: r20616-20630 (GncOwner)

Geert Janssens
In reply to this post by John Ralls-2
On maandag 16 mei 2011, John Ralls wrote:

> On May 16, 2011, at 6:10 AM, Derek Atkins wrote:
> > John Ralls <[hidden email]> writes:
> >> I'm not at all sure that the plugin architecture gets us anything in
> >> return for the added complexity, though. AFAIK there aren't any
> >> plugins. The various libraries in Gnucash proper that are dloaded
> >> instead of being dynamically linked sure doesn't get us anything
> >> except longer load times and missed optimization opportunities.
> >
> > Technically the business features were designed to be a plugin.  When I
> > originally worked on that code a decade ago my idea was that packagers
> > could build a "gnucash" app and then supply a secondary
> > "gnucash-business" plugin that would supply all the business code, GUIs,
> > etc.   Obviously that never happened, but that WAS the original design.
>
> Then why are the data objects in src/engine? Did they get moved there
> later?
>
Christian moved them into the engine because both the python interface and
cutecash would loose the saved business data when manipulating a datafile that
had such data.

> Regardless, is there a good reason to keep (or restore) that architecture?
> It fails the test Christian advocated for aqbanking, because the business
> stuff has no special dependencies. It's also difficult to separate the
> data objects into an optional module and maintain referential integrity
> across the module interface.
>
I understand Derek's original intention to create a business pluging, but in
practice I see more drawbacks than advantages in keeping the business features
in a separate module. There has never been a distribution that created a
business-free GnuCash build with a separate gnucash-business module. That
would risk data loss as I explained above.

Geert
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
12