Source directory restructuring

classic Classic list List threaded Threaded
31 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Source directory restructuring

Geert Janssens-4
So after my houskeeping message in which I have announced the changes to src/
business and src/libqof, I'd like to bring up my eventual goal for the source
tree.

My main motivation to do all this restructuring is to simplify. There are
currently plenty of directories and I often have to guess where to expect a
file. The qof vs engine story was one example. Is gnc-date something for qof
or for engine ? I find myself regularly searching for a file in the wrong
directory.

So here follows a first proposal for the directory structure I'm targetting:

* data (for things like art, checks, account hierarchy templates,...)
* doc (for all documenation)
* lib (for all source required to build "libgnucash", see below)
* ui (for all the user interfaces the project currently supports)

I am omitting some smaller directories here, such as util and macros. They
will probably stay on the current top level for now.

I'm envisioning "libgnucash" as the core library that encapsulates all gnucash
related concepts (the accounting concepts such as transaction, split, as well
as the data backends). This library is what all applications in the gnucash
sphere should depend on to implement the "gnucash" experience. The most
obvious is of course the current gnucash as we know it. However at some point
this library should ideally also become the core of the Android app and *who
knows* one day an IOS app. And closer to the current state, it should also be
the library that the guile and python bindings depend on. So all functionality
encapsulated in one single library, the API. In practice I think the following
directories belong in this libgnucash: core-utils, gnc-module, engine, the
backends, app-utils. (Note I would like to get rid of gnc-module still, but
that's a whole other discussion and task).

The ui directory will have a subdirectory for each user interface we support.
This is not necessarily a *graphical* user interface though. At this point
there are already a number of them:
gnucash
cutecash
bindings (with subdirectories for python and guile)

The bulk of the other directories are support directories for one of the ui's
and I propose to make them subdirectories of each respective ui.

For example gtkmm is only used by cutecash, so let's store it there (until
another ui would also require it, which I consider unlikely to happen still).

The other directories (gnome-utils, gnome-search, gnome, register, html,
report, import-export,...) are all used only in the gnucash application and
hence should be moved there. In the move I'd like to try and reduce the 3
gnome* directories to one and call it gtk as we're not using any gnome
specific technology any more.

In a later phase I think it would be nice if the core libgnucash could also
spit out html reports, but that would require us to refactor the report
modules, which I consider too much work to be done at the same time. When this
eventually gets done the non-ui part of the report system can then be added in
libgnucash to the benefit of all consumers (gnucash/cutecash/Gnc4Android/...).

Feedback or questions ?

Geert
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Source directory restructuring

Derek Atkins
Sounds like a great plan to me!!!

-derek

Geert Janssens <[hidden email]> writes:

> So after my houskeeping message in which I have announced the changes to src/
> business and src/libqof, I'd like to bring up my eventual goal for the source
> tree.
>
> My main motivation to do all this restructuring is to simplify. There are
> currently plenty of directories and I often have to guess where to expect a
> file. The qof vs engine story was one example. Is gnc-date something for qof
> or for engine ? I find myself regularly searching for a file in the wrong
> directory.
>
> So here follows a first proposal for the directory structure I'm targetting:
>
> * data (for things like art, checks, account hierarchy templates,...)
> * doc (for all documenation)
> * lib (for all source required to build "libgnucash", see below)
> * ui (for all the user interfaces the project currently supports)
>
> I am omitting some smaller directories here, such as util and macros. They
> will probably stay on the current top level for now.
>
> I'm envisioning "libgnucash" as the core library that encapsulates all gnucash
> related concepts (the accounting concepts such as transaction, split, as well
> as the data backends). This library is what all applications in the gnucash
> sphere should depend on to implement the "gnucash" experience. The most
> obvious is of course the current gnucash as we know it. However at some point
> this library should ideally also become the core of the Android app and *who
> knows* one day an IOS app. And closer to the current state, it should also be
> the library that the guile and python bindings depend on. So all functionality
> encapsulated in one single library, the API. In practice I think the following
> directories belong in this libgnucash: core-utils, gnc-module, engine, the
> backends, app-utils. (Note I would like to get rid of gnc-module still, but
> that's a whole other discussion and task).
>
> The ui directory will have a subdirectory for each user interface we support.
> This is not necessarily a *graphical* user interface though. At this point
> there are already a number of them:
> gnucash
> cutecash
> bindings (with subdirectories for python and guile)
>
> The bulk of the other directories are support directories for one of the ui's
> and I propose to make them subdirectories of each respective ui.
>
> For example gtkmm is only used by cutecash, so let's store it there (until
> another ui would also require it, which I consider unlikely to happen still).
>
> The other directories (gnome-utils, gnome-search, gnome, register, html,
> report, import-export,...) are all used only in the gnucash application and
> hence should be moved there. In the move I'd like to try and reduce the 3
> gnome* directories to one and call it gtk as we're not using any gnome
> specific technology any more.
>
> In a later phase I think it would be nice if the core libgnucash could also
> spit out html reports, but that would require us to refactor the report
> modules, which I consider too much work to be done at the same time. When this
> eventually gets done the non-ui part of the report system can then be added in
> libgnucash to the benefit of all consumers (gnucash/cutecash/Gnc4Android/...).
>
> Feedback or questions ?
>
> Geert
> _______________________________________________
> gnucash-devel mailing list
> [hidden email]
> https://lists.gnucash.org/mailman/listinfo/gnucash-devel
>
>

--
       Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
       Member, MIT Student Information Processing Board  (SIPB)
       URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
       [hidden email]                        PGP key available
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Source directory restructuring

Sumit Bhardwaj
This sounds like a great plan to me. It will definitely make understanding
the code easier than the current state.

-Sumit

On Mon, Aug 7, 2017 at 7:23 AM, Derek Atkins <[hidden email]> wrote:

> Sounds like a great plan to me!!!
>
> -derek
>
> Geert Janssens <[hidden email]> writes:
>
> > So after my houskeeping message in which I have announced the changes to
> src/
> > business and src/libqof, I'd like to bring up my eventual goal for the
> source
> > tree.
> >
> > My main motivation to do all this restructuring is to simplify. There are
> > currently plenty of directories and I often have to guess where to
> expect a
> > file. The qof vs engine story was one example. Is gnc-date something for
> qof
> > or for engine ? I find myself regularly searching for a file in the wrong
> > directory.
> >
> > So here follows a first proposal for the directory structure I'm
> targetting:
> >
> > * data (for things like art, checks, account hierarchy templates,...)
> > * doc (for all documenation)
> > * lib (for all source required to build "libgnucash", see below)
> > * ui (for all the user interfaces the project currently supports)
> >
> > I am omitting some smaller directories here, such as util and macros.
> They
> > will probably stay on the current top level for now.
> >
> > I'm envisioning "libgnucash" as the core library that encapsulates all
> gnucash
> > related concepts (the accounting concepts such as transaction, split, as
> well
> > as the data backends). This library is what all applications in the
> gnucash
> > sphere should depend on to implement the "gnucash" experience. The most
> > obvious is of course the current gnucash as we know it. However at some
> point
> > this library should ideally also become the core of the Android app and
> *who
> > knows* one day an IOS app. And closer to the current state, it should
> also be
> > the library that the guile and python bindings depend on. So all
> functionality
> > encapsulated in one single library, the API. In practice I think the
> following
> > directories belong in this libgnucash: core-utils, gnc-module, engine,
> the
> > backends, app-utils. (Note I would like to get rid of gnc-module still,
> but
> > that's a whole other discussion and task).
> >
> > The ui directory will have a subdirectory for each user interface we
> support.
> > This is not necessarily a *graphical* user interface though. At this
> point
> > there are already a number of them:
> > gnucash
> > cutecash
> > bindings (with subdirectories for python and guile)
> >
> > The bulk of the other directories are support directories for one of the
> ui's
> > and I propose to make them subdirectories of each respective ui.
> >
> > For example gtkmm is only used by cutecash, so let's store it there
> (until
> > another ui would also require it, which I consider unlikely to happen
> still).
> >
> > The other directories (gnome-utils, gnome-search, gnome, register, html,
> > report, import-export,...) are all used only in the gnucash application
> and
> > hence should be moved there. In the move I'd like to try and reduce the 3
> > gnome* directories to one and call it gtk as we're not using any gnome
> > specific technology any more.
> >
> > In a later phase I think it would be nice if the core libgnucash could
> also
> > spit out html reports, but that would require us to refactor the report
> > modules, which I consider too much work to be done at the same time.
> When this
> > eventually gets done the non-ui part of the report system can then be
> added in
> > libgnucash to the benefit of all consumers (gnucash/cutecash/Gnc4Android/
> ...).
> >
> > Feedback or questions ?
> >
> > Geert
> > _______________________________________________
> > gnucash-devel mailing list
> > [hidden email]
> > https://lists.gnucash.org/mailman/listinfo/gnucash-devel
> >
> >
>
> --
>        Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
>        Member, MIT Student Information Processing Board  (SIPB)
>        URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
>        [hidden email]                        PGP key available
> _______________________________________________
> gnucash-devel mailing list
> [hidden email]
> https://lists.gnucash.org/mailman/listinfo/gnucash-devel
>
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Source directory restructuring

John Ralls
In reply to this post by Geert Janssens-4

> On Aug 7, 2017, at 2:56 PM, Geert Janssens <[hidden email]> wrote:
>
> So after my houskeeping message in which I have announced the changes to src/
> business and src/libqof, I'd like to bring up my eventual goal for the source
> tree.
>
> My main motivation to do all this restructuring is to simplify. There are
> currently plenty of directories and I often have to guess where to expect a
> file. The qof vs engine story was one example. Is gnc-date something for qof
> or for engine ? I find myself regularly searching for a file in the wrong
> directory.
>
> So here follows a first proposal for the directory structure I'm targetting:
>
> * data (for things like art, checks, account hierarchy templates,...)
> * doc (for all documenation)
> * lib (for all source required to build "libgnucash", see below)
> * ui (for all the user interfaces the project currently supports)
>
> I am omitting some smaller directories here, such as util and macros. They
> will probably stay on the current top level for now.
>
> I'm envisioning "libgnucash" as the core library that encapsulates all gnucash
> related concepts (the accounting concepts such as transaction, split, as well
> as the data backends). This library is what all applications in the gnucash
> sphere should depend on to implement the "gnucash" experience. The most
> obvious is of course the current gnucash as we know it. However at some point
> this library should ideally also become the core of the Android app and *who
> knows* one day an IOS app. And closer to the current state, it should also be
> the library that the guile and python bindings depend on. So all functionality
> encapsulated in one single library, the API. In practice I think the following
> directories belong in this libgnucash: core-utils, gnc-module, engine, the
> backends, app-utils. (Note I would like to get rid of gnc-module still, but
> that's a whole other discussion and task).
>
> The ui directory will have a subdirectory for each user interface we support.
> This is not necessarily a *graphical* user interface though. At this point
> there are already a number of them:
> gnucash
> cutecash
> bindings (with subdirectories for python and guile)
>
> The bulk of the other directories are support directories for one of the ui's
> and I propose to make them subdirectories of each respective ui.
>
> For example gtkmm is only used by cutecash, so let's store it there (until
> another ui would also require it, which I consider unlikely to happen still).
>
> The other directories (gnome-utils, gnome-search, gnome, register, html,
> report, import-export,...) are all used only in the gnucash application and
> hence should be moved there. In the move I'd like to try and reduce the 3
> gnome* directories to one and call it gtk as we're not using any gnome
> specific technology any more.
>
> In a later phase I think it would be nice if the core libgnucash could also
> spit out html reports, but that would require us to refactor the report
> modules, which I consider too much work to be done at the same time. When this
> eventually gets done the non-ui part of the report system can then be added in
> libgnucash to the benefit of all consumers (gnucash/cutecash/Gnc4Android/...).
>
> Feedback or questions ?

Geert,

These are to be top-level directories, so the current src goes away, right? The rest assumes that to be the case.

Let's call the libgnucash directory libgnucash instead of just lib and keep lib as the place put code pinched from elsewhere like the bits of libgoffice.

I want to get rid of gnc-module too, but it's not easy.

App-utils has stuff that belongs in libgnucash and other stuff (e.g. options) that belongs in the application directory (which I propose calling "gnucash" below). There's also code splattered all over everywhere else that belongs in libgnucash. How much of a prerequisite to rearrangement is getting the code in the right place?

I think that having separate documentation directories (there are two, doc and src/doc) is an abject failure from a maintenance standpoint. A lot of stuff was written 15-20 years ago and was marked obsolete 10 years ago because it's been left to rot. All documentation needs to be in the source files as Doxygen comments. We should look through the docs and src/docs stuff and copy anything that's still relevant into comments in the appropriate source files and delete the docs directories.

I'd prefer that the application code go into a directory named gnucash rather than "ui". I'm inclined toward removing cutecash entirely. It made sense for Christian to keep it in the Gnucash repository when we were using SVN, but now he should just publish it in his own Github repo.

The bindings are mostly (and should probably be entirely) libgnucash bindings. They really belong in libgnucash or failing that in their own TLD, not part of the application.

Reports might stand on their own as a separate dependency of gnucash or they could be a subdirectory of gnucash. The same is true of import-export. In one approach  the GUI parts of each would be part of the application while the functional bits would be part of libgnucash. Are the matchers libgnucash or gnucash?

Gtk+ is very much Gnome and depends on other pieces of the Gnome project. The actual name of the directory isn't really important so we can call it gtk if you'd rather, but there are lots of other Gnome project dependencies that aren't going away as long as we're using Gtk+. Remember that in addition to gnome, gnome-util, and gnome-query there's also register-gnome, report-gnome, and (until your latest changes) business-gnome. There's also Gtk+ code in import-export, html, and probably a few other places that don't immediately come to mind. Setting that aside we still have all of that GValue and g_object_set/get code that we use for communicating between the engine, the backends, and the importers.

I'll point out once again that there's an immense amount of untangling to separate the actual UI code from glue and actual bits of libgnucash that are all mangled together in the various gnome directories.

Does it really make sense to push much further with this for 2.8? I'd like to move towards releasing that as quickly as we can, then we can move on to extraction and rearrangement.

Regards,
John Ralls


_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Source directory restructuring

Geert Janssens-4
On maandag 7 augustus 2017 21:27:18 CEST John Ralls wrote:
> These are to be top-level directories, so the current src goes away, right?
> The rest assumes that to be the case.
>
Yes, I would drop src.

> Let's call the libgnucash directory libgnucash instead of just lib and keep
> lib as the place put code pinched from elsewhere like the bits of
> libgoffice.
>
That's reasonable.

> I want to get rid of gnc-module too, but it's not easy.
>
Indeed. I didn't plan this in the first phase. Except perhaps for some low
hanging fruit (like what I did with a couple of reports subdirectories) I
don't think this is something to target for 2.8.

> App-utils has stuff that belongs in libgnucash and other stuff (e.g.
> options) that belongs in the application directory (which I propose calling
> "gnucash" below). There's also code splattered all over everywhere else
> that belongs in libgnucash. How much of a prerequisite to rearrangement is
> getting the code in the right place?
>
There's a gui and non-gui aspect to the options. The gui aspect clearly should
go to the "gnucash" part. The non-gui aspect probably not. In my long term
idea libgnucash should be able to generate reports (but not display them) to
allow the bindings (and eventual smartphone apps) access to this feature. As
things stand now, the report system requires the options. We all agree of
course the options are currently a complicated mixture of C and guile code
which needs serious modernization. That will not be for 2.8.

But your general message stands. Lots of code is in the wrong functional
source group. And there definitely won't be time to reorganize all of that.

> I think that having separate documentation directories (there are two, doc
> and src/doc) is an abject failure from a maintenance standpoint. A lot of
> stuff was written 15-20 years ago and was marked obsolete 10 years ago
> because it's been left to rot. All documentation needs to be in the source
> files as Doxygen comments. We should look through the docs and src/docs
> stuff and copy anything that's still relevant into comments in the
> appropriate source files and delete the docs directories.
>
Good idea.

> I'd prefer that the application code go into a directory named gnucash
> rather than "ui". I'm inclined toward removing cutecash entirely. It made
> sense for Christian to keep it in the Gnucash repository when we were using
> SVN, but now he should just publish it in his own Github repo.
>
I realize I'm somewhat hesitant to drop cutecash from the repo. It looks I'm a
bit emotionally attached to it (although I never used it and only built it
once to test). That's probably because it's a decent first attempt to use qt
instead of gtk. And qt for me is one of the few options that can bring us one
code base for all form factors. (Drifting off into dreaming about Utopia...)

Anyway sticking with the current situation I believe you are right to consider
removing it. It's not maintained, and based on the soon to be obsolete Qt4 and
most importantly it can be recovered should we ever want to do so or are ready
to go the Qt way and need a base to start from.

> The bindings are mostly (and should probably be entirely) libgnucash
> bindings. They really belong in libgnucash or failing that in their own
> TLD, not part of the application.
>
TLD then.

> Reports might stand on their own as a separate dependency of gnucash or they
> could be a subdirectory of gnucash. The same is true of import-export. In
> one approach  the GUI parts of each would be part of the application while
> the functional bits would be part of libgnucash. Are the matchers
> libgnucash or gnucash?
>
Long term I would like to see the functional part of both reports and import/
export be part of libgnucash and the gui's go into gnucash. I think there are
use cases where smartphone apps and the bindings would benefit from this.

For 2.8 that would again be too much work. For this initial step I'd consider
the plugins directory to be the location to store self-contained, loadable
modules. There are already a few smaller ones in there. I'd add the report and
import/export directories in there as well until we're ready to split them up.
My work on the csv importer is an important step in the right direction. For
the next major cycle I hope to continue this for the other importers and
exporters as well.

> Gtk+ is very much Gnome and depends on other pieces of the Gnome project.
> The actual name of the directory isn't really important so we can call it
> gtk if you'd rather, but there are lots of other Gnome project dependencies
> that aren't going away as long as we're using Gtk+. Remember that in
> addition to gnome, gnome-util, and gnome-query there's also register-gnome,
> report-gnome, and (until your latest changes) business-gnome. There's also
> Gtk+ code in import-export, html, and probably a few other places that
> don't immediately come to mind. Setting that aside we still have all of
> that GValue and g_object_set/get code that we use for communicating between
> the engine, the backends, and the importers.
>
> I'll point out once again that there's an immense amount of untangling to
> separate the actual UI code from glue and actual bits of libgnucash that
> are all mangled together in the various gnome directories.
>
All true. I preferred gtk as that's the name of the toolkit the code is based
on, whereas gnome these days refers more to a user community and a desktop
environment.

> Does it really make sense to push much further with this for 2.8? I'd like
> to move towards releasing that as quickly as we can, then we can move on to
> extraction and rearrangement.

I share your concern about getting 2.8 out as soon as possible. I'm trying to
balance this concern with a potential maintenance burden in the next major
cycle. If we do the directory shuffling early in the next major cycle the 2.8
(then to be maint branch) and the new master branch will have a rather
diverging source directory structure which means developers have to keep two
structures in their head for one major cycle.

I understand it's impossible to clean it all up in the few weeks we have left
before we start the 2.8 end game. So I don't expect to be able to do more than
some very basic top-level restructuring. My hope is this will more clearly set
the intention for each source directory and may help in deciding which code
should go where. And 2.8 maintenance and new master will have the same basic
source structure reducing some of the confusion.

The real uncluttering will be for the next major cycle.

Geert
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Source directory restructuring

John Ralls

> On Aug 8, 2017, at 10:29 AM, Geert Janssens <[hidden email]> wrote:
>
> On maandag 7 augustus 2017 21:27:18 CEST John Ralls wrote:
>> These are to be top-level directories, so the current src goes away, right?
>> The rest assumes that to be the case.
>>
> Yes, I would drop src.
>
>> Let's call the libgnucash directory libgnucash instead of just lib and keep
>> lib as the place put code pinched from elsewhere like the bits of
>> libgoffice.
>>
> That's reasonable.
>
>> I want to get rid of gnc-module too, but it's not easy.
>>
> Indeed. I didn't plan this in the first phase. Except perhaps for some low
> hanging fruit (like what I did with a couple of reports subdirectories) I
> don't think this is something to target for 2.8.
>
>> App-utils has stuff that belongs in libgnucash and other stuff (e.g.
>> options) that belongs in the application directory (which I propose calling
>> "gnucash" below). There's also code splattered all over everywhere else
>> that belongs in libgnucash. How much of a prerequisite to rearrangement is
>> getting the code in the right place?
>>
> There's a gui and non-gui aspect to the options. The gui aspect clearly should
> go to the "gnucash" part. The non-gui aspect probably not. In my long term
> idea libgnucash should be able to generate reports (but not display them) to
> allow the bindings (and eventual smartphone apps) access to this feature. As
> things stand now, the report system requires the options. We all agree of
> course the options are currently a complicated mixture of C and guile code
> which needs serious modernization. That will not be for 2.8.
>
> But your general message stands. Lots of code is in the wrong functional
> source group. And there definitely won't be time to reorganize all of that.
>
>> I think that having separate documentation directories (there are two, doc
>> and src/doc) is an abject failure from a maintenance standpoint. A lot of
>> stuff was written 15-20 years ago and was marked obsolete 10 years ago
>> because it's been left to rot. All documentation needs to be in the source
>> files as Doxygen comments. We should look through the docs and src/docs
>> stuff and copy anything that's still relevant into comments in the
>> appropriate source files and delete the docs directories.
>>
> Good idea.
>
>> I'd prefer that the application code go into a directory named gnucash
>> rather than "ui". I'm inclined toward removing cutecash entirely. It made
>> sense for Christian to keep it in the Gnucash repository when we were using
>> SVN, but now he should just publish it in his own Github repo.
>>
> I realize I'm somewhat hesitant to drop cutecash from the repo. It looks I'm a
> bit emotionally attached to it (although I never used it and only built it
> once to test). That's probably because it's a decent first attempt to use qt
> instead of gtk. And qt for me is one of the few options that can bring us one
> code base for all form factors. (Drifting off into dreaming about Utopia...)
>
> Anyway sticking with the current situation I believe you are right to consider
> removing it. It's not maintained, and based on the soon to be obsolete Qt4 and
> most importantly it can be recovered should we ever want to do so or are ready
> to go the Qt way and need a base to start from.
>
>> The bindings are mostly (and should probably be entirely) libgnucash
>> bindings. They really belong in libgnucash or failing that in their own
>> TLD, not part of the application.
>>
> TLD then.
>
>> Reports might stand on their own as a separate dependency of gnucash or they
>> could be a subdirectory of gnucash. The same is true of import-export. In
>> one approach  the GUI parts of each would be part of the application while
>> the functional bits would be part of libgnucash. Are the matchers
>> libgnucash or gnucash?
>>
> Long term I would like to see the functional part of both reports and import/
> export be part of libgnucash and the gui's go into gnucash. I think there are
> use cases where smartphone apps and the bindings would benefit from this.
>
> For 2.8 that would again be too much work. For this initial step I'd consider
> the plugins directory to be the location to store self-contained, loadable
> modules. There are already a few smaller ones in there. I'd add the report and
> import/export directories in there as well until we're ready to split them up.
> My work on the csv importer is an important step in the right direction. For
> the next major cycle I hope to continue this for the other importers and
> exporters as well.
>
>> Gtk+ is very much Gnome and depends on other pieces of the Gnome project.
>> The actual name of the directory isn't really important so we can call it
>> gtk if you'd rather, but there are lots of other Gnome project dependencies
>> that aren't going away as long as we're using Gtk+. Remember that in
>> addition to gnome, gnome-util, and gnome-query there's also register-gnome,
>> report-gnome, and (until your latest changes) business-gnome. There's also
>> Gtk+ code in import-export, html, and probably a few other places that
>> don't immediately come to mind. Setting that aside we still have all of
>> that GValue and g_object_set/get code that we use for communicating between
>> the engine, the backends, and the importers.
>>
>> I'll point out once again that there's an immense amount of untangling to
>> separate the actual UI code from glue and actual bits of libgnucash that
>> are all mangled together in the various gnome directories.
>>
> All true. I preferred gtk as that's the name of the toolkit the code is based
> on, whereas gnome these days refers more to a user community and a desktop
> environment.
>
>> Does it really make sense to push much further with this for 2.8? I'd like
>> to move towards releasing that as quickly as we can, then we can move on to
>> extraction and rearrangement.
>
> I share your concern about getting 2.8 out as soon as possible. I'm trying to
> balance this concern with a potential maintenance burden in the next major
> cycle. If we do the directory shuffling early in the next major cycle the 2.8
> (then to be maint branch) and the new master branch will have a rather
> diverging source directory structure which means developers have to keep two
> structures in their head for one major cycle.
>
> I understand it's impossible to clean it all up in the few weeks we have left
> before we start the 2.8 end game. So I don't expect to be able to do more than
> some very basic top-level restructuring. My hope is this will more clearly set
> the intention for each source directory and may help in deciding which code
> should go where. And 2.8 maintenance and new master will have the same basic
> source structure reducing some of the confusion.
>
> The real uncluttering will be for the next major cycle.

Geert,

OK, I think we're in complete agreement. Let's go ahead and move current directories into gnucash and libgnucash, recognizing that a) there's plenty of code that's going to get moved after we branch 2.8 and b) that the two will be interdependent--meaning that the build system may have to bounce back-and-forth building modules in one and then the other until we get it untangled.

CMake will do better at optimizing the dependencies once it's informed of them all so we should probably dump autotools as part of this rearrangement.

Regards,
John Ralls

_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Source directory restructuring

Sumit Bhardwaj
John,

If the plan is to dump autotools, should we ask also devs to make sure that
they can build using cmake? I have had problems in the past and therefore,
I have stuck with autotools so far.

What are the things we want to confirm in the cmake toolchain?
cmake
cmake install
cmake check

Anything else?

Thanks,
Sumit

On Tue, Aug 8, 2017 at 7:36 AM, John Ralls <[hidden email]>
wrote:

>
> > On Aug 8, 2017, at 10:29 AM, Geert Janssens <[hidden email]>
> wrote:
> >
> > On maandag 7 augustus 2017 21:27:18 CEST John Ralls wrote:
> >> These are to be top-level directories, so the current src goes away,
> right?
> >> The rest assumes that to be the case.
> >>
> > Yes, I would drop src.
> >
> >> Let's call the libgnucash directory libgnucash instead of just lib and
> keep
> >> lib as the place put code pinched from elsewhere like the bits of
> >> libgoffice.
> >>
> > That's reasonable.
> >
> >> I want to get rid of gnc-module too, but it's not easy.
> >>
> > Indeed. I didn't plan this in the first phase. Except perhaps for some
> low
> > hanging fruit (like what I did with a couple of reports subdirectories) I
> > don't think this is something to target for 2.8.
> >
> >> App-utils has stuff that belongs in libgnucash and other stuff (e.g.
> >> options) that belongs in the application directory (which I propose
> calling
> >> "gnucash" below). There's also code splattered all over everywhere else
> >> that belongs in libgnucash. How much of a prerequisite to rearrangement
> is
> >> getting the code in the right place?
> >>
> > There's a gui and non-gui aspect to the options. The gui aspect clearly
> should
> > go to the "gnucash" part. The non-gui aspect probably not. In my long
> term
> > idea libgnucash should be able to generate reports (but not display
> them) to
> > allow the bindings (and eventual smartphone apps) access to this
> feature. As
> > things stand now, the report system requires the options. We all agree of
> > course the options are currently a complicated mixture of C and guile
> code
> > which needs serious modernization. That will not be for 2.8.
> >
> > But your general message stands. Lots of code is in the wrong functional
> > source group. And there definitely won't be time to reorganize all of
> that.
> >
> >> I think that having separate documentation directories (there are two,
> doc
> >> and src/doc) is an abject failure from a maintenance standpoint. A lot
> of
> >> stuff was written 15-20 years ago and was marked obsolete 10 years ago
> >> because it's been left to rot. All documentation needs to be in the
> source
> >> files as Doxygen comments. We should look through the docs and src/docs
> >> stuff and copy anything that's still relevant into comments in the
> >> appropriate source files and delete the docs directories.
> >>
> > Good idea.
> >
> >> I'd prefer that the application code go into a directory named gnucash
> >> rather than "ui". I'm inclined toward removing cutecash entirely. It
> made
> >> sense for Christian to keep it in the Gnucash repository when we were
> using
> >> SVN, but now he should just publish it in his own Github repo.
> >>
> > I realize I'm somewhat hesitant to drop cutecash from the repo. It looks
> I'm a
> > bit emotionally attached to it (although I never used it and only built
> it
> > once to test). That's probably because it's a decent first attempt to
> use qt
> > instead of gtk. And qt for me is one of the few options that can bring
> us one
> > code base for all form factors. (Drifting off into dreaming about
> Utopia...)
> >
> > Anyway sticking with the current situation I believe you are right to
> consider
> > removing it. It's not maintained, and based on the soon to be obsolete
> Qt4 and
> > most importantly it can be recovered should we ever want to do so or are
> ready
> > to go the Qt way and need a base to start from.
> >
> >> The bindings are mostly (and should probably be entirely) libgnucash
> >> bindings. They really belong in libgnucash or failing that in their own
> >> TLD, not part of the application.
> >>
> > TLD then.
> >
> >> Reports might stand on their own as a separate dependency of gnucash or
> they
> >> could be a subdirectory of gnucash. The same is true of import-export.
> In
> >> one approach  the GUI parts of each would be part of the application
> while
> >> the functional bits would be part of libgnucash. Are the matchers
> >> libgnucash or gnucash?
> >>
> > Long term I would like to see the functional part of both reports and
> import/
> > export be part of libgnucash and the gui's go into gnucash. I think
> there are
> > use cases where smartphone apps and the bindings would benefit from this.
> >
> > For 2.8 that would again be too much work. For this initial step I'd
> consider
> > the plugins directory to be the location to store self-contained,
> loadable
> > modules. There are already a few smaller ones in there. I'd add the
> report and
> > import/export directories in there as well until we're ready to split
> them up.
> > My work on the csv importer is an important step in the right direction.
> For
> > the next major cycle I hope to continue this for the other importers and
> > exporters as well.
> >
> >> Gtk+ is very much Gnome and depends on other pieces of the Gnome
> project.
> >> The actual name of the directory isn't really important so we can call
> it
> >> gtk if you'd rather, but there are lots of other Gnome project
> dependencies
> >> that aren't going away as long as we're using Gtk+. Remember that in
> >> addition to gnome, gnome-util, and gnome-query there's also
> register-gnome,
> >> report-gnome, and (until your latest changes) business-gnome. There's
> also
> >> Gtk+ code in import-export, html, and probably a few other places that
> >> don't immediately come to mind. Setting that aside we still have all of
> >> that GValue and g_object_set/get code that we use for communicating
> between
> >> the engine, the backends, and the importers.
> >>
> >> I'll point out once again that there's an immense amount of untangling
> to
> >> separate the actual UI code from glue and actual bits of libgnucash that
> >> are all mangled together in the various gnome directories.
> >>
> > All true. I preferred gtk as that's the name of the toolkit the code is
> based
> > on, whereas gnome these days refers more to a user community and a
> desktop
> > environment.
> >
> >> Does it really make sense to push much further with this for 2.8? I'd
> like
> >> to move towards releasing that as quickly as we can, then we can move
> on to
> >> extraction and rearrangement.
> >
> > I share your concern about getting 2.8 out as soon as possible. I'm
> trying to
> > balance this concern with a potential maintenance burden in the next
> major
> > cycle. If we do the directory shuffling early in the next major cycle
> the 2.8
> > (then to be maint branch) and the new master branch will have a rather
> > diverging source directory structure which means developers have to keep
> two
> > structures in their head for one major cycle.
> >
> > I understand it's impossible to clean it all up in the few weeks we have
> left
> > before we start the 2.8 end game. So I don't expect to be able to do
> more than
> > some very basic top-level restructuring. My hope is this will more
> clearly set
> > the intention for each source directory and may help in deciding which
> code
> > should go where. And 2.8 maintenance and new master will have the same
> basic
> > source structure reducing some of the confusion.
> >
> > The real uncluttering will be for the next major cycle.
>
> Geert,
>
> OK, I think we're in complete agreement. Let's go ahead and move current
> directories into gnucash and libgnucash, recognizing that a) there's plenty
> of code that's going to get moved after we branch 2.8 and b) that the two
> will be interdependent--meaning that the build system may have to bounce
> back-and-forth building modules in one and then the other until we get it
> untangled.
>
> CMake will do better at optimizing the dependencies once it's informed of
> them all so we should probably dump autotools as part of this rearrangement.
>
> Regards,
> John Ralls
>
> _______________________________________________
> gnucash-devel mailing list
> [hidden email]
> https://lists.gnucash.org/mailman/listinfo/gnucash-devel
>
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Source directory restructuring

John Ralls

> On Aug 8, 2017, at 5:50 PM, Sumit Bhardwaj <[hidden email]> wrote:
>
> John,
>
> If the plan is to dump autotools, should we ask also devs to make sure that
> they can build using cmake? I have had problems in the past and therefore,
> I have stuck with autotools so far.
>
> What are the things we want to confirm in the cmake toolchain?
> cmake
> cmake install
> cmake check
>
> Anything else?
Sumit,

No. cmake <args> srcdir && make && make check && make install or (quite a bit faster) cmake -G Ninja <args> srcdir && ninja check && ninja install.

You generally need to at least specify a -DCMAKE_INSTALL_PREFIX unless you want GnuCash installed in /usr/local which back in the day was a reasonable thing to do but isn't really anymore. Because of normal linker behavior and GnuCash's overuse of loadable modules you also need to uninstall before building, especially when changing branches. The incantation for that in cmake is xargs rm < install_manifest.txt.

Geert and I use the cmake+ninja build system most of the time and the Windows automated build has been using it for just over a year. I think that it's well tested. There's a known problem that the dependency graph doesn't capture everything especially for some of the scheme modules so allowing too much parallelism (setting -j too high on a many-core machine) will try to build some things before their dependencies are done. That's not a blocker to dropping autotools. The only loose end at present is that there are still a few rough edges in the dist target that need to be cleaned up.

Regards,
John Ralls


_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Source directory restructuring

Aaron Laws
In reply to this post by Geert Janssens-4
On Mon, Aug 7, 2017 at 7:56 AM, Geert Janssens <[hidden email]>
wrote:

> So after my houskeeping message in which I have announced the changes to
> src/
> business and src/libqof, I'd like to bring up my eventual goal for the
> source
> tree.
>
> My main motivation to do all this restructuring is to simplify. There are
> currently plenty of directories and I often have to guess where to expect a
> file. The qof vs engine story was one example. Is gnc-date something for
> qof
> or for engine ? I find myself regularly searching for a file in the wrong
> directory.
>
> So here follows a first proposal for the directory structure I'm
> targetting:
>
> * data (for things like art, checks, account hierarchy templates,...)
> * doc (for all documenation)
> * lib (for all source required to build "libgnucash", see below)
> * ui (for all the user interfaces the project currently supports)
>
> I am omitting some smaller directories here, such as util and macros. They
> will probably stay on the current top level for now.
>
> I'm envisioning "libgnucash" as the core library that encapsulates all
> gnucash
> related concepts (the accounting concepts such as transaction, split, as
> well
> as the data backends). This library is what all applications in the gnucash
> sphere should depend on to implement the "gnucash" experience. The most
> obvious is of course the current gnucash as we know it. However at some
> point
> this library should ideally also become the core of the Android app and
> *who
> knows* one day an IOS app. And closer to the current state, it should also
> be
> the library that the guile and python bindings depend on. So all
> functionality
> encapsulated in one single library, the API. In practice I think the
> following
> directories belong in this libgnucash: core-utils, gnc-module, engine, the
> backends, app-utils. (Note I would like to get rid of gnc-module still, but
> that's a whole other discussion and task).
>
> The ui directory will have a subdirectory for each user interface we
> support.
> This is not necessarily a *graphical* user interface though. At this point
> there are already a number of them:
> gnucash
> cutecash
> bindings (with subdirectories for python and guile)
>
> The bulk of the other directories are support directories for one of the
> ui's
> and I propose to make them subdirectories of each respective ui.
>
> For example gtkmm is only used by cutecash, so let's store it there (until
> another ui would also require it, which I consider unlikely to happen
> still).
>
> The other directories (gnome-utils, gnome-search, gnome, register, html,
> report, import-export,...) are all used only in the gnucash application and
> hence should be moved there. In the move I'd like to try and reduce the 3
> gnome* directories to one and call it gtk as we're not using any gnome
> specific technology any more.
>
> In a later phase I think it would be nice if the core libgnucash could also
> spit out html reports, but that would require us to refactor the report
> modules, which I consider too much work to be done at the same time. When
> this
> eventually gets done the non-ui part of the report system can then be
> added in
> libgnucash to the benefit of all consumers (gnucash/cutecash/Gnc4Android/
> ...).
>
> Feedback or questions ?
>
> Geert
>

I echo the other sentiments: This will make it much easier to grok and
contribute to gnucash.

Concerning cutecash, "It's not maintained" may be the case now, but I
remember working on some libqof changes before and having to "maintain"
cutecash since it was breaking because of the changes. It was a bit
irritating. If it's a difficult thing to bring yourself to, perhaps you
could have a friend pull the trigger? ;-)

Thanks for thinking through this and putting it on paper. I look forward to
seeing the fruit of this labour.
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Source directory restructuring

Geert Janssens-4
In reply to this post by John Ralls
On dinsdag 8 augustus 2017 19:01:44 CEST John Ralls wrote:

> > On Aug 8, 2017, at 5:50 PM, Sumit Bhardwaj <[hidden email]> wrote:
> >
> > John,
> >
> > If the plan is to dump autotools, should we ask also devs to make sure
> > that
> > they can build using cmake? I have had problems in the past and therefore,
> > I have stuck with autotools so far.
> >
> > What are the things we want to confirm in the cmake toolchain?
> > cmake
> > cmake install
> > cmake check
> >
> > Anything else?
>
> Sumit,
>
> No. cmake <args> srcdir && make && make check && make install or (quite a
> bit faster) cmake -G Ninja <args> srcdir && ninja check && ninja install.
>
> You generally need to at least specify a -DCMAKE_INSTALL_PREFIX unless you
> want GnuCash installed in /usr/local which back in the day was a reasonable
> thing to do but isn't really anymore. Because of normal linker behavior and
> GnuCash's overuse of loadable modules you also need to uninstall before
> building, especially when changing branches. The incantation for that in
> cmake is xargs rm < install_manifest.txt.
>
> Geert and I use the cmake+ninja build system most of the time and the
> Windows automated build has been using it for just over a year. I think
> that it's well tested. There's a known problem that the dependency graph
> doesn't capture everything especially for some of the scheme modules so
> allowing too much parallelism (setting -j too high on a many-core machine)
> will try to build some things before their dependencies are done. That's
> not a blocker to dropping autotools. The only loose end at present is that
> there are still a few rough edges in the dist target that need to be
> cleaned up.
 
Which dist target rough edges are you referring to here ?

I know there is the issue I can't add Makefile.in files to the dist tarball
(because Rob's dist rules prevent my cmake version from running the necessary
commands). Other than that the dist tarball generated via both build systems
are identical on my system after I ironed out a few minor issues last month.
Since we're right now discussing dropping autotools, this very issue becomes
moot.

Do you know other ones ?

Geert
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Source directory restructuring

Geert Janssens-4
On woensdag 9 augustus 2017 18:51:52 CEST Geert Janssens wrote:

> > Geert and I use the cmake+ninja build system most of the time and the
> > Windows automated build has been using it for just over a year. I think
> > that it's well tested. There's a known problem that the dependency graph
> > doesn't capture everything especially for some of the scheme modules so
> > allowing too much parallelism (setting -j too high on a many-core machine)
> > will try to build some things before their dependencies are done. That's
> > not a blocker to dropping autotools. The only loose end at present is that
> > there are still a few rough edges in the dist target that need to be
> > cleaned up.
>
> Which dist target rough edges are you referring to here ?
>
> I know there is the issue I can't add Makefile.in files to the dist tarball
> (because Rob's dist rules prevent my cmake version from running the
> necessary commands). Other than that the dist tarball generated via both
> build systems are identical on my system after I ironed out a few minor
> issues last month. Since we're right now discussing dropping autotools,
> this very issue becomes moot.
>
> Do you know other ones ?

Oh, and there is this little detail called Travis... Perhaps I should first
test whether it can switch to cmake or not.

Geert
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Source directory restructuring

John Ralls

> On Aug 9, 2017, at 8:02 PM, Geert Janssens <[hidden email]> wrote:
>
> On woensdag 9 augustus 2017 18:51:52 CEST Geert Janssens wrote:
>>> Geert and I use the cmake+ninja build system most of the time and the
>>> Windows automated build has been using it for just over a year. I think
>>> that it's well tested. There's a known problem that the dependency graph
>>> doesn't capture everything especially for some of the scheme modules so
>>> allowing too much parallelism (setting -j too high on a many-core machine)
>>> will try to build some things before their dependencies are done. That's
>>> not a blocker to dropping autotools. The only loose end at present is that
>>> there are still a few rough edges in the dist target that need to be
>>> cleaned up.
>>
>> Which dist target rough edges are you referring to here ?
>>
>> I know there is the issue I can't add Makefile.in files to the dist tarball
>> (because Rob's dist rules prevent my cmake version from running the
>> necessary commands). Other than that the dist tarball generated via both
>> build systems are identical on my system after I ironed out a few minor
>> issues last month. Since we're right now discussing dropping autotools,
>> this very issue becomes moot.
>>
>> Do you know other ones ?
>
> Oh, and there is this little detail called Travis... Perhaps I should first
> test whether it can switch to cmake or not.

I did when I was testing it but now I've forgotten. Having the tarball have both systems configs and makefiles and so on was certainly one of them, and you're right that it's moot when we drop autotools.

I'm sure that Travis can do cmake, but not sure that it can do a new enough cmake: We need at least 3.0 and 3.2 or later is preferred. ISTR that Travis has only 2.8.

Regards,
John Ralls



_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Source directory restructuring

Alex Aycinena-2
In reply to this post by Geert Janssens-4
>
>
>
> ---------- Forwarded message ----------
> From: John Ralls <[hidden email]>
> To: Sumit Bhardwaj <[hidden email]>
> Cc: gnucash-devel <[hidden email]>
> Bcc:
> Date: Tue, 8 Aug 2017 20:01:44 +0300
> Subject: Re: Source directory restructuring
>
> > On Aug 8, 2017, at 5:50 PM, Sumit Bhardwaj <[hidden email]> wrote:
> >
> > John,
> >
> > If the plan is to dump autotools, should we ask also devs to make sure
> that
> > they can build using cmake? I have had problems in the past and
> therefore,
> > I have stuck with autotools so far.
> >
> > What are the things we want to confirm in the cmake toolchain?
> > cmake
> > cmake install
> > cmake check
> >
> > Anything else?
> Sumit,
>
> No. cmake <args> srcdir && make && make check && make install or (quite a
> bit faster) cmake -G Ninja <args> srcdir && ninja check && ninja install.
>
> You generally need to at least specify a -DCMAKE_INSTALL_PREFIX unless you
> want GnuCash installed in /usr/local which back in the day was a reasonable
> thing to do but isn't really anymore. Because of normal linker behavior and
> GnuCash's overuse of loadable modules you also need to uninstall before
> building, especially when changing branches. The incantation for that in
> cmake is xargs rm < install_manifest.txt.
>
> Geert and I use the cmake+ninja build system most of the time and the
> Windows automated build has been using it for just over a year. I think
> that it's well tested. There's a known problem that the dependency graph
> doesn't capture everything especially for some of the scheme modules so
> allowing too much parallelism (setting -j too high on a many-core machine)
> will try to build some things before their dependencies are done. That's
> not a blocker to dropping autotools. The only loose end at present is that
> there are still a few rough edges in the dist target that need to be
> cleaned up.
>
> Regards,
> John Ralls
>
>

I switched to cmake and ninja a few months ago when I had trouble building
with autotools and I thought everything was working fine. I pushed a commit
and found that some changes to unit tests that I had made didn't work and
so I accidently broke the build. I had assumed that ninja check, which had
been successful, had run the unit tests and hadn't bother to check for
sure. I was finally able to get autotools to work by not including the
debug arg in configure and so was able to run the unit test, fix it and
push the fix. I have been using autotools since to make sure the tests are
all run.

My question is whether cmake now runs all the same unit tests.

Regards,

Alex
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Source directory restructuring

John Ralls

> On Aug 9, 2017, at 9:16 PM, Alex Aycinena <[hidden email]> wrote:
>
>>
>>
>>
>> ---------- Forwarded message ----------
>> From: John Ralls <[hidden email]>
>> To: Sumit Bhardwaj <[hidden email]>
>> Cc: gnucash-devel <[hidden email]>
>> Bcc:
>> Date: Tue, 8 Aug 2017 20:01:44 +0300
>> Subject: Re: Source directory restructuring
>>
>>> On Aug 8, 2017, at 5:50 PM, Sumit Bhardwaj <[hidden email]> wrote:
>>>
>>> John,
>>>
>>> If the plan is to dump autotools, should we ask also devs to make sure
>> that
>>> they can build using cmake? I have had problems in the past and
>> therefore,
>>> I have stuck with autotools so far.
>>>
>>> What are the things we want to confirm in the cmake toolchain?
>>> cmake
>>> cmake install
>>> cmake check
>>>
>>> Anything else?
>> Sumit,
>>
>> No. cmake <args> srcdir && make && make check && make install or (quite a
>> bit faster) cmake -G Ninja <args> srcdir && ninja check && ninja install.
>>
>> You generally need to at least specify a -DCMAKE_INSTALL_PREFIX unless you
>> want GnuCash installed in /usr/local which back in the day was a reasonable
>> thing to do but isn't really anymore. Because of normal linker behavior and
>> GnuCash's overuse of loadable modules you also need to uninstall before
>> building, especially when changing branches. The incantation for that in
>> cmake is xargs rm < install_manifest.txt.
>>
>> Geert and I use the cmake+ninja build system most of the time and the
>> Windows automated build has been using it for just over a year. I think
>> that it's well tested. There's a known problem that the dependency graph
>> doesn't capture everything especially for some of the scheme modules so
>> allowing too much parallelism (setting -j too high on a many-core machine)
>> will try to build some things before their dependencies are done. That's
>> not a blocker to dropping autotools. The only loose end at present is that
>> there are still a few rough edges in the dist target that need to be
>> cleaned up.
>>
>> Regards,
>> John Ralls
>>
>>
>
> I switched to cmake and ninja a few months ago when I had trouble building
> with autotools and I thought everything was working fine. I pushed a commit
> and found that some changes to unit tests that I had made didn't work and
> so I accidently broke the build. I had assumed that ninja check, which had
> been successful, had run the unit tests and hadn't bother to check for
> sure. I was finally able to get autotools to work by not including the
> debug arg in configure and so was able to run the unit test, fix it and
> push the fix. I have been using autotools since to make sure the tests are
> all run.
>
> My question is whether cmake now runs all the same unit tests.
>

Alex,

IIRC you flagged those tests as not run and Geert fixed them. Do you remember what they were?

I think that cmake and autotools are running the same set of tests but I haven't done a test-by-test comparison in a while.

Regards,
John Ralls


_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Source directory restructuring

Sumit Bhardwaj
John and Geert,

As I remembered, I am still having problem with cmake. I have pasted the
error message below. Is this a known problem? If not, will it be better to
wait for Geert's restructuring and then try to fix it? For reference,
autotools work.

Thanks,
Sumit

​[3/942] cd /home/bhardwajs/ac/devel/gnucash/po && /usr/bin/cmake ...e -D
PO_DIR=/home/bhardwajs/ac/devel/gnucash/po -P check-po.cmake
FAILED: po/CMakeFiles/check-po
cd /home/bhardwajs/ac/devel/gnucash/po && /usr/bin/cmake -D
INTLTOOL_UPDATE=/usr/bin/intltool-update -D PO_DIR=/home/bhardwajs/ac/deve
l/gnucash/po -P check-po.cmake
The usage of POTFILES.ignore is deprecated. Please consider moving the
content of this file to POTFILES.skip.
The following files contain translations and are currently not in use.
Please
consider adding these to the POTFILES.in file, located in the po/ directory

If some of these files are left out on purpose then please add them to
POTFILES.skip instead of POTFILES.in. A file 'missing' containing this list
of left out files has been written in the current directory.
Please report to [hidden email]
CMake Error at check-po.cmake:22 (MESSAGE):
 POTFILES.in is missing files.  See 'missing' in
 /home/bhardwajs/ac/devel/gnucash/po


On Wed, Aug 9, 2017 at 11:54 AM, John Ralls <[hidden email]>
wrote:

>
> > On Aug 9, 2017, at 9:16 PM, Alex Aycinena <[hidden email]>
> wrote:
> >
> >>
> >>
> >>
> >> ---------- Forwarded message ----------
> >> From: John Ralls <[hidden email]>
> >> To: Sumit Bhardwaj <[hidden email]>
> >> Cc: gnucash-devel <[hidden email]>
> >> Bcc:
> >> Date: Tue, 8 Aug 2017 20:01:44 +0300
> >> Subject: Re: Source directory restructuring
> >>
> >>> On Aug 8, 2017, at 5:50 PM, Sumit Bhardwaj <[hidden email]>
> wrote:
> >>>
> >>> John,
> >>>
> >>> If the plan is to dump autotools, should we ask also devs to make sure
> >> that
> >>> they can build using cmake? I have had problems in the past and
> >> therefore,
> >>> I have stuck with autotools so far.
> >>>
> >>> What are the things we want to confirm in the cmake toolchain?
> >>> cmake
> >>> cmake install
> >>> cmake check
> >>>
> >>> Anything else?
> >> Sumit,
> >>
> >> No. cmake <args> srcdir && make && make check && make install or (quite
> a
> >> bit faster) cmake -G Ninja <args> srcdir && ninja check && ninja
> install.
> >>
> >> You generally need to at least specify a -DCMAKE_INSTALL_PREFIX unless
> you
> >> want GnuCash installed in /usr/local which back in the day was a
> reasonable
> >> thing to do but isn't really anymore. Because of normal linker behavior
> and
> >> GnuCash's overuse of loadable modules you also need to uninstall before
> >> building, especially when changing branches. The incantation for that in
> >> cmake is xargs rm < install_manifest.txt.
> >>
> >> Geert and I use the cmake+ninja build system most of the time and the
> >> Windows automated build has been using it for just over a year. I think
> >> that it's well tested. There's a known problem that the dependency graph
> >> doesn't capture everything especially for some of the scheme modules so
> >> allowing too much parallelism (setting -j too high on a many-core
> machine)
> >> will try to build some things before their dependencies are done. That's
> >> not a blocker to dropping autotools. The only loose end at present is
> that
> >> there are still a few rough edges in the dist target that need to be
> >> cleaned up.
> >>
> >> Regards,
> >> John Ralls
> >>
> >>
> >
> > I switched to cmake and ninja a few months ago when I had trouble
> building
> > with autotools and I thought everything was working fine. I pushed a
> commit
> > and found that some changes to unit tests that I had made didn't work and
> > so I accidently broke the build. I had assumed that ninja check, which
> had
> > been successful, had run the unit tests and hadn't bother to check for
> > sure. I was finally able to get autotools to work by not including the
> > debug arg in configure and so was able to run the unit test, fix it and
> > push the fix. I have been using autotools since to make sure the tests
> are
> > all run.
> >
> > My question is whether cmake now runs all the same unit tests.
> >
>
> Alex,
>
> IIRC you flagged those tests as not run and Geert fixed them. Do you
> remember what they were?
>
> I think that cmake and autotools are running the same set of tests but I
> haven't done a test-by-test comparison in a while.
>
> Regards,
> John Ralls
>
>
> _______________________________________________
> gnucash-devel mailing list
> [hidden email]
> https://lists.gnucash.org/mailman/listinfo/gnucash-devel
>
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

cmake failure (was:Re: Source directory restructuring)

Geert Janssens-4
On donderdag 10 augustus 2017 07:08:38 CEST Sumit Bhardwaj wrote:

> John and Geert,
>
> As I remembered, I am still having problem with cmake. I have pasted the
> error message below. Is this a known problem? If not, will it be better to
> wait for Geert's restructuring and then try to fix it? For reference,
> autotools work.
>
> Thanks,
> Sumit
>
> ​[3/942] cd /home/bhardwajs/ac/devel/gnucash/po && /usr/bin/cmake ...e -D
> PO_DIR=/home/bhardwajs/ac/devel/gnucash/po -P check-po.cmake
> FAILED: po/CMakeFiles/check-po
> cd /home/bhardwajs/ac/devel/gnucash/po && /usr/bin/cmake -D
> INTLTOOL_UPDATE=/usr/bin/intltool-update -D PO_DIR=/home/bhardwajs/ac/deve
> l/gnucash/po -P check-po.cmake
> The usage of POTFILES.ignore is deprecated. Please consider moving the
> content of this file to POTFILES.skip.
> The following files contain translations and are currently not in use.
> Please
> consider adding these to the POTFILES.in file, located in the po/ directory
>
> If some of these files are left out on purpose then please add them to
> POTFILES.skip instead of POTFILES.in. A file 'missing' containing this list
> of left out files has been written in the current directory.
> Please report to [hidden email]
> CMake Error at check-po.cmake:22 (MESSAGE):
>  POTFILES.in is missing files.  See 'missing' in
>  /home/bhardwajs/ac/devel/gnucash/po

That's a weird error and I can't reproduce.

What platform/os are you on ?
What are the exact commands you are using to get here ?
What's the contents of /home/bhardwajs/ac/devel/gnucash/po/missing ?

Geert
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: cmake failure (was:Re: Source directory restructuring)

Aaron Laws
On Thu, Aug 10, 2017 at 3:13 AM, Geert Janssens <[hidden email]>
wrote:

> On donderdag 10 augustus 2017 07:08:38 CEST Sumit Bhardwaj wrote:
> > John and Geert,
> >
> > As I remembered, I am still having problem with cmake. I have pasted the
> > error message below. Is this a known problem? If not, will it be better
> to
> > wait for Geert's restructuring and then try to fix it? For reference,
> > autotools work.
> >
> > Thanks,
> > Sumit
> >
> > ​[3/942] cd /home/bhardwajs/ac/devel/gnucash/po && /usr/bin/cmake ...e
> -D
> > PO_DIR=/home/bhardwajs/ac/devel/gnucash/po -P check-po.cmake
> > FAILED: po/CMakeFiles/check-po
> > cd /home/bhardwajs/ac/devel/gnucash/po && /usr/bin/cmake -D
> > INTLTOOL_UPDATE=/usr/bin/intltool-update -D
> PO_DIR=/home/bhardwajs/ac/deve
> > l/gnucash/po -P check-po.cmake
> > The usage of POTFILES.ignore is deprecated. Please consider moving the
> > content of this file to POTFILES.skip.
> > The following files contain translations and are currently not in use.
> > Please
> > consider adding these to the POTFILES.in file, located in the po/
> directory
> >
> > If some of these files are left out on purpose then please add them to
> > POTFILES.skip instead of POTFILES.in. A file 'missing' containing this
> list
> > of left out files has been written in the current directory.
> > Please report to [hidden email]
> > CMake Error at check-po.cmake:22 (MESSAGE):
> >  POTFILES.in is missing files.  See 'missing' in
> >  /home/bhardwajs/ac/devel/gnucash/po
>
> That's a weird error and I can't reproduce.
>
> What platform/os are you on ?
> What are the exact commands you are using to get here ?
> What's the contents of /home/bhardwajs/ac/devel/gnucash/po/missing ?
>
> Geert


I remember getting that error a long time ago, but I don't remember how to
fix it. I think it has something to do with deleting a generated file and
having make re-generate it? It seems like we have a generated file in
version control? Is POTFILES.in generated and version controlled for
example?
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: cmake failure (was:Re: Source directory restructuring)

Geert Janssens-4
On donderdag 10 augustus 2017 14:15:24 CEST Aaron Laws wrote:
> I remember getting that error a long time ago, but I don't remember how to
> fix it. I think it has something to do with deleting a generated file and
> having make re-generate it? It seems like we have a generated file in
> version control? Is POTFILES.in generated and version controlled for
> example?

It is, but that's not the problem. However your reply did ring a bell and
here's the issue:
If your build directory is a subdirectory of your source directory make check
will fail because it will falsely believe the built files should be added to
POTFILES.in. Building with ninja-build doesn't exhibit this issue so I assume
it's a make quirk.

The immediate solution is simple: choose a build directory that's not a
subdirectory of your source directory. How to nudge cmake to deal with this
properly may be another matter.

Regards,

Geert
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: cmake failure (was:Re: Source directory restructuring)

Sumit Bhardwaj
Actually, I was using ninja when I ran into this problem. Here is the
sequence of commands that lead to the error:

$> pwd
devel/gnucash
$mkdir gbuild && cc gbuild
$ cmake -D WITH_AQBANKING=OFF -D WITH_OFX=OFF -G Ninja .. && ninja check &&
ninja install

I am attaching po/missing.

Note that the following commands also didn't failed for the same reason.
$> pwd
devel/gnucash
$> cd .. && mkdir gbuild && cd gbuild
$> cmake -D WITH_AQBANKING=OFF -D WITH_OFX=OFF -G Ninja ../gnucash/ &&
ninja check && ninja install

Thanks,
Sumit

On Thu, Aug 10, 2017 at 6:09 AM, Geert Janssens <[hidden email]>
wrote:

> On donderdag 10 augustus 2017 14:15:24 CEST Aaron Laws wrote:
> > I remember getting that error a long time ago, but I don't remember how
> to
> > fix it. I think it has something to do with deleting a generated file and
> > having make re-generate it? It seems like we have a generated file in
> > version control? Is POTFILES.in generated and version controlled for
> > example?
>
> It is, but that's not the problem. However your reply did ring a bell and
> here's the issue:
> If your build directory is a subdirectory of your source directory make
> check
> will fail because it will falsely believe the built files should be added
> to
> POTFILES.in. Building with ninja-build doesn't exhibit this issue so I
> assume
> it's a make quirk.
>
> The immediate solution is simple: choose a build directory that's not a
> subdirectory of your source directory. How to nudge cmake to deal with this
> properly may be another matter.
>
> Regards,
>
> Geert
>

_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

missing (7K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Source directory restructuring

Frank H. Ellenberger-3
In reply to this post by Geert Janssens-4
Hi,

Am 08.08.2017 um 09:29 schrieb Geert Janssens:
> On maandag 7 augustus 2017 21:27:18 CEST John Ralls wrote:
>> These are to be top-level directories, so the current src goes away, right?
>> The rest assumes that to be the case.
>>
> Yes, I would drop src.
>

Ahem, if I understand them right, most tools, I am using for e.g.
I18N/L10N and documentation expect a directory src.

Regards
Frank
_______________________________________________
gnucash-devel mailing list
[hidden email]
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
12
Loading...