Automated win32 build scripts and UPDATE_SOURCES, packaging dir,...

Geert Janssens janssens-geert at telenet.be
Sat Aug 11 11:29:07 EDT 2012


I have just added some scripts to run an automated build from git 
instead of svn.

I copied the svn behaviour as closely as possible. While doing so I got 
to question parts of the general setup we promote [1]. The guidelines 
state that the contents of packaging/win32 should be copied locally, and 
it shouldn't be in the repo checkout that will later be used for building.

I see several issues with this, some of which may be easier to fix than 
others:
- I first wondered why the packaging directory should be separate from 
the repo used to build. After some pondering I came up with this: when 
UPDATE_SOURCES=yes, and we're using the packaging dir from the repo, the 
install scripts may update from under us while building. Obviously not good.
- So I started thinking about UPDATE_SOURCES. Why is that option in 
there ? I suppose for the "automated" mantra. Having it in there, allows 
you to always build the most recent state of a branch. I wonder though 
if this benefit outweighs the restrictions and limitations it brings 
with it.
- Copying the packaging directory is a bootstrap thing and has other 
issues: the guidelines don't explain how to copy this directory. On a 
clean Windows system your option is to manually download each file or 
use some kind of download manager. At this stage the user isn't supposed 
to have svn or git available to checkout the directory - that means the 
guidelines don't ask him to do so, because it will be installed later on 
"automatically". This looks like a hurdle to me for new potential 
contributors on the Windows platform (there are many more, I know...)
- Next, by simply copying the files to a local directory, the user 
looses the main advantage a checked out repo has: easy update to newer 
versions of the build scripts. The user has to guess if/when the scripts 
were updated and manually copy the over again from somewhere. So if the 
user isn't careful, later builds may be using outdated build scripts.
- The fix for the last problem would be to use some form of checkout of 
the packaging directory. But at bootstrap time, there's no subversion or 
git available. That can be solved by asking the user to install either 
tool manually.
- For svn, the story ends here. The user uses svn to checkout the 
packaging directory and continues the setup. To get the most recent 
build scripts, it suffices to run svn update.
- But for git users, there's another catch: git doesn't allow to 
checkout one directory only from a git repo. So to satisfy the original 
requirement (separate packaging directory) he needs to clone the 
complete git directory. The build itself will clone it once more. That's 
twice the disk space for the complete gnucash repository used for only 
ten or something scripts.

And all that because UPDATE_SOURCES risks of invalidating the build 
scripts mid-build.

I see two possible fixes:
- drop UPDATE_SOURCES. It's not that hard to update your build repo for 
either svn or git. For git which is much more branch focussed than svn, 
I think an active developer would probably disable UPDATE_SOURCES 
anyway, because it would rarely do what you actually wanted. With 
UPDATE_SOURCES out of the way, the bootstrap procedure would be 
something like: install your favourite revision control tool, check out 
the proper repository, (the other bootstrappers, such as mingw-dtk,...), 
cd into packaging/win32 and off you go. This could still work with 
building from tarball as well.
- alternatively if UPDATE_SOURCES is to stay, we could also move the 
packaging/win32 directory to a separate repository (similar to what the 
OS X build does). That would allow UPDATE_SOURCES to remain and have a 
local packaging directory that is revision controlled from the start. 
Doesn't make much difference for svn users, but for git users the 
separate packaging repository would be a small clone instead of having 
to checkout the whole gnucash repository. So for this solution the steps 
would be: install your favourite revision control too, checkout the 
packaging repository. Do the other bootstrapping stuff and off you go. 
The packaging repo can be updated whenever needed.

Personally I prefer the first option. From a new user's point of view it 
would look so much more intuitive to be able to either install a tarball 
or the preferred rcs and checkout the proper repo and from there on 
start building (I deliberately ingore some other bootstrap steps here to 
create the picture). But the second option would work as well for me.

What do others think ? Is my reasoning correct ? What do you see as 
solutions ?

Geert

[1] 
http://wiki.gnucash.org/wiki/Windows#Instructions_for_an_.28almost.29_automated_build


More information about the gnucash-devel mailing list