[Patch] Bug 712640: gnc-engine.c parallel optimization

[John Ralls]

On Nov 19, 2013, at 4:26 AM, Alex Kempster <alkempster.cu at gmail.com> wrote:

> - It looks like someone has already reverted the wiki change regarding openmp.
> - I also added doxygen to:
> and a few required or useful tools
> 
> sudo apt-get install libtool swig subversion libgnomeui-dev xsltproc doxygen
> 
> This has also been reverted (under ubuntu 12.04). Can i make that change back? 
> I would consider doxygen a useful tool that is not included in that specific 
> package and I know it something that gnucash has/does use...

OK, but why only on that one version of Ubuntu? It is 
> 
> - on the same page and section this existed:
> 
> 
> The main branch in git is conventionally named master, whereas in this repository it is named trunk (due to the fact that it is derived from a subversion repository I believe). This is not a problem but if you would rather have it named master then cd into the gnucash directory and
> 
> git branch -t master refs/remotes/origin/trunk git checkout master
> 
> I  removed it as cloning the repository already sets you to branch master. would you like me to revert that change?

Yes, that's fine. Like Geert, I didn't realize that had changed. Since we'll be 
> 
> Under the code section on the development page i appended:
> 
> Note: The source code repository is changing from subversion to git. 
> 
> so it reads:
> 
> We use Subversion as our primary Version Control System. There is an official Git mirror on Github. Detailed instructions for using each are provided on the Subversion page. This includes commit conventions--e.g. backporting conventions. Note: The source code repository is changing from subversion to git.
> would you like me to revert this?
> 
No, but you can rewrite the whole paragraph so that it explains that SVN is currently the canonical repo, that we're in a transition between SVN and git, and we'll complete the transition shortly after releasing 2.6.

> >I’m negative about this attempt because you so clearly took on something way beyond your ability to execute and beyond Gnucash’s ability to accept. If you’ve looked through >the sources you can’t escape noticing all of the static variables and the total absence of any sort of locking; if you’re experienced in multi-threaded development that would have >raised a huge red flag. It didn’t. But in my reply to your introductory letter I pointed it out:
> >"As for MP, we need to get to multi-user access to the database back end first. This will unfortunately require a major rewrite, for which we're planning to use C++. Thread safety >should certainly be a design consideration in that new implementation; the current one is stuffed with statics and utterly thread unsafe.”
> 
> >You ignored me and jumped in with an incompetent attempt, going so far as to announce “a new feature” in the Wiki. Why did you think you’d get anything but a negative >response?
> 
> No, I am not experienced with multi-threaded development.

But you said "I also have a few years experience working on parallelization of programs using OpenMP."
Shall we charitably call that an exaggeration, then?

> I thought wiki changes, like changes to code, were moderated before being accepted. It was not my attention to make it seem like I was announcing a new feature, just never made wiki changes before. I think it raises an important question: shouldn't those changes be moderated to prevent someone making changes that are not wanted? It is obviously important ans i see it as the equivalent of not wanting changes to be made to source code but allowing anyone to merger their code (as an example).

You've never met a wiki, either? Have you been hiding under a rock for the last 15 years?

Wikis are intentionally post-moderated. By design, anyone can edit them. Anyone else can then edit the edits. Unfortunately this makes them spam magnets, so they have to be closely monitored. Frank Ellenberger, Christian Stimming and I check the changes page frequently for spam and misguided edits. Spammers get blocked, misguided edits are just reverted so that the editor can try again. You're not blocked, but I suggest you read up on how Wikis work and what they're about before you dive back in.

> 
> >The documentation on patches is geared to someone who already knows how to use git and what a patch is.
> >Knowing about config.h requires knowing about auto tools. That’s a prerequisite for contributing to any project in the Gnome universe.
> >It’s well documented on the web. Gnucash can’t provide its own documentation of every tool we use. It’s incumbent upon potential contributors to get up to speed on them on their >own.
> 
> I know how to use git (not fully, but I can get around), I have used it for many personal and academic projects. And a patch is simply a set of code which repairs or fixes a bug or known problem with code. The difference here is that this is the first time I have used git where multiple people are using the same repository. There is going to be a difference between using a private repo with a single user compared to a public repo with many users which is a hybrid of git and svn (according to documentation). This is the first project i have tried contributing, let alone one in the gnome universe. I had no idea that auto tools was a prerequisite for gnome based projects, with all the dependencies the project has it would be impossible for me to get familiar with all aspects in a week. I agree gnucash can't provide documentation for all the tools it uses but it would be little difficulty to provide a link or two to the places which do document it, just a suggestion for future development of the wiki. Yes, it is potential contributors responsibility to get up to speed, but with all of the tools and libraries used it can be very easy to make a mistake such as putting another include before config.h as it is one of the many pitfalls a beginner may make. I thought that was a reason why potential contributors have to go through a trusted contributor to filter out the mistakes which are bound to happen. I could spend the next month reading up on every tool and still make a simple mistake like that.

A patch is at its most basic an input file for patch(1), created by diff(1). Git in particular offers `git am` for applying patches and `git format-patch` for making them. The patches section on the Wiki that I pointed you to explains that, but it's up to you to then read and understand the man pages for those commands.

I often recommend Scott Chacon's "ProGit". I keep a paper copy by my computer, but it's available free in a variety of electronic formats at http://git-scm.com/documentation.

> 
> >No, without the -fopenmp flag it won’t compile at all, because the compiler can’t find <omp.h> without the flag. How can someone with years of experience not know that?
> 
> That is strange. I was able to compile without the -fopenmp flag from a fresh install of everything. In fact, every project in which I have used OpenMP I have been able to include or not include that compiler option without any compiler warnings or errors. I believe that this is because OpenMP is built into the gcc compiler 4.7+, thus omp.h is in the default search directory for header files.

Maybe on your system. See http://stackoverflow.com/questions/12106204/omp-h-header-file-not-found-in-suse-linux and http://stackoverflow.com/questions/13999255/xcode-c-omp-h-file-not-found for some other people's experiences.

We support a variety of distributions as well as OSX and Windows. You can't assume that any two distros, even those with a clone relationship like Debian->Ubuntu->Mint, have the same, or even close to the same, tools installed. That's especially true of Ubuntu, which is extremely aggressive in pushing the latest development versions on its users. You also can't assume that everyone who wants to build Gnucash is running the latest versions. We include in the README the oldest RHEL version we support for a particular development cycle.

> I said i had a few years programming experience with C, i have had little under a year with OpenMP and then that was infrequent due to the area i am in. I have experience mainly with embedded platform development with C, assembler and even hard coding. I have done a few minor programs for desktops including the first few steps for gtk, opengl, sdl, win32 and so forth. Yes i have a few years experience in C programming but it is in no way the same kettle of fish. It would be like assuming that person x can make programs for windows so they can make the next blockbuster computer game. I apologise if i mislead you to my familiarity of programming for projects in this area.

You said:
"I am a competent C programmer and I have dabbled a little bit on GTK, SDL and OpenGL. I also have a few years experience working on parallelization of programs using OpenMP. "

That's pasted directly from your first post to this list. You've created a bit of a credibility problem for yourself.

> 
> >For what value of “beginner”? Beginning programmer? We don’t have time to teach. First time working in open source? Maybe, but there’s probably other material out there that’s >better than what we’d be likely to write. It might be worthwhile adding pointers to some of that material on http://wiki.gnucash.org/wiki/Development .
> 
> Yes, that would be beneficial. But i was considering more of, this developer is new to this project, before he or she contributes they should be familiar with: auto-make, check this page to know the general community ethos, this page for current plans such the change to C++ that has been mentioned. There are a few things which you have mentioned that it would have been impossible for me to find on the wiki. Just having a page that new developers could use as a check-list before attempting to contribute perhaps could avert problems like this from happening in the future.

http://wiki.gnucash.org/wiki/Development and the pages it links to are intended to provide that information. At http://wiki.gnucash.org/wiki/Development#Tools you'll find the line:
"Gnucash uses the Autotools as build system "

I agree that there should be some more links, including http://wiki.gnucash.org/wiki/Development_Process and http://wiki.gnucash.org/wiki/Roadmap.

> 
> >I said *that particular file* wasn’t a good candidate.
> 
> >OK. What are your actual strengths? Can you refactor? Write unit tests? 
> >Consolidate API? (Not until January, though, we’re not committing 
> >anything that doesn’t directly fix bugs until we release 2.6.0 at the 
> >end of next month.)
> 
> I misunderstood, would you recommend a file which would be a good candidate? Like i have previously mentioned, i mainly work with embedded projects, so i am used to lower level coding rather than allot of abstraction through APIs, simply put, they don't really exist in the embedded world. I have used a few APIs but not to any extent i would say i could immediately start making code for it. I can refactor, never written a unit test. I can consolidate APIs, i have done so at an embedded level, i would of course need to get up to speed with the relevant API first. I can handle files, complex bit logic, reduce variable waste, analyse bottlenecks. Most importantly i can learn, if there is a particular area i need to get familiar with and i can develop it.
Look through src/app-utils, src/core-utils, and if you want to polish your Gtk-fu, src/gnome-utils. There's plenty of "smelly code" in there to refactor. To make sure we're using the same meaning, I mean refactor as explained in http://martinfowler.com/books/refactoring.html .

From a professional development view I recommend learning about unit tests and test-driven development. If you've no experience with object-oriented programming, you'd better get going on that too unless you're planning to spend your life using C as a glorified assembler. Nothing wrong with that, of course, it can pay extremely well. But it's a skill that isn't much applicable outside the embedded world. Grokking OO is mandatory for modern application development, including Gnucash. It isn't easy and it might take you several years; it did for me. 
> >> Perhaps you could set up a wiki change pre approval system so mistakes like that don’t occur in the future. I made that 
> >> change as the wiki said to make documentation changes before changing 
> >> code.
> 
> >That wasn’t a documentation change, it was an announcement of something I explicitly told you we weren’t ready to do.
> 
> Again, i didn't intend for it to be like that. This was more to an effect of: a change was made by someone who didn't have the right to make that change. 
> 
> The change should not have been able to be made in the first place. There is nothing stopping someone from making changes without pre approval. Anyone with an email
> 
> could go along and post a completely different game plan, or delete text from entire selections. I’m saying, yes it happened, something that you didn't want to, we should 
> 
> find a way to ensure that can't happen again, either as a mistake or as an intentional way to cause trouble. One method of doing that would be to allow changes to be 
> 
> made but force them to be moderated before they are made visible to anyone else.

Again, that's how wikis work. There's tons of material on the web about that, beginning with the somewhat recursive
http://en.wikipedia.org/wiki/Wiki
> 
> I hope that i have clarified my position and gone some way to fix the problems i caused. Please say if you would prefer me to stop attempting to work on this project and i 
> 
> will go and find another way to develop my skills.
I'm not going to tell you whether or not to work on Gnucash, but I'll tell you that the skills you've claimed in this latest post are not a good fit with Gnucash or any other  application project, and you'll have to learn to think in a very different way before you can make significant contributions. Contributing to Gnucash--or any other project--isn't a way to develop skills. It can be a good way to practice new skills that you develop by studying. I That might interfere with your day job and it will probably take a long time. You'll have to decide for yourself whether it's worth the study time or if you should instead look for a project where your close-to-the-metal skills would be more helpful; compilers and kernel development come immediately to mind.

One more thing: Can you do something about your mailer's weird line spacing?

Regards,
John Ralls