[GNC-dev] Import roadmap

[john]

Those XML changes sound like the wrong direction. XML code needs to stay in libgnucash/backend, not get mixed in with engine/QOF, especially since the long-range plan is to turn XML into a backup format with SQL being the primary storage and data access mechanism.

I understand the reluctance to cause a regression, but no matter how you go about it you're going to be writing new code. Unlike a C -> C++ conversion you can't do a function at a time with Scheme -> C++, nor can you use the same tests for both the old and new code. If you wire into the existing C import code you'll at least avoid creating bugs in those parts of the process.

Regards,
John Ralls




> On Feb 4, 2024, at 16:50, Brian Rater <blrnh94 at gmail.com> wrote:
> 
> Great.  Thank you for the insights.  Before I got overwhelmed with work and other commitments last year, I was experimenting with a set of base classes that would replace the lower level qof infrastructure.  As part of that, I was also looking at an alternative xml concept where xml provided some fundamental utilities, but each class such as book and transaction would use those to read and write themselves.  This keeps the data structures contained in the class itself rather than xml w needing to know about engine class data structures.  That effort did not progress far enough to bring up for discussion here or do a development branch pr, however.
> 
> I’m comfortable with redesigning and rearchitecting, but I am planning to go cautiously with import/QIF as it sounds like a lot people use that and the format is not well defined.  I don’t want to introduce new bugs.
> 
> On Sun, Feb 4, 2024 at 6:50 PM John Ralls <jralls at ceridwen.us <mailto:jralls at ceridwen.us>> wrote:
>> 
>> 
>> > On Feb 4, 2024, at 12:11 PM, Brian Rater <blrnh94 at gmail.com <mailto:blrnh94 at gmail.com>> wrote:
>> > 
>> > I've been looking at the import code and documentation and I'm wondering
>> > what the overall plan and path is.  It looks like there is a "generic"
>> > implementation in the top level directory which has utilities that are
>> > being converted to C++.  I can see the importers making some calls into
>> > this.  What is the plan and where are we at?
>> > 
>> > In particular, what needs to happen with QIF, which is still written in
>> > guile?
>> > 
>> > I'm asking because I'm looking for a task to work on.  My particular
>> > passion is writing C++ classes to replace legacy code along with adding
>> > tests and documentation.  I've been reading over the source code and
>> > following the development list for several months, but I've had limited
>> > bandwidth until recently to make contributions.
>> > 
>> > I've been looking at converting Import/QIF to C++ because it is written in
>> > guile (higher priority than converting the C code in my opinion), is fairly
>> > self contained, and the only alternative guile conversion code is reports,
>> > which is way too large in scope for where I am.  Converting QIF has
>> > drawbacks such as needing extensive testing, and having limited existing
>> > tests for regression testing, however.
>> > 
>> > My current thought is to directly translate the existing algorithms and not
>> > use the generic import to minimize introducing bugs, but others have
>> > probably put more thought into how to handle this.
>> > 
>> > So I'm looking for some direction either on what the end goal is and how to
>> > achieve it, or direction to leave QIF alone for now and work on some other
>> > area of the codebase (suggestions welcome).
>> > 
>> 
>> Hi Brian,
>> 
>> Welcome to GnuCash.
>> 
>> Rewriting the QIF importer to C++ is a reasonable task if that's what motivates you. It's not on the roadmap in more than the most generic "get rid of Guile" sense, but it does seem like a good project to take on. It's as good a legacy conversion project as any at this point. The rest are kind of blocked by the XML->in-memory SQL DB so that we can lose QOFQuery and stop creating huge linked lists of every object in the book. That's design-and-implement rather than rewrite an existing design so maybe not something you'd be passionate about.
>> 
>> IMO it would be better to rewrite the QIF parser and transaction creator into C++ and then wire it in to the "generic" infrastructure so that it presents the same user experience as other imports. 
>> I'm not sure how well that fits with your passion either, but I'm also not convinced that it makes sense to reimplement functional algorithms as OO classes. I reimplemented the Scheme report options system into C++ a couple of years ago and it turned out to be a largely start-from-scratch redesign.
>> 
>> Regards,
>> John Ralls
>>