dlopen error with newly introduced (but still unused) c++ class

[Rob Gowin]

Geert,

The reason you are not seeing these errors at link time is because, as
Derek hinted at, when you link a shared object, the link step is only
checking that the object files in the shared object are internally
consistent. It will not check that any external symbols that are used are
actually available.

Without the extern "C" you will encounter runtime crashes in the CSV
importer when you try to access a function that your code thinks is C++,
but the core thinks is C. This will be similar to the runtime errors you
ran into a couple of months ago (
http://lists.gnucash.org/pipermail/gnucash-devel/2015-December/039239.html)
that I fixed with a PR to handle some extern "C" issues.

For example, consider the "xaccParseAmount" function used by
gnc-csv-imp-trans.o. Using the 'nm' tool, we can poke around and see the
link symbols used for this function, which is defined in app-utils. If you
run 'nm' (on Fedora 23) on src/app-utils/.libs/gnc-ui-utils.o you'll see

 $ cd gnucash/src/app-utils/.libs
 $ nm  gnc-ui-util.o | grep xaccParse
   0000000000003030 T xaccParseAmount
   0000000000002760 T xaccParseAmountExtended

The 'T' stands for "text' and means the function is defined in this object
file.

If you do the same thing for an object file in the csv-importer that does
not have the extern "C", you'll see that the function names are mangled
(where 'U' means undefined):

  $ cd gnucash/src/import-export/csv-imp/.libs
  $ nm gnc-csv-imp-trans.o | grep xaccParse
    U _Z15xaccParseAmountPKciP12_gnc_numericPPc
    U _Z23xaccParseAmountExtendedPKcijjjS0_S0_P12_gnc_numericPPc

The 'nm' command has a handy argument '-C' that will demangle the name:

  $ nm -C gnc-csv-imp-trans.o | grep xaccParse
     U xaccParseAmount(char const*, int, _gnc_numeric*, char**)
     U xaccParseAmountExtended(char const*, int, unsigned int, unsigned
int, unsigned int, char const*, char const*, _gnc_numeric*, char**)

To resolved overloaded functions, C++ needs to encode the types of the
function parameters in the function name, which is what all of the extra
symbols in the C++ names are doing.

After adding extern "C", the undefined symbols now have "C" linkage, are
not mangled, and will be matched with the functions in app-utils.

  $ nm gnc-csv-imp-trans.o | grep xaccParse
                 U xaccParseAmount
                 U xaccParseAmountExtended

--

The reason you don't see the kinds of errors until runtime is because when
GnuCash does the dlopen(), it does it lazily. The code is in gnc-module.c:

  303     /*   g_debug("(init) dlopening '%s'\n", fullpath); */
  304     gmodule = g_module_open(fullpath, G_MODULE_BIND_LAZY);
  305     if (gmodule == NULL)
  306     {
  307         g_warning("Failed to dlopen() '%s': %s\n", fullpath,
g_module_error());
  308         return NULL;
  309     }

This means that any symbols in your library, like the inadvertent C++
symbols because of missing extern "C", are not resolved (looked up) until
the point they are actually needed. If the symbols are not found, you will
get a runtime crash. While you are developing your csv-imp module it might
be a good idea to change "G_MODULE_BIND_LAZY" on line 304 to 0, and change
line 308 to exit() so that GnuCash attempts to resolve symbols right away
(when the shared object is loaded) and exit if any are missing. (But don't
check those changes in.)

Or not. It seems on the Mac that symbols for shared objects are fully
resolved at link time (this is how I noticed the missing extern "C" on
IRC). So when I eventually get your new code running on CMake on the Mac,
I'll come hunt you down when things don't link. :-)

Regards,

Rob
(codesmythe on IRC)





On Sun, Jan 31, 2016 at 1:09 AM, Geert Janssens <geert.gnucash at kobaltwit.be>
wrote:

> On Saturday 30 January 2016 08:33:56 John Ralls wrote:
> > > On Jan 30, 2016, at 6:04 AM, Geert Janssens
> > > <geert.gnucash at kobaltwit.be> wrote:
> > > When running gnucash in this setup, I get this warning:
> > >
> > > WARN <gnc.module> Failed to dlopen()
> > > '/home/janssege/Lokaal/installs/gnucash/master/lib/gnucash/libgncmo
> > >
>
> > I'm a bit puzzled about how exactly the C++ bits are integrated with
> > the rest of the importer, but I think your problem is that you're
> > including C-linkage headers into C++ without declaring them as such.
> > For example in gnc-csv-imp-trans.hpp, you have
> >
> > #include "config.h"
> > #include "Account.h"
> > #include "Transaction.h"
> > #include <vector>
> > #include <memory>
> >
> > it needs to be
> >
> > extern "C"
> > {
> > #include "config.h"
> > #include "Account.h"
> > #include "Transaction.h"
> > }
> > #include <vector>
> > #include <memory>
> >
> > to tell the compiler to use unmangled names for the symbols from those
> > headers. That lack of mangling is called C linkage. You need to do
> > that in both headers and implementations. If a header will be
> > included in a C file, wrap the 'extern "C" {' and closing } with
> > #ifdef __cplusplus … #endif so that the C compiler doesn't see it.
> >
> > Any C++ functions called from C need to also have C linkage, and
> > obviously must be free functions, not class members. Only the
> > declarations need be wrapped in extern "C" {…}. The definitions
> > shouldn't be and can be as C++-y as you want. You can look at
> > qof/gnc-numeric to see an example.
> >
> > Regards,
> > John Ralls
>
> Thanks for the detailed explanation John.
>
> I can imagine you are puzzled about the integration of the c++ parts with
> the rest of the
> importer - they aren't yet... I just wanted to run gnucash to experiment
> with how the old
> importer works when I ran into the dlopen issue.
>
> The dlopen issue was effectively resolved on IRC by codesmythe. He pointed
> out I had a virtual
> member function GncTokenizer::tokenize that didn't have an implementation.
> Adding a minimal
> implementation made the error go away, even without the extern "C"
> wrappers (though to be
> complete codesmythe did suggest to do that as well).
>
> I'll add the extern "C" wrappers next.
>
> Geert
> _______________________________________________
> gnucash-devel mailing list
> gnucash-devel at gnucash.org
> https://lists.gnucash.org/mailman/listinfo/gnucash-devel
>