[offtopic] marshalling

[Tyson Dowd]

On 04-Jan-2001, linas@linas.org <linas@linas.org> wrote:
> It's been rumoured that Tyson Dowd said:
> > If you develop super-SWIG for N languages, either it
> > 	(a) understands just one sort of "header file" (e.g. C header files)
> > 	   and generates N different bindings 
> > 	(b) understands N different "header files" and generates N
> > 	   different bindings
> > 
> > Doing (a) is just the same as using an IDL, 
> 
> No its not; especially not if your code is native C.  No different as
> saying that the output of reflection is the common intermediate form.

Perhaps this is a matter of viewpoint.

Only one language on earth is C, so everyone not programming in C has to
learn C to write the "header file" for SWIG.  I feel this is just the
same as using an IDL, except that you might already be programming in C
or already know C, in which case it's a little easier.

> > Besides, as the implementor of super-SWIG, you have to maintain N
> > different bindings, while those languages go their separate directions,
> > which is a tough job.
> 
> But that is always the case, no matter what.  standardizing on a VM
> or on some intermediate representation e.g. the output of reflection,
> doesn't change that.

Will, I don't think it's quite the same -- if you write a 3rd party tool
to do interop, it's very hard to get the languages to help write your
tool or keep it up to date.

But if you provide a VM that does the interop automatically, as long as
people are interested in running on your VM, you don't have to worry so
much about tracking each of the languages running on it, just provide
the best VM you can and they will have to interop together.

But this is the "best case" for the VM writer -- I'm sure that in
practice they do have to do some work to keep the languages interoping
(at least they have to support major features if languages start adding
them).  Similarly I'm sure that SWIG does get some help from the
languages themselves.

> > Doing (b) is an N x N problem, where N is the number of languages that
> > want to interoperate.  This is N times tougher than (a).
> 
> No its not. e.g. for the pnm image conversion suite: convert anything to
> pnm, and pnm to anything; its a n+m problem.   Likewise, convert
> anything to some intermediate reflection-like language, and generate
> all bindings from that intermediate form.  The intermediate language
> doesn't have to be standard, it just has to be malleable.

Ok, if you use an intermediate form that's doable.  But your
intermediate form must both:
	- be a superset of all N languages
	- be reducible to any subset i (0 < i <= N)
It's a tough requirement, and I'm sure that compromises will be necessary.

I think this is why MS provided a VM that does most of the N language
features, but they reduced the interoperability down to a "standard"
subset of that, with the rest being "optional". 

> > But I also think that a open-source universal VM for Linux et al would
> > be a very nice thing too.  
> 
> Would it? Do you really think that you could convince the developers
> of perl, tcl, guile and kaffe  to use this VM? or even one of them to
> do so?  This seems highly unlikely to me.   First, there are social
> issues (try convincing a BSD user to use linux: the kernels are
> *almost* the same, and the libraries & apps really are the same).

Social issues are tough.  It requires marketing, spin, luck,
partnerships and probably money to do this.  And a moderately compelling
technical solution.

> Next, there are probably deep, powerful technical issues.  
> I once partook in an effort to put opengl and phigs on top of the 
> same 3D engine.  Despite both of them being 3D API's, its 
> fundamentally undoable. Its undoable in the sense that you 
> couldn't live with the result: you'd loose performance,
> you'd gain extreme complexity, you'd loose ability to optimize 
> certain parts of the pipeline, you couldn't implement certain
> function in hardware.   I can't imagine that the issues confronting
> programming languages are easier than 3D.

They aren't any easier.  I don't underestimate the size or scope of such
a project.

The right approach will solve some of these issues, but you will still
be making a trade-off of interoperability vs performance.

In a 3-d engine such a trade-off might be unacceptable.  But in general
purpose programming it might.  Many scripting language implementations
are very slow, but as people don't do grunt work with them it just
doesn't matter.

> I would guess the reason that VB7 won't be backwards compatible with
> VB6 is precisely because of these issues: they couldn't share a VM
> with C# and still have it work.

Not in any useful sense of the word "share".

But VB breaks in every iteration.  The book publishers and consultants
love it.

> But I guess we've really gone off-topic now.

And then some.

It's a very complex topic, but I enjoy discussing it.  Of course, some
of my thesis ranges into this field, so you have to be careful when you
get me started on it.

-- 
       Tyson Dowd           # 
                            #  Surreal humour isn't everyone's cup of fur.
     trd@cs.mu.oz.au        # 
http://www.cs.mu.oz.au/~trd #