GDA save missing records

[Mark Johnson]

Graham Leggett wrote:
> Keith Bellairs wrote:
>
>> Speaking as a user and not someone busting his butt on this, I hate 
>> the idea of "unlimited" everything when we go to a DB. Most of our 
>> databases have a mechanism (BLOB/CLOB) to store really big things, 
>> usually at the cost of indexing or searching (other than with special 
>> hacks -- Oracle Text, for example).
>>
>> gnc is not, and should not be, a doc mgmt system. I want fast, fast 
>> retrieval and summarization. Having a place to store a reference to a 
>> doc is a great idea; plugging up the data with the docs, not so much.
>>
>> Of course, it is unforgiveable to just drop rows. Even silently 
>> truncating data is pretty dubious. Don't know Postgres and Mysql; 
>> can't we throw an exception so we have a chance to do the right thing 
>> (what the user needs)?
>>
>> I'd ask the developers to pick some reasonable size for each column. 
>> Then publish the schema. Granted this is a big change from the 
>> unlimited everything, but it seems necessary. If I don't like your 
>> column size, I should be able to ALTER TABLE and set my own 
>> favorites, so please do not hard-code the column sizes into the code.
>
> The problem with this is that it introduces inconsistency into the 
> code. The XML backend has no concept of line lengths, and is so 
> "unlimited". The problem was originally found when an attempt was made 
> to import this "unlimited" data into a "limited" system, such as the 
> current DB system.
>
> Suddenly we have introduced the possibility that perfectly valid data 
> in one backend is no longer valid in another. Add to that a user 
> ability to change the line lengths and suddenly all bets are off.
>
> Fixed length string widths are an optimisation that helps if you are 
> manipulating fixed length strings, but if you aren't - such as with a 
> description in a register - the fixed length serves no purpose at all.
Technically, varchar(n) is not a fixed string length.  That would be 
char(n).  The varchar is a variable-length string with a maximum length. 

Each of MySql, PostgreSQL, and SQLite treat them differently on insert - 
truncation, rejection, and ignoring the maximum, respectively.
>
> As someone who spends a lot of time tracking down nasty problems in 
> software, I can tell you that this is exactly one of those seemingly 
> harmless issues that can cause some very difficult to find, and 
> therefore very expensive bugs in systems. In this case, it was only 
> found because mysql and postgresql have different behaviour when 
> string lengths are too long, and that was found by a very lucky accident.
Ahem, it was not an "accident"; it was a test.
>
> Regards,
> Graham
> -- 
Mark