We all know that Delphi is a really great tool. Many consider it #1 in Desktop Windows development. So do I, but perhaps I’m biased. I use it from so many years, and I still have to find something better that it, especially when it comes to talk about complex GUIs, speed of development, speed and size of executable, power and flexibility given to the developer, mainly when we speak about database-related applications.
Pretty nice advertisement, eh?…
Well, many of us think that this is true, especially with the latest versions of it (I mean D2007, D2009) – guys, no religious wars, please! The post is about another thing.
It’s about a thing which, while “inexistent” (note the quotes) from technical POV (perhaps) it was (and still is!) one of the Most Expensive Human Bugs which I found in my entire career…
One of the most basic (and used) features in an n-Tier application is a so called ‘in-memory DataSet’ which (for ones who doesn’t know) is a TDataSet descendant which holds all its data in memory. Commonly used for caching / buffering, fetching / briefcasing data from various DB servers, as well as stand-alone, file based, data engine etc. Anyway a very used workhorse. And one of the most important features is, of course, speed.
The Genesis of the Thing
Many years ago, (almost ten, I think), Borland (then the owner of Delphi) introduced an in-memory DataSet called TClientDataSet. We all jumped on it but it was sloooooow (and it required a DLL – thing which latter was addressed by having the option to embed the DLL right in the .exe). Dog slow. All DB developers (a huge mass) perhaps would prefer it, but we’re forced to switch to other 3rd party solutions which were much faster. To have an idea, there were (and still are) in-memory datasets which are 10 times faster. (!) (Yep, folks, we did benchmarks as anyone else).
The Domino Effect
But TClientDataSet was a very nice piece of code. And Delphi team built underneath a data access framework (DBExpress – DBX for short) which even if it was also ‘a very nice piece of code’ it relied on TClientDataSet to do CRUD operations with the data in a true RAD way. Of course, you can display/edit/insert the data “by hand”, but…
So, even if DBX was (and is) a solid infrastructure, the final link was a very weak (slow) one, and as we all know, the chain is a weak as it’s weakest link. On the other part, of course, there were a blooming market of 3rd party vendors providing their frameworks (masterpieces of code btw), exploiting this weakness which dragged down the entire official Delphi database access solution. And mind you, this is one of the main Delphi sweet spots…
What amazed me is that TClientDataSet didn’t like at all as a pile of junk. Well thought, nice features, a joy to use theoretically.. Same applied to DBX. Retest it as new versions of Delphi appeared. Looking at Help. Nothing…
After the Borland’s spun off, the Delphi team (which suffered some changes) became more open and we begun to know them. They seemed wery good programmers, thing which increased my surprise. Finally, I “hooked” one of DB engineers in a newsgroup and I showed him the benchmarks. He was very kind and promised that he’ll “have a look”. Indeed after few ours (a very short response time, considering his workload) he spot the “problem”. And the answer was stunning:
Try to set the TClientDataSet’s.LogChanges property to ‘False’. And please re-run the benchmarks and tell me how is going.
…and indeed that was it. After ten years. I looked immediately again at Help. Nothing. In Object Inspector… nothing. Perhaps a Readme.txt… somewhere… Release Notes… Google… Aha! There were some guys who knew about it. But many others who don’t (read the entire thread).
And the bug?
Is in human communications. Such a thing which has such severs impacts on performance should be known by anyone who had an interest in evaluating that solution. Imho, they should made the property ‘published’ (in order to appear in the Object Inspector), to document the property in Help and to specify in a distinctive formating something like “NOTE: The LogChanges property logs all the dataset’s changes in order to allow you to post the things back to the database. But for large datasets it severely affects performance.”
Ok, we are very happy with out 3rd party solution. But for us was an unforgettable lesson that the human factors prevail the technical ones. We should never forget that the ones for which we code are humans and we must guide them how to use our product. And we must be sure that we know the pain points of our user base.