RAD Studio 2010 Review #11: – DataSnap

DataSnapGuys, don’t forget that we’re talking about a prerelease beta. Anything can change till release. Also, let us not forget to thank Embarcadero for giving us the permission to talk about this.

First of all, what is very nice in the new DataSnap is that you have the feeling of a n-tier solution which is usable in the real sense of the word. The great thing is that because there is a DataSnap implementation on .NET and because DataSnap can use JSON, you can have a seamless data exchange between native Windows, .NET, and Java applications. Pretty neat I’d say.

But let’s see some from the DataSnap features. First, you have… a wizard which will do for you the plat work of creating and configuring a DataSnap server:


As you see above there are very ‘interesting’ features: Authentication, Http etc.

Also when we look at the result in the Form Designer and in the Object Inspector we’ll see at a glance that the new DataSnap supports REST. Also, an important thing is the ‘Filters’ property with which you can apply any transformations to the data stream. A ZLibCompression filter is provided out of the box, but if someone wants to add any other filter like eg. encryption is free to do so. See for yourself :


How it works?

The simplest way to find out is to run the demo shipped with the new Delphi. Yep, there are new demos in the package and you can learn enough from them in a very short amount of time. In my opinion this is a very good move to advertise the new features and to attract the new users.

But for the ones who want to know more from now, the principle is very simple: While in Indy (which DataSnap uses behind the scenes) you deal with streams of bytes, in Data Snap you deal with Delphi artifacts.

First, you create the code which will be executed on DataSnap server. In your Server project you’ll add as many methods you like, writing them in a classical Delphi way. Let’s say that we have the following:

procedure TParametersServerModule1.DirExists(aDir: string; out aExists: OleVariant);

Compile and run the server outside of IDE (or Run without Debugging) in order to freely close the project while our DataSnap Server is running. Now, if we’ll open our DataSnap client project and add a TSQLServerMethod instance correctly configured to point to our running DataSnap server through an TSQLConnection component then in the Object Inspector we should see our new method:

DS3 By setting up this property, the ‘Params’ property is updated accordingly. But of course we can do this also in code:

procedure TForm13.Button4Click(Sender: TObject);
    SqlServerMethod1.CommandText := 'TParametersServerModule1.DirExists'; //the full qualified name of our server method
    SqlServerMethod1.Prepared := True; //in order to get the parameters from server

    SqlServerMethod1.Params[0].Value :='C:\';

    SqlServerMethod1.ExecuteMethod; //execute the method on the server

    if SqlServerMethod1.Params[1].Value then //read and test the result
      ShowMessage('Directory exists')
      ShowMessage('There isn''t such a directory!');

    SqlServerMethod1.Close; //cleanup

…also you can use the fabulous JSON units (source code included) to serialize your class instances over the wire. The main class which is responsible for this (TJSONMarshall) has an open plug-in architecture in order to allow you to write readers and writers (also called converters) for your own custom data types. A small example to see how the things work:

procedure TForm13.Button4Click(Sender: TObject);
  LContact: TContact;
  oMarshaller: TJSONMarshall;
  crtVal: TJSONValue;
  LContact:=TContact.Create; //our custom class
  LContact.Age:=20; //fill with some data
  oMarshaller:=TJSONMarshal.Create(TJSONConverter.Create); //our engine
    crtVal:=oMarshaller.Marshal(LContact); //serialize to JSON
    Memo1.Text:=crtVal.ToString; //display
  finally //cleanup

Of course that are much more to say about DataSnap, but I think that for a quick overview is enough :-). Also I think that perhaps we should mention that DataSnap implements callbacks for server to notify the client and asynchronous method execution. Also I do think that in the current iteration, DataSnap is becoming a solid framework for distributed computing. You?

21 thoughts on “RAD Studio 2010 Review #11: – DataSnap

  1. Until DataSnap start to offer *real* enterprise-class features (i.e. Kerberos support, AD/LDAP integration, per-method ACLs, etc. – DCOM does), it won’t be a useful framework for high-end and data-sensitive applications. Right now, a C/S approach is “better”.
    Just adding a zlib filters follow the mindset of the past few years just to make simple what already is, but leaving all the hard stuff out. For example AFAIK Delphi still lacks a default encryption library (and the old TP Lockbox is pretty obsolete nowadays), thereby adding proper and strong encryption to Datasnap is not trivial.
    There other “trivial” issues, i.e. taking into account user rights on the server side (aka, impersonation, delegation, or the like) that need to pass user credential along the connection – safely.
    I am sorry the Delphi team still amazed because “hey, we can pass data via HTTP, look, we have callbacks!” when the bar is much higher today. They are still playing in the kids league.
    And I really wish they could take “SQL” away from any remoting component.

    • How could it have AD integration OOTB when it is designed as a platform neutral n-Tier solution?
      Also implementing ACLs (or any other authentication method for the server – not for the communication) is the developer’s responsibility, because in fact, the developer writes the server. Imho, of course.
      On the encryption: we agree that Delphi should have an encryption library OOTB. OTOH, is trivial to add such a library to DataSnap. (ie. isn’t trivial to write one but is trivial to use such a library in DataSnap once you have it).
      As I made clearer in my previous post, DataSnap has no connection whatsoever with DBX and/or SQL. Yes, we agree that, perhaps, the name of the component should be changed from TSQLConnection to TConnection but is too late now.

      • “How could it have AD integration OOTB when it is designed as a platform neutral n-Tier solution?”
        Because when you’re on Windows you need it to simplify management? Because AD is LDAP compatible? Because that’s where users and permission are stored? Because I wish I could avoid to reinvent the wheel? Because “enterprises” rely on directory services for a lot of tasks? Because that’s what customers ask when deploying such kind of applications in large companies?
        I never said it should be the only avalable method – I wrote “LDAP” too – but when all of your applications are in a Windows domain (and not only, there are ways to use AD from other OSes too, lots of Linux users here authenticating against our AD domain) are you going to mantain your own users database, and handle everything yourself?

        “Also implementing ACLs….” but you need the hooks. You need a protocol that can transfer user credential safely. Right now AFAIK a DataSnap call has no idea about who’s calling the server, and little control under which privileges the call is executed – unless you write a lot of code. Up to the point you can easily write the data exchange and method calls too….

        “OTOH, is trivial to add such a library to DataSnap”. Try, and tell me how easy is to add such feature. It’s not just adding a “Crypt/DecryptBuffer” call. You have to authenticate the endpoints, exchange the keys safely. Protect from “man-in-the-middle” and “reply” attacks, and the like. Again, what do you use to get authentication data? Where do you store the keys?

        There are some standard technologies like SPNEGO/GSSAPI to implement some of these features, but Delphi is careful to keep itself away from any real “enterprise” technology.

    • @LDS:
      I should first confess : yes I like “progress”.
      Though, progress does not mean “perfection”, despite the “P”s.
      Your comment is IMHO valuable – as a matter of kinda “(.Net) reflection” i.e. on a developpers PoV.
      Frankly speaking:
      most existing *real* enterprise-class features have NEVER been properly deployed – in real life, most of the companies (even some big players I know about) deploy/implement “nonsense”.

      it’s more of a “design/architecture” related choice to integrate it or not.
      As for example consider “firebird”.
      There you have (had?) a design “without” encryption of the DB-files or of the streamed data.
      The Firebird team states explicitly: encryption is matter of the network/server admin. the admin should be able to protect the file and ensure safe connections.

      Datasnap “optional” (developper’s responsability) encryption complies with flexible architecture design and may better adapt to the future technologies/frameworks.

      • “The Firebird team states explicitly: encryption is matter of the network/server admin. the admin should be able to protect the file and ensure safe connections.”

        Until one of the server/network admins steals your database files, or sniff data on the wire, and sells them 🙂 You may want to restrict access to data to DBAs (and not all of them, that’s why Oracle has Vault) and people needing them. You don’t want server operators or hardware technician be able to copy and access your sensitive data. There are cases when you have to protect data in memory too.
        Firebird team can say what they say because they work on a small database rarely used to store very sensitive data. And if you have different application accessing the same data, you wish you could have a single, proven, standard way to protect data, you don’t want each application using its own, maybe flawed, protection technique.

        I know there’s a lot of bad implemented and depolyed “enterprise”-class applications. That’s not an excuse to add more. IMHO multi-tier without proper protection and functionalities is just useless, and adds more complexity without any real advantage. And data protection today is not an option – is a must.

  2. I really wasn’t impressed with Midas. I think it’s fair to say that Borland mismanaged it.

    The latest DataSnap sounds like a significant, if long overdue improvement. It now seems to do most of what I need for n-tier projects, so it’s time to take another look.

    I agree with LDS that encryption support should be available out of the box, though.

    • Midas was a good idea when it was released in Delphi 3, but made very little progress since then. At least the DCOM based version can (could?) take advantage of the DCOM features (i.e authentication, authorization, integrity and confidentiality) , although it was complex to develop and manage, and the implementation suffered from Win9x support and lack of real updates later.
      Unluckily the new Datasnap looks more a dbExpress byside product useful to transfer data to/from web pages than a real attempt to implement a full remoting library useful for native applications. Look at WCF – it has much more features and power. And that’s what they have to compete with – not PHP xmlrpc…
      I am using DCOM extensively, but I need far more power and control than what DataSnap offers now.
      Don’t know if they’re avoiding explicitly to enter RemObjects realm – they gopt Prism from them – maybe they could license remoting too…

      • No, DataSnap has nothing in common with DBX. (See above my previous comment – also see my previous post). No, DataSnap isn’t designed specifically to transfer data/from to web pages. I wrote IntraWeb applications and I never felt the need for something like DataSnap to fill the ‘holes’. (OTOH, I see value in DataSnap) Why do you think that?

        • Nothing in common? C’mon, look at your own code – that’s calling stored procedures, that’s not RPC! 🙂 Ok, you can have a proxy built and hide that awful code – but that’s how it works.
          And if you look at the genesis of this Datasnarl, you will see it rooted in some dbExpress low level ways of exchanging data. It was not designed from scratch to be a remoting protocol, or it wouldn’t be designed that way.

          “isn’t designed specifically to transfer data/from to web pages”. So why it uses JSON instead of packing and transferring binary data – at least as an option? Because JavaScript is not very well suited at manipulating binary data? Why should data be transferred as strings??? That could be ok for web pages where almost everything is text – much worse for application that transfer mostly – and a lot of – numbers and binary data. Ok, ok, there’s endianness, there’s streams and so on, but strings are just a stupid way to transfer non-text data, believe me. They are for humans, not computers. Unluckily it looks most web developers can handle only strings – I’ve saw one writing code to do bitwise operations transforming data into strings and then looking for ‘0’ and ‘1’… still shuddering thinking about it.
          And its lack of good ways to properly protect the communication channel shows it was designed with only one kind of application in mind – the web application.

          “I never felt the need for something like DataSnap”. I’ve been using extensively RCP, DCOM and – unluckily – Datasnap for years – that’s why I feel this implementation looks flawed since its inception.

          Believe me, if that’s how Delphi implements remoting, they should rename the “Enterprise” edition to “Inprise”. That’s where they are still.

          • JSON is just a language and OS agnostic way to exchange data, like XML, except it’s lighter weight and more human readable. If DataSnap used a proprietary binary format, clients and servers could only be written in Delphi on Windows. I’m glad they’re using an open standard.

            Which reminds me. Is there a widely accepted (read: not just Microsoft) binary standard for exchanging data? If there is, then it might make sense to support that, too.

            • DCE/RPC has a standard to exchange binary data. It’s complex, though, because it has to take into account endiannes, character encoding – can JSON take into account EBCDIC? 😉 -, different floating point encodings (IEEE, Cray, IBM, etc.), and other issues. See DCE/RPC 1.1 Technical Standard, “Transfer Syntax NDR”.
              Anyway Delphi is a tool to write *native* Windows applications. If I do not need to exchange data outside Delphi applications (and data representation has more to do with the underlying CPU than the OS), why should I incur the penalty of transforming everything to and back from strings? WCF does offer an optimized binary representation of data when perfomances are important. Because it wasn’t designed to be used by ASP.NET only.
              Another issue I see now is they tightly coupled the data representation and the remoting protocol. They have Datasnap/JSON and separately WS/SOAP. It does little sense to me.

                • I would have no problem, but DCOM support in Delphi is really stale and more or less the same it was in Delphi 4 or 5 when they integrated some of the improvements Dan Miser made. Since Miser no longer used Delphi Midas saw very little improvements. And because MS “deprecated” DCOM in favor of WCF and the new Datasnap has been introduced there won’t be any improvements. But the new Datasnap is not a suitable replacement yet.
                  It looks we are in the same situation when they dropped BDE support for RDBMS but dbExpress was so poor and buggy it wasn’t a replacement at all. We ended up using third party components. But we are already using too many to add another library too…. each upgrade is getting to be a nightmare and IDE setup too.

              • DataSnap sends and receives binary data. The communication layer, which is based on Indy as I said, doesn’t know what it sends. For your convenience, if you want, you can use some units to stream objects using the JSON format. Also, you can transform them in XML. But of course you can write any serialization engine you like. JSON isn’t tied to DataSnap. You can use the JSON units to write a .dfm-like file for example.

                • “DataSnap sends and receives binary data.” Because TCP/IP does not know anything about strings, and so HTTP does, when you’re bytes are going on the wire they’re binary data – there were proposal to use xml and thereby strings to transport network data over IP (see RFC 3252) ;), but they were never implemented… 😉 Just because they didn’t had JSON in 2002, I guess 😉 But or this post is wrong (http://blogs.embarcadero.com/pawelglowacki/2009/03/03/38672), or Datasnap does use JSON for its data represetnation across the network.
                  “But of course you can write any serialization engine you like.” Actually, I could even write my own remoting library 🙂 The problem is what I get out of the box, without having to write a lot of low-level code and instead be able to concentrate on writing application code. It’s the usual price/feature ratio.

    • Indy may not be optimal, but Delphi has little more – unluckily. There are Synapse and ICS also, but Indy AFAIK is the most extensive one – although sometimes quality and some design choices may be questionable. IMHO Embarcadero should take control of Indy development, because TCP/IP is no longer an “auxiliary” library, and the VCL still lacks a lot of needed libraries.

  3. Indy is good – I use it a lot in my projects.

    It’s really good to read of the improvements to DataSnap in D2010, especially that it is no longer restricted to DCOM – which is hard to configure and maintain.

  4. I’m truly enjoying the design and layout of your blog. It’s a very
    easy on the eyes which makes it much more pleasant
    for me to come here and visit more often. Did you hire out a designer
    to create your theme? Excellent work!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s