The Memory Barrier: Poll results, Comments, Solutions.

aquariumHi again – we will return to strategy posts soon, but till then, I owe you some results. Asking “Would you like a solution delivered quickly which would allow accessing more than 2/4 GB?” (see here) it seems that the results are quite balanced, however it seems most of you want improvements in this area in a form or another. But let’s see our pie:…

Memory64

As you see there is no clear winner on this but again, it is clear that we need a 64-bit compiler. And the sooner the better, even if the pressure isn’t so big as in the case of cross-platform.

Anyway, from all data which I saw (comments, polls, newsgroups, other blogs etc.) it is pretty clear that the year 2010 will be a decisive year for Delphi. And I don’t mean only from technical point of view, there are increasing pressure in finding new business models, lowering the prices etc. And yes, I do think that they must adapt.

But let’s return to our needs: Breaking the memory barrier. Because I don’t think that they will give us an AWE solution “now” I’ll present some ‘workarounds’ bellow. Find what it is applicable for you (if any):

  • First, try to ‘compress’ your data. A fist step would be to use appropriate language elements in order to gain some memory. An important memory saver is using records instead of objects. Also consider the use of packed. Of course you can have a look at your numerals – perhaps you can choose a type with fewer bytes. And of course, it would help if you can use a true compression algorithm to compress your data.
  • You can use AWE. However as discussed in the comments of our previous post, this can be tricky – it depends on what you want to achieve. But it has very little overhead comparing with the other solutions which follow. Anyway, here you have a sample code on how this would work (Beware: It DOESN’T work on WinXP – 32bit). The code is adapted and ‘decorated’ with ShowMessage calls in order to mark the steps.First we have an unit with the bare-bones the AWE interface:
    Unit AWEManager;
    //
    // It doesn't work on WinXP 32-bit
    //
    // Is the Delphi translation of the C code from:
    // http://msdn.microsoft.com/library/default.asp?url=/library/en-us/memory/base/awe_example.asp
    //
    // Translated and enhanced from the french original: http://nono40.developpez.com/sources/source0044/
    
    Interface
    
    Uses
      Windows,
      Dialogs, SysUtils;
    
    Const MEM_PHYSICAL = $400000;
    
    Function AllocateUserPhysicalPages(hProcess:THandle;NumberOfPages:Pointer;UserPfnArray:Pointer)
              :Bool;StdCall; External 'Kernel32.dll' Name 'AllocateUserPhysicalPages';
    Function MapUserPhysicalPages(lpAddress:Pointer;NumberOfPages:Cardinal;UserPfnArray:Pointer)
              :Bool;StdCall; External 'Kernel32.dll' Name 'MapUserPhysicalPages';
    Function FreeUserPhysicalPages(hProcess:THandle;NumberOfPages:Pointer;UserPfnArray:Pointer)
              :Bool;StdCall; External 'Kernel32.dll' Name 'FreeUserPhysicalPages';
    
    Function _AdjustTokenPrivileges(TokenHandle: THandle; DisableAllPrivileges: BOOL;
      NewState: PTokenPrivileges; BufferLength: DWORD;
      PreviousState: Integer; ReturnLength: Integer): BOOL;stdcall;
      external advapi32 name 'AdjustTokenPrivileges';
    
    Function LoggedSetLockPagesPrivilege ( hProcess:THANDLE;bEnable:Boolean):Boolean;
    
    implementation
    
    Function LoggedSetLockPagesPrivilege ( hProcess:THANDLE;bEnable:Boolean):Boolean;
    Var
      Info  : TTokenPrivileges;
      Token : THandle;
      Res   : Boolean;
    Const SE_LOCK_MEMORY_NAME='SeLockMemoryPrivilege';
    Begin
      // Open the token.
      Res := OpenProcessToken ( hProcess,TOKEN_ADJUST_PRIVILEGES,Token);
    
      If Not Res Then
      Begin
        ShowMessage('Cannot open process token.' );
        Result:=False;
        Exit;
      End;
    
      // Enable or disable?
      Info.PrivilegeCount := 1;
      If  bEnable
        Then Info.Privileges[0].Attributes := SE_PRIVILEGE_ENABLED
        Else Info.Privileges[0].Attributes := 0;
    
      // Get the LUID.
      Res := LookupPrivilegeValue ( Nil,SE_LOCK_MEMORY_NAME,Info.Privileges[0].Luid);
    
      If Not Res Then
      Begin
        ShowMessage('Cannot get privilege value For SE_LOCK_MEMORY_NAME.' );
        Result:=False;
        Exit;
      End;
    
      // Adjust the privilege.
      Res := _AdjustTokenPrivileges ( Token, FALSE,@Info, 0, 0, 0);
    
      If Not Res Then
      Begin
        ShowMessage('Cannot adjust token privileges, error '+IntToStr(GetLastError));
        Result:=False;
        Exit;
      End
      Else
      Begin
        If GetLastError<>ERROR_SUCCESS Then
        Begin
          ShowMessage('Cannot enable SE_LOCK_MEMORY privilege, please check the local policy.');
          Result:=False;
          Exit;
        End;
      End;
    
      CloseHandle( Token );
    
      Result:=True;
    End;
    
    End.
    

    And now, let’s use it (I’ll post only the interesting part of the code – It is a form which has the code which shows the functionality in an OnClick handler).

    
    implementation
    
    {$R *.dfm}
    
    uses
      AWEManager;
    
    Const MEMORY_REQUESTED=1024*1024; //We allocate 1(one) MB
    
    procedure TForm2.Button1Click(Sender: TObject);
    Var
      bResult              : Boolean;     // generic Boolean value
      NumberOfPages        : Cardinal;    // number of pages to request
      NumberOfPagesInitial : Cardinal;    // initial number of pages requested
      aPFNs                : ^Cardinal;   // page info; holds opaque data
      lpMemReserved        : Pointer;     // AWE window
      sSysInfo             : TSystemInfo; // useful system information
      PFNArraySize         : Integer;     // memory to request for PFN array
    Begin
      GetSystemInfo(sSysInfo);  // fill the system information structure
    
      ShowMessage('This computer has a page size of '+IntToStr(sSysInfo.dwPageSize)+' bytes.');
    
      // Calculate the number of pages of memory to request.
    
      NumberOfPages := MEMORY_REQUESTED Div sSysInfo.dwPageSize;
      ShowMessage('Requesting '+IntToStr(NumberOfPages)+' pages of memory.');
    
      // Calculate the size of the user PFN array.
      PFNArraySize := NumberOfPages * sizeof (Cardinal);
    
      GetMem(aPFNS,PFNArraySize);
      If (aPFNs = Nil)
      Then Begin
        ShowMessage('Failed to allocate on heap.');
        Exit;
      End;
    
      // Enable the privilege.
      If Not LoggedSetLockPagesPrivilege( GetCurrentProcess, TRUE ) Then Exit;
    
      // Allocate the physical memory.
      NumberOfPagesInitial := NumberOfPages;
      bResult := AllocateUserPhysicalPages( GetCurrentProcess,@NumberOfPages,aPFNs );
      If Not bResult Then
      Begin
        ShowMessage('Cannot allocate physical pages, error '+IntToStr(GetLastError));
        Exit;
      End;
      If  NumberOfPagesInitial <> NumberOfPages Then
      Begin
        ShowMessage('Allocated only '+IntToStr(NumberOfPages)+' pages.');
        Exit;
      End;
    
      // Reserve the virtual memory.
      lpMemReserved := VirtualAlloc( Nil,
                                     MEMORY_REQUESTED,
                                     MEM_RESERVE Or MEM_PHYSICAL,
                                     PAGE_READWRITE );
      If lpMemReserved = Nil Then
      Begin
        ShowMessage('Cannot reserve memory.');
        Exit;
      End;
    
      // Map the physical memory into the window.
      bResult := MapUserPhysicalPages( lpMemReserved,NumberOfPages,aPFNs );
      If Not bResult Then
      Begin
        ShowMessage('MapUserPhysicalPages failed to map, error '+IntToStr(GetLastError));
        Exit;
      End;
    
      ShowMessage('The new memory is mapped here!');
      //****************************************
      //
      //Here you can use the memory area pointed
      //by lpMemReserved.
      //
      //****************************************
    
      // unmap - deallocate the area
      bResult := MapUserPhysicalPages( lpMemReserved, NumberOfPages, Pointer(0) );
      If Not bResult Then
      Begin
        ShowMessage('MapUserPhysicalPages failed to unmap, error '+IntToStr(GetLastError));
        Exit;
      End;
    
      // Free the physical pages.
      bResult := FreeUserPhysicalPages( GetCurrentProcess,@NumberOfPages,aPFNs );
      If Not bResult Then
      Begin
        ShowMessage('Cannot free physical pages, error '+IntToStr(GetLastError));
        Exit;
      End;
    
      // Free virtual memory.
      VirtualFree( lpMemReserved, 0, MEM_RELEASE );
    
      // free the memory hold by the array
      FreeMem(aPFNS,PFNArraySize);
    End;
    
    

    Well, that’s it. As you see you must allocate and deallocate pages in a window of your memory. Not so convenient for certain tasks.

  • Use an AWE Server like the NexusDB AWE Server. For the ones which don’t know, NexusDB is based on the old FlashFiler hence is a ‘Delphi friendly’ product. It supports Delphi 2010, but it is not free.
  • Use an AWE Library like this one. It should be faster than a separate server but you must code all the logic (especially searching) by yourself. Cheap but not free.
  • Use the 64-bit version of Firebird configured to keep the data in memory. It has the advantage that you have a full blown 64-bit SQL server (so, no AWE paging etc.) and if the data is stored locally then the server uses the local protocol which is sensibly faster than TCP/IP. And of course is free. How to configure Firebird to avoid writes on disk?
    You must set the following values:

    Forced writes = OFF (at the database level)
    MaxUnflushedWrites = -1,
    MaxUnflushedWriteTime = -1 (in firebird.conf)

    Credits goes to Paul Beach and Dimitry Yemanov.

  • Use memory mapped files. Torry has several free implementations for this. Choose the one which fits to you. Very fast, smallest overhead and it’s up to you to write the entire logic. However you have a flat memory space to play with. (Ok, you’ll see it as a file or stream but the ‘file’ actually resides in memory). Perhaps what’s worth mentioning is that mapped files use the kernel’s virtual space and don’t count against the 32-bit 2 Gb user process memory limit. Also it adds persistence, so you can ‘save’ data between sessions. Next time, of course, there will be a performance hit when the data will be read again from the disk. But if you don’t need to save the data between the sessions then everything will work as expected.
  • Use a RAMDisk. Google for them. The implementations are either free either cheap. For example this one. However there is some overhead implied but you can use it for other tasks as well. Also is very useful if you have entities outside of your control which write data on ‘disk’. Of course, a RAM disk require installation.

Did I miss something?

…and which is your favorite? Besides of 64-bit compiler of course 🙂


25 thoughts on “The Memory Barrier: Poll results, Comments, Solutions.

  1. Hi WOW, thanks for the article.

    In our particular situation we use memory mapped files, but they seem to tap out at around 700MB, perhaps because that is the largest contiguous block of memory it can allocate. Does you know if AWE can be used in conjunction with memory mapped files?

    Not sure if 64-bit would help us either.

    Cheers

    • “they seem to tap out at around 700MB, perhaps because that is the largest contiguous block of memory it can allocate.”
      Yes, you’re right. But there are workarounds.

      “Does you know if AWE can be used in conjunction with memory mapped files?”
      Yes.

    • Also, in order to not depend on a big continuous memory block you can have split your data on more mapped files (depending on your application).

      • Splitting it up might be doable. I tried the code in the article and it fails with

        “Cannot enable SE_LOCK_MEMORY privilege, please check the local policy.”

        After some research I’ve found that under Vista Home Premium (which I’m using) you can’t enable this privilege, because you can’t change the local policy. Hard to believe but apparently true – at least I couldn’t find any way to do it. If that’s the case, then I don’t think AWE can be used on this OS version.

        I’ll try it on a Vista Business machine if I get the chance.

        • Yes (unfortunately). Microsoft policy to push the users to higher SKUs. Exactly the same behavior is on WinXP 32bit. Not that technically wouldn’t work, just that it’s disabled because of “commercial issues” (TM).

    • The legend is (in this order):

      1. Yessss!!! Now!!! …but this doesn’t mean that real 64bit is canceled
      2. No, need real 64bit
      3. Neutral, do as you wish
      4. Perhaps is better to focus your resources on cross-platform first
      5. If the next release will have this only and cross-platform it will be ok.
      6. Moderately favorable.
      7. Totally against.
      8. Other

  2. Nice article. I am a bit bemused by the following though: “An important memory saver is using records instead of objects.”

    Objects are implemented as a pointer to a record structure. If you need lots of instances and your record is really, really small (in which case packed won’t make much of a difference either), then the three additional pointers (one for the object, one for the VMT and one for the monitor) will probably be a deal breaker. But if the record is quite sizeable, these are probably the least of your problems.

    • Someone who is fighting with the memory barrier reported us privately that it gained approx. 30% memory savings from switching from objects to records. But I don’t know the architecture of its application though.

        • If you use records you can save memory if you pay attention in the way you sequence the fields on it.
          For istance the following two records contain the same information but the first (A) needs 32 byte while the second (B) only 20:
          TRecA = record
          R1: integer;
          R2: byte;
          R3: integer;
          R4: byte;
          R5: integer;
          R6: byte;
          R7: integer;
          R8: byte;
          end;

          TRecB = record
          R1: integer;
          R3: integer;
          R5: integer;
          R7: integer;
          R2: byte;
          R4: byte;
          R6: byte;
          R8: byte;
          end;

    • Actually, this looks like a very workable solution for us in the meantime.

      Our DB architecture already has a caching layer on top of the file system database, and I could definitely create an L2 cache to store DB information in a more raw form…

      • After some small amount of coding, I now have an L2 layer for our DB that utilises the ‘temporary’ temp file approach mentioned above.

        It isn’t as nice as a proper 64 bit memory addressing solution, but will certainly improve things.

  3. Pingback: Firebird News » “The memory barrier” – post mentions Firebird

  4. It would be interesting to see what sort of performance benefits could come from not having to thunk between a 32bit app and a 64 bit kernel with each and ever single call the application makes.

    Memory pressures aside, that might (or might not) make a compelling case all on its own.

    I can’t say I have seen anyone discuss the topic yet (not that I follow it that extensively)

  5. You forgot to mention another important factor:

    If you have lots of strings and don’t need Unicode then either stay on Delphi 2007 or earlier or go through your code and change all your “string” and “char” declarations to “ANSIString” and “ANSIChar”.

    In the All-Unicode-or-Nothing default behaviour of Delphi 2009+, every string in your application will occupy more than DOUBLE the amount of memory that it would occupy in Delphi 2007.

    MORE than double because in 2009+ there are additional payload descriptor fields in addition to each character occupying 2 bytes rather than 1. That additional descriptor data also applies to ANSIStrings, so *every* string in Delphi 2009+ occupies more memory, but ANSI strings will incur *only* the additional descriptor but the string data itself is the same size as in 2007 and earlier.

    But I would caution against asking for a “quick” solution to 64-bit.

    The approach to Unicode was arguably determined by a desire to get it out quick and to make it quick for people to migrate.

    Both were a mistake imho and have resulted in a compromised implementation that suits nobody, least of all anyone that wishes to *correctly* implement Unicode support in a Delphi application and even less for anyone for whom Unicode was/is simply irrelevant to their needs.

  6. Once again, I would like to know how much votes you received – 20% of how many? If you discern between 20 and 21 I guess there were at least 100 votes – but I hope, there are quite more delphi programmers out there 😉

    I think Embarcadero should launch such a poll from the welcome page in delphi – so at least all users of delphi 2005 and ahead would be noticed about such a poll. So it should be more representative, I guess.

    But please: do what all serious inquiry companies do: mention the absolute size of of the probe!

    Regards, Fritz

    • “Once again, I would like to know how much votes you received”

      …hehehe… ~ 200. Much less than on the other questions where we had till 1350. Ken was right: many of them even didn’t bother to vote – sign that this problem generally speaking isn’t so stringent to the community. So, even if Embarcadero will put such a poll in Delphi’s welcome page I don’t think that the results will be different. Many of them will just not vote (due of the nature of the question) and, perhaps, some of them will be annoyed.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s