When it comes to scaling Web applications, every experienced Web architect eventually realizes that Disk is the New Tape. Getting data from off of the hard drive disk is slow compared to getting it from memory or from over the network. So an obvious way to improve the performance of your system is to reduce the amount of disk I/O your systems have to do which leads to the adoption of in-memory caching. In addition, there is often more cacheable data on disk than there is space in memory since memory to disk ratios are often worse than 1:100 (Rackspace's default server config has 1GB of RAM and 250 GB of hard disk ). Which has led to the growing popularity of distributed, in-memory, object caching systems like memcached and Microsoft's soon to be released Velocity.
memcached can be thought of as a distributed hash table and its programming model is fairly straightforward from the application developer's perspective. Specifically, There is a special hash table class used by your application which is in actuality a distributed hashtable whose contents are actually being stored on a cluster of machines instead of just in the memory of your local machine.
With that background I can now introduce Terracotta, a product that is billed as "Network Attached Memory" for Java applications. Like distributed hash tables such as memcached, Terracotta springs from the observation that accessing data from a cluster of in-memory cache servers is often more optimal than getting it directly from your database or file store.
Where Terracotta differs from memcached and other distributed hash tables is that it is completely transparent to the application developer. Whereas memcached and systems like it require developers to instantiate some sort of "cache" class and then use that as the hash table of objects that should be stored, Terracotta attempts to be transparent to the application developer by hooking directly into the memory allocation operations of the JVM.
The following is an excerpt from the Terracotta documentation on How Terracotta Works
Terracotta uses ASM to manipulate application classes as those classes load into the JVM. Developers can pick Sun Hotspot or IBM's runtime, and any of several supported application servers … The Terracotta configuration file dictates which classes become clustered and which do not. Terracotta then examines classes for fields it needs to cluster, and threading semantics that need to be shared. For example, if to share customer objects throughout an application cluster, the developer need only tell Terracotta to cluster customers and to synchronize customers cluster-wide. Terracotta looks for bytecode instructions like the following (not an exhaustive list): GETFIELD PUTFIELD AASTORE AALOAD MONITORENTRY MONITOREXIT On each of those, Terracotta does the work of Network Attached Memory. Specifically: BYTECODE Injected Behavior GETFIELD Read from the Network for certain objects. Terracotta also has a heap-level cache that contains pure Java objects. So GETFIELD reads from RAM if-present and faults in from NAM if a cache miss occurs. PUTFIELD Write to the Network for certain objects. When writing field data through the assignment operator "=" or through similar mechanisms, Terracotta writes the changed bytes to NAM as well as allowing those to flow to the JVM's heap. AASTORE Same as PUTFIELD but for arrays AALOAD Sames as GETFIELD but for arrays MONITORENTRY Get a lock inside the JVM on the specified object AND get a lock in NAM in case a thread on another JVM is trying to edit this object at the same time MONITOREXIT Flush changes to the JVM's heap cache back to NAM in case another JVM is using the same objects as this JVM
Terracotta uses ASM to manipulate application classes as those classes load into the JVM. Developers can pick Sun Hotspot or IBM's runtime, and any of several supported application servers … The Terracotta configuration file dictates which classes become clustered and which do not. Terracotta then examines classes for fields it needs to cluster, and threading semantics that need to be shared. For example, if to share customer objects throughout an application cluster, the developer need only tell Terracotta to cluster customers and to synchronize customers cluster-wide.
Terracotta looks for bytecode instructions like the following (not an exhaustive list):
On each of those, Terracotta does the work of Network Attached Memory. Specifically:
The instrumented-classes section of the Terracotta config file is where application developers specify which objects types should be stored in the distributed cache and it is even possible to say that all memory allocations in your application should go through the distributed cache.
In general, the approach taken by Terracotta seems more complicated, more intrusive and more error prone than using a distributed hash table like Velocity or memcached. I always worry about systems that attempt to hide or abstract away the fact that network operations are occurring. This often leads to developers writing badly performing or unsafe code because it wasn't obvious that network operations are involved (e.g. a simple lock statement in your Terracotta-powered application may actually be acquiring distributed locks without it being explicit in the code that this is occuring).
Now Playing: Dream - I Luv Your Girl (Remix) (feat. Young Jeezy)