Hmm, so what are we to explore today...
Every machine has some amount of RAM where the OS/programs store frequently accessed data. Have you ever tried storing arrays in memory so that you can access it very frequently. You might have. Even i have done a similar stuff. Storing a binary tree in memory as doubly linked list - so that parsing of the tree becomes fast.
Here is something to ponder over known as memcache. Defined as "A high performance, distributed memory object caching system". You can get it here http://www.danga.com/memcached/
How does this work.. Well, firstly just download the tar.gz source file. Untar and compile it. A simple ./configure , make and make install would do.
So you will have the "memcached" binary ready. Then all you have to do is run the memcached binary in daemon mode and assign it some amount of memory where it can store data. Just do a memcached -h and it will list you all the available options
[jayant@jayant memcached-1.2.1]$ ./memcached -h
-d run as a daemon
-r maximize core file limit
-M return error on memory exhausted (rather than removing items)
-k lock down all paged memory
-v verbose (print errors/warnings while in event loop)
-vv very verbose (also print client commands/reponses)
-h print this help and exit
-i print memcached and libevent license
-b run a managed instanced (mnemonic: buckets)
To start memcached in daemon mode with 128 MB RAM, which listens on localhost port 11211 the following command would need to be run
memcached -d -m 128 -l 127.0.0.1 -p 11211
using the options available above, you can configure memcached as per your needs.
So now the server is up and running and you need to use clients to connect and store data over there. For that there are apis available with different languages which allow you to connect to memcached daemon and store/retrieve variables, arrays and objects from it. APIs for perl, python, ruby, java, C# and C are available on the website.
Since i generally do work on php, i wanted an API for php. Well for that the simplest way to install a memcache API for php is by running the following command as root
pecl install memcache
It will automatically download, compile and install the memcache API for php.
Cool, so now we are ready. We have the server running and the client API ready. All we need to do now is build a program which puts and gets information from memcache. I will stick to php for this. http://in.php.net/manual/en/ref.memcache.php lists the functions available with memcached. I wont be giving you a detailed program on how to use memcached api. But just for an idea sake...
You will have to connect to the memcached daemon using the memcache_connect function and then use the memcache_add, memcache_get, memcache_set and memcache_delete to add, retrieve, update and delete objects from the memcachd daemon.
Points to ponder upon:
1. Memcache is very fast. It uses the libevent to scale to any number of open connections.
2. You can start any number of memcached servers on different machines. Different instances of the server do not replicate data across each other. The client api uses the different servers to create a pool and distribute and store data among them.
3. LiveJournal.com uses memcache on a major basis for serving dynamic pages. It helped them reduce the page load times and also reduce database load by a major extent.
4. Memcache does not allow you to store objects which are language independent. So if you use php API to store an object, you cannot use java API to extract the object. Well, even if you extract it, you wont be able to figure out the object - it would be incomprehensible.
If you can figure out a way to serialize all objects of different languages in a well defined manner, then the same object could be stored and accessed by different languages from memcached. This should make things much simpler.