Saturday, December 30, 2006

mysql cluster

The NDB mysql cluster is an in-memory storage engine which offers high availability and data-persistence features. Mysql cluster is available with mysql-5.1 and with mysql5.0-max binaries. This can also be compiled after obtaining the source and the binaries can be used.

We will have a quick start to mysql ndb cluster here...
The basics first :

There are 3 nodes/services necessary to start when you are planning to run a mysql cluster

1. The NDB management node daemon ndb_mgmd.
2. The SQL node - a simple mysql server (mysqld).
3. Data nodes - running NDBD daemon

The best configuration would be to have at least 4 differet machines to run the mysql cluster on. The management node could be any small machine. All the management node does is check the cluster on periodic basis and other management functionalities like stop a node, start another node and stuff like that.

The Data nodes need to be high configuration machines - having lots of RAM. This is because the mysql cluster is a memory based cluster which means that the data resides in memory. The more memory you have the better the engine would perform.

And finally the SQL node should be a good machine since it will be doing some processing also. Though it should not require high amounts of RAM.

Steps to install and get mysql cluster running :

step 1:
install mysql 5.1 (binary) on 3 machines (2 data nodes and 1 sql node).
my.cnf on these nodes can be as below:

[mysqld]
ndbcluster
ndb-connectstring=ip.of.mgmnode

[mysql_cluster]
ndb-connectstring=ip.of.mgmnode

step 2:
untar the binary of mysql-5.1 and copy the files ndb_mgm & ndb_mgmd to /usr/local/bin
There is no need to keep the binary of mysql. Once the ndb_mgm* files are copied to bin directory, the untared binary of mysql can be deleted.

step 3:
configuring the mgm node.
create a directory "mkdir /var/lib/mysql-cluster".
create a file config.ini and incorporate mysql cluster settings in it

config.ini
[NDB DEFAULT] #affects settings of all data nodes
#No of replicas. Must divide evenly into the no of data nodes. So for 2 data nodes the no of replicas can be 1 or 2.
NoOfReplicas=2
# Memory allocated for data storage. Used for storing actual records and indexes. The memory space allocated consists of 32K pages. Once a page is allocated, it cannot be returned to the free pool, except by deleting the table. Default value is 80 MB. Minimum is 1 MB and max is dependent on available system memory.
DataMemory=1024M
# Memory allocated for Index storage (Hash indexes in mysql cluster). Hash indexes are used for primary key, unique indexes. Default value is 18 MB. Minimum is 1 MB and max is dependent on available system memory.
IndexMemory=128M
# Log levels. Used to log messages for various events. By default all loggings are disabled. For our reference we would set the log level to 5 for Startup messages and 10 for Error messages/warnings. Minimum level is 0 (disabled) and maximum is 15.
LogLevelStartup=5
LogLevelError=10

[TCP DEFAULT] # TCP/IP options
# no settings required for normal setup

[NDB_MGMD] # Options for the management process
hostname=ip.of.mgmnode
# Data directory - Directory where output files from the management server will be placed. It includes cluster log files, process output files and daemon's pid file.
datadir=/var/lib/mysql-cluster

[NDBD] # Options for data node "A"
hostname=ip.of.datanode_A
# datadir specifies the directory on data node "A" where trace, log, pid and error log files are placed
datadir=/usr/local/mysql/data # assuming the path on node "A"

[NDBD] # Options for data node "B"
hostname=ip.of.datanode_B
datadir=/usr/local/mysql/data # assuming the path on node "B"

[MYSQLD] # Options for the SQL node
hostname=ip.of.sqlnode

step 4:
Start the cluster. Hang on. The way to start the cluster is to first start the management node. Use the following command on management node machine.

ndb_mgmd -f /var/lib/mysql-cluster/config.ini

now run "ndb_mgm". It will connect to the mgm daemon on port 1186. use the "show" command to list the current settings.

ndb_mgm> show
Connected to Management Server at: localhost:1186
Cluster Configuration
---------------------
[ndbd(NDB)] 2 node(s)
id=2 (not connected, accepting connect from ip.of.datanode_A)
id=3 (not connected, accepting connect from ip.of.datanode_B)

[ndb_mgmd(MGM)] 1 node(s)
id=1 @ip.of.mgmnode (Version: 5.1.14)

[mysqld(API)] 1 node(s)
id=4 (not connected, accepting connect from ip.of.sqlnode)


Now start the data nodes. On the data nodes run the following command.

./ndbd --initial

The --initial command deletes all log files and clears the data on node machines. Use this parameter only for starting the data node for the first time.

And start the sql node like when you start a mysql server

./bin/mysqld_safe --user=mysql &

And run the "show" command on the mgm node again.

ndb_mgm> show
Cluster Configuration
---------------------
[ndbd(NDB)] 2 node(s)
id=2 @ip.of.datanode_A (Version: 5.1.14, Nodegroup: 0, Master)
id=3 @ip.of.datanode_B (Version: 5.1.14, Nodegroup: 0)

[ndb_mgmd(MGM)] 1 node(s)
id=1 @ip.of.mgmnode (Version: 5.1.14)

[mysqld(API)] 1 node(s)
id=4 @ip.of.sqlnode (Version: 5.1.14)



Ok, now your mysql cluster is up and running.

If you need to restart the mysql cluster safely run the following commands

ndb_mgm -e shutdown # shuts down the mysql management and data nodes
ndb_mgmd -f /var/lib/mysql-cluster/config.ini #start the management node
ndbd # run this on the data nodes to start mysql cluster on data nodes
./bin/mysqld_safe --user=mysql & # And also shutdown and restart the sql node - like you shutdown and restart a normal mysql server.

To create a table using NDB cluster specify "ENGINE=NDBCLUSTER" in the create table syntax. Eg:

Create table testndb (id int, val varchar(255) ENGINE=NDBCLUSTER;

One error that i encountered while working with the NDB cluster is "ERROR 1114 (HY000): The table '#sql-15c1_2' is full". After a bit of digging i found the following in the log files on the mgm node.

2006-12-30 12:50:59 [MgmSrvr] INFO -- Node 3: Data usage increased to 80%(13205 32K pages of total 16384)
2006-12-30 12:50:59 [MgmSrvr] INFO -- Node 2: Data usage increased to 80%(13220 32K pages of total 16384)
2006-12-30 12:51:29 [MgmSrvr] INFO -- Node 3: Data usage increased to 90%(14757 32K pages of total 16384)
2006-12-30 12:51:29 [MgmSrvr] INFO -- Node 2: Data usage increased to 90%(14769 32K pages of total 16384)
2006-12-30 12:51:45 [MgmSrvr] INFO -- Node 2: Local checkpoint 12 started. Keep GCI = 168 oldest restorable GCI = 192
2006-12-30 12:52:13 [MgmSrvr] INFO -- Node 3: Data usage decreased to 86%(14112 32K pages of total 16384)
2006-12-30 12:52:13 [MgmSrvr] INFO -- Node 2: Data usage decreased to 86%(14109 32K pages of total 16384)
2006-12-30 12:52:17 [MgmSrvr] INFO -- Node 3: Data usage decreased to 74%(12212 32K pages of total 16384)
2006-12-30 12:52:17 [MgmSrvr] INFO -- Node 2: Data usage decreased to 74%(12207 32K pages of total 16384)
2006-12-30 12:52:33 [MgmSrvr] INFO -- Node 2: Local checkpoint 13 started. Keep GCI = 193 oldest restorable GCI = 192


I had allocated 512MB as the DataMemory, which was getting exhausted. So i increased the parameter to 1024M (1G) and since then the things are running fine.

The only problem i am facing is that I need huge amounts of RAM for storing the NDB tables. And i am looking for an option to reduce RAM usage and use simple data on disk clusters.

References:
Mysql Cluster Configuration
Mysql Cluster Complete documentation

Sunday, December 17, 2006

cars

Where do i start from?? Ummm, well, I have got a new car - a chevrolet aveo 1.6 LT (limited edition). Pic below...



The question is why did i get an aveo. Why not any other car? First of all, i never wanted to get a small car. I could have got a santro or a zen or an alto or a swift (i would have preferred the swift). But i wanted my car to be loooong. The options that i had were

1. honda city
2. chevrolet aveo
3. ford fiesta (petrol & diesel)
4. hyundai verna (petrol & diesel)
5. hyundai ascent (petrol & diesel)
6. Maruti Esteem
7. Maruti Baleno

If you look at both comfort and mileage and ease of maintenance, the best car is maruti baleno. But i dont like the looks of baleno. Ummm, let me give you some chart for comparison. Which is available from any other site (try carwale.com).

First of all, i was confused between getting a petrol or a diesel car. Though theoretically diesel cars are supposed to be much stronger and last longer(you can refer to my earlier posts for info on this.), but the initial cost and maintenance cost of the car is high. So unless and until you have to drive more than 100 kms in a day, it is not advisable to get a diesel car. The diesel car is approximately 1.2-1.5 lakhs costlier than its petrol counterpart and the maintenance of the diesel comes to be about 3000-5000 Rs extra per year.

If I would have gone for a diesel car, the cost of the fully loaded car would go beyond my budget. I liked the fiesta diesel and the verna diesel. Though the turbo lag is very much pronounced in the verna, but the pickup and power of verna is too good. Since my daily run would not be more than 50 Kms, i decided to go for a petrol car. I could get a fully loaded petrol car for around 7.6 Lakhs. Where as if i get a fully loaded diesel car, it would cost me around 9 lakhs.

This narrowed down my list of options to

1. hyundai ascent viva crdi
2. hyundai verna xxi
3. ford fiesta 1.6 sxi
4. maruti baleno
5. honda city
6. Chevrolet aveo 1.6 LT

Again knocking out what i did not like. Hyundai ascent viva crdi - wonderful car at reasonable price. I could have got this car for 7.2 lakhs and saved on petrol also. But the point was that the crdi technology in this car is somewhat old.

Well, I dont like the "ass" of baleno, so that also gets knocked out. But baleno has the best average among all cars. Baleno would give you an average of 14 in city and 17 on highway. And the interiors of baleno is very classic.

So now i am left with 4 options. And this is the comparison chart i made.

















CarFord FiestaHonda CityHyundai vernaChevrolet Aveo
Price7,65,000N/A (but 8L approx.)N/A (but 7.2L approx.)7,00,000
Engine TechnologyDurateci-DSIVTVT (Variable timing valve train)VGIS (Variable Geometry intake system)
Engine capacity (cc)1596149715991598
Power (PS@RPM)101@650077@5000103@5500102@6000
Torque (NM@RPM)146@3400125@2700146@3000147@3500
Fuel systemSEFIi-DSIN/ASequencial Fuel Injection with 32 bit ECM
Turning radius (m)4.94.955
Average (kmpl)11121111
Length (mm)4282439041304310
Width (mm)1686169516951710
Height (mm)1468149514901505
Wheel base (mm)2486245025002480
Ground clearance (mm)168160170181
Music system6 CD player with RADIO (no Mp3)Radio + CD/MP3 player with remoteRadio+mp3+cd player (4 speakers)Alpine music system with CD+MP3+Radio (6 speakers)


After going through all the options thoroughly, i zeroed down on the aveo. Well why not fiesta - coz it is costly to maintain. Another point was the music system. The music system in fiesta is damn good, but it cannot play mp3s. Imagine creating Audio cds - 6 of them to play in car and they would carry around 60, maybe 70 songs. Whereas one mp3 cd carries more than a 100 mp3 songs of really good quality. And I can cut one whenever i like. And the biggest point is that the ground clearance of aveo is very good. If you drive a fiesta on a road with potholes and mountains, you can expect some scratching on its base. But with aveo, the chances are grossly reduced. Aveo has got all wheel independent suspension - much better to drive on indian roads. And the best part is that it is cheaper than the other cars.

The city is a piece of trash - no power all style. Verna is a also a good car, but the diesel version is a lot better than the petrol one.

Lets see how things go...

Friday, December 15, 2006

insomnia

insomnia is defined as inability to sleep or take rest.

Here i am referring to the nights we spend in office working. How many days+nights would you have worked in a single go? I have done 4days+nights without sleep. And i believe there are people who have done a lot more.

It is not that I am workaholic. No dear, it is just that the work needs to be completed on time and when deadlines approach the efficiency of the human brain also increases and so does the period during which the brain will be active. The human brain is a wonderful thing. During emergency situations both efficiency and activity of the human brain increases by a good factor.

Oooops!, i think i am going off topic. This post is to discuss what you feel and how you feel when you have not slept for some time.

Lets start with the situation first. I am not a script writer, so i hope this turns out to be fine...

Act 1, Scene 1 - project is delayed and need to be deployed tonight. We have not done proper planning for the deployment.

Act 2, Scene 2 - It is 10'o clock in the night and we are terribly hungry. We need to get some food into our stomach and also start the actual deployment

Act 3, Scene 3 - We have had a heavy diner (sponsored by the company) and it is 11'o clock. Our stomach is full and our eyes are drowsy. And so now the deployment starts.

Act 4, Scene 4 - We start cracking senseless jokes. And to add on it we start laughing on those senseless jokes. We are drowsy. Simply copy pasting commands from the deployment doc to the system. Brain utility is at the minimum. And we are having coffee after coffee to keep us awake.

Act 5, Scene 5 - The project is deployed and all are happy. Now we have started testing the project to see if we have missed anything. Boooom!! a flaw is found, second flaw is found, third..... We are now wide awake. The brain is fully active. Heartbeats are up. The heart is pumping more blood to the brain. We fight. We debug and decode our own scripts. And finally after much struggle around 4 in the night we think we are done.

Act 6, Scene 6 - Now we need to decide whether we should go to sleep and return early to handle problems or go to some place, party and be back in the morning without sleeping. By this time your eyes are closing. You feel a bit heavy. Your food has still not gone down. You need to drink a lot of water. After a lot of struggle and arguments, we decide that a 2 hour sleep would do some good. So we rush home and fall on our beds.

Act 7, Scene 7 - It is 7 in the morning. And the maid is downstairs ringing the bell for some time now. She is paitent. I have woken up, but dont want to get out of the quilt. Wishing that she gives up and goes away. But she is very insistent. Why does she has to clean the room today. Cant she take rest for a day. Well, finally i get up with around 2-2.5 hours of sleep, open the door and let her in. And then jump back in bed hoping to catch 30 minutes of sleep more. But the maid has her own set of questions. Where are the clothes to be washed? Why have we not got the resources (bucket, soap, etc) to aid her cleaning process? I just wish she would do her work quietly and shut the door when she goes. But nooooo. Well, I answer her - "will get the resources today, etc.". Just when the maid is about to go and i am dreaming about my peaceful sleep, the bell is rung again. The car-wash person is here and wants me to open up the car so that he can clean it from inside. I suddenly have a notion that the world is very dirty today morning. And all cleaning needs to be done today itself. Why dont people simply stop cleaning everything? Why is there so much dust in this world?

Act 8, Scene 8 - So I am now fully awake. The car-wash person has gone and my car is gleaming bright red. I can feel that my brain is tired but I know that no matter how much i try, i would not be able to sleep now. Well anyways, I take a bath to freshen up and shoo away the sleep. But the bath does not help in any way - except increase the effort in changing clothes. I feel as if I am a day old. I am hungry but my stomach is full. Anyways, i start my bike and drive to office. I feel as if there is a lag between the situations i see and the reactions which i am supposed to do.

Act 9, Scene 9 - I am in office, hungry and thirsty. Park my bike, punch my card. Plugin my lappy and look at the mails. The project deployed has bug reports already. Good!!! Start the debugging process. Drink a lot of water to keep myself awake and make a few no of trips to the toilet. Take coffee.

Act 10, Scene 10 - It is 12 am. My body has started reacting to sleeplessness. The mind has somehow communicated the lack of sleep to the body. The body now has a tingling sensation. Stomach is still full and hunger is still there. The hunger is becoming more prominent. Eyes are drowsy. Crack pot jokes have gone down. People came and went with their "good work" slogans. I just wish they had the courage enough to stay up a few nights so that they realize our "sacrifice".

Act 11, Scene 11 - It is 1:26PM and it is lunch time. Finally i am going to get some food to eat. But I am not sure where i am going to put it as my stomach is still full. Tiredness has increased. Stuff myself up with whatever junk and stale food that was available.

Act 12, Scene 12 - It is 2:18 PM and i am feeling very very sleepy. My eye lids are not agreeing the command that i am giving and are trying their best to close my eyes. Suddenly i am drowsing on the chair itself. My hand is on the keyboard, eyes half open - half closed. And this is just the moment when my boss makes his appearance - bad luck and wakes me up. All work gone to drain. Get calls from banks offering some stupid credit card. Just want to sleep. My mouth is dry. The uncomfortable chair that i sit upon has now become very comfortable. If i leave before 4:30, it will be considered as a half day. So i will have to be in office and be unproductive till 4:30.

Act 13, Scene 13 - It is 4:15 and i cant take it any more. My eyes are by default half closed. And i can fall sleep walking. Dont remember how much water have i drunk. Just remember that i have been to the loo lots of times. My stomach has suddenly grown huge and full. I dont feel hunger any more - just the need to sleep. I pick up my lappy and am back driving home. I dont care about the half day i would get due to my going home 15 minutes before time.

Act 14, Scene 14 - I am back home. Wanting to sleep. Buy my screwed up brain wont allow me to sleep now. I am on my bed with my lappy and watching movies - reading novels. I realize that there are stretches of time when i have not read anything or saw the movie. I have missed an entire scene and i dont remember what i have done during that time. I had not slept. And i had not pondered on any issue. Have not thought any thought. I was just blank...

Act 15, Scene 15 - It is 8 PM and i dont feel hungry any more. I dont feel thirsty any more. I dont feel the need to watch a movie. I dont feel the need to sleep also. My mind is still tired and my stomach is still full. My mouth is still dry. And now i can meditate - if that is what you call when a person can remain blank and without feeling anything around him. I am at peace.

Act 16, Scene 16 - It is 10 PM and i am again stuffed with junk food. I am on my bed and sleeping very heavily. I dont dream anything. I dont feel anything. You can cut me in my sleep and i wont feel you cutting me.

I wish that i can sleep like this every night.

P.S. =>> Some scenes have been tampered with (a little) to make the story more interesting and readable.

Thursday, December 07, 2006

sociallization

The question here is how do people manage their social life... Well, frankly, i am tired...

To start off, i am engaged, have a 9 to 6 office, have to prepare for my marriage, have to get a house and a car and prepare for my coming married life. And i am not sure how to get all these things running in one go.

Have to talk to my mom, have to talk to my sisters. Make small talks. I dont know why, but i find it difficult to make small talk. According to me, people should just talk when they have something to convey, something to discuss or something to get done.

I get up at 7 in the morning, the maid comes and starts ringing the bell. She has to be let in so she can start doing her household works. Then my fiance would buzz me in the morning to wake me up - so sweet of her. I like this about her. I get ready and reach office by 9:30. And then there is work. Mails to respond to, meetings to attend and issues to solve.

In the evening, i already have a list of todos which need to be done. And of course go for food. Talk to mom, discuss her issues. Talk to the carwala and blast him off. On top of it people complain that i dont interact with them. People get cross and then again i have to work on it.

I dont know how people tend to live in joint families. I dont even tend to remember the names of all my relatives and neither do i remember their relations with me...

When should i get my car serviced and when should i get the water purifier installed. Well, thats comparatively easy. When should i call people who expect to be in touch with me and what small talk should i discuss with them. I have nothing to say to them.

And ofcourse, my friends - some freaking out/ hanging out in the night. Watch a movie - go and explore a new place.

I just dont get it. How do people manage it, keeping in touch with all the people you know.

Maybe sometime down the line, i will be able to do all that without a hitch. The problem i think is that i intend to live up to everybody's expectations.

How to people manage to keep more than 1 girl friends? Handling even 1 girl is tiring...

Whoooooooooo, lets c where i go from here....

Monday, October 30, 2006

404 - a legend




You've felt it. You know the power of 404. You're surfing one night, you've got your new modem, your fast Mac, your ergonomic mouse, your precise mousing surface. Bring it on. You've got 7 windows open at a time, and you flip through them like dealing cards. You could tile them on the screen, like a little website mosaic, but you choose to keep them stacked on each other, so that each flip is a new adventure. Then suddenly, it screeches to a halt: 404. You don't want to believe it; that site was there last week! You reload, hoping it was a fluke. How could your painstakingly-compiled bookmarks betray you? 404 glares back at you, challenging you to contact the referring page's administrator. Questioning your spelling skills. What does 404 want from you?

Relax! 404 is your friend. It just wants to help you get where you want to go. It might intimidate you at first, with its stark white background and unadorned black text. But just think about it for a minute: 404 is baring its soul for you. It gives you its message and asks nothing in return. No login and password, no banner ads, no mailing list to keep you informed of future updates. All 404 has, it offers to you, knowing the likelihood that you will scorn it nonetheless, and leave as quickly as you came in. And 404 will continue to do so for every visitor, regardless of color, religion, or gender. 404 is nothing if not fair.

But why leave 404 so quickly? Why not stay a while and have a drink? 404 is an oasis on the web. It's like a rest stop with clean bathrooms on the interstate. 404 doesn't ask you to 'Click Here' or 'Visit our Sponsor'. It's perfectly satisfied if you just sit there and do nothing. 404 doesn't care how many visitors it's had since 8/1/96, and it's not tracking your click-through rate. So consider just hanging out for a while and relaxing. 404 is easy to get along with.

404 is full of intrigue. What did it used to be? What internet delight has escaped you? Will it return? 404 will never tell. Its mystery drives you to return again and again. Where there is 404, there is always the potential for something new. 404 is the eternal ebb and flow of life. One day you will return and 404 will be gone, replaced by a new page about South Park or the webmaster's cats. And it will be filled with the bittersweet memory of 404. You will be driven to seek out 404 in other places. Your desire for 404 will start to overshadow your career, your loved ones, even your passion for role-playing video games. 404 draws you deeper and deeper into its vortex. You must admit that you are powerless before it. 404 is not evil, it is a natural force that defies control. 404 is wild and free.

404 believes in your abilities. It doesn't try to lose you in a crazy series of redirects, it challenges your problem-solving skills. It asks, "Now that you're here, what are you going to do?" 404 willingly hands you the reins. Have you ever just closed a window on 404? No! You've considered your options, exercised your reasoning skills, and firmly chosen a course of action to deal with your situation. When you successfully navigate 404, you feel the blood coursing in your veins, the wind in your hair, and it's good to be alive. 404 is life-affirming. When you find 404, you know that even though the thing you were looking for no longer exists, there is still 404, stepping in to fill the void.

Where there was darkness, there is now 404. And all is right with the world.

Thursday, October 19, 2006

Your Dominant Intelligence is Logical-Mathematical Intelligence

You are great at finding patterns and relationships between things.
Always curious about how things work, you love to set up experiments.
You need for the world to make sense - and are good at making sense of it.
You have a head for numbers and math ... and you can solve almost any logic puzzle.

You would make a great scientist, engineer, computer programmer, researcher, accountant, or mathematician.

Wednesday, October 18, 2006

Petrol engine versus diesel engine

An engine is a lump of metal which makes the vehicles go zoom. Major types of engines are petrol and diesel. The engines required to run petrol and diesel are different because petrol and diesel are different types of fuel. Petrol is a highly volatile fuel and gets ignited very easily whereas diesel is comparatively heavy and dirtier fuel. We would be talking about 4 stroke engines only, the ones used in cars.

People who know me must be wondering why am I writing this article, i being a computer engineer. Hmmm, because, i am supposed to buy a car now and have been doing some research on them. The first question I came upon was whether a petrol or a diesel car. And, all i want to do is to share all that i have learned with you people. I am still unable to decide whether i should go for ford fiesta diesel or ford fiesta petrol, the petrol verson being a lot cheaper than the diesel version.

Lets start with what does a 4 stroke engine mean. It means that the engine has 4 strokes - inlet, compression, expansion and exhaust. Lets see stroke by stroke how both engines work

Petrol Engine


Diesel Engine


Inlet stroke
> In petrol engines the mixture of air and petrol is drawn in by the falling piston
> In diesel engines only air is drawn in by the falling piston

Compression stroke
> In petrol engine, the mixture is compressed upto about 1/8th to 1/12th of its original size.
> In diesel engine, only air is compressed upto about 1/14th to 1/25th of its original size.

Expansion stroke
> In petrol engine, the air and fuel mixture is ignited using a spark plug and burns expanding and forcing the piston down.
> In diesel engine, fuel is injected at a high pressure into the hot, compressed air in the cylinder, causing it to burn and force the piston down. No spark is required.

Exhause stroke
> In both petrol and diesel engines, the burned mixture of air and fuel is pushed out of the cylinder by the rising piston.

A diesel engine is also known as a "compression ignition" engine. Since the air is compressed to very high pressure raising its temperature and then diesel is injected in a very fine spray which causes the diesel to ignite and explode. Whereas a petrol engine is known as a "spark ignition" engine. Since a spark plug is required to ignite the mixture of petrol and air in the combustion chamber.

Diesel engine


Petrol engine


Lets also note down more differences of the same


  • A diesel engine is more easily turbocharged than a petrol engine. A petrol engine cannot be easily turbocharged due to the fact that if the compression ratio and the pressure in the cylinder is to high during the inlet stroke, the mixture starts to burn to soon, while the piston is on its way up. The diesel engine has no fuel in the cylinder, thus letting the turbocharger suck as much air as it can without creating any problems. (A turbo charger is a simple air compressor which compresses air in the combustion chamber for burning). Some diesel engines also have an intercooler which helps in blowing cold and oxygen rich air in the combustion chamber.

  • Electronic engine management not necessary in diesel engines. Some modern diesel engines are gaining electronically controlled injection pumps, but the vast majority of them out there have purely mechanical pumps. In fact no electricity is required to make a diesel engine run, except for a simple fuel cut off solenoid so that you can switch the thing off! If your alternator stops working, then you’re gonna get home in a diesel. This also means that a diesel engine does not have any ignition breakers, ignition coils, distributors and ignition wires to go bad. So a diesel engine should start no matter if it is dry or rainy or wet.

  • Petrol destroys lubrication and burns the engine whereas diesel doesnt. So a diesel engine would last longer than a petrol engine.

  • Petrol engines are lighter than diesel engines.

  • Diesel engines have higher torque than petrol engines. What does this mean? Well, this means that a diesel engine would pull heavy loads easily than a petrol engine. Though the pickup of a petrol engine would be much more than that of a diesel engine, the diesel engine would be steady and carry heavier loads to longer distances.

  • Diesel engines have better fuel efficiency as compared to petrol due to the fact that they have higher compression ratio.

  • Diesel engines dont need an ignition system, which reduces their complexity. But they are more noisy and may require frequent maintenance as compared to petrol engines. Also they are more durable.

  • Diesel engines may also need glow plugs in extreme cold conditions which heat up the cylinder so that a cold engine can start easily.

  • And now the most important part, fuel economy. Diesel wins in both ways. Diesel engines give better mileage than petrol engines and In india diesel is much cheaper than petrol. So running on diesel would make you go farther at a lower cost than running on petrol.



Even after writing all this, i am still undecided where to go. Should i risk taking a diesel engine hoping that the maintenance costs would be bearable. Well, lets c what happens...

Tuesday, October 17, 2006

Reverse Proxy using squid

what is proxy server caching? hmm... lets start with what apache or any other web server does. Whenever you send a request to apache, the request (whether static or dynamic) is processed. The appropriate file is located in the file system and, the content type is identified and data is then streamed from the file to the web and hence to the browser. This is what the apache does. So where does proxy server come in picture? Proxy servers like squid are set up at the gateway level in cyber cafes or large companies. The proxy server caches web content in its internal cache and whenever a request comes to the proxy server, the server matches the modified time of the web content with that at the server, and if the time is same, the content is served from the proxy cache, else the content is fetched from the remote server and served and cached for future purposes.

So, now, what is reverse proxy caching? Reverse proxy is something which is installed in the neighbourhood of a web server. All incoming traffic to the web server is passed through the reverse proxy server. This reduces the load on a busy server by using a web cache between the server and the internet.

Following benefits are derived by deploying reverse proxy servers alongside web servers:

1. increase capacity of existing servers and prevent purchase of new servers.
2. static content is easily cached at the proxy server level, leaving the web server to handle dynamic content.
3. Increase response time of any web request.
4. proxy server acts like an additional layer of defence against hacking.
5. Load balancing: reverse proxy can be used to distribute load on several web servers.
6. Compression: web content can be optimized and compressed to speed up download times.

A reverse proxy server intercepts requests to the Web server and responds to the request out of a store of cached pages. Dynamic web content cannot be cached. Reverse proxy caches static pages / images based on HTTP header tags that are returned from the web page. Important HTTP header tags are:

Last-Modified -> when was the page last modified
Expires -> when would the page expire so that it can be removed from proxy server cache
Cache-control -> should the page be cached
Pragma -> similar to Cache-control, deciding factor whether the page should be cached or not

Here is what i did to install squid:

>> download the squid source gz file.
>> tar -xvzf squid-2.6.STABLE4.tar.gz - creates a directory squid-2.6.STABLE4
>> cd squid-2.6.STABLE4
>> ./configure --disable-internal-dns - makes squid use the /etc/hosts file for dns lookup.
>> make - compile the files
>> make install (as root) - copies the compiled files to /usr/local/squid
>> vi /usr/local/squid/etc/squid.conf
>> make the following changes. The configuration file format has been changed in squid 2.6. I am putting the new configuration settings here. Older directives of httpd_accel_* have been depricated.

http_port 80 vhost
socket address where squid listens to httpd client requests. Default is 3128 (for proxy server). Various options can be put after the port number. Like
transparent : support for transparent proxies
vhost : accelerator using vhost directive
vport : accelerator with IP virtual host support
defaultsite= : main website name for accelerators
protocol= : protocol to reconstruct accelerated requests with. Default is httpd.
no-connection-auth : prevent forwarding of microsoft connection oriented authentication
tproxy : support for linux TPROXY for spoofing outgoing connections using the client IP address


cache_peer [options]

for apache running on localhost on port 81, the configuration for reverse proxy - cache_peer directive would be

cache_peer localhost parent 81 0 originserver
hostname : cache peer to which connection has to be established type : how cache peer is treaded (as parent, sibiling or multicast)
parent -> the child cache will forward requests to its parent cache. If the parent does not hold the requested object, it will forward the request on behalf of the child.
sibling -> a peer may only request objects already held in the cache. a sibling cannot forward cache misses on behalf of the peer.
multicast -> multicast packet is from one machine to one or more.
proxy port : port no where cache listens to the peer requests
icp port : used for querying neighbour caches about objects
options : lots of options available like
proxy-only -> objects fetched from this cache should not be saved locally
weight=n -> specify weight of parent
round-robin -> define a set of parents to be used in a round robin way.
weighted-round-robin -> define a set of parents to be used in a round robin way, frequency of each parent being based on the round trip time.
originserver -> contact this parent as a origin server. used for accelerator setups

Thats it, just start your apache and squid and everything should run fine.
Hope this helps...

Source : http://www.visolve.com/squid/whitepapers/reverseproxy.php with some customizations to upgrade it for the new version.

Thursday, September 28, 2006

python

Yes python. No, I am not refering to the snake, I am refering to the programming language "python". OOPS!!! Sorry for the confusion.

Python is an interpreted, interactive and object oriented programming language. Often compared to perl, java, ruby etc. As per http://en.wikipedia.org/wiki/Python_programming_language, python was created by some dude named "Guido van Rossum" in 1990. It uses automatic memory management and is a High level language. Oooh.. And python is hot... Why?? Cause it has a very clean syntax and is easy to use and packs a lot of power inside it... Thats why, i think so... So lets get some tits and bits about python...

But before i start, just check out http://www.python.org, the official website of python software foundation the creators and maintainers of python.

The thing which will strike you at the start is the way indentation is important in python. Python groups sentenses on the basis of how it is indented. So i would write a for loop as follows


>>> arr = ['a1','b2','c3','d4','e5','f6'] # defining the array
>>> for x in arr: # starting the for loop
>>> print x # print the element and its length
>>> print len(x)
>>> print 'end' # print end the for loop. Indentation says for loop is over


The best thing i like about this is that developers would automatically indent their code making it good to look at - ummm beautiful may be the right word.

python has a nice set of string operators and regular expressions. You can create and iterate over a range very nicely, write functions, create stacks and queues, do file handling and exception handling. You have complete java style - try->catch level exception handling. And ofcourse you can create classes. Serialize and store objects in file very easily and then restore the objects later. Also there are lots of extensions available which would allow you to interact with the net, create threads, do data compression and decompression. control garbage collection.

Wooo, and the main thing is that all this is possible with very easy and clean syntax . I have seen small and large - in fact very large programs being written in python. And maybe sometime down in future python would replace perl for small programs.

You have got to try it out to see the power of python...

Wednesday, September 27, 2006

Your Love Element Is Metal

In love, you inspire and respect your partner.
For you, love is all about fusing together for one incredible life experience.

You attract others with wit and a bit of flash.
Your flirting style is defined by making others want and value you.

Greatness and optimism are the cornerstones of your love life.
You may let go too easily, but you never get weighed down by your past.

You connect best with: Earth

Avoid: Fire

You and another Metal element: will control and smother each other

Monday, September 25, 2006

where have i been??

Oh dear, one whole month gone, and there is just 1 blog - my contribution to the world of bloggers... Too bad "jayant"... Where have you been?? What have you been doing?? The world is moving on and you are still at the same place. Doing nothing, not even writing blogs...

My dear friends, actually there has been certain changes in my life... And i think, i had been a bit busy in these. Waise the start of this month was very eventful. We had a trip to agra. And for the first time in my life, i saw "Taj Mahal". You would say - "well buddy, whats there in a taj mahal. It is one of the wonders of the world, but still, it is just a monument made out of white marble. Isnt it?" And i would say, "yes yes, it is, it is. But you cannot guess the beauty of the monument just by looking at the images. You have to go and look at the monument to enjoy its beauty"....

To cut the long story short. The trip to agra was gud. We took our own cars and drove down to agra and came back in the same day. Went to see the "Taj Mahal" first thing in the morning and then saw the "Agra Fort". Then went to see the Fort at "Fatehpur Sikri". The Taj was gud, no doubt. Made out of white marbel and there are precious stones embedded in the marble. There are carvings and design patterns on the marble created using precious stones of red, orange, green and other colors. A lot of real hard work must have gone in creating the monument. Also take into account the fact that at the time when the taj was built, sophesticated instruments were not available . And the workers took around 22 years to build up the "Taj" - A monument of love created by the mughal emperor "Shah Jahan" for his beloved wife "Mumtaj Mahal" after her death.






Ok ok, so agreed, you were a bit busy in the starting of this month - around 3rd due to your trip to agra. What happened to the rest of the month buddy. Today is 26th. That is 23 days, without a blog. Hmmmmm, Hmmmmm, Aaaaaa, i have nothing to say. Ummm, i have a friend in orkut who keeps on writing lots of scraps and i had to reply her back. So it also took some of my precious time - you know..

Yes, of course!!! i know. And i had to plan my trip home. And ofcourse, another important thing - should i write.... umm maybe... Well, i have been getting lots of calls lately - lots of job offers... Though i keep on telling them that "i am not interested", but these companies and consultants dont leave me alone. And specially Yahoo!!! God, they have been after my life. But, for the time being i will continue with my current organization.

And ofcourse there were certain deadlines to meet in september. And we had some meetings over that and things were finalized. So, i got a bit engaged in those things. Still, saturdays and sundays - you had a bit of those free. You could have written something then.

Umm, yes, but i got busy... Hang on buddy, i will let you know everything after i come back from home. Oh, i will be going home on the 29th of september. And i am not getting 30th off, so i will be forced to take 1st and 2nd as leave increasing the count of leaves taken from 2 to 5. A loss of 3 days.

And i promice, will write a lot of details once i am back from home....

Till then....Just hang on...

Sunday, September 10, 2006

about love

The big question : what is love??

And the important fact for people reading this blog : i am NOT in love... This is just a thought which i wanted to pen down or maybe blog down...

So returning back to the big question. what is love? how would you define love. Is it the relation between a mother and a child, where the mother cares for the child and vice versa. Or is it the feeling between a brother and a sister. It is said that they share a bonding of "love" and care for each other. Or is it the friendship between two friends. Both friends depend on each other for something or other. They just feel good when they are together. Or is it the relation between a boyfriend and a girlfriend, where there is passion sometimes with and sometimes without commitment. Or is it the relation between a husband and a wife - a lifelong relationship where there is commitment, passion, understandings and minunderstandings, small and major fights, caring and dependency upon each other.

Lets explore each of these relations and try to define love out of these. From what i have seen, people generally tell "i love you" to either their boyfriend or girlfriend. Love in this sense symbolyses passion. Dont you think so?

Well lets start from the mother/father who loves her/his child. From the day the baby is born, parents feed and take care of the baby. Fulfill all his desires and try to make a man out of him. Give him education, food, shelter. Take care of him when he is sick. This, i think is love. Though "i love you" is rarely exchanged in this relation. Still, why does the parent take so much care of the child. Because, you feel that the child is your part. You are the creator of the child, and it is your responsibility. This feeling leads to a type of love and bonding. Where the child demands and the parent fulfills. There are fights and misunderstandings, but they are overcome, because you are bound to live together and care for each other....

The relation between a brother and a sister is, i feel somewhat similar to that between two very good childhood friends, except for the fact that the brother and sister have to live together. Again the "bonding" comes into picture. Which makes this relation stronger than that between two very good friends. There is love over here too, born out of the need to share thoughts and exchange ideas. You love and care for your sibling. What about 2 friends? what i feel is that the "love" between two friends is very selfless and fragile. Small fights can break up friendship unless there is good understanding between the friends. Friendships break because there is no "bond" between the friends. Well, friends are people with whom secrets can be shared, the secrets, which, otherwise may be considered "bad" in the family. Friends are people with whom you can form groups to accomplish something in life. You would never say "i love you" either to your sibling or your friend. Well, in some cases an "i love you" to a friend can break the friendship and hearts. So avoid it. Though sharing an "i love you" with your sibling would ofcourse matter and make the relation stronger.

Now lets come to the most important section of "i love you". The one between a boy friend and a girl friend. The boyfriend keeps on saying "i love you" to the girlfriend and vice versa. They generally become possessive of each other. There are cases where fights break out and there are broken hearts. Or one of them starts liking someone else and ditches the other. Whoa, yes there is a "love" between then but then again there is no "bonding" between them to keep the love, and hence them together. But of-course, there are lots of cases where boyfriend and girlfriend have a "love marriage" and live a very happy life.

Moving ahead lets come to the "love" between a husband and a wife. Well, there is commitment and the need to share the same life. To create a new life. There is a bonding between them that they should stay together - no matter what. Well, though in various cases, this bonding goes for a toss and they do separate. But we are dealing with ideal situations over here. Their life starts with passion and lust and moves to caring and understanding and then to parenthood and responsibilities. Do they love each other. Well, cant say. Newly married couples would say "i love you" maybe once every hour. And after ten years, the same couple would talk to each other maybe once in a day and that too for just 5-10 minutes. They still stay together and care for each other, but where has the "love" gone? Dont look like that at me. Even i dont have an answer to that. I am still unmarried and still have to experience all these things....

So from all the above relations what can be derived regarding the definition of love? Is love simply passion? NO... Is it caring and understanding? Well, maybe, but not completely. Then what is love?

I, myself am unsure about the definition. But just for the records i would simply pen down what i think love is. Or maybe what i believe, ideally love should be.

Love is an invisible bonding. A feeling of togetherness. Strengthened by trust, understanding and caring. Love is something which cannot be expressed, but only felt.

If you love someone, you would care for that person. You would be possessive about that person. You would care for him. And you would depend on him and want him to depend on you. But, in addition to being possessive, you will also need to set the person free and trust that the person does not go against your will. If you dont set him free, there would be suffocation in the relationship and both u and ur love would feel sad. And if the person goes against what you want him to do or does something wrong, you will need to understand why? If two people love each other their "love" should become stronger with each fight/misunderstanding they have. Since with each such situation, they would come closer and their understanding about each other will improve. And ideally there should be no place for "ego" in love.

Though there are people who become "deewana/deewani" in love. They have a sense of self sacrifice. To die/give up everything for someone they "love". To bear the pain and sorrow just to see that the one they "love" is happy. This, i believe is also "love".

I m thru... Let me know your ideas...

Wednesday, August 23, 2006

RAM disk in linux

A RAM disk is a portion of RAM which is being used as if it were a disk drive. RAM disks have fixed sizes, and act like regular disk partitions. Their access time is much faster for a RAM disk than for a real, physical disk. However, any data stored on a RAM disk is lost when the system is shut down or powered off. RAM disks can be a great place to store temporary data which needs to be accessed frequently.

For RAM disk to be accessed, the support for the same should be compiled with the linux kernel. As far as i remember Kernel 2.4.* and 2.6.* have RAM disk support compiled in it.

RedHat/Fedora creates 16 ram disks by default. Although they are not usable by default. The size of these ram disks is 4096K (4 MB).

To list available RAM disks use the command - ls -lh /dev/ram*.
To know default size of RAM disk use - dmesg | grep RAMDISK

As it can be seen, using the default configuration only 4 MB RAMDISK can be created. To create larger RAMDISKS you need to configure the kernel parameters so that larger RAMDISKS can be created. How ?? By simply passing the size of the RAMDISK as a parameter during booting of the kernel.

The kernel option for RAMDISK size is: ramdisk_size=<size in kb>.

All you have to do is type this at the boot prompt during booting. Or if you are using some boot loader like lilo or grub, you need to change the boot command in either lilo.conf or grub.conf. For example in grub.conf you would be having something like


# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You do not have a /boot partition. This means that
# all kernel and initrd paths are relative to /, eg.
# root (hd0,4)
# kernel /boot/vmlinuz-version ro root=/dev/hda5
# initrd /boot/initrd-version.img
#boot=/dev/hda
default=0
timeout=5
splashimage=(hd0,4)/boot/grub/splash.xpm.gz
hiddenmenu
title Fedora Core (2.6.11-1.1369_FC4)
root (hd0,4)
kernel /boot/vmlinuz-2.6.11-1.1369_FC4 ro root=LABEL=/1 ramdisk_size=512000 quiet
initrd /boot/initrd-2.6.11-1.1369_FC4.img
title Windoze XP
rootnoverify (hd0,1)
chainloader +1


The above configuration file would create ramdisks of size 512MB. There is an upper limit to the max size of RAMDISK that can be created and it is also controlled by the amount of available RAM. So if you have 512MB of RAM, you would not be able to create RAMDISKS of more than 512MB. But depending on the kernel configuration, if you have a 4GB RAM system, you may not be able to create RAMDISK of more than 1GB maybe. You will have to check this limit either with the kernel documents or by trying out different sizes at boot prompt.

Once the RAMDISK size has been configured, all that needs to be done is to create a file system on the RAMDISK and then mount the RAMDISK as a file system directory in linux.

To create an ext2fs file system, the following command is used :

mke2fs -m 0 /dev/ram0
The -m 0 option keeps mke2fs from reserving any space on the file system for the root user. This makes the complete ramdisk space available to any regular user.

To mount the ramdisk :


mkdir /mnt/RAMDISK
mount /dev/ram0 /mnt/RAMDISK


Run mount to check whether the RAMDISK has been mounted.
Now the RAMDISK can be used as a regular directory. You can read, write, delete and modify files on the RAMDISK as if you are working on a regular hard disk. Linux would handle it as if it were handling a regular directory on the disk. The difference between RAMDISK and normal DISK would be invisible to a regular user.

The only problem is that if the system is either rebooted or crashes. In that case, all data in the RAMDISK is lost. You will need to recreate the RAMDISK and redo all that has been done.

The benefit of using RAMDISk is that reads and writes ot the RAMDISK would be extremely fast, since everything is in RAM and not on the DISK. To prevent loss of data, it is advisable to keep a backup of data in RAMDISK on the local HDD.

The commands used to create the RAMDISK and loading of data in RAMDISK can be automated by issueing the command in the rc.local or some other startup script.

More details of effectively creating and using RAMDISKS can be obtained from the kernel documentation.

http://www.vanemery.com/Linux/Ramdisk/ramdisk-kerneldoc.txt

Monday, August 21, 2006

trip to kasauli + chail

I have been a bit busy and the blogger was really missing me... anyways, had gone for a trip to kasauli & chail (both are near to Shimla).

Well, initially we were planning to go to jaipur, but we found that due to rains the santuaries would be closed. Moreover, there might be lots of heat in jaipur. And we were planning to go on our bikes. Me on my bullet. So, this got dropped. Next we decided to go to some nearby hillstation. After a lot of research, we decided to visit kasauli and chail. Both are near to each other.

Again the plan of going there on bikes was unsuccessful because one of our group members had a leg injury and it was difficult for him to drive for that long on his bike. It was supposed to be a 6-7 hours/300 kms drive.

Kasauli is around 300 kms from delhi and all we have to do is catch the NH1 to chandigarh and from there to panchkula and then to kasauli/shimla/chail. There is a single road to shimla, on taking a left you reach kasauli and on taking a right you reach chail.

Chail is around 60 kms from kasauli. You have to go down to a small river and then there is a very steep climb up a hill via a narrow road surrounded by greenery and trees. Lovely place to drive through.




We left at around 3 in the morning, gathered at a friends house and then moved on from there in his car. We were able to get on the highway by around 5:30 after filling air and petrol in the car. The highway was good and the drive was great. After a very long time i was able to see the sun rise and we also saw 2 rainbows one above the another on the way.




We reached kasauli around 12 in the afternoon. And we were shocked to see the rates of hotels. We were unable to find anything below 700 for a two bed and nothing below 800 for a 4 bed set. Anyways after lots of research we finalized to stay at R Maidens' some nice hotel with a 4 bed big and decent room. Then had something to eat and went to sleep. Roamed about a bit in the evening and relaxed at night.




Well, the next day we left at around 11 am for chail. Had parathas on the way and enjoyed the hills and the cool and clean air. I was able to see lots of people on bikes and really missed my bullet. At chail, the two things that we visited were - worlds highest cricket pitch and a temple. All the time we were in the clouds. The temple was very peaceful. And the drive wonderful. We came back in the evening and then relaxed.

Next day we went to manki point at kasauli. It was the best place we visited. It is situated inside a military layout, so you could not take any electronic items to that place, neither click photos of the place. We had to leave everything including the our cells, cameras and locking keypad of car inside the car. Had to climb for about 500 meters on a 75 degree slope. On top of th e place, there is a helipad and near that there is a temple. It is a very small place. After reaching the top we were inside the clouds. All we could see was clouds above and below us.

The monkeys over there are very aggressive. During our climb downhill, one of the monkeys took all prasad that we had. Anyways, the visit was good. And the place was worth all the effort.

Later that day, we did some shopping and then relaxed. Next day we left kasauli around 12 pm and were in delhi at around 6:30 pm. Back to the city, heat, pollution and our everyday busy lives.

Saturday, August 05, 2006

mysql partitioning

Table partitioning is a concept which allows data to be stored in multiple locations on one or more disks and access to the data is through SQL queries. The way data is stored and retrieved is invisible to the end user.

Mysql allows the user to select the rules based on which data would be spread over multiple partitions in a table. Mysql supports horizontab partitioning whereby the rows of a table are spread across multiple locations. Support for vertical partitioning is not there. So using mysql partitioning, you cannot assign different columns to different physical partitions of a table.

Benefits of partitioning a table :
1. Being able to store more data in one table than can be held on a single disk or filesystem partition. Though current operating systems allow extremely huge files on the disk.
2. Data that loses its usefulness can often be easily be removed from the table by dropping the partition containing only that data. And adding of new data can be facilitated by adding a new partition specially for that data.
3. Some queries can be greatly optimized in virtue of the fact that data satisfying a given WHERE clause can be stored only on one or more partitions, thereby excluding any remaining partitions from the search. Data can also be re-organized to move more frequently accessed data to one partition.
4. Queries involving aggregate functions such as SUM() and COUNT() can easily be parallelized.
5. Greater query throughput can be achieved by spreading data seeks over multiple disks.

I had used partitioning with MYIASM tables and found out that they were very useful. The complete logic of how rows were divided and stored and how they need to be accessed was invisible to me. What happened when i created some partitions for the table is that the same number of .MYI and .MYD files were created. Though the table definition remained in the single .frm file. A separate .PAR file was created for the table which i think was being used to define the logic of how database was being partitioned.

Moreover, an insert on the table locks all the partitions for a small time that is, the time needed by mysql to decide in which partition the record should be put.

The logic using which a table can be partitioned are listed as below:

RANGE partitioning -> Here table partitions are defined based on a range of data to be stored in each partition. For example -

CREATE TABLE abc (
name VARCHAR(100) NOT NULL,
email VARCHAR(40),
dob DATE NOT NULL
)
PARTITION BY RANGE( YEAR(dob) ) (
PARTITION p0 VALUES LESS THAN (1960),
PARTITION p1 VALUES LESS THAN (1970),
PARTITION p2 VALUES LESS THAN (1980),
PARTITION p3 VALUES LESS THAN (1990),
PARTITION p4 VALUES LESS THAN MAXVALUE
);

This would create 5 partitions in the table abc. Partition p0 will contain records whose dob is from 0 to 1959, partition p1 will contain records whose dob is from 1960 to 1969 and so on. In case a row is entered whose dob cannot be accomodated in any of the partitions, an error is thrown and the data is not inserted in the table.

LIST partitioning -> Here table partitions are defined based on a list of data for each partition. Rows which satisfy a criteria list is inserted in that partition.

CREATE TABLE abcs (
name VARCHAR(100),
dob DATE NOT NULL DEFAULT '1979-01-01'
age INT
)
PARTITION BY LIST(age) (
PARTITION p0 VALUES IN (3,5,6,9,17),
PARTITION p1 VALUES IN (1,2,10,11,19,20),
PARTITION p2 VALUES IN (4,12,13,14,18),
PARTITION p3 VALUES IN (7,8,15,16)
);

This would create 4 partitions in the table abcs. Partition p0 would contain records whose age would be 3,5,6,9 or 17. Again if you try to insert a record whose dob is not in any of the partition list created, an error is thrown and the record is not inserted.

LINEAR HASH partitioning -> Here table data is evenly divided between all partitions using some simple functions. for example

CREATE TABLE abc (
name VARCHAR(100),
dob DATE NOT NULL DEFAULT '1970-01-01',
age INT
)
PARTITION BY HASH( YEAR(dob) )
PARTITIONS 4;

Here 4 partitions are created and data is distributed using the following formula
Partition number = MOD(YEAR(dob),number_of_partitions);
So since my date of birth is 1980, my record will be in partition number 0.

LINEAR HASH partitioning -> This is almost similar to hash partitioning except for the fact that the algorithm used to divide data is different. The syntax is also almost same. We use PARTITION BY LINEAR HASH instead of PARTITION BY HASH.

The algorithm used is :

Given an expression expr, the partition in which the record is stored when linear hashing is used is partition number N from among num partitions, where N is derived according to the following algorithm:

  • Find the next power of 2 greater than num. We call this value V; it can be calculated as:
    V = POWER(2, CEILING(LOG(2, num)))

  • Set N = F(column_list) & (V - 1).

  • While N >= num
    {
    Set V = CEIL(V / 2)
    Set N = N & (V - 1)
    }



The advantage in partitioning by linear hash is that the adding, dropping, merging, and splitting of partitions is made much faster, which can be beneficial when dealing with tables containing extremely large amounts of data. The disadvantage is that data is less likely to be evenly distributed between partitions as compared with the distribution obtained using regular hash partitioning.

KEY partitioning ->
Partitioning by key is similar to partitioning by hash, except that where hash partitioning employs a user-defined expression, the hashing function for key partitioning is supplied by the MySQL server. The syntax used is PARTITION BY KEY instead of PARTITION BY HASH.

In most of the cases either a primary key or an unique key is used to create partitions.

Mysql also supports sub partitioning whereby a partition can be divided into further sub partitions of similar type.

I think i am creating a very long blog for mysql partitioning. However there are a list of syntaxes available on the mysql site to alter, analyze, optimize, rebuild, checking and repairing partitions. Please check http://dev.mysql.com/doc/refman/5.1/en/partitioning.html. It will give a more detailed idea of whatever ranting i have done over here...

I will be signing off from here now. Next blog may or may not contain more info about partitions...

Anyways will keep blogggging...

Saturday, July 29, 2006

Maybe....

How do you handle emotions? What do you do when you feel depressed? You may feel as if there is no goal in your life. You feel lost. You dont know what to do. And to top it up, you are not absolutely sure about the reason for all these emotions/feelings. You try to find a reason.

Maybe your friend did something wrong. No, but he/she was right in her own way. What he/she did is in no way wrong for him/her. But then, why are you upset about what he/she did? Maybe there are issues at your college or work place. Your professor never takes you seriously. No matter what you write, he would not give you good marks. Whereas, his favourite student gets all the good marks and all the praise. Or maybe, your boss had a problem at his place and decided to bring it all out on you. Or maybe you are not getting the responsibilities you deserve. Or maybe you are underpaid and on searching you are not getting a job that you would like. Or maybe, your collegues just dont get along with you. Or maybe you had a fight with your collegue over some small issue in the project. Or maybe your girlfriend/boyfriend or wife/husband cares for someone more than he/she cares for you. Maybe you are right. Maybe you are wrong. You are not sure.

What do you do in such a situation? You feel lonely. You feel lost. You may want to end up your life. How? Take loads of sleeping pills. Or maybe hang yourself from the cealing. Or maybe just jump off the tallest building in your city. But then again, there is no reason for doing so. There is no reason to kill yourself. Why should you kill yourself when you have not achieved what you were meant to achieve in life. But then again what are you supposed to achieve? You are not sure of anything. Well or maybe you are afraid of death. Buy why?? After all some day you are going to die. Is there a reason to live? Why are you alive? What is the purpose of your life?

Maybe you would sit in a corner and cry. And you try to hide yourself from the world. Maybe your friend comes to console you. And maybe you cry on his/her shoulder or maybe you just shoo him away. You expect someone to take care of you. But when someone comes and asks you what is wrong, you dont know how to explain. Maybe you can trust the person or maybe you cant. Maybe the person will understand you or maybe not. You are not sure. You are alone. You are lonely. You struggle with yourself trying to decide what to do. What is right and what is wrong.

And then maybe the next day, you wake up and you feel lucky. Or maybe, you get a call from someone you really care about. Or something unexpectedly good happens. Or maybe the grief simply fades with time. Your mind hides the grief in one of its secret places. And you decide never to look in that place. You cope up with yourself. You feel good. You continue to live...

You laugh, you cry. You get angry. You get depressed. You get excited. That, i think is life...

Sunday, July 23, 2006

why?? why??


  • Why are people jealous of each other?

  • Why do people have an ego?

  • Why do people have an attitude?

  • Why can't people consider each other as equal?

  • Why are there caste restrictions?

  • Why is a girl child not considered as equal to a boy child?

  • Why is there more than 50% reservation for SC/ST/OBC people?

  • Why do we give and take dowery?

  • Why are most of the cops corrupt?

  • Why is there crime in society?

  • Why can't we simple admire beauty without tampering with it?

  • Why are we possessive of the ones we love?

  • Why do terrorists kill innocent people?

  • Why do we fight with a rickshaw wala for Rs 2/- while we spend 150/- on a movie without a word of protest?

  • Why are people emotional?

  • Why do some relations break?

  • Why is love painful?

  • Why do girls mostly answer "I used to think of you only as a friend" OR "I never had such thoughts about you" when you propose her?

  • Why are there issues with property?

  • Why cant we simply take what is ours and let others have what is theirs?

  • Why do we have to tip the waiter always?

  • Why cant we do what we love to do?

  • Why do people steal what is not theirs?

  • Why do people kill?

  • Why do people rape?

  • Why are homo sapiens self destructive?

  • Why is it difficult to let go of someone you love?

  • Why do we fall in love?

  • Why is there a censor board in a democratic country?

  • Why is there still illetracy in India?

  • Why do i like India but still hate it?

  • Why do we fight with someone we love?

  • Why is it that most of the times boys have to propose and be rejected?

  • Why cant girls propose?

  • Why are there love triangles?

  • Why are we afraid of the truth?

  • Why can't I simply be what i am no matter whatever the situation is?

  • Why is sholay still considered to be the best bollywood movie?

  • Why are some people fat and others thin?

  • Why are most of the girls cute?

  • Why is it difficult to live alone?

  • Why do we need to marry?

  • Why are people not ok with gay and lesbian relations?

  • Why cant everybody be in peace and happy?

  • Why is it difficult to live?

  • Why are we afraid of death?


No!!, this is not something i am writing under desperation or depression. I am as happy as a lark. Have been thinking about writing this for a long time... It is something i think is right but... whatever.......

Maybe this is how the world is, and, if everything would be right, there would be no challenges worth facing in life. These are the questions which i think should define life (a struggle) as it is. There are still lots of whys still left out.

And finally, why do i like my bullet so much...

Saturday, July 22, 2006

firefox 2.0 is out

Can be downloaded from http://developer.mozilla.org OR from http://www.mozilla.org/projects/bonecho/all-beta.html

Major features in firefox 2.0 from the end user view point are

User experience

  • Inline spell checking for text areas lets you compose with confidence in web forms.

  • Microsummaries provide a way to create bookmarks that display information pulled from the site they refer to, updated automatically. Great for stock tickers, auction monitoring, and so forth.

  • Extension Manager user interface has been enhanced.

  • Search engine manager lets you rearrange and remove search engines shown in the search bar.

  • Tabbed browsing enhancements include adding close buttons to each tab, adjustments to how Firefox decides which tab to bring you to when you close the current tab, and simplified preferences for tabs.

  • Autodetection of search engines allows search engines that offer plugins for the Firefox search bar to offer to install their plugins for you.

  • Search suggestions allow search engines to offer suggested search terms based on what you've typed so far in the search bar.



Security and privacy

  • Anti-phishing feature to warn users when the web site you're looking at appears to be a forgery.

Wednesday, July 19, 2006

Freedom Of Blogging

Censorship, or limiting access to certain specific data is a big drawback for democracy. It limits freedom of speech, thought, expression, the basic fundas of democracy.

We say india is a democratic country. Why? There is censorship in almost all movies, in addition to rating given by the censor board.

Well forget that. But why is my blog not opening up since yesterday. I had to do some blogging but since my blog was not opening up, i thought, it might be down. But that is not the case...

Take a look at the traceroute below (for techies):

C:\Documents and Settings\Admin>tracert jayant7k.blogspot.com

Tracing route to blogspot.blogger.com [66.102.15.101]
over a maximum of 30 hops:

1 314 ms 298 ms 300 ms 97.239.1.15
2 298 ms 320 ms 299 ms 97.239.2.1
3 297 ms 320 ms 499 ms 202.138.101.57
4 517 ms 540 ms 519 ms 220.224.184.178
5 517 ms 540 ms 519 ms 62.216.145.221
6 557 ms 539 ms 559 ms eqixva-google-gige.google.com [206.223.115.21]
7 638 ms 639 ms 640 ms 216.239.47.120
8 539 ms 539 ms 540 ms 72.14.238.235
9 718 ms 740 ms 719 ms 66.249.94.233
10 717 ms 720 ms 719 ms 66.249.95.246
11 717 ms 719 ms 719 ms 216.239.46.44
12 637 ms 619 ms 619 ms 72.14.236.9
13 617 ms 639 ms 620 ms 72.14.233.121
14 639 ms 619 ms 659 ms 216.239.49.78
15 * * * Request timed out.
16 * * * Request timed out.
17 * * * Request timed out.
18 * * * Request timed out.
19 * * * Request timed out.
20 * * * Request timed out.
21 * * * Request timed out.
22 * * * Request timed out.
23 * * * Request timed out.
24 * * * Request timed out.
25 * * * Request timed out.
26 * * * Request timed out.
27 * * * Request timed out.
28 * * * Request timed out.
29 * * * Request timed out.
30 * * * Request timed out.

Trace complete.


After 14 hops, suddenly the request for data times out.

Well take a look at this... http://www.withinandwithout.com/?p=854. The GREAT indian government has decided to do censorship on internet also. Blocking freedom of blogging. This is very bad. The government says that terrorists are using blogs to communicate. Well agreed. But then there is google earth.

If you dont have any idea about what it is, you can do a search on google and download and take a look at it. Using google earth, you can zoom in on any place around the world be it "the white house" or "The parliament". You will be having the map of that place in front of you, no matter where you are.

If you have good internet connectivity on moon, maybe you can plan a terrorist mission from there using google earth. What will the government do then, decide to ban "google earth". Say that from now onwards, there would be no maps made for delhi.

It was just an example, of what technology can provide. Using it depends on who uses it. Just like an atom bomb. It can be used both for construction and destruction. Dynamites and explosives can be used for breaking rocks and making roads or for killing innocent people. With advanced technology, nothing would be impossible.

What i would say is that instead of blocking access to data, the government should focus on strengthening its own infrastructure. Look at the amount of corruption that is there in india. The cost of a traffic cop in DELHI/NCR is only around 100/- for whatever was done or not done. The feeling of doing something good for the nation has to be there, not only in the traffic cops but in the ministers and all IAS/IPS officers.

People in india want to become MPS, IAS, IPS because there is lots of under table money in these lines.

Sometimes, i just try to imagine how can this country run? But it is still running. Though a single earthquake, single famine or a single terrorist attack can bring it to a standstill any day.

Well, for example, we had a heavy downpour in delhi today, and believe me, more than half of delhi was immobile due to it. Why cant the government do something about it?

Well, anyways lets hope this does not go on for long and we can access our precious blogs soon.

To bypass the block, you can use www.pkblogs.com. And read whatever blogs that were blocked. The news is that blogspot, Typepad and geocities have been affected due to the ban. You can post blogs but cannot view your blog.

And almost all ISPs have dirtied their hands in this game. Have a look at http://censorship.wikia.com/wiki/List_of_ISPs_that_seem_to_have_blocked_blogger

Keep blogging........
Let the world know where we stand......

Tuesday, July 18, 2006

Honda Vallyrle Rune

Take a look... Costs something around 25K dollars...
6 Cylinder and styleeeee...






And this is a Triumph Rocket III - 3 cylinder :

Saturday, July 08, 2006

document scoring/calculating relevance in lucene

How is scoring of a document calculated in lucene? Well, first of all what is soring and what does it do. Scoring is a factor which decides the order by which documents are returned after a search is done using lucene. Something like "order by relevance". If we provide an order by/sort by clause in the search query, then ordering is done on this. Else, ordering is done on the score of the document. All documents returned from the search have a score no associated with it. The document with the highest score comes on top and that with the lease score comes at the bottom.

Scoring is implemented in the similarity class of lucene (org.apache.lucene.search.Similarity).

Lets see how the score of a document is calculated.

The score of a query q for document d is defines as follows (t refers to a term):





score(q,d)=
Σ (tf(t in d) * idf(t)^2 * getBoost(t in q) * getBoost(t.field in d) * lengthNorm(t.field in d) ) * coord(q,d) * queryNorm(sumOfSqaredWeights)
t in q


where


sumOfSqaredWeights =
Σ ( idf(t) * getBoost(t in q) )^2
t in q


Now, lets decode it...

tf = term/phrase(t) frequency in a document(d). Terms and phrases repeated in a document indicate the topic of the document, so implementations of this method usually return larger values when freq is large, and smaller values when freq is small.

idf = term document frequency(no of documents which contain the term). Terms that occur in fewer documents are better indicators of topic, so implementations of this method usually return larger values for rare terms, and smaller values for common terms.

getBoost (t in q) = Gets the boost for this clause in the query. Documents matching this clause will (in addition to the normal weightings) have their score multiplied by b. The boost b=1.0 by default.

getBoost (t.field in d) = Gets the boost for the field in the document if the term is in the field. Again boost for the field b is 1.0 by default.

lengthNorm(t.field in d) = normalization value for a field given the total number of terms contained in a field. These values, together with field boosts, are stored in an index and multipled into scores for hits on each field by the search code. Matches in longer fields are less precise, so implementations of this method usually return smaller values when numTokens is large, and larger values when numTokens is small. These values are computed under IndexWriter.addDocument(Document d) and stored. Thus they have limited precision, and documents must be re-indexed if this method is altered.

coord = Score factor based on the fraction of all query terms that a document contains. This value is multiplied into scores. The presence of a large portion of the query terms indicates a better match with the query, so implementations of this method usually return larger values when the ratio between these parameters is large and smaller values when the ratio between them is small.

queryNorm = Computes the normalization value for a query given the sum of the squared weights of each of the query terms. This value is then multipled into the weight of each query term. This does not affect ranking, but rather just attempts to make scores from different queries comparable.

So affectively the score of a document returned, in search done, using a query is calculated keeping in mind the frequency of the term/phrase in the document, frequency of documents containing the term, boosting factor of a clause in the query, boosting factor of a field, total number of terms in a field, fraction of all query terms contained in a document and some normalization factor based in the idf and boost of term in query.

If you are searching a single field using a query, the results should be very relevant, provided the text entered is meaningful and gramatically correct. With the logic here, even if you are searching multiple fields, the order of relevance of results should be good. But relevance can be increased by assigning boost factor to fields either during the query formation process or during the indexing process. Fields could be boosted according to their order of importance resulting in very relevant results.

Boosting during the query formation process provides the flexibility to change the boost factor dynamically but slows down the search and scoring due to dynamic computation of score. Whereas if fields are boosted during the indexing process, then though the search will be much faster, you would loose the flexibility of altering the boost factor dynamically.

Anything i have missed, please let me know and i will add it...

Friday, July 07, 2006

blogrolling

Finally got some way of connecting other blogs to my site. Editing the template every time i want to add some blog to my site is a very tedious job...

blogrolling is cool.

The "i am online" link from yahoo is a bit buggy though. Shows me online even at times when i am NOT. Seems that yahoo is having problems with its database/messaging systems.

Clustermaps is also cool. Lots of people who visit me are from india, europe and North America. Good to know where i am getting fame. Will have to plan my celebrity trip accordingly ;-).

cbox.ws though gives no control on spam. People just write down anything in it. It should have a image verification process...

Anyways, i think i have got an almost perfect site.

Still missing something which gives referral details. Like from which site, people got links to my site. Will need to plan my advertising strategy accordingly ;-).

Searcing for more stuff ....

Wednesday, July 05, 2006

simply babbling...

Long time again and no blogs...And i have my excuses ready...

Well firstly there was nothing good enough to write...bad excuse!!, i could have written about my bike ride from noida to greater noida. Although i have gone through the ride again and again, but this time it was different. Me and my collegue, ankit raced through the 20-25 km express way connecting greater noida with noida/delhi. I think, i could have done an average of 100 kmph on my thunderbird. The bike has done only 6000 kms and people say it will open up after around 15000 kms. After that it may touch 120 kmph.

Had Shipra and IP (friends) for the drive. See the pics below...












Had loads of fun. Though there was nothing to do in Greater Noida. No malls, no food joints. All we could find was a Dominos pizza where we had some pizza and then we drove back. Great drive, something which inspired me to think for a delhi-jaipur drive sometime later.

Next, had a wedding in the family (my sis). Getting married on 5th of july 2006. So came down to baroda on 30th - my b'day and was a bit busy with the wishes. just enjoying myself.

And lastly, do remember that indian weddings are lengthy procedures which include shopping (specially for the wedding), lots of customs and traditions and finally the wedding followed by the vidai. Oh!!! a complete week would go by and still you would miss someone by the end of it. Dont know why people follow such traditions.

In the meantime, i tried downloading some games for linux. The best one yet is Quake-III Arena. Downloaded cube and tuxracer, but could not get them running....

Lets c what comes next...