A brief on using amazon web services.

Recently we decided to use aws for doing some benchmarks. Why - it is cheaper than getting a system, setting them up and configuring them. But using aws for a long time may prove costly.

So, i went to http://aws.amazon.com and signed up with the service. You need to provide a valid credit card no and your phone no before the services are activated. The UI is quite confusing and at times you are not aware whether you are logged in or not. There might also be a case where you think that you are logged in but you would still be asked your password to access some other service.

The best place to start is the "Your Account" section on the top right hand corner. There are two services which are of interest.

Amazon s3 : Simple storage service
Amazon ec2 : Elastic compute cloud

There are two ways to access these services. All ways are available in the "security credentials" tab on the left hand panel after logging in. One way is to create an access key - which provides a access key id and a secret access key - similar to a username & password. Both need to be passed while communicating with the machines. Another way is to generate a X.509 certificate - it also generates a public and a private key.

S3 is a storage space on amazon. The easiest way to access it is using the s3cmd tool available at http://s3tools.org/s3cmd. You can create your own directory and put and get files from the s3 store. Another way of accessing S3 is by using the firefox extension - elasticfox. Once the keys are in place in elasticfox, it provides an interface similar to fileftp with drag-drop features. It looks cool. S3 could only be used to store and retrieve data. If you are thinking about mounting an s3 drive on an ec2 instance to transfer data - better forget it. It would require you to mount the s3 device using fuse and it might take a lot of time to set it up. And there is another way of storing data for EC2.


EC2 is a virtual machine available for computing. You can start an instance of the virtual machine and run your processing on it. The good thing is that there is no need to install the OS of your choice. All you need to do is simply choose the type of machine and the OS and within seconds, the machine is available. You can simply ssh into the machine and start running your processing on it. The bad thing about ec2 is that it does not store your data. Once your processing is done and you turn off or terminate the machine, there is no way of getting back the machine or the data that was there on the machine. The data is lost in the cloud. Like discussed earlier storing data in S3 and mounting it on EC2 might be an option. But a better option is to create an EBS(Elastic Block Storage) volume and attach it to the EC2 instance.

The way to go about all this is logging in to the AWS management console - which is again on the left hand panel. It gives you a dashboard. First of all try launching an instance. It would ask you for the type of instance you want to launch - so you can launch a 32 bit centos instance or a 64 bit ubuntu instance and all you have to do is choose the related AMI (Amazon Machine Image). But be careful, there are some paid AMIs as well - so something like the RHEL instance might be paid and you will have to pay to launch it.

For creating an EBS, you will have to go to the left hand panel again and create a volume. You have to specify the size and the availability zone. An EBS can only be mounted on an Instance if it is in the same availability zone. So be careful about it. Select the zone, where your instance is running. You can create as large a volume you want because you are charged for the data in the volume and not for the volume size. So, if you create a 100 GB volume and put data of only 5 GB in it, you will be charged only for 5GB. Once the volume is created - attach it to the instance - the option is availabe on the web interface.

Now to access the volume you will need to create a filesystem on it and mount it. So, for example if you have attached the EBS volume at /dev/sdf on the instance currently running simply create a filesystem on it.

ssh -i private_key.pem root@ec2-public-dns-name-for-your-instance.com [login to your ec2 instance]
mke2fs -j /dev/sdf [create ex3 filesystem]
mkdir /mnt/vol
mount /dev/sdf /mnt/vol [mount the volume]

Now you can work on your ec2 machine and store data on /mnt/vol. When you are done, it is better to take a snapshot of your volume using the tools in the amazon web console and then turn off your instance. Next time you need to work, simply mount the EBS volume on a new instance that you have started and all your data is readily available.

Another way of going about it is creating an amazon AMI after you have done setup of your machine - and use the AMI to boot further machines that you would need. You can download amazon ec2 api tools and the amazon ec2 ami tools which can help in creating AMIs and running them.

If you want to setup networking between the multiple instances, you need to do some extra effort. Of course by default all amazon running instances could ssh to each other using their private IPs and public DNSes. But to communicate on other ports, the ports need to be opened up. To do this simply use the "security groups" link on the left hand panel of the AWS management console. Select a group - default if you did not create any other group and open up the ports that you need.

And the most important advice - do not forget to turn down your instance after you are through with it - remember, you are billed by the hour.

permutation & combination

Definition:

Permutation:
An arrangement is called a Permutation. It is the rearrangement of objects or symbols into distinguishable sequences. When we set things in order, we say we have made an arrangement. When we change the order, we say we have changed the arrangement. So each of the arrangement that can be made by taking some or all of a number of things is known as Permutation.

Combination:
A Combination is a selection of some or all of a number of different objects. It is an un-ordered collection of unique sizes.In a permutation the order of occurence of the objects or the arrangement is important but in combination the order of occurence of the objects is not important.


Formula:

Permutation = nPr = n! / (n-r)!
Combination = nCr = nPr / r!
where,
n, r are non negative integers and r<=n.
r is the size of each permutation.
n is the size of the set from which elements are permuted.
! is the factorial operator.


Example:Find the number of permutations and combinations: n=6; r=4.


Step 1: Find the factorial of 6.
6! = 6×5×4×3×2×1 = 720

Step 2: Find the factorial of 6-4.
(6-4)! = 2! = 2

Step 3: Divide 720 by 2.
Permutation = 720/2 = 360

Step 4: Find the factorial of 4.
4! = 4×3×2×1 = 24

Step 5:Divide 360 by 24.
Combination = 360/24 = 15

GPG Error: ... : NO_PUBKEY D739676F7613768D

You run an apt-get update and it gives some errors which are difficult to comprehend. The output looks some like this

Fetched 924B in 2s (352B/s)
W: GPG error: http://ppa.launchpad.net karmic Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY D739676F7613768D
W: GPG error: http://ppa.launchpad.net karmic Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 2836CB0A8AC93F7A
W: GPG error: http://ppa.launchpad.net karmic Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 2836CB0A8AC93F7A


What to do... How to remove these errors. Well, some public keys are unavailable due to which these errors are happening. Ok, ok, but how to go about it? How do i fix it?

The easiest way is to get this script. The script does not enable the PPAs that are disabled. But for all enabled PPAs it fetches their keys and installs them

jayant@gamegeek:~/bin$ cat launchpad-update 
#! /bin/sh                                  

# Simple script to check for all PPAs refernced in your apt sources and
# to grab any signing keys you are missing from keyserver.ubuntu.com.  
# Additionally copes with users on launchpad with multiple PPAs        
# (e.g., ~asac)                                                        
#                                                                      
# Author: Dominic Evans https://launchpad.net/~oldman                  
# License: LGPL v2

for APT in `find /etc/apt/ -name *.list`; do
    grep -o "^deb http://ppa.launchpad.net/[a-z0-9\-]\+/[a-z0-9\-]\+" $APT | while read ENTRY ; do
        # work out the referenced user and their ppa
        USER=`echo $ENTRY | cut -d/ -f4`
        PPA=`echo $ENTRY | cut -d/ -f5`
        # some legacy PPAs say 'ubuntu' when they really mean 'ppa', fix that up
        if [ "ubuntu" = "$PPA" ]
        then
            PPA=ppa
        fi
        # scrape the ppa page to get the keyid
        KEYID=`wget -q --no-check-certificate https://launchpad.net/~$USER/+archive/$PPA -O- | grep -o "1024R/[A-Z0-9]\+" | cut -d/ -f2`
        sudo apt-key adv --list-keys $KEYID >/dev/null 2>&1
        if [ $? != 0 ]
        then
            echo Grabbing key $KEYID for archive $PPA by ~$USER
            sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com $KEYID
        else
            echo Already have key $KEYID for archive $PPA by ~$USER
        fi
    done
done

echo DONE

Make the script executable

$ chmod a+x launchpad-update

And run the script

jayant@gamegeek:~/bin$ sudo ./launchpad-update 
Grabbing key 7613768D for archive vlc by ~c-korn
Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --secret-keyring /etc/apt/secring.gpg --trustdb-name /etc/apt/trustdb.gpg --keyring /etc/apt/trusted.gpg --recv-keys --keyserver keyserver.ubuntu.com 7613768D                                                                                                                
gpg: requesting key 7613768D from hkp server keyserver.ubuntu.com                                                                                                          
gpg: key 7613768D: public key "Launchpad PPA named vlc for Christoph Korn" imported                                                                                        
gpg: no ultimately trusted keys found                                                                                                                                      
gpg: Total number processed: 1                                                                                                                                             
gpg:               imported: 1  (RSA: 1)                                                                                                                                   
Already have key 4E5E17B5 for archive ppa by ~chromium-daily                                                                                                               
Grabbing key 8AC93F7A for archive backports by ~kubuntu-ppa                                                                                                                
Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --secret-keyring /etc/apt/secring.gpg --trustdb-name /etc/apt/trustdb.gpg --keyring /etc/apt/trusted.gpg --recv-keys --keyserver keyserver.ubuntu.com 8AC93F7A                                                                                                                
gpg: requesting key 8AC93F7A from hkp server keyserver.ubuntu.com                                                                                                          
gpg: key 8AC93F7A: public key "Launchpad Kubuntu Updates" imported                                                                                                         
gpg: no ultimately trusted keys found                                                                                                                                      
gpg: Total number processed: 1                                                                                                                                             
gpg:               imported: 1  (RSA: 1)                                                                                                                                   
Already have key 8AC93F7A for archive staging by ~kubuntu-ppa                                                                                                              
DONE                           

Thats it... Done...
Now if you run sudo apt-get update, you should not get any errors about PUBKEYs...

Bombay & Delhi - very precise article describing both...

He is a large amiable boy who smiles at breasts as if they are acquaintances. He suspects he is good looking, and he probably is, especially when he is quiet. A pretty white girl walks into this London pub and he nudges the elbow of a friend. As his eyes follow her to the far corner, his face assumes a sudden seriousness. He then takes her pictures with his phone camera. He tries to meet the eyes of any woman in the pub so that he can flash a smile. He has heard that white women are broadminded. “I like to hear white women scream under me,” he says. A Bengali sitting beside him says he must become a midwife then. Everybody laughs. He stares at the Bengali who is a much smaller man, and he slaps him a few times. Others now rise and try to drag him away. He growls, not metaphorically but really growls. And he says, “I’m a baaad guy, I’m a baaad guy.” He is, of course, a jat from Delhi whose matrimonial ad had once said, accurately, that he is from a good family. He has travelled the world. He studied briefly in the First World, even. There are thousands like him in Delhi, the natural habitat of a kind.

Delhi is a vast medieval town of indisputable botanical beauty, spectacular red ruins, Sheila Dixit, and other charms. Its women, rumoured to be high maintenance as if there is another kind, take so much care of themselves that one would think the men are worth it (but they make a gesture that suggests puking when asked to confirm). Space is not compressed here. Everything is far from everything else. There are real gardens where you do not see the exit when you stand at the entrance. It has sudden small parks that in Bombay would have been called, ‘Chhatrapati Shivaji Mini Forest’. Homes have corridors, and they are called corridors, not half-bedrooms. Yet, Delhi has a bestial smallness of purpose.

Those men there who drive the long phallic cars, sometimes holding a beer bottle in one hand, there is something uncontrollable about them. Even for a man, it is hard to understand their mutation. What is the swagger about? What is the great pride in driving your father’s BMW, what is the glory in being a sperm? And what is the great achievement in stepping on the accelerator? It is merely automobile engineering—press harder on the pedal and the car will move faster. Why do you think a girl will mate with you for that? It is somehow natural that the contemporary version of Devdas, Anurag Kashyap’s Dev D, would be set in Delhi, where a man can debase himself because life does not challenge him, he has no purpose, whose happiness is a type of sorrow. This motiveless Delhi male, you can argue, can be found in Bombay too, where not all BMWs are hard earned. But that’s not very different from saying Bombay, too, has bungalows.

Like a rich man’s son, Delhi is a beneficiary of undeserved privileges. That is at the heart of Bombay’s contempt for Delhi. Bombay is a natural city, like all great port cities of the world. It was not created. It had to arrive at a particular moment in time, it was an inevitability caused by geography, shipping and shallow waters. Bombay eventually earned its right to be a financial force through the power of enterprise, which created a system that allowed, to some extent, anyone to stake a claim at wealth through hard work. That culture still exists. It is the very basis of Bombay. That is the whole point of Bombay.

But Delhi as a centre of power is an inheritance, a historical habit. An unbearable consequence of this is the proximity of easy funds for various alleged intellectual pursuits which has enabled it to appropriate the status of intellectual centre. It is a scholarship city, a city of think tanks, of men who deal in discourse, debates and policies. And of fake serious women who wear the sari the other way and become leftists, nature lovers and diurnal feminists.

Delhi, often, confuses seriousness with intelligence and humour with flippancy. People will not be taken seriously here if they are not, well, serious. There is much weight attached to the imagined sophistication of talk, of gas. It is a city of talkers. There is always The Discussion. When you are in Delhi, you have to talk, and then they talk, and they appear to be solving an enigma, they seem headed towards achieving some revelation. But then, you realise, they were peeling an onion, an act that leads to more peels and at the heart of it all, there is nothing. Delhi is an onion. It is a void-delivery device.

Of course, all this is a generalisation, but then generalisation is a form of truth. One of the most repulsive images I bear in mind of Delhi is a scene in JNU, when Venezuelan president Hugo Chavez delivered a special lecture. It was like a rock concert and Chavez, who is a scion of the same imbecilic philosophy that once destroyed the great economies of South America, was the star. As students pumped their hands in the air and cheered him for his anti-capitalist calling, I looked at their faces. I knew those faces. They were from homes that once profited from India’s socialist corruption, and then from Manmohan’s revolution. They were hip. They would, of course, later join MNCs and chuckle at their youthful romanticism. That moment in JNU was despicable because it captured a meaningless aspect of Delhi’s fiery intellectuality, and also laid bare the crucial difference between intellectuality, which is borrowed conviction, and intelligence, which is creativity, innovation and original analysis.

It is for the same reason that the greatest misfortune of Indian journalism is that it is headquartered in Delhi. Needless to say, like in any other city, Delhi has astonishingly talented editors, journalists and writers, but there is a Delhi mental condition which is incurable—a fake intensity, a fraudulent concern for ‘issues’, the grand stand. Readers, on the other hand, have many interests today apart from democracy, policies and the perpetual misery of the poor. But the Indian media, based in Delhi, refused to see it until recently and very grudgingly, when The Times of India proved it. It is not a coincidence that The Times Group, the most profitable media organisation in India, is based in Bombay. It is not a coincidence that the game changer came from here. In Bombay it is hard to convert air from either side of your alimentary canal into cash. You have to do something here. You have to work. It is appropriate that the National School of Drama, with its phoney distaste for money, is in Delhi. And commercial cinema is in Bombay.

It must be said though that in recent times Delhi has become somewhat more endearing. This is partly because of Bombay’s own degradation and its loss of modernity, and partly because of a remarkable cultural irony. Bombay’s films were increasingly becoming pointless because, like Delhi has those silver sperms in BMWs, Bombay’s film industry, too, suffers the curse of the privileged lads whose fathers were something. As actors with no real talent they could still survive, but some who did not look so good could do nothing more than remaking movies about love and parental objection. Then two things happened. The flops of the brainless boys from the film families gave opportunities to talent that had arrived from all over the country, including what is called North India. They were waiting, and when they got a chance they created a new kind of commercial cinema in which Bombay was not necessarily the focus. That resulted in the startling revelation here that Bombay is a culturally impoverished, rootless setting compared to Delhi. What films like Oye Lucky! Lucky Oye! and Dev D have achieved as hilarious, poignant and self deprecatory narrations of the North Indian way of life, has changed Hindi cinema, probably forever. So Delhi is being seen a bit differently in Bombay, with some affection too. Though, the best thing about Delhi will always be its winter. When there is this mist. And you do not see its people.

Source : http://www.openthemagazine.com/article/nation/it-s-a-city-of-undeserved-privilege