Setting up password-less SCP or SSH

I had about 500GB of data on a hard drive and I needed to run some experiments on a small portion from every file. It was not possible to mount the harddrive on the big Linux cluster on which we used to do experiments. So it boiled down to mounting the disk on my local machine, cut every file accordingly and then copy these small files to remote cluster, which ofcourse I wanted to automate. Secure copy (SCP) was the bottleneck here as it works in password authentication mode in its normal form. Answer lies in public key authentication (RSA or DSA). I had to waste 2-3 hours to figure out how to prevent it from password based authentication. Here is a summary of the steps which I have compiled from various sources, to save time of others and even mine if I forget later (which I always do).

Def: client/local machine - the machine from which we are connecting (which issues the 'scp' command),
target/remote machine/server - the other machine to which we are connecting

1> Make sure the target machine's SSH server accepts public key authentication: Open "ssh_config" on target machine, usually located in "/etc/ssh/sshd_config". Make sure these lines are available and uncommented:

PubkeyAuthentication yes
AuthorizedKeysFile %h/.ssh/authorized_keys

2> Make sure SSH client knows where the private key is located: Open "ssh_config" on client machine, usually located in "/etc/ssh/ssh_config". Make sure these lines are available and uncommented:

IdentityFile ~/.ssh/identity
IdentityFile ~/.ssh/id_rsa #if using RSA
IdentityFile ~/.ssh/id_dsa #if using DSA

3> Create a pair of public and private keys: Public key authentication relies on a key pair - public key (which is known to everyone) and private key (known only to the owner). In most flavors, sender encodes the message with receiver's public key and it can be decoded only with the corresponding private key, which is with the intended receiver. In some flavors, sender can use its own private key to encode the message which can be decoded only with the corresponding public key. In this flavor, the receiver needs to have sender's public key. I am guessing that SSH uses this second flavor to authenticate the client. Client generates a key pair and makes it public key known to the remote machine. During authentication, client sends a message encrypted by its private key to the server. If the server is able to decode it using the public key it has against the name of the incoming client (username@client), it opens its doors for the client. So the security of the server lies in being careful while storing the public keys of clients. If a client is successful in storing its public key in the server's repository, it can connect anytime using its private key without a need for password. This is the reason that privilege of storing the public keys of wishful clients lies with the superuser of the remote machine. Anyways, to create the key pair at client end, login with the 'username' who will use the SCP and issue these commands:

local_user@client$ ssh-keygen -t dsa

At one stage, it will ask for passphrase. Hit 'Enter' key without typing any passphrase as we dont want password for public key authentication mode also.

It will store private key in "~/.ssh/id_dsa" and public key in "~/.ssh/id_dsa.pub".

4> Now we need to make this public key known to the server. Log into the remote machine as the user who would be the beneficiary of password-less SCP. A sample command sequence for this would go like:

local_user@client$ scp ~/.ssh/id_dsa.pub remote_user@server:~/
remote_user@server$ cat id_dsa.pub >> ~/.ssh/authorized_keys

Be careful not to overwrite the existing entries in "authorized_keys" file. The login process requires that public and private keys on both machine should be in a folder which is only accessible by the corresponding users. Hence,

local_user@client$ chmod -R 700 ~/.ssh
remote_user@server$ chmod -R 700 ~/.ssh

Now, ideally at this point we should be all set to enjoy the password-less SCP/SSH to connect to the server. However, I was not that lucky. It still asked for password from me. After struggling for sometime I found that the login process checks the permissions of remote home directory and requires them to be "755". As I believe in open source, I had kept my permissions "777". I corrected it as :

remote_user@server$ chmod 755 ~

and yes! I was able to bypass the password authentication and the following command didn't ask for password.

local_user@client$ scp dummy.file remote_user@server:

If you still face some problem and are not able to make it work, I am sorry that I can't help you. But if you find its solution later, please do post it in the comments as it can help everyone.