Whether you’re setting up a new EC2 instance or recovering from a lost .pem key, this guide covers the essential steps — plus easy SSH config tricks for smoother access.

1. Setting Up a New EC2 Instance with an SSH Key Pair

  1. Create a Key Pair Go to: EC2 Dashboard → NETWORK & SECURITY → Key Pairs → Create Key Pair Give it a name and download the .pem file.

  2. Launch the Instance Pick an AMI and instance type. While configuring details, select the key pair you just created.

  3. Set Key Permissions (VERY Important) Before using the key, set correct permissions:

chmod 400 /path/to/downloaded-key.pem

2. Recovering from a Lost Key Pair

Lost your private key? Here’s the way back in.

  1. Generate a New Key Pair Create and download a new .pem file from the EC2 dashboard.

  2. Set Permissions on the New Key

chmod 400 /path/to/new-key.pem

  1. Extract the Public Key Use your new .pem file to extract the corresponding public key:

ssh-keygen -y -f /path-to/key-file-pair.pem -t rsa -b 4096

  1. Update authorized_keys on the EC2 Instance

Use EC2 Instance Connect (browser-based login) or any other method to access the server and run:

echo "your-new-public-key" >> ~/.ssh/authorized_keys

You’re back in.

3. Configuring SSH for Easy Access

  1. Edit SSH Config On your local machine:

nano ~/.ssh/config

Add this block:

Host server
    HostName [instance-IP]
    User [username]
    IdentityFile /path/to/key-file-pair.pem

Now you can simply run:

ssh server

And you’re in.

4. Configuring .pem File Permissions (Ubuntu + Windows)

For Ubuntu/Linux:

chmod 400 ~/keys/*.pem

For Windows:

Use icacls to fix key file permissions:

icacls "C:\Users\hp\Documents\keys\*.pem" /reset
icacls "C:\Users\hp\Documents\keys\*.pem" /grant:r %USERNAME%:F
icacls "C:\Users\hp\Documents\keys\*.pem" /remove "Users" "Authenticated Users" "Everyone" /T /C

Proper permissions help avoid "unprotected private key" errors when using SSH on Windows via WSL, Git Bash, or other tools.

Pro Tips

  • Back up your .pem files somewhere safe. These are your literal keys to the kingdom.

  • Use descriptive names per project/org to avoid confusion.

  • Always chmod 400 or set ACLs, or SSH will scream at you.

← Back to Blog Index

Learn more about me.

Got feedback? Ping me on LinkedIn.

My GitHub