(UPDATED: 05/31/2010)
This how to article will go over creating a Amazon Elastic Compute Cloud (EC2) Machine Image (AMI) from scratch. In this particular example we are creating a Centos 5.3 64bit AMI from beginning to end. For those that are interested in taking advantage of cloud computing, hopefully you find this blog article helpful. Note this blog also can be generalized for Centos 5, 5.1, 5.2 most versions of Redhat, and Fedora 64bit and 32bit.
First thing is first you are going to need to sign up for an Amazon Web Services account specifically S3 Storage and EC2 Elastic Compute Cloud here. You will need a credit card and some basic info, and will immediately get the following info:
[root@server]$ vi /mnt/ami-centos5.3-64bit-base/etc/fstab If you are using 64BIT use these fstab entries below /dev/sda1 / ext3 defaults 1 1 /dev/sdb /mnt ext3 defaults 0 0 none /proc proc defaults 0 0 none /sys sysfs defaults 0 0 If you are using 32BIT use these fstab entries below /dev/sda1 / ext3 defaults 1 1 none /dev/pts devpts gid=5,mode=620 0 0 none /dev/shm tmpfs defaults 0 0 none /proc proc defaults 0 0 none /sys sysfs defaults 0 0 /dev/sda2 /mnt ext3 defaults 0 0 /dev/sda3 swap swap defaults 0 0 |
You will need to work off of a server with the same distro you would like your image to be in my case that would be Centos 5 also note you will want to work on a server with enough disk space to create and image on.
Time to login become root and go to town:
Put your two .pem files the cert and pk in the directory ~/.ec2
[root@server]$ mount -t proc none /mnt/ami-centos5.3-64bit-base/proc |
Create some directories to work from and put your working files in make sure its on your largest partition in my case its /opt.
[root@server]$ vi /opt/EC2YUM/yum-ami.conf [main] cachedir=/var/cache/yum keepcache=1 debuglevel=2 logfile=/var/log/yum.log pkgpolicy=newest distroverpkg=redhat-release tolerant=1 exactarch=1 obsoletes=1 gpgcheck=1 plugins=1 metadata_expire=1800 [base] name=CentOS-5 - Base mirrorlist=http://mirrorlist.centos.org/?release=5&arch=x86_64&repo=os #baseurl=http://mirror.centos.org/centos/5/os/x86_64/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5 #released updates [updates] name=CentOS-5 - Updates mirrorlist=http://mirrorlist.centos.org/?release=5&arch=x86_64&repo=updates #baseurl=http://mirror.centos.org/centos/5/updates/x86_64/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5 #packages used/produced in the build but not released [addons] name=CentOS-5 - Addons mirrorlist=http://mirrorlist.centos.org/?release=5&arch=x86_64&repo=addons #baseurl=http://mirror.centos.org/centos/5/addons/x86_64/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5 #additional packages that may be useful [extras] name=CentOS-5 - Extras mirrorlist=http://mirrorlist.centos.org/?release=5&arch=x86_64&repo=extras #baseurl=http://mirror.centos.org/centos/5/extras/x86_64/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5 #additional packages that extend functionality of existing packages [centosplus] name=CentOS-5 - Plus mirrorlist=http://mirrorlist.centos.org/?release=5&arch=x86_64&repo=centosplus #baseurl=http://mirror.centos.org/centos/5/centosplus/x86_64/ gpgcheck=1 enabled=0 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5 #contrib - packages by Centos Users [contrib] name=CentOS-5 - Contrib mirrorlist=http://mirrorlist.centos.org/?release=5&arch=x86_64&repo=contrib #baseurl=http://mirror.centos.org/centos/5/contrib/x86_64/ gpgcheck=1 enabled=0 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5 |
Download and extract the Amazon AMI tools into the /opt/EC2TOOLS you should have subdirectories that look like /opt/EC2TOOLS/bin etc…
[main] cachedir=/var/cache/yum keepcache=1 debuglevel=2 logfile=/var/log/yum.log pkgpolicy=newest distroverpkg=redhat-release tolerant=1 exactarch=1 obsoletes=1 gpgcheck=1 plugins=1 metadata_expire=1800 [base] name=CentOS-5 - Base mirrorlist=http://mirrorlist.centos.org/?release=5&arch=i386&repo=os #baseurl=http://mirror.centos.org/centos/5/os/i386/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5 #released updates [updates] name=CentOS-5 - Updates mirrorlist=http://mirrorlist.centos.org/?release=5&arch=i386&repo=updates #baseurl=http://mirror.centos.org/centos/5/updates/i386/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5 #packages used/produced in the build but not released [addons] name=CentOS-5 - Addons mirrorlist=http://mirrorlist.centos.org/?release=5&arch=i386&repo=addons #baseurl=http://mirror.centos.org/centos/5/addons/i386/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5 #additional packages that may be useful [extras] name=CentOS-5 - Extras mirrorlist=http://mirrorlist.centos.org/?release=5&arch=i386&repo=extras #baseurl=http://mirror.centos.org/centos/5/extras/i386/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5 #additional packages that extend functionality of existing packages [centosplus] name=CentOS-5 - Plus mirrorlist=http://mirrorlist.centos.org/?release=5&arch=i386&repo=centosplus #baseurl=http://mirror.centos.org/centos/5/centosplus/i386/ gpgcheck=1 enabled=0 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5 #contrib - packages by Centos Users [contrib] name=CentOS-5 - Contrib mirrorlist=http://mirrorlist.centos.org/?release=5&arch=i386&repo=contrib #baseurl=http://mirror.centos.org/centos/5/contrib/i386/ gpgcheck=1 enabled=0 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5 |
Setup your .bashrc file to have proper pathing to EC2 resources
[root@server opt]$ cd /opt/EC2YUM [root@server EC2YUM]$ yum -c yum-ami.conf --installroot=/mnt/ami-centos5.3-64bit-base -y groupinstall Base [root@server EC2YUM]$ cp yum-ami.conf /mnt/ami-centos5.3-64bit-base/etc/yum.conf |
Enable your .bashrc settings
[root@server EC2YUM]$ yum -c /opt/EC2YUM/yum-ami.conf --installroot=/mnt/ami-centos5.3-64bit-base install *openssh* [root@server EC2YUM]$ /usr/sbin/chroot /mnt/ami-centos5.3-64bit-base /sbin/chkconfig sshd --add [root@server EC2YUM]$ /usr/sbin/chroot /mnt/ami-centos5.3-64bit-base /sbin/chkconfig --level 12345 sshd on |
Create an empty file that is about 10GB where the OS files will be worked.
[root@server]$ wget http://3.84.23.23/wp-content/uploads/2009/05/kernel-modules-261633-xenu.tgz [root@server]$ gunzip -c kernel-modules261633-xenu.tgz | tar -xvf - [root@server]$ mv 2.6.16.33-xenU /mnt/ami-centos5.3-64bit-base/lib/modules [root@server]$ /usr/sbin/chroot /mnt/ami-centos5.3-64bit-base /sbin/depmod -ae 2.6.16.33-xenU |
Create an ext3 filesystem inside the image file.
[root@server]$ wget http://3.84.23.23/wp-content/uploads/2009/05/kernel-modules2616-xenu.tgz [root@server]$ gunzip -c kernel-modules2616-xenu.tgz | tar -xvf - [root@server]$ cd lib/modules [root@server]$ mv 2.6.16-xenU /mnt/ami-centos5.3-32bit-base/lib/modules [root@server]$ /usr/sbin/chroot /mnt/ami-centos5.3-32bit-base /sbin/depmod -ae 2.6.16-xenU |
Mount the image file using the loop-back option, allowing you to treat the image file as if it was a standard disk drive.
[root@server]$ /usr/sbin/chroot /mnt/ami-centos5.3-64bit-base /sbin/chkconfig --del kudzu |
Prepare the empty image filesystem with paths for system devices and configuration files.
[root@server]$ vi /mnt/ami-centos5.3-64bit-base/etc/ssh/sshd_config UseDNS no PermitRootLogin without-password |
Create the file fstab to store the filesystem configuration.
[root@server]$ cd /mnt/ami-centos5.3-64bit-base/etc/sysconfig/ [root@server sysconfig]$ vi network NETWORKING=yes HOSTNAME=localhost.localdomain [root@server sysconfig]$ vi /mnt/ami-centos5.3-64bit-base/etc/sysconfig/network-scripts/ifcfg-eth0 ONBOOT=yes DEVICE=eth0 BOOTPROTO=dhcp |
Mount the image’s proc device in advance to avoid problems with using yum.
[root@server]$ vi /mnt/ami-centos5.3-64bit-base/etc/init.d/getssh #!/bin/bash # chkconfig: 2345 95 20 # description: getssh # processname: getssh # export PATH=:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin # Source function library. . /etc/rc.d/init.d/functions # Source networking configuration. [ -r /etc/sysconfig/network ] && . /etc/sysconfig/network # Check that networking is up. [ "${NETWORKING}" = "no" ] && exit 1 start() { if [ ! -d /root/.ssh ] ; then mkdir -p /root/.ssh chmod 700 /root/.ssh fi # Fetch public key using HTTP /usr/bin/curl -f http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key > /tmp/my-key if [ $? -eq 0 ] ; then cat /tmp/my-key >> /root/.ssh/authorized_keys chmod 600 /root/.ssh/authorized_keys rm /tmp/my-key fi # or fetch public key using the file in the ephemeral store: if [ -e /mnt/openssh_id.pub ] ; then cat /mnt/openssh_id.pub >> /root/.ssh/authorized_keys chmod 600 /root/.ssh/authorized_keys fi } stop() { echo "Nothing to do here" } restart() { stop start } # See how we were called. case "$1" in start) start ;; stop) stop ;; restart) restart ;; *) echo $"Usage: $0 {start|stop}" exit 1 esac exit $? ###END OF SCRIPT |
Create a YUM config file, note please adjust version and architecture according to which version you are using.
For Centos 5.3 64bit use the following:
#Fixed typo 07/09 [root@server]$ /bin/chmod +x /mnt/ami-centos5.3-64bit-base/etc/init.d/getssh |
If you are creating a 32bit Centos 5 create the following YUM config file instead:
[root@server]$ /usr/sbin/chroot /mnt/ami-centos5.3-64bit-base /sbin/chkconfig --level 34 getssh on |
Perform the operating system installation via yum, installing base OS.
[root@server EC2YUM]$ cd /opt/EC2YUM/ [root@server EC2YUM]$ yum -c yum-ami.conf --installroot=/mnt/ami-centos5.3-64bit-base install ruby [root@server EC2YUM]$ yum -c yum-ami.conf --installroot=/mnt/ami-centos5.3-64bit-base install curl |
Install openssh dependencies to allow you to connect via SSH
[root@server]$ cd /mnt/ami-centos5.3-64bit-base/ [root@server]$ /usr/sbin/chroot /mnt/ami-centos5.3-64bit-base ./jre-6u12-linux-x64.bin |
Install precompiled modules supplied by Amazon, since the kernel modules installed by yum are not appropriate for the EC2 environment.
[root@server]$ /usr/sbin/chroot /mnt/ami-centos5.3-64bit-base (put your commands here to do installs for the image) |
If you are installing 32bit use the following download instead
[root@server ami-centos5.3-64bit-base]$ cd /opt/EC2YUM/ [root@server EC2YUM]$ yum -c yum-ami.conf --installroot=/mnt/ami-centos5.3-64bit-base clean all [root@server EC2YUM]$ sync [root@server EC2YUM]$ umount /mnt/ami-centos5.3-64bit-base/proc [root@server EC2YUM]$ umount /mnt/ami-centos5.3-64bit-base |
Delete Kudzu from startup since for some reason it messes up your network settings on first startup of the image.
[root@server]$ cd /opt/EC2TOOLS/bin/ [root@server bin]$ ./ec2-bundle-image --image /opt/EC2AMIIMAGE/ami-centos5.3-64bit-base.img --prefix ami-centos5.3-64bit-base --cert ~/.ec2/cert-yourcert.pem --privatekey ~/.ec2/pk-yourprivatekey.pem --user youramazonaccountnumber --destination /opt/EC2AMIFILES --arch x86_64 |
Configure SSH to allow Root login only via key.
[root@server]$ cd /opt/EC2TOOLS/bin/ [root@server bin]$ ./ec2-upload-bundle --manifest /opt/EC2AMIFILES/ami-centos5.3-64bit-base.manifest.xml --bucket phils-amis --access-key accesskeyhere --secret-key secretkeyhere --location (US, UK, us-west-1, ap-southeast-1) [root@server bin]$ history -c |
Configure the images network settings
# .bashrc # User specific aliases and functions alias rm='rm -i' alias cp='cp -i' alias mv='mv -i' # Source global definitions if [ -f /etc/bashrc ]; then . /etc/bashrc fi export PATH=$PATH:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/Users/phil/EC2/bin export EC2_HOME=/Users/phil/EC2 export EC2_PRIVATE_KEY=~/.ec2/pk-yourprivatekey.pem export EC2_CERT=~/.ec2/cert-yourcertificate.pem export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Home/ |
This script grabs the public key credentials for your root login.
[phil@desktop]$ cd /Users/phil/EC2/bin [phil@desktop]$ ./ec2-register phils-amis/ami-centos5.3-64bit-base.manifest.xml IMAGE ami-youramicode |
Give the getssh proper permissions
[phil@desktop]$ ./ec2-add-keypair phils-keypair-raw KEYPAIR phil-keypair 1f:51:ae:28:bf:89:e9:d8:1f:25:5d:37:2d:7d:b8:ca:9f:f5:f1:6f -----BEGIN RSA PRIVATE KEY----- MIIEoQIBAAKCAQBuLFg5ujHrtm1jnutSuoO8Xe56LlT+HM8v/xkaa39EstM3/aFxTHgElQiJLChp HungXQ29VTc8rc1bW0lkdi23OH5eqkMHGhvEwqa0HWASUMll4o3o/IX+0f2UcPoKCOVUR+jx71Sg 5AU52EQfanIn3ZQ8lFW7Edp5a3q4DhjGlUKToHVbicL5E+g45zfB95wIyywWZfeW/UUF3LpGZyq/ ebIUlq1qTbHkLbCC2r7RTn8vpQWp47BGVYGtGSBMpTRP5hnbzzuqj3itkiLHjU39S2sJCJ0TrJx5 i8BygR4s3mHKBj8l+ePQxG1kGbF6R4yg6sECmXn17MRQVXODNHZbAgMBAAECggEAY1tsiUsIwDl5 91CXirkYGuVfLyLflXenxfI50mDFms/mumTqloHO7tr0oriHDR5K7wMcY/YY5YkcXNo7mvUVD1pM ZNUJs7rw9gZRTrf7LylaJ58kOcyajw8TsC4e4LPbFaHwS1d6K8rXh64o6WgW4SrsB6ICmr1kGQI7 3wcfgt5ecIu4TZf0OE9IHjn+2eRlsrjBdeORi7KiUNC/pAG23I6MdDOFEQRcCSigCj+4/mciFUSA SWS4dMbrpb9FNSIcf9dcLxVM7/6KxgJNfZc9XWzUw77Jg8x92Zd0fVhHOux5IZC+UvSKWB4dyfcI tE8C3p9bbU9VGyY5vLCAiIb4qQKBgQDLiO24GXrIkswF32YtBBMuVgLGCwU9h9HlO9mKAc2m8Cm1 jUE5IpzRjTedc9I2qiIMUTwtgnw42auSCzbUeYMURPtDqyQ7p6AjMujp9EPemcSVOK9vXYL0Ptco xW9MC0dtV6iPkCN7gOqiZXPRKaFbWADp16p8UAIvS/a5XXk5jwKBgQCKkpHi2EISh1uRkhxljyWC iDCiK6JBRsMvpLbc0v5dKwP5alo1fmdR5PJaV2qvZSj5CYNpMAy1/EDNTY5OSIJU+0KFmQbyhsbm rdLNLDL4+TcnT7c62/aH01ohYaf/VCbRhtLlBfqGoQc7+sAc8vmKkesnF7CqCEKDyF/dhrxYdQKB gC0iZzzNAapayz1+JcVTwwEid6j9JqNXbBc+Z2YwMi+T0Fv/P/hwkX/ypeOXnIUcw0Ih/YtGBVAC -----END RSA PRIVATE KEY----- |
Configure your init script to be launched in run level 3 and 4
[phil@desktop]$ vi phil-key-pair -----BEGIN RSA PRIVATE KEY----- MIIEoQIBAAKCAQBuLFg5ujHrtm1jnutSuoO8Xe56LlT+HM8v/xkaa39EstM3/aFxTHgElQiJLChp HungXQ29VTc8rc1bW0lkdi23OH5eqkMHGhvEwqa0HWASUMll4o3o/IX+0f2UcPoKCOVUR+jx71Sg 5AU52EQfanIn3ZQ8lFW7Edp5a3q4DhjGlUKToHVbicL5E+g45zfB95wIyywWZfeW/UUF3LpGZyq/ ebIUlq1qTbHkLbCC2r7RTn8vpQWp47BGVYGtGSBMpTRP5hnbzzuqj3itkiLHjU39S2sJCJ0TrJx5 i8BygR4s3mHKBj8l+ePQxG1kGbF6R4yg6sECmXn17MRQVXODNHZbAgMBAAECggEAY1tsiUsIwDl5 91CXirkYGuVfLyLflXenxfI50mDFms/mumTqloHO7tr0oriHDR5K7wMcY/YY5YkcXNo7mvUVD1pM ZNUJs7rw9gZRTrf7LylaJ58kOcyajw8TsC4e4LPbFaHwS1d6K8rXh64o6WgW4SrsB6ICmr1kGQI7 3wcfgt5ecIu4TZf0OE9IHjn+2eRlsrjBdeORi7KiUNC/pAG23I6MdDOFEQRcCSigCj+4/mciFUSA SWS4dMbrpb9FNSIcf9dcLxVM7/6KxgJNfZc9XWzUw77Jg8x92Zd0fVhHOux5IZC+UvSKWB4dyfcI tE8C3p9bbU9VGyY5vLCAiIb4qQKBgQDLiO24GXrIkswF32YtBBMuVgLGCwU9h9HlO9mKAc2m8Cm1 jUE5IpzRjTedc9I2qiIMUTwtgnw42auSCzbUeYMURPtDqyQ7p6AjMujp9EPemcSVOK9vXYL0Ptco xW9MC0dtV6iPkCN7gOqiZXPRKaFbWADp16p8UAIvS/a5XXk5jwKBgQCKkpHi2EISh1uRkhxljyWC iDCiK6JBRsMvpLbc0v5dKwP5alo1fmdR5PJaV2qvZSj5CYNpMAy1/EDNTY5OSIJU+0KFmQbyhsbm rdLNLDL4+TcnT7c62/aH01ohYaf/VCbRhtLlBfqGoQc7+sAc8vmKkesnF7CqCEKDyF/dhrxYdQKB gC0iZzzNAapayz1+JcVTwwEid6j9JqNXbBc+Z2YwMi+T0Fv/P/hwkX/ypeOXnIUcw0Ih/YtGBVAC -----END RSA PRIVATE KEY----- |
Yum install Ruby and CURL to the image
[phil@desktop]$ /Users/phil/EC2/bin/ec2-run-instances ami-youramiid --instance-type m1.large -k phil-key-pair --region (US, UK, us-west-1, ap-southeast-1) |
Install Java to the image, get Java file for your distro jre-6u12-linux-x64.bin and put it in the /mnt/ami-centos5.3-64bit-base
[phil@desktop]$ /Users/phil/EC2/bin/ec2-describe-instances RESERVATION r-0dc52965 027409037432 default INSTANCE i-yourinstance ami-youramiid pending phil-key-pair 0 m1.large 2009-02-13T23:51:11+0000 us-east-1c |
Here is the part where you can install other software you wish or remove it simply by using the chroot command
[phil@desktop]$ /Users/phil/EC2/bin/ec2-describe-instances RESERVATION r-0dc57965 024439027432 default INSTANCE i-yourinstance ami-youramiid ec2-176-122-149-109.compute-1.amazonaws.com domU-12-31-39-00-12-C1.compute-1.internal running phil-key-pair 0 m1.large 2009-02-13T23:51:11+0000 us-east-1c |
Clean up after yourself lowering image size
[phil@desktop]$ /Users/phil/EC2/bin/ec2-authorize default -p 22 |
Bundle your image
[phil@desktop]$ ssh -i phil-key-pair root@ec2-176-122-149-109.compute-1.amazonaws.com The authenticity of host 'ec2-176-122-149-109.compute-1.amazonaws.com (174.129.149.109)' can't be established. RSA key fingerprint is cb:77:33:4f:a0:62:c0:a6:c8:40:99:09:25:4f:5d:ef. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'ec2-174-129-149-109.compute-1.amazonaws.com,174.129.149.109' (RSA) to the list of known hosts. [root@domU-11-33-39-00-12-C1 ~]$ |
The next step is to ensure you have a target bucket for your EC2 instance with the correct ACL’s you may want to use the FireFox plugin S3 Fox to do this but their are great libraries out their for your desired language. I created a bucket called phils-amis
Give the ACL permission to the bucket you create for your AMI:
6aa5a366c34c1cbe25dc49211496e913e0351eb0e8c37aa3477e40942ec6b97c
Next you will upload your files to the bucket you just designated in my case phils-amis
[phil@desktop]$ /Users/phil/EC2/bin/ec2-terminate-instances i-yourinstance INSTANCE i-yourinstance running shutting-down |
Now you are just about done all you need to do is register your AMI. I did this part from my Macbook Pro since I had my Amazon Web Services Tools there. You can get these from Amazon keep in mind these tools are different then the AMI tools you have been using. I put mine in /Users/phil/EC2 also I created a .ec2 directory with my cerificate and private key at /Users/phil/.ec2 Also know that you can use the AWS console from their site to register AMI’s add security groups and launch them as well.
My .bashrc file looks like this on my laptop
[phil@desktop]$ /Users/phil/EC2/bin/ec2-deregister ami-youramiid [root@server bin]$ /Users/phil/EC2/bin/ec2-delete-bundle -b phils-amis -a yourkey -s yoursecretkey -m /opt/EC2/ami-centos5.3-64bit-base.manifest.xml [phil@desktop]$ :>~/.bash_history |
You now can register our new AMI and get your AMI code
[root@server]$ mount -o loop /opt/EC2AMIIMAGE/ami-centos5.3-64bit-base.img /mnt/ami-centos5.3-64bit-base [root@server]$ mount -t proc none /mnt/ami-centos5.3-64bit-base/proc |
You will want to create a keypair to launch your AMI
[phil@desktop]$ ./ec2-add-keypair phils-keypair-raw KEYPAIR phil-keypair 1f:51:ae:28:bf:89:e9:d8:1f:25:5d:37:2d:7d:b8:ca:9f:f5:f1:6f -----BEGIN RSA PRIVATE KEY----- MIIEoQIBAAKCAQBuLFg5ujHrtm1jnutSuoO8Xe56LlT+HM8v/xkaa39EstM3/aFxTHgElQiJLChp HungXQ29VTc8rc1bW0lkdi23OH5eqkMHGhvEwqa0HWASUMll4o3o/IX+0f2UcPoKCOVUR+jx71Sg 5AU52EQfanIn3ZQ8lFW7Edp5a3q4DhjGlUKToHVbicL5E+g45zfB95wIyywWZfeW/UUF3LpGZyq/ ebIUlq1qTbHkLbCC2r7RTn8vpQWp47BGVYGtGSBMpTRP5hnbzzuqj3itkiLHjU39S2sJCJ0TrJx5 i8BygR4s3mHKBj8l+ePQxG1kGbF6R4yg6sECmXn17MRQVXODNHZbAgMBAAECggEAY1tsiUsIwDl5 91CXirkYGuVfLyLflXenxfI50mDFms/mumTqloHO7tr0oriHDR5K7wMcY/YY5YkcXNo7mvUVD1pM ZNUJs7rw9gZRTrf7LylaJ58kOcyajw8TsC4e4LPbFaHwS1d6K8rXh64o6WgW4SrsB6ICmr1kGQI7 3wcfgt5ecIu4TZf0OE9IHjn+2eRlsrjBdeORi7KiUNC/pAG23I6MdDOFEQRcCSigCj+4/mciFUSA SWS4dMbrpb9FNSIcf9dcLxVM7/6KxgJNfZc9XWzUw77Jg8x92Zd0fVhHOux5IZC+UvSKWB4dyfcI tE8C3p9bbU9VGyY5vLCAiIb4qQKBgQDLiO24GXrIkswF32YtBBMuVgLGCwU9h9HlO9mKAc2m8Cm1 jUE5IpzRjTedc9I2qiIMUTwtgnw42auSCzbUeYMURPtDqyQ7p6AjMujp9EPemcSVOK9vXYL0Ptco xW9MC0dtV6iPkCN7gOqiZXPRKaFbWADp16p8UAIvS/a5XXk5jwKBgQCKkpHi2EISh1uRkhxljyWC iDCiK6JBRsMvpLbc0v5dKwP5alo1fmdR5PJaV2qvZSj5CYNpMAy1/EDNTY5OSIJU+0KFmQbyhsbm rdLNLDL4+TcnT7c62/aH01ohYaf/VCbRhtLlBfqGoQc7+sAc8vmKkesnF7CqCEKDyF/dhrxYdQKB gC0iZzzNAapayz1+JcVTwwEid6j9JqNXbBc+Z2YwMi+T0Fv/P/hwkX/ypeOXnIUcw0Ih/YtGBVAC -----END RSA PRIVATE KEY----- |
Now create the key basically cut and paste —–BEGIN RSA PRIVATE KEY—– to —–END RSA PRIVATE KEY—– including the two lines into a your keypair file.
[phil@desktop]$ vi phil-key-pair -----BEGIN RSA PRIVATE KEY----- MIIEoQIBAAKCAQBuLFg5ujHrtm1jnutSuoO8Xe56LlT+HM8v/xkaa39EstM3/aFxTHgElQiJLChp HungXQ29VTc8rc1bW0lkdi23OH5eqkMHGhvEwqa0HWASUMll4o3o/IX+0f2UcPoKCOVUR+jx71Sg 5AU52EQfanIn3ZQ8lFW7Edp5a3q4DhjGlUKToHVbicL5E+g45zfB95wIyywWZfeW/UUF3LpGZyq/ ebIUlq1qTbHkLbCC2r7RTn8vpQWp47BGVYGtGSBMpTRP5hnbzzuqj3itkiLHjU39S2sJCJ0TrJx5 i8BygR4s3mHKBj8l+ePQxG1kGbF6R4yg6sECmXn17MRQVXODNHZbAgMBAAECggEAY1tsiUsIwDl5 91CXirkYGuVfLyLflXenxfI50mDFms/mumTqloHO7tr0oriHDR5K7wMcY/YY5YkcXNo7mvUVD1pM ZNUJs7rw9gZRTrf7LylaJ58kOcyajw8TsC4e4LPbFaHwS1d6K8rXh64o6WgW4SrsB6ICmr1kGQI7 3wcfgt5ecIu4TZf0OE9IHjn+2eRlsrjBdeORi7KiUNC/pAG23I6MdDOFEQRcCSigCj+4/mciFUSA SWS4dMbrpb9FNSIcf9dcLxVM7/6KxgJNfZc9XWzUw77Jg8x92Zd0fVhHOux5IZC+UvSKWB4dyfcI tE8C3p9bbU9VGyY5vLCAiIb4qQKBgQDLiO24GXrIkswF32YtBBMuVgLGCwU9h9HlO9mKAc2m8Cm1 jUE5IpzRjTedc9I2qiIMUTwtgnw42auSCzbUeYMURPtDqyQ7p6AjMujp9EPemcSVOK9vXYL0Ptco xW9MC0dtV6iPkCN7gOqiZXPRKaFbWADp16p8UAIvS/a5XXk5jwKBgQCKkpHi2EISh1uRkhxljyWC iDCiK6JBRsMvpLbc0v5dKwP5alo1fmdR5PJaV2qvZSj5CYNpMAy1/EDNTY5OSIJU+0KFmQbyhsbm rdLNLDL4+TcnT7c62/aH01ohYaf/VCbRhtLlBfqGoQc7+sAc8vmKkesnF7CqCEKDyF/dhrxYdQKB gC0iZzzNAapayz1+JcVTwwEid6j9JqNXbBc+Z2YwMi+T0Fv/P/hwkX/ypeOXnIUcw0Ih/YtGBVAC -----END RSA PRIVATE KEY----- |
Time to start your AMI! Note that if you have a 64bit AMI it needs to be started as a large instance.
[phil@desktop]$ /Users/phil/EC2/bin/ec2-run-instances ami-youramiid --instance-type m1.large -k phil-key-pair --region (US, UK, us-west-1, ap-southeast-1) |
Time to check the status
[phil@desktop]$ /Users/phil/EC2/bin/ec2-describe-instances RESERVATION r-0dc52965 027409037432 default INSTANCE i-yourinstance ami-youramiid pending phil-key-pair 0 m1.large 2009-02-13T23:51:11+0000 us-east-1c |
When it running it will look like this
[phil@desktop]$ /Users/phil/EC2/bin/ec2-describe-instances RESERVATION r-0dc57965 024439027432 default INSTANCE i-yourinstance ami-youramiid ec2-176-122-149-109.compute-1.amazonaws.com domU-12-31-39-00-12-C1.compute-1.internal running phil-key-pair 0 m1.large 2009-02-13T23:51:11+0000 us-east-1c |
Open the AWS Firewall ingress port 22
[phil@desktop]$ /Users/phil/EC2/bin/ec2-authorize default -p 22 |
Now you can ssh using your keypair as root!
[phil@desktop]$ ssh -i phil-key-pair root@ec2-176-122-149-109.compute-1.amazonaws.com The authenticity of host 'ec2-176-122-149-109.compute-1.amazonaws.com (174.129.149.109)' can't be established. RSA key fingerprint is cb:77:33:4f:a0:62:c0:a6:c8:40:99:09:25:4f:5d:ef. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'ec2-174-129-149-109.compute-1.amazonaws.com,174.129.149.109' (RSA) to the list of known hosts. [root@domU-11-33-39-00-12-C1 ~]$ |
To terminate the instance you can do the following use the describe instance command to find the i-id then do the below
[phil@desktop]$ /Users/phil/EC2/bin/ec2-terminate-instances i-yourinstance INSTANCE i-yourinstance running shutting-down |
If you ever need to stop deregister and delete your AMI you can do the following
[phil@desktop]$ /Users/phil/EC2/bin/ec2-deregister ami-youramiid [root@server bin]$ /Users/phil/EC2/bin/ec2-delete-bundle -b phils-amis -a yourkey -s yoursecretkey -m /opt/EC2/ami-centos5.3-64bit-base.manifest.xml [phil@desktop]$ :>~/.bash_history |
If you need to modify your image do the above two steps of deregistering and deleting your bundle in S3 as well as delete the files in /opt/EC2AMIFILES then you can remount the image to make your changes using the same commands you did before
[root@server]$ mount -o loop /opt/EC2AMIIMAGE/ami-centos5.3-64bit-base.img /mnt/ami-centos5.3-64bit-base [root@server]$ mount -t proc none /mnt/ami-centos5.3-64bit-base/proc |
After you remount the image you can modify your software and then re-bundle, re-upload, re-register your ami.
For help setting up your environment or things I may have left out please refer to the documentation at Amazon or write me a comment.
One last note make sure you turn your EC2 Instances off if your not using them they do cost money $ 😛
Happy AMI building!
Also a great resource for free Realtime AWS uptime data is Systems Watch
This is a good instruction thanks
Glad you found it useful.
Awsome, I tried to followed the Amazones doc to create AMI from scratch but no luck. So i created my AMI by using public AMI but this tutor is realy very useful for me.
Thanks
Mr. Chen….. YOU ROCK!
Thanks Vince glad you like 😛
http://3.84.23.23/wp-content/uploads/2009/05/kernel-modules2616-xenu.tgz
This link seems to be broken, any chance you’d re-host it?
I’ve tried recompiling the correct kernel modules (patched) with what’s listed on the ec2 repository, but there are a lot of conflicts. If you’ve already done this using CentOS 5.3, I’d love to give those files a whirl as I’m banging my head up against the wall at the moment.
Hello John,
Sorry about that I just updated those and typo’d it, the below two links should work now:
64bit http://3.84.23.23/wp-content/uploads/2009/05/kernel-modules-261633-xenu.tgz
32bit http://3.84.23.23/wp-content/uploads/2009/05/kernel-modules2616-xenu.tgz
Both have worked for me using CentOS 5.3 32bit and 64bit
Hi Phil,
This is a very good post. By the way, did you compile the kernel modules yourself? In your blogs, you said “Install precompiled modules supplied by Amazon”, but I can’t find anywhere we can download that from Amazon. Can you post the link where you download that from Amazon? if you compile the kernel module, do you mind share the info? ( I got lot of conflict like other people said when I compile the kernel and modules. ) thanks in advance!
Hello Barrow,
I didn’t compile the modules I used Amazons for compatability reasons with Xen, but I did download them from them and provide them from the below urls:
64bit http://3.84.23.23/wp-content/uploads/2009/05/kernel-modules-261633-xenu.tgz
32bit http://3.84.23.23/wp-content/uploads/2009/05/kernel-modules2616-xenu.tgz
Let me know if I am missing your point, this is included in the post.
See below for the instructions after you retrieve them also in my post:
64bit:
[root@server]$ wget http://3.84.23.23/wp-content/uploads/2009/05/kernel-modules-261633-xenu.tgz
[root@server]$ gunzip -c kernel-modules261633-xenu.tgz | tar -xvf –
[root@server]$ mv 2.6.16.33-xenU /lib/modules
[root@server]$ depmod -ae 2.6.16.33-xenU
32bit:
[root@server]$ wget http://3.84.23.23/wp-content/uploads/2009/05/kernel-modules2616-xenu.tgz
[root@server]$ gunzip -c kernel-modules2616-xenu.tgz | tar -xvf –
[root@server]$ cd lib/modules
[root@server]$ mv 2.6.16-xenU /lib/modules
[root@server]$ depmod -ae 2.6.16-xenU
Thanks Phil, I got the files from the link you provided ( http://www.philchen.com.. ) but I was wondering where you got this file from Amazon ( ie the URL where you download this file ). This is just in case your site is not available :).
another question:
should this
[root@server]$ mv 2.6.16.33-xenU /lib/modules
be
[root@server]$ mv 2.6.16.33-xenU /mnt/ami-centos5.3-64bit-base/lib/modules
Hello Barrow,
I updated the blog entry with the correct path for /lib/modules in this example which is /mnt/ami-centos5.2-64bit-base/lib/modules and also added the chroot for depmod.
As for where I got the modules from it looks like Amazon doesn’t have them anymore linked publicly so I guess your safer with me providing them 😉
You could look at the following article: http://developer.amazonwebservices.com/connect/thread.jspa?messageID=96651𗦋 however to make your own.
Regarding your warnings you have been receiving.
Sounds like your using rpms and not yum? I can’t really tell you why your getting those errors with the above info?
Phil
I also found lot of warning “group xxxxxx does not exist use root” how can we fix that?
Hi Phil,
We have a web application running at Client place, for which we are planning to use AmazonEC2.
For creating AMI for that, do I need to create it on the Server machine on which the app is running or can I use any machine at my disposal?
Hi Ashwin,
The method I have outlined requires you to create your AMI on the architecture and OS in question. For this particular article it was CentOS 5.3 on a 64bit and 32bit architecture. The method I use is to build a base AMI using this blog post then launch it, ssh in adding your unique application and take snapshots as needed, which essentially creates separate AMI’s for different application servers off the base one. I have another blog post on snapshots.
Hello Phil,
One of my software during the ./configure,make,make install process needs compatible
kernel-devel package for the kernel that is running ..
if i do the yum install kernel it will give me the present kernel and kernel-devel packages but
to make it as present kernel, the only way i know is edit /boot/grub/grub.conf to the new kernel
so how should i handle incase of ec2 architecutre to deploy my application ..
please reply to me
Srinivas, sounds like your answering your own question, unless I am missing the point. I have two articles one is to create a default AMI using a plain YUM install process for 5.3 CentOS, and one is to snapshot a existing instance into a new modified AMI. You could create a default AMI then make your kernel modifications and snapshot them which might be easier. Again I am a bit unclear on your question.
Thanks for creating this great How-To Phil!! I’m able to get the AMI instance stored into S3 & running under EC2. However, when I try to SSH to my instance, I get the following (where is the public IP of the amazon instance):
ssh -i ross-key-pair root@ec2-.compute-1.amazonaws.com
The authenticity of host ‘ec2-.compute-1.amazonaws.com ()’ can’t be established.
RSA key fingerprint is 74:db:……
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘ec2-.compute-1.amazonaws.com,’ (RSA) to the list of known hosts.
Connection closed by
I’m not able to ssh to my instance as it always gives the “Connection closed by ” message. I have already opened up the ACL (firewall) for port TCP:22 & nmap shows it is open. Have you encountered anything similar to this? Any ideas? Thanks
I apologize for the double posting, the FORM stripped out my "<IP>" tags ….. I’ve changed it to [IP].
Thanks for creating this great How-To Phil!! I’m able to get the AMI instance stored into S3 & running under EC2. However, when I try to SSH to my instance, I get the following (where [IP] is the public IP of the amazon instance):
ssh -i ross-key-pair root@ec2-[IP].compute-1.amazonaws.com
The authenticity of host ‘ec2-[IP].compute-1.amazonaws.com ([IP])’ can’t be established.
RSA key fingerprint is 74:db:……
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘ec2-[IP].compute-1.amazonaws.com,’ (RSA) to the list of known hosts.
Connection closed by [IP]
I’m not able to ssh to my instance as it always gives the “Connection closed by †message. I have already opened up the ACL (firewall) for port TCP:22 & nmap shows it is open. Have you encountered anything similar to this? Any ideas? Thanks
Hi Ross,
Glad the ariticle is helping you. Is it possible for you to post the ssh -v output for verbose ssh debugging purposes.
Also did you make sure to have the below in your sshd_config file in your AMI
UseDNS no
PermitRootLogin without-password
Also double check your:
/mnt/ami-centos5.3-64bit-base/etc/init.d/getssh
script and make sure the syntax is correct like the post, also make sure you:
[root@server]$ /usr/sbin/chroot /mnt/ami-centos5.3-64bit-base /bin/chmod +x /mnt/ami-centos5.3-64bit-base/etc/init.d/getssh
^^
(Looks like my post had a typo and didn’t include the /usr/bin/chroot /mnt/ami-centos5.3-64bit-base before the command) I fixed the post. This could have made it so you couldn’t chkconfig getssh on for run level 3 and 4)
[root@server]$ /usr/sbin/chroot /mnt/ami-centos5.3-64bit-base /sbin/chkconfig –level 34 getssh on
Let me know if that helps after the above commands I would make sure you check to see that getssh is turned on for runlevel 3 and 4 by doing:
[root@server]$ /usr/sbin/chroot /mnt/ami-centos5.3-64bit-base /sbin/chkconfig –list
sorry for the typo its hard remembering to chroot your build, when working on a existing machine sometimes.
Thanks for the response Phil.
1. I’ve doubled checked my “/etc/ssh/sshd_config” in the mounted image & it contains the proper directives you mentioned.
UseDNS no
PermitRootLogin without-password
2. Double checked “getssh” & it was properly set to startup in run-levels 3&4 and was executable. I even went as far as to put debug code in the script, which then prints out the RSA public cert during boot. I then copy-n-pasted the same key to another Linux server I have locally & was able to access the other server using SSH keys. So I even know that the keys are correct.
3. The output of the SSH connection is as follows using the “-vvv” option:
# ssh -vvv -i mytest-keypair.pem root@ec2-[IPaddr].compute-1.amazonaws.com
OpenSSH_4.3p2, OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to ec2-[IPaddr].compute-1.amazonaws.com [IPaddr] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug3: Not a RSA1 key file mytest-keypair.pem.
debug2: key_type_from_name: unknown key type ‘—–BEGIN’
debug3: key_read: missing keytype
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug2: key_type_from_name: unknown key type ‘—–END’
debug3: key_read: missing keytype
debug1: identity file mytest-keypair.pem type -1
debug1: loaded 1 keys
debug1: Remote protocol version 2.0, remote software version OpenSSH_4.3
debug1: match: OpenSSH_4.3 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_4.3
debug2: fd 3 setting O_NONBLOCK
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour128,arcfour256,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se,aes128-ctr,aes192-ctr,aes256-ctr
debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour128,arcfour256,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se,aes128-ctr,aes192-ctr,aes256-ctr
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour128,arcfour256,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se,aes128-ctr,aes192-ctr,aes256-ctr
debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour128,arcfour256,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se,aes128-ctr,aes192-ctr,aes256-ctr
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: mac_init: found hmac-md5
debug1: kex: server->client aes128-cbc hmac-md5 none
debug2: mac_init: found hmac-md5
debug1: kex: client->server aes128-cbc hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug2: dh_gen_key: priv key bits set: 122/256
debug2: bits set: 531/1024
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug3: check_host_in_hostfile: filename /root/.ssh/known_hosts
debug3: check_host_in_hostfile: match line 8
debug3: check_host_in_hostfile: filename /root/.ssh/known_hosts
debug3: check_host_in_hostfile: match line 8
debug1: Host ‘ec2-[IPaddr].compute-1.amazonaws.com’ is known and matches the RSA host key.
debug1: Found key in /root/.ssh/known_hosts:8
debug2: bits set: 500/1024
debug1: ssh_rsa_verify: signature correct
debug2: kex_derive_keys
debug2: set_newkeys: mode 1
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug2: set_newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: mytest-keypair.pem ((nil))
debug1: Authentications that can continue: publickey,gssapi-with-mic,password
debug3: start over, passed a different list publickey,gssapi-with-mic,password
debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup gssapi-with-mic
debug3: remaining preferred: publickey,keyboard-interactive,password
debug3: authmethod_is_enabled gssapi-with-mic
debug1: Next authentication method: gssapi-with-mic
debug3: Trying to reverse map address [IPaddr].
debug1: Unspecified GSS failure. Minor code may provide more information
No credentials cache found
debug1: Unspecified GSS failure. Minor code may provide more information
No credentials cache found
debug1: Unspecified GSS failure. Minor code may provide more information
No credentials cache found
debug2: we did not send a packet, disable method
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Trying private key: mytest-keypair.pem
debug1: read PEM private key done: type RSA
debug3: sign_and_send_pubkey
debug2: we sent a publickey packet, wait for reply
debug1: Connection closed by [IPaddr]
4. Based on the output above, I tried enabling iptables with rules opening up TCP:22 to everyone, but that did not work either (same SSH error log) …. thinking that the firewall was somehow disconnecting me.
5. I was able to get an EC2 instance running CentOS 5.3 32-bit, which was based on a 3rd Party (Community) AMI image. This proved that my security settings were working for my “Default” Security Group & validated the public cert/private key I generated using Amazon tools.
6. My version of CentOS is Release 5.3 (Final) with kernel 2.6.16-xenU (i386).
Not sure where to go from here?
Hi Phil,
Thanks for the great article, I have able to create and launch a Cent OS 5.3 64 bit instace just fine, but when I try to log into the instance via ssh I get a “Server refused our key” error. I have been looking around but can’t find any real help for this. I was wondering if you ever faced the error while creating your instance.
Thanks,
Dhruv
very good tutorial many thanks
but i am bit confused in the step:
this script grabs the public key credentials
http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key
how do i change this line for myself? or it is the same for everyone….. i could not find any further information about openssh-key how do i generate this key for myself?
many thanks
Excellent guide Phil; one thing to note when I ran through this (on Fedora 11) was that at the point which I ran the yum base install I got a lot of yum errors. I got round them by temporarily renaming /etc/yum.repos.d on the host machine in order to stop yum from trying to pick up extra repos.
I have the same problem as Ross. ssh -v indicates the public-key packet
has been sent and then I get the connection closed message…
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Trying private key: very-first-key.pem
debug1: read PEM private key done: type RSA
debug3: sign_and_send_pubkey
debug2: we sent a publickey packet, wait for reply
Connection closed by 174.129.104.149
getssh appears ok..chkconfig is ok, and permissions are ok.
(One thing I notice in the output thing on the AWS console is a missing
/etc/resolv.conf). Could that be a problem?
Hi Phil,
Great stuff you have posted here. I’m trying to execute the steps but got stuck at ‘yum -c yum-ami.conf –installroot=/mnt/ami-centos5.3-64bit-base -y groupinstall Base’ command. It seems that the link to atrpms is broken. It trying to look into http://dl.atprms.net/el$releasever-x86_64/… . The $releasever value is not seem to be right. It should be 5 or 5.3. How can I proceed? Btw, I have copied the exact yum configuration.
Thanks.
Hi again,
I have sorted out the issue. I have changed the entry in /etc/yum.repos.d/atrpms.repo to reflect the exact url. If you have other options do let me know. Thanks.
I am trying to create the CentOS 4.7 64bit image using this instruction. I have updated the yum.conf file to point to 64 bit URLs. However, I get dependency errors like
Error: Missing Dependency: /usr/bin/md5sum is needed by package redhat-lsb-3.0-8.EL.i386 (base)
I wonder why it is trying to install redhat-lsb-3.0-8.EL.i386 and not redhat-lsb-3.0-8.EL.x86_64 version.
Any idea?
Iam running into exactly the same problem that “Ross” had mentioned 🙁
and also, I have a comment on this command :
$ /usr/sbin/chroot /mnt/ami-centos5.3-64bit-base /bin/chmod +x /mnt/ami-centos5.3-64bit-base/etc/init.d/getssh
shouldn’t this be :
$ /usr/sbin/chroot /mnt/ami-centos5.3-64bit-base /bin/chmod +x /etc/init.d/getssh
“/mnt/ami-centos5.3-64bit-base” in front of “/etc/init.d/getssh” will throw a “no such file or directory” right ?
Hi Phil,
I am stuck at the chroot command, it says ‘/bin/bash’ file or directory does not exist
/usr/sbin/chroot /tmp/ami-linux
Please help
Regards,
Manjeet
Hello,
I tried the above method, but i am stuck at the yum OS install. It is showing lots of missing dependency while installing the OS. Do you have a solution for this?
Thanks
Phil, thanks for writing this article and posting it. Your getssh script helped me fix the problem I was having with building my AMI and this is much better documentation than Amazon has on their site.
This is really a very good doc. I found it very useful as i am beginner on amazon.
Thanks a lot dude.
Sandeep
Hello Phil
Great job!. This article was very helpful for me. Thanks.
I found the modules for 32 bits in Amazon:
http://ec2-downloads.s3.amazonaws.com/linux-2.6.16-ec2.tgz
Regards,
Endrigo Rivas.
Sorry, the link is
http://ec2-downloads.s3.amazonaws.com/modules-2.6.16-ec2.tgz
Endrigo Rivas.
Thank you for the instructions this helped me alot but i am stuck at the instructions which says
yum -c yum-ami.conf –installroot=/mnt/ami-centos5.3-64bit-base -y groupinstall Base
and I get the error.
Repository updates is listed more than once in the configuration
I am trying to create the ami on one of the fedora 8 32bit instance. and using the centos 64 bit repo you mentioned, could I know if anyone knows how to resolve this.
I disabled the repos in my yum.repo.d directory and now I get an error which says.
Error: Cannot find a valid baseurl for repo: fedora
Max glad its of help. Make sure you have the correct 32bit Yum config file I have outlined in the article. Sounds like your mixing the 32bit and 64bit. Hope it works out for you.
Hi Phil,
Firstly, thanks for the wonderful step-by-step document. It worked fine until I got to the below step.
When I try changing the permission on the ‘getssh’ script, I get the below error.
[root@localhost]# /usr/sbin/chroot /mnt/ami-centos5.4-32bit-base /bin/chmod +x /mnt/ami-centos5.4-32bit-base/etc/init.d/getssh
/bin/chmod: cannot access `/mnt/ami-centos5.4-32bit-base/etc/init.d/getssh’: No such file or directory
I am able to do the below but not with ‘chroot’:
[root@localhost]# /bin/chmod +x /mnt/ami-centos5.4-32bit-base/etc/init.d/getssh
Is it ok if I change the permission without chroot? If not can you please let me know how to resolve the above error I mentioned? Any help is much appreciated.
Thanks
Aravind.
I think I got it. Like
“blogananda says: on August 27, 2009 at 11:09 pm” in this blog comments, i guess it should be
[root@localhost]# /usr/sbin/chroot /mnt/ami-centos5.4-32bit-base /bin/chmod +x /etc/init.d/getssh
since we are already ‘chroot’ing to the base directory, giving the entire path again will throw the error ‘no such file or directory’
Thanks Blogananda.
Cheers
Aravind.
SO, ALL THIS IS DONE IN MY MACHINE OR DO I HAVE TO GET FIRST HTE INSTANCE RUNNING BEFORE I ALTER IT?
Hi Phil,
Great tutorial on building a EC2 AMI. Any suggestions on how should one go about building a custom AMI for Debian or Ubuntu instead of CentOS?
Thanks,
Manish
Phil,
First off great article. Now to my issue. I followed your steps but instead of using Centos I used Fedora 12. I can get the instance running, but cannot connect to it. Putty always throws a Netork Error: Connection Timed Out message.
Forgive me as I’m a bit of a noob with Linux, but would really like to get this working somehow. Any insight you can provide would be greatly appreciated.
Best regards,
-bob
I tried executing the above with Centos 5.0 successfully.
In addition i am looking for
1) building the image should contain my custom file (example file.zip). How to do ?
example: Adding custom file file.zip, jboss installation
if i create the instance, the os should contain with jboss installation, with my custom file etc…
2) Like to add welcome message say for example “welcome to my xxxxx” to image
I am having the same problem that Max had regarding the yum-ami.conf.
I am trying to make a base image of CentOS 4.8 – 32bit. I am making the AMI while running on CentOS 4.8 in VirtualBox. I have modified all the references from 5.4 to 4.8 and 64bit to 32bit, throughout the process.
However, I am getting this error:
Repository base is listed more than once in the configuration
Repository contrib is listed more than once in the configuration
Repository centosplus is listed more than once in the configuration
Repository addons is listed more than once in the configuration
Repository extras is listed more than once in the configuration
Setting up Group Process
Setting up repositories
not using ftp, http[s], or file for repos, skipping – Null is not a valid release or hasnt been released yet
Cannot find a valid baseurl for repo: update
Error: Cannot find a valid baseurl for repo: update
I have tried to uncomment the baseurl sections and running it, but that did not work. The urls appear to be valid when using it for 4.8-32bit. Any insight on how to fix would be appreciated.
I am wondering if there is a way to take a working install and turn it into an AMI? Currently I think the process would be to create this base install, then rebuild the application installation on the AMI. This seems an extra step as I already have a running version (not instanced) on some local hardware and really just wish to translate it over to an AMI. Is this clear? Any help would be appreciated.
Thanks Phil for this great article. I’ve followed many online how-to’s but only yours actually worked.
This is an excellent guide and helped me to overcome all the issues I was having with creating custom AMIs. Thanks Phil!
Hi Phil,
I see your article is the best knowledge by far in creating custom AMI from a vanilla distro. I am wondering if all the comments from this blog have been folded into the article itself or we have to take additional steps to fix any issues by going thru the users’ comments. Thanks for contributing to the knowledge.
Right now, I have just signed up for the S3 and EC2 accounts and am ready to kick tires with CentOS 5.4 64 bit. I am confused about the certificate.pem file. I know I have the S3 keys I got when I signed up, and I have the key-pair.pem file I created first time on EC2. How do I get the certificate file? Is it the same as X.509 certificate or something else?
Great steps Phil!. Can you point me to creating customer AMI for SUSE Linux.
Thanks.
Thanks for sharing this information … any chance you have a public ami you’ve created using these steps we could launch as a base?
I followed these steps verbatim with the exception of “usr/sbin/chroot /mnt/ami-centos5.3-64bit-base /bin/chmod +x /mnt/ami-centos5.3-64bit-base/etc/init.d/getssh” which generated file not found errors … I made the ami public ami-7f4fa616
Kevin Bridges instead of /usr/sbin/chroot /mnt/ami-centos5.3-64bit-base /bin/chmod +x /mnt/ami-centos5.3-64bit-base/etc/init.d/getssh you can do /bin/chmod +x /mnt/ami-centos5.3-64bit-base/etc/init.d/getssh instead since you are only doing a file permission change.
So I decided to make a base Install 64bit CentOS 5.3 AMI available in US, us-west-1, UK, ap-southeast-1 for those of you that have requested it. You can find them in the AWS EC2 Dashboard under public AMI’s named below:
US East Virginia – ami-b945acd0
US West California – ami-498cdd0c
UK West Ireland – ami-993d17ed
Asia Pacific Singapore – ami-45e29d17
I hope these are helpful.
Also a couple of notes for those that have issues following my blog and ssh’ing into their instances. I believe a couple of things are happening for you:
1) You did not allow the proper AWS security group for port 22
2) the getssh script is not executing at the correct runlevel due to permissions or improper chkconfig settings and therefore is not adding the public key to the authorized_keys in your roots .ssh
3) SSHD didn’t get installed for whatever reason, it doesn’t come with the base install
4) Your network setting is not working possibly due to kudzu horking things
I have followed the instructions again myself and they work for me. So good luck and happy AMI making!
Hi!
Great tutorial. I need to create an AMI for Suse 10 SP3 but I have only seen tutorials using redhat based distros.
I can follow your steps but I get stuck on the YUM commands. Has anyone came across a guide/tutorial which describes how to create an AMI for Suse.
Regards Ola
Excellent article Phil. Any tutorial for Windows AMI?
For future reference could you tell us how you determine what kernel to use. Im currently creating an instance following you instructions verbatim but the latest kernels for ec2 are 2.6.21 and my current centos install is 2.6.18. And Id rather not upgrade or downgrade the kernel unless it’s unavoidable.
Thanks
Hi,
I followed the steps mentioned , exactly and I am glad to report that works perfectly fine today!
Thanks alot!
Deependra
I’m also curious about the kernel update to 2.6.18, and to generalize these instructions.
Would this be the right modules to try: http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1345
Hi Marc,
Yes you could use those modules.
These seem like great steps. Are there similar kind of steps / tutorial for Windows users? I am familiar with “unix”. It’s just that I do not have access to one right now. So, I would like to do all the above things on Windows to create an image
Thanks!
could i build an AMI using LFS?
http://www.linuxfromscratch.org/
Hi,
Thanks for your web page and instruction for generating centos based ami.
I am looking at getssh and would you explain what the line below is doing?
/usr/bin/curl -f http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key > /tmp/my-key
What machine is 169.254.269.254?
Thank you.
Hello Tena,
169.254.0.0-169.254.255.255 is what IANA Automatic Private IP Addressing
In Amazon Web Services case they use it for API calls to retrieve information about your instance on startup. This particular API call I use in getssh has to do with your SSH key to add to your authorized_keys
Phil
Hi Phil,
Thanks for your response.
I would like to report a possible / typo on your page where you do the registration:
$ ./ec2-register phils-amis/ami-centos5.3-64bit-base.manifest.xml
When I issue the command similar to the above, it complains as to name is missing.
With the current incarnation of ec2-register, it seems “-n a-name-4-my-ami” is required.
Tena
Hi Phil,
I don’t understand what the command below means:
[phil@desktop]$ ./ec2-add-keypair phils-keypair-raw
What’s phils-keypair-raw? And what’s the rest of the stuff (keypair + private key) in the panel? How is this related to ec2-add-keypair command?
Please explain.
Thank you.
Tena,
Check out:
http://docs.amazonwebservices.com/AmazonEC2/dg/2006-10-01/CLTRG-create-keypair.html regarding the ec2-add-keypair.
To be honest you can just use the AWS console for your key pair management.
Any chance someone can post a cent os AMI that’s ready to go? maybe just take out their SSH key or anything that’s private??
Troy,
See my blog post:
http://3.84.23.23/2010/05/31/amazon-ec2-64bit-centos-5-3-base-public-ami-s
I have created public versions in each region. The key for root is whatever you start the image with. Hope that helps.
Hi Phil,
I have been successful creating centos 5.5 ami from your instruction and using the instance. Many thanks for your article. Thanks also for responses to my questions (and to others’s as well).
May I suggest in the next incarnation of the post, you include s3fs? s3fs sounds like an excellent tool to make “permanent” file system out of S3. I think it is a bit complicated due to underlying/required components, fuse for instance. Maybe this is a bit beyond “it is left as student exercise” and requires a helping hand from professor Phil. 🙂
Tena
Hi Phil,
I want to run an instance and then have that instance launch more instances. To do so, I need to include at least ec2-run-instances and other ec2-bla-bla tools in my ami. Would you please comment as to how I go about this? I.e., Where would be a good place to copy these amazon tools? What pitfalls might there be?
Thank you.
Tena
Hi Tena
Thanks for the feedback Tena I will look into writing a post for s3fs or a topic that meets the same requirement. As for your other question.
You can include the amazon EC2 API tools within your AMI, by starting an instance of the AMI you created then wget the api tools in something like /usr/local/ami-tools write a bash, perl, or whatever scripting language you desire to execute starting instances given whatever your requirement is and then follow my snapshot article to create a new AMI, bundling all this.
I actually don’t recommend doing it this way though, because you will need to have your amazon security credentials (pem files) embedded in your AMI. This is not the best idea. I wouldn’t bundle AWS security info in an AMI ever. I would think creating a server maybe not cloud that runs a RESTful API service that you can call via a token authentication system which in turn fires up new EC2 Instances would be better. On this server you probably should run a HIDS like OSSEC.
So the workflow would be that your current AMI has a script or daemon that runs looking for your criteria of starting EC2 instances, when the criteria is met then it makes a secure API call to a RESTful API service server you have centralized which is hardened and has EC2 API TOOL’s and starts whatever number of instances you would like.
Just my take on it. I could be missing what your asking me though. Security is a concern so just be careful on where you put your credentials. I always to the command history -c whenever I do snapshotting also so that my credentials never get stored in the snapshot.
How do I change this for fedora 13 x86_64?
Most is pretty straightforward, but what do I do with
Install precompiled modules supplied by Amazon, since the kernel modules installed by yum are not appropriate for the EC2 environment.
Hi Phil,
Great work! Thanks so much for taking your time to write up this guide. This is very helpful!
One question I have, since I only have a desktop (installed centos 5.5) at home. Does hardware matter at all? Do I need a real 4u server to create an ami?
Thanks!
K
Hi Kevin, you should be fine with using your CentOS 5.5 desktop build to build your AMI with my instructions. The only thing you need to be aware of is if you are using a 32bit or 64bit desktop environment. Using my method if you have a 32bit desktop you can only build the 32bit AMI and a 64bit desktop will allow you to build a 64bit AMI. I have blurbs for either, also. Having a 4u chassis doesn’t matter its really the OS and architecture that does.
AWESOME!!! This is going to be my thanksgiving project!
Have a great turkey! Thanks again, Phil.
K.
Kevin,
You have a great thanksgiving too. And good luck on your EC2 project I am sure it will turn out great! Also Kevin if you decide to build a 64bit AMI you will need to launch at minimum a large EC2 instance when done, if you are using 32bit AMI you can use small and up.
Hello Phil,
I was wondering if you would show us mere motals how to build EBS based ami…
Thank you.
Tena Sakai
Hi Tena,
I have been so busy I have totally neglected blogging :-/ but this holiday I think I can start eeking out some time for more how to’s! Thank you for the feedback, I think a EBS based AMI would be a great post.
Just a quick note that I didn’t see mentioned elsewhere. Any yum.conf file will, by default, look at /etc/yum.d/repos as well, so in effect your using what you’ve created plus what’s already defined on the system. Normally it may not be a big deal, but if you add a:
reposdir=
to the yum-ami.conf file you can make sure that only your custom yum.conf file is used instead of any system specific repos.
Great post!
Thanks Jason for that info, good to know. Glad you like the post.
Greetings Phil,
I have emulated your post and generated an ami, which is working fine, but it doesn’t have c compiler. I would like to build another ami with compilers (including fortran). Can this be done by adding lines to yum-ami.conf, which in your example, resides in /opt/EC2YUM/ directory? Would you mind showing me what lines that would be?
Thank you.
Tena Sakai
Hi Phil,
Please ignore my previous post. There was no need to do anything to yum-ami.con file. I was able to run yum commands on the running EC2 instance and then snapshot (thanks for your other web page) the image.
Thank you.
Tena Sakai
Hi guys, I am new to this concept and would like to learn
Can any1 please comment here
What is an instance ? Does it mean SERVER ?
and why do we take snapshots of instance ? What purpose does a snapshot serve ?
Maida,
http://www.amazon.com/Host-Your-Web-Site-Cloud/dp/0980576830 is a good book that recently came out to start with. Other then that just check out all the blogs and give it a whirl.
Hi, guys here seem to be Pros working on Amazon cloud computing. what is the best place to get training on Amazon cloud computing? Thanks
Hi Phil,
First thank you for the post. It is by far the best and most practical and informative guide I’ve seen around.
I’ve searched around in the aws docs, but can’t find informative or practical links / downloads about the EC2 kernel modules. Where did you get the ones you used for this article?
Thanks!
Hi Roy,
Glad you found the article helpful.
http://alestic.com/2008/05/kernel-modules-2-6-21 is a good article on kernel modules.
Hi
So now i uploaded the files to S3, but i can only see it as filename.00 filename.01. till filename.51, how to combine them as .img file now?
i did it partially through ec2- tool from console, and through upload link at aws manage link.
Its very useful for the beginner.
Magento Development Company
Nicole,
The manifest file is the way all the individual files come together. You should see that file at the end of the different segmented files.
Yogesh,
Your welcome
Phil,
Thanks for the great site.I have a query regarding installing gcc in the image i am creating.I had followed your steps and had registered an image in EC2.When i tried to install php from there,the gcc was missing.please let me know how to install gcc.
Note: I am installing the OS my mounting it in any linux directory and pointing my YUM conf to that directory.
Phil,
Thanks a lot for this great document. I was successfully create EC2 image and placed on s3. Can I discuss some other issue related to EC2 image bundling. Recently we got an instance –> created snapshot –> Now it seems to be placed in EBS. I am looking to bundle this instance on S3 bucket but when I try to do so, it throws “ERROR: Can’t access instance metadata. If you are not bundling on an EC2 instance use –no-inherit.” error 🙁 . I tried to follow Amazon Ec2 forum and couldn’t get anything. Could you please suggest?
Thanks,
Shah
Phil
Thanks for the practical help on creating EC2 AMIs. I have been following your article, but for an m1.small Redhat 32-bit instance. When I get to the creation of the 10GB file to hold the OS files I get a message saying I’ve run out of space:
[root@server]$ dd if=/dev/zero of=/opt/EC2AMIIMAGE/ami-rhel6.1-32bit-base bs=1M count=10240
dd: writing `/opt/EC2AMIIMAGE/ami-rhel6.1-32bit-base’: No space left on device
4225+0 records in
4224+0 records out
4429357056 bytes (4.4 GB) copied, 233.488 s, 19.0 MB/s
When I check the amount of available disk with “df -h” I get
[root@server]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/xvde1 5.7G 5.7G 0 100% /
none 837M 0 837M 0% /dev/shm
Do you know how to fix this problem.
Thanks
David
Hi David,
You could try making the file smaller like 5G say:
dd if=/dev/zero of=/opt/EC2AMIIMAGE/ami-rhel6.1-32bit-base bs=1M count=5120
Or you could build your AMI from a machine with more disk space if you need more disk.
Really nice post. I was wondering if there is any way you can show how top create EBS backed ami instead of the instance store.
Hi,
we have few instances in EC2. I need to save it in EBS. Can you please guide me.
Thanks
Earnest paul
This is great and has really helped me out, one thing that would be very useful remains, would it be possible to include LVM filesystems in this image?
Hi Phil,
Its Great article. but right now i’m using Cetos6.2-32bit version as. & so far its woked gr8 for me bt hav stucked while installing Java to the image!!! Please help me in that.
if i use this commad — “-/usr/sbin/chroot /mnt/ami-centos6.2-32bit-base ./jre-6u12-linux-x64.bin”
it gives me error as — “/usr/sbin/chroot: failed to run command `./jre-6u12-linux-x86.bin’: No such file or directory”
& even i try to install java manually like u said via other command — “/usr/sbin/chroot /mnt/ami-centos6.2-32bit-base yum install jdk-1.6”
it throws the error as– “http://ftp.iitm.ac.in/centos/6.2/os/i386/repodata/repomd.xml: [Errno 14] PYCURL ERROR 6 – “Couldn’t resolve host ‘ftp.iitm.ac.in'”
Trying other mirror.”
rather for every dependancy it thowing dis error — “ERROR 6 – “Couldn’t resolve host ‘ftp.iitm.ac.in'”
Trying other mirror.”
So please help me in that & possibly create d othr guide for newer versions of it!!
Thanks phil.. 🙂
For anyone who is having issues or wants to see the EBS backed build version of this please see my latest blog: http://3.84.23.23/2012/04/30/how-to-create-an-amazon-elastic-compute-cloud-ec2-machine-image-ami-centos-6-s3-backed-or-ebs-backed
Thanks
Hi Phil,
I am new to AWS as well to Linux and I am trying to make a CentOS 5.8 AMI with MySQL, Tomcat and Jasper Report Server installed in the image. But installing MySQL is throwing error for error 2002 /var/lib/mysql/mysql.sock not found. My question is what kind of configuration and settings are required for MySQL to run on Image file. How to test it and how to configure it?
Any input will be helpful.
Thanks