How to create an Amazon Elastic Compute Cloud EC2 Machine Image (AMI) CentOS 6 S3 Backed or EBS Backed

April 30, 2012

This how to article will go over creating a Amazon Elastic Compute Cloud (EC2) Machine Image (AMI) S3 backed as well as EBS backed. In this particular example we are creating a Centos 6 64bit AMI from beginning to end. I wrote a blog about this topic in 2010 and decided to freshen it up a bit with a CentOS 6 build.

First you will want to hop on a CentOS 6 box you have available so that all the yum commands will work below.

This first step is to setup your basic environment to start building.

yum -y install e2fsprogs ruby java-1.6.0-openjdk unzip MAKEDEV
 
mkdir -p /root/.ec2
 
cp amikey.pem /root/.ec2
 
cp amicert.pem /root/.ec2
 
mkdir -p /opt/EC2TOOLS /data /opt/EC2YUM
 
curl -o /tmp/ec2-api-tools.zip http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip
 
cd /tmp
 
unzip /tmp/ec2-api-tools.zip
 
cp -r /tmp/ec2-api-tools-*/* /opt/EC2TOOLS
 
curl -o /tmp/ec2-ami-tools.zip http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.zip
 
cd /tmp
 
unzip ec2-ami-tools.zip
 
cp -r /tmp/ec2-ami-tools-*/* /opt/EC2TOOLS

Save your X509 certificates in the following directories:

/root/.ec2/amikey.pem
/root/.ec2/amicert.pem
cat <<EOT >/root/.bashrc
export PATH=\$PATH:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/opt/EC2TOOLS/bin
export EC2_HOME=/opt/EC2TOOLS
export EC2_PRIVATE_KEY=~/.ec2/amikey.pem
export EC2_CERT=~/.ec2/amicert.pem
export JAVA_HOME=/usr
EOT
 
source ~/.bashrc
export RELEASE=6
 
export ARCH=`uname -i`

This step is to create your disk image and filesystem and file structure.

dd if=/dev/zero of=/data/CentOS$RELEASE-$ARCH-base.img bs=1M count=10240
 
mkfs.ext4 -F -j /data/CentOS$RELEASE-$ARCH-base.img
 
mkdir /mnt/ec2-image
 
mount -o loop /data/CentOS$RELEASE-$ARCH-base.img /mnt/ec2-image/
cd /mnt/ec2-image
 
mkdir -p proc etc dev sys var/cache var/log var/lock var/lib/rpm
cat <<EOT >/mnt/ec2-image/etc/fstab
/dev/xvde1    /        ext4        defaults,noatime,nodiratime    1    1
EOT

This step is to create your Yum conf that you will use to install the base OS

mkdir -p /opt/EC2YUM
 
cat <<EOT >/opt/EC2YUM/yum-ami.conf
[base]
name=CentOS-$RELEASE - Base
mirrorlist=http://mirrorlist.centos.org/?release=$RELEASE&arch=$ARCH&repo=os
#baseurl=http://mirror.centos.org/centos/$RELEASE/os/$ARCH/
gpgcheck=1
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-$RELEASE
 
[updates]
name=CentOS-$RELEASE - Updates
mirrorlist=http://mirrorlist.centos.org/?release=$RELEASE&arch=$ARCH&repo=updates
#baseurl=http://mirror.centos.org/centos/$RELEASE/updates/$ARCH/
gpgcheck=1
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-$RELEASE
 
[extras]
name=CentOS-$RELEASE - Extras
mirrorlist=http://mirrorlist.centos.org/?release=$RELEASE&arch=$ARCH&repo=extras
#baseurl=http://mirror.centos.org/centos/$RELEASE/extras/$ARCH/
gpgcheck=1
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-$RELEASE
 
[centosplus]
name=CentOS-$RELEASE - Plus
mirrorlist=http://mirrorlist.centos.org/?release=$RELEASE&arch=$ARCH&repo=centosplus
#baseurl=http://mirror.centos.org/centos/$RELEASE/centosplus/$ARCH/
gpgcheck=1
enabled=0
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-$RELEASE
 
[contrib]
name=CentOS-$RELEASE - Contrib
mirrorlist=http://mirrorlist.centos.org/?release=$RELEASE&arch=$ARCH&repo=contrib
#baseurl=http://mirror.centos.org/centos/$RELEASE/contrib/$ARCH/
gpgcheck=1
enabled=0
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-$RELEASE
EOT
yum -c /opt/EC2YUM/yum-ami.conf --installroot=/mnt/ec2-image -y groupinstall Base
yum -c /opt/EC2YUM/yum-ami.conf --installroot=/mnt/ec2-image -y install openssh-server yum-plugin-fastestmirror.noarch e2fsprogs dhclient

This step is to allow root login without password, since you will be using your key

cat <<EOT >>/mnt/ec2-image/etc/ssh/sshd_config
UseDNS no
PermitRootLogin without-password
EOT

Create devices so we can use chroot for the image

MAKEDEV -d /mnt/ec2-image/dev -x console
 
MAKEDEV -d /mnt/ec2-image/dev -x null
 
MAKEDEV -d /mnt/ec2-image/dev -x zero
 
mount -o bind /dev /mnt/ec2-image/dev
 
mount -o bind /dev/pts /mnt/ec2-image/dev/pts
 
mount -o bind /dev/shm /mnt/ec2-image/dev/shm
 
mount -o bind /proc /mnt/ec2-image/proc
 
mount -o bind /sys /mnt/ec2-image/sys

Create network settings

cat <<EOT >/mnt/ec2-image/etc/sysconfig/network
NETWORKING=yes
HOSTNAME=localhost.localdomain
EOT
cat <<EOT >/mnt/ec2-image/etc/sysconfig/network-scripts/ifcfg-eth0
ONBOOT=yes
DEVICE=eth0
BOOTPROTO=dhcp
NM_CONTROLLED=yes
EOT

Create grub.conf and boot settings so the Amazon Kernel Image can boot into the new Kernel

cat <<EOL > /mnt/ec2-image/boot/grub/grub.conf
default=0
timeout=0
title CentOS$RELEASE
root (hd0)
kernel /boot/vmlinuz ro root=/dev/xvde1 rd_NO_PLYMOUTH selinux=0 console=hvc0 loglvl=all sync_console console_to_ring earlyprintk=xen nomodeset 
initrd /boot/initramfs
EOL
ln -s /boot/grub/grub.conf /mnt/ec2-image/boot/grub/menu.lst
 
kern=`ls /mnt/ec2-image/boot/vmlin*|awk -F/ '{print $NF}'`
 
kernver=${kern#vmlinuz-}
 
ird=`ls /mnt/ec2-image/boot/initramfs*.img|awk -F/ '{print $NF}'`
 
sed -ie "s/vmlinuz/$kern/" /mnt/ec2-image/boot/grub/grub.conf
 
sed -ie "s/initramfs/$ird/" /mnt/ec2-image/boot/grub/grub.conf

Create a script that grabs the public key credentials for your root login.

vi /mnt/ec2-image/etc/init.d/getssh
 
#!/bin/bash
# chkconfig: 2345 95 20
# description: getssh
# processname: getssh
#
export PATH=:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin
# Source function library.
. /etc/rc.d/init.d/functions
 
# Source networking configuration.
[ -r /etc/sysconfig/network ] && . /etc/sysconfig/network
 
# Check that networking is up.
[ "${NETWORKING}" = "no" ] && exit 1
 
start() {
  if [ ! -d /root/.ssh ] ; then
          mkdir -p /root/.ssh
          chmod 700 /root/.ssh
  fi
  # Fetch public key using HTTP
/usr/bin/curl -f http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key > /tmp/my-key
  if [ $? -eq 0 ] ; then
          cat /tmp/my-key >> /root/.ssh/authorized_keys
          chmod 600 /root/.ssh/authorized_keys
          rm /tmp/my-key
  fi
  # or fetch public key using the file in the ephemeral store:
  if [ -e /mnt/openssh_id.pub ] ; then
          cat /mnt/openssh_id.pub >> /root/.ssh/authorized_keys
          chmod 600 /root/.ssh/authorized_keys
  fi
}
 
stop() {
  echo "Nothing to do here"
}
 
restart() {
  stop
  start
}
 
# See how we were called.
case "$1" in
  start)
    start
    ;;
  stop)
    stop
    ;;
  restart)
    restart
    ;;
  *)
    echo $"Usage: $0 {start|stop}"
    exit 1
esac
 
exit $?
###END OF SCRIPT

Make your script init ready

 
/bin/chmod +x /mnt/ec2-image/etc/init.d/getssh
 
/usr/sbin/chroot /mnt/ec2-image /sbin/chkconfig --level 34 getssh on

Clean up after yourself

yum -c /opt/EC2YUM/yum-ami.conf --installroot=/mnt/ec2-image -y clean packages
 
rm -rf /mnt/ec2-image/root/.bash_history
 
rm -rf /mnt/ec2-image/var/cache/yum
 
rm -rf /mnt/ec2-image/var/lib/yum

Check for what Amazon AKI’s are available you will need this info when starting your ec2 instance

/opt/EC2TOOLS/bin/ec2-describe-images -H --region us-east-1 -x all|grep "pv-grub-hd0"

Bundle your new image

/opt/EC2TOOLS/bin/ec2-bundle-image --image /data/CentOS$RELEASE-$ARCH-base.img --prefix ami-CentOS$RELEASE-$ARCH-base --cert ~/.ec2/amicert.pem --privatekey ~/.ec2/amikey.pem --user awsaccountnumber --destination /data --arch $ARCH

Upload your new image

/opt/EC2TOOLS/bin/ec2-upload-bundle --manifest /data/ami-CentOS$RELEASE-$ARCH-base.manifest.xml --bucket yourbucket --access-key accesskeyhere --secret-key secretkeyhere

Register your image

I recommend you do this via the Amazon Web Services Management console, under IMAGES->AMIs “Register New AMI” you will just need to enter bucketname/manifest.xml file for your new AMI.

Steps to create a EBS Backed AMI from your S3 Backed Instance. *Only if you want an EBS Backed AMI do these steps

Step 1

Start your newly created EC2 instance

Step 2

Go into the Amazon Web Services Management console and create a Volume at whatever size you choose

Step 3

Attach your Volume to your new running EC2 Instance

Step 4

Login to your new instance

ssh -i yourkey root@yournewinstance

Step 5

Create your filesystem type in this case EXT4 then make a directory and mount it to the newly formated filesystem. After that we rsync the root of your ec2 instance, then unmount.

mkfs.ext4 /dev/xvdj
mkdir /mnt/ebs && mount /dev/xvdj /mnt/ebs
rsync -avHx / /mnt/ebs
rsync -avHx /dev /mnt/ebs
sync;sync;sync;sync && umount /mnt/ebs

Step 6

Go into the Amazon Web Services Management Console section Volumes and detach the Volume

Step 7

Go into the Amazon Web Services Management Console section Volumes and create a Snapshot of your Volume

Step 8

Go into the Amazon Web Services Management Console section Snapshots and select your new Snapshot and choose “create image” Fill in the name and description and the AWS AKI you chose in my case for us-east-1 I chose aki-08ed0761 for x86_64 bit hd0

Step 9

Start your new EBS backed AMI

Also a great resource for free Realtime AWS uptime data is Systems Watch

Comments for “How to create an Amazon Elastic Compute Cloud EC2 Machine Image (AMI) CentOS 6 S3 Backed or EBS Backed”

  1. Amos Shapira commented on May 24, 2012

    I’m trying your instructions (on Debian testing with yum installed) and found what I suspect to be a typo, in the mkdir -p stage, shouldn’t it be:
    mkdir -p proc etc dev var var/cache var/log var/lock var/lib/rpm sys
    i.e. relative paths for the var/* and var/lib/rpm instead of var/loc/rpm?

  2. Phil Chen commented on May 30, 2012

    Thanks for the correction Amos. You are correct there was a typo, and the paths should be relative. It shouldn’t have effected anything though since the yum half of the instructions create the dir for you. But just the same I will correct the instructions. Thanks.

  3. yz commented on June 7, 2012

    Hi,

    Thanks for the instructions. I followed it to create an image, but after I uploaded it into S3, I was not able to boot it up. The syslog says root disk not found.

    I am not clear about the relationship between my ami image, ari image, and aki image or if I need to specify them at all. I tried both aki-427d952b and “default” (whatever that means!), but I did not provide an ari image (left its selection at “default”).

    So any insight would be appreciated!

    Thanks

    yz

  4. Steven Lambert commented on June 23, 2012

    Phil:

    Thanks for the excellent summary. Followed instructions and had only one hiccup.

    At the script init section when I issued the command:

    /usr/sbin/chroot /mnt/ec2-image /sbin/chkconfig –level 34 getssh on

    I got the following error:

    service getssh does not support chkconfig

    am running this on this system (uname -a)

    Linux ip-10-118-193-67 2.6.32-220.4.2.el6.x86_64 #1 SMP Tue Feb 14 04:00:16 GMT 2012 x86_64 x86_64 x86_64 GNU/Linux

    Any ideas?

    Congrats on the engagement

    Cheers

  5. Hi Phil, Thanks very much for this post. I modified it to create an Oracle Enterprise Linux flavor. Your steps worked great and this website really helped me move forward on this. It’s a real gem and thank you for taking the time to post. Gil

  6. Jaap commented on October 24, 2012

    Hi Phil,

    Very interesting! I so far never created my own AMI so i am looking around a bit and your tutorial gives an enlightening peek into how to set up a clean Linux image, whether it is for EC2 or anything else. My compliments.

    I was wondering, are there any specific reasons to create the image by hand? I mean, you can also let CentOS (or any distro) do the heavy work by letting them install to the image themselves. For example, you can use libvirt (virt-install) or VirtualBox and just run the installer. I believe for VirtualBox you will then need a real host, but virt-install can run on a virtualized guest as well using qemu. And, you can do it from MacOS or Windows: chroot is not needed so you can create the image on your laptop for example.

  7. Myra commented on November 5, 2012

    Hi Phil, Thank you so much for the post. It’s been very helpful. Do you happen to know if it is possible to create separate /boot partition on /dev/sda1 in EC2? Thank you very much and any help is greatly appreciated! -Myra

  8. Hi Phil, Thanks for the great post, good work to create customized AMI. … You have created any Git repo for deployment of configuration.

Leave a Reply