How to create an Amazon Elastic Compute Cloud EC2 Machine Image (AMI) CentOS 6 S3 Backed or EBS Backed

April 30, 2012

This how to article will go over creating a Amazon Elastic Compute Cloud (EC2) Machine Image (AMI) S3 backed as well as EBS backed. In this particular example we are creating a Centos 6 64bit AMI from beginning to end. I wrote a blog about this topic in 2010 and decided to freshen it up a bit with a CentOS 6 build.

First you will want to hop on a CentOS 6 box you have available so that all the yum commands will work below.

This first step is to setup your basic environment to start building.

Copy Code 


yum -y install e2fsprogs ruby java-1.6.0-openjdk unzip MAKEDEV

mkdir -p /root/.ec2

cp amikey.pem /root/.ec2

cp amicert.pem /root/.ec2

mkdir -p /opt/EC2TOOLS /data /opt/EC2YUM

curl -o /tmp/ec2-api-tools.zip http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip

cd /tmp

unzip /tmp/ec2-api-tools.zip

cp -r /tmp/ec2-api-tools-*/* /opt/EC2TOOLS

curl -o /tmp/ec2-ami-tools.zip http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.zip

cd /tmp

unzip ec2-ami-tools.zip

cp -r /tmp/ec2-ami-tools-*/* /opt/EC2TOOLS

Save your X509 certificates in the following directories:

Copy Code 


/root/.ec2/amikey.pem
/root/.ec2/amicert.pem

Copy Code 


cat </root/.bashrc
export PATH=\$PATH:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/opt/EC2TOOLS/bin
export EC2_HOME=/opt/EC2TOOLS
export EC2_PRIVATE_KEY=~/.ec2/amikey.pem
export EC2_CERT=~/.ec2/amicert.pem
export JAVA_HOME=/usr
EOT

source ~/.bashrc
Copy Code 


export RELEASE=6

export ARCH=`uname -i`

This step is to create your disk image and filesystem and file structure.

Copy Code 


dd if=/dev/zero of=/data/CentOS$RELEASE-$ARCH-base.img bs=1M count=10240

mkfs.ext4 -F -j /data/CentOS$RELEASE-$ARCH-base.img

mkdir /mnt/ec2-image

mount -o loop /data/CentOS$RELEASE-$ARCH-base.img /mnt/ec2-image/

Copy Code 


cd /mnt/ec2-image

mkdir -p proc etc dev sys var/cache var/log var/lock var/lib/rpm
Copy Code 


cat </mnt/ec2-image/etc/fstab
/dev/xvde1    /        ext4        defaults,noatime,nodiratime    1    1
EOT

This step is to create your Yum conf that you will use to install the base OS

Copy Code 


mkdir -p /opt/EC2YUM

cat </opt/EC2YUM/yum-ami.conf
[base]
name=CentOS-$RELEASE - Base
mirrorlist=http://mirrorlist.centos.org/?release=$RELEASE&arch=$ARCH&repo=os
#baseurl=http://mirror.centos.org/centos/$RELEASE/os/$ARCH/
gpgcheck=1
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-$RELEASE

[updates]
name=CentOS-$RELEASE - Updates
mirrorlist=http://mirrorlist.centos.org/?release=$RELEASE&arch=$ARCH&repo=updates
#baseurl=http://mirror.centos.org/centos/$RELEASE/updates/$ARCH/
gpgcheck=1
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-$RELEASE

[extras]
name=CentOS-$RELEASE - Extras
mirrorlist=http://mirrorlist.centos.org/?release=$RELEASE&arch=$ARCH&repo=extras
#baseurl=http://mirror.centos.org/centos/$RELEASE/extras/$ARCH/
gpgcheck=1
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-$RELEASE

[centosplus]
name=CentOS-$RELEASE - Plus
mirrorlist=http://mirrorlist.centos.org/?release=$RELEASE&arch=$ARCH&repo=centosplus
#baseurl=http://mirror.centos.org/centos/$RELEASE/centosplus/$ARCH/
gpgcheck=1
enabled=0
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-$RELEASE

[contrib]
name=CentOS-$RELEASE - Contrib
mirrorlist=http://mirrorlist.centos.org/?release=$RELEASE&arch=$ARCH&repo=contrib
#baseurl=http://mirror.centos.org/centos/$RELEASE/contrib/$ARCH/
gpgcheck=1
enabled=0
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-$RELEASE
EOT

Copy Code 


yum -c /opt/EC2YUM/yum-ami.conf --installroot=/mnt/ec2-image -y groupinstall Base
Copy Code 


yum -c /opt/EC2YUM/yum-ami.conf --installroot=/mnt/ec2-image -y install openssh-server yum-plugin-fastestmirror.noarch e2fsprogs dhclient

This step is to allow root login without password, since you will be using your key

Copy Code 


cat <>/mnt/ec2-image/etc/ssh/sshd_config
UseDNS no
PermitRootLogin without-password
EOT

Create devices so we can use chroot for the image

Copy Code 


MAKEDEV -d /mnt/ec2-image/dev -x console

MAKEDEV -d /mnt/ec2-image/dev -x null

MAKEDEV -d /mnt/ec2-image/dev -x zero

mount -o bind /dev /mnt/ec2-image/dev

mount -o bind /dev/pts /mnt/ec2-image/dev/pts

mount -o bind /dev/shm /mnt/ec2-image/dev/shm

mount -o bind /proc /mnt/ec2-image/proc

mount -o bind /sys /mnt/ec2-image/sys

Create network settings

Copy Code 


cat </mnt/ec2-image/etc/sysconfig/network
NETWORKING=yes
HOSTNAME=localhost.localdomain
EOT

Copy Code 


cat </mnt/ec2-image/etc/sysconfig/network-scripts/ifcfg-eth0
ONBOOT=yes
DEVICE=eth0
BOOTPROTO=dhcp
NM_CONTROLLED=yes
EOT

Create grub.conf and boot settings so the Amazon Kernel Image can boot into the new Kernel

Copy Code 


cat < /mnt/ec2-image/boot/grub/grub.conf
default=0
timeout=0
title CentOS$RELEASE
root (hd0)
kernel /boot/vmlinuz ro root=/dev/xvde1 rd_NO_PLYMOUTH selinux=0 console=hvc0 loglvl=all sync_console console_to_ring earlyprintk=xen nomodeset 
initrd /boot/initramfs
EOL

Copy Code 


ln -s /boot/grub/grub.conf /mnt/ec2-image/boot/grub/menu.lst

kern=`ls /mnt/ec2-image/boot/vmlin*|awk -F/ '{print $NF}'`

kernver=${kern#vmlinuz-}

ird=`ls /mnt/ec2-image/boot/initramfs*.img|awk -F/ '{print $NF}'`

sed -ie "s/vmlinuz/$kern/" /mnt/ec2-image/boot/grub/grub.conf

sed -ie "s/initramfs/$ird/" /mnt/ec2-image/boot/grub/grub.conf

Create a script that grabs the public key credentials for your root login.

Copy Code 


vi /mnt/ec2-image/etc/init.d/getssh
 
#!/bin/bash
# chkconfig: 2345 95 20
# description: getssh
# processname: getssh
#
export PATH=:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin
# Source function library.
. /etc/rc.d/init.d/functions
 
# Source networking configuration.
[ -r /etc/sysconfig/network ] && . /etc/sysconfig/network
 
# Check that networking is up.
[ "${NETWORKING}" = "no" ] && exit 1
 
start() {
  if [ ! -d /root/.ssh ] ; then
          mkdir -p /root/.ssh
          chmod 700 /root/.ssh
  fi
  # Fetch public key using HTTP
/usr/bin/curl -f http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key > /tmp/my-key
  if [ $? -eq 0 ] ; then
          cat /tmp/my-key >> /root/.ssh/authorized_keys
          chmod 600 /root/.ssh/authorized_keys
          rm /tmp/my-key
  fi
  # or fetch public key using the file in the ephemeral store:
  if [ -e /mnt/openssh_id.pub ] ; then
          cat /mnt/openssh_id.pub >> /root/.ssh/authorized_keys
          chmod 600 /root/.ssh/authorized_keys
  fi
}
 
stop() {
  echo "Nothing to do here"
}
 
restart() {
  stop
  start
}
 
# See how we were called.
case "$1" in
  start)
    start
    ;;
  stop)
    stop
    ;;
  restart)
    restart
    ;;
  *)
    echo $"Usage: $0 {start|stop}"
    exit 1
esac
 
exit $?
###END OF SCRIPT

Make your script init ready

Copy Code 



/bin/chmod +x /mnt/ec2-image/etc/init.d/getssh

/usr/sbin/chroot /mnt/ec2-image /sbin/chkconfig --level 34 getssh on

Clean up after yourself

Copy Code 


yum -c /opt/EC2YUM/yum-ami.conf --installroot=/mnt/ec2-image -y clean packages

rm -rf /mnt/ec2-image/root/.bash_history

rm -rf /mnt/ec2-image/var/cache/yum

rm -rf /mnt/ec2-image/var/lib/yum

Check for what Amazon AKI’s are available you will need this info when starting your ec2 instance

Copy Code 


/opt/EC2TOOLS/bin/ec2-describe-images -H --region us-east-1 -x all|grep "pv-grub-hd0"

Bundle your new image

Copy Code 


/opt/EC2TOOLS/bin/ec2-bundle-image --image /data/CentOS$RELEASE-$ARCH-base.img --prefix ami-CentOS$RELEASE-$ARCH-base --cert ~/.ec2/amicert.pem --privatekey ~/.ec2/amikey.pem --user awsaccountnumber --destination /data --arch $ARCH

Upload your new image

Copy Code 


/opt/EC2TOOLS/bin/ec2-upload-bundle --manifest /data/ami-CentOS$RELEASE-$ARCH-base.manifest.xml --bucket yourbucket --access-key accesskeyhere --secret-key secretkeyhere

Register your image

I recommend you do this via the Amazon Web Services Management console, under IMAGES->AMIs “Register New AMI” you will just need to enter bucketname/manifest.xml file for your new AMI.

Steps to create a EBS Backed AMI from your S3 Backed Instance. *Only if you want an EBS Backed AMI do these steps

Step 1

Start your newly created EC2 instance

Step 2

Go into the Amazon Web Services Management console and create a Volume at whatever size you choose

Step 3

Attach your Volume to your new running EC2 Instance

Step 4

Login to your new instance

Copy Code 


ssh -i yourkey root@yournewinstance

Step 5

Create your filesystem type in this case EXT4 then make a directory and mount it to the newly formated filesystem. After that we rsync the root of your ec2 instance, then unmount.

Copy Code 


mkfs.ext4 /dev/xvdj
mkdir /mnt/ebs && mount /dev/xvdj /mnt/ebs
rsync -avHx / /mnt/ebs
rsync -avHx /dev /mnt/ebs
sync;sync;sync;sync && umount /mnt/ebs

Step 6

Go into the Amazon Web Services Management Console section Volumes and detach the Volume

Step 7

Go into the Amazon Web Services Management Console section Volumes and create a Snapshot of your Volume

Step 8

Go into the Amazon Web Services Management Console section Snapshots and select your new Snapshot and choose “create image” Fill in the name and description and the AWS AKI you chose in my case for us-east-1 I chose aki-08ed0761 for x86_64 bit hd0

Step 9

Start your new EBS backed AMI

Also a great resource for free Realtime AWS uptime data is Systems Watch

Posted in Systems

Tagged , , , , ,

Comments for “How to create an Amazon Elastic Compute Cloud EC2 Machine Image (AMI) CentOS 6 S3 Backed or EBS Backed”

  1. Amos Shapira commented on May 24, 2012

    I’m trying your instructions (on Debian testing with yum installed) and found what I suspect to be a typo, in the mkdir -p stage, shouldn’t it be:
    mkdir -p proc etc dev var var/cache var/log var/lock var/lib/rpm sys
    i.e. relative paths for the var/* and var/lib/rpm instead of var/loc/rpm?

  2. Phil Chen commented on May 30, 2012

    Thanks for the correction Amos. You are correct there was a typo, and the paths should be relative. It shouldn’t have effected anything though since the yum half of the instructions create the dir for you. But just the same I will correct the instructions. Thanks.

  3. yz commented on June 7, 2012

    Hi,

    Thanks for the instructions. I followed it to create an image, but after I uploaded it into S3, I was not able to boot it up. The syslog says root disk not found.

    I am not clear about the relationship between my ami image, ari image, and aki image or if I need to specify them at all. I tried both aki-427d952b and “default” (whatever that means!), but I did not provide an ari image (left its selection at “default”).

    So any insight would be appreciated!

    Thanks

    yz