This how to article will go over creating a Amazon Elastic Compute Cloud (EC2) Machine Image (AMI) S3 backed as well as EBS backed. In this particular example we are creating a Centos 6 64bit AMI from beginning to end. I wrote a blog about this topic in 2010 and decided to freshen it up a bit with a CentOS 6 build.
First you will want to hop on a CentOS 6 box you have available so that all the yum commands will work below.
This first step is to setup your basic environment to start building.
yum -y install e2fsprogs ruby java-1.6.0-openjdk unzip MAKEDEV mkdir -p /root/.ec2 cp amikey.pem /root/.ec2 cp amicert.pem /root/.ec2 mkdir -p /opt/EC2TOOLS /data /opt/EC2YUM curl -o /tmp/ec2-api-tools.zip http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip cd /tmp unzip /tmp/ec2-api-tools.zip cp -r /tmp/ec2-api-tools-*/* /opt/EC2TOOLS curl -o /tmp/ec2-ami-tools.zip http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.zip cd /tmp unzip ec2-ami-tools.zip cp -r /tmp/ec2-ami-tools-*/* /opt/EC2TOOLS |
Save your X509 certificates in the following directories:
/root/.ec2/amikey.pem /root/.ec2/amicert.pem |
cat <<EOT >/root/.bashrc export PATH=\$PATH:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/opt/EC2TOOLS/bin export EC2_HOME=/opt/EC2TOOLS export EC2_PRIVATE_KEY=~/.ec2/amikey.pem export EC2_CERT=~/.ec2/amicert.pem export JAVA_HOME=/usr EOT source ~/.bashrc |
export RELEASE=6 export ARCH=`uname -i` |
This step is to create your disk image and filesystem and file structure.
dd if=/dev/zero of=/data/CentOS$RELEASE-$ARCH-base.img bs=1M count=10240 mkfs.ext4 -F -j /data/CentOS$RELEASE-$ARCH-base.img mkdir /mnt/ec2-image mount -o loop /data/CentOS$RELEASE-$ARCH-base.img /mnt/ec2-image/ |
cd /mnt/ec2-image mkdir -p proc etc dev sys var/cache var/log var/lock var/lib/rpm |
cat <<EOT >/mnt/ec2-image/etc/fstab /dev/xvde1 / ext4 defaults,noatime,nodiratime 1 1 EOT |
This step is to create your Yum conf that you will use to install the base OS
mkdir -p /opt/EC2YUM cat <<EOT >/opt/EC2YUM/yum-ami.conf [base] name=CentOS-$RELEASE - Base mirrorlist=http://mirrorlist.centos.org/?release=$RELEASE&arch=$ARCH&repo=os #baseurl=http://mirror.centos.org/centos/$RELEASE/os/$ARCH/ gpgcheck=1 gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-$RELEASE [updates] name=CentOS-$RELEASE - Updates mirrorlist=http://mirrorlist.centos.org/?release=$RELEASE&arch=$ARCH&repo=updates #baseurl=http://mirror.centos.org/centos/$RELEASE/updates/$ARCH/ gpgcheck=1 gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-$RELEASE [extras] name=CentOS-$RELEASE - Extras mirrorlist=http://mirrorlist.centos.org/?release=$RELEASE&arch=$ARCH&repo=extras #baseurl=http://mirror.centos.org/centos/$RELEASE/extras/$ARCH/ gpgcheck=1 gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-$RELEASE [centosplus] name=CentOS-$RELEASE - Plus mirrorlist=http://mirrorlist.centos.org/?release=$RELEASE&arch=$ARCH&repo=centosplus #baseurl=http://mirror.centos.org/centos/$RELEASE/centosplus/$ARCH/ gpgcheck=1 enabled=0 gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-$RELEASE [contrib] name=CentOS-$RELEASE - Contrib mirrorlist=http://mirrorlist.centos.org/?release=$RELEASE&arch=$ARCH&repo=contrib #baseurl=http://mirror.centos.org/centos/$RELEASE/contrib/$ARCH/ gpgcheck=1 enabled=0 gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-$RELEASE EOT |
yum -c /opt/EC2YUM/yum-ami.conf --installroot=/mnt/ec2-image -y groupinstall Base |
yum -c /opt/EC2YUM/yum-ami.conf --installroot=/mnt/ec2-image -y install openssh-server yum-plugin-fastestmirror.noarch e2fsprogs dhclient |
This step is to allow root login without password, since you will be using your key
cat <<EOT >>/mnt/ec2-image/etc/ssh/sshd_config UseDNS no PermitRootLogin without-password EOT |
Create devices so we can use chroot for the image
MAKEDEV -d /mnt/ec2-image/dev -x console MAKEDEV -d /mnt/ec2-image/dev -x null MAKEDEV -d /mnt/ec2-image/dev -x zero mount -o bind /dev /mnt/ec2-image/dev mount -o bind /dev/pts /mnt/ec2-image/dev/pts mount -o bind /dev/shm /mnt/ec2-image/dev/shm mount -o bind /proc /mnt/ec2-image/proc mount -o bind /sys /mnt/ec2-image/sys |
Create network settings
cat <<EOT >/mnt/ec2-image/etc/sysconfig/network NETWORKING=yes HOSTNAME=localhost.localdomain EOT |
cat <<EOT >/mnt/ec2-image/etc/sysconfig/network-scripts/ifcfg-eth0 ONBOOT=yes DEVICE=eth0 BOOTPROTO=dhcp NM_CONTROLLED=yes EOT |
Create grub.conf and boot settings so the Amazon Kernel Image can boot into the new Kernel
cat <<EOL > /mnt/ec2-image/boot/grub/grub.conf default=0 timeout=0 title CentOS$RELEASE root (hd0) kernel /boot/vmlinuz ro root=/dev/xvde1 rd_NO_PLYMOUTH selinux=0 console=hvc0 loglvl=all sync_console console_to_ring earlyprintk=xen nomodeset initrd /boot/initramfs EOL |
ln -s /boot/grub/grub.conf /mnt/ec2-image/boot/grub/menu.lst kern=`ls /mnt/ec2-image/boot/vmlin*|awk -F/ '{print $NF}'` kernver=${kern#vmlinuz-} ird=`ls /mnt/ec2-image/boot/initramfs*.img|awk -F/ '{print $NF}'` sed -ie "s/vmlinuz/$kern/" /mnt/ec2-image/boot/grub/grub.conf sed -ie "s/initramfs/$ird/" /mnt/ec2-image/boot/grub/grub.conf |
Create a script that grabs the public key credentials for your root login.
vi /mnt/ec2-image/etc/init.d/getssh #!/bin/bash # chkconfig: 2345 95 20 # description: getssh # processname: getssh # export PATH=:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin # Source function library. . /etc/rc.d/init.d/functions # Source networking configuration. [ -r /etc/sysconfig/network ] && . /etc/sysconfig/network # Check that networking is up. [ "${NETWORKING}" = "no" ] && exit 1 start() { if [ ! -d /root/.ssh ] ; then mkdir -p /root/.ssh chmod 700 /root/.ssh fi # Fetch public key using HTTP /usr/bin/curl -f http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key > /tmp/my-key if [ $? -eq 0 ] ; then cat /tmp/my-key >> /root/.ssh/authorized_keys chmod 600 /root/.ssh/authorized_keys rm /tmp/my-key fi # or fetch public key using the file in the ephemeral store: if [ -e /mnt/openssh_id.pub ] ; then cat /mnt/openssh_id.pub >> /root/.ssh/authorized_keys chmod 600 /root/.ssh/authorized_keys fi } stop() { echo "Nothing to do here" } restart() { stop start } # See how we were called. case "$1" in start) start ;; stop) stop ;; restart) restart ;; *) echo $"Usage: $0 {start|stop}" exit 1 esac exit $? ###END OF SCRIPT |
Make your script init ready
/bin/chmod +x /mnt/ec2-image/etc/init.d/getssh /usr/sbin/chroot /mnt/ec2-image /sbin/chkconfig --level 34 getssh on |
Clean up after yourself
yum -c /opt/EC2YUM/yum-ami.conf --installroot=/mnt/ec2-image -y clean packages rm -rf /mnt/ec2-image/root/.bash_history rm -rf /mnt/ec2-image/var/cache/yum rm -rf /mnt/ec2-image/var/lib/yum |
Check for what Amazon AKI’s are available you will need this info when starting your ec2 instance
/opt/EC2TOOLS/bin/ec2-describe-images -H --region us-east-1 -x all|grep "pv-grub-hd0" |
Bundle your new image
/opt/EC2TOOLS/bin/ec2-bundle-image --image /data/CentOS$RELEASE-$ARCH-base.img --prefix ami-CentOS$RELEASE-$ARCH-base --cert ~/.ec2/amicert.pem --privatekey ~/.ec2/amikey.pem --user awsaccountnumber --destination /data --arch $ARCH |
Upload your new image
/opt/EC2TOOLS/bin/ec2-upload-bundle --manifest /data/ami-CentOS$RELEASE-$ARCH-base.manifest.xml --bucket yourbucket --access-key accesskeyhere --secret-key secretkeyhere |
Register your image
I recommend you do this via the Amazon Web Services Management console, under IMAGES->AMIs “Register New AMI” you will just need to enter bucketname/manifest.xml file for your new AMI.
Steps to create a EBS Backed AMI from your S3 Backed Instance. *Only if you want an EBS Backed AMI do these steps
Step 1
Start your newly created EC2 instance
Step 2
Go into the Amazon Web Services Management console and create a Volume at whatever size you choose
Step 3
Attach your Volume to your new running EC2 Instance
Step 4
Login to your new instance
ssh -i yourkey root@yournewinstance |
Step 5
Create your filesystem type in this case EXT4 then make a directory and mount it to the newly formated filesystem. After that we rsync the root of your ec2 instance, then unmount.
mkfs.ext4 /dev/xvdj mkdir /mnt/ebs && mount /dev/xvdj /mnt/ebs rsync -avHx / /mnt/ebs rsync -avHx /dev /mnt/ebs sync;sync;sync;sync && umount /mnt/ebs |
Step 6
Go into the Amazon Web Services Management Console section Volumes and detach the Volume
Step 7
Go into the Amazon Web Services Management Console section Volumes and create a Snapshot of your Volume
Step 8
Go into the Amazon Web Services Management Console section Snapshots and select your new Snapshot and choose “create image” Fill in the name and description and the AWS AKI you chose in my case for us-east-1 I chose aki-08ed0761 for x86_64 bit hd0
Step 9
Start your new EBS backed AMI
Also a great resource for free Realtime AWS uptime data is Systems Watch
I’m trying your instructions (on Debian testing with yum installed) and found what I suspect to be a typo, in the mkdir -p stage, shouldn’t it be:
mkdir -p proc etc dev var var/cache var/log var/lock var/lib/rpm sys
i.e. relative paths for the var/* and var/lib/rpm instead of var/loc/rpm?
Thanks for the correction Amos. You are correct there was a typo, and the paths should be relative. It shouldn’t have effected anything though since the yum half of the instructions create the dir for you. But just the same I will correct the instructions. Thanks.
Hi,
Thanks for the instructions. I followed it to create an image, but after I uploaded it into S3, I was not able to boot it up. The syslog says root disk not found.
I am not clear about the relationship between my ami image, ari image, and aki image or if I need to specify them at all. I tried both aki-427d952b and “default” (whatever that means!), but I did not provide an ari image (left its selection at “default”).
So any insight would be appreciated!
Thanks
yz