How to Launch a Cloud Encoding Computer
In this tutorial, you'll learn how to launch a cloud computer on AWS EC2. Though this tutorial will involve an Ubuntu computer upon which I'll load FFmpeg, most of the key lessons are generic. If you need to rent a cloud computer for any application, much of what's covered will apply.
Specifically, in the tutorial I'll cover:
- How to choose an Amazon Machine Image (AMI) and computer to run it on and launch that instance
- How to connect to and control the cloud computer with PuTTY
- How to install FFmpeg on the cloud instance
- How to connect to the EC2 instance with Filezilla
- How to save your own AMI and how to Stop and restart the image for later use.
None of this is hard or complex, but several of these processes involve more steps than I can document in this tutorial. So, I'll cover the high points herein and will point you to YouTube videos and other sources to fill in any blanks.
Choose An Amazon Machine Image (AMI)
You'll need an AWS account to get started. Many users start on Amazon for S3 storage, and certainly that was the case for me, but for this tutorial, you'll be working in the the Elastic Cloud Computing service, or EC2. You can find this by logging into your AWS account and clicking EC2 to navigate to the EC2 Dashboard shown in Figure 1. Click Launch Instance to get this party started. To follow along with a YouTube video, click here.
Figure 1. Launching an instance in EC2
Amazon has simplified the process with a wizard-like approach, with the first step shown in Figure 2. Here you choose an Amazon Machine Image (AMI) which is basically an operating system and collection of applications that you can install on a cloud computer.
If you look on the left in Figure 2, you'll see tabs for My AMIs, which would include the AMI you'll create at the end of this tutorial, plus the AWS Marketplace, which takes you to AMIs built by hundreds of companies with their software preinstalled. We'll install a generic version of Ubuntu Server, but the procedure would be very similar if you were installing an AMI from the marketplace.
Figure 2. Here's the AMI that we'll install.
I'll install the 64-bit (x86) version of the Ubuntu Server 18.04, which you designate by clicking the 64-bit radio button beneath Select. Briefly, Ubuntu Server is the same as regular Ubuntu but lacks a GUI, which doesn't matter in this instance since I won't be using one. It is possible to control your remote Ubuntu instance with a GUI, so if you want to do this, install Ubuntu from a different AMI, not Ubuntu Server. Canonical has a free AMI you can access from the AWS Marketplace (and check out this article entitled "Remote Desktop on an Ubuntu EC2 Instance").
Choose the 64-bit (x86) radio button then click Select, and you'll be taken to Step 2: Choose an Instance Type.
Choose an EC2 Instance Type
The EC2 Instance is the type of computer that you'll launch the AMI on, which controls performance and cost. There are multiple types of instances (general purpose, compte, memory, and storage optimized, accelerated computing) and multiple options within each category. In researching the best instance type for FFmpeg, I ran across an article from codec and encoder vendor Ittiam entitled, "Making Optimal Deployment Choices for an Encoding System on AWS," which recommended a C4.2xlarge system that costs about forty cents an hour. Another option is the newer c5d.2xlarge that delivers slightly better performance for a few cents less per hour. I select the latter in Figure 3.
Figure 3. I'm going with the c4d.2xlarge that costs about $0.384/hour in my zone
Then click Review and Launch. Note that the aforementioned YouTube video goes through all the screens from Choose Instance Type to Review (see the top of Figure 3) but most of this is important when you'll be configuring a site for public usage, not personal consumption. The only option worth considering is extra storage which you might need if uploading lots of videos to the cloud system for encoding (or if you're performing a different application altogether). If you're encoding videos located in one bucket and delivering them to another, the 200 GB of storage included with the system should suffice.
You'll probably see some security-related messages in the Step 7: Review Instance Launch screen. If you're configuring an image for temporary FFmpeg encoding, you can probably ignore these. If you're creating a site for sensitive long term use, you should seek help elsewhere and resolve them. For our purposes, I'll assume you're good to go so press Launch. This takes you to the Key Pair setting screen shown in Figure 4.
Choosing Your Key Pair Settings and Launch Your Image
AWS uses key pairs to ensure that only authorized users can start an instance. You create the key pair in AWS and the public key is stored in AWS. You download the private key which you'll input to every application used to communicate with the instance. Lose the key and you can't open the instance, so save it carefully.
If you've created a key pair setting before, you can choose and use that here. Otherwise, choose Create a new key pair (Figure 4), name it, and click Download Key Pair to save the private key with a .pem extension. Then choose that Key Pair Setting in the drop down list, click the check-box that will appear warning you about losing your key file, and then Launch Instances. Congratulations, you've launched an instance and your hourly charges are accruing.
Figure 4. Choosing a key pair setting
The Launch Status page will open with the name of the instance (see Figure 5). Click the name to navigate to the Instances screen shown in Figure 6.
Figure 5. Here's your instance. Click it to open the Instance page
Naming Your Instance
Two things are going on in Figure 6, which shows our running instance. First, in the grayed area, I named the instance by clicking what was the blank area under the Name field. Then I typed in Sourcebook 2019 Cloud Computer and saved the name. This is obviously a good idea if you'll be creating multiple instances. Then click Connect on the upper left (in the grayed area in the Figure) to open the Connect To Your Instance screen.
Figure 6. Naming your instance and opening the Connect screen
Connecting to your Instance with PuTTY
You need to connect to your remote computer to configure and run it, and you'll do this via a free SSH client named PuTTY. To follow along with a YouTube video, click here, but it's pretty straightforward. To run PuTTY, you'll need the Host name and User name, both shown in Figure 6 (by default, the user name for Ubuntu is always ubuntu, lower case). You'll also need the private key you created in step 3.
The only wrinkle is that you can't use the private key as downloaded; you'll have to convert the private key from Amazon with a .PEM extension to a PuTTY private key with a .PPK extension. You'll do this with a utility named PuTTY Key Generator. This takes a few steps, but it's not difficult; here are the highlights, all shown in the YouTube video.
a.Download the 64-bit PuTTY with installer from https://www.putty.org/. This installs PuTTY and PuTTYGen.
b. Run PuTTYGen, or PuTTY Key Generator (Figure 7). Click Conversions > Import Key and then navigate to and load the .PEM file you downloaded from Amazon and save it as a PPK file (Save private key). Don't password protect the key at this point.
Figure 7. Converting the .pem Amazon key to a .ppk PuTTY key
c. Run PuTTY and as shown in Figure 8, click Connection, SSH, then Auth to get to the screen shown. From there, use the Browse button to locate and load the PPK file you just created.
Figure 8. Loading the PPK key
d. Add the Auto-login user name in the Data connection field (see Figure 9). Be sure to use lowercase ubuntu as uppercase Ubuntu won't work.
Figure 9. Adding the user name
e. Copy and paste the Host Name from the Connect to Your Instance screen shown in Figure 6 to PuTTY's Host Name field (Figure 10). Keep the port at 22. If you'd like, save the session using the Saved Sessions controls mid-screen so you can easily return to it later. Then click Open on the bottom right.
Figure 10. Adding the host name and saving the session
You'll see a warning screen that you can ignore (click Yes) and you should be rewarded with the screen shown in Figure 11. You're in.
Figure 11. You're in. Welcome to your Ubuntu cloud instance
Connecting to Your EC2 Instance with FileZilla
You may need to upload files to and download files from your instance, and open source utility FileZilla is perfect for this. Use the same basic procedure to connect to the instance with FileZilla (Figure 12). In FileZilla, open the Settings dialog, click SFTP on the left and then click Add key file to select and add the PPK file. Then insert the Host and User names from Figure 6 to the top of the FileZilla interface, and you're in. This should do it, but if you're having problems, check out a short tutorial entitled "How to Connect to Amazon EC2 Using FileZilla."
Figure 12. Connecting to your EC2 instance with FileZilla
Configuring Your Instance
You setup programs onto your instance just as you would for a local Ubuntu computer. I updated to FFmpeg 4.x on my system after referring to this article entitled, "How to Install FFmpeg 4.0.2 in Ubuntu 18.04/16.04 via PPA." It just took a few moments.
Once you've got your system configured, you can save it as a custom AMI by right clicking the instance in the EC2 Dashboard and choosing Create Image (Figure 13). That makes the image accessible to you next time you start the process via the My AMIs tab shown on the left in Figure 2. There is no charge for creating an AMI.
Figure 13. Saving your configuration as a custom AMI
You stop the instance by right clicking it and choosing Instance State > Stop (Figure 14) which terminates the hourly machine charges associated with that system. Stopping the instance flushes data from the local (ephemeral) hard drive but not the attached bootable EBS volume, which means some storage charges will continue to accrue. Unless you’re absolutely sure that your files will be preserved it’s best to download them to a local system before stopping the remote instance. Terminating the system via the Instance State > Terminate command flushes all data and stops all compute and storage charges.
Figure 14. Stopping the charges on my EC2 instance
If you Stop the instance (as opposed to Terminate) Amazon stores the instance in the EC2 Dashboard (Figure 15) where you can restart it by clicking the instance and choosing Instance State > Start. If you Terminate the system, you can’t restart it.
Figure 15. Restarting the instance
If you’re concerned about charges associated with stopped instances, check out this article entitled "What Is The Difference Beween Terminating and Stopping an EC2 Instance?" What I took away was that if you’ll be using the system frequently, it’s better to Stop the instance so you can easily restart with critical data intact. Otherwise, you should Terminate the instance to stop all charges (and knowing that you can resurrect most of the functionality via the AMI you saved in Figure 13.
While meticulous, none of this is particularly hard and you should be able to get up and running in 30 minutes or less. Once you have access to the unlimited resources in the cloud, you’ll wonder how you ever got along without it.
Related Articles
Discovery Communications' sports division spent €1.3 billion on pan-European digital and linear rights to Olympic Games until 2024, and aims to make every second of every event from PyeongChang available on multiple platforms.
30 Jan 2018
At NAB, execs from Amazon, Netflix, 20th Century Fox, and Warner Bros. shared their insights into how virtualized workflows in the cloud are helping them meet international demand
11 May 2017
New offerings in cloud video encoding let companies both large and small find the option that makes the most sense for them.
09 Nov 2016