Streaming Forum Keynote: Sky Spins Up New Channels in Minutes
What does the phrase "Powered by Sky" really mean to a traditional media company, especially one that operates an OTT live streaming platform—at scale, for millions of customers every day—across multiple countries?
Jeffrey Webb, principal streaming architect at Sky, used the opening keynote at Streaming Forum 2018 to answer that question.
Webb started by giving a brief CV, noting that he'd been at Sky for thirteen years, and is responsible for live streaming strategy at Sky in the UK.
"I want to share what we've been working on the last twelve months," said Webb. "As a broadcaster you do all these clever technologies, but our goal is to provide our customers with a quality experience. If the customer isn't seeing buffering, or latency, if the [support desk] phone isn't ringing, they're having a good experience."
Webb said the journey is from an old world to a new world, moving from an appliance-focused to container-focused video workflow.
"When we were designing the first generation, we started with appliances," said Webb. "We had separate encoders, packagers, origin servers."
The second generation, several years ago, took a hybrid approach, with Webb noting that examples of virtualization in the second-generation solution being the packager, even while the encoder and origin server remained appliance-focused.
One of the biggest reasons to move to the Streaming DevOps (SDS) version was a significant time savings when it came to standing up new channels.
"Our previous live streaming platform delivery timescales, from a streaming perspective was up to four weeks," said Webb, "with additional costs due to project timescales and the risk of manual channel build."
New live streaming platform delivery timescales, by contrast, allows each channel to be built consistently in just minutes, using templates and a modular architecture. SDS is the merging of two skill sets to optimize the video workflow.
"It's about predicability," said Webb, noting that the engineering skills and knowledge inform the modification of channels via a customized template approach.
Streaming DevOps introduces Reduced Fault Domain, using continuous deployment and continuous Integration (CD/CD) automation pipeline.
In addition, Streaming DevOps offers centralized monitoring and aggregation reparation, as well as automated audio and video quality monitoring. At the same time, Webb said, Sky looks to maintain existing highly quality streaming services for clients.
Webb explained that the Reduced Fault Domain protects channels in a much simpler way than previous generation's approaches, using containers rather than appliances.
"This allows the ability to recover channels whilst not affecting other channels," said Webb.
The SDS focus on an end-to-end automated deployment of channels is key, as is the use of a lights-out recovery approach in the event of disaster.
"The same software stack can be deployed on or off premise, further reducing risk and support," said Webb, noting that version control of channel configurations uses software engineering best practices.
Webb introduced a slide that shows a quasi-OSL layering model, offering a layered approach to an on-premise streaming architecture. Starting from the bottom, Webb's model builds on base network and server layers, followed by virtualization, all of which powers the containers and container orchestration approach Webb mentioned earlier.
"Containers offer a lightweight footprint, making efficient use of the available compute in your cluster," said Webb, adding that containers start up much faster than appliances.
Within the containerization approach, core services are foundational to video workflows and content delivery network (CDN) services.
"Core services have to become before the video workflow," said Webb, "because if you can't measure it, you can't improve it."
Webb notes that Jenkins is used for continue integration and delivery, and use Grafana for creating dashboards. Docker is used as a container engine and Kubernetes is used to orchestrate containers, with Prometheus providing container metrics reporting.
Kibana allows for providing visualization, a sort of custom dashboard, at the core services layer.
Sky client applications, for various client platforms—from mobile to desktop to set-top box—is the top of Webb's layer, the only end user-facing layer in the entire stack.
From a CDN perspective, Webb notes there's still a large need for a multi-CDN approach.
"We're at the terabytes of volume, and at this level even the biggest CDNs can face issues, so you have to split up the delivery," said Webb.
Release schedule, though, is structured so that nothing can be released to production without human intervention.
What about the cloud?
"As long as you're on Intel Xeon hardware," said Webb, SDS is designed for deployment anywhere. "We have to choose particular instance types, when it comes to the cloud. Sometimes it depends on the provider, whether they have newer or older generation Xeons with particular instruction sets."
SDS shortens the development cycle by not having to wait for infrastructure to be deployed, allowing for rapid initial response to business needs.
"SDS introduced software engineering best practices for live streaming consistency," said Webb. "We're driving towards a single-pane-of-glass to monitor the end-to-end video workflow."
Finally, Webb offered lessons learned.
"What have we learned so far in our SDS journey?" asked Webb. First, there needs to be tight collaboration between streaming engineers and DevOps engineering, and Webb sees is as a cultural collaboration binging the best of both worlds.
"Engineers now have time for research and development to enhance our services," said Webb.
Future plans including further improvements in the end-to-end monitoring visibility, while exploring new compression technologies. Webb said Sky will also further explore machine learning options.
"One of the challenges we've found is that SDS requires more effort, but the rewards are much greater," said Webb. "When we want to move to the cloud we will be able to do so much more quickly."
Related Articles
In this Streaming Forum keynote preview, Sky portfolio & program manager Deepa Kutty shares the broadcaster's current and future plans for targeted programming and advertising
24 Jan 2019
Pay TV operators pay £4.464 billion for the majority of English Premier League rights through 2022, with two more packages yet to be awarded
13 Feb 2018
Pay TV operator, soon likely to be owned by Disney, details plans to ditch the dish and continues drive to go beyond its traditional long-term subscription with streaming services.
25 Jan 2018
When does delivery from an origin server make sense, and when does a CDN—or a multi-CDN strategy—make sense? Read on for insights from Sky's principal streaming architect
08 Jan 2018
Sky's prinicipal streaming architect Jeff Webb will keynote Streaming Forum 2018 in London. In the first article in a five-part series, he takes a look at the transcoding challenges involved in delivering live sports and entertainment OTT content to Sky's viewers.
14 Dec 2017