DIY: Streaming to Multiple Destinations
How do I stream to multiple destinations like YouTube and Facebook at the same time? I hear this question a lot, particularly at the budget end of the market—the occasional streamer or the amateur who's not afraid of doing a little coding but doesn't have the need for a commercial service to solve the problem.
It's no surprise: The majority of contribution options into most live streaming services offer just RTMP. This is true for example for Facebook Live, YouTube Live and Twitch TV at the time of writing. Each of these offers a single RTMP sign on. On top of this – in order not to congest those all-important contribution feed internet connections - most encoding software only outputs to one target, and at best two.
While in the distribution from the core service to the edge RTMP has largely been replaced with HLS/HTTP and end-user media decoder frameworks/players are almost universally heading that way too, for the contribution feed into the distribution network, session-based (rather than chunked) protocols offer features such as much better feedback as to the quality of the contributed feed. The session based protocol RTMP is still significantly lower latency than most HTTP distribution models, and for a live engineer this service level commitment to do their part with as little latency as possible is helping RTMP stick around, too.
Of course there are many reliable UDP transports that might improve on this, but few as yet have commonly available libraries for encoder developers to just pick up and use. RTMP is widespread. SRT is worth investigating, but it will be some time until it reaches ubiquity like RTMP.
But typically these encoders can usually only send a single stream to a single end point or, at best, on occasion they might send two end points, offering a back up path.
But if you want to send a single stream to multiple outputs, this also puts pressure on the contribution feed's network link. If a single stream can be split out to multiple destinations from an end point with abundant connectivity—such as a virtual cloud machine offers—then this means that the field engineer can limit the bandwidth requirement of the network link. If this is a bonded 3G or 4G network, a DSL network, or an expensive VSAT link, then this reduces the risk of multiple streams contending for bandwidth if there is any variance (let alone cost of over-provisioning to ensure this risk is reduced).
There are some fantastic commercial services emerging focusing on this specific “origination” role, and many offer extended features such as paywalls, DRM, transcoding, archiving, and ad insertion, and some will include your player, metrics, and the full CDN service. Companies providing such services as SaaS models include the likes of Livescale, Vimeo, Switchboard Live, and Brightcove, as well as many others you will find in the annual Streaming Media Sourcebook Industry Directory.
Sending a stream to multiple distribution networks (I call it stream splitting) is also a very simple thing to do in a very rudimentary way with some open source code that is freely available online, and I thought it might be useful to provide a simple walkthrough to show you how to set this up for your own purposes. Be this something you just play with one quiet evening, or something that goes on to be the bedrock of your commercial operation, I hope the following proves stimulating. I'm also hoping those of you stuck on Windows or Mac will give a Linux workflow a try.
I hasten to note that of my own company's team I am the one not allowed to deliver production code. In fact, the rest of my team programs using the functional language Erlang, where I am a 'lowly' PHP and Shell Script hacker, with enough capability to read a C/C++ program and get the idea of what is going on sufficiently to find critical hooks for a model, but no where near enough ability to write anything serious in any compiled languages. Fortunately the rest of my team are pretty good at turning my hacks and ideas into something production ready!
However, my own code in these DIY articles, however should NOT be used in high-scale, high-availability environments!
With this, the first of a series of DIY articles, on the one hand I hope to provide something quick and interesting for software engineers who are proficient in more complex languages and methods, but curious about specific functions required by webcasters.
On the other hand, I also hope to provide and interesting “hands-on” article for the interested amateur webcaster who wants to implement some of the functionality they require themselves in a practical way. I feel this will help those who—like me—increasingly spend more time in the business of streaming than in the engineering of streaming to maintain a clearer picture of what is going on with the evolving services and capabilities under the hood of the various operators we all work with.
Let me explain the workflow we are going to build:
Software RTMP encoder>->Virtual Machine>-> Software plug in>NGINX>Multiple CDN End points (in this case Twitch, Facebook, and YouTube).
Note that the video is encoded only once—this is a transmuxing workflow, meaning the same encoded video is re-packaged from one container/format/transport as it passes through the workflow, but NOT re-encoded. Re-encoding is expensive (in terms of CPU cycles which could be used for other things) and will also require more time/add latency. We don't need to do that with this workflow. All we do is copy the video into different output sessions, with the virtual machine completing authentication for each session and acting as a “splitter” for our signal.
I am going to assume that you have your own virtual cloud service of choice. I have flipped a coin and decided to use a Google Cloud Platform machine and to deploy in Ubuntu 16.04 which I am familiar with. Of course you could set this up on your local machine or even on a locally run virtual machine, or in fact anywhere, but for ease let's just spin up a VM on GPC for now.
I launched an n1-standard-1 (1 vCPU, 3.75GB memory) machine in GCP.
Note turn it off when you are not making money or learning from it!
Before we do anything to the box we will need to allow the inbound RTMP stream through the cloud firewall, so pop into the VPC Network Firewall Rules settings and add tcp:1935 for 0.0.0.0/0 (anyone—so be careful to lock that down if you want to make things a bit more secure!)
Then on the machine's management page, hit the SSH box, and choose Open in Browser Window:
A few moments later—assuming you are using Chrome—you will have a command prompt in your browser!
Now you need to type the following:
Let it install. It will take a couple of minutes. Near the end of the process it will open a simple text editor looking at the system's config file.
This is easy—it's a very simple word processor called nano. Just use the cursor keys to move around. And on a Mac remember use Control key, not Command, for the functions at the end.
Now scroll down and find the following lines:
Edi these lines to reflect your RTMP destinations. You can add or remove extra lines for more destinations.
Once you have finished, press <ctrl-x> then confirm by pressing “y.”
The install will complete and finish with a line that tells you:
The IP address I have highlighted will vary with the virtual machine.
Using this you can now set your encoder to send an RTMP stream to (replace '<ip-address>' with the one found on your virtual machine as above):
Server: 'rtmp://<ip-address>/stream'
Key: 'source'
And within a few moments your stream will be live on all the different platforms.
Ta-da!
Kudos to Roman Arutyunyan who published the niginx-rtmp-module at the heart of this.
This splitter is extremely basic. But network links aside, it is also so simple it is reasonably reliable. If you get stuck into the module configuration it is possible to extend the capability to—for example—record a copy of the stream on the virtual machine, or to setup monitoring and statistics…enjoy!
Related Articles
In the latest installment of our do-it-yourself series, we'll look at how to set up an Icecast server on an AWS cloud instance.
21 May 2018
For the second installment in our do-it-yourself series, we look at how to set up a very basic audio stream between two computers on the same LAN. While the coding is simple, the concepts underneath are crucial to an understanding of how streaming works.
04 Apr 2018