Telestream CTO Simon Clarke Talks AI, Cloud, and Wirecast and Vantage Trajectories
In a recent interview, Simon Clarke, Chief Technology Officer of Telestream, discussed two key products, Wirecast and Vantage. Celebrating Wirecast’s 20th anniversary, Clarke reflected on its evolution from a nascent streaming tool to a robust application used by a diverse range of streaming producers, including educators, business professionals, and houses of worship.
The discussion highlighted several key areas: Wirecast's development trajectory, the introduction of AI-driven features, Vantage's transition to cloud-based workflows, and future enhancements aimed at improving user experience and functionality. Clarke’s insights reveal how Telestream is adapting to industry changes and user needs, focusing on innovation and practical application.
This article will explore the main topics discussed with Clarke, providing an overview of Wirecast’s significant milestones, the impact of AI and machine learning on Telestream’s products, the role of cloud integration, and planned enhancements to Wirecast and Vantage.
Evolution of Wirecast
As Clarke recounted, Wirecast was initially designed to facilitate the creation of live shows by integrating multiple cameras, graphics, and pre-recorded video. According to Clarke, the product was ahead of its time when it launched in 2004, during the era of Windows Media and RealNetworks. Early challenges included dealing with buggy graphics drivers and the lack of general-purpose GPUs, which made the software difficult to run effectively.
Over the years, Wirecast has undergone several key evolutions. By 2010, the product had transitioned to primarily supporting RTMP and H.264, which became the leading formats for streaming. Notable feature expansions included screen recording capabilities and the ability to integrate social media feeds into live productions.
The COVID-19 pandemic had a notable impact on the usage of Wirecast. As remote work and virtual communication became essential, the demand for streaming solutions increased dramatically. Clarke noted that while the surge in usage has since stabilized, the primary user base—comprising education, house of worship, and business applications—remains consistent.
Wirecast's functionality has expanded to include features like multi-camera inputs, conferencing capabilities, and the ability to stream to multiple destinations simultaneously. Clarke explained that many users leverage Wirecast’s robust title engine and virtual camera driver, which allows the output of Wirecast to be used as an input for other programs like Zoom and Microsoft Teams.
Additionally, Wirecast includes a feature called Rendezvous, which facilitates remote conferencing by integrating guests into a Wirecast presentation via WebRTC. Clarke mentioned that while Zoom and Microsoft Teams have become dominant in the remote conferencing, Wirecast's integration with Zoom, introduced in version 16, has been well-received. This integration allows users to bring Zoom guests directly into Wirecast, providing advanced mixing capabilities not available in Zoom alone.
AI Integration in Wirecast
Clarke discussed how AI has been incorporated into the platform, focusing on practical applications that benefit users. One significant AI integration is the virtual assistant, introduced in Wirecast 16. This assistant, accessible from the help menu, provides users with instant support for common issues and questions. Clarke explained that the assistant leverages a generative AI system built around Wirecast-specific content and streaming knowledge. Users can ask questions such as troubleshooting connectivity issues with YouTube or addressing streaming problems, and the assistant provides relevant and helpful answers based on internal knowledge.
Another AI-driven Wirecast feature is the virtual PTZ (Pan Tilt Zoom), which uses machine learning to detect and track individuals within the video frame, allowing the camera to follow them as they move. Clarke noted that this is particularly useful in high-resolution settings as it ensures that the subject remains in focus without requiring manual camera adjustments.
Wirecast also includes an AI-based background removal feature, similar to those found in platforms like Skype and Zoom. This allows users to replace their background with virtual ones without needing a greenscreen.
Looking forward, Clarke revealed that Telestream is exploring the integration of AI-driven captioning within Wirecast, attaching them to the live stream in real-time. While various technologies are being considered, including OpenAI's Whisper, the focus is on finding a balance between high-quality results and efficient resource usage.
Vantage and its Evolution
Vantage, Telestream's flagship media supply chain workflow solution, has also seen significant advancements and integration of AI. By way of background, Vantage is designed to handle various media processing tasks, making it a cornerstone product for many media organizations. It enables users to create complex workflows through a user-friendly interface known as the Workflow Designer. This interface allows non-programmers to build intricate decision-tree workflows by placing and connecting action blocks on a canvas.
Telestream recently integrated generative AI into the Workflow Designer, a feature it showed at NAB. The new AI-powered feature acts as a co-pilot for operators, allowing them to create workflows using natural-language prompts. Users can describe the desired workflow in plain language, and the AI generates the corresponding workflow automatically. This functionality aims to help both novice and experienced users create and optimize workflows more efficiently.
Vantage also leverages AI to enhance existing workflows. Clarke highlighted a feature that provides natural-language descriptions of workflows, making it easier for users to understand and manage complex workflows, especially those created by others or by former employees. This feature also facilitates searching for specific workflows and identifying duplicates, further streamlining operations.
Another significant evolution is Vantage's transition to the cloud. Initially an on-premises solution, Vantage now supports various cloud integration models. Users can send parts of their workflows to the cloud for processing or host entire workflows in the cloud. Clarke described two primary models: ground-to-cloud orchestration, where certain tasks are offloaded to the cloud, and fully hosted workflows that run entirely in the cloud. These models provide flexibility for users, allowing them to leverage the cloud’s scalability and cost-efficiency while maintaining control over their media processing.
Clarke also discussed the integration of AI into Vantage for codec development and content adaptive encoding. While Telestream is not a codec vendor, it works with leading codec developers to incorporate AI-driven improvements. For instance, AI can enhance rate distortion performance in widely used codecs like H.264, offering better compression efficiency without compromising quality. Clarke noted that performance and speed are critical considerations, as any AI-based improvements must not significantly increase processing time.
Future Developments for Wirecast and Vantage
Clarke shared insights into some of the future developments planned for Wirecast and Vantage, focusing on improving usability, functionality, and integration with emerging technologies.
Wirecast
One key focus for Wirecast is the introduction of next-generation ingress protocols. Clarke mentioned that the industry has relied on RTMP (Real-Time Messaging Protocol) for a long time, but there is a growing anticipation for new protocols that offer lower latency and better overall performance. The adoption of these next-gen protocols will help improve the streaming experience by reducing delays and enhancing the quality of live broadcasts.
One significant upcoming feature is cloud redistribution or the ability for users to send a single stream to the cloud, where it can be redistributed to multiple destinations. This simplifies the streaming process, as users no longer need to manage multiple streams from their local setup.
Vantage
One of Vantage's primary areas of development is improving deployment models, specifically making Vantage more versatile and easier to deploy in various environments. This includes enhancing the cloud integration capabilities, allowing users to seamlessly transition between on-premises and cloud-based workflows.
AI and machine learning will continue to play a significant role in Vantage’s future. Clarke discussed the potential for AI-driven quality-of-life improvements, such as predictive analytics for workflow monitoring. By leveraging machine learning algorithms, Vantage can provide users with real-time insights into workflow performance, identifying potential bottlenecks or issues before they impact operations. This proactive approach will help users maintain optimal workflow efficiency and reduce downtime.