February 1, 2012
By Jan Ozer Contributing Editor
Featured Articles
For the rest of the Winter 2011 issue of Streaming Media magazine please click here

Quality Control Roundup: Catch the Mistakes Before they Go Live

Action templates include the ability to trigger emails at the start of the job, at the completion of the job, or upon job failure and the ability to copy or move a file upon success or error. Or you can create a script that incorporates any or all of the communications or file-transfer functions defined previously.

Cerify’s reporting includes web-based See and Solve reports (Figure 6), which let you drill down into thumbnails of problem regions, plus machine-parsable XML files for your encoding or MAM systems.

Rhozet Workflow System 1.4

The Rhozet Workflow System is an encoding/QC system with quality control functionality provided by the tightly integrated Rhozet Quality Control System. Like other QC systems,

Carbon breaks its quality control tasks into audio, video, and container evaluations. Within these categories are comparative tests, such as SSIM and PSNR, and single-test quality measures, most notably for blockiness, letterbox/pillar-box, black frame detection for video, and levels and silence detection for audio.

In terms of operation, the basic building block is a job, which links a workflow with either a file or a watch folder. A work-flow consolidates pretransform tasks, transforms, and job success and failure tasks and is created in the workflow editor (Figure 2). You create presets for quality control, filtering, and encoding in a separate presets editor. While this may sound disjointed, it’s not: In operation I found the process simple and elegant.

Figure 2 shows a workflow with pre-encoding and post-encoding tasks. In the figure, I’m checking the incoming file for resolution, letterboxing, and black frames, sending any nonconforming file to a reject bin with email notification. Then, I’m encoding to a single H.264 preset and testing the encoded file for drop frames and blockiness and making sure it meets my minimum standards for SSIM and PSNR.

Failures here are sent to the separate Rhozet QC Manager for review (Figure 7). From there, you can elect to approve or reject the file, escalate the issue to another reviewer, and add a comment. There’s also a separate interface that lets you play the video file itself in real time, with all errors prominently marked and detailed, and a PDF report with descriptions and thumbnails to further document the problems.

In version 1.4, which I reviewed, Rhozet can’t reroute a failed file for re-encoding at a higher rate, one of the most prominent features of Telestream Vantage’s “self-healing” workflow. However, the product manager informed me that cascading workflows, a feature that would allow such a process, were on the published road map for version 1.5, which was discussed at the IBC show in September 2011.

One of Rhozet’s traditional strengths has been outstanding performance on multiple-core workstations, and that remains true with the quality control functionality, which can check multiple files simultaneously when checking pre-encoding or post-encoding. From a reporting perspective, WFS integrates with the Microsoft SQL Enterprise database and can send email, invoke web scripts, or make command-line calls at any stage of the workflow.

To purchase all the functionality discussed here, you’d have to buy the Rhozet Workflow System ($20,000) plus separate versions of ProMedia Carbon ($5,995) and the Rhozet Quality Control System ($5,995). In the first year, there’s also a 15% software maintenance con- tract for support and updates.

Interra Baton Version 3.2.1

Baton is a stand-alone, file-based quality control/automated content verification pro- gram. Unfortunately, Interra doesn’t publish its pricing, so we can’t share that with you. Like most QC systems, you can drive operation directly, via watch folders or via an XML- RPC API supported by a 246-page manual.

You run Baton via a browser-based interface. The building blocks of Baton verification projects are content locations, test plans, and alerts. First, you register a content location, which can be a local or a network folder or an FTP or an AMP server. After registering a content location, you can then set it up as a watch folder or access it for manual or automated operation.

Test plans contain the verification tests to be performed, and you can use supplied templates or create your own by choosing the container and audio and video formats you intend to test in a wizard-based workflow. As shown in Figure 4, you can test the audio, video, and file container separately using three categories of tests for format-independent features, quality features, and format-specific features.

Video quality tests are particularly extensive, and they include searching for comb artifacts, blockiness, blurriness, pixelation, freeze frames, flashy video, and much more. Audio-related quality tests include the usual silence, clipping, and loudness, but Baton can also search for transient noise, wow and flutter, misplaced channels, and jitter noise.

When you run a job in Baton manually, the source files are not moved after testing, so you have to do this manually. However, when you set up a watch folder, you choose both a quarantine folder and a playout folder for files that fail or pass inspection. The processes of creating a job and a watch folder are both wizard-based, which simplifies operation. After setting up test plans or watch folders, you can create alerts for either folder to complete the information flow.

System reporting is extensive, with PDF, XML, and a unique multimedia report that you can play in the Baton Media Player (Figure 5). As you can see, the player lets you examine the issues found during analysis in the actual media file itself, so you can very quickly gauge their severity.

Telestream Vantage Version 3.0.29.73950

Of the four programs reviewed, Vantage was voted “most likely to require a programmer to operate” by our panel of judges, which, of course, included only me. This isn’t necessarily a bad thing, but a simple reminder of Robert Heinlein’s TANSTAAFL theory, which stands for “there ain’t no such thing as a free lunch.” From a workflow perspective, Vantage appears exceptionally functional, but the likely price is a few hours to understand how it all works and fits together, though the investment should prove well worth it for even mod- erately complicated workflows.

With Vantage, you design your workflows using the drag-and-drop components shown in Figure 8. For basic operations, such as grab this file, encode it, and move it there, operation is very straightforward. On the other hand, when creating if-then-type analyses, you have to use multiple components such as Examine, Decide, and Deploy, which means coming to grips with concepts such as generating variables and setting a workflow state, which aren’t exactly intuitive for right-brain dominant compression wonks such as myself.

OK, enough about ease of use. From a QC perspective, Vantage offers three analysis tools, Examine, Compare, and Identify, which you can insert at any point in the workflow or in multiple points. Compare’s sole function is to compare the encoded file to the original file to compute PSNR. The Examine tools, which are shown in Figure 8, incorporate macroblocking detection, which is the primary video-quality metric besides PSNR, as well as detecting curtains, black frames, and periods of silence in the audio. The Identify category includes variables such as Extract Media Properties, which lets you identify characteristics such as resolution, aspect ratio, codec, bitrate, and the like.

As mentioned earlier, you can drive different workflow decisions from the information derived from the analysis. For example, if the system detects excessive blockiness, you could re-encode the original file at a higher data rate. This is the “self-healing” that Telestream mentions in the Vantage product literature, and it’s currently unique to Vantage in the products that I tested.

After designing your workflow, you activate it to take it live. While processing live, Vantage shows color-coded job status (see Figure 9), with blue tasks completed, green tasks in process, and yellow tasks in waiting. Beyond this visual information, Vantage reports can be published to web services, as metadata in media files, or exported as XML. As mentioned in the body of the article, Vantage operators can set triggers for operator review of any file flagged by the QC review at any point in the worfklow.

To acquire the functionality discussed, you’ll need Vantage Workflow ($8,500 per server), Vantage Transcode ($5,500 per server), and Vantage Analysis ($7,500 per server).