Online round table “Broadcasting 2021. Video Compression Systems” from the 13th of July 2021. Christoph Jurkuhn, Head of Product and Presales at Hiscale, and Jiri Janiga, Sales Director, with their presentation “Dockerized transcoding for independent and scalable platform management”.
Moderator – Timur Kulgarin, technical director of CTC Media.
Jiri Janiga: My name is Jiri Janiga. I left ChyronHego a few months ago. When deciding where to go, Hiscale approached me. I didn’t know them, because it is quite a new and small company. When I saw their products and customers – including Fox, France TV, and big German customers – I decided that this was the company I wanted to work for. I am really happy that we can show you our solutions. They are still not in Russia, but we are approaching Russian companies to begin our cooperation.
Christoph Jurkuhn: A few words about our company. My name is Christoph Jurkuhn, and I am the Head of Product and Presales at Hiscale. We are a software development company based in Germany, and we focus on building software solutions for the broadcast and media industry. We have been around since 2014 doing scalable media products. We have a couple of different product families that we support – the main topic today will be our transcoding product family. They are called FLICS. We also have a workflow orchestration product called ‘Jobs’. For us, automation workflow and orchestration are separate from transcoding because some customers already have workflow engines and only need transcoding, or vice-versa. While these are two different products, they of course go together well.
We are a relatively new company based in Germany, so most of the customers we work with are in the German-speaking market. This includes the German public broadcasters, private broadcasts, production companies. Last year we started to grow outside of the German-speaking market. For example, we have a big installation at Fox in the US, and France TV is using our transcoding system. As well as this, we are also growing in Eastern Europe.
We are not as well-known in the Russian-speaking market, so I want to go over why we started this company.
We come from media asset management – we worked on enterprise media asset management projects which always involve transcoding. So, we have worked on and integrated transcoders from other vendors. We are familiar with the process of customer projects when it comes to transcoding. This is where we got started: enterprise customer projects were always struggling with purchasing. People were not sure how much to purchase to have enough resources for worst-case scenarios, but not to spend too much money. It is always difficult to balance this.
At that time, everything was still done on-premises. Now, in 2021, you can solve scaling issues with the cloud. However, this comes with its own downsides. Not everyone wants to go with cloud processing – it doesn’t always make sense. For example, you may be dealing with high bandwidth production content which would take a long time to transfer. It also involves some cost and depends on what the transcoding requirements are.
This is where we started six years ago. We thought that, for the typical customer we work with, it would be most beneficial to have a hybrid system. This is how we started developing the product which is now called ‘FLICS’, an elastic transcoding cluster.
This is built from the ground up on cloud-native technology. It is built in Docker, for example, for the deployment part. This makes it fairly platform-independent. You can deploy the system on standard hardware in your data center, in virtual environments as well as cloud environments. With this, you can also build hybrid systems. We have customers who work only on-premise, some who work only on the cloud, and some that can build hybrid systems with system resources on-premises in their data center for their average use. The Tokyo Olympic Games are coming up, and for some of our customers, this is a peak during their production because there is much more to transcode. This system can automatically, based on the system load, scale in a public cloud. It can detect if there are a lot of things to do, spin up a machine, and deploy the containers. This is all automated, so there is no manual interaction needed. It can also spin them down if they are not needed anymore.
We orchestrate the cloud platform or the platform the customer provides. We don’t provide a cloud service that we host and are paid for. The customer brings in the hardware, virtual environments, or cloud subscription. Of course, the solution generally takes care of managing all of the infrastructure as well as the integration capability. The product is a transcoder, so it is built with Restful API interface for easy integration. There is existing integration from companies like Avid or IBM, so this is already working in various customer sites and projects.
With transcoders, everyone needs to do the same sorts of things. I think what can make the difference is the way you can operate and orchestrate. Before we look at this in a live demo, I want to talk about what we address with our transcoding product.
You could put what we address into four different use cases. First, we address typical broadcast workflows – essentially file transcoding use cases. You get something in, and you transcode it to your HiRes formats, Proxy formats, or other things you might need to do in file-based workflows. This may include stitching of content, extraction of pieces of video, KeyFrame generation, and much more.
Secondly, with the same transcoder and the same containers deployed, you can also change from a file source to a live IP-based source. This is what we call ‘live ingest’. You can use SRT or transport streams as a source from a live signal feed that you receive. With the same transcoder, you could then generate your HiRes and Proxy formats and then use them for editing. For example, we can spin up a machine in a cloud and receive the SRT live signal. Then, in parallel, we record and generate a low-resolution and high-resolution format. While the low resolution gets checked into the MAM immediately, it will be available for HiRes editing at the same time. So, while recording, you can start your workflows.
As well as this, with the same transcoder, we support video-on-demand workflows. We include a packager to generate, but customers can also use an external packager that we can integrate. 
The latest addition to the transcoding product is a live transcoder. This is, again, the same transcoder that we deploy, just with a different license. We can do live encoding for events and 24/7 streaming.
We have a transcoding pipeline that looks the same for most transcoding vendors. In the beginning, you decode, then you can do processing, such as manipulation of the images. This also includes things like frame-rate conversion. We work with a partner doing motion-compensated frame rate conversion in the UK. Their product has high-quality conversion and has been chosen for the Olympic Games by the German public broadcasters. They have tested seven different solutions and chosen FLICS with this British FrameFormer plugin to do the conversion for the Tokyo Games. This is available for file encoding as well as live encoding. This is interesting because as a software-based product, we can instantly spin up a machine with live frame rate conversion, which used to be really expensive hardware that needed to be physically installed. After this, you do the encoding to a certain format that you require. You can then do packaging, or simply deliver the file somewhere.
This is not something where you can set yourself apart as a transcoding company – everyone needs to stick to the latest trends and keep up and adapt to new formats. When we deploy a transcoding container, it contains this pipeline. This is also what we license.
We also have the management application orchestrating all of this. As a customer, you don’t need to be aware of Docker containers and where they are – we build the management application. There are different ways you could purchase this, we call it ‘FLICS SOLO’. This is a standalone deployment on one machine. You can install it for easier use cases or workloads that are not as high. You can also install it as a distributed system – a bigger system with multiple transcoding nodes; this is what we call ‘FLICS Cluster’.
This is also available as a live encoder, ‘FLICS LIVE’, and a special edition called FrameFormer Live, with a motion-compensated live frame rate conversion. It is available with the same options as a SOLO product – one box deployment as well as a clustered solution, where you can dynamically scale and spin up machines.
Now, let’s look at what it actually looks like.
We are deployed on Linux machines, so the Docker containers are deployed on Linux servers. You are looking at a FLICS Cluster system right now, what we call a ‘cluster view’. This is where you see your current system status. My system here is configured – I have a backend management machine running user interface, Restful API, and so on. The green boxes we call a ‘pool’, this is essentially configuration. You configure resources on the same location with the same hardware specifications. We support Intel, as well as AMD. We also support GPU acceleration and other acceleration features.
In my system, I have a hybrid configuration. I have a pool I call ‘local’, with two machines, and you can see some transcoding going on here. I also have a pool called ‘cloud’, where currently nothing is happening. There are no machines attached, meaning there is no cost at this stage. If I go to the dashboard and I look at my current system usage, I see that the blue line – active transcoding at this moment – vs the red line – how much is in the queue. If I force the queue to go up, I am using Restful API to insert new tasks. This could also be someone coming in throwing in tasks from the hard drive.
A lot of transcoding requests means that the queue goes up. What will my system do? I have a table view, I have a filter showing me everything that is running, failed, etc. I can look at the details of a transcode, I can cancel it, I can download the log file and look at the logs. If I have a long queue, and only two machines on-premise, my system will fill up the resources that it has. In a minute, it will start spinning up machines in the cloud after realizing that this is not enough.
On the transcoding side, you configure what is typical for transcoders with various different settings. You configure what we call a template, where you find the entire workflow. You lay out the kind of profile you want to create, the target destination, the filing pattern. You can also apply filters, such as subtitle insertion. For VOD outputs, you also create a template, but you configure the renditions right in the template. You do a similar thing for audio.
It looks fairly similar for the live product. It is the same product with the same deployment, just a different license. There are also different views, such as a channel-based view. The focus of the live product is a lot more on monitoring. What happens if the incoming stream dies? You can switch to an alternative source, new playback, etc. What happens if my frame rate changes? I want to be notified. All of our products can be integrated for monitoring. You can also send a message to a Slack or Teams channel, for example, for monitoring purposes.
As mentioned before, when it comes to file transcoding, you typically need to have some logic before transcoding. You would analyze the file and decide what you need to do. You may already have a media asset management workflow engine that you can use and then use FLICS as a transcoder. Alternatively, you can choose our orchestration tool JOBS, and build workflows that orchestrate your daily file-based operations. It comes with a BPMN-based workflow editor which has integration to standard products from the broadcast industry, such as Baton for quality control. It also includes different transcoders as well as notifications.
The difference here compared to most solutions doing workflows is the capability of customization. This is a product – we do not want to spend years building workflows with customers. The customers can get training and build the workflows themselves. They can even develop things themselves to integrate into other systems. For example, you can start by writing JavaScript code right from a workflow. Anything you can do with a Linux terminal you can do from a workflow. If you have developers, you can even have them develop integrations with Java. This is highly customizable, but we of course help customers when they do projects. We also help customers with development, if this is required. But you are not bound to that and can do it yourself. This is the real focus here.
[pdf id=15770]
We have done quite a few installations that we can show you. FLICS is being used as a central transcoding hub for the German broadcasters for Tokyo Olympic Games content. Everything they retrieve from archives, they must convert before they do the editing. The same goes for the other way around, anything they receive from Tokyo or post-production needs to be converted before it gets aired, which they do with FLICS.
You can build many other applications as well. There are combinations with JOBS and FLICS – for example, we have built Mobile Ingest for a German public broadcaster. They bought 40 iPhones for their journalists to shoot video and need an easy-to-use solution. A journalist can upload the files, and they will find them in their media asset management. With iPhones, you can do a lot of different things, so there can be a lot of variants that come out. We have built a little UI – they can enter media data, but the actual magic is that the job’s workflow gets triggered by the UI. After an upload of one or multiple files, it will analyze all the files and detect what needs to be done. It does temporary transcoding and then starts the HiRes transcoding for the house format. Journalists can look for the data they have entered on their mobile device and find it in their media asset management. This sounds easy, but in detail, there are lots of things you have to keep in mind. This is one of the two sample use cases we have done, with a combination of our reorchestration projects which includes file analysis and integrates transcoders and our application FLICS.
Timur Kulgarin: TV companies will be interested in input interfaces, to know if you support HD-SDI, if your systems fit with cable if you support SMPTE 2022, or 2021-10. Do you support this?
Christoph Jurkuhn: As I mentioned, the live product is the newest edition to our transcoding portfolio, which was actually released at the end of last year. We are two weeks away from the second major release, which will come to include things that the customers have been asking such as subtitle support. To your question, as of now, we support IP-based protocols, and we have started integration for two hardware vendors for SDI cards. This will not be in the next release, but it will be available in the next version, around the Fall. We will integrate SDI cards for HD-SDI, and in the same release, we will also have support for SMPTE 2110. NDI is also something that we have integrated already, again this will be the major input format release.
Timur Kulgarin: Is RIST protocol in your plans as well?
Christoph Jurkuhn: This is obviously on our roadmap, and we are looking closely at what is happening there. I have to admit, I have heard a lot of talk about RIST, but I haven’t yet met a customer who actually uses it. 2110, SDI, and NDI are the focus points for us. If we have time left, we will look at RIST.
Timur Kulgarin: From what I understand, subtitles and closed captioning are also on your roadmap. So, they are not supported yet but will be soon?
Christoph Jurkuhn: Yes, we are building the release as we speak, it will be available in two weeks.
Timur Kulgarin: Can you tell us about ways to monitor your system using external monitoring software?
Christoph Jurkuhn: We have several ways of integrating monitoring. We use a Message Bus solution, so anyone could integrate into this. Of course, we have a Restful API interface that can be used for monitoring purposes. As I mentioned, based on events that happen in the system that you can configure, you can actually send or create events. Then, you can send emails to monitoring systems. Typically, this is different from customer to customer, depending on their focus. We have an alarming mechanism – you can create alarms based on anything we have available. You can send a custom message and apply an alarm to any live channel. When the alarm goes off, you can have an event in your system that can trigger something like an email. It is highly flexible and requires some configuration of course. But again, different customers have different requirements, so we make this as flexible as possible.
Ilya Doronin, Diantis System House, Canada: There are several German, Austrian and Israeli companies that are working on similar solutions. What are the advantages that make you different?
Timur Kulgarin: Generally, we have many different competitors in different areas. We see ourselves as providing a solution that the customer can own. Though, this isn’t a necessity – we also have subscription models, so you can pay as you go. It is more flexible in terms of deployment, so you can have it in your own hands. It is also more flexible in terms of licensing compared to competitor products. This is where we see ourselves – our product is highly integratable, a new way of doing things. This is not a legacy product that we are trying to push into the cloud, this has been built from the ground up.
Obviously, if you only need cloud transcoding and if you are fine with H.264 output, you can find a long list of transcoders. In our case, this is our own transcoder. We worked with coding developers that have dedicated resources for us, as well as native integrators with which we have contracts.
If you are using someone who pushes FFmpeg as the main transcoder, this is fine up to a certain level. Especially with broadcast formats, we see that there are always bits and pieces that need to be different, that need to be adapted. Typically, it is really hard to do this in FFmpeg and find the right people who can do this. This is also an advantage in terms of the cloud service – we actually have the resources and the people who can change the transcoding pipeline and adapt something if it is needed.
