Skip to main content

Stream processing

What if the stream isn't quite how you like it?​

We are going to oversimplify a few things here, mainly to give an introduction that can be followed at all levels of expertise. You can always refer to our glossary if you feel like you need more information.

Each media stream can be explained by 3 categories:

  • Codec: How audio/video data is stored. Note that not every codec fits within every container or protocol.
  • Container: How multiple audio and/or video codecs should be mixed to create a "stream"
  • Protocol: How Codecs (inside their container) are transported over the internet

When MistServer runs under default settings it leaves the codec part of the equation completely untouched and automates most of the decisions you would take for both container and protocol. You can however have full control within each of these 3 steps.

Changing Codecs​

Changing codecs is done through encoding, changing the way the audio/video data is stored. Encoding describes the act of changing this data, while transcoding is specifically for changing already compressed video/audio formats into differently compressed formats. When in doubt, simply call it encoding and you cannot go wrong.

To change codecs you will need to use an encoder, two of the most commonly used encoders are FFmpeg and Gstreamer. There are a few MistProc processes dedicated to encoding which would help you here.

The most common reasons to change encoding are:

  • Bitrate changes
  • Resolution changes
  • Codec changes

Since encoding is a big part of processing we recommend reading up on the encoding introduction and encoding guide.

Changing Container and Protocol​

Within MistServer changing a container or protocol is completely automated and a near free operation. Thanks to our DTSC format we can change any stream container to another upon request. The only requirement is that the encoding/codecs of the stream tracks can fit the requested container format. This allows changes from MP4 to MPEG-TS or HLS to MP4 instantly. This applies to protocols as well and you can easily go from older formats such as RTSP to modern formats such as WebRTC.

Changes on this level is one of the main reasons why MistServer was created. Generally speaking you shouldn't run into any issues with containers or protocols whether this is from legacy to modern formats or the other way around.

There are a few operations that aren't necessarily making changes to a container, but can add/remove tracks from a container. For the ease of understanding we'll place these here as well. Typical changes would include:

  • Changing container format (MP4 to MPEG-TS, MKV, etc)
  • Adding/removing/editing of video, audio, subtitle or meta-data tracks
  • Media gateway operations such as going from RTMP to SRT

This entire process is fully automated and best explained in distribution.

Combining streams​

With MistServer 3.10 MistProcCompose has been added, which allows you to add multiple (raw) video and audio tracks, combine them and rebuild a stream out of them. This can be used for various tasks such as:

  • Picture in Picture
  • Fallback streaming
  • Gridview
  • Watermarking
  • Syncing multiple streams