How to Use Matroska In/Out with GStreamer

In this guide, we’ll cover tips and tricks to improve your encoding experience using Matroska In/Out with your GStreamer scripts and settings. Be warned—the topic is broad, so there’s quite a bit to go through!

This guide contains advanced knowledge, so if you’re confused or get stuck, feel free to contact us.

Requirements

• MistServer installed and running
• GStreamer installed

Steps in This Guide

1. When to Use Encoding
2. Why Matroska In/Out and GStreamer?
3. Setup and Usage
4. Best Practices & Tips
5. Examples
6. Troubleshooting

1. When to Use Encoding

Simply put, whenever you media isn't compatible with your current target or goal. This could be because the quality is too high, or because it is incompatible with your end device.

The two main components of media delivery are the codec (compression method) and the protocol. MistServer covers the protocol part with efficient transmuxing, so you’re set there. However, encoding is needed when you want to change aspects like the resolution, framerate, bitrate, keyframe interval, or codec. This is particularly important when the original media isn’t compatible with your intended use or end user.

warning

Encoding will always impact the quality of your stream. You cannot upscale low-quality video and get high-quality results. In most cases, the best you’ll achieve is matching your input quality.

2. Why Matroska In/Out and GStreamer?

We provide minimal encoding through FFmpeg, but for more customization, you’ll need to write your own script. Matroska In/Out with GStreamer (or FFmpeg) offers more flexibility. While there is no immediate advantage of one tool over the other, this guide will focus on GStreamer.

3. Setup and Usage

If you already have basic knowledge of GStreamer, you should have an easy time here. The core of GStreamer is to first tell it how to access the video, then decode the audio/video, provide the encoder and settings, and finally specify how to push the output.

In this case, the input will always be matroska, and the expected output will also be matroska. This simplifies the chain. We’ll also only be using standard input and standard output.

GStreamer can be somewhat hard to learn at first and may seem overwhelming. We’ll provide a (very limited) explanation here, but we highly recommend reading the GStreamer documentation.

MistServer basics

Encoding is done through stream processes which can be set up when editing or creating a stream. A stream process is a method of telling MistServer that something needs to change with the stream data itself. However since MistServer itself isn't capable of doing so we will need to enlist the help of an encoder.

GStreamer Basics

To encode video or audio, you’ll need to combine elements. Each element has a source (input), filter (options), and sink (output). The entire setup of all elements is called the pipeline.

Each element has a single focus. For example, if you have a video with HEVC video and want to convert it to H264, you will need an element capable of creating H264 and check what it can accept as its source. Unless you have a very specific element, you should assume that video/audio needs to be converted to a raw format before it can be encoded.

Elements are separated by an exclamation point (!), and filter options are space-separated within the element. While GStreamer can automatically determine compatibility between two elements, you can also specify compatibility manually using filters.

Video-Only Example of a GStreamer Pipeline for MistServer

gst-launch-1.0 -q fdsrc ! decodebin ! videoconvert ! ENCODE ! matroskamux ! fdsink

Key points to remember:

• The first source should always be standard input: fdsrc.
• The last sink should always be standard output: fdsink.
• You’ll always need to package your stream into the Matroska format using matroskamux.

decodebin is optional but is a great method for automatically decoding and demuxing the input. It’s like a shortcut to getting raw video/audio. -q removes unnecessary output from GStreamer, which can conflict with the encode if MistServer ingests it expecting Matroska. ENCODE should include the encoder and any additional encoder settings/filters.

Convert Any Video to H264 (Software) Example Using GStreamer

gst-launch-1.0 -q fdsrc ! decodebin ! videoconvert ! x264enc ! matroskamux ! fdsink

Since this is a video-only encode, you will need to set up source selectors to avoid adding audio to the pipeline. To ensure the best quality, we recommend filtering with video=maxbps&audio=none.

Since we didn’t set any x264enc options, it will use defaults: ~2 Mbps bitrate, same resolution, and no bframes.

This is the bare minimum needed to use a GStreamer script with MistServer, but GStreamer can do much more than just converting to H264.

Setting Specific Options (filters) in GStreamer

GStreamer separates parts of its chain with exclamation points (!), and spaces separate options to override the defaults. You can find out the default settings using the following terminal command:

gst-inspect-1.0 ELEMENT

Where ELEMENT can be anything from a specific encoder to a packager. For example, to see what options are available within the x264enc element, you can use:

gst-inspect-1.0 x264enc

Using this command, you would find that the default bitrate is approximately 2 Mbps, which is often too low for full HD outputs. You can set the bitrate to 10 Mbps by specifying the option during the ! x264enc step of the chain:

gst-launch-1.0 -q fdsrc ! decodebin ! videoconvert ! x264enc bitrate=10000 ! matroskamux ! fdsink

If you're inspecting x264enc, you might notice that resolution options are missing. This is because the resolution is inherited from the decode process and passed along. To change the scale, you need to insert the filter immediately after decoding (! decodebin ! videoconvert) and before encoding (! x264enc):

! videoscale ! video/x-raw,width=WIDTH,height=HEIGHT

Hopefully, this provides enough information for you to start experimenting and exploring other options.

4. Best Practices and Tips

Creating your command

To properly create your command, we recommend running it in the terminal without the -q flag. The -q flag suppresses error feedback that could be critical for troubleshooting issues.

Although running directly from MistServer may have lower latency, you can simulate the same conditions using curl to send the Matroska output of MistServer into the script. Attaching a player helps verify the output:

curl http://localhost:8080/STREAMNAME.mkv 2> /dev/null | gst-launch-1.0 fdsrc ! COMMAND ! fdsink | mpv -

For example, to run a 720p video with Opus audio conversion:

curl http://localhost:8080/STREAMNAME.mkv 2> /dev/null | gst-launch-1.0 -q fdsrc ! decodebin name=demux ! videoconvert ! videoscale ! video/x-raw,width=1280,height=720 ! x264enc bitrate=2500 ! queue ! muxer. demux. ! audioconvert ! audioresample ! opusenc inband-fec=true perfect-timestamp=true ! queue ! matroskamux name=muxer ! fdsink | mpv -

You can use other players like ffplay or vlc with the same syntax.

Using ' or " Requires a Script

MistServer does not correctly handle commands that include single (') or double (") quotes. To resolve this, convert your command into a bash script and use the script instead of the direct command.

gst-inspect-1.0

If you encounter errors, it’s likely you’re trying to combine incompatible components. The most reliable way to debug is to inspect the components using gst-inspect-1.0.

Running gst-inspect-1.0 without arguments lists all available components. Adding the component name provides detailed information about its input/output capabilities.

For example, the H264 encoder nvh264enc is incompatible with matroskamux without an intermediary. nvh264enc produces the following h264 output:

    Capabilities:
      video/x-h264
                  width: [ 145, 4096 ]
                 height: [ 49, 4096 ]
              framerate: [ 0/1, 2147483647/1 ]
          stream-format: byte-stream
              alignment: au
                profile: { (string)main, (string)high, (string)high-4:4:4, (string)baseline, (string)constrained-baseline }

Meanwhile, matroskamux accepts:

      video/x-h264
          stream-format: { (string)avc, (string)avc3 }
              alignment: au
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]

Here, the stream-format mismatch (byte-stream vs. avc,avc3) is the issue. Using h264parse bridges the gap, as it supports both formats:

    Capabilities:
      video/x-h264
                 parsed: true
          stream-format: { (string)avc, (string)avc3, (string)byte-stream }
              alignment: { (string)au, (string)nal }

Adding ! h264parse gives GStreamer a method to fix the incompatibility.

Cluster Durations for matroskamux

By default, GStreamer estimates the cluster size, which may vary. For live streams, smaller cluster sizes improve latency. You can set this explicitly:

gst-launch-1.0 -q fdsrc ! decodebin ! videoconvert ! x264enc ! matroskamux min-cluster-duration=100000000 max-cluster-duration=500000000 ! fdsink

This sets the cluster size between 0.1 and 0.5 seconds, ensuring output starts within this range and is sensible for a live stream.

Stream masks to avoid creating unnecessary qualities

MistServer has masks, inhibitors and selectors in order to make an automatic decision on whether a stream process should start and what it can be used for.

masks these determine whether the source (input) or output are hidden from certain processes within MistServer after the encode has started. Using this you can make sure to not re-use an already encoded track by another stream process or avoid viewers watching the source quality of a track.

inhibitors these decide when a process should not start. You could for example have it check for the codec h264 and not run if it's present. Or only start a process if the resolution is above 1280x720.

selectors these determine what tracks are passed to the encoder. You can specifically remove video or audio to avoid having to deal with it within the encoder. Or you can select a specific codec, resolution or bitrate to grab when there's multiple. Usually maxbps is a fairly good selector to get the highest quality.

5. Examples

Encode Anything to H264 Video

gst-launch-1.0 -q fdsrc ! decodebin ! videoconvert ! x264enc ! matroskamux min-cluster-duration=100000000 max-cluster-duration=500000000 ! fdsink

tip

Add the bitrate option to x264enc, as the default is only ~2 Mbps.

Recommended settings:

• source track mask: Any
• output track mask: Viewer and pushes tasks (not processes)
• track inhibitor: video=h264
• track selector: video=maxbps&audio=none

Encoding Video (H264 720p)

gst-launch-1.0 -q fdsrc ! decodebin ! videoconvert ! videoscale ! video/x-raw,width=1280,height=720 ! x264enc bitrate=10000 ! matroskamux min-cluster-duration=100000000 max-cluster-duration=500000000 ! fdsink

tip

Scaling should be done at the raw video step, not during encoding.

Recommended settings:

• source track mask: Any
• output track mask: Viewer and pushes tasks (not processes)
• track inhibitor: video=h264,|<1280x720
• track selector: video=maxbps&audio=none

Encoding Audio

Audio to Opus

gst-launch-1.0 -q fdsrc ! decodebin ! audioconvert ! audioresample ! opusenc inband-fec=true perfect-timestamp=true ! matroskamux ! fdsink

tip

we always recommend adding the forward error correction for OPUS through inband-fec=true. It adds a noticeable improvement.

Recommended settings:

• source track mask: Any
• output track mask: Viewer and pushes tasks (not processes)
• track inhibitor: audio=opus
• track selector: audio=maxbps&video=none

Audio to AAC

gst-launch-1.0 -q fdsrc ! decodebin ! audioconvert ! audioresample ! faac perfect-timestamp=true ! matroskamux ! fdsink

tip

Pay attention to encoder names, as they may not follow consistent naming schemes.

Recommended settings:

• source track mask: Any
• output track mask: Viewer and pushes tasks (not processes)
• track inhibitor: audio=aac
• track selector: audio=maxbps&video=none

Using Hardware Encoding

NVIDIA NVENC

NVIDIA's nvenc can be used with nvh264enc (encoding) and nvh264dec (decoding). For nvh264enc we recommend using the ultra-low-latency - High Performance preset, it gives a good balance between quality and low latency:

gst-launch-1.0 -q fdsrc ! decodebin ! videoconvert ! nvh264enc preset=5 bitrate=10000 ! h264parse ! matroskamux min-cluster-duration=100000000 max-cluster-duration=500000000 ! fdsink

Recommended settings:

• source track mask: Any
• output track mask: Viewer and pushes tasks (not processes)
• track inhibitor: video=h264
• track selector: video=maxbps&audio=none

Encode H264 720p and Opus in One Command

gst-launch-1.0 -q fdsrc ! decodebin name=demux ! videoconvert ! videoscale ! video/x-raw,width=1280,height=720 ! x264enc bitrate=5000 ! queue ! muxer. demux. ! audioconvert ! audioresample ! opusenc inband-fec=true perfect-timestamp=true ! queue ! matroskamux name=muxer ! fdsink

Recommended settings:

• source track mask: Any
• output track mask: Viewer and pushes tasks (not processes)
• track inhibitor: None
• track selector: video=maxbps&audio=maxbps

This command processes both audio and video together, which normally would ensure sync, however starting a process through MistServer already covers that. It still makes it easier to track the resource usage, however you do lose some flexibility in when you "should" run this process. This most likely gets the best use from running every time.

6. Troubleshooting

tip

Remove the -q flag from your GStreamer command to view debug information.

Pre-roll and Pipeline Errors

These errors typically indicate incompatible elements in the pipeline. Simplify the script to isolate the issue and remove filters to identify where the problem starts.

Use Generated Sources for Debugging

GStreamer provides test sources like videotestsrc and audiotestsrc for debugging. We recommend using these to simplify the setup if you get stuck. It helps tremendously knowing that the source isn't the problem.