What should you know about streaming
protocol describes the way computers can communicate with each other. This is used in streaming media to create a connection between servers and viewers. Media itself is in a
container format, which describes how media is packaged. Then Within those packages are
Codecs which actually describe how the media can be used. The streaming methods described below are combinations of these three that will be able to fit in every type of media delivery.
To keep things as basic as can be there's 3 methods of handling media data that you want to remember and almost every method of delivering media will fit in these categories.
Real time streaming
Stream based protocols are true streaming protocols. It has two-way communication between the server and the client and maintain an open connection. This allows for faster, more efficient delivery and lower latency. These properties make it the best option, if available. On the downside you need a player that supports the protocol, which often makes it unavailable for browsers.
This is a method where you have a HTTP(S) server and you pretend all streams are files. However it's not always a file, it could actually be a live stream and you're just sending it pretending there’s a file. You lose out on latency, but gain a lot of compatibility as most players will be able to use this as long as the
container format is supported.
This is the current most popular method. Here you cut up a stream in several parts and you have an index that shows how to reach every part. That index updates when new parts become available. As long as the player knows how to make sense of the index it should be able to play the stream and an advantage is that due to the nature of how it works trick play options (seeking, reverse playback, fast forward, etc.) is easier to do.
Protocol, connection type and Container format are a little blurry
This is something to keep in mind,
Connection type and
Container format are related and influence each other. Certain combinations are not allowed, or required. Since this is an introductionary article we'll simplify things a little bit, in reality things are not so black and white.
Then for the transport types you also have several options. The two big ones that everything is based on are:
- User Datagram Protocol (UDP)
- Transmission Control Protocol (TCP)
Then we have the transport types that build onto TCP and UDP:
- Hyper Text Transport Protocol (HTTP(S))
- Secure Reliable Transport (SRT)
- Reliable Internet Stream Transport (RIST)
Note: "Protocol" in the cases above does not refer to a protocol in the same context as discussed in the rest of the topic. UDP, TCP and HTTP are transport protocols, while in most of the article we're talking about streaming protocols.
What you need to know about these transport types:
UDP is a type of connection where one side is listening and anyone can send towards it. There's no form of handshake or control, what you receive is what you get. Using this method packets tend to go missing or appear in the wrong order for all intends and purposes this is very much a fire and forget connection. The big benefit is that it's fast, but will be unreliable if your connection isn't good.
These are UDP based transports. They attempt to "fix" the packet loss and packet reordering issues that UDP normally has in various ways.
TCP is a connection where both sides are aware of each other. First a handshake is made and both sides know exactly what is send, needs to be send and what the order is. This is a slower type of connection as almost everything gets double-checked, but you're guaranteed that everything comes in as you want.
HTTP is the transport most commonly known for serving webpages. It has gone through several modernisation updates, currently the latest version is HTTP3, but HTTP1.1 is still widely used. HTTP versions under 3 use TCP as their underlying transport, while HTTP3 uses UDP as its underlying transport. This means the exact behaviour and pros and cons are dependant on the version and specific device in use.
Summing up the methods and connection types:
|Streaming Method||Latency||Trick play||Player support|
|Real time streaming||Low||Yes||Custom players|
|Pseudo streaming||Medium||No||Native playback|
|Segmented streaming||High||Yes||Native playback|
|Connection type||Latency||packet loss||Reliable|
|UDP||Low||Yes||Only in good condition|
|TCP||Medium||No||In almost all conditions|
So what should I use?
This is the obvious question to ask, and really the answer is any of these are a good option. You should pick the method that best fits your or your users needs.
Ultra low latency
The best latency will always be a
Real time streaming over
UDP combination. There will always be two challenges with this combination however:
- Bad connections will be problematic
- A custom player is almost always necessary
Therefore we recommend having a
Pseudo streaming or
Segmented streaming as option to fall back towards.
For reliability you mostly want to look at options using
TCP. That's not to say that
UDP cannot be reliable, it's just easier to achieve when using
TCP. This is especially true when you're talking about how to reach your end users.
When we are talking between server traffic you can generally assume the connection is good enough to use
UDP, which allows you to share media quickly between servers and do
TCP for the part of the connection where the public internet tends to become more unreliable.
Where the protocols that MistServer supports line up
|Common name||Streaming method||Transport type||Container||Where||Expected Latency|
|RTMP||Real time streaming||TCP||Flash Video||To servers||< 3s|
|WebRTC||Real time streaming||UDP||RTP||Browsers||< 1s|
|RTSP||Real time streaming||UDP and/or TCP||RTP||Custom players, servers||< 1s|
|SRT||Real time streaming||SRT||MPEG2-TS||to servers||< 3s|
|RIST||Real time streaming||RIST||MPEG2-TS||to servers||< 3s|
|MPEG-TS||It's complicated||It's complicated||MPEG2-TS||It's complicated||It's complicated|
|MP4 (MPEG4)||Pseudo streaming||HTTP||MP4||Browsers, players||< 5s|
|WS/MP4||Pseudo streaming||HTTP||MP4||Browsers||< 3s|
|MKV / WebM / EBML||Pseudo streaming||HTTP||Matroska||Storage, Edge, players||< 5s|
|FLV||Pseudo streaming||HTTP||Flash Video||Legacy||< 5s|
|HLS||Segmented streaming||HTTP||MPEG2-TS / CMAF||Apple, scripted players||< 30s|
|LL-HLS||Segmented streaming||HTTP||CMAF||Apple, scripted players||< 5s|
|HSS||Segmented streaming||HTTP||CMAF||Legacy||< 30s|
|DASH||Segmented streaming||HTTP||MPEG2-TS / CMAF||Browsers, non-Apple||< 30s|
|HDS||Segmented streaming||HTTP||Flash Video||Legacy||< 30s|
|MP3||Pseudo streaming||HTTP||MP3||Browsers, Players||< 5s|