Streaming with more than one PC has been the leader in H.264 encoding for years, but NVIDIAs Turing and Ampere generation has put a significant dent into that lead. The new generation of GPUs with the brand new encoder brought comparable quality x264 medium – if you can find a GPU that is. Let’s take a look at what’s needed to set up your stream for massively improved quality.
The guide has been updated for:
StreamFX v0.10 and OBS Studio 27.0
Setting up NVENC (for Streaming)
Modern OBS Studio has two ways to achieve the expected quality: the built-in NVENC H.264 (new) and the addition from StreamFX called NVIDIA NVENC H.264/AVC (via FFmpeg). Both Options can achieve similar quality to x264 medium, but the latter is able to exceed that and rival x264 medium/slow in various situations. Whichever you pick, both of them support zero-copy encoding, and they’re both valid options for streaming.
Built-In: OBS Studio NVENC H.264 (new)
The built-in NVENC option in OBS Studio is by far the simplest option and will give you almost identical quality on Maxwell, Pascal, Turing and Ampere, though Turing and Ampere will make use of the new improvements of the NVENC chip. Maxwell and Pascal users can expect to reach x264 veryfast/faster-like quality, while Turing and Ampere users can expect to hit fast/medium-like quality. Below are the settings you need to set:
|Preset||Quality||Max Quality or Quality|
|Psycho Visual Tuning||Enabled/Checked||Enabled/Checked|
|Max B-Frames||1 or 2||3 or 4|
StreamFX: NVIDIA NVENC H.264/AVC (via FFmpeg)
If you’re new to StreamFX’s NVENC integration, it will most likely overwhelm you with the settings it offers. But thanks to all those settings, you can actually go above the default quality by quite a significant amount. Note that I will only cover critical settings, as other settings like Bitrate, Buffer Size and Key Frame Interval are explained elsewhere.
|Preset||High Quality||High Quality|
|Rate Control Options|
|► Mode||High Quality|
|► Two Pass||Disabled||Enabled|
|► Look Ahead||8||Any value between 16 and 32|
|► Adaptive I-Frames||Disabled||Disabled|
|► Adaptive B-Frames||Enabled||Enabled|
|► Spatial (AQ)||Enabled||Enabled|
|► Spatial (AQ) Strength||<Any>||<Any>|
|► Temporal (AQ)||Enabled||Enabled|
|► Maximum B-Frames||1 or 2||3 or 4|
|► B-Frame References||“middle”||“middle”|
|► Zero Latency||Disabled||Disabled|
|► Weighted Prediction||Disabled||Disabled|
|► Non-reference P-Frames||Enabled||Enabled|
|FFmpeg Options (Optional)|
|► Custom Settings||-refs=#|
The correct value for
-refs= depends on the H.264 Level you have chosen and while higher levels do have more reference frames available to them, the devices supporting this are limited in number. In order to target H.264 Profile Level 4.2, you would have to set
-refs=4 at 1920×1080,
-refs=6 at 1600×900, and
-refs=9 at 1280×720. It is possible to calculate this value with the data and formulas provided in “H.264 Annex A.3.1”.
Setting up Resolution and Framerate to match the Bitrate
It is no secret than H.264/AVC is an outdated codec and that platforms should have paved the way for better codecs a long time ago, but it is the solution that we are stuck with until AV1 is adopted by the masses. So in order to get the best quality out of our stream, we should aim to also set up our stream according to the bitrate and codec we use. Below is the average result of a few hundred thousand tests at various resolutions and bitrates, according to PSNR and VMAF (weighted PSNR 30:70 VMAF):
A 10 is perfect, 9 is near lossless, 8 is indistinguishable, 7 is high quality, 5 is acceptable quality and 3 is watchable.
Please note that watchable in video encoding means that you can decode information within it with reasonable accuracy, instead of it having turned to full garbage. Higher resolutions than 1920×1080 were omitted from the table as the rows would be filled with values between 0 and 1, which just are not very useful to us.
This means that at 3.5mbit, the highest resolution and framerate for a variety streamer is 1280×720 at 30 FPS, or 960×540 at 60 FPS. The equation shifts slightly for 6.0mbit, where you can either go for 1536×864 at 30 FPS or 1280×720 at 60 FPS. Finally at 8.5mbit you are looking at a maximum resolution and framerate of 1920×1080 at 30 FPS or 1536×864 at 60 FPS.
In the past few years NVIDIA has made massive improvements to their encoder, which has evened the playing field far beyond what was expected. With no need to transfer frames from the GPU to the CPU, and quality comparable to x264 medium (or better), NVIDIAs Turing NVENC is pushing the boundaries of what is possible in a single consumer PC.
Whether you use it or not is entirely up to you however. If you already have a working Dual-PC setup that can achieve x264 medium (or better) quality, then you don’t gain much from moving to Turing NVENC. But if you’re currently stuck on anything below x264 medium, or have a Turing GPU ready to test it out – why not give it a shot?