x264 is the open-source software H.264 encoder built into OBS — it compresses your stream using the CPU rather than dedicated GPU hardware.
x264 is the reference software implementation of H.264 and the default encoder in OBS. It runs entirely on the CPU, which makes it universally available — no specific GPU required — but also means it competes with your game for processor time.
x264 exposes a speed/quality dial: ultrafast, superfast, veryfast, faster, fast, medium, slow. Slower presets spend more CPU cycles per frame and produce better quality at the same bitrate. veryfast is the practical default for single-PC streaming; anything slower usually starves the game.
As a rough guide, 1080p60 at veryfast wants a modern 8-core CPU with the game running. 720p60 is far more forgiving. If OBS reports "Encoding overloaded", the CPU could not finish frames in time — step the preset faster, drop the frame rate, or move to hardware encoding.
On an RTX card, NVENC matches x264 medium while using almost no CPU, so it is the better choice for most people. x264 still makes sense on a dedicated encoding PC, on AMD systems with many idle cores, or at low bitrates where slower presets genuinely extract more detail per bit.
Most "my game stutters when I stream" reports come from x264 at a preset the CPU cannot sustain. Understanding the preset dial lets you trade quality for headroom deliberately rather than by accident.
veryfast for single-PC streaming at 1080p, faster or fast if you have CPU headroom, superfast if OBS reports encoding overload. Presets slower than fast are only realistic on a dedicated encoding machine.
At the same preset level, x264 medium and modern NVENC are close. But single-PC streamers cannot run x264 medium while gaming, so in practice NVENC on an RTX card delivers better real-world quality.
Because the CPU could not compress a frame before the next one arrived. Lower the preset, resolution or frame rate, close background applications, or switch to a hardware encoder.