When you talk about “watching a video,” you’re probably thinking about a combination of one video and one audio stream. They are not two different files, you just have the video file. It could be an AVI file, MP4 file or another kind of video codec. These are just container formats, like a ZIP file that hold multiple kinds of files within it. The container format defines how to store the video and audio streams in a single file.
When you “watch a video,” your video player is doing at least three things at once:
1. Interpreting the container format to find out which video and audio tracks are available, and how they are stored within the file so that it can find the data it needs to decode next
2. Decoding the video stream and displaying a series of images on the screen
3. Decoding the audio stream and sending the sound to your speakers
4. A video codec is an algorithm by which a video stream is encoded. Your video player decodes the video stream according to the video codec, then displays a series of frames on the screen. Most modern video codecs use all kinds of tricks to minimize the amount of information required to display one frame after the next. Some videos don’t actually change all that much from one frame to the next which allows for high compression rates resulting in a smaller file sizes.
There are lossy and lossless video codecs. Lossless video is much too large to be useful on the internet, so let’s focus on lossy codecs. A lossy video codec means that information is being irretrievably lost during the encoding process. Like copying an audio cassette tape, you’re losing information when encoding the video source, and degrading the quality every time you encode. Instead of the “hiss” of an audio cassette, re-encoding a few times will make the video look blocky, especially during scenes with a lot of motion. (This can happen even if you encode straight from the original source, if you choose a poor video codec or set the wrong parameters.) On the bright side, lossy video codecs can offer amazing compression rates by smoothing over blockiness during playback, to make the loss less noticeable to the human eye.