So what you're saying is that it's probably better to use a browser, because your bandwidth between your PC and your TV is likely to be much better, as well as having the capability to cast not only web content but also anything on your desktop. Plus a mirror is much easier to control, and some cast capable devices may not have the memory to "download" video files.
What am I missing here? You seem to think it's a bad thing.
sending a video stream from a file on your disk on a computer results in higher quality video on the television than playing a video in your browser and encoding a stream of your browser window and sending that to the chromecast to display on the TV.
What am I missing here? You seem to think it's a bad thing.