If you have any interest in online gaming, yes, you do have to buy an Xbox Gold subscription. This is also true for PlayStation 4, by the way, which requires a PS+ subscription for most online games. Of course, there are still some excellent single player games that don't require access to online multiplayer servers, but you are really excluding yourself from some of the best gameplay experiences if you don't pony up for Gold.
Microsoft did make a change to their policies a while back, so even without Gold you can now watch online services (Netflix, YouTube, etc.) without it. This was a welcome change. I HATED it when Microsoft demanded $60 a year for Gold to watch streaming services that they didn't host or provide content for, to the point that even though I owned an original Xbox 360, I bought a PS3 because Sony didn't extort me the way Microsoft did. I also bought a PS4 and skipped the Xbox One because Microsoft held content hostage. As I said, they have since reversed that policy, but at this point the damage is done as far as I'm concerned, and while I will continue to enjoy some of my X360 games, I'll probably never buy another Xbox console.