IDG Answers is a community of experts who are passionate about technology. Ask a question or answer one below.
I think to a large degree it is people being uncomfortable with new technology, even though most of the technology in Glass is repackaged existing tech. Christopher makes a good point, though - currently if you are walking around being a creeper with a cell phone, it is pretty obvious. Of course, a serious creeper can just buy a dedicated camera that is far less obvious than Google Glass. Don't believe me? Just google "wearable spy camera" and see what you can get for a couple hundred dollars.
I also don't think most people are aware of the amount of time they are on camera, especially if they live in a large city. I lived in London, and there there are so many cameras that you are more likely than not on camera once you step out of your flat. It's a Brave New World, and the social norms that govern behavior on this are not really firmly established. With Google Glass being such a high profile item, it has become representative of the growing loss of a sense of some privacy when in public spaces.
True, everyone has a camera-enabled smartphone these days, but the vast majority of these people don't walk around as if they're about to take a photo or video of you. The few people who actually do that are annoying and usually told to stop. I think if you're out in public, your expectations of privacy have to be reasonable. But in a work or meeting setting, many people may not feel comfortable with the possibility that they're being videotaped without their consent. And you can't tell if a Glass wearer is recording video.
Any time a new technology enables potential surreptitious recording or other forms of invading privacy, it's responsible to ask about implications regarding personal rights.