IDG Answers is a community of experts who are passionate about technology. Ask a question or answer one below.
One thing that I would add to what Christopher posted is to consider spending a little more money and get a good quality microphone and pop stopper. There may be no need to spend $1000 plus for a Shure VP88 field mic when you are dipping your toes into podcasting, but you can get a solid studio mic like a Studio Projects B1 for around $120. A decent mic will make the sound a lot fuller, and should last you for years. The pop stopper will stop "pops" when you use words with hard consonants like "Ps" and "Bs" - I would put that on the must have list.
The person who wrote this article recommends starting small. That means you need a host, a domain, and a computer with a microphone and headphones. He says you can do all that for less than $100.
Of course, the most important thing you need is a good idea for a podcast and a strategy for developing ideas and an audience. Once you start to 1) get better 2) grow an audience 3) become more ambitious (more than one host, guests, remote locations), you can consider buying more equipment.