Microphones
Condenser microphones are probably going to be in your budget. Ditch the single USB and head off to better mics for you and your guest.
One option to consider is the Audio-Technica AT2020PK Vocal Microphone Pack for Streaming/Podcasting, Includes XLR Cardioid Condenser Mic, Adjustable Boom Arm, and Monitor Headphones, Black. While you are buying your microphones, also consider the additional equipment to make a real professional setup.
The Audio Interface
One option might be the Focusrite Scarlett 2i2 (3rd Gen) USB Audio Interface
