Bauhaus -- yup, that's pretty much all you need. Unless you like the "Slice of Life" stuff, in which case you may need to delve into Tones on Tail. They have a two-disk set called Everything which is, literally, everything they've ever done. And then there's Love & Rockets...
Re: country -- I don't usually listen to country, but one of my best friends is in a country band, and I've quite taken to them. They have some delicious guitars and amps and microphones...it's just too bad they don't have a version of their quintessential song, "If I'd Shot You Sooner, I'd Be Out of Jail by Now". Lollers. They also do a lot of what I call retro revival -- think Sun Studios era rock'n'roll.