Dusty:
Yeah, that's right. So far. So, you're asking the same question on the amp side. The L/R- is unterminated. I definitely don't want a switch there.
So, in summary the problem is with a 4-pin jack, or even two 3-pin jacks, that are used on the output of a four channel design, you cannot "convert" this using any combination of adapters, custom or otherwise, for use by a single ended terminated headphone. You need a ground reference.