Capsacasin activates pain receptors rather than the standard flavour (like menthol does with cold)
Your butthole has a similar set of receptors (not quite taste buds, but as far as chili is concerned it is), so if you haven't denaturised it through digestion, you'll 'taste' it on the way out.