I think a big part of it, whether rightfully so or not, is that the conflict between imagery and abstraction has been equated with the conflict between spirituality and intellect. At one time poets usually did sit in the Church’s crooked hand, and all the shiny cogs of the Enlightenment disdained it either for that or because it played with illogic (whichever came first). And modern poetry is quick to seize on the romanticized poet as seer because it sprinkles a little glamour on an otherwise squalid profession which of course fits in very well with any monastic role-playing we might want to throw in the mix. This is all the more appealing since modern society has freed itself from the fetters of religion, or so we are told, and poetry can be this gleaming modern and secular spirituality. Of course the problem in both cases is setting up pins. You are either for or against, and not allowed both. Even if you try to integrate them, public opinion squarely proclaims you in one camp or the other.