But I suggest that the literality of this story should be the least of your concerns.
The two Gospels containing infancy narratives, Luke and Mathew, give inconsistent accounts of the genealogy of Jesus as well as the time and place of his birth and attendant visitors (shepherds in Luke and the Magi in Matthew). This distinction was prevalent in the art of the Middle Ages and while it may be glossed over today, academic theologians accept that where these accounts conflict, then at least one of them cannot literally be true.
Realizing that the Word of God represented more than the literal text, early and medieval Christianity developed the allegorical method of reading scripture in order to interpret the inner meaning of the literal text. By the time of Thomas Aquinas in the 13th century , the four levels hermeneutic had become widely accepted as the means by which Scripture was to be interpreted, its method summed up in a well-known verse:
The letter teaches you the facts
Allegory what you should believe
Morality how you should act.
And Anagoge what to hope for.
The primary task of the four levels approach was to bring the Old and New Laws (Testaments) into unity through a double structure of prophecy.
Since the events of the Old Testament prefigure the mission of Christ, the Old Law is a prophecy of the New (allegory). The New Law in turn is a prophecy of the Kingdom of Heaven upon Earth at the Second Coming (anagoge). Understanding the allegories of the Bible was also the gateway to the moral meanings of the various stories and a guide to Christian conduct (tropological or moral image of ‘the Truth’). For ordinary priests, who might find the four levels hermeneutic difficult, standard interpretations of Bible stories were devised to aid with the anagogics (mystery interpretations).
But does this mean that you have to accept these standard interpretations as the only ‘truth’?
For example, the facts pertaining to the life of Jesus we know them could be interpreted as a patchwork of events in the lives of those who came before him: born to a carpenter and a virgin, like Krishna: born on 25th December, like Mithras; heralded by a star in the East, like Horus; walking on water and feeding the five thousand from a small basket, like Buddha; performing miracles, like Pythagoras; raising from the dead, like Elisha; executed on a tree, like Adonis; and ascending to Heaven like Hercules, Enoch and Elijah.
Looking at the Nativity through similarly expanded eyes, in Mary, you might sense of the presence of Isis; in Joseph, the patriarch with a crooked staff, Osiris, the luminous babe in the manger, Krishna; the ox (Taurus) and the ass (Aries), the two ages leading the new Age of Pisces, the Magi’s guiding star, the spirit of Zarathustra, and the angel announcing the birth, the spirit of the Buddha.
What if one of the Magi were Pythagoras reincarnated? What if the Magi had been initiated by the prophet Daniel? What if instead of one Jesus, there were two as depicted by the Leonardo Cartoon in the National Gallery in London and on the north portal at Chartres?
What if…well, you get the idea.
May peace be with you this holiday season and may hermeneutics take you as far as you’re willing to go.