I see it as a seperation of church and state issue. Right now, two men can be married in a Unitarian church, but the marraige would not be recognized by the state of Indiana or the federal government. Likewise, a person who got divorced but did not get an annulment prior to getting remarried would not have their marraige recognized by the Catholic Church, yet it would still be recognized by the state that it took place in.
Marraige usually consists of two things. It consists of a expensive party in a fancy building with stained glass windows and a legal contract. Let the entities who own the fancy buildings decide who may get married inside. And let any consented adult enter into whatever legal contract they choose.
Or we can go back to the good old days, when a man had his wife, and his secretary, and the babysitter. And a woman had her husband, and her gardener, and the mailman.