I am a strong believer in the separation of church and state. There is little overlap between religion's interests and Uncle Sam's interests, and there is no reason to try to force an alignment of those interests.
Marriage is one of the few institutions that today straddles the line between being a civil matter and a religious matter. The sacrament of marriage has long been a cornerstone of virtually every religion. In addition to the historical precedent, there is also widespread strong attachment to the religious nature of marriage. The image of a wedding couple - with bride wearing a flowing white dress and veil - saying their vows before a congregation of their family and friends is much more powerful than the image of the couple signing a computerized form at the local registrar.
The state has a proper interest in the welfare of all children, and some interest in the process of inheritance, but otherwise, I can see no reason for the state to have any involvement with the sacrament of marriage. Questions of inheritance are always best addressed through wills, and inheritance laws are only applied in the unfortunate absence of a will. Today's laws regarding inheritance are murky, with various states having quite different rules regarding how legitimate and illegitimate children are defined and the extent to which each is eligible for inheritance. Inheritance laws themselves need a great deal of clean up, but are certainly no reason to perpetuate the government's presence in the arena of marriage.
The government's proper interest in children's welfare extends, of course, to all children, regardless of the circumstances under which they were conceived, and regardless of the family or institutional setting in which they are nurtured. So the state's protection of children in no way creates a valid interest of the government in licensing or regulating marriage.
How about tax breaks for marriage? As one who favors simplified tax structures, with no favored treatment for any special interest groups, I have a hard time justifying any special deals on inheritance taxes or income taxes for anyone. Current income tax laws don't actually help most married couples much anyway, and sometimes even create a penalty. Inheritance laws are their own mess, and are unfair in many ways. Fix the inheritance laws, but don't use them as a justification for perpetuating the state's meddling in the institution of marriage.
Eliminating all marriage laws sounds incredibly radical. The government's regulation of marriage has come to be accepted as a law of nature - as real as the law of gravitation, but why? Why polarize our nation over the government's definition of marriage when the state has no business being concerned about marriage in the first place? Imagine a new unfettered society in which the tradition of marriage is returned to it's proper home, in the church, and in the hearts of those choosing to live as one.