I have never believed it before coming to RTS, and also now been following Reformed teaching, it has opened my eyes to see the significance and blessing of infant baptism.
Our infant children are also covenant children and should receive the sign of the covenant. Baptism is one of the two sacraments in the protestant faith, Catholic has seven. Sacrament by definition is a sign and seal of the covenant promises of God towards us. Baptism is a dynamic equivalent of circumcision in the Abrahamic covenant with God. The Jewish believers in Jesus do not get circumcision but baptism as a sign of being part of the covenant with God in Christ. Same applied to Gentile believers.
Circumcision was not only given to children of promise of Abraham, Isaac, but also to non-promised child, Ismail. It was administered to Abraham and his children. Circumcision pointed to belonging, discipleship, covenant obligations, and eventually to the future faith of the child.  Water baptism has the same spiritual meaning. Children are to be marked to be part of the covenant of God, just like Abraham’s children, didn’t necessarily believe in Yahweh yet, but they were circumcised.
Col 2 : 11 In him also you were circumcised with a circumcision made without hands, by putting off the body of the flesh, by the circumcision of Christ, 12 having been buried with him in baptism, in which you were also raised with him through faith in the powerful working of God, who raised him from the dead.
Baptism and circumcision carry the same spiritual meaning.
Not only that, God deals with households as a unit and welcomes whole families into the church. Acts 16:13-15, 32-34, 1 Cor 1:16. Household concept is precious and beautiful. Would you like to consider your children outside of the covenant of God growing up in a Christian family and believing parents, you? So, don’t exclude your kids. The whole idea of repentance and believing first before baptism is only found in Peter’s preaching in Acts…repent and be baptized, you & your family shall be saved. Repent and baptism can be independent, nothing here binds one to the other. Its just a sequence.
History of early church been baptizing infants. They have been doing that through big names Augustine, Iraneous, etc. Not until the 16th century, the whole thing is called into question. 
Finally-baptism is a sign of inclusion in the covenant community just like circumcision was. Children are holy by believing parents.
1 Cor 7 14 For the unbelieving husband is made holy because of his wife, and the unbelieving wife is made holy because of her husband. Otherwise your children would be unclean, but as it is, they are holy.