Common numbers thrown around in psychology is that a person's adult personality is determined by 50% genes, 40% peer-relationships (starting from childhood), and 10% by parent upbringing. There's no research basis for these exact numbers, but they're used to point out the huge effect of genes, and the much-lower-than-commonly-expected effect of parenting.
Also, in addition to what Blank said, while attachment does begin really early on, humans are very responsive to changes and children will not be extremely traumatized even if their caretaker suddenly changes. I remember doing some course work on this, and reading in some research that poor parent-child attachment only explained 10%-15% of teenage behavioral problems. The general trend seems to be, that while having good early attachment and good parenting is good for the child's development, it's not a huge factors (as compared to poor attachment and not so-good-parenting, NOT compared to completely socially deprived, "locked in the cellar" cases).
(BTW, I'm a psychology major, but developmental psychology is definitely not my main interest.)