Even if the universe were infinite, our technology only allows us to see back in time about 15 billion years at this stage.
However, there are a number of supporting observations to suggest that the universe is only about 15 billion years old, and seems to have expanded from a point (hence the Hubble observation of the Doppler effect and his constant, etc).
I do remember seeing a model of the expanding universe and it was a bunch of concentric "shells" of galaxies, clusters and cosmological elements, seperated by large voids. We just can't see beyond the last layer, fifteen billion light years away. (And even then we need a super-bright source called a Pulsar, which gives off the equivalent of the Sun's energy in a second, or something similar.)
So, as far as we can tell, the universe is not infinite. Although there is no reason why it couldn't be.
The latest theory I read the other day was that black holes contained entire universes. I'll dig it up after breakfast if you like.
One of the more interesting areas in Astrophysics is today is question of how the expansion expands. One of the most puzzling things about the universe, right now, is that matter doesn't seem to be distributed homogeneously. Some places seem to be more densely packed than others, which would, apparently, indicate that the expansion isn't regular. This, if it's true of course, might require a complete revision of the Big Bang hypothesis.
What's that black hole theory about meta?