Astronomers do not directly observe the Big Bang itself when they look through telescopes like the Hubble Space Telescope. The Big Bang occurred roughly 13.8 billion years ago, and the light from that early period has long since traveled vast distances and been redshifted due to the expansion of the universe. However, by studying the oldest and most distant objects they can observe, astronomers can indirectly probe the early universe and gain insights into its early stages.
When astronomers say they are observing objects that are "13 billion light-years away," they are referring to the distance that light has traveled from those objects to reach us over the course of 13 billion years. The light from those objects was emitted when the universe was much younger.
The Hubble Space Telescope, along with other telescopes, allows astronomers to look deep into space, observing very distant and ancient galaxies. These galaxies appear as they were when their light was emitted billions of years ago. By analyzing the light from these distant objects, astronomers can study the composition, structure, and evolution of the early universe.
If astronomers were to look in other directions of space, they would indeed find other older galaxies. The universe is vast and contains billions of galaxies spread across immense distances. As telescopes improve and our observational capabilities advance, astronomers continue to discover and study galaxies at increasingly greater distances and thus further back in time.
It's important to note that the further we look into space, the more we are effectively looking back in time. By observing objects at greater distances, we are observing them as they were in the past. Therefore, studying different regions of space allows astronomers to explore different eras in the history of the universe.