Yes, you're absolutely correct. When we observe distant stars in the night sky, we are indeed looking into the past due to the finite speed of light. Light travels at a constant speed of approximately 299,792 kilometers per second (or about 186,282 miles per second) in a vacuum. Since the distances between stars and galaxies are vast, it takes time for the light emitted by those celestial objects to reach us here on Earth.
This means that when we observe a star that is, for example, 1,000 light-years away, we are essentially seeing the star as it appeared 1,000 years ago. The light emitted by the star has traveled through space for 1,000 years before reaching our telescopes.
Considering the vastness of the universe and the time it takes for light to travel, it is indeed possible that some of the stars we observe may no longer exist by the time their light reaches us. In fact, it is entirely possible that some stars we see in the sky have already undergone significant changes or even ended their life cycles, but we are still observing their light because of the time it takes to reach us.
This aspect of observing distant objects in space allows astronomers to study the evolution of stars, galaxies, and the universe itself over vast periods of time. By observing objects at various distances, they can effectively look back in time and gain insights into the past states and events of the universe.