The phenomenon you are describing is known as dispersion, and it occurs when white light passes through a prism or when light undergoes diffraction through a double slit. Dispersion is primarily a result of the different wavelengths (or colors) of light having different indices of refraction within the material.
When white light passes through a prism, the light is refracted, or bent, as it enters the prism and again as it exits the prism. The extent of refraction depends on the wavelength of light. Different colors of light, corresponding to different wavelengths, experience varying degrees of bending, resulting in the separation of colors.
Red light, which has a longer wavelength, is less refracted than blue light, which has a shorter wavelength. This causes the red light to deviate less from its original path, resulting in it appearing closer to the original direction of the incident white light. On the other hand, blue light, with its shorter wavelength, is refracted more and deviates from the original path, causing it to appear at the bottom or further away from the initial direction.
Similarly, when light passes through a double slit and undergoes diffraction, different wavelengths are diffracted at different angles. This results in the separation of colors, with longer wavelengths (such as red light) being diffracted at smaller angles compared to shorter wavelengths (such as blue light).
In both cases, the phenomenon is primarily a consequence of the different wavelengths (or colors) of light interacting with the medium or the diffracting elements. The differences in wavelength correspond to differences in frequency as well, but in this context, it is the wavelength that primarily determines the observed separation of colors.