Actually it only happens when ray passes through F. Here it is coming from focal plane. So the refracted rays are parallel to each other only and not principal axis.