Observation of collisions between cold Li atoms and Yb$^+$ ions