The impact of initial conditions on quasi-normal modes