A note on the $K$-stability on toric manifolds