Quasi-powerful $p$-groups