Structure of potentials with $N$ Higgs doublets