$\mathbb{Z}_N$ lattice gauge theories with matter fields