Astro2020: Empirically Constraining Galaxy Evolution