Critical fields on the M5-brane and noncommutative open strings