Calibrating the Cepheid Period-Luminosity relation from the infrared surface brightness technique II. The effect of metallicity, and the distance to the LMC