Flexible Models for Galaxy Star Formation Histories Both Shift and Scramble the Optical Color-M/L Relationship