On the Coupling of Gravitons to Matter