Sparsity, variance and curvature in multi-armed bandits