The Importance of the Current Input in Sequence Modeling