Sign in

Object Recognition as Next Token Prediction

By Kaiyu Yue and others at
LogoUniversity of Maryland
We present an approach to pose object recognition as next token prediction. The idea is to apply a language decoder that auto-regressively predicts the text tokens from image embeddings to form labels. To ground this prediction process in auto-regression, we customize a non-causal attention mask for the decoder, incorporating two... Show more
March 31, 2024
Loading PDF…
Loading full text...
Similar articles
Loading recommendations...