Abstract: Sequential advertisement recommendation is an important task in intelligent advertising systems. It needs models that can learn user-ad interaction sequences from different types of data.