Recent evidence suggests that infants as young as 12 month of age use pointing to communicate about absent entities. The tacit assumption underlying these studies is that infants do so based on tracking what their interlocutor experienced in a previous shared interaction. The present study addresses this assumption empirically. In three experiments, 12-month-old infants could request additional desired objects by pointing to the location in which these objects were previously located. We systematically varied whether the adult from whom infants were requesting had previously experienced the former content of the location with the infant. Infants systematically adjusted their pointing to the now empty location to what they experienced with the adult previously. These results suggest that infants’ ability to communicate about absent referents is based on an incipient form of common ground.