In conversation, individual utterances are almost always ambiguous, with this ambiguity resolved by context and discourse history (common ground). One important cue for disambiguation is the topic under discussion with a particular partner (e.g., “want to pick?” means something different in a conversation with a bluegrass musician vs. with a book club partner). Here, we investigated 2- to 5-year-old American English-speaking children's (N = 131) reliance on conversational topics with specific partners to interpret ambiguous or novel words. In a tablet-based game, children heard a speaker consistently refer to objects from a category without mentioning the category itself. In Study 1, 3- and 4-year-olds interpreted the ambiguous pronoun “it” as referring to another member of the same category. In Study 2, only 4-year-olds interpreted the pronoun as referring to the implied category when talking to the same speaker but not when talking to a new speaker. Thus, children's conception of what constitutes common ground in discourse develops substantially between ages 2 and 5.