Introduction: In the present study, we assessed crossmodal associations between odors and both color and shape, with particular interest in the principles beneath these mappings. We hypothesized that visual associations of odors would primarily reflect observable features of a smelling object and thus vary with different source assumptions of the very same smell. Methods: We asked 30 participants to visualize their odor associations on a drawing tablet, freely deciding on color and shape. Additionally, subjects provided ratings on perceptual and shape-related dimensions as well as a verbal label for each sample. Results: With respect to color selection, the results confirmed a source-based mapping approach: odors rated as familiar were associated with very particular colors that typically resembled the appearance of their source. For less familiar odors, color selection was rather inconsistent but still then went along with assumed odor objects. Shape ratings changed with odor identifications as well, but considerably less than for color associations. Shape ratings and shape drawings produced very different results. While shape ratings were unlikely rooted in the mental imagery of a shape, drawings frequently displayed concrete objects that depended on odor label. Conclusions: Results confirm the existence of stable odor–vision correspondences and suggest that language plays a major part in mediating these mappings. The frequently assumed hedonic foundation of crossmodal matchings could not be confirmed for this stimuli set. Implications: Odor sensations may trigger odor naming spontaneously. Assumptions about an odor’s identity, as well as the multisensory knowledge we have acquired on it, affect the visual associations of an odor.