Agroforestry is a promising method for enhancing land-use sustainability in the Bolivian Andes. However, its benefits in terms of rural development are under-recognized due to gaps in understanding users’ perceptions while taking into consideration both local and global environmental goals. Our study aimed to narrow these gaps by developing an analytical framework for analyzing the site-specific socio-ecological factors and interactions related to local woody species and assessing their ecological, economic, and socio-cultural plant values in order to identify the most promising agroforestry species. The framework was then tested in an indigenous community at 2,760–3,830 m a.s.l., incorporating vegetation surveys, environmental studies, and interviews on plant functions. Ecological, economic, and socio-cultural values and the ecological apparency of plants were calculated, and detrended correspondence and principal component analyses helped to reveal the socio-ecological context of significant factors for plant distribution and uses. Results showed dominating seral woody species along an altitudinal gradient. Although shrubs were more ecologically apparent than trees, trees were perceived to be more valuable as the usefulness and cultural importance of species increased with plant height and timber availability. Phytosociological factors played a minor but still significant role in perceived usefulness. Schinus molle and Prosopis laevigata (<3,200 m a.s.l.), Polylepis subtusalbida (>3,200 m a.s.l.), and Baccharis dracunculifolia (both zones) were evaluated as most promising for agroforestry use. In conclusion, our analytical framework proved to be a valuable tool for context-specific agroforestry plant selection. Nonetheless, economic, technical, and socio-cultural limitations of cultivating native agroforestry species were revealed as well. Agroforestry science and practice should, therefore, focus on enhancing reproductive potentials of existing woody vegetation, as well as problem-oriented horizontal dialogues between indigenous, expert, and scientific actors.