Detecting areas with high social-ecological vulnerability (SEV) is essential to better inform management interventions for building resilience in coastal systems. The SEV framework, developed by the Intergovernmental Panel on Climate Change, is a robust method to identify SEV of tropical coastal systems to climate change. Yet, the application of this framework to temperate regions and other drivers of change remains underexplored. This study operationalizes the SEV framework to assess the social-ecological implications of fishing and tourism in temperate coastal systems. We spatially represented the SEV of coastal systems and identified the social and ecological vulnerability dimensions underpinning this SEV. Our results demonstrate that different dimensions contribute differently to the SEV, suggesting the need for distinctive management intervention to reduce the vulnerability of coastal systems. Our findings also highlight that livelihood diversification and the protection of marine areas may be plausible strategies to build resilience in temperate coastal systems that face fishing and tourism pressures. With this study, we hope to encourage the application of the SEV framework to other drivers of change for building more resilient coastal systems.