The transition from a fossil fuel-based economy to a bio-based economy necessitates the exploitation of synergies, scientific innovations and breakthroughs, and step changes in the infrastructure of chemical industry. Sustainable production of chemicals and biopolymers should be dependent entirely on renewable carbon. White biotechnology could provide the necessary tools for the evolution of microbial bioconversion into a key unit operation in future biorefineries. Waste and by-product streams from existing industrial sectors (e.g., food industry, pulp and paper industry, biodiesel and bioethanol production) could be used as renewable resources for both biorefinery development and production of nutrient-complete fermentation feedstocks. This review focuses on the potential of utilizing waste and by-product streams from current industrial activities for the production of chemicals and biopolymers via microbial bioconversion. The first part of this review presents the current status and prospects on fermentative production of important platform chemicals (i.e., selected C2–C6 metabolic products and single cell oil) and biopolymers (i.e., polyhydroxyalkanoates and bacterial cellulose). In the second part, the qualitative and quantitative characteristics of waste and by-product streams from existing industrial sectors are presented. In the third part, the techno-economic aspects of bioconversion processes are critically reviewed. Four case studies showing the potential of case-specific waste and by-product streams for the production of succinic acid and polyhydroxyalkanoates are presented. It is evident that fermentative production of chemicals and biopolymers via refining of waste and by-product streams is a highly important research area with significant prospects for industrial applications.