Corporate social responsibility (CSR) implies that companies have collective responsibility beyond legal requirements. This ‘beyond’ can be described along the ‘triple bottom line’ of sustainability (social, environmental and economic dimensions). Dealing with these dimensions is under discussion in societies and some propose the stakeholder concept as a framework for companies to identify their responsibilities. However, who counts as a key stakeholder and what is a legitimate stake are difficult to determine. This article discusses reasons, possibilities and difficulties of managing responsibility in innovation, taking two cases of genetically modified organisms as examples. It is argued that the stakeholder approach is necessary to provide different perspectives, but is not enough or even completely unsuitable for defining responsibilities. Moral responsibility involves judgement that differs between stakeholders and might vary over time, owing to emerging insights and changing priorities. Therefore, pursuing or abandoning a particular project does not constitute a responsible organization; rather, it is the preparation of decisions, accountability and the resulting pattern of behaviour that renders an organization responsible. The various notions of responsibility set the scene for discussion but do not provide a clear answer to the questions of right or wrong and good or ill intent. Key conclusions are that companies need to organize for responsibility and that responsibility needs to be complemented with accountability.