We provide a new explanation why the effective tax rate is smaller for larger firms, even in the absence of common channels such as profit shifting and lobbying activities. This result emerges in a heterogeneous firms model with endogenous markups based on Melitz & Ottaviano (2008). Our framework features imperfect pass-through of corporate taxes into prices and partial deductibility of production costs. Corporate taxes reduce mark-ups and hence pre-tax profits, especially for high cost firms. As production costs are only partially deductible, low productivity firms are relatively more responsive to tax policy than high productivity firms. We further show that shocks which affect mark-ups through the toughness of competition, such as trade liberalization, reinforce the heterogeneity in effective tax rates across firms.