This paper addresses the question if and to which extent linguistic surface features of test items in a physics assessment affect item difficulty. In an experimental study, linguistic features of test items in physics were varied systematically on three levels based on a heuristic model of linguistic demands. The results show that item difficulty can be predicted by linguistic features, but only for a limited number of items and not in a consistent way.