In recent years, factory production systems have shifted from low-mix high-volume production to high-mix low-volume production and the robots such as arm typed robots and mobile robots are installed in a factory. Integrating robots, sensors and the other devices into a system is costly. To solve the problems in regard to the costs, a method to automatically generate control code for a robot using a large language model (LLM) is considered. Controlling a robot using an LLM makes it possible for non experts in on-site adjustment of robots to adjust motion of a robot installed in factory by giving text-base instructions. In case to use an LLM, we must devise how to write instructions to obtain desired motion of the robot. In this paper, we address pick and place task using a robotic arm and propose a method to learn how to paraphrase instructions to high quality ones for desired controlling a robotic arm using an LLM in reinforcement learning.