This paper presents a single-input and single-output (SISO) adaptive sliding mode control (SMC) combined with an extended Kalman filter (EKF), which is used as an observer to control stimulus-responsive polymer fibres as an actuator. Conductive metal-polymer fibres are the fundamental core for wearable technology and an integral part of smart textiles. To control this actuator a SMC is combined with an EKF and used as an observer to estimate the velocity. Despite the particular simplified model of the considered actuator, the EKF presents a nonlinear Jacobian matrix. The parameter settings of the system and measurement covariance matrix, together with their initial values, are done heuristically. Because of the slow velocity of the fibre, the EKF produces poor estimation results. Therefore, a derivative approximation structure is proposed to estimate the velocity though the measurement of the position. Both estimations are analysed based on simulations. The simulation results indicate that the proposed algorithm is effective and robust.