During visual servoing space activities, the attitude of free-floating space robot may be disturbed due to dynamics coupling between the satellite base and the manipulator. And the disturbance may cause communication interruption between space robot and control center on earth. However, it often happens that the redundancy of manipulator is not enough to fully eliminate this disturbance. In this paper, a method named off-line optimizing visual servoing algorithm is innovatively proposed to minimize the base disturbance during the visual servoing process where the degrees-of-freedom of the manipulator is not enough for a zero-reaction control. Based on the characteristic of visual servoing process and the robot system modeling, the optimal control method is applied to achieve the optimization, and a pose planning method is presented to achieve a second-order continuity of quaternion getting rid of the interruption caused by ambiguity. Then simulations are carried out to verify the method, and the results show that the robot is controlled with optimized results during visual servoing process and the joint trajectories are smooth.