Robotic hands use rolling contact to manipulate a grasped object to a desired location, even when the finger and the palm linkage mechanisms lack degrees of freedom. This paper presents a systematic approach to the forward and inverse kinematics of in-hand manipulation. The moving frame method in differential geometry is integrated into the product of exponential formula to establish a pure geometric framework of the kinematics of a robot hand. The forward and inverse kinematics of a multifingered hand are obtained in terms of the joint rates and contact trajectories. A two-fingered planar robot hand and a three-fingered spatial robot hand are used to demonstrate the proposed approach. The proposed formulation amounts to solving a univariate polynomial, providing an alternative to the existing ones that require numerical integration.