The relationship between frailty and glycemic control in older adults with diabetes remains uncertain, mainly due to the fact that previous studies have not accounted for measures of body composition. In older adults with diabetes, we examined the association between three types of frailty measures and glycemic control, while accounting for fat-free mass (FFM) and waist circumference (WC). Eighty older adults (age ≥65, 27 women and 53 men, mean age 80.5 ± 0.6 years) had gait speed, Cardiovascular Health Study Index (CHSI), Rockwood Clinical Frailty Scale (RCFS), and glycosylated hemoglobin (HgA1C) measured. HgA1C showed a negative association only with CHSI (standardized β = −0.255 ± 0.120, p = 0.038), but no association with gait speed or the RCFS. Even after accounting for FFM and WC, we demonstrated a negative association between glycated hemoglobin and increasing frailty in older adults with diabetes.