In order to investigate the benefits of compound waterways more fully, this study reveals vessel navigational mode and traffic conflicts in a compound waterway through a case analysis, following which a type of simplified prototype of a compound waterway is proposed and three key conflict areas are specified. Based on the three key sub-models of slot allocation for vessels in a waterway entrance, traffic flow conversion of a main and auxiliary waterway in a precautionary area, and traffic flow coordination of division and confluence in a Y crossing area, a vessel traffic scheduling optimisation model is presented, with the minimum waterway occupancy time and minimum total waiting time of vessels as the objective. Furthermore, a multi-objective genetic algorithm is proposed to solve the model and a simulation experiment is carried out. By analysing the optimised solution and comparing it with other scheduling schemes in common use, the results indicate that this method can effectively improve navigation safety and efficiency in a compound waterway.