This paper investigates the structure of cyclically pure (or C-pure) projective modules. In particular, it is shown that a ring R is left Noetherian if and only if every C-pure projective left R-module is pure projective. Also, over a left hereditary Noetherian ring R, a left R-module M is C-pure projective if and only if M = N ⊕P, where N is a direct sum of cyclic modules and P is a projective left R-module. The relationship C-purity with purity and RD-purity are also studied. It is shown that if R is a local duo-ring, then the C-pure projective left R-modules and the pure projective left R- modules coincide if and only if R is a principal ideal ring. If R is a left perfect duo-ring, then the C-pure projective left R-modules and the pure projective left R- modules coincide if and only if R is left K¨othe (i.e., every left R-module is a direct sum of cyclic modules). Also, it is shown that for a ring R, if every C-pure projective left R-module is RD-projective, then R is left Noetherian, every p-injective left R-module is injective and every p-flat right R-module is flat. Finally, it is shown that if R is a left p.p-ring and every C-pure projective left R-module is RD-projective, then R is left Noetherian hereditary. The converse is also true when R is commutative, but it does not hold in the non-commutative case.