Caveolin-1 (cav-1) is an important membrane protein that plays a vital role in cellular signaling and trafficking by organizing membrane domains. The peptide interacts with cholesterol-rich membranes and induces large morphological changes in them, forming microdomains such as caveolae. Here, we use coarse-grain molecular dynamics simulations to study the interaction of cav-1 peptides with several model bilayer systems mimicking biological scenarios, such as cholesterol-rich domains, cholesterol-depleted domains, and unsaturated lipid domains. We show that cholesterol modulates the depth as well as orientation of cav-1 binding to membranes. Furthermore, the presence of cholesterol stabilizes more open conformations of cav-1, and we speculate that the binding modes and open conformations could be responsible for inducing morphological changes in the bilayer. We also calculated the partitioning free energy to different bilayers and show that binding to cholesterol-rich bilayers is more favorable than cholesterol-depleted bilayers and the binding to unsaturated bilayers is the least favorable. Binding to cholesterol-rich bilayers also changes the pressure profile of the bilayer to which it is bound and thereby affects the local spontaneous curvature. Our results highlight molecular details of protein-lipid interplay and provide new insights into the effects of cav-1 in tuning the morphology of cholesterol-rich membranes.