Srimad Bhagavatam Canto 11, Chapter 14, Text 29

SB 11.14.29

strinam stri-sanginam sangam
 tyaktva durata atmavan
kseme vivikta asinas
 cintayen mam atandritah
 
Translation: 
 
Being conscious of the eternal self, one should give up association with women and those intimately associated with women. Sitting fearlessly in a solitary place, one should concentrate the mind on Me with great attention.
 
Purport: 
 
One who has intimate contact with women and becomes attached to them will gradually lose his determination to go back home, back to Godhead. Association with lusty men gives exactly the same result. Therefore, one is advised to be fearless and to sit down in a solitary place, or a place where there are no lusty men and women committing spiritual suicide. Without fear of failure or of unhappiness in life, one should remain with sincere devotees of the Lord. Atandrita means that one should not compromise this principle but should be rigid and cautious. All this is possible only for one who is atmavan, or fixed in practical understanding of the eternal soul.
Srimad Bhagavatam Canto 11, Chapter 14, Text 28
Srimad Bhagavatam Canto 11, Chapter 14, Text 30