Srimad Bhagavatam Canto 11, Chapter 23, Text 54

SB 11.23.54

karmastu hetuh sukha-duhkhayos cet
 kim atmanas tad dhi jadajadatve
dehas tv acit puruso ’yam suparnah
 krudhyeta kasmai na hi karma mulam
 
Translation: 
 
If we assume that fruitive work is the cause of happiness and distress, we still are not dealing with the soul. The idea of material work arises when there is a spiritual actor who is conscious and a material body that undergoes the transformation of happiness and distress as a reaction to such work. Since the body has no life, it cannot be the actual recipient of happiness and distress, nor can the soul, who is ultimately completely spiritual and aloof from the material body. Since karma thus has no ultimate basis in either the body or the soul, at whom can one become angry?
 
Purport: 
 
The material body is composed of earth, water, fire and air, just like bricks, stones and other objects. Our consciousness, falsely absorbed in the body, experiences happiness and distress, and fruitive work (karma) is performed when we falsely consider ourselves to be the enjoyers of the material world. False ego is thus the illusory combination within our minds of the self and the body, which are actually two separate objects. Since karma, or material work, is based on illusory consciousness, these activities are also illusory and have no factual basis in either the body or the soul. When a conditioned soul falsely considers himself to be the body, and consequently the enjoyer of the material world, he tries to find pleasure in illicit connection with women. Such sinful activity is based on his false concept of being the body and thus the enjoyer of women and of the world. Since he is not the body, his activity of enjoying a woman does not actually exist. There is merely the interaction of two machines, namely the two bodies, and the interaction of the illusory consciousness of the man and woman. The sensation of illicit sex occurs within the material body and is falsely assimilated by the false ego as its own experience. Thus the miserable or pleasurable reactions of karma ultimately act upon the false ego and not upon the body, which is composed of dull matter, nor upon the soul, which has nothing to do with matter. False ego is the illusory concoction of the mind; it is specifically this false ego that is suffering happiness and distress. The soul cannot become angry at others, since he is not personally enjoying or suffering. Rather, the false ego is doing this.
Srimad Bhagavatam Canto 11, Chapter 23, Text 53
Srimad Bhagavatam Canto 11, Chapter 23, Text 55