Right - but the problem is that your souncard is probably "taken over" by Skype,and maybe not available to Reaper.
There probably is a way,but I'm not familiar with that kind of setup.
I can't say for sure,but you might be able to do it by using the free ASIO4all driver.
A workaround would be if you had an external recorder (like maybe a ZoomH4) where you could record from the headphones out,and hear it through the monitor.
Just found a web page on How To Record A Skype Call :
http://skypetips.internetvisitation....ype_calls.html