Dad wants me to visit him every weekend. Any advice?

I love my father dearly. He lives in a senior housing about 20 minutes from me. My problem is, he wants me to visit him every weekend. Come Thursday, he's asking me about coming over. The problem is, I have a full time job, and I'm only off on the weekends. I'm single and sometimes do activities on weekends. That leaves me with maybe one day off to do chores, etc for myself. I have no problem visiting him every few weeks (say every two to three weeks), but when I tell him I can't make it he seems really disappointed. I'm torn between trying to see him as often as I can - when I can't I feel incredibly guilty. It doesn't help that my two sisters (one who lives right around the corner) rarely see him. I love him but I'm trying to balance having a life and seeing him. Sometimes I feel like I don't go to activities or socialize so that I can spend time with him. Does anyone have any suggestions for trying to balance my life with seeing him. Sometimes, even when I'm out, I feel bad that I'm not with him. Its incredibly stressful.