Giving up caregiving, am I wrong?

I have posted several times about my situation. My bro has banned me from seeing or talking to my mom. I did take care of her 6 years ago along with caring for her BF. No one helped or called or even asked how things were going. My si flat out said she would never take care or our parents. My bro lets my sister off by saying, "There's always one in the family." But he expects me to do the sole care giving. My mom and her BF lived with me for that 6 years. I kicked the BF out because of abusing narcotic drugs. My mom chose to live with him. A year later they moved down by me and my sibs. I did everything, docs, Rep. Payee, shopping and anything else my mom needed. I wouldn't do for the BF. (Long story) My mom's BF went into the hospital and my bro calls and tells me to go over and watch my mom. For 3 weeks I had my bro come over in the evening where I could go home for a couple hours, eat and sleep. I had to tell my bro and sis I couldn't do it anymore. I have a mental illness and am on disability for it. I threw the whole thing in my sibs lap. As punishment, I can't see or speak to my mom. Am I wrong for giving up her caregiving? I know if I went back to say I would help with my mom I would be where I started from. I would be sole caregiver. I can't wrap my head around why is there life more important than mine. I cared for my mom's mom for 2 years before she passed away. My sis didn't have a place to live so I opened my house to her. She just paid utilities. We couldn't get along as she was telling me what I could or couldn't do in my house so I moved out. For 2 years. Then I told her I needed my house back. She was NOT happy. All my life I've been made to feel guilty and do everything no one else in my family wanted to do. My mom is my best friend. I haven't seen her in 2 months. Last time I talked to her and told her I loved her, she cried. My bro considered that drama so I can't see her anymore. My family is very dysfunctional. My bro raped me as a child for over 4 years and my father molested my sis for most of her childhood. I've called agencies and the police and an attorney and they say without my mom's address there is nothing they can do. I'm at my wits end. I tried to take care of her again but I'm 50 years old and have been doing everything my family has told me to most all of my life. I'm venting and need some advice or encouragement or to be told how wrong I am. I'm beating myself up over this. At this point, I don't think I will ever see my mom again.