How do you know when to get POA?

My fil has been in out of hospitals. He has chf, kidney failure, on dialysis recently, and diabetes. My sil had been basically taking over, which I had stepped back since she was the daughter. And like a lot of you had said, "Let her if that is what she wants." But, she wants to be caregiver, when she wants to be. Her husband was in the hospital and she hadn't come up (she lives in another state) so it was left up to me and my husband. I called her everyday and kept her updated. While he was in the hospital, he aspirated and was moved to ICU. I have another bil and sil who stopped talking to my mil, but resumed talking and seeing her prior to her passing. After her passing, wouldn't speak or see my fil for over a year. Now that he is sick, yep you guessed it, they are coming around. Only two people allowed in ICU. My husband and I went to visit him in ICU, and they are in there. Nurse comes in and tells us only 2 at a time. SIL says "WE drove all this way(150 miles) and we aren't staying long, we aren't leaving)." Husband, turned around and walked out. I just waited to talk to the nurse, and then walked out. Fuming yes. The day before, I found my fil, and thought he was dying, the were suctioning him out, due to the aspiration. I was so relived to see him sitting him talking. I went over later that evening, and guess who walks in. Yep. BIL sits down, and nurse tells sil, only two to a room. SIL walks out. I'm sitting there and think"Oh heck, there are going home, I'll get up and let her say by. I say to her " Go ahead Carla, you can go in" she says "Thanks, I'm surprised though". All of them, come around to visit. My husband keeps him lawn mowed, garage was broke, he fixed it, gutters need cleaned he did it. When he got released from the hospital we took him home. We drove over in the car, and they said OH we are releasing him tonight. Had to drive 20 miles back to get the SUV to take him back to rehab. I call and make his appointments, I call to get charitable financing for him, he wants to change doctors, he has confusion from dialysis I calm him, he calls at 2:00 in the morning, I wake up and take the call. Where are they? His daughter, brings his bills to him to sign, but my husband has authority to sign on his checkbook and I've told and told her that. We have his checkbook, she just got a new checks. It's like no respect. SO, wondering when is the right time to get POA? He is so confused, he wants to go home, he isn't strong enough, but I told him if he gets well enough to walk around his house, I'll move in for awhile and drive him to out patient PT. His daughter took his dog to her daughter's house, because nobody was home to take care of it, and borrowed money from him to put a fence up at daughter's house...for the dog. I have his insurance cards, she has his wallet with his credit card. He asked where his wallet was, and I asked her. She said I have it, he doesn't need it. Just not sure where to step back. When she is here, she wants to be "the caregiver" when she leaves, she "allows" my husband and I. We do all the work really, she just visits, day and night granted. Even the discharge nurse calls her I've never talked to the discharge nurse, and I was there when he was discharged! When is the time to get POA, for his protection?? sorry it's so long