Can my dad's POA keep him away from his family if he is mentally incompetant?

The real question, is, since you do not give us further information here is....WHY is your Dad's POA keeping him away from his family? Without any information I must assume that he or she has some reason?
Is his POA a member of his family? A Spouse? A significant other? When did your father confer POA on this person. That would have been a choice he made when he was mentally competent. Do you know why your Dad chose "this person" instead of another family member.
There is a good deal of this story missing. If family has been disruptive to a now mentally incapacitated person, the POA, who was put there by Dad to protect him and his health and assets I assume, would have a right to protect him.
So why not tell us more about this story?