Do I need permission from the POA to go see my mom?

The power of attorney for financial is responsible for paying your mother's bills and cost of living from what financial resources she has in a way that she would do were she still capable. A Power of Attorney for health care is responsible for making decisions for your Mom's health in a way that she would do if she were able, in so far as that POA understands her wishes were or would be were she capable today. So all that said, there are of course circumstances under which the POA, having certain knowledge, could make a decision about visitors, who, what, when and for how long. Any suspicion that a visit would be distressing during this end of life care would likely result in the power of attorney asking that no such visits occur. You do not tell us why you are thus estranged from your father, and I presume (?) your mother. If you wish to make and end of life visit to reassure your mother that you do love her very much, it would be wise to first approach the lion guarding the gate, that being your Dad, to ask his permission, and tell him why you wish to make the visit. That is to say, throw yourself upon his mercy, as I suspect he is now in charge of what he believes is his last duty of protection of his wife. Without your giving us further information I am afraid we are stuck with this as a guessing game, so that would be my advice to you, and wishing you good luck in this sad time, hoping for some peace at the end.

Is there more to this story? I kinda think so.

You don't. Sounds like he's let being POA go to his head as a power trip. He does not have any legal standing to not allow you as her POA.