With all siblings listed as POA, how does this effect the assistance provided in financial matters by the primary caregiver?

I was recently told by a lawyer I consulted that, by the POA handling the expenses keeping "meticulous" records and making regular financial reports (e.g. quarterly) to the others, they have the chance to question items right away and so have little ground to question them later. Any heir might question the finances later, not just the POAs. While people might question the financial decisions the dad made, and litigious people might bring it to court (and hence bring costs up), I can't see they'd have much of a leg to stand on. But I've seen friends brought down by siblings who questioned how they spent the money, and it makes me nervous. What if you don't want to be the POA, but there's no choice because the parent is no longer competent?