Why is my Dad picking his poop out of his bottom?

Ok thank you your answers. It makes sense.

From my nursing career it sounds like your Father may have what they call Sundowners. As daylight fades, especially Winter, and the staff changes in late afternoon,, the sounds are different, TVs go on and sounds get louder. I'm not sure they know positively what causes Sundowners but behaviors can change, more depression, and crying is noted. Some people get stubborn or even strike out at their caregivers, and short term memory seems to get worse. You might Google Sundowners and read about it.
Another reason for your Dad , what nursing homes call "digging" is possibly constipation, or it can be just a behavior. Sometimes, nursing home caregivers put socks on the hands of the patients. Ok, sounds odd, but they do.

There might be a medical reason why he is doing that... something is making him feel very uncomfortable. Maybe an internal hemorrhoid.