Does it hurt anything if he pounds on his chair? If it doesn't, I would say to ignore it the best you can and do the things you need to do. I suspect that he is just reacting to you being gone.
If the problem isn't dementia, calmly say that kind of behavior is why people get placed in a nursing home. It worked with my Dad, although the problem wasn't the same, just obnoxious behavior.
Alexander, I can just imagine how that could grind on the nerves. If the sound of him pounding on his chair sounds like a boxer practicing on a punching bag, maybe use that to relate to the sound.... then that way it won't be so grinding.