My parents are comfortable but not wealthy. As their caregiver, should I be added to their bank accounts to protect them?

Do this Now while they are able to understand. It could get tricky if they become incompetent. If that happens you would have to go through guardianship. This happend to my daughter (only child and heir) after her dad was in a horrible car accident with brain damage. Guardianship if governed by the court. It can be challenged. In her case she was just out of college (high degree and extreemely capable) and some old aunt and uncle of her dad challenged her (for the $$) She spent many many years dealing with this, in court! - it was a nightmare, people become nuts when money is involved - lawfuit involved from company that wrecklessly caused the accident-(especially if they do nothing to deserve it)...after 9 years he passed (he was in a vegetative state) and died wihtout a will and the aunts and uncles got nothing. But they made her life hell for 9 years.
Do it now before it is too late. Your siblings may say something now...but $$ is the root of all evis. Get your ducks in a row, it may save you a lot of stress later.

Get a durable Power of Attorney. Make sure it is Durable and includes health concerns. You may want to have them sign a advance directive (something like that ie living will).

I get that you're NOT trying to take over their finances at this point. I would tell them that 'just in case' your name needs to be added to their accounts.
It's no different doing that, then it is planning for a rainy day. They should understand that. And golfbhard is right, make sure it includes the medical aspect too.

thank you for your good advice I appreciate it!!!

When you go to the bank, make sure they know that you want to be added as a "signer" only...so that you are not personally responsible for any of the accounts. This allows you to write checks for them, pay bills, and view important documents. I have also established online banking - makes it so much easier to view the accounts and make sure that an unauthorized person is not accessing their accounts.
You also need both PoAs for medical and financial. I downloaded the documents online and had them filed according to my state's instructions. (whomever does this for you, make sure that the forms are updated according to your state guidelines)
Most of all, find out about your parent's wishes now and document it so there will be no second-guessing in the future.
good luck