How do I handle finances with a caregiver? Do I pay the agency directly and they pay for groceries or do I give them grandma's debit card?

Talk to the agency about how they would prefer this to be set up.

I can't imagine giving a new caregiver Gma's debit card.

NEVER give a debit card to anyone but the POA.

You can order groceries delivered online, either through the grocery store direct, (Walmart.com/free delivery), or via Instacart. The small fees plus a 5% tip will be much less than you pay a caregiver to shop.

Do not hand over your Grandmother's debit card to anyone, for any reason.

A person can enter the card on any website and shop away without your knowledge, even after they give the card back, and have left your employment.
You may not discover the theft until much later, and after having to inquire of Grandma, did you buy this?

As a fiduciary, find out ways to protect Grandmother's finances. Suggestions can be found online. You can learn...

I would order and pay for items on line. The caregiver can either pick up or you can elect to have the items delivered.
I would not give a caregiver a credit or debit.
If you can not order on line and pay that way give the caregiver a preloaded card. You can buy them at most grocery stores in any dollar amount you wish. I would keep the amount small but reasonable. And you must ask for the original reciept

For a period of time just pre-COVID we had someone coming in to do some of the chores, making meals, laundry… Moms grocery store had recently started a delivery or pick up option where we ordered her groceries on line and then could opt to either have them delivered (we never did this for coordination reasons) or pick up where you pulled into a designated parking spot, called them and they brought the order out and loaded into the car. We started having the woman who was coming in to help out do the pick up, it was far less expensive than paying her to take time doing the shopping and I just paid for the groceries when I ordered them no need to worry about giving her a card or reimbursing her. We still use the service but my brother or I, when I’m in town, do the pick up since we haven’t reintroduced the helper yet.

I'd like to think agencies are much better a decade later, but when I first arrived at my grandmother's house and looked over services supplied and billed by her CGs, there was literally no food in the refrigerator, and some small cans of Vienna Sausages on the pantry shelf. I saw what she was being billed for, and food worth hundreds was supposedly bought every other week. There was also a charge to an old Sears card that I later found on her credit report that was traced back to a previous agency caregiver.

Ask the agency how they usually handle this, and go along with it and monitor things if that's easier -- but with so many delivery services now, like Instacart, do you really need to go through them at all for this one?

And welcome to AC!

I have been with families where I have cared for their Mom or Dad for often more than 5 years and the families have always trusted me with their parents debit card. The present client I am with the family lives in USA and trusts me completely with their moms debit card. I pay all the accounts which are not on debit order. I purchase what is needed in the house and they pay for my food without ever complaining. I also withdraw a certain amount to have as petty cash. But the bulk of the purchases I use the debit card so that the money spent can be checked

I do have to say that I do a spending analysis every month of all the expenses for the cash withdrawals and debit card payments to enable the family member who control the money to check the bank statement at anytime

It is a good feeling to feel you are being trusted and relied on.

I have been doing dementia care for 20 years and never felt I was not being trusted.

You give the caregiver a grocery list and a certain amount of cash to go to the store with. Then you tell her to bring back a receipt and change if she doesn't spend the whole amount. This is how it's done in private-pay care as well. Grocery shopping or other errands that involve the caregiver making purchases on the client's behalf do not go through the agency they work for.
If there's a good client/caregiver relationship where they trust each each other, the caregiver will put in a few dollars if the bill goes over what they were given, but they're paid the difference right away by the client or family.
I've done this personally many times. I've paid for client shopping bills in full out of my own pocket if they didn't have access to cash. Many elderly people don't. I would produce store receipts and they would give me a check separate from my pay for money spent. I always refused to use a client's credit or debit card for shopping because this leaves a door open for a caregiver to be accused of stealing or some other nonsense an elderly client gets into their head and becomes fixated on. It's a matter of trust with caregiver/client and family. It's also a matter of the caregiver protecting themselves and their reputation. Cash and store receipts are the way to go for shopping needs.
I've worked for other clients and families who would send me to a store with a list and nowhere near enough cash to cover the bill. So, I'd get the items I could and they did without the rest or someone else went and got them. I would not so much as put up a nickel of my own money because I knew they wouldn't pay. It all depends on what the caregiver. client, and family agree to.

Have the caregiver make a grocery list. You can either place order with the grocery store for pick up by the caregiver or enlist a "shopper" (there are several services you can choose from online). You pay for the groceries. You can either have the groceries delivered to grandma's home or have the caregiver pick them up at the store.

NEVER, NEVER, NEVER give anybody access to financial information.

Mom used to send her aid to the store and give her a check for whatever was on the receipt. That ended up including her aid's weekly groceries for her family. Mom was also paying for the time her aid took to shop, drive home and put her groceries away, pick up her son from school, and come back to help Mom with dinner. In other words, Mom was stolen from to the tune of hundreds of dollars per week. She didn't do her aid any favors either. Thinking she could get away with it she repeated at her next job. She was arrested and prosecuted. Don't tempt anyone by making it easy to help themselves. Several of the caregivers here have been amazing in the care and compassion they show the people they worked for, but this seems the exception today.