After a Medicaid patient dies, does the money left in his checking account go to the County?

No, he was allowed to have 2k or whatever your state allows. If the NH was payee for his SS and pension (if he had one),the Personal Needs Acct set up for personal needs goes to his estate as does his bank acct. Ask the NH what you need to do to receive that money. Don't let them tell you anything different. That is your fathers money that was deducted from his SS check each month. In my State is $50 a month.

I was able to get Moms by getting a short certificate from Probate showing I was able to handle Moms finances.

This is actually a question for Medicaid in your state because Medicaid estate recovery is estate specific and if he died with more than $2k, the state may be entitled to it.

Also how the checking account is set up makes a difference,
like if he set the checking account so that it is to be POD / pay on death to a named individual; then that person is the beneficiary and the $ passes to them outside of probate; if $ or asset goes outside of probate, it is not an asset of the deceased estate so not an asset subject to an attempt by the state or it’s outside contractor for MERP / medicaid estate Recovery
BUT
if he did not name a POD, then the $ becomes an asset of his estate.

Assets of an estate can get involved as you have to have some sort of document to show you are administrator, executor or other recognized by the court authorized individual for his estate to use the $ or sell assets or divide up assets to heirs. Like JoAnn got a short certificate from her county probate court to do things for her moms estate, I went full traditional probate for my moms estate. We both had valid wills but did different paths to get things done as probate is very state specific for laws and administrative codes. Her mom was in NJ, mine in TX.

But back to you.... so how are you managing to settle this affairs? Like were you named in his will to be the executor, or were you already a signatory on his bank account and it’s POV to you, or for the Personal needs trust account set up at the NH (if he was there on LTC Medicaid there almost always is a PNA set up) were you a designated person to draw from it?
or
are you actually paying for after death stuff from your own $? If so, is there a will? And in the will are you or your hubs (his son) named executor?

have you or your hubs or whomever was his old DPOA gotten a MERP questionnaire from the state yet?

In the U.S. states are federally mandated to recoup costs from everyone who received Medicaid/ Medicare after 55. Estate Recovery laws are easy to locate online. I've heard it's only for Medicaid, whilst some states include Medicare.

Since you are wanting the money for a marker, I was wondering if your FIL was a Vet? If so, they will provide a marker but not the placement or marble that it is attached to. Actually there are two choices and one doesn’t require the marble or granite if I’m remembering correctly.