If my mother dies and there is no will, but I have been the caregiver living with her, can they force me out of the house when she dies?

By they do you mean your siblings? With no will, someone will have to become Administrator. This person does everything the Executor would have. The difference, the State determines who inherites. If your an only child you will. Siblings, Moms estate gets split. Meaning if she only has the house, it may have to be sold and the proceeds split between her children.

Even if they allow you to remain in the house, do you have the finances to maybe pay them rent, pay taxes, utilities and upkeep on the house?

Who is "they"? Your other siblings? If there's no Will, the house and your Mom's remaining estate will need to go through probate and then the court will decide who inherits the house. I don't see why the estate wouldn't be split evenly between you all. The house would need to be sold and the procedes distributed, or you'd need to buy out the others. You don't get to keep living there. If your Mom passes and is on Medicaid at the time, Medicaid puts a lien on the house that will need to be satisfied by the inheritors or next owner(s).

Does she have a husband, or other children besides you? Is the house paid for, and is your mother the owner, or does she rent?

Most likely yes. They may have to formally evict you which will take at least 30 days.

But honestly, it depends on how many heirs, who they are, what state you are in etc.