Where do I turn for protection from my brother who lives on the same property as my mom but has never actually cared for her?

Sorry you're going through this. Talk to the attorney who added your son to the deed or to an elder care attorney who is versed in trust and estate law. Depending on the facts, you might want to report him for elder abuse if he is pestering your mother about her financial affairs.

Can you return to your mother's attorney who had assisted with adding the grandson (your son, who is now 20?) to the deed?

He can speak to whether the deed is sound.
Whether or not your mother was competent at the time of executing the deed.
Whether or not, you, as his mother, would have a conflict of interest.
And, whether or not there is a will that backs up her wishes for her grandson to inherit the property.

The attorney can write a letter to your brother.

Arguing about an inheritance before your mother passes? Is this merely sabor-rattling? Does brother even have a legal leg to stand on?
Have an attorney review the will.

Is the home free of a mortgage?

Was your Mother fully competent when she added her grandson to her property deed? How old was the grandson? Did you have any part in getting the grandson added onto a deed while serving as your Mom's POA? Because if the latter is true you may be convicted of self-enrichment, which amounts to elder abuse.
You have apparent been put on notice by a "lawyer letter" that your brother is considering taking you to court. You now need not the advice of this Forum but of a good attorney. A POA has a Fiduciary Duty to act only in the best interest of the ELDER. Not in his or her own interest or in the interest of his or her family. So you need advice at once. And indeed, depending on the circumstances, the Grandson's name may need to be removed.
This is a serious legal question. See an attorney. Wish you luck and hope you will update us when this is cleared up.

Another thought; your profile indicates your mother is 93. This kind of sibling conflict can't be good for her and could negatively affect her health.

If your brother persists in harassing and threatening you, you might want to ask the attorney who you will need to consider an injunction preventing him from harassing both you, your family, and your mother. A PPO (Personal Protection Order) might be necessary to accomplish this.

1. On what basis are you "required" to remove your son from the deed? If your mother wanted him to be an heir to the property, that was her choice, not your brother's.

2. Not to be nosy, but I'd really like to know if this demand was in the letter, signed by an attorney? If so, have you checked out the attorney to get insight into his/her background, including whether or not he/she's ever been cited by the state bar?

3. What criteria did the letter establish, and what supporting data was presented for the charge of elder abuse? Is there something else going on in this situation that needs to be aired to assist others in responding?

4. If your brother is going to attempt to force you out of the picture by legal manipulation, you are more likely going to need to retain an attorney, including to avoid being "dragged through court". Fight fire with fire.

5. An attorney experienced in estate litigation can protect you better than you can yourself, as your brother's attorney could very likely institute proceedings to remove you as proxy under a DPOA, and attorneys are better at hostile proceedings than individuals w/o legal training.

6. What are the issues surrounding your brother's living on the property? Is he living in the same house as your mother? Is there a separate building that he occupies? Are there any contracts between your mother and your brother establishing the terms of his living on her property?

More information on why he's living there in the first place could help establish what his goals are and why he's pursuing them in such a hostile manner.