A swift defection of at least four Senate Republicans has all-but-doomed the chance of Herman Cain to win a seat on the Federal Reserve’s board of governors, serving as one of the most striking rebukes to President Trump’s personnel choices since he took office in 2017.

Sens. Mitt Romney of Utah, Lisa Murkowski of Alaska, Cory Gardner of Colorado and Kevin Cramer of North Dakota all said they would oppose Cain’s appointment to the Fed. Republicans control 53 seats of the 100 chamber Senate, and losing the support of four members means Cain would need help from Democrats, which appears unlikely.

“If I had to vote today, I would vote no,” Cramer said Thursday.

Trump in recent days has said he wants to add Cain to the Fed’s seven-member board of governors at a time when he has been repeatedly chiding the central bank’s leadership for their management of interest rates. Trump has also said he wants to nominate conservative economist Stephen Moore to the Fed. Moore and Cain are both vocal Trump supporters and have called for slashing interest rates to make the economy grow even faster.

Cain ran for president in the 2012 Republican primary, but lost momentum amid accusations of sexual harassment, among other things. Cain has denied wrongdoing and recently became a full-throated supporter of Trump.

During his first 18 months in office, Trump relied on Treasury Secretary Steven Mnuchin and other economic advisers to pick Fed nominees, but he has wrested control of the process in recent months after blaming Mnuchin for steering him towards Jerome Powell as Fed chairman. Powell was easily confirmed by the Senate, but Trump has told aides he regrets appointing Powell to the job because the Fed took several steps last year to raise interest rates and Trump has blamed this for slowing economic growth.