WASHINGTON – Secret Service Director Mark Sullivan apologized Wednesday before a Senate panel for the prostitution scandal that has embarrassed his agency and overshadowed President Obama’s April trip to the Summit of the Americas, saying it was the result of a few agents doing “really dumb things,” and not a sign of a workplace culture that turns a blind eye to misbehavior.

Skeptical lawmakers, however, questioned whether the carousing in Cartagena, Colombia, was truly a one-time incident.

The Senate Homeland Security Committee revealed 64 allegations of sexual misconduct against Secret Service employees over the past five years, including a report of non-consensual sex and of soliciting a prostitute.

“I apologize for the misconduct of these employees and the distraction that it has caused,” Sullivan said, referring to the agents who took prostitutes to their hotel rooms on a trip ahead of Obama’s visit. “The men and women of the U.S. Secret Service are committed to continuing to live up to the standards that the president, the Congress and the American people expect and deserve.”

Sullivan sought to portray the event as rogue agents most likely influenced by alcohol and “the environment,” but Senate Homeland Security Committee Chairman Joe Lieberman, a Connecticut independent, and Sen. Susan Collins of Maine, the panel’s top Republican, were nearly incredulous, though they had no proof that it was otherwise.

“It is hard for many people, including me, I will admit, to believe that on one night in April 2012 in Cartagena, Colombia, 12 Secret Service agents there to protect the president suddenly and spontaneously did something they or other agents had never done before,” Lieberman said.

Collins noted that the men had openly registered the women at their hotel and had gone out partying with their supervisors.

“That surely sends a message to the rank and file that this kind of activity is tolerated on the road,” said Collins, who called the behavior “morally repugnant.” She added that the “numbers involved,” as well as the supervisors, “lead me to believe that this was not just a one-time event. Rather, the circumstances unfortunately suggest an issue of culture.”

Sullivan’s testimony came as The Washington Post reported that four employees are fighting their dismissals for misconduct, arguing that they’re being punished for behavior that the agency has long condoned.

Sullivan noted that the story had “cited numerous anonymous sources” and said he’d encourage anyone with information to report it to investigators.

“I’m confident this is not a cultural issue, this is not a systemic issue,” he said.

A review of the agency’s disciplinary records found the 64 complaints mostly involved sexually explicit emails or having such material on a government computer, Lieberman said. He noted that “discipline was imposed in most of the cases.”

Sullivan said law enforcement had conducted an “intense” investigation into the report of non-consensual sex and decided not to proceed with charges.