Antonin Scalia, who was known to be fond of Latin, was probably familiar with the phrase nemo judex in causa sua, or “No one should be a judge in his own cause.” His death at a Texas resort where he was staying free has raised questions about how seriously the U.S. Supreme Court takes that notion.

Although justices are not above the law, they have exempted themselves from the code of conduct that applies to all other federal judges. When lower-court judges face a motion to step aside owing to a possible conflict of interest, other judges settle the question. Supreme Court justices, however, get to decide questions about their own impartiality by themselves, sometimes arriving at dubious conclusions.

In 2004, Scalia declined to recuse himself from a case involving Vice President Dick Cheney and the energy industry, shortly after Cheney and Scalia had gone on a duck hunting trip together to a private camp owned by an oil industry executive. (Scalia voted in favor of Cheney.) Right or wrong, the fact that Scalia made the decision himself did nothing to bolster public confidence in the court’s impartiality.

Justices, like members of Congress, may also accept junkets – and they often do. A recent analysis of their travel showed that from 2004 to 2014, justices took over 1,000 trips paid for by outside groups. But the rules regarding disclosure are unclear.

The Supreme Court does comply with the financial disclosure requirements that apply to all federal judges, but Chief Justice John Roberts has suggested that they only do so out of the goodness of their hearts. The court, Roberts wrote in 2011, has “never addressed whether Congress may impose those requirements on the Supreme Court” – a veiled warning to Congress that it shouldn’t press its luck.

Congress needn’t be cowed. If the court rejects its authority to establish clearer ethical standards for the judiciary, a constitutional amendment may be necessary. The public deserves a court that is more transparent and accountable.