DENVER — An effort to decriminalize polygamy by the family from the TV show “Sister Wives” reached its highest level of the legal system so far Thursday as federal appeals judges questioned a lawyer for Utah about whether the state needs to ban plural marriages.

A three-judge panel of the 10th U.S. Circuit Court of Appeals in Denver put some of its most pointed questions to Utah’s attorney as it heard arguments on the state’s appeal of a ruling that struck down key parts of a law banning polygamy.

But they also asked how the law hurt Kody Brown and his four wives in a state with a longstanding policy against prosecuting otherwise law-abiding adults in polygamous marriages.

Judge Nancy Moritz asked state lawyer Parker Douglas why the law is used so rarely if Utah claims it is needed to curb abuses such as underage unions.

“Doesn’t that sort of negate those state interests?” she said.

Prosecutors do not bring charges often, but when they do, the law helps in gathering evidence and strengthening cases against those abuses, Douglas said. Polygamy can be associated with crimes such as sexual assault, statutory rape and exploitation of government benefits, prosecutors have said.

Brown and his wives are urging the court to uphold a ruling that found key parts of Utah’s bigamy law forbidding cohabitation violated the family’s right to religious freedom.

Bigamy, or holding multiple marriage licenses, is still illegal. Brown has a license for only one of his marriages and says his other unions are spiritual.

The family says its TLC reality show “Sister Wives” reveals that polygamous unions can be as healthy as monogamous ones and argues that making such marriages a crime violates the right to privacy and freedom of religion.

The case concerns “important issues of personal and religious freedom that we are committed to defend,” Jonathan Turley, a George Washington University law professor who is representing the family for free, said after the hearing.

Utah does not believe the law infringes on free speech or discriminates based on religion, Douglas told the judges, who asked why Utah had said so little about those constitutional issues.

“Maybe you disagree with them. Maybe you don’t. But I don’t see it in your brief. And that’s a problem,” Judge Scott Matheson told Douglas.

The judges also asked Turley what harm the law causes. State attorneys have said they don’t plan to charge the family if the law stands, but Turley said a chilling threat lingers.

He noted that the Browns moved to Nevada for fear of prosecution and would like to return to Utah someday, which he described as the home of their religion. He also said the Browns have been publicly labeled felons by Utah officials.

“That’s the type of harm that can be hard to imagine if it’s never happened to you,” Turley told the judges.

The legal fight comes after a landmark 2013 ruling by U.S. District Judge Clark Waddoups, which removed the threat of arrest for plural families.