WASHINGTON — A second top Internal Revenue Service official has announced plans to leave the agency amid the controversy over the targeting of tea party groups.

An internal IRS memo says Joseph Grant, commissioner of the agency’s tax-exempt and government entities division, will retire June 3. Grant joins Steven Miller, who was forced to resign as acting IRS commissioner on Wednesday.

As part of his duties, Grant oversaw the IRS division that targeted tea party groups for additional scrutiny when they applied for tax-exempt status.

Grant joined the IRS in 2005.