Lawyers for a former child model accusing ‘X-Men’ director Bryan Singer of sex abuse say they want out of the case because their relationship with the accuser has deteriorated.

Attorneys Jeff Herman and Mark Gallagher filed a motion in U.S. District Court in Honolulu on Tuesday asking to withdraw as counsel for Michael Egan III. They say they no longer talk to Egan except through a new lawyer.

Egan, now 31, accuses Singer in a lawsuit of sexually abusing him during trips to Hawaii when he was 17 in 1999. Singer has denied the allegations.

Egan previously dropped three similar Hawaii lawsuits against other Hollywood figures.

The lawsuits were filed suit under an unusual state law that created a window for civil cases in sex abuse cases where the statute of limitations has passed.