A federal judge denied Hawaii businessman Albert S.N. Hee’s request Monday for more time to turn himself in to begin serving his prison sentence for tax fraud.

That means Hee needs to arrive at the U.S. Bureau of Prison’s Rochester Federal Medical Center in Minnesota on Wednesday morning.

In July a federal jury found Hee, 62, guilty of filing false personal income tax returns for 2007-2012 and for corruptly impeding the IRS from correctly calculating and collecting his taxes.

Between 2002 and 2012, the government maintains, Hee had his company, Waimana Enterprises Inc., pay approximately $2.3 million in personal and family expenses that Waimana deducted as business expenses but which Hee did not claim as income.

U.S. District Judge Susan Oki Mollway in January sentenced Hee to 46 months in prison, fined him $10,000 and ordered him to pay the IRS $431,793 in back taxes. She gave him until April 4 to turn himself in.

Two months later, at Hee’s request and with agreement from the federal prosecutor, Mollway extended Hee’s turn-in date to Wednesday.

Hee’s lawyers filed papers last week asking for another extension, to July 29. They said the reason for the request was to allow Hee to get evaluated for his reaction to new medication for his food allergies.

Hee has said he takes Benadryl for his allergies and carries epinephrine in case he goes into anaphylactic shock. Those drugs are not available in federal prison, so defense lawyer Michael Purpura told Mollway that an allergy doctor who tested Hee at the University of California, San Francisco, prescribed two other drugs that are allowed in prison.

Purpura said Hee has an appointment with another allergy specialist at UCSF next month to see how he reacts to the new drugs.

Beyond that, Purpura said, tests conducted at UCSF and by his primary doctor in Hawaii have resulted in a “breakthrough” that could lead to a diagnosis as to why Hee has allergic reactions to certain foods.

Assistant U.S. Attorney Lawrence Tong told Mollway he agreed to the first extension because of the tests Hee’s doctor conducted in March. Tong asked whether he should expect a call for more tests and extensions if Hee does not tolerate the new drugs and doctors are not able to make a diagnosis.

“Enough is enough,” he said. Tong asserted that Hee needs to serve his prison sentence to appreciate the seriousness of his actions, “and the time has come.”

Mollway told Hee and his lawyers that Hee should have anticipated going to prison when a jury found him guilty nearly a year ago, and should have obtained information about drug-use policies before sentencing in January.

She said the Bureau of Prisons went along with Hee’s initial request and her recommendation for Rochester FMC because of Hee’s medical condition. Following the BOP’s designation, however, Hee asked to be incarcerated in a different facility in New York, where it’s not as cold as Minnesota, Mollway said.

When Hee’s lawyers filed the extension request last week, Mollway said she was on her way to a federal court sentencing conference. At the conference, she said, she learned that not only is the Rochester FMC in the same city as the renowned Mayo Clinic, but also that the two hospitals have a working relationship. That would give Hee at least the same level of care he can get at UCSF Medical Center, and at taxpayer expense, she said.

“No one is telling me only UCSF can do the tests,” Mollway said.

Of the Waimana Enterprises money that went to personal and family expenses, more than $1.6 million in wages and benefits was paid to Hee’s wife and three children, who provided little or no work for the company; more than $736,900 was spent on tuition and housing for Hee’s three children while they were attending college on the mainland; $119,909 went to credit card charges for personal and family travel; and $96,000 paid for twice-weekly massages.

In 1995 the Hawaiian Homes Commission awarded Waimana a nonbid, exclusive license to provide telephone and telecommunications services on state Department of Hawaiian Homelands properties. Hee created separate subsidiary companies to purchase the equipment, install the infrastructure, provide the service and even insure their loans.

One of those subsidiaries, Sandwich Isles Communications, collects subsidies from the Federal Communications Commission that are funded by a fee charged to all telephone users nationwide. In 2005 Sandwich Isles’ subsidy was about $13,700 per customer, or 100 times the average cost for rural phone service on the mainland.