NEW YORK » If Ben Jay had known the depth of the task confronting him at the University of Hawaii, he said Thursday, he might not have taken the job as its athletic director.

Jay, who announced his resignation Tuesday, two years after accepting the job, said, "I thought I knew what I knew, but it wasn’t until I really got there and went through some of it that I got more of an idea of some of the underlying issues about why we are in the position we’re in."

Jay said, "Honestly, if I knew more I may not have taken it."

Jay cited mounting frustration with fundraising at UH; the firing of his boss, Tom Apple, as Manoa chancellor; and the toll speculation about his job status was taking on his family for the decision to resign.

Jay’s first extensive comments since the resignation came while he was attending the IMG Intercollegiate Athletics Forum in New York City with more than 100 fellow ADs and athletic administrators. Jay, in a suit and tie, was a panelist on issues confronting non-Power Five conference schools, an appearance scheduled before his announced departure.

Ohio State athletic director Gene Smith, Jay’s former boss and mentor, who was in attendance, said, "I just know that if (Hawaii) is not the toughest job in college sports it is awfully close."

Jay said he met Dec. 5 with interim Manoa Chancellor Robert Bley-Vroman and discussed his evaluation, the budget situation and whether to retain football coach Norm Chow.

Jay said he supported keeping Chow for a fourth year despite an 8-29 record, which he said Bley-Vroman promised to take under advisement. Jay did not receive a good evaluation but said he was not asked for his own resignation.

Jay said he told the chancellor he was considering stepping down, in part, because of family issues, many connected to his job.

Over the weekend, Jay said he decided to leave UH and informed Bley-Vroman on Monday afternoon.

The move caught many at UH by surprise despite more than a month of rampant rumors about whether he would get to begin the final 12 months of his three-year contract in January.

Jay was hired in December 2012 by Apple, who lasted until this summer, when he was replaced by Bley-Vroman.

"I was hired by Tom and we shared some similar thoughts," Jay said. "It was like we were a team, a good team, and without Tom it just didn’t seem like we could move in the direction we had been planning."

But despite Apple coming up with $17 million to retire a decade-long athletic department deficit, the 21-team, $32 million program has continued to run in red ink that could top $4 million for the current fiscal year that ends June 30, 2015.

A large part of the fiscal difficulties was attributed to football underperforming on the field and at the gate. But Jay said the fundraising efforts of the booster club, Koa Anuenue, and his own initiatives, including calling on 50 major backers, did not meet expectations.

"We had goals and we didn’t meet them," Jay said. "There are some things I think I could have done differently, especially in some things I said."

He acknowledged he went too far in August when he talked about the possibility of dropping football but said he was trying to get across "the financial crisis that we’re in."

"Everybody is waiting for somebody else to take care of the problem, somebody else to fund us," Jay said. "And the message I tried to get across is that we all have to work together or we all lose. If you want UH athletics to succeed, you have to support us, and people took exception to that."

He said he was surprised by the state of the program when he arrived in January 2013.

"Especially for those of us who come from big programs, you figure there is a certain minimum amount of support and infrastructure," Jay said. "But as I got into it from a business perspective, you realize that even some of the minimal stuff is just not there. All the things that we have at the tip of our (fingers) at other places, we didn’t have here so you have to build it. Which is what we were trying to do here."

The task, he said, was wearing on his family.

"My family has supported me my entire career, has seen me go away, travel and I just felt like it was time for me to step back a bit," he said. "My kids (ages 15, and twin girls soon to be 12) are becoming teens and I don’t want to miss that. When you hear from your kids ‘We kids, we never see you,’ it hits you. You sit back and think about that."

Jay said, "One of the things my son and I like to do is go out to the airport, sit there on the roof and watch with binoculars as the planes take off and land and listen to radio traffic. We don’t get to do that anymore. The same thing with my daughters, what time do we have to share? You only get these times once."

So, Jay said, "I’m willing to take a step back in (my career) if it means I get more time with them."

He plans to stay at UH until June, and said he has begun looking for a new job.

"I have to find the right situation," Jay said.

Following Jay’s appearance at the IMG Forum Thursday, several peers and past colleagues caught up with him, offering hugs and pats on the shoulder to express condolences and support over his decision to resign from UH.

Smith, of Ohio State, said, "Hawaii has got some problems, so to expect that you are going to hire somebody and they are going to walk in and all of a sudden your $3 million deficit is going to go away when, frankly, you have some systemic problems in your model, well, that’s just not going to happen.

"When you have the stadium issues that you have, when you have the travel costs that you have and you want to be able to compete, there is a reality to those issues," Smith said. "When I look at the job, frankly, it is not just the AD. They have to understand that, hey, wait a minute, we need to rethink this model on how we fund Hawaii."

Bley-Vroman said in a statement that Hawaii is not alone in its fiscal struggles.

"Almost every mid-major and some major Division I programs are dealing with the same challenges because of the changing landscape of college athletics," he said, adding that Hawaii’s location in the Pacific Ocean further compounds UH’s problems.