Fagan said on Thursday he was shocked and disappointed when White asked for a release from his contract with two years remaining.
"This came out of the blue," he told ABC radio in Canberra.
The South African was brought to Canberra in 2011, steering the team to a Super Rugby final just two months ago.
Fagan conceded that being overlooked for the Wallabies head coach role in July in favour of Ewen McKenzie had affected White.
"And that's sort of at the heart of all this," Fagan said.
"He missed out on that job and he was extremely disappointed. At the time, when it happened, he was contemplating a release.
"I had a long chat with him, as did others. He settled back down and recommitted to the club - including expressing that to the board just prior to his vacation (to South Africa).
"Obviously back home, he said he's had a rethink and he said he doesn't feel as though he can progress in international rugby through Australian Super Rugby and the sacrifice of being away from his family who he's very close to.
"If that carrot's not there, he felt as though that was just too much and his heart wasn't going to be in it.
"I tried to talk him out of it. My chairman spoke with him; (captain) Ben Mowen spoke with him. We tried our best to turn that around.
"But for those primary reasons, he just said he couldn't make that decision."
The CEO suggested the club had little choice but to grant White a release, saying: "We can't go and kidnap him and drag him back across the ocean and force him to coach".
Fagan all but confirmed World Cup-winning Wallaby Stephen Larkham or Laurie Fisher would replace White as head coach.
Larkham is the team's current attack coach, while Fisher - who was head coach from 2005-08, had handled the forwards under White.
"So we've got two head coaches in waiting, effectively, and I think it would be unlikely that we would go outside the group - given what significant role they've played in the last two years," he said.
"We're not looking for change. We're looking to continue to evolve what's in place and those guys have the most intimate knowledge of that."
Fagan added that he did not expect any player to walk out on the Brumbies as a result of White's departure.
"There are no clauses in contracts that relate to Jake White or any other staff in the organisation. We're not a club who does that," he said.