Panesar on Thursday returned to Test ranks for the first time in nine months.
But after being selected for the second Test against Australia in Adelaide, Panesar was unwittingly the centre of another racial controversy.
Cricket Australia (CA) apologised after an inflammatory picture regarding the 31-year-old was posted on CA's official Twitter account.
CA posted a picture of four bearded men wearing turbans, with the caption: "Will the real Monty Panesar please stand up".
Panesar, a Sikh of Indian descent, was not in the picture and the tweet sparked claims of racism before it was deleted by CA.
And last weekend, a stadium announcer at the England tour match against a CA Chairman's XI in Alice Springs was fired after using a mock Indian accent to introduce Panesar.
The incidents come after Panesar was fined by police in August for urinating on nightclub bouncers in England. He later said a bitter divorce was a factor in his misbehaviour.
But to Panesar's teammates, the off-field troubles aren't an issue, fellow spinner Swann says.
"Monty is Monty. He has always been a bit left-field and a bit different to everyone else and it's one of the reasons we love him so much," Swann told reporters.
"We don't care what has happened in the last 12 months off the field.
"He is one of the boys and we embrace him as ever and we love seeing him do well."
Swann says Panesar justified his Test comeback, after being overlooked since March.
The left-armer took 1-68 from 24 overs and had a simple catch from his bowling dropped late in the day by Michael Carberry.
"His bowling was excellent," Swann said.
"It's never easy coming back in to a team because obviously a lot of spotlight goes on you, a lot of expectation.
"But I thought he applied himself really well. It was a beauty to get his wicket as well, could have had two (wickets) by the end which is unfortunate for him.
"But he did the job that we wanted him to do and that is all Monty ever does, he just turns up and plays his game."