Fronting a media conference in Sydney on Friday, the swimmers admitted using the sedative Stilnox as part of a "bonding session".
They now face sanctions from the Australian Olympic Committee (AOC) which could include bans and fines.
The AOC banned athletes using Stilnox and other prescription sedatives about three weeks before the start of the London Games.
The six freestyle relay squad members - James Magnussen, Tommaso D'Orsogna, Cameron McEvoy, James Roberts, Eamon Sullivan and Matthew Targett - admitted to all taking one Stilnox tablet during their bonding session at the pre-Games camp in Manchester.
Australian Swimmers' Association general manager Daniel Kowalski read out a statement on the relay swimmers' behalf at Friday's media conference.
"We all acknowledge that at the time Stilnox was consumed, it had been recently prohibited by the AOC," Kowalksi read.
"We own up to it ... and are deeply sorry for it."
Favoured to win the gold medal in London, a team of Magnussen, Targett, Sullivan and Roberts swam the final and finished fourth.
The swimmers believed the bonding session was "harmless fun".
All six will now also face a Swimming Australia integrity panel.
Roberts later said that despite Sullivan's comment that they all took one Stilnox pill, he had never taken the drug and did not do so then.
"I didn't take Stilnox that night," Roberts said.
The team admitted some of them played pranks on other team members such as knocking on their doors but denied entering any rooms of teammates.
The swim team's head coach Leigh Nugent was told of the pranks the following morning.