The loss, which was their fourth in a row in the Premier League, left the Baggies two points off the relegation places.
Clarke becomes the fourth Premier League manager to lose his job this season.
"We have reluctantly come to the decision to relieve Steve of his duties after very careful consideration," said West Brom technical director Richard Garlick.
"It has been well documented that we have not had the rub of the green in certain games this season but that does not cloud the generally disappointing points return during this calendar year, culminating in today's fourth-successive defeat at Cardiff."
The 50-year-old Clarke, a former assistant coach at Chelsea and Liverpool, took over at West Brom in June 2012.
West Brom have won only seven of their 34 Premier League games during the calendar year although they did finish eighth last season, their best performance in the top flight.
Joint-assistant head coach Keith Downing will assume control of first-team affairs until a new full-time appointment is made, the club said in a statement.
Clarke's team lost their season-opener 1-0 at home to Southampton at the Hawthorns.
But under his guidance, West Brom went on to celebrate a 2-1 victory against champions Manchester United, their first win at Old Trafford in 35 years.
Clarke admitted after Saturday's game that his team were not enjoying great fortune.
"We created some chances as we normally do, but at the moment things aren't quite going for us," he admitted.
Sunderland's Paolo Di Canio was the first Premier League manager to go this season followed by Ian Holloway at Crystal Palace and Martin Jol at Fulham.
Managerial departures in the English leagues in the 2013/2014 season so far following Steve Clarke's sacking at West Brom on Saturday:
Kevin MacDonald (Swindon)
Greg Abbott (Carlisle), Paolo Di Canio (Sunderland), Nigel Clough (Derby)
David Weir (Sheffield United), Martin Allen (Gillingham), Kevin Blackwell (Bury), Tony Mowbray (Middlesbrough), Ian Holloway (Crystal Palace), Chris Kiwomya (Notts County)
Brian Laws (Scunthorpe), Guy Whittingham (Portsmouth), Richie Barker
(Crawley), Sean O'Driscoll (Bristol City), David Flitcroft (Barnsley)
Martin Jol (Fulham), Dave Jones (Sheffield Wednesday), Owen Coyle (Wigan), Steve Clarke (West Brom)