South Africa won the final match of the series reaching 9-264 from its 50 overs chasing down New Zealand's 9-260, but it took until the very last ball.
Dale Steyn was out for four off the second last ball of the match leaving South Africa three to win with Ryan McLaren facing James Franklin, and McLaren hit a massive six as he remained 25 not out to get the Proteas home.
The Black Caps can certainly consider themselves unlucky, though, not to have to won the series 3-0 after facing just a five-ball 49th over in their innings. The umpires realised the mistake, but decided to ignore it and that one delivery could have made all the difference in the end.
However, winning the one-day series 2-1 was still a tremendous result for New Zealand after the disastrous Test series where they lost by an innings and 27 runs in Cape Town and by an innings and 193 runs in Port Elizabeth.
New Zealand captain Brendon McCullum won the toss in the third and last one-day clash in South Africa and had no hesitation batting first, but the Black Caps made a poor start falling to 4-68 with Martin Guptill making just five, Kane Williamson six, BJ Watling 20 and himself 13.
Grant Elliott then teamed with Colin Munro for a 61-run stand for the fifth wicket to steady things before Elliott was out for 54 from 67 deliveries.
Munro kept on going, though, and found terrific support in the form of Franklin. The pair added and 67 to take the score to 5-196 before Munro fell for 57 from just 62 balls.
While the Black Caps again lost some quick wickets with Nathan McCullum out for two, Jimmy Neesham for 13 and Kyle Mills for seven, Franklin was still there and hit New Zealand to a total of 9-260 as he remained unbeaten on 53 from 52 balls.
Lonwabo Tsotsobe claimed 4-45 for South Africa from 10 overs with McLaren also picking up 4-52 from his 10.
That set South Africa a challenging run chase needing 261 to win and the Proteas made a good start with Quinton de Kock and Graeme Smith putting on an opening stand of 83 before de Kock was bowled by Franklin for 31.
Smith remained the rock of the innings, though, with Colin Ingram (25), Faf du Plessis (19), David Miller (15) and McLaren offering him good support as he reached his 10th one-day international century.
He was eventually out for 116 when caught behind off Williamson with South Africa 6-229 needing 32 runs still with 26 balls remaining.
Rory Kleinveldt (six), Aaron Phangiso (nine) and Steyn (four) all lost their wickets, but then McLaren stepped up to hit his six on the final ball to get South Africa home.
Mills, Mitchell McClenaghan, Franklin and Williamson all picked up two wickets apiece for the Black Caps.
Du Plessis was captaining South Africa in the absence of the suspended AB de Villiers and was pleased to come away with the win even if it wasn't good for his nerves.
"It was nerve-racking at the end, but it's just nice to see that the younger guys coming in towards the end held their head and played beautifully. It was a great finish and it was probably the shot of the day to end it with a lap shot into the stands," du Plessis said.
"We needed this win. I still think we didn’t play a particularly great game, but we will take a win and there's a lot to work on for the next one-day series against Pakistan."
Black Caps skipper Brendon McCullum was delighted to come away with the series win after the horrors of the two Test matches.
"I'm incredibly proud of this young team. It has shown the fighting characteristics and style of cricket we want to play. Especially after the tough Test series to be able to bounce back with a few new faces the way we did and come away with the spoils is incredibly satisfying," McCullum said.
"I wouldn’t say we were demoralised after the Test series, we were taught a lesson and that was a great learning opportunity for us as a team. When you come up against the best team in the world and get to learn those lessons in a short space of time, it gives you an opportunity to respond.
"That's what we saw in this series and we managed to put the best one-day team in the world under pressure at times. We found out that we are actually good enough and that's massive for us."
New Zealand will now return home to host England in three Twenty20 internationals, three one-day clashes and three Tests matches starting on February 9 while South Africa remains home to take on Pakistan in three Tests, two Twenty20's and five one-dayers starting February 1.