The Black Caps, set 259 to win on Sunday, chased down the target with seven balls to spare.
McCullum was unbeaten on 69 and opener Guptill, who had earlier retired hurt on three after pulling a hamstring, was 27 not out.
Despite his handicap, Guptill, who rejoined the match with New Zealand at 219-7 and needing 41 from 25 balls, struck five boundaries, including a six over third man, and also hit the winning run.
When the pair came together, England appeared in the box seat.
The contest began tilting in the tourists' favour when top-scorer Kane Williamson was run out for 74 in a mix-up with McCullum to leave the home side at 142-4.
While he did survive one dropped catch, Williamson was in fine touch as he played shots to all parts of Seddon Park in a 99-ball innings that contained six boundaries.
McCullum made up for his error in calling Williamson through and then sending him back by keeping New Zealand's victory pursuit alive, hitting six fours and three sixes in his 61-ball knock.
Guptill was one of two injury concerns for the Black Caps as the three-match series heads to Napier on Wednesday.
Seamer Mitchell McClenaghan, who had the match's best figures of 4-56, fell victim to a side strain in grabbing his last wicket in his ninth over, and went off two balls later.
Earlier, England, after losing the toss, had looked on track for a big total before dropping their last eight wickets for 74 runs.
They had made a slow start, with skipper Alastair Cook managing just four before returning to the pavilion.
However, Ian Bell and Jonathan Trott set about laying the groundwork for the heavy hitters down the order.
Bell had a couple of reprieves but was finally caught behind for 64 to be the first of James Franklin's three scalps.
Joe Root joined Trott at 95-2 and the pair were looking ominous as they put on 89 together before England were set back by two quick wickets.
Kyle Mills bowled Trott for 68 and then had Eoin Morgan brilliantly caught by Guptill at backward point for one.
From there, England lost momentum, with Root's innings ending on 56 when he charged at Franklin and was bowled.
McCullum then produced a superb keeper's catch to remove Chris Woakes for 17.