The tourists, following on 396 runs behind, reached 2-168 at stumps on the third day after a more determined batting display second time around.
They still require 228 to make the home side bat again.
Opener Kirk Edwards, who made 59, said they felt they had to put things right in the second innings after a below-par effort.
"We wanted to go out and fight to restore some pride. Getting skittled out for such a low total is not good for any team."
Tim Southee, who took 4-43 in the first innings, said despite being well ahead in the Test they had a lot of work ahead of them to win their first Test under captain Brendon McCullum.
"Very rarely do you see a side fold twice in a game, so they're always going to come out harder in the second innings and they'll have a bit more of a point to prove.
"It's tough, it's Test cricket, it's what it's about.
"We've got a chance to win a game and it is going to be tough over the next day or two," he said.
On a benign wicket the Black Caps hit their straps early at University Oval to claim three wickets before lunch after the tourists, replying to New Zealand's massive 9-609 declared, resumed on 2-67.
Southee grabbed three wickets before Trent Boult and Ish Sodhi claimed two apiece after the interval to dismiss the tourists for 213.
Only Shivnarine Chanderpaul, who became the seventh man to bring up 11,000 Test runs in making 76, got past 50 before he shouldered arms to a ball from Boult to fall lbw to a delivery that was clipping the top of middle stump.
After his wicket the end came quickly and McCullum wasted little time in enforcing the follow-on with his bowlers delivering only 38.1 overs until then.
Southee ended with 4-52 with new-ball partner Boult grabbing 3-40 and Sodhi 2-63.
Boult struck first second time around to have Kieran Powell caught at third slip but a 117-run stand between Kirk Edwards (59) and Darren Bravo helped stymie the fall of wickets.
Sodhi, who had been guilty of bowling a few loose balls, slid a quicker ball to trap Edwards lbw after Bravo had survived a referral to the television umpire.
Marlon Samuels also endured a referral off Sodhi to reach the close on 17 while Bravo was unbeaten on 72.