The hosts had reached 1-77 at stumps off 33 overs, still 134 runs behind after being made to follow on at the Basin Reserve on Saturday.
England allrounder Stuart Broad claimed 6-51 as New Zealand succumbed in their first innings for 254, still 12 runs short of the follow-on mark.
Opener Peter Fulton, who looked badly out of sorts in his first innings, had reached 41 and put on 52 for the second wicket with Kane Williamson (16).
Earlier, Hamish Rutherford provided spinner Monty Panesar with his first scalp of the Test when caught at leg slip for 15.
Showers are forecast for the next two days, which would break a lengthy drought in Wellington and potentially save a New Zealand side who have been outplayed throughout.
England made batting look relatively easy in scoring 465 on an even pitch yet their seam attack has generated significant bounce and pace over four sessions.
Broad, 26, showed he has fully overcome a heel injury which curtailed his recent tour of India by recording the third-best innings figures of his career.
He found a spot short of a length which the home side struggled to handle, ripping through the last three wickets for 15 runs with the second new ball.
Broad rated that late burst as his most important because of its follow-on implications.
"It (follow-on) is not often enforced because the bowlers tend to like a bit of a rest and it's good to get their batsmen back out in the field and build a big lead with scoreboard pressure," he said.
"But with the weather around, it's unsure how much cricket will be left in the next two days. That was the reason behind it."
The tourists claimed five wickets in the second session, including Brendon McCullum and BJ Watling in their 60s.
McCullum struck a lively 69 off 94 balls, sharing a 100-run partnership with wicketkeeper Watling (60) after New Zealand were teetering at 5-89 early on day three.
The skipper struck nine fours and one six in raising his 27th half-century before edging Finn to second slip.
He will be annoyed not to have pushed on, having registered scores from 69 to 79 in six of his past seven innings on the English tour across all forms of the game.
Watling reached a fourth half-century but his dismissal, caught behind off Broad, exposed the tail.
Despite the sound knock, Watling was disappointed with himself for not guiding New Zealand past the follow-on mark.
"I think that was a crucial stage that I needed to be there at the end," he said.
"There's still two days of cricket and we've got to do a lot of things a lot better than in the first innings and put a couple of big partnerships on the board."
Spinner Bruce Martin was unbeaten on 21 but he received little support from Tim Southee (3), Neil Wagner (0) and Trent Boult (2).
New Zealand began the day at 3-66 and reached 5-169 at lunch after overnight pair Williamson (42) and Dean Brownlie (23) were dismissed within the space of four deliveries.