Nadal, the world number one's 6-1, 6-4, 6-4 win in Valencia drew the tie level at 2-2, before veteran David Ferrer battled to a marathon 7-6 (7/1), 3-6, 7-6 (7/4), 4-6, 7-5 victory over Philipp Kohlschreiber to finish the job.
Two breaks in each set were enough for Nadal to seal an impressively comfortable victory over world number four Zverev in what was only the 31-year-old's second outing since returning from a hip injury.
Nadal said after the match he is still regaining sharpness.
"The first set was fantastic," Nadal said. "There were times when I lost a bit of control because the match was tough, I had not competed for a long time and to maintain the level of the first set was complicated for me. "But in general it was a very important match and I am very happy."
Nadal was forced to retire from his Australian Open quarter-final against Marin Cilic in January but, back on his favoured clay, the 16-time Grand Slam champion showed little sign of rust against Zverev, whom he has now beaten four times out of four.
It was also the 24th consecutive match Nadal has won in the Davis Cup.
He beat Kohlschreiber in straight sets on Friday after Zverev had overcome Ferrer. Germany then took a 2-1 lead when Tim Puetz and Jan-Lennard Struff won the doubles rubber against Feliciano and Marc Lopez.
But Nadal kept up his end of the bargain on Sunday and the 36-year-old Ferrer showed all of his trademark never-say-die qualities to keep Spain's hopes of a sixth Davis Cup title alive.
Kohlschreiber twice fought back from a set down, but former French Open runner-up Ferrer broke in the 11th game of the decider and held his nerve on serve after almost five hours on court.
Elsewhere, defending champions France booked a home semi-final against Spain as Lucas Pouille claimed a hard-fought win over Fabio Fognini in Genoa to give the visitors a 3-1 victory in the tie.
"I won without playing my best tennis, those are the best types of victories," Pouille said after his first Davis Cup win over a top-20 player.
World number 11 Pouille, who had beaten Andreas Seppi in five sets on Friday, came from behind to see off Fognini 2-6, 6-1, 7-6 (7/3), 6-3 and send Yannick Noah's France into the semis for the third time in four years.
"18 months ago, he was playing for the first time, and now he is our leader on and off the court," captain Noah said of Pouille.
Nicolas Mahut and Pierre-Hugues Herbert had cruised to a straight-sets victory over Fognini and Simone Bolelli in the doubles on Saturday to put France in control, despite Fognini's bad-tempered four-set win against Jeremy Chardy.
Croatia will host the United States in the other semi-final in September, after Cilic thrashed Mikhail Kukushkin 6-1, 6-1, 6-1 in Varazdin.