I haven't used the N4 and I'm sure it's quality, but did it really have any big, new feature for that time? If it didn't, I think that's what the Nexus title should entail. Bring something new to the table and something that other OEMs can look at and make standard within a few months.
Otherwise, so long as they make the S5 with a Play Edition variant, I'll be happy to buy it if the next Nexus ends up being kinda "meh." I just think if it's coming from Google, it shouldn't just be a minor software update, like ICS and JB kind of have been. Sure, there are tweaks under the hood, but how about some features like the S4 brought in, like one of the best cameras, the eye-sensor stuff, etc. That's what makes these phones exciting, whether people use them or not. These Nexus devices should set standards.