The factory images (all build LMY47O) will flash on top of any ROM or previous version of the stock software. By default they remove all your data in the process, but you can remove the wipe command if you're just going from the last official build. You will need an unlocked bootloader, though.
The Nexus 4 and LTE Nexus 7 already have OTAs floating around, so that's your best bet for a simple update. For N7 WiFi owners and anyone who needs a fresh start, grab the images at the link below. Make sure to check out our factory image roundup to see what is and isn't out. Nexus 9? Nope, that's the sole Nexus device without 5.1.