Loaded/unloaded doesn't matter from the magazine's perspective. As far as wear goes, it's using them that wears them out, not sitting on a shelf. I prefer loaded myself. An empty magazine does no good when it is needed.
I have a good number loaded, however I also have a larger number brand new in the wrappers. I will keep it that way for the foreseeable future. Most every gun I own that takes a one has a ample supply of new, unused magazines saved for a rainy day.