There's ups and downs to it all. If you want to be able to move around, having your NV optic on your gun is going to turn out badly. Also keep in mind that if your only NV optic is on your rifle, you're going to point your rifle at everything just to look around for game.
There's a reason that traditional (daylight) hunters glass with binoculars and not their rifle scopes.
There's a reason that traditional (daylight) hunters glass with binoculars and not their rifle scopes.