Back when the phrase came about, all of the canons had to be tied up and restrained. Recoil would send them rolling back from the gunports, and the gun crew would reload, then winch them forward into position and lash them down again for the next shot. A loose cannon had the potential to roll out of place while firing, careen about the gun deck, crush the gun crew, and generally snatch defeat from the jaws of victory.