...
While holding the button down, reconnect the power supply to the device.
Wait for checkpoints to appear in the serial console, ~20-40 seconds.
...
You can hold the button indefinitely, however as long as you hold it, the device will keep rebooting this is expected behavior.
...