One place I'd be into exploring is Centralia, Pennsylvania. There's been a fire burning in the mines beneath Centralia since 1962 and there's enough coal down there to keep it going for 250 more years. With less than 10 residents remaining and most buildings demolished or in various states of deterioration, Centralia is on it's last legs- not even qualifying as a dot on today's maps.
It may not be the most romantic honeymoon destination but I did take him to the putrid dead-fish laden shores of the Salton Sea on our first date and he did say it was by far the best first date he'd ever been on. So, maybe hazardous areas of abandonment and decay is our thing.Photos by: Urban Ghosts, Nile Guide, Boiteaoutils