The home was mostly empty from the 1930s until the 1960s. Its towering cupola was destroyed by lightning and the boat houses fell into the water. Things got so bad that passing tour boat narrators made fun of the home, telling their passengers that the house – its exterior paint peeling, extremely faded and sorely in need of a paint job — was owned by a paint manufacturing company. The family made extensive repairs in the 1960s and 70s including adding a boathouse, docks and, yes, new paint.