I know about the famous yellow book that lists private gardens in the UK open to the public, but didn’t realize we have the same thing here. For some reason, no Virginia gardens are included in the Garden Conservancy Open Days program, but any travel plans could be cross-checked against the list. Filter by state, and a pleasant diversion might appear.
Caroline's Garden and Travel Record 