Oh yeah mate, i think about stuff like that all the time here..
My Mrs took her patents on a guided tour of Castle Drogo, it's a national trust place about 5 miles from us, the most modern proper castle in England...during the tour the guide asked my wife where she lived, when she said the village she new our house abd went on to give quite a lot of info about the place and how interesting historically it is...Mrs got a bit embarrassed, but yeah it's mad to think what has been seen in this house, there is priest hole in one of the walls back to Oliver Cromwell times...