The Grey Friars or Franciscans, were established in Old Winchelsea in the early 13th century. When the town was destroyed by the sea, the Grey Friars accompanied the townsfolk to their new quarters on Iham Hill. After the dissolution the property came into private hands.

The present house, now a private house, was built in the early nineteenth century on the site of earlier buildings, and has been much altered. The ruins of the chancel of the chapel built 1310-1320 still remain and traces of the cloister and monastic buildings south of the chapel may still be traced on the lawn in dry summers. The chancel ruins are no longer open to the public. This is currently the location for the annual Grey Friars Opera held in aid of Charity each June