After her two and a half year stay at Sheffield Castle, Mary Stuart was, for the next eleven years, moved around some of the other houses of the Earl of Shrewsbury. First to Sheffield Manor, then to Buxton, where she stayed in Buxton Old Hall on three occasions. This house no longer stands but the site is occupied today by the "Old Hall Hotel". She would also return to Sheffield Castle, and on four occasions to Chatsworth Hall. Mary was at Sheffield Castle when the Battle of Lepanto took place, 7th October 1571. She would have been at Chatsworth when St Edmund Campion and his companions were executed, 1st December 1581.
She also stayed once at Worksop Manor, pictured above as it would have been a century later. George Talbot had built this house but it was burnt down in 1761.
In the summer of 1584 she was moved back to Sheffield Castle for the last time. It was at this time that the Priviy Council decided to change her custodian from the hospitable Geroge Talbot to the more strict governance of Ralph Sadler, someone who was more sychophantic of the Elizabethan regime.
On 2nd September 1584 Mary was taken to Wingfield for the last time, her new custodian having received instructions to hold her more securely again in Tutbury Castle.