The castle was built in the 15th century by Ralph, 3rd Lord Cromwell[?], Lord Treasurer of England, on the site of a previous 13th century castle. The centerpiece of the surviving ruins is the 110 foot tall red-brick medieval keep tower.
The castle was left to the National Trust by Lord Curzon of Kedleston on his death.