Little now remains of Rylstone Hall but the site. "Some garden flowers still, as when Whitaker wrote, mark the site of the pleasaunce. The house fell into decay immediately after the attainder of the Nortons; and, with the estates here, remained in the hands of the Crown until the second year of James I., when they were granted to the Earl of Cumberland. Although Wordsworth makes the Nortons raise their famous banner here, they assembled their followers in fact at Ripon (November 18, 1569), but their Rylstone tenants rose with them."