About 1250 the Servites bought so much more land than was necessary for their church, convent and orchard, that a large space was left in front of the church. In winter the mud was knee deep, and in summer the faithful were enveloped in clouds of dust when they went to mass. So the friars petitioned the Captains of the Guelph party to make a paved road from the Via de’ Servi to the S.S. Annunziata. In 1464 the Prior of the Order decided to sell some of the waste land near the Via de’ Servi, and the first buyer was Puccio Pucci. He did not carry out the covenants of the sale, and the land was sold to Roberto de’ Ricci in 1515; some thirty years later his sons ceded their rights to the brothers Griffoni. Ugolino Griffoni, secretary to Ramazzotto, the confidential adviser of Cosimo I., held, among other rich benefices, that of Maestro dell’ Altopascio and had the title of Monsignore as an Apostolic Pronotary. He, like many of the Duke’s courtiers, called in Bartolomeo Ammannati as his architect, and the contract between them of 4th September 1563 still exists in the State archives.40 Ten years later the marble coat-of-arms, which is now on the façade in the Piazza, was placed above the central window in Via de’ Servi. The balcony was one of the first works by Gian Bologna, but the palace was only entirely finished in 1772 by Pietro Griffoni.

Cinelli writes in the Bellezze della Città di Firenze, “This beautifully proportioned and ornate palace, with a fine frieze under windows of the Doric order, was built by Bernardo Buontalenti;” but he cannot have seen the above-mentioned contract, nor can Vasari have known of it, as he states that the architect was Giuliano di Baccio d’Agnolo. This is impossible, as Giuliano died two years before Ugolino Griffoni demolished the shops belonging to the first owner of the land. Additional proof that Ammannati was the architect of the fine palace, one of the few instances in Florence of an unplastered red brick building, exists in the Riccardiana library, in a book treating of arithmetic, geometry, etc., in which Ammannati made architectural sketches. Among them are a sketch of chimney-piece, the plan of a loggia, and some drawings of doors, all marked “for I’Altopasso.”

In 1800 Gaetano Griffoni sold his family palace to the Marquess Ferdinando Riccardi at whose death it went to his heir the Marquess Mannelli (who took the name of Riccardi). He sold it to the Antinori family in 1847. When, about forty years later, the palace was bought by Cav. Gattai and his son-in-law Signor Budini, but little remained of the beautiful friezes; the stone-work of the windows was crumbling away, and the cornice had not been completed. The palace has been admirably restored by the architect Boccini, who among other things did away with the shutters, which disfigured the façade and were destroying the fine ornamentation of the windows.