In 1540 a page named Antonio Ramirez di Montalvo came from Spain in the train of the Cardinal Don Giovanni di Toledo, Bishop of Burgos and uncle to the Duchess Eleonora, wife of Cosimo I. de’ Medici. The Cardinal passed some weeks in Florence, and when obliged to proceed to Rome left the lad Antonio, who was ill, in the charge of the Duchess. She took him into her service, and the clever young Spaniard gained the good graces of the Duke, became tutor to the young Prince Francesco de’ Medici, and married Donna Giovanna di Ghevara, one of the ladies-in-waiting of the Duchess. In 1568 he bought a house with a tower and a garden, in Borgo degl’Albizzi, from Giovanni Bonafedi, and afterwards two or three smaller ones adjoining, and then built the palace we now see. By the Duke’s order the overseer of the works at the Duomo supplied him with the wood necessary for the roof, doors, windows, etc., and it appears that Cosimo also gave considerable sums towards the cost of building. With good reason, therefore, the grateful Spaniard placed his master’s arms, instead of his own, on the façade with the inscription: COSMOS;MAGN;FLOR;ET;SEN;D;II.

The palace is decorated in grafite, and on a line with the arms are the favourite emblems of the Duke, a capricorn, a tortoise with a sail and two crossed anchors. It has always been attributed to Bartolomeo Ammanati, though neither Baldinucci nor any other contemporary writers give the name of the architect. But as Ammanati was at that time (1566–1569) engaged in rebuilding the bridge of Sta. Trinita, and was Cosimo’s favourite architect (he built the courtyard of Palazzo Pitti), it is more than probable that he was also employed by his courtiers. When Cosimo I. instituted the order of S. Stefano Don Antonio Ramirez di Montalvo was one of the first knights created, and was given a rich Commenda in perpetuity. The name of his grand-daughter, Donna Leonora, is well-known in Florence as the foundress of the convent of Le Quiete, the great school where the daughters of the nobility are educated by nuns called the Signore della Quiete, all ladies of good birth who do not take solemn vows. The life of Donna Leonora, written in 1740, is curious reading. As a baby she reproved her nurse if she was idle, and as a girl she must have been a trial to her father confessor from her scruples and incessant fears that everything was a deadly sin. She had visions, during which Our Lord and the Virgin Mary talked with her; she predicted the restoration to health, or the death, of many people and performed many miraculous cures. A singular mixture of Spanish bigotry and Tuscan common-sense was Donna Leonora. Many of the rules she laid down for the teaching of the young ladies at Le Quiete would be considered admirable at the present day. “Let them be trained to order,” she writes, “and to cleanliness. Teach them to put away their clothes properly.... Let them not be lazy or negligent, but sprightly and diligent. They should know how to sweep and clean a room, how to make a bed, to take charge of the linen and woollen things, and of what pertains to the furniture of the house. They should learn how to nurse a sick person with care, according to the doctor’s orders, and how to prepare all kinds of milk dishes, cordials, candied fruit and sweet pastry. It would be well, too, that they should know how to sew and wash fine linen, such as is used in a sacristy. Not that they are to perform such fatiguing duties, save for exercise and their own pleasure; on the contrary, I desire that they should be served in seemly fashion, but so that in case of necessity they may be able to direct how things ought to be done, for that is most necessary for the good ruling of a household.”50 After all these excellent, but rather commonplace, precepts, the bewildered reader is suddenly plunged into chapter after chapter of prophecies made, and miracles worked, by Donna Leonora; the miracles continued even after her death in 1659. She did, however, good work in founding the convent of the Signore della Quiete. Her brother, Don Antonio, died in 1581, and was succeeded by the eldest of his five sons. According to his friend Vasari, he drew well, and was a munificent patron of art. The last of the family, another Antonio, inherited the tastes of his ancestor. He was President of the Academy of Drawing in Florence, and Director of the Palatine gallery in Palazzo Pitti; it was under him that the gallery was first thrown open to the public on Sundays and half-holidays in 1833.

In 1739 the palace was let to Baron von Stosch, an antiquary, and a spy in the service of the English government to watch the doings of the Pretender. That delightful old gossip, Sir Horace Mann, notes in 1757, “Baron Stosch is dead at last.... His effects consist only in his Collection, which is very great, and worth a large sum. It is to be offered to the Emperor. He has appointed me and Abbé Buonacorsi his executors, and has left him a picture, and me a cameo, which I might have bought some years ago for six zecchins.” The palace still belongs to a collateral descendant of the old Spanish family, the Count Matteucci Montalvo.