If we have a look at the article about the Grand Condé, which came out in the Historama magazine in 1963, we can find the following information :
In a château, nowdays destroyed, used to leave lady Bouthillier de Chavigny, who educated a five years old child protected by her uncle, the Cardinal of Richelieu.
This child, Claire-Clémence de Maillé-Brézé, was the children of the Marshal of France, Urbain de Maillé, Marquis de Brézé, who had married the niece of the Cardinal, Nicole du Plessis. This household was divided. Urbain was spending most of his time hunting and reading. His wife was insane and died in 1635. The old Marshal set up home with one of his lackey widow.
We can understand why the Cardinal of Richelieu wanted to protect his niece from this situation. Sickly and a bit mentally retarded, Claire-Clémence was however cherished by the Cardinal. This one made the plan to give her an important name by the marriage and his choice concerned the young Duke of Condé...".
Although the dates don't match completely, it could be possible that the destroyed was the old Beaujeu. In this château should have leave the young Claire-Clémence de Maillé-Brézé, future wife of the Grand Condé, because this one has been educated by a lady Bouthillier de Chavigny, owner of the château de Beaujeu.