La Granja de San Ildefonso is best known for the beautiful Palace and gardens which were built by King Philip V in 1721. This place is relatively close to Madrid (1 hour and 15 minutes by car at most) and 10-minutes just up the road from Segovia. The three best days to come are July 25, May 25 and Aug. 25. The first two dates are when all of the fountains are turned on and the last one is the date of the town festivals. There are plenty of nice restaurants in town, but you'll need reservations if you come on a holiday.
A CC Wikimedia photo by pav3 of La Granja with the Sierra of Madrid in the background. I believe the Puerto de Navacerrada would be straight to the right of the photographer and Segovia would be behind them.
A photo of the valley of Valsain.
(CC photo from Wikimedia)
A map of the valley of Valsain. At the southwest end, notice the Puerto de Navacerrada which is where the trailhead of the
Camino
Schmid is found. La Granja is at the north end of the map.
Segovia is a 10-minute drive towards the west from there.
You can also have a picnic near La Granja at Boca del Asno. Or, you can go to Segovia.