You need at least 1 day for Toledo and half a day for El Escorial. Visit them as separate trips.
If you must combine El Escorial with another place, combine it with Avila instead of Toledo. El Escorial and Avila are in the same direction from Madrid.
Toledo and Escoril are on the opposite sides!!
It is impossible to combine Toledo and El Escorial in one day. First of all Toledo deserves more than a day by itself. And on top of that Toledo and El Escorial are not close to each other. It can take 2,5 hours just to travel from Toledo to El Escorial.
If you combine them, you will spend at least 4 to 6 hours traveling between the places. This will give you no time to see anything.