I also find golf to be hard on my knees since I use a lot of leg drive when hitting. It stresses my knee in a different way (twisting) than tennis does. In tennis it's the lateral movement and impact on hard court that affect my knees.
If you can, play on clay as much as possible. So much nicer on the body. Also, most of the better male players at my club over 35 stick to doubles because of the shorter points and having less court to cover and leave the singles spots for the young guns .