Michele,
Going to the gym regularly (the one for exercise, not sex) will pay dividends for the rest of your life. Easy for me to say, of course. Does it feel good when you go even if you don't achieve the running speed that you used to do? For me there is something about the exercise itself that just makes me feel better. I do have goals too but even when I don't achieve them I know that the exercise itself is good for me.
As far as the man goes, it sounds to me like there is something you worry about being with him. Maybe he feels good in bed but in your heart you don't trust him. I think you should listen closely to your heart and give him up. Find a new one (maybe at the gym!) so your body and your mind will be happy.
Be well,
Emma