I have lower back pain. Are there any exercises that I should avoid?

I do the treadmill for 30 min, and the stair climber for 20 min, and sometimes the elliptic trainer. I also do weight circuit. I am just wondering if there are things that I really need to avoid. Ohh, I have been to the Doc, 3 times. Its not like I am in pain 24-7. Mostly when I lay down and than get up. Never when I am exercising or sitting.