The Minnesota Power utility gets credit for this informative graphic on the steps to home energy efficiency. The things at the bottom of the pyramid are relatively low investment and high payoff and should be prioritized. As you ascend the pyramid, you get to progressively bigger investments.
Unfortunately, I can't get much beyond the third step. The fourth step is all about climate control, and in Los Angeles I don't use a heater or an air conditioner at all, so it doesn't really apply. At the fifth step you get into replacing appliances, which, as a renter, I'm not about to do. But if I did own a home, I think I would just start working my way up this pyramid.