How much is intelligence affected by upbringing?

I grew up in a really awful family and never did well in school. I never really developed a work ethic and I don't understand the concept of being passionate about stuff so it has really dumbed me down. I've always thought this was because of how my parents raised me, but I'm still trying to improve myself.

What do you guys think about this idea? Could it just be that I'm a 'tard and didn't have a chance to begin with?