Middle school has had no affect whatsoever on me.
In fact, I failed most if not all of my classes through middle school (I was lazy, lol). But then in high school I exhausted all the hardest math and science classes, and then took classes at a college through my Junior/Senior year (right now I'm near the end of my Senior year in high school). The classes I've taken at the college so far are, Calculus III, Vector Calculus, Differential Equations, Linear Algebra, Electrical Circuit Analysis I & II & III, Computer Science I & II, Data Structures, Calculus Physics I & II & III, Linear Circuits, Micro controllers, digital logic I & II.
As you can see... Middle school literally had no affect on high school for me.
So, don't be too hard on yourself.

Colleges never look at middle school grades anyway, high school is what's most important. Good luck!