Too late to change schools?

My son is in grade 2 (grade 3 next year) and my other son starts prep next year.

I have been feeling like I made the wrong decision with schooling.... The school they currently and are going to go to is very large approx 600 children and my son has had behavior issues since he started.

My eldest is quite intelligent and my second struggles and I am concerned I should have chosen a smaller school.

I feel like because my son is already part way through school I shouldn't change schools however I feel like he would excel in a smaller school and behavior may improve. I feel silly that I based my decisions on school on the wrong reasons (area and having friends going to the same school from kinder). Should I change schools or stick it out...