Any parents who let their kids play a contact sport are consenting to a certain level of what they are willing to define as acceptable risk.
This case raises the issue of where a line can reasonably be drawn between "acceptable" and "unacceptable" risk.
Forfeiting a game against a team because it has more talent or is better coached than your team is against the spirit of sport...getting whipped by a superior team is part of the growth experience of sports.
But forfeiting a game because the conference in which your child is playing is poorly managed and has allowed a disparity in the core profile of the teams to arise is not against the spirit of sport and is certainly consistent with responsible parenting.
Something is clearly off in this case. The whole point of a Division or Conference is to have teams that are (relatively) evenly matched, not as to talent (which can be coached or found in any kid at random) but as to the height and weight profile of the teams. Sounds like the Conference in which this team is playing has some work to do.
The parents of students on the other teams have a legitimate interest in having the issue addressed in a more productive manner than putting them in a position where they feel they have to forfeit games for the sake of their children's safety.
Perhaps there is another conference in which that larger team can play, where it would get to test itself against its true peer group, i.e., someone "its own size." Or perhaps the conference should put in a height and weight profile for its players.
As the parent of a 13 year old, who's a scrapper but also already had one diagnosed concussion, there's no way I'd let him play any contact sport against a team that belongs in another "division." Therefore, I'm not at all critical of parents who won't let their kids play that team. But, that's a suboptimal way to address the deeper issues.