Imagine living in a 7 bedroom share-house with strangers that just stay in their room. The house is fairly full, all is well, everyones got their personal space, no one really frequents the living room or the kitchen or the game room that often accept that one random dude. Now a bunch of other people come along and start to fill up space. It even works out to where you have to share your room with a couple of them. I know when I'm in an enclosed living area, the less people there are around stinking the area up the better.
Sure that's a personality trait of the type of fish I would be. Others may be more social. You could read up on which fish do well with each other, but that will only take you so far until you run into other forum posts about fish acting aggressive towards the fish that they're supposed to get along with. Adding little by little will not only allow the bio load to adjust properly, but it will also allow fish time to get used to their new tank mates. Since it's a 120, you could probably get away with more than one at a time. Since you have mostly pairs, you could probably throw in a pair a week. Getting used to a new environment with a buddy will probably go over better. Perhaps those fish will be less likely to get picked on.