Maybe the vilagers will form worker unions against you … Or maybe if you missuse them too much they will aggrevate iron golems against you, that would be quite funny. At first I thought maybe it could cause agrevated iron golems to spawn but then that could be used as an iron golem farm