Why can we assume that the gathering of white citizens grew more raucous as the evening progressed?

In the novel Invisible Man by Ralph Ellison, the main character, an African American, suffers from a harsh exhibition of prejudice and discrimination caused by the color of his skin. He was invited to give the same farewell speech he said to his high school classmates in front of a town's meeting which had a majority of white male citizens. In its place, there was a fight which involves his classmates and him. At the end, he tries to recite his speech and he mistakenly uses "social equality" instead of "social responsibility" and that's when the crowd starts to get angry at him, reminding him of his place in this world. The crowd started to get more upset since they had a misconception of African Americans at that time, they believed they were nothing but an isolated group of society and they were considered a low-life population; additionally, the behavior of the Invisible Man and his classmates was not their best analyzing the standards of morally correct citizens they wanted to achieve. Subsequently, the Invisible man tries to find his place in this world, a place opposite to the opinion that a white America has of him.