Douglass declares to his audience that "This Fourth [of] July is yours, not mine." What does he mean in this statement?

Douglas means that the 4th of July is a celebration of freedom, but how could African Americans celebrate a day of freedom if they were still enslaved? The audience spoken to was primarily white, so in turn, the white population of America may celebrate their freedom on the 4th but for all the enslaved blacks it was not a day to rejoice because they had not yet gained their freedom