What is the density of the rock

You would have to find the mass and volume of the rock and then do mass divided by volume to get your density

The scale reads 3 so it is displacing 5 kg of water

find volume of rock using volume of water it displaces:

volume of rock = 5 kg (m^3 / 1000 kg) =.005 m^3

mass of rock = 8 kg

density = 8 / 0.005 (kg/m^3)

1600 Kg / m^3