Santiago often expresses pity for the fish and calls it his friend. Why then, does he want to kill it so badly

Because that's just the natural order of things - fishermen hunt the fish, and the fish resists and tries to run away. Santiago's persistence resembles the marlin's - the one is determined to kill, the other is determined to run away if possible. In a way, they live the same kind of life - harsh and ruthless, and share the same fate, lonely and isolated from the rest of the world. This is why Santiago sees the fish as his fellow suffering (and resisting) soul.