Given that our solar system is now about 4.6 billion years old, when will we need to worry about the sun running out of hydrogen for fusion?

The approximate lifetime of a star like our Sun is in the range of 10.3 billion years. At a current age of 4.6 billion years, this will leave (10.3 - 4.6) or approximately 5.7 billion years before the supply of hydrogen fuel (and thereby, helium) will run out.