What concerns might arise when congress delegates decision-making authority to these unelected leaders of federal agencies, sometimes called the fourth branch of government?

One of the main concerns is that those people won't work to do what's best for the people since they weren't elected by the people and might thus be looking out for their self interest or someone else's self interest if those people have lobbyists, and they often do.