Which was a major source of controversy for President Andrew Jackson?

I don't know what options you had, but controversy concerning President Jackson was his marriage to Rachel Donelson Robalds, who, it turned out, was still married when she and President Jackson started living as a married couple (she later divorced and they married lawfully.)