According to the preamble, the phrase "a more perfect Union" refers to

According to the preamble, the phrase "a more perfect Union" refers to improving upon a previous attempt at creating a new union.

The phrase "to form a more perfect Union" was stated to refer to the shift to the Constitution from the Articles of Confederation. The Union had a federal government established by the Constitution that had enough power to act upon citizens directly.

This is the preamble of the Constitution:

We the People of the United States, in Order to form a more perfect Union, establish Justice, insure domestic Tranquility, provide for the common defence, promote the general Welfare, and secure the Blessings of Liberty to ourselves and our Posterity, do ordain and establish this Constitution for the United States of America.