Which type system allows energy to enter or leave the system, but does not allow matter to cross into or out of the system?

A closed system can allow transfer of energy into and out of the system without allowing matter to transfer as well. An example of such a closed system is the Earth itself, which transfers massive amounts of energy into and out of the atmosphere while transferring almost no matter.