Is Labor Important to Our Country's Wealth

Economics 101 taught that the earth, capital and labor are the sources of all wealth.  I’d like for a recognized economist to explain what would be the result for a prosperous country that gives up labor to other countries.

Much of the goods we purchase were produced in China, Indian, Indonesia and other countries.

Temple, Oklahoma, once had a clothing factory that employed 300 laborers.  The factory moved to Mexico and later to South East Asia.  The same has happened in thousands of USA communities.

Where are the workers who once produced our goods now?  How many hours of labor have been outsourced during the past 20 years? I have a feeling that the loss of wealth from labor will impair USA wealth for a larger time than can be imagined now.