Defense Secretary Mark Esper said he is sticking to a plan that could draw down U.S. forces in Africa.