Inflation-weary consumers have also been slammed by high borrowing costs, but the Fed is cautious about sticky inflation.