A New Jersey teacher has been arrested after a potential threat to staff and students shut down an entire school district on Monday, authorities said.