The system, called DeepText, is based on recent advances in artificial intelligence and a concept called word embeddings, which means it is designed to mimic the way language works in our brains. When the system encounters a new word, it does what we do and tries to deduce meaning from all the other words around it.
White, for instance, means something completely different when it’s near the words snow, Sox, House, or power. DeepText is designed to operate the way a human thinks, and to improve over time, like a human too.
DeepText was built as an in-house tool that would let Facebook engineers quickly sort through mass amounts of text, create classification rules, and then build products to help users. If you’re on Facebook griping about the White Sox, the system should quickly figure out that you’re talking about baseball, which, at a deeper level, it should already know is a sport. If you’re talking about the White House, you might want to read the news. If you use the word white near snow, you might want to buy boots, unless you also use the words seven and dwarfs. If you’re talking about white power, maybe you shouldn’t be on the platform.
Getting access to DeepText, as Facebook explains it, is akin to getting a lesson in spear fishing (and a really good spear). Then the developers wade out into the river.
