Context Clouds, Co-occurrence, Relatedness and BERT (Dawn Anderson with Jason Barnard)

She starts with the quote 'You shall know a word by the company it keeps', which is really cool. She uses seashore as an example, which is perfect because we are sitting on the beach in Brighton with the seagulls in the background (but that isn't the reason she chooses the example, surprisingly). Dawn is a super interested in academic papers and their importance to industry, so we end up talking a great deal about that… I assume that academic papers come from university, and Dawn gently corrects me and gives several really interesting examples of academics in industry at Google, Spotify, Yahoo… Conclusion is that industry would not be funding these studies if there were no commercial potential. Her (convincing) argument is PhDs are often the precursors of commercial implementation and patents, so she has the jump on pretty much all of us. Finally, NLP and BERT - whereas NLP will look left, then right of a word, whereas BERT looks both ways at the same time and thus vastly widens the context window. And that is going to revolutionise machine understanding of context clouds, co-occurence, relatedness (and probably RankBrain). She then goes on the theorise of what is coming next, which is wildly mind-blowing (and VERY hard to keep up with her). Listening back now to the conversation, I can picture multiple jaw drops. The future is about machines understanding language more and more within the idea of entities, knowledge graphs and… context.