Neural Information Retrieval

Information retrieval fundamentals, information retrieval evaluation, word representational learning, word embeddings, language modeling, Word2Vec, FastText, word embeddings in information retrieval, query expansion with word embeddings, application to patent retrieval, neural networks, neural network methods in NLP, Sequence Modeling with CNNs (Convolutional Neural Networks) and RNNs (Recurrent Neural Networks): modeling word n-grams with CNN, hierarchical CNNs, recurrent neural networks, simple RNN, RNN as Encoder, LSTM, LSTM gating mechanism, encoder-decoder architecture, attention mechanism, encoder-decoder, and attention, Transformer and BERT: transformer architecture, contextualization via self- attention, transformer – positional encoding, masked language modeling, BERT, and extractive question answering, Neural Re-ranking: text-based neural models, properties of neural IR models, neural re-ranking models, re-ranking evaluation, KNRM (Kernel based Neural Ranking Model), and convolutional KNRM, Transformer Contextualized Re-ranking: web search with BERT, Re-ranking with BERT, splitting BERT - PreTTR and ColBERT, transformer-kernel ranking, TK (Transformer-Kernel) ranking, TKL (Transformer- Kernel for Long documents)and a hybrid approach IDCM (Intra-Document Cascade Model), domain specific information retrieval applications, Dense Retrieval Models – Knowledge Distillation: neural methods for IR beyond re-ranking, dense retrieval, dense retrieval and re-ranking, dense retrieval lifecycle, BERTDOT model, nearest neighbour search, nearest neighbour search – GPU brute-force, approximation of nearest neighbour search, knowledge distillation, DistilBERT, distillation in IR, deep learning based recommender systems.