Avinash Madasu

Avinash Madasu
Chapel Hill, NC

I am a graduate student and research assistant at University of North Carolina, Chapel Hill in the computer science department. I am fortunate to be advised by Prof. Gedas Bertasius. My research focuses on multi-modal AI. Previously, I worked with Prof. Shashank Srivastava on inductive biases in language models.

I was a senior engineer at Samsung R & D Institute India - Bangalore, for 3 years where I worked on Bixby (Samsung's voice assistant). My job was to design NLU models to help improve Bixby. I had the opportunity to work under Prof. Asif Eqbal on multi-modal dialog systems. I graduated from National Institute of Technology Tiruchirappalli with Bachelors in Computer Science. While pursuing UG studies, I worked closely with Prof. Sivasankar on statistical feature extraction techniques for sentiment analysis.

I am interested in contributing to open source frameworks that empower neural networks. I am also a member of Distributed Deep Machine Learning Community and reviewer of gluonnlp, Amazon's NLP library.

Research Interests

My area of interests lie in Multi-modal AI especially in the intersection of Natural Language Processing, Computer Vision and Robotics. I had worked in the areas of multi-modal dialog systems, domain adaptation, text classification etc.

I like to reproduce results from the papers published in CVPR, ICCV, Neurips, ICLR etc.

News

Paper accepted at EMNLP (findings).
New pre-print on video retrieval. Work done at Intel AI Labs as summer intern.
Paper accepted at ACM 22 .
New pre-print on dialog systems.
Paper accepted at ICPR 22.
New pre-print on interactive video retrieval systems.
I will be joining Intel AI as a research intern in summer 2022.
Serving as a reviewer for ACL rolling review .
Working as RA under Prof. Shashank on inductive biases in language models.
Started my Masters program at UNC Chapel Hill.
Paper submitted to ACM TOMM journal on end-to-end slot identification in multi-modal dialog systems.
Served as reviewer for ACL'21

Pre-prints

	Improving video retrieval using multilingual knowledge transfer Avinash Madasu*, Estelle Aflalo, Gabriela Ben Melech Stan, Shao-Yen Tseng, Gedas Bertasius, Vasudev Lal, arxiv In this paper, we propose a framework MKTVR, that utilizes knowledge transfer from a multilingual model to boost the performance of video retrieval.
	A Unified Framework for Emotion Identification and Generation in Dialogues Avinash Madasu, Mauajama Firdaus and Asif Ekbal arxiv In this paper, we propose a multi-task framework that jointly identifies the emotion of a given dialogue and generates response in accordance to the identified emotion.

Peer Reviewed Publications

	What do Large Language Models Learn beyond Language? Avinash Madasu and Shashank Srivastava EMNLP (findings), 2022 [Paper] [Code] In this paper, we investigate if pre-training on text also confers these models with helpful `inductive biases' for non-linguistic reasoning. On a set of 19 diverse non-linguistic tasks involving quantitative computations, recognizing regular expressions and reasoning over strings
	Learning to Retrieve Videos by Asking Questions Avinash Madasu, Junier Oliva and Gedas Bertasius ACM MM, 2022 [Paper] [Code] [Poster] We propose a novel framework for Video Retrieval using Dialog (ViReD), which enables the user to interact with an AI agent via multiple rounds of dialog. We also demonstrate that our proposed approach also generalizes to the real-world settings that involve interactions with real humans, thus, demonstrating the robustness and generality of our framework.
	A Syntax Aware BERT for Identifying Well-Formed Queries in a Curriculum Framework Avinash Madasu, Vijjini Anvesh Rao ICPR, 2022 arxiv We propose a BERT based model to improve the search queries. We further imbibe BERT with parts-of-speech information and train the model in a Curriculum Learning way. The best approach achieves accuracy of 83.93%, outperforming previous state-of-the-art at 75.0% and reaching close to the approximate human upper bound of 88.4%.
	Sequential Domain Adaptation through Elastic Weight Consolidation for Sentiment Analysis Avinash Madasu, Vijjini Anvesh Rao ICPR, 2020 arxiv We present a anti-curriculum based sequential domain adaptation. The sequential domain adaptation is trained elastic weight consolidation. The proposed approach outperformed previous SOTA architectures and the training time is very less compared to previous methods. It is also architecture agnostic.
	A Position Aware Decay Weighted Network for Aspect based Sentiment Analysis Avinash Madasu, Vijjini Anvesh Rao NLDB, 2020 arxiv We propose a model that leverages the positional information of the aspect. The proposed model introduces a decay mechanism based on position. This decay function mandates the contribution of input words for ABSA. The performance is measured on two standard datasets from SemEval 2014 Task 4.
	Sequential Learning of Convolutional Features for Effective Text Classification Avinash Madasu, Vijjini Anvesh Rao EMNLP, 2019 arxiv We propose a Sequential Convolutional Attentive Recurrent Network (SCARN). Extensive experiments establish that SCARN outperforms other recurrent convolutional architectures with significantly less parameters. Furthermore, SCARN achieves better performance compared to equally large various deep CNN and LSTM architectures
	Efficient Feature Selection techniques for Sentiment Analysis Avinash Madasu, Sivasankar E Multimedia Tools and Applications, 2019 arxiv In this paper, we aim to study the performance of different feature selection techniques for sentiment analysis. Ensemble techniques are applied on classifiers to enhance the performance on sentiment analysis. We show that, when the best FS techniques are trained using ensemble methods achieve remarkable results on sentiment analysis and outperforms neural networks.
	Gated Convolutional Neural Networks for Domain Adaptation Avinash Madasu, Vijjini Anvesh Rao NLDB, 2019 arxiv In this paper, we propose Gated CNN for domain adaptation in sentiment analysis. Extensive experimentation on two standard datasets reveal that training with Gated Convolutional Neural Networks give significantly better performance on target domains than regular convolution and recurrent based architectures.
	Effectiveness of Self Normalizing Neural Networks for Text Classification Avinash Madasu, Vijjini Anvesh Rao CICLing, 2019 arxiv In this paper we aim to show the effectiveness of proposed, Self Normalizing Convolutional Neural Networks(SCNN) on text classification. We analyze their performance with the standard CNN architecture used on several text classification datasets. Our experiments demonstrate that SCNN achieves comparable results to standard CNN model with significantly fewer parameters. Furthermore it also outperforms CNN with equal number of parameters.
	A Study of Feature Extraction techniques for Sentiment Analysis Avinash Madasu, Sivasankar E IEMIS, 2018 arxiv We perform a study on the performance of unsupervised feature extraction techniques TF-IDF and Paragraph2Vec for sentiment classification.

Projects

	Utilizing BERT Intermediate Layers for Aspect Based Sentiment Analysis and Natural Language Inference Avinash Madasu, Fine-tuned BERT Base Uncased model on SemEval 2014 datasets Laptop and Restaurant for Aspect Based Sentiment Analysis. Hidden representations are taken from CLS token in each of the 12 hidden layers. These representations are trained using LSTM. Achieved near State-of-the-art results of 84% on Restaurant and 77% on Laptop (Metric:Accuracy).
	Adaptive Methods for Nonconvex Optimization Avinash Madasu, Successfully reproduced the results of the paper \Adaptive Methods for Nonconvex Optimization". Implemented Yogi optimizer as proposed in the paper. This implementation is included in the open source project pytorch-optimizer".
	Highway Networks Avinash Madasu, Implemented a Highway Network for performing Image Classification on CIFAR-10 dataset. Achieved an accuracy of 70.35% with a simple 3 layer Highway Network. Implemented Yogi optimizer as proposed in the paper. This implementation is included in the open source project pytorch-optimizer".
	All you need to know about Normalization Avinash Madasu, Studied the effects of using different Normalization techniques like Batch Normalization, Layer Normalization and RMS Normalization on CNN for Image Classification. Evaluated pros and cons of each of the Normalization techniques and their dynamics while training CNN.
	Recurrent-Neural-Filters Avinash Madasu, Implemented a class of Convolutional Neural Networks that utilize LSTM networks as convolutional filters. Achieved a competitive accuracy of 88% on SST-2 dataset..

Design taken from here & inspiration from here