"The world is full of magical things"

About me

Hi, I'm Tejas Vaidhya 👋
Building Nolano AI, stay tuned!
I am grad student in computer science at MILA and University of Montreal supervised by Prof. Irina Rish . I received my undergraduate degree from Indian Institute of Technology, Kharagpur.
My research primarily focuses on the efficiency and scaling of large language models (LLMs). I study scaling laws and explore efficient model representations, including low-bitwidth quantization and other compression strategies, to make LLMs more computationally and memory efficient.
In general, I am curious about everything. Recently, I've developed an interest in economics, psychology, and philosophy.
Email: iamtejasvaidhya@gmail.com

News

• 31 May 2023: Nolano secured the 3rd position in the Mozilla Responsible AI Challenge and received a grant of 20K USD.
• 15 Apr 2023: Nolano is one of the top 10 teams in the Mozilla Responsible AI Challenge
• 1 Mar 2023: Building nolano.org, stay tuned!
• 1 Sep 2022: Joining MILA as a Master’s student.
• 1 April 2020: Become organizer of ACL Year-Round Mentorship.
• 10 Feb 2022: Accepted at MILA for MSc in Computer science.
• 24 Sep 2021: Our paper titled “Causal Direction of Data Collection Matters: Implications of Causal and Anticausal Learning for NLP” is selected to present as main conference paper at EMNLP 20222
• 17 Sep 2021: Our system improving GPT-3 for text simplication won the best design award at AllenNLP 2021 Hackathon.
• 14 Sep 2021: Secured 1st position and cash prize of Rs. 1,00,000 in the Hackathon, Intelligence Augmentation (IA) for AI
• 26 Aug 2021: Our paper titled “Mining the Cause of Political Decision-Making from Social Media” is selected in the Findings of EMNLP.
• 7 July 2021: Attending Eastern European Machine Learning Summer School, 2021
• 1 May 2021: Mentoring two project in Google Summer of Code, 2021 and Julia Summer of Code, 2021.
• 1 March 2021: Starting Research internship at Mrinmaya’s Lab, ETH Zürich More

Publications

• "Logical Fallacy Detection",
  Zhijing Jin∗, Abhinav Lalwani∗, Tejas Vaidhya, Xiaoyu Shen, Yiwen Ding, Zhiheng Lyu, Mrinmaya Sachan, Rada Mihalcea and Bernhard Schölkopf
  Pre-Print
  Code Pdf Cite Slides Poster More
Reasoning is central to human intelligence. However, fallacious arguments are common, and some exacerbate problems such as spreading misinformation about climate change. In this paper, we propose the task of logical fallacy detection, and provide a new dataset (Logic) of logical fallacies generally found in text, together with an additional challenge set for detecting logical fallacies in climate change claims (LogicClimate). Detecting logical fallacies is a hard problem as the model must understand the underlying logical structure of the argument. We find that existing pretrained large language models perform poorly on this task. In contrast, we show that a simple structure-aware classifier outperforms the best language model by 5.46% on Logic and 3.86% on LogicClimate. We encourage future work to explore this task as (a) it can serve as a new reasoning challenge for language models, and (b) it can have potential applications in tackling the spread of misinformation



• "Leveraging Event Specific and Chunk Span features to Extract COVID Events from tweets",
  Ayush Kaushal and Tejas Vaidhya
  Oral Presentation at the 6th Workshop on Noisy User-generated Text (W-NUT) at the 2020 Conference on Emperical Methods in Natural Language Processing (EMNLP) Shared Task Winners
  Code Pdf Cite Slides Poster More
Twitter has acted as an important source of information during disasters and pandemic, especially during the times of COVID-19. In this paper, we describe our system entry for WNUT 2020 Shared Task-3. The task was aimed at automating the extraction of a variety of COVID-19 related events from Twitter, such as individuals who recently contracted the virus, someone with symptoms who were denied testing and believed remedies against the infection. The system consists of separate multi-task models for slot-filling subtasks and sentence-classification subtasks while leveraging the useful sentence-level information for the corresponding event. The system uses COVID-Twitter-Bert with attention-weighted pooling of candidate slot-chunk features to capture the useful information chunks. The system ranks 1st at the leader-board with F1 of 0.6598, without using any ensembles or additional datasets.



• "Domain specific BERT representation for Named Entity Recognition of lab protocol.",
  Tejas Vaidhya and Ayush Kaushal
  Proceedings of the 6th Workshop on Noisy User-generated Text (W-NUT) at the 2020 Conference on Emperical Methods in Natural Language Processing (EMNLP)
  Code Pdf Cite Poster More
Supervised models trained to predict properties from representations, have been achieving high accuracy on a variety of tasks. For instance, the BERT family seems to work exceptionally well on the downstream task from NER tagging to the range of other linguistic tasks. But the vocabulary used in the medical field contains a lot of different tokens used only in the medical industry such as the name of different diseases, devices, organisms, medicines, etc. that makes it difficult for traditional BERT model to create contextualized embedding. In this paper, we are going to illustrate the System for Named Entity Tagging based on Bio-Bert. Experimental results show that our model gives substantial improvements over the baseline and stood the fourth runner up in terms of F1 score, and first runner up in terms of Recall among 13 teams with just 2.21 F1 score behind the best one.




• "Hostility Detection in Hindi leveraging Pre-Trained Language Models",
  Ojasv Kamal, Adarsh Kumar, and Tejas Vaidhya
  Oral Presentation at the Constraint workshop at Advancement Of Artificial Intelligence
  Code Pdf Cite Slides More
Hostile content on social platforms is ever increasing. This has led to the need for proper detection of hostile posts so that appropriate action can be taken to tackle them. Though a lot of work has been done recently in the English Language to solve the problem of hostile content online, similar works in Indian Languages are quite hard to find. This paper presents a transfer learning based approach to classify social media (i.e Twitter, Facebook, etc.) posts in Hindi Devanagari script as Hostile or Non-Hostile. Hostile posts are further analyzed to determine if they are Hateful, Fake, Defamation, and Offensive. This paper harnesses attention based pre-trained models fine-tuned on Hindi data with Hostile-Non hostile task as Auxiliary and fusing its features for further sub-tasks classification. Through this approach, we establish a robust and consistent model without any ensembling or complex pre-processing. We have presented the results from our approach in CONSTRAINT-2021 Shared Task[21] on hostile post detection where our model performs extremely well with 3rd runner up in terms of Weighted Fine-Grained F1 Score1.



• "Spanish Pre-Trained Language Models for HealthCare Industry",
  Tejas Vaidhya and Jalaj Harkawat
  Oral Presentation at Proceedings of the Iberian Languages Evaluation Forum
  Code Pdf Cite Slides More
Currently transformer based model have shown high accuracy and good prediction on downstream tasks like Named Entity Recognition, Sentiment analysis etc. But the terminologies used in Healthcare sector such as names of different diseases, medicines and departments makes it difficult to predict with high accuracy. In this paper we are going to show a system for Named Entity tagging based on BETO (SpanishBERT). Experimental results have shown that our model gives better results than the current baseline of MEDDOPROF Shared task.



• "Causal Direction in Data Matters: Implications of Causal and Anticausal Learning in NLP",
  Zhijing Jin, Julius von KÃgelgenâ, Jingwei Ni, Tejas Vaidhya, Ayush Kaushal, Mrinmaya Sachan and Bernhard Schoelkopf
  Long Paper at the 2021 Conference on Emperical Methods in Natural Language Processing (EMNLP)
  Code Pdf Cite Slides More
The principle of independent causal mechanisms (ICM) states that generative processes of real-world data consist of independent modules which do not influence or inform each other. While this idea has led to fruitful developments in the field of causal inference, it is not widely known in the NLP community. In this work, we argue that the causal direction of the data collection process bears non trivial implications that can explain a number of published NLP findings, such as differences in semi-supervised learning (SSL) and domain adaptation (DA) performance across different settings. We categorize common NLP tasks according to their causal direction and empirically assay the validity of the ICM principle for text data using minimum description length. We conduct an extensive meta-analysis of over 100 (SSL) and 30 (DA) published studies, and find that the results are consistent with our expectations based on causal insights. This work presents the first attempt to analyze the ICM principle in NLP, and provides constructive suggestions for future modelling choices .


• "Mining the Cause of Political Decision-Making from Social Media: A Case Study of COVID-19 Policies across the US States",
  Zhijing Jin, Zeyu Peng, Tejas Vaidhya, Bernhard Schoelkopf and Rada Mihalcea
  Findings of EMNLP at the 2021 Conference on Emperical Methods in Natural Language Processing (EMNLP)
  Code Pdf Cite Slides More
Mining the causes of political decision making is an active research area in the field of political science. In the past, most studies have focused on long-term policies that are collected over several decades of time, and have primarily relied on surveys as the main source of predictors. However, the recent COVID 19 pandemic has given rise to a new political phenomenon, where political decision-making consists of frequent short-term decisions, all on the same controlled topic—the pandemic. In this paper, we focus on the question of how public opinion influences policy decisions while controlling for covariates such as COVID-19 case increases or unemployment rates. Using a dataset consisting of Twitter data from the 50 US states, we classify the sentiments toward governors of each state, and conduct controlled studies and comparisons. Based on the compiled samples of sentiments, policies, and covariates, we conduct causal inference to discover trends in political decision making across different states


• "ArP-Gen: Architectural Plan Generator",
  Tejas Vaidhya and Shubham Kumar Pandey
  Work in Progress
  Code Pdf Cite Demo More
In architecture, planning is a unique problem where the goal is to generate mapping from the inputs, namely given site boundary, required building foot print and fundamental spatial program requirements. A lot of study has been already done in the area of generative algorithm in order to create spatial floor plan layouts with the help of iterative process. In this paper, we attempt to propose first end-to-end Architectural plan generation model called ArP-Gen: Architectural Plan Generator, that create architectural plan from basic user inputs like site boundary, entrance point, and room programming and new Spatial Mapping Task (SMT), which maps spatial or semantically segmented floor plans to detail architectural plan, learning detailing and room semantics. Our other contribution also include creation of precisely annotated and colour coded image data set for the purpose of mapping spatial plan to detailed architectural floor plan layout. Experiments and evaluation of output by architects demonstrate the potential of ArP-Gen framework to generate feasible Architectural solutions.

You can also check out the other things I do here