Abstract

We propose CLIP-Head, a novel approach towards text-driven neu- ral parametric 3D head model generation. Our method takes simple text prompts in natural language, describing the appearance & facial expressions, and generates 3D neural head avatars with ac- curate geometry and high-quality texture maps. Unlike existing approaches, which use conventional parametric head models with limited control and expressiveness, we leverage Neural Parametric Head Models (NPHM), offering disjoint latent codes for the dis- entangled encoding of identities and expressions. To facilitate the text-driven generation, we propose two weakly-supervised map- ping networks to map the CLIP’s encoding of input text prompt to NPHM’s disjoint identity and expression vector. The predicted latent codes are then fed to a pre-trained NPHM network to gener- ate 3D head geometry. Since NPHM mesh doesn’t support textures, we propose a novel aligned parametrization technique, followed by text-driven generation of texture maps by leveraging a recently pro- posed controllable diffusion model for the task of text-to-image syn- thesis. Our method is capable of generating 3D head meshes with arbitrary appearances and a variety of facial expressions, along with photoreal texture details. We show superior performance with ex- isting state-of-the-art methods, both qualitatively & quantitatively, and demonstrate potentially useful applications of our method.

Abstract

Researchers have extensively studied the field of vision and language, discovering that both visual and textual content is crucial for understanding scenes effectively. Particularly, comprehending text in videos holds great significance, requiring both scene text understanding and temporal reasoning. This paper focuses on exploring two recently introduced datasets, NewsVideoQA and M4-ViteVQA, which aim to address video question answering based on textual content. The NewsVideoQA dataset contains question-answer pairs related to the text in news videos, while M4-ViteVQA comprises question-answer pairs from diverse categories like vlogging, traveling, and shopping. We provide an analysis of the formulation of these datasets on various levels, exploring the degree of visual understanding and multi-frame comprehension required for answering the questions. Additionally, the study includes experimentation with BERT-QA, a text-only model, which demonstrates comparable performance to the original methods on both datasets, indicating the shortcomings in the formulation of these datasets. Furthermore, we also look into the domain adaptation aspect by examining the effectiveness of training on M4-ViteVQA and evaluating on NewsVideoQA and vice-versa, thereby shedding light on the challenges and potential benefits of out-of-domain training.

Abstract

This study employed the P-MiSTIC method to analyze decadal temperature patterns in contiguous Peninsular India. A gridded temperature dataset from IMD spanning 1990-2020 was utilized, generating three decade-wise datasets. P-MiSTIC identified similar zones in decadal and overall datasets, revealing localized data characteristics. The 30-year trend for diurnal temperature range (DTR) showed a significant increase of 0.002oC/year for all zones, while the Western Himalayas and Karakoram range exhibited higher increasing trends of 0.019oC/year and 0.028oC/year, respectively. The DTR lapse rate over 30 years was quantified as -0.133oC/km, with a sharp decline observed from the first two decades (0.012 & 0.028 oC/km) to the third decade (-0.533 oC/km). Differential elevation-dependent temperature changes were observed, with the Karakoram region displaying the most prominent increase, suggesting the emergence of Elevation Dependent Warming in contiguous Peninsular India.

Abstract

Abusive comments in online platforms have become a significant concern, necessitating the development of effective detection systems. However, limited work has been done in low resource languages, including Dravidian languages. This paper addresses this gap by focusing on abusive comment detection in a dataset containing Tamil, Tamil-English and TeluguEnglish code-mixed comments. Our methodology involves logistic regression and explores suitablef embeddings to enhance the performance of the detection model. Through rigorous experimentation, we identify the most effective combination of logistic regression and embeddings. The results demonstrate the performance of our proposed model, which contributes to the development of robust abusive comment detection systems in low resource language settings. Keywords: Abusive comment detection, Dravidian languages, logistic regression, embeddings, low resource languages, code-mixed dataset.

Abstract

This study addresses the challenge of detecting fake news in Dravidian languages by leveraging Google’s MuRIL (Multilingual Representations for Indian Languages) model. Drawing upon previous research, we investigate the intricacies involved in identifying fake news and explore the potential of transformer-based models for linguistic analysis and contextual understanding. Through supervised learning, we fine-tune the ”muril-base-cased” variant of MuRIL using a carefully curated dataset of labeled comments and posts in Dravidian languages, enabling the model to discern between original and fake news. During the inference phase, the fine-tuned MuRIL model analyzes new textual content, extracting contextual and semantic features to predict the content’s classification. We evaluate the model’s performance using standard metrics, highlighting the effectiveness of MuRIL in detecting fake news in Dravidian languages and contributing to the establishment of a safer digital ecosystem. Keywords: fake news detection, Dravidian languages, MuRIL, transformer-based models, linguistic analysis, contextual understanding.x

Abstract

Detecting abusive language in multimodal videos has become a pressing need in ensuring a safe and inclusive online environment. This paper focuses on addressing this challenge through the development of a novel approach for multimodal abusive language detection in Tamil videos and sentiment analysis for Tamil/Malayalam videos. By leveraging state-of-the-art models such as Multiscale Vision Transformers (MViT) for video analysis, OpenL3 for audio analysis, and the bert-basemultilingual-cased model for textual analysis, our proposed framework integrates visual, auditory, and textual features. Through extensive experiments and evaluations, we demonstrate the effectiveness of our model in accurately detecting abusive content and predicting sentiment categories. The limited availability of effective tools for performing these tasks in Dravidian Languages has prompted a new avenue of research in these domains. Keywords: abusive language detection, sentiment analysis, multimodal analysis, video analysis, Dravidian languages.

Abstract

Machine Learning models face increased concerns regarding the storage of personal user data and adverse impacts of corrupted data like backdoors or systematic bias. Machine Unlearning can address these by allowing post-hoc deletion of affected training data from a learned model. Achieving this task exactly is computationally expensive; consequently, recent works have proposed inexact unlearning algorithms to solve this approximately as well as evaluation methods to test the effectiveness of these algorithms. In this work, we first outline some necessary criteria for evaluation methods and show no existing evaluation satisfies them all. Then, we design a stronger black-box evaluation method called the Interclass Confusion (IC) test which adversarially manipulates data during training to detect the insufficiency of unlearning procedures. We also propose two analytically motivated baseline methods~(EU-k and CF-k) which outperform several popular inexact unlearning methods. Overall, we demonstrate how adversarial evaluation strategies can help in analyzing various unlearning phenomena which can guide the development of stronger unlearning algorithms.

Abstract

Prior work has shown that pretrained language models often make incorrect predictions for negated inputs. The reason for this behaviour has remained unclear. It has been argued that since language models (LMs) don’t change their predictions about factual propositions under negation, they might not detect negation. We show encoder LMs do detect negation as their representations across layers reliably distinguish negated inputs from non-negated inputs, and when negation leads to contradictions. However, probing experiments show that these LMs indeed don’t use negation when evaluating whether a factual statement is true, even when fine-tuned with the objective of changing outputs on negated sentences (Hosseini et al., 2021). We hypothesize about why pretrained LMs are inconsistent under negation: when the statement could refer to multiple ground entities with conflicting properties, negation may not entail a change in output. This means negation minimal pairs in different training samples can have the same completion in pretraining corpora. We argue pretraining may not provide enough signal to learn the distribution of ground referents a token could have, confusing the LM on how to handle negation.

Abstract

No Results Found

Abstract

Second language learners often make grammatical mistakes, which can impact the performance of Spoken Language Understanding (SLU) systems. SLU systems consist of two key components: Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU). This work focuses on the effects of grammatical errors which were synthetically generated by manipulating Parts of Speech (POS) tokens on the NLU module. Our analysis comprises three main aspects. Firstly, we assess the impact of grammatical errors on the overall performance of NLU systems, specifically in the domains of Intent Detection and Slot Filling. Secondly, we investigate NLU performance concerning POS tags. Lastly, we utilize an Attention-based NLU model to evaluate the significance of different POS. This study evaluates NLU models on ATIS and SNIPS datasets for Intent Detection and Slot Filling tasks with introduced grammatical errors, leading to a significant performance drop as large as 4.7% for Intent Detection and 5.1% for Slot Filling. However, training on datasets with synthetic errors helps mitigate this drop, and the models are shown to be sensitive to increasing levels of corrupted tokens. Additionally, attention analysis reveals Proper Nouns’ higher weights in determining intent, while specific POS corruption, particularly Auxiliary Verbs and Nouns, is more likely to cause intent misclassification.