Abstract

Supervised methods in deep learning approaches hinge on the assumption that the training and testing data are sampled from the same distribution. However, this is not the case in most realistic scenarios and thus leads to poor performance when these models are deployed in domains with different data distribution than the training set. Unsupervised Domain Adaptation (UDA) tackles the problem of adapting the data distributions between a labelled source domain and an unlabeled target domain. In contrast, Semi-supervised Domain Adaptation (SSDA) assumes a partially labelled target domain, a more realistic scenario in many computer vision tasks. Domain randomization is a popular approach wherein models are trained on synthetically generated data. With complete control over the synthetic data generation process, domain randomization introduces randomness in various properties of the object as well as the scene. Similar to data augmentation techniques in deep learning, the hope here is to introduce invariance for different non-causal features of the data and even nudge the model towards learning the causal correlations for the task at hand. In this thesis, we explore and study various approaches to domain adaptation. First, we present the image level domain adaptation methods, which use image-level manipulation or transformations to achieve domain invariance. We first analyze the domain randomization approach used in an object detection setting. For this, we use synthetically generated data and train a FasterRCNN model aimed at the object detection task. Domain randomization helps boost the performance of object detection models, and a model trained entirely on synthetic data outperforms the one trained on real data. With fine-tuning, the performance of the model trained on synthetically generated data increases drastically. Next, we extend the work on domain adaptation in the frequency domain, wherein the image level adaptation occurs in the frequency domain. To this end, we propose new combination strategies to combine the frequency components. We propose masking techniques which consider the frequency of the components in the combination process. Fourier domain adaptation techniques have seen some success in the image segmentation task from synthetic like GTA5 [1] and SYNTHIA [2] to realistic domains like Cityscapes [3]; however, these domains contain syntactically similar images. For the synthetic dataset that we use, we find that these frequency domain bases stylization methods do not improve performance over the domain randomization method. Finally, we present two novel methods for domain adaptation using feature level alignment. One of the primary challenges in SSDA is the skewed ratio between the number of labelled source and target samples, causing the model to be biased towards the source domain. Recent works in SSDA show that aligning only the labelled target samples with the source samples potentially leads to incomplete domain alignment of the target domain to the source domain. In our first approach, we train the source and target feature spaces separately. To ensure that the feature space of the target domain is generalized well, we employ semi-supervised methods to leverage the labelled and unlabeled samples. The Domain Adapters, which are parametric functions, are then trained to learn the feature level transformation from the target domain to the source domain. During inference, we use the target domain’s feature extractor and then pass the features to the respective Domain Adapter for that target-source pair. The transformed feature representation in the source domain is then fed to the source classifier. We show that keeping the feature extractors separate is advantageous if the domain gap between the source and the target domain is insignificant. Finally, we present SPI, which leverages contrastive losses to learn a semantically meaningful and a domain agnostic feature space using the supervised samples from both domains. To mitigate challenges caused by the skewed label ratio, we pseudo-label the unlabeled target samples by comparing their feature representation to those of the labelled samples from both the source and target domains. Furthermore, to increase the support of the target domain, these potentially noisy pseudo-labels are gradually injected into the labelled target dataset over the course of training. Specifically, we use a temperature scaled cosine similarity measure to assign a soft pseudo-label to the unlabeled target samples. Additionally, we compute an exponential moving average of the soft pseudo-labels for each unlabeled sample. These pseudo-labels are progressively injected (or removed) into (from) the labelled target dataset based on a confidence threshold to supplement the alignment of the source and target distributions. Finally, we use a supervised contrastive loss on the labelled and pseudo-labelled datasets to align the s

Abstract

Text Classification is a technique for assigning a set of categories to give text. A significant type of text classification is Sentiment Analysis. Sentiment Analysis is a technique in natural language processing to identify and quantify subjective states and information systematically. With the exponential increase in traffic over the internet, we have a lot of textual content available in different settings. This data provides an opportunity for efficient processing and application in solving problems across multiple domains. Few such critical applications include analysing public behaviour online and how they interact. Other is to analyse news content by different media houses that can cause public divide and polarisation due to its subjective nature. Firstly, we aim to create a novel dataset for bias detection in Hindi. We also explore classification tasks in a multi-modal setting to analyse and understand how information in multiple modalities affects classification outcomes. We further experiment with deep learning models to classify text where the language is more subtle and often unrealised for humans making its classification a tricky task. Recently, we have seen a rise in content from highly subjective news media outlets created to polarise and create a divide between people. This reporting can have far-reaching consequences, from orchestrating riots and public agitations to influencing the results of an election. This biased reporting makes its analysis and detection crucial. We aim to do it for the Hindi Language. As no data were available, we decided to develop our dataset. Our dataset consists of 1388 Hindi news articles and their headlines from various Hindi news media outlets, which were annotated for being biased towards, against, or neutral towards the BJP (Bhartiya Janata Party). Further, we explore a multi-modal classification task of identifying misogynous memes posted online. In this work, we also explore how the visual modality affects the classification task. We build an ensemble of deep learning text-transformers and vision-language transformers like UNITER and OSCAR and compare them to better understand the role of vision in multi-modal classification tasks. We also experiment with task-specific pretraining of our vision-language(VL) transformers. Lastly, we experiment with employing transformers trained on similar tasks on other similar datasets to explore transfer learning. Lastly, We employ deep-learning transformers to detect patronising and condescending language targeting vulnerable communities (e.g. refugees, homeless people, and low-income families). We experiment with techniques like focal-loss and weighted random sampling to deal with class imbalances and to improve our results. We also take up the task of classifying more fine-grained categories of condescending language.

Abstract

ML-powered document image analysis approaches can enable intelligent solutions in bringing handwritten information into the digital world. Two major components of handwriting understanding include handwritten text recognition(HTR) and handwritten search. The former task enables the conversion of handwritten text to digital format, whereas the latter task provides easy access to the handwritten information scattered across books, archives, manuscripts and so on. We primarily focus on both these problem statements in this thesis. Handwritten document image analysis for Indic scripts is still in its nascent stage compared to Latin scripts. For example, many commercial applications and open-source demonstrations are available for Latin scripts and there are hardly such known applications for Indic scripts. It is challenging to develop solutions for Indic scripts due to (i) variety of scripts within the Indic subcontinent, (ii) lack of huge annotated datasets and challenges in collecting data for multiple scripts, and (iii) inherent challenges in Indic scripts like inflections, joining multiple glyphs to form an akshara(equivalent to a character in Latin scripts). While challenging, it is also crucial to develop HTR and handwritten search approaches for Indic scripts. Therefore, this thesis is majorly focused on discussing approaches for Indic HTR and Indic handwritten search. In the last two decades, large digitization projects converted paper documents and ancient historical manuscripts into digital forms. However, they remain often inaccessible due to the unavailability of robust HTR solutions. Recognizing handwritten text is fundamental to any modern document analysis system. In recent years, efforts toward developing text recognition systems have advanced due to the success of deep neural networks and the availability of annotated datasets. This is especially true for Latin scripts. In this thesis, we discuss the standard text recognition pipeline that comprises various neural network modules. Then, we present a simple and effective way to improve the text recognition pipeline and training approach. We report the improvement from our approach on four benchmark datasets in Latin and Indic scripts. The existing state-of-the-art approaches for Latin HTR and Latin handwritten search are highly datadriven. Due to the lack of availability of large-scale data, developing Indic HTR and Indic handwritten search is challenging. Therefore, we release a collective Indic handwritten dataset with text images from majorly spoken 10 Indic scripts. We establish a high baseline for text recognition in prominent Indic scripts. Our recognition scheme follows the contemporary design principles from other recognition literature, and yields competitive results on English. We also explore the utility of pre-training for Indic HTRs. We hope our efforts will catalyze research and fuel applications related to handwritten document understanding in Indic scripts. Finally, we investigate the problem of handwritten search and retrieval for unlabeled collections. Handwritten search pipelines are needed in online platforms like E-libraries and digital archives. Such pipelines can efficiently search through handwritten collections and present relevant results, much like Google Search. With its ease of access and time-saving capability, the handwritten search application can prove to be valuable to many communities that study such historical documents. In this thesis, we present one such pipeline for handwritten search that performs retrieval on new and unseen collections. The proposed retrieval is not fine-tuned for specific writing styles or unknown vocabulary in the new collection. Therefore, it can be applied to new unlabeled collections.

Abstract

Linguistic diversity is one of the defining traits of the Indian subcontinent. With thousands of official languages and countless mother tongues, multilingualism is a way of life for the people. Today, as speech technology gains popularity due to the rising number of smart assistants, this abundance of natural languages is an obvious target for developing new speech systems. A majority of Indian languages happen to be low resource due to any of the following reasons that arise from the Indian context: 1) A lack of native speakers of the desired language; 2) A lack of recorded spontaneous speech for modelling natural speaker behaviour; 3) Poor quality of recordings due to external noise or minimal proper documentation; 4) Use of prolific code-switching in natural speech. Such limitations to the data that can be collected encourages us to look for alternatives that can both compensate for the lack of target data as well as properly utilise the feature of the languages available. Models to handle multilingualism are becoming increasingly complex, but they fail to be robust to the Indian context. Without any precise understanding of the way the models operate, it is impossible to adapt them to Indian language requirements. In addition, there is a lack of linguistic studies which have been conducted on Indian multilingualism in the context of developing speech systems. Due to this, even the requirements for making a system robust to the Indian context are largely unclear. The first step to adapting existing speech systems is understanding the models currently in use. Then, it is important to understand the context – in this case, to become familiar with the Indian context and the way Indian languages interact. This provides a theoretical basis which helps properly utilise existing resources with lower data reliance to manage Indian speech systems. This also forms the foundation for developing new frameworks which can either leverage the unique context of Indian languages or benefit from their shared features. In this thesis we have produced tools and results which will form the basis for future work on both multilingual and acquired language studies. We analyse the working of the joint acoustic model in multilingual automatic speech recognition systems, and determine which phonological properties of languages most affect model performance. We also demonstrate how the output patterns and accuracy are affected by the choice of language for each of the architectures that was used. Such results can be used to make appropriate choices for architectures based on the available data as well as the desired outputs for different applications, avoiding any unnecessary complications. We also study acquired language pronunciations, providing a dictionary for converting American English lexicons to Indian English as well as phonetic transformation rules which characterise Indian English relative to British Received Pronunciation. In our analysis, we also create a linguistic profile for 18 major Indian languages which describes the phonemes and phonological behaviours of Indian native languages which may be considered "universal" or regional. These tools can be adapted for handling variations in pronunciation or adapting accents to make existing systems more robust. Thus, the work in this thesis is a foundation for further studies of Indian language speech systems.

Abstract

The configurational sampling is central to characterize the equilibrium properties of complex molecular systems, but it remains a significant computational challenge. The conventional molecular dynamics simulations of limited duration often result in inadequate sampling and thus inaccurate equilibrium estimates. Replica exchange with non-equilibrium switches (RENS) is a collective variable-free computational technique to achieve extensive sampling from a sequence of equilibrium and non-equilibrium MD simulations without modifying the underlying potential energy surface of the system. Unlike the conventional replica exchange molecular dynamics (REMD) simulation, which demands a significant number of replicas for better accuracy, RENS employs non-equilibrium heating and cooling work simulations prior to configurational swaps to improve the acceptance probability for replica exchange. Here, we have implemented the RENS algorithm on four model systems and examined its performance against the conventional MD and REMD simulations. The desired equilibrium distributions were generated by RENS for all the model systems, whereas REMD and MD simulations could not do so due to inadequate sampling on the same timescales. The calculated work distributions from RENS obeyed the expected non-equilibrium fluctuation theorem. The results also demonstrated that the switching time of the nonequilibrium work simulations can be adaptively altered to fine-tune the acceptance probability and the reduced work of switching. The modular implementation of RENS algorithm not only enables us to readily extend it to multiple replicas, but also paves the way for extension to larger molecular systems in the future

Abstract

INTERNET is the most familiar word from everyone’s mouth nowadays. Children or adults of all ages are utilizing it in some way. Major change has been seen from communicating with pigeons and then moving on with telephone to voice-over IPs. Internet connects smaller businesses to larger ones and makes communication faster, reliable and accessible. Every second, the world creates an incredible amount of data, out of which the majority is unstructured. It has become pivotal to understand this vast amount of unstructured, raw data to get innovations and data-driven decisions. But the question that remains is what gives coherence to this raw data? The answer to this is through Natural Language Processing (NLP). In simple terms, NLP refers to the capability of computers to decipher human speech or, as it is spoken or written to text. NLP is a part of Artificial Intelligence that causes the systems to understand, grasp, and interpret how these systems communicate with human languages. It includes teaching computers how to make sense of the natural language humans use and to derive meaning from it. Many algorithms work on this aspect and train machines to understand natural language. Most of these algorithms are based on Machine learning, in which the computer program is trained using a large amount of data about the language to understand the language with some accuracy. This work will mainly focus on improving two NLP applications for Indian languages, viz Machine Translation and Text Classification. India is a multi-lingual country with thousands of languages and regional dialects spoken by its citizens. Some of them are lost, while others are still in use with significant speakers. Most of the Indian languages are related to each other due to shared ancestry or prolonged contact, and they share a lot of linguistic and structural features due to this relatedness. So, it is essential to utilize this languagerelatedness to build efficient NLP systems. We make use of relatedness between Indian languages and attempt to quantify their similarity. In the process of similarity score calculation, we examine different factors (script, string metric, size and type of parallel corpus, etc.) that can affect that similarity value. We also try to exploit language relatedness to train efficient multilingual NMT systems (one-to-many and many-to-one) and multilingual text classification systems for languages of the Indian subcontinent. We also focused on contributing to resource creation and annotation strategy for low-resource domains for Indian languages, further motivating researchers to curate new datasets for low-resource fields with little or no annotated data.

Abstract

Virtual assistants are trained to understand and process natural language, which in general are used to follow simple commands like making calls, controlling devices, setting alarms, etc. Customer chat supports in multiple applications also use virtual assistants for resolving simple issues or raising tickets. Human assistants come into action when there is a problem with understanding the issue or understanding the language. The aim here is to have the issues resolved quickly and efficiently by AI instead of a human. The virtual assistance in the past decade has improved vastly over the past decade. Widely used assistants like SIRI, Alexa, and Google Assistant were developed in the 2000s and had significant improvement in understanding natural language over time. Multiple virtual assistants have extensive uses in various fields like entertainment, essential tasks, voice search, etc. Virtual assistance in education is equally necessary as today’s world is much more dependent on online schooling. Learning is a crucial task in a student’s life, which needs assistance in many stages. We started by working on generating questions that will help in cross-checking key points from a passage of text. We worked on the grammatical part of the language Telugu which could give us better results than the state-of-art systems. We tried to create rules that capture the grammatical patterns for potential POS(Parts of Speech) tags that carry important information (instead of tags like auxiliaries) and suffixes of words in a Telugu sentence. In this work we present a learning assistant that tests one’s knowledge and gives feedback that helps a person learn at a faster pace. A learning assistant (based on an automated question generation) has extensive uses in education, information websites, self-assessment, FAQs, testing ML agents, research, etc. Multiple researchers and companies have worked on Virtual Assistance, but majorly in English. We built our learning assistant for the Telugu language to help with teaching in the mother tongue, which is the most efficient way of learning. Our system is built primarily based on Question Generation in Telugu. Many experiments have been conducted on Question Generation in English using multiple methods like rule/template based, supervised, semi supervised [39, 81, 31, 26, 1]. We have built the first hybrid machine learning and rule-based solution in Telugu, which proves efficient for short stories and short passages in children’s books. Our work covers the fundamental question forms with question types such as adjective, yes/no, adverb, verb, when, where, whose, quotative, and quantitative (how many/how much). We constructed rules for question generation using Part of Speech (POS) tags and Universal Dependency (UD) tags along with linguistic information of the surrounding relevant context of the word. Our system is primarily built on question generation in Telugu and is also capable of evaluating the user’s answers to the generated questions

Abstract

Semantic segmentation is an essential primitive in real-time systems such as autonomous navigation, which require processing at high frames per second. Hence for models to be practically applicable, it is essential that they have to be compact, fast as well as achieve high prediction accuracies. Previous research into semantic segmentation has focused on creating high-performance deep learning architectures. Most of the time, these best-performing models are complex, deep, have large processing times, and demand a significantly higher amount of processing capacity. Another relevant area of research is model compression, by which we can obtain lightweight models. Considering that there also have been works that produced mainstream light-weight semantic segmentation models at the expense of performance, we design models that bring a desirable balance between performance and latency. Specifically, methods and architectures that give a high performance while being real-time and working on resourceconstrained settings. We identify the redundancies in the existing state-of-the-art approaches and propose compact architecture family called ESSNet with accuracy comparable to the state-of-the-art while utilizing only a fraction of the space and computational power of those networks. We propose convolutional module designs with sparse coding theory as a premise and we also present two real-time encoder backbones employing our proposed modules. We empirically evaluate the efficacy of our proposed layers and compare them with existing approaches. Secondly, we explore the need for optimization during the training phase in the proposed models and present a novel training method called Gradual Grouping that results in models with improved implementation efficiency vs accuracy trade-offs. Additionally, we conduct extensive experiments by varying architecture hyper-parameters such as network depth, kernel sizes, dilation rates, split branching and additional context extraction modules. We also present a compact architecture using multi-branch separable convolutional layers with different dilation rates.

Abstract

Dance movement is not just an external body movement but internal too. It is both a physical and mental phenomena. As per Vaiśeṣika theory both these phenomena are different in nature. Mental phenomena, like thoughts, are sequential in nature, as it is in pure time. Physical phenomena, like body movement, are in time as well as space. If thoughts are to be put in front of the world, then they need body which utilizes space. How does the thought that is in mind finds expression in space using body? The link between the two, mind (mental) and body (physical), is manas (mind). Manas connects mental and physical phenomena. It helps us to understand the internal body which plays a major role in dance. It is through body that a dancer dances. Manas resides and moves in the internal body. Hence tracing the movement of manas is essential for the construction of accurate grammar of dance. We take a model of human as per Vaiśeṣika theory to interpret dance movement. According to this theory human being is made up of three basic elements, ātman (self), manas (mind) and śarīra (body). Ātman with the help of manas makes an effort (prayatna, ceṣṭā) on the muscles and joints of body to create dance. Manas is the link between ātman and śarīra. When we do a simple movement, manas actually goes to millions of places in the body. Even if we take gross [aggregate] movements of the manas moving in the body, we can model dance movement. Thus, understanding the movement of manas will help us to understand and derive the grammar of movement. We plan to specify syntax of movement, which integrates physical as well as mental aspect of the movement. Thus when the movement is executed as a conscious experience then it is noticed that certain structures emerge. These structure are the phenomenal concepts also known as movement words. These movement words are further classified giving the detail. These details are then used to reconstruct the movement. Keywords: Dance, Philosophy of mind, Phenomenology, Ontology, Bharatanāṭyam, Vaiśeṣika Darśana, Kinaesthetic Experience, Embodied Dance

Abstract

In India, energy consumption has increased by 50% during the last decade, and with rapid electrification, ever-growing population and increasing incomes, it is expected to grow by 2.3 times in the next two decades. Residential sector alone contributes about 24% of India’s total energy use and is set to go upwards in the coming years. Factors such as appliance ownership, occupant behaviour, thermal comfort levels and the climate conditions are the major contributors to household electricity consumption in India. In the Indian dwellings about 45% of total energy consumed is used to meet the requirement of thermal comfort. Air-Conditioners (AC) usage patterns influence the annual load curve significantly even though it is used only for a few months, for few hours every year. Due to the availability of the limited energy resources, meeting these increasing energy demands is a major challenge. In order to reduce these demands, a deeper understanding of energy consumption patterns and demand profiles along with the factors that are affecting the usage levels will help. This thesis proposes a well-organised data driven approach that is helpful towards future solutions to address this challenge. Based on the recent studies, a thorough understanding of residential energy consumption is required for designing and execution of energy saving measures. Whereas most of the studies in India focused on survey-based approach in identifying electricity usage patterns of residential sector, a very few studies were done using the captured real-time data of electricity and indoor temperature patterns. Therefore, there is a need to accurately collect and understand data on energy consumption at household level, which is of interest to end-customers, utilities, appliance manufacturers and most importantly policy makers. In this thesis, a total of 25 homes from Hyderabad, covering various built types (standalone houses/apartments), income levels (low/medium/high income), occupancy levels and ownership of AC units were monitored during three different seasons (summer, monsoon and winter) by installing electricity and indoor temperature loggers. A detailed survey questionnaire was also conducted to understand dwelling and occupancy characteristics. Using the monthly energy bills and the survey questionnaire data, a correlation matrix was developed to identify the factors that affect the household electricity consumption. It was found that, factors such as ‘connected load’ of home appliances, ‘number of ACs’, ‘number of rooms’ and ‘income levels’ had significant impact on the ‘Annual Energy Consumption (AEC)’ of a dwelling. Seasonal consumption trends were identified using the data from the loggers installed in the dwellings and it was observed that AC home consume nearly three times more than a non-AC home during summer, but the gap between them was minimal during monsoon and in winter. Further study was done on AC homes to determine the usage patterns of the primary-AC and the true contribution of it towards the total home energy. On an average, primary-AC was used for 6.2 hours a day with mean daily consumption of 6 kWh/day during summer. It was found that primary-AC was contributing about 44% towards the total home energy.