Abstract

In India, energy consumption has increased by 50% during the last decade, and with rapid electrification, ever-growing population and increasing incomes, it is expected to grow by 2.3 times in the next two decades. Residential sector alone contributes about 24% of India’s total energy use and is set to go upwards in the coming years. Factors such as appliance ownership, occupant behaviour, thermal comfort levels and the climate conditions are the major contributors to household electricity consumption in India. In the Indian dwellings about 45% of total energy consumed is used to meet the requirement of thermal comfort. Air-Conditioners (AC) usage patterns influence the annual load curve significantly even though it is used only for a few months, for few hours every year. Due to the availability of the limited energy resources, meeting these increasing energy demands is a major challenge. In order to reduce these demands, a deeper understanding of energy consumption patterns and demand profiles along with the factors that are affecting the usage levels will help. This thesis proposes a well-organised data driven approach that is helpful towards future solutions to address this challenge. Based on the recent studies, a thorough understanding of residential energy consumption is required for designing and execution of energy saving measures. Whereas most of the studies in India focused on survey-based approach in identifying electricity usage patterns of residential sector, a very few studies were done using the captured real-time data of electricity and indoor temperature patterns. Therefore, there is a need to accurately collect and understand data on energy consumption at household level, which is of interest to end-customers, utilities, appliance manufacturers and most importantly policy makers. In this thesis, a total of 25 homes from Hyderabad, covering various built types (standalone houses/apartments), income levels (low/medium/high income), occupancy levels and ownership of AC units were monitored during three different seasons (summer, monsoon and winter) by installing electricity and indoor temperature loggers. A detailed survey questionnaire was also conducted to understand dwelling and occupancy characteristics. Using the monthly energy bills and the survey questionnaire data, a correlation matrix was developed to identify the factors that affect the household electricity consumption. It was found that, factors such as ‘connected load’ of home appliances, ‘number of ACs’, ‘number of rooms’ and ‘income levels’ had significant impact on the ‘Annual Energy Consumption (AEC)’ of a dwelling. Seasonal consumption trends were identified using the data from the loggers installed in the dwellings and it was observed that AC home consume nearly three times more than a non-AC home during summer, but the gap between them was minimal during monsoon and in winter. Further study was done on AC homes to determine the usage patterns of the primary-AC and the true contribution of it towards the total home energy. On an average, primary-AC was used for 6.2 hours a day with mean daily consumption of 6 kWh/day during summer. It was found that primary-AC was contributing about 44% towards the total home energy.

Abstract

Open-Vocabulary Keyword spotting solves the problem of spotting audio keywords in an utterance. The keyword set can incorporate keywords the system has seen and not seen during training, functioning as zero-shot keyword spotting. The traditional method involves using ASR to transcribe audio to text and search in the text space to spot keywords. Other methods include obtaining the posteriors from a Deep Neural Network (DNN) and using template matching algorithms to find similarities. Keyword spotting does not require transcribing the entire audio and focuses on detecting the specific words of interest. In this thesis, we aim to explore the usage of the Automatic Speech Recognition (ASR) system for Keyword Spotting. We demonstrate that the intermediate representation of ASR can be used for open vocabulary keyword spotting. With this, we show the effectiveness of using Connectionist Temporal Classification (CTC) loss for learning word embeddings for keyword spotting. We propose a novel method of using the CTC loss function with the traditional triplet loss function to learn word embeddings for keyword spotting on the TIMIT English language audio dataset. We show this method achieves an Average Precision (AP) of 0.843 over 344 words unseen by the model trained on the TIMIT dataset. In contrast, the Multi-View recurrent method that learns jointly on the text and acoustic embeddings achieves only 0.218 for out-of-vocabulary words. We propose a novel method to generalize our approach to Tamil, Vallader, and Hausa low-resource languages. Here we use transliteration to convert the Tamil language script to English such that the Tamil words sound similar written with English alphabets. The model predicts the transliterated text for input Tamil audio with CTC and triplet loss functions. We show that this method helps transfer the knowledge learned from high resource language English to low resource language Tamil. We further reduce the model size to make it work in a small footprint scenario like mobile phones. To this extent, we explore various knowledge distillation loss functions such as MSE, KL Divergence, and CosEmbedding loss functions. We observe that small-footprint ASR representation is competitive with knowledge distillation methods for small-footprint keyword spotting. This methodology makes use of existing ASR networks trained with massive datasets. It converts them into open vocabulary keyword spotting systems that can also be generalized to low-resource language.

Abstract

Automatic Speech Recognition (ASR) refers to the task of transcribing a given audio sample into its corresponding words. In the recent years, interest in improving the ASR systems has grown tremendously as the commercial viability of the devices such as the Google Assistant, Alexa, and Siri that use these systems has grown exponentially. As is true for most tasks that use Machine Learning-based systems, building robust ASR systems requires large amounts of annotated data. However, publicly available resources to build ASR systems for Indian languages is scarce. Therefore, in order to counter this scarcity, we explore how language relatedness can improve the performance of Language Models (LM) and thereby the ASR systems that use them. We begin by investigating how pooling the datasets of closely related Indian languages such as Marathi and Hindi and building a HMM-based Hybrid multilingual ASR system can help improve the quality of transcriptions as measured by Character Error Rate (CER) and Word Error Rate (WER). Having found little evidence that such models outperform baseline monolingual ASR systems, we look to increase the lexical overlap between the two languages by using subword-based language models instead of the default choice, i.e., word-based languages. We investigate the performance of such models under various conditions and show that they outperform word-based counterparts as measured by upto 26.35% CER. With the increased interest in End-to-End ASR systems, we shift our focus to how LMs play a role in them. Previous research has found that Acoustic Models (AM) of an Automatic Speech Recognition (ASR) system are susceptible to dialect variations within a language, thereby adversely affecting the ASR. To counter this, researchers have proposed to build a dialect-specific AM while keeping the LM constant for all the dialects. In this thesis, we start by studying how dialect mismatched LMs adversely affect the performance of the ASR by considering three regional Telugu dialects: Telangana, Rayalaseema, and Coastal Andhra. We show that dialect variations that surface in the form of a different lexicon, grammar, and occasionally semantics can significantly degrade the performance of the LM under mismatched conditions. Therefore, this degradation has an adverse effect on the ASR even when dialect-specific AM is used so much so that it actively hinders the performance of the ASR, i.e., performs worse than an ASR system with no LM. Next, we look to remove the need for building dialectspecific model by proposing a multi-dialect ASR system that outperforms the dialect-specific ones. We find that simple pooling of the data and training a multi-dialect ASR benefits the low-resource dialect the most (Rayalaseema). Subsequently, we incorporate dialect-specific information by adding a Dialect ID into the LM of an End-to-End ASR. We train these End-to-End ASR systems under a Multi-Task Learning (MTL) framework, where the the primary task is to transcribe the audio and the secondary task is to predict the dialect. Such a model outperforms naive multi-dialect ASRs by up to 8.24% in relative WER.

Abstract

Due to rapid urbanization and rising population, there has been a significant increase in the global demand for electricity in the building sector. Windows significantly influence building energy performance; while it helps provide daylight for the occupants, it may aggravate the building's heating or cooling load and results in higher energy consumption if not appropriately designed. Proper window design may help to reduce the power consumption in the building. The relation between the building energy consumption and the window-to-wall ratio (WWR) is complex since increasing the window size or WWR, may initially result in electricity savings due to increased daylight utilization. Still, the increase in solar radiation inside the space might increase the cooling load after a certain WWR, reducing the benefits. So, there will be a point of inflection where the user can get the maximum energy benefit/saving. This point will vary based on the type of glazing, solar heat gain coefficient (SHGC), visible light transmittance (VLT), thermal transmittance (U-value), WWR, location, and the orientation of the window. Modern buildings have different types of glazing, such as clear glass, Low-E, semi-transparent photovoltaic (STPV), electrochromic (EC) glazing, and glazing with dynamic blind to provide comfort to the occupants and achieve building energy efficiency. Thus, there is a need to determine what type of window and WWR would give the optimal performance for a given orientation for different window systems. This work proposes a simulation-based approach that does parametric simulation analyses for different window systems with several WWR and orientations to compute and compare the annual energy performance in EnergyPlus. An index, net electricity consumption (NEC), is used based on the electricity consumption by air conditioning, artificial lighting, equipment, and electricity generation in the case of STPV glazing to evaluate the annual energy performance and perform a comparative analysis with different glazing. Performing parametric simulation and calculating NEC of different combinations require expertise in modeling and building energy simulation using software such as EnergyPlus. A web-based parametric tool can assist the user in carrying out the desired studies without requiring extensive technical knowledge. Based on this approach, an automated parametric simulation tool is developed, and a case study is conducted in two distinct climatic zones of India. This study examines the benefits of single and double pane semi-transparent photovoltaics (STPV) glazing over other regular glazing systems with similar opto-thermal parameters as STPVs. Further, the impact of dynamic internal blind on the STPV window and electrochromic glazing on the annual energy consumption is analyzed. The study adopts an Energy Management System (EMS) approach to model EC window systems. The simulation results consist of case studies of Hyderabad and Kolkata, which lie in two different climatic zones of India. It shows benefits in adopting the STPV window system across all directions and higher optimal WWR in the STPV window compared to the traditional window with the same opto-thermal characteristics. The study shows that the optimal NEC in the case of STPV windows is 10-12% less than simple glazing’s optimal NEC. Additionally, the result suggested that glare-controlled interior blinds in the STPV window further reduce the NEC by up to 15% compared to conventional glazing. Similarly, installing the EC glazing reduces the NEC consumption by up to 5% compared to standard glazing

Abstract

All the parsers built for English till date are either syntactic (constructs possible syntactic structures of the sentence) or semantic (maps text to formal meaning representations). CPG is syntactico-semantic, a machine-understandable representation of the sentence with a syntactic structure as well as semantic labels. More than 80% of the dependency relations can be represented with just 7 of the Karaka relations. We first designed and developed a rule-based mapper to convert Stanford dependencies(42) to Karaka labels(7). The accuracy of the mapper was 52% and is domain-independent. This was used in the academic/courses domain based NLIDB [27] system to identify and map to verb frames/templates to further generate the SQL query. More details on that in further chapters. The next version was to analyze the English Tree-bank (manually annotated Karaka labels) data and build a generic mapper using ML methods. We examined the Karaka dependencies and listed some features that could affect the nature of the Karaka relation. We took the English Tree-bank data, manually annotated by LTRC and converted it into a format suitable for application of statistical methods. After cross-validation within the data set, accuracy of the results went upto 67% using CRF++. Re-examining and adding one more feature increased the accuracy by 2%. Then using SVM (LIBSVM) on the same data took the overall accuracy to 74%. The previous 2 experimental results were applied in the context of Language (NLIDB) model only with limited dataset. Compared to other Vision + Language tasks, datasets for Embodied AI tasks tend to be longer and contain more actions, which makes it naturally suitable for CPG analysis as CPG is also centered around actions. As the concluding experiment, we used the Karaka relations in a MultiModal system to improve the syntactic and semantic understanding of the instruction set with a small set of labels. We compared the results with other labeling systems like NLTK (PoS tags) and Stanford parser (Dependency labels).

Abstract

Ellipsis is deemed important for downstream Natural Language Processing (NLP) tasks that handle data with ellipsis, such as Information Retrieval (IR), event extraction, Machine Translation (MT), Dialogue systems, Summarization, etc. The simplest and most straightforward way to confirm this to add ellipses resolution as an additional step to the existing NLP system pipelines and measure the change in their performance. However, previous computational work on ellipsis resolution is not only limited, but has also majorly focused on only one type of ellipsis, namely Verb Phrase Ellipsis (VPE) and some phenomena closely related to it such as sluicing, gapping, etc. In other words, the computational support for other ellipses types is even more sparse. Another major challenge this field is facing is the lack of use of existing theoretical understanding of this phenomenon in linguistics, which as I find out, has made the task harder than it really is. This thesis presents the first computational treatment to another major form of ellipses namely–noun ellipses, (also referred to as noun phrase ellipses and nominal ellipses in some linguistic textbooks), drawing from linguistic features of the phenomenon. I present several procedures for the detection and resolution of noun ellipsis in English, as well as the motivation behind and the impact of doing so. The computational experiments draw heavily from the syntactic analysis of noun ellipses in traditional and more recent theoretical linguistic studies. I begin with a rule based approach– where I exploit existing syntactic structural analysis of different types of ellipses in general and noun ellipses in particular, so as to identify cues for its detection and resolution. These result into novel syntactic rules, which are then later represented and optimised as manual features in a supervised Machine Learning framework One of the challenges of using linguistic theory of ellipsis is the discrepancy in the definition of the phenomenon used in previous work. Traditionally, many linguists have discussed noun ellipsis exhibiting two strategies in English, namely true lexical elision and one-anaphora. However, more recent linguistic analysis shows they are completely different phenomena. This claim finds support from linguistic analysis of the word taking into account its morphological, syntactic and semantic properties, and is backed by statistical insights on its occurrence and distribution from data-driven studies carried on several popular English corpora. In this thesis, I use this linguistic analysis to modify the system for the detection and resolution of noun ellipsis presented earlier and successfully handle the detection and resolution of oneanaphora in English with high accuracy. The investigation and the computational experiments that lead towards it highlight the importance of linguistic analysis in NLP research. Apart from presenting end-to-end pipelines for noun ellipses and noun anaphora resolution that outperform state-of-the-art models significantly, another major contribution of this work will comprise novel findings on the impact of noun ellipses (along with verbal and clausal ellipses) as well as the anaphoric instances of the word one on Machine Translation (MT). I also test the efficacy of a simple resolution procedure as a preprocessing step for improving the performance of English-Hindi NMT–a simple syntactic procedure that can be extended to other language pairs and can be easily incorporated into an existing model without making any modifications in the model itself Finally, there are some byproducts of this work, including two huge hand annotated corpora for the analysis of one-anaphora and noun ellipsis in English, one small curated dataset containing several instances of noun ellipses, and a parallel testset to aid research in English-Hindi MT

Abstract

In recent years, the Internet of Drones (IoD) consisting of Unmanned Aerial Vehicles (UAVs), also called drones, achieves a great momentum due to its high mobility to difficult-to-access places with minimum intervention. The drones are remotely piloted aircrafts, which are widely installed from military mission to civilian applications. For various IoD applications, the drones communicate over public (insecure) channels under the wireless sensor networks (WSN) where the drones are usually deployed in various crucial applications and terrains. Various security threats like replay, man-in-the-middle, impersonation, privileged-insider, physical drones capture attacks, etc. exist in an IoD environment. Therefore, it becomes a challenging job to design security mechanisms for providing the authenticity of transmitted information during communication over the public channels. An access control mechanism is a security mechanism that controls who or what can see, use or access the assets in the IoD system to ensure security/access control for the private data. The IoD applications produce a huge volume of data that is mainly confidential and it needs to be stored securely. Once the data is gathered by the UAVs, the data can be stored securely in the distributed servers (ledgers), such as blockchain, due to a single server failure issue in a traditional centralized storage platform. Once the data is stored in a blockchain, the data cannot be modified, deleted or altered by a malicious entity. Motivated by these issues, in this thesis, the blockchain-based access control schemes have been designed in order to provide strong security in IoD environment to store and access the information for the UAVs applications. The first study presents a new blockchain-based access control scheme in an Internet of Things (IoT)-enabled IoD deployment. In this scheme, several drones are deployed in different flying zones where the drones residing in each zone can securely communicate with each other in order to exchange crucial information. Next, the information is securely collected by their respective Ground Station Server (GSS) of the drones. Secure data gathered by the GSS form transactions, and the transactions are then made into the blocks. The blocks are finally added in the blockchain by the cloud servers which form a peer-to-peer (P2P) cloud servers network via the voting-based “Ripple Protocol Consensus Algorithm (RPCA)”. We provide all sorts of security analysis including formal security under the random oracle model, informal security and simulation-based formal security verification using the widely recognized “Automated Validation of Internet Security Protocols and Applications (AVISPA)” tool to assure that the proposed scheme can resist various potential attacks with high probability needed for an IoD environment. A meticulous comparative analysis among the proposed scheme and other closely related existing schemes shows that our scheme offers more functionality attributes and better security, and also low communication and computation costs as compared to other schemes. In addition, a real testbed experiment has been also demonstrated to show the feasibility study of the proposed scheme for the access control part. In the second study, we propose a novel access control scheme for unauthorized UAV detection and mitigation in an IoD environment, called ACSUD-IoD. With the help of the blockchain-based solution incorporated in ACSUD-IoD, the transactional data having both the normal secure data from a drone (UAV) to the GSS and the abnormal (suspected) data for detection of unauthorized UAVs by the GSS are stored in private blockchain which are considered as authentic and genuine. As a result, the Big data analytics can be performed on the authenticated transactional data stored into the blockchain. Through the detailed security analysis including formal security under the broadly-accepted Real-Or-Random (ROR) model, formal security verification using the widely-applied AVISPA tool and non-mathematical security analysis, we show the robustness of the proposed scheme against a number of potential attacks needed in an IoD environment. The testbed experiments for various cryptographic primitives using the broadly-accepted “Multi-precision Integer and Rational Arithmetic Cryptographic Library (MIRACL)” have been performed under both server and Raspberry PI 3 configurations. Furthermore, a detailed comparative analysis and blockchain-based simulation study have been conducted to show the effectiveness of the proposed scheme. Finally, in the last but not least study, we focus on designing a new blockchain-envisioned secure data delivery and collection scheme for the 5th generation mobile network (5G)-based IoT-enabled IoD environment which relies on the elliptic curve cryptography (ECC). This scheme tackles efficiently the security and privacy challenges during commun

Abstract

Language identification (LID) refers to the task of recognizing the language (e.g., Hindi,English, Telugu, etc.) from the spoken utterance. In a multilingual country like India, automatic LID systems play a key role in developing speech applications. Indian languages majorly belong to two language families i.e., Indo-Aryan and Dravidian. The languages from these families share a common phonetic space, which makes language identification a challenging task in the Indian scenarios. Extracting language discriminative information from speech plays a vital role in developing LID systems. Lack of large scale datasets covering various diversities such as speakers and dialects adds additional level of complexity of the task. Large scale datasets are a pre-requisite for effectively adapting the recent advancements in deep neural networks (DNNs) for developing the LID systems. Lack of this data poses an additional challenge for developing LID systems in Indian scenarios. The goal of this thesis is to improve the performance of Indian LID systems. As an initial step we have collected a speech database comprising data of 23 Indian languages. The database has 103.5 hours of speech data comprising of 1150 speakers covering multiple dialects. In this thesis, we have explored various feature extraction techniques and modelling approaches for developing LID systems. Modeling the long-temporal context is a key for developing LID systems. We have explored the use of stacked shifted delta cepstral ( Stacked-SDC ) features for developing LID systems. These features capture the long-term temporal context to improve the performance of LID systems. To effectively capture this information, a feature that can model the long-term temporal context is required. This study aims to capture the long-term temporal context by appending stacked shifted delta cepstral ( Stacked-SDC ) features. We also explore DNN with attention model for developing LID systems. These models have the sequence-summarizing capability, which helps to better model the long-temporal context. A multi-head attention mechanism is explored to develop LID systems, as having multiple heads improves the capacity of the models. The neural hidden representations derived from a joint acoustic model (JAM) of a multilingual end-to-end automatic speech recognition system are used as features for developing LID system. These features represent contextual information learned by the acoustic model. In this work, an LSTM-CTC ( Connectionist Temporal Classification ) framework is used for training the JAM models. Recently Self-supervised pre-training (wav2vec2.0) has been used to train a neural network which can generate representations with discriminative properties. We explore the use of features from the pre-trained network for developing LID system. The performance of LID systems is at its best when there is a match between training and operating environments. A large degradation is observed when there is a mismatch in the quality of the data in the presence of noise. To overcome the mismatch problem due to noisy operating environments, curriculum learning strategies are explored for training LID systems. These training methods have led to more stable LID systems that can be operated in noisy environments

Abstract

Digital (discretized and quantized) samples are used to represent speech signals in a computer. Beyond recording and playback, these representations as such have limited use. Most speech processing applications involve extraction of relevant information from the speech signals. For example, speech recognition requires extraction of parameters or features that describe the different sound units uniquely. In speaker recognition, it is necessary to extract the speaker-specific features to discriminate one speaker from another. In emotion recognition, the emotion-specific characteristics need to be identified and extracted. In speech compression, the features of speech production need to be captured, so that the speech can be effectively and efficiently used for transmission and storage. This thesis focuses on extraction and representation of speech production features, not only for understanding the link between the speech signal and the dynamic vocal tract system, but also to examine the possibility of compact representation for transmission and storage. The first attempt is to produce a spectrographic display that shows different types of segments with good temporal resolution, and also changes in the formant contours reflecting the rapid movement of the articulators of the dynamic vocal tract system. Methods are proposed to extract formants and their bandwidths. Glottal source processing methods are developed to obtain the characteristics of the glottal vibration within each cycle. While these source and system features are useful in understanding the production process, they are not adequate to reproduce all the important characteristics of speech, as evidenced by the poor quality of the formant vocoder developed using the extracted features. For a compact representation of the speech signal for reproduction, the highly successful linear prediction (LP) based source-system model is used. Even approximate separation of the source and system components helps in effectively compressing the speech information using artificial neural network (ANN) models for these components. The compressed parameters are also useful for other applications such as speech recognition, speaker recognition and speech enhancement

Abstract

Numerical simulations of condensed matter systems, for example biomolecular systems, are notorious for slow dynamics and consequent incomplete sampling lead to incorrect inferences on system thermodynamics and kinetics. In this study, we propose a protocol for efficient exploration of relevant phase space using machine learning techniques. The suggested protocol explicitly performs machine learning in two concurrent parallel stages: (i) map different metastable states into different classes by a classification model with assumption that large jumps that result in anomalous data (to the current class) are indicative of transitions between metastable states and (ii) construct a non-linear map within individual free energy minima (here, a class) by use of regression model that can learn local information. We apply the protocol on alanine tetrapeptide and show that a significant increase in sampling in all relevant phase space of the system; specifically, in comparison to the benchmark study using metadynamics using a well designed collective variable. We report sampling of all highly probable metastable states, and a significant increase in the sampling of relevant low probability metastable states, along with the concomitant increase in the sampling of the transitions between all such metastable states. The protocol proposed in the current study will be useful in construction of complete metastable states connectivity graph, thus enabling computation of thermodynamic and kinetic process of systems with very large degrees of freedom.