Abstract

The effectiveness of machine learning techniques is strongly dependent on the choice of data representation and the features available. In recent years, the importance of feature engineering has been confirmed by the exceptional performance of deep learning techniques, that automate this task for some applications. For others, feature engineering requires substantial manual effort in designing and selecting features and is often tedious and non-scalable. An AI system must fundamentally understand the world around us which can only be done if it can learn to identify and disentangle the underlying implicit patterns present in the observed low-level sensory data. Feature learning presents a way to take advantage of human ingenuity and prior knowledge to compensate for that weakness. In order to make it possible to apply machine learning to different domains, it is very important to make learning algorithms less dependent on manual feature engineering, so that novel AI based applications could be developed faster.This thesis is about automated feature learning, i.e., learning representations of the data that make it easier to extract useful information when building classifiers or other predictors. A good representation is one that is useful as input to a supervised predictor. Among the various ways of learning representations, this thesis focuses on automating the task of feature learning without any domain knowledge via regression models. Our proposed algorithm is scalable regression-based feature learning algorithm. Being data driven, it requires no domain knowledge and is applicable to any dataset having numeric attributes. Such a generic representation is learnt by mining pairwise feature associations, identifying the linear or non-linear relationship between each pair, applying regularized regression and selecting those relationships that are stable. Our experimental evaluation on benchmark UC Irvine and gene expression datasets across different domains provides evidence that the features generated through our learning model can improve the prediction accuracy significantly for different classifiers without using any domain knowledge.

Abstract

The last decade has witnessed digitization of much governmental data. With the increasing usage of internet, tons of data is being digitized which includes parliamentary proceedings as well. A major downside of this digitization is that the data is in an unstructured format. There is a need to structure the data for use in machine learning, data mining and NLP. Much work has been done on parliamentary data such as Hansard, American congressional floor-debate data on various aspects but there isn’t enough work on the argumentation aspects of it using computational methods. Text summarization of the political discourse, particularly in the parliamentary proceedings is an area of research that is rela- tively explored less. Also, argumentation is relatively a new area of research which has many potential applications and can be used as a feature in machine learning models. In this thesis, we work on these rough edges: creating structured data of parliamentary debates, creating an annotated dataset of arguments on the debates and then propose a novel approach for summarizing these debates using argumentation theory. Creating structured data from unstructured PDF’s was a challenge. We developed a software parser which leveraged the patterns associated with each debate type to convert it into a structured format and store it into a database. The dataset created consisted of 768 debates from the Lok Sabha of the Indian Parliament spanning over 189 sessions. We also discuss the potential use cases of the dataset such as experimenting various natural language processing techniques like stance classification, sentiment analysis, detection of blame and appreciate speeches, generating statistical analysis of parties and their members and statistical analysis of the parliamentary debates. We explain the significance of the dataset by performing a few preliminary experiments such as stance classification, calculating sentiment po- larity and detection of speeches which capture raising of Issues, Blame, Appreciate or Call for Action. We achieved an accuracy scores of 0.80, 0.74, 0.60, 0.84 and 0.80 for stance classification, detection of issue, blame, appreciate and call for action speeches respectively. In the next part of the thesis, we harness this dataset for creating an argument annotated dataset. This dataset is ideally suited for creation of argument dataset because of the argumentative nature of the parliament debates. We annotate the debates with the argumentative theory which contains two argument components ‘claim’ and ‘premise’ and two argument relations support and attack which connect these two components. Few researchers have developed argument datasets of essays, news articles and online debate data but not of parliamentary proceedings. Creating an argument annotated dataset is a non-trivial task due to the complexity in the theory. Annotation of arguments is a non-trivial task as identification of argument components and relations between them needs critical attention and complete understating of the theory. We achieve an average score of 0.31 using multi-π inter annotator agreement metric before training of the annotators. After providing guidelines and training the annotators we achieve an average score of 0.73 using using multi-π inter annotator agreement metric which is a significant improvement and shows that arguments can be reliably annotated in parliamentary proceedings. To show the potential and novel application of the argumentation theory, we created a pipeline based approach for generating extractive summaries of the parliamentary debates using the argumentative theory. We use argumentation as a feature and design a semi automatic pipeline for generating debate sum- maries. The proposed approach considers the relevance of the topic, argumentative nature, sentiment and context features for generating summaries. We test our approach on the argument annotated dataset of debates developed previously. We discuss and analyze the results obtained using the pipeline. Our proposed methodology achieves F-scores of 0.44, 0.21, 0.41 using ROUGE-1, ROUGE-2 and ROUGEL evaluation metrics over other summarization algorithms in the case of 1500 word summary generation.We also evaluate our approach with other high performing popular summarization algorithms such as TextRank, Nenkova and LexRank.

Abstract

How to make machines speak more like humans? With the emerging technologies and computing abilities, once prescient ideas to emulate the human nature in machines are eventually achievable in the recent past. We specifically discuss the artificial production of speech by leaning into Text-to-Speech (TTS) synthesis and Voice Conversion (VC) sys-tems. TTS systems carry out the conversion of linguistic information present in the text to the minimal sound units which are further synthesized into speech using a vocoder. TTS systems are employed for many applications such as public announcements, voice assistants in mobile phones, screen readers for visually challenged and language learning systems. Incorporating the speaker variation would facilitate us in building a variety of synthetic voices. VC technology enables the speaker variations through the artificial imitation of the desired target speaker. We have thus explored VC frameworks for speaker variations (conversion) with and without parallel corpus. VC systems mainly find their applications in emotion conversion, personalized TTS systems, Normal to Lombard speech conversion and speech to singing conversion. In our thesis, we discuss artificial speech generation for: expressive speech synthesis, text-dependent Voice (speaker) conversion, and the major contributions through our work to Cross-Lingual Voice Conversion (CLVC). TTS Synthesis and VC involve modeling of sequential data. Thus, they highly rely on the contextual information. Accordingly, various approaches to appropriate feature representation and modeling of sequences in building the above-mentioned speech systems are discussed in this thesis. Our contributions: the continuous representation of text for prosody modeling, integrating signal correlation as one of the sub costs for phone-based concatenative speech synthesis. Furthermore, a light-weighted approach to CLVC is presented. Key ideas of this thesis are: • The sentence level modeling of text provides supra-segmental information required for the pre- diction of phone durations in Text-to-Speech Synthesis. Hence, we have incorporated a Recurrent Neural Network based Language Modeling (RNNLM) for continuous representation of sentences in addition to the matrix factorization for phones and words. Accordingly, a unique representation of text has been leveraged for the expressive speech synthesis. Embedding the signal correlation information while calculating the join cost facilitates an optimal selection of units for concatena-tive speech synthesis. Thus, Cross-Correlation was included in the join cost to further minimize the total cost when selecting the units. • A text-independent and transcription free approach was proposed for CLVC. Stacking of source and target speaker’s speech has enabled the projection of the language and speaker dependent features onto a common space. An auto-encoder is utilized as a parallel corpus generator to mitigate the issues involved in CLVC. A traditional parallel VC is then carried out between the generated and natural speech in target speaker’s voice. Further, a cross-language conversion is conducted for the target language. We conclude that an error reduction network can also be utilized to address the mapping of unaligned sequences.

Abstract

Tandemly repeated structural motifs within a protein form highly stable super secondary structural folds and provide multiple binding sites giving rise to diverse protein-protein/DNA/RNA interactions. Thus, these proteins provide a repertoire of cellular functions such as cell-cycle regulation, transcriptional regulation, cell differentiation, apoptosis, plant defense, bacterial invasion, etc.The evolutionary conservation and disease association of repeat proteins show their structural and functional importance and suggest the need for their identification. The multiple copies of repeats originate by the intragenic duplication and recombination events resulting in identical repeating units, both at the sequence and structure level. During the course of evolution, individual copies of the repeat accumulate mutations resulting in undetectable sequence similarity within repeating units, though the structural fold remains conserved. The traditional approaches of repeat identification and classification have mainly been based on identifying sequence patterns, and have limited coverage, especially for repeats with low sequence conservation. Thus, identification and classification of repeats at the structural level is desirable. In this thesis, we propose computationally efficient graph based approaches for the identification of repeats in protein structures, and use them to build a database of structural repeat proteins. The three dimensional topology of a protein structure is well captured by a graph, making it possible to analyze a complex protein structure as a mathematical entity. We observe that the principal eigen spectra of the adjacency matrix (eigenvector centrality) is able to capture the conserved residue interaction pattern within and between the repeating units, which are responsible for the regular tertiary fold of the repeat domain. The efficacy of the measure is tested on one of the most abundant repeat families, Ankyrin, by developing a rule based approach based on the observed pattern in eigenvector centrality profile of all known members of the family. It is evaluated on a benchmark dataset and the prediction compared with UniProt annotation, and other state-of-the-art sequence and structure based approaches. We extended the approach by developing a generalized algorithm, named PRIGSA, for the identification of structural repeats using the eigenvector centrality profile (Alevc) and secondary structure architecture (SSA). The algorithm identifies members of known families by comparing the Alevc profile and SSA of a query structure with pre-computed profiles of known families, and also de novo captures novel patterns to identify previously uncharacterized novel repeat proteins. The performance of the approach is tested on other known benchmark datasets and the performance compared with two repeat identification methods. The algorithm is improved to develop PRIGSA2 algorithm by incorporating structure based validation of predicted repeats by structural alignment and checking the relative orientation of secondary structures within the repeat copy. The predictions of the approach for 13 well-represented repeat families are compared with annotation in UniProt, comprising both solenoid and non-solenoid repeats. The PRIGSA2 algorithm is executed on all protein chains reported in the Protein Data Bank to build a database of structural repeats in proteins, DbStRiPs. The de novo predicted repeats are clustered and manually curated. We report one novel repeat family, Left-handed Beta Helix (LbH), and 31 De novo Protein Repeat Clusters (DPRCs). We anticipate the clusters of structurally similar repeat proteins, DPRCs, may lead to novel families in subsequent updates of DbStRiPs with the increase in structural data. In DbStRiPs, 11,901 repeats are reported in 10,816 PDB chains, which are categorized into known protein repeat families (KPRF) (74%), DPRC (6%) or Unclassified (20%). The repeat families and clusters are grouped in the database based on Kajava’s structural classification, and the unclassified repeats are grouped based on the secondary structure composition of the repeating unit. We also developed a web server for network analysis of protein structures, NAPS, providing a comprehensive platform for construction and analysis of protein structure graphs. Five types of networks can be constructed and various types of analyses including node/edge centrality, shortest paths, k-clique and graph spectral analyses can be carried out. The web server provides browser independent interactive platform for visual analysis of protein structure and network along with options to download the results in suitable formats.

Abstract

There has been an immense increase in the use of Deep Neural Networks in recent times due to the availability of more data and greater computing power. With their recent success, it has been a trend to use them extensively in real-time applications. However, the size of deep models can render them incapable to be used in devices with memory-constraints. In this thesis, we explore the several neural network compression techniques for three separate tasks namely i) visual place recognition, ii) object recognition and iii) visual localization. We explore explicit compression methods for the visual place recognition task and the object recognition task, achieved by making modifications to the learned weight matrices. Furthermore, we look at compression attained through architectural modifications in the network itself, proposing novel training procedures and new loss functions for object recognition and visual locali zation. The task of visual place recognition requires us to correctly identify a place given its image, by finding out images of the same place in the world(dataset). Performing this on low memory devices such as mobile phones and robotics systems, is a challenging problem. The state of the art models for this task uses deep learning architectures having close to 100 million parameters which take over 400MB of memory. This makes these models infeasible to be deployed in low memory devices and gives rise to the need of compressing them. Hence, we study the effectiveness of explicit model compression techniques like trained quantization and pruning, on one of the most effective visual place recognition models. We show that a compressed network can be created by starting with a pre-trained model and then fine-tuning it via trained pruning and quantization. Through this training method, the compressed model is able to produce the same mAP as the original uncompressed network. We achieve almost 50% parameter reduction through pruning with no loss in mAP and 70% reduction with close to 2% mAP reduction, while also performing trained 8-bit quantization. Furthermore, together with 5-bit quantization, we perform about 50% parameter reduction by pruning and get only about 3% reduction in mAP. The resulting compressed networks have sizes of around 30 MB and 65 MB which makes them easily usable in memory constrained devices. We next move on to compression through low rank approximation for the task of image classification. Traditional compression algorithms in deep networks involves performing low-rank approximations on the learned weight matrices after the training procedure has been completed. We propose to perform low rank approximation during training itself and make the parameters of the approximated matrix learnable too by using a suitable loss function. We show that by using our method, we are able to compress a base-model providing 89% accuracy, by 10x, with some loss in performance. Using our compression based training procedure, our compressed model is able to achieve an accuracy of about 84%. Next, we focus on developing compressed models for the object recognition task and propose a novel architecture for the same. Deep neural networks for image classification typically consists of a convolutional feature extractor followed by a fully connected classifier network. The predicted and the ground truth labels are represented as one hot vectors. Such a representation assumes that all classes are equally dissimilar. However, classes have visual similarities and often form a hierarchy. We propose an alternate architecture for the classifier network called the Latent Hierarchy Classifier which can discover a latent hierarchy of the classes while simultaneously reducing the number of parameters used in the original classifier. We show that, for some of the best performing architectures on CIFAR and Imagenet datasets, our proposed alternate classifier and training procedure, recovers the accuracy. Also, our proposed method significantly reduces the parameter complexity of the classifier. We achieve a reduction in the number of parameters of the classification layer by 98% for CIFAR 100 and 41% for the Imagenet 1K dataset. We also verify that many visually similar classes are grouped together, under the learnt hierarchy. Finally, we address the problem of Visual Localization where the task is to predict the camera orientation and pose of the given input scene. We propose an anchor point classification based solution for this task by using single camera images only. Our proposed three-way branching of the feature extractor into an Anchor Point Classifier, a Relative Offset Regressor and an Absolute Regressor, is able to achieve <2m translation localization and <5 ◦ pose localization on the Cambridge Landmarks dataset, while also obtaining state-of-the-art in median distance localization for orientation for all the 6 scenes. Our method not only uses fewer parameters than previous deep learning base

Abstract

Risk is an estimation of a desired outcome not achieved, but is central to decision making. Here, context and condition define decision making and risk. Hence, it is imperative for research to examine human responses in different situations to reduce stereotype threat. In this thesis, an attempt is made to understand general stereotypes around gender, trust/emotions and education training with risk taking propensity. Two gambling experiments were designed for this work. One uses the European Roulette Wheel while the other is a modified version of the traditional two player Centipede Game. Emotional changes due to the evaluation of trust and the decision making anxiety are measured using somatic state markers (skin conductance and heart rate). To look for activation at the neural or cortical level electroencephalography (EEG) has been used and the P300 event-related potential waveforms were analysed for variation in game play. Individual game play using the roulette wheel corroborates past research of women being more risk averse than men while the two player game analysis shows women to be taking a higher/similar risk than men. Results from the education stream (engineering and social sciences) comparison indicate that participants from social sciences take significantly higher risks than participants from engineering in individual play though co-player presence shows the opposite trend, which we attribute to probable competition in engineering training. Gender gap was observed in the individual play in both the education streams. While no gender gap was noticed in engineering when co-playing, the gap still persisted in social sciences. Results of this experiment are important as it recommends a re-look at stereotypes, either gender based or education stream, determining access to decision making roles. The behavioral results from the centipede game reveal trust as a probable dominant factor as the results show a direct roportionality between game play turns and trust scores. This was estimated by the termination of the game at a later node with higher trust score between the players. Considering the physiological variation, an overall increase in skin conductance and heart rate during the game-play is noticed compared to the baseline data as expected when undertaking any cognitive task. Interestingly, opponent's action showed higher skin conductance amplitude than during self-play for the same decision, which implies that the cognitive process applied to form prediction model of other's game choices is higher than during self decision making. Preliminary EEG data showed that P300 amplitudes were higher for positive feedback (opponent's pass) than for negative feedback (opponent's take). Additionally, the P300 amplitude was higher for a ’take’ than ’pass’ and we attribute this difference to competitive game play dynamics. P300 waveform was identified from the electrodes covering the frontal and parietal brain region, the areas supporting a cognitive process like decision making. Though not conclusive due to the sparse channel EEG device used, the findings indicate the possibility of measuring a complex process like decision making under risk using EEG technique. Further work is being conducted using dense channel EEG system for conclusive results of neural correlates.

Abstract

User generated content, mainly through social media, blogs and review websites etc. offers a powerful outlet for peoples thoughts and feelings. This presents an enormous ever-growing source of texts ranging from everyday observations to involved discussions. This thesis approaches problems of sentiment analysis from recently popularized perspectives. We explore how sentiment of the user generated text can be computed using a combination of heuristics and machine learning approaches. This thesis contributes to the field of sentiment analysis, which aims to extract emotions and opinions from text. Social Media and other mediums of user generated content provide a powerful source for aggregating peoples’ thoughts and feelings towards a particular entity, event or topic. A robust sentiment analysis system is helpful for the business entities towards which the sentiment is expressed, providing them a solution to understand users’ perception towards the product and services they offer. It is also helpful for other users in form of summary of reviews about a product and service the user wishes to pay for. There are hundreds of websites providing opinionated user-generated content, it is difficult to go through all and then make some decision. Automated sentiment detection helps identify pros and cons of a product with ease. Sentiment analysis system presents a reliable tool for decision making for various parties. This thesis begins with sentiment analysis on social media using machine learning methods. Various signals were specifically extracted for social media setting. A method for fine grain sentiment analysis is presented using a sequence labelling approach combined with heuristics for improved features. To further enhance the sentiment analysis on social media, a solution is proposed for sentiment analysis of Code Mixed data, which is otherwise treated as noise and thus huge amount of information is lost. We conclude that feature engineering for standard machine learning methods and deep learning provide a reliable solution for sentiment analysis.

Abstract

In a last few years, there has been a surge of interest in the field of autonomous navigation and exploration using drones. While there have been many advances made in the field, there has been various challenges, yet to be tackled, for a safe navigation of Quad-copters amidst uncertain dynamic obstacles. In this work, we attempt to tackle this problem by incorporating positional uncertainties of drone and moving obstacles into a joint trajectory optimization framework. In this thesis, we formulate a novel trajectory optimization scheme that takes into consideration the state uncertainty of the robot and obstacle into its collision avoidance routine. The collision avoidance under uncertainty is modeled here as an overlap between two distributions that represent the state of the robot and obstacle respectively. Our framework is a generic framework and the idea proposed here can be used to for any sets of distributions with characterizable overlap. In the scope of this thesis, we model these distributions as Gaussian distributions. We adopt the minmax procedure to characterize the area of overlap between two Gaussian distributions, and compare it with the method of Bhattacharyya distance. Bhattacharyya distance is an approximate closed form characterization of overlap between two Gaussian distribution. The Bhattacharyya distance is generalizable to other distributions too, if their overlap can be characterized through some analytical closed form solution. In this work, We provide closed form expressions that can characterize the overlap as a function of control. We establish that our closed form approximations to characterize overlap is less erroneous when compared with entropic distances like Bhattacharyya distance. Our proposed algorithm can avoid overlapping uncertainty distributions in two possible ways. Firstly when a prescribed overlapping area that needs to be avoided is posed as a confidence contour lower bound, control commands are accordingly realized through a MPC framework such that these bounds are respected. Secondly in tight spaces control commands are computed such that the overlapping distribution respects a prescribed range of overlap characterized by lower and upper bounds of the confidence contours. We test our proposal with extensive set of simulations carried out under various constrained environmental configurations. We show usefulness of proposal under tight spaces where finding control maneuvers with minimal risk behavior becomes an inevitable task.

Abstract

With an immense amount of data growing on the web in Hindi, a text summariser is helpful in summarising government data, medical reports, news, and research articles or any other digitalized documented data. The volume of available text has increased beyond the reading capacity of a single user — a search term on the web results in thousands of relevant pages. Recent improvements in the search engines yield better ranked documents that aim to provide precise information. However, these documents in itself are significantly huge to be read in a few minutes. Hence, the availability of summarisation tools in this domain is of utmost importance.According to Wikipedia, Hindi is the fourth most-spoken first language in the world. Hindi is written in the Devanagari script. Hindi is growing digitally and serves nearly a billion people residing in India. Summarising Hindi news articles is essential. As of now, not enough work has been done for summarising Hindi news articles automatically. Although the dominant form of news consumption in India is through newspapers, digital news consumption is becoming increasingly popular. With the world moving towards virtual reality and digitalization using smartphones, there is a need to summarise the data to make it more readable and concise. Summaries provide an efficient way to deal with the growth of data. A news summariser can be built to manage the issue of data growth on a large scale. A gradual move towards summarization of the digitally available Hindi news would lead to a higher population reading news efficiently. We have aimed to analyze the following approaches for text summarization: 1. Frequency-based approach 2. Graph-based approach 3. Feature-based approach We developed an end to end extractive text summariser for news articles and tested it on a dataset of about 24 thousand articles. We achieved excellent results so far. The extractive dataset of 200 article-summary pair developed as a part of this thesis is of great use for the research community. Currently, there is no public dataset for extractive summarisation available in Hindi. To solve the issue, we extracted a dataset of 24253 news articles. We evaluated extractive summary results on various parameters with manual gold abstractive summaries of exactly 60 words each. For evaluation purposes, we created a standard article summary pair, of the same type of summaries. In the gold data of extracted news articles, summaries were in abstractive form, and we developed an extractive summariser. So we manually created the gold dataset of extractive summaries of 200 articles to test the summariser on the ideal platform.

Abstract

In recent years, with the proliferation of devices capable of capturing and consuming multimedia content, there is a phenomenal increase in multimedia consumption. And most of this is dominated by video content. This creates a need for efficient tools and techniques to create videos and better ways to render the content. Addressing these problems, in this thesis we focus on (a) Algorithms for efficient video content adaptation (b) Automating the process of video content creation. To address the problem of efficient video content adaptation, we present a novel approach to optimally retarget videos for varied displays with differing aspect ratios by preserving salient scene content discovered via eye tracking. Our algorithm performs editing with cut, pan and zoom operations by optimizing the path of a cropping window within the original video while seeking to (i) preserve salient regions, and (ii) adhere to the principles of cinematography. Our approach is (a) content agnostic as the same methodology is employed to re-edit a wide-angle video recording or a close-up movie sequence captured with a static or moving camera, and (b) independent of video length and can in principle re-edit an entire movie in one shot. The proposed retargeting algorithm consists of two steps. The first step employs gaze transition cues to detect time stamps where new cuts are to be introduced in the original video via dynamic programming. A subsequent step optimizes the cropping window path (to create pan and zoom effects), while accounting for the original and new cuts. The cropping window path is designed to include maximum gaze information and is composed of piecewise constant, linear and parabolic segments. It is obtained via L(1) regularized convex optimization which ensures a smooth viewing experience. We test our approach on a wide variety of videos and demonstrate significant improvement over the state-of-the-art, both in terms of computational complexity and qualitative aspects. A study performed with 16 users confirms that our approach results in a superior viewing experience as compared to the state of the art and letterboxing methods, especially for wide-angle static camera recordings. As the retargeting algorithm takes a video and adapts it to a new aspect ratio, we can only use the existing information in the video which limits the applicability. In the second part of the thesis, we address the problem of automatic video content creation by looking into the possibility of using deep learning techniques for automating cinematography. This type of formulation gives more freedom to the users to create content according to some their preferences. Specifically, we investigate the problem of predicting shot specification from the script by learning this association from real movies. The problem is posed as a sequence classification task using Long Short Term Memory (LSTM) network, which takes as input the sentence embedding and a few other high level structural features (such as sentiment, dialogue acts, genre etc.) corresponding a line of dialogue and predicts the shot specification for the corresponding line of dialogue in terms of Shot-Size, Act-React and Shot-Type categories. We have conducted a systematic study to find out effect of the combination of features and the effect of input sequence length on the classification accuracy. We propose two different formulations of the same problem using LSTM architecture and extensively studied the suitability of each of them to the current task. We also created a new dataset for this task which consists of 16000 shots and 10000 dialogue lines. The experimental results are promising in terms of quantitative measures (such as classification accuracy and F1-score).