Abstract

This thesis addresses the critical challenge of improving road safety by introducing novel approaches to predictive modeling of accident-prone zones and action recognition in critical traffic scenarios. It makes two key contributions: the early identification of accident-prone zones using Advance Driving Assistance System (ADAS) data and the development of IDD-CRS, a comprehensive dataset for action recognition in unstructured road environments. In the first study, geo-tagged collision alert data from a fleet of 200 ADAS-equipped city buses in Nagpur, India, is leveraged to proactively identify high-risk zones across urban road networks. Using Kernel Density Estimation (KDE), this study captures the spatiotemporal distribution of collision alerts, enabling the detection of emerging blackspots before accidents occur. A novel recall-based metric evaluates the alignment of these predicted zones with historical blackspots, while Earth Mover Distance (EMD)-based analysis identifies previously unreported accident-prone areas. This predictive framework provides civic authorities with actionable insights for targeted interventions, such as traffic-calming measures and infrastructure improvements, thereby enhancing public safety. The second part of the thesis introduces the IDD-CRS dataset, a large-scale collection of traffic scenarios recorded using ADAS and dash cameras. IDD-CRS fills a critical gap in existing datasets by focusing on complex interactions between vehicles and pedestrians, with scenarios such as high-speed lane changes, unsafe vehicle approaches, and near-miss incidents. With precise temporal annotations powered by ADAS technology, the dataset ensures accurate event boundaries, providing a robust benchmark for action recognition and long-tail action recognition tasks. It includes 90 hours of footage spanning 5,400 one-minute videos and 135,000 frames, with hard negative examples to challenge existing models. Initial benchmarks highlight the limitations of current video backbones in recognizing rare events, emphasizing the need for further advancements. Together, these contributions provide a holistic framework for improving road safety through proactive accident prevention and robust action recognition in traffic scenarios. By addressing both spatial accident prediction and temporal event recognition, this work offers foundational resources and actionable insights to advance research and practical solutions for safer road environments.

Abstract

Absolute separable (AS) quantum states are those states from which it is impossible to create entanglement, even under global unitary operations. It is known from the resource theory of non-absolute separability that the set of absolute separable states forms a convex and compact set, and global unitaries are free operations. We show that the action of a quantum switch controlled by an ancilla qubit over the global unitaries can break this robustness of AS states and produce ordinary separable states. First, we consider bipartite qubit systems and find the effect of quantum switch starting from the states sitting on the boundary of the set of absolute separable states. As particular examples, we illustrate what happens to modified Werner states and Bell diagonal (BD) states. For the Bell diagonal states, we provide the structure for the set of AS BD states and show how the structure changes under the influence of a switch. Further, we consider numerical generalization of the global unitary operations and show that it is always possible to take AS states out of the convex set under switching operations. We also generalized our results in higher dimensions. Several recent studies have demonstrated the utility of the quantum SWITCH as an important resource for enhancing the performance of various information processing tasks. In a quantum SWITCH, the advantages appear significantly due to the coherent superposition of alternative configurations of the quantum components which are controlled by an additional control system. Here we explore the impact of increasing the Hilbert-space dimension of the control system on the performance of the quantum SWITCH. In particular, we focus on a quantifier of the quantum SWITCH through the emergence of non-Markovianitxy and explicitly study their behavior when we increase the Hilbert-space dimension of the control system. We observe that increasing the Hilbert-space dimension of the control system leads to the corresponding enhancement of the non-Markovian memory induced by it. Our study demonstrates how the dimension of the control system can be harnessed to improve the quantum SWITCH-based information processing or communication tasks

Abstract

A document has information emerging out of the existence of various physical entities or regions such as headings, paragraphs, figures, captions, tables, and backgrounds along with the textual content. To decipher the information in a document, a human reader uses a variety of additional cues, such as context, conventions, and information about language, script, location, and a complex reasoning process. Millions of documents are created and distributed daily over the Internet and printed media. Understanding, analyzing, sorting, and comparing a massive collection of documents in a limited time is a hectic job for humans. Here, the automatic document image understanding systems (DIUS) help humans do this tedious task within a limited time. The DIUS have typically a document image layout segmentation module and information extraction modules. This thesis focuses on the challenging problems related to document image layout segmentation in various types of documents and their applications. In this thesis, first, we analyse the various document images using deep features. The deep features are the features extracted using a pretrained deep neural network. To study deep texture features, we propose a deep network architecture that independently learns texture patterns, discriminative patches, and shapes to solve various document image analysis tasks. The considered tasks are document image classification, genre identification from book covers, scientific document figure classification, and script identification. The presented network learns global, texture, and discriminative features and combines them judicially based on the nature of the problems. We compare the performance of the proposed approach with state-of-the-art techniques on multiple publicly available datasets such as Book covers, RVL-CDIP, CVSI and DocFigure. Experiments show that our approach obtains significantly better performance over state-of-the-art for all tasks. Next, we focus on the problem of document image layout segmentation and propose a solution for a class of document images, including historical, scientific, and classroom slide document images. The historical document image segmentation problem is modeled as a pixel labeling task where each pixel in the document image is classified into one of the predefined labels, such as text, comment, decoration, and background. The method first extracts deep features from the superpixels of the document image. Then, we learn SVM classifier using these features and segment the document image. The pyramid pooling module is used to extract the logical regions of scientific document images. In the classroom slide images, the logical regions are distributed based on the location of the image. To utilize the location of the logical regions for slide image segmentation, we propose the architecture, Classroom Slide Segmentation Network (CSSN). The unique attributes of this architecture differ from most other semantic segmentation networks. We validate the performance of our segmentation architecture using publicly available benchmark datasets. Next, we analyze the output regions of document layout segmentation. Figures used in the documents are the complex regions to decipher the information by a DIUS. Hence, document figure classification (DFC) is an important stage of the DIUS. The design of a DFC system required well-defined figure categories and datasets. Existing datasets related to the classification of figures in the document images are limited in size and category. We introduce a scientific figure classification dataset named DocFigure. The dataset consists of 33K annotated figures of 28 different categories present in the document images, which correspond to scientific articles published in last several years. Manual annotation of such a large number (33K) of figures is time-consuming and cost-ineffective. We design a web-based annotation tool that can efficiently assign category labels to many figures with the minimum effort of human annotators. To benchmark our generated dataset on the classification task, we propose three baseline classification techniques using the deep feature, deep texture feature, and both. Our analysis found that the combination of both deep and texture features is more effective for document figure classification tasks than individual features. Finally, we propose the application backed by the research of this thesis. Slide presentations are an effective and efficient tool for classroom communication used by the teaching community. However, this teaching model can be challenging for blind and visually impaired (VI) students as such students require personal human assistance to understand the presented slide. This shortcoming motivates us to design a Classroom Slide Narration System (CSNS) that generates audio descriptions corresponding to the slide content. This problem poses an image-to-markup language generation task. Ext

Abstract

Unlike the classical scenario, conditional entropy in quantum systems can be negative. Such systems play a pivotal role in information-processing tasks such as super dense coding, one-way entanglement distillation, and state merging, to name a few. Quantum conditional entropy’s central role led to the formulation of a resource theoretic study, establishing the notion of free and resource states and a char- acterization of free operations. In this thesis, we delve into the study of quantum channels that affect the negative conditional entropy of a quantum system. The main focus of this thesis is the characterization of these channels from topological and information- theoretic perspectives. We consider two classes of quantum channels - Negative Conditional Entropy Breaking (NCEB) channels and Negative Conditional Entropy Annihilating (NCEA) channels. We de- fine such channels and examine their properties when combined serially. We also consider the property of parallel combinations of NCEB channels. Our exploration extends to complementary channels of NCEB, whereby we discover the information-leaking nature of NCEB channels. We exemplify NCEB and NCEA channels through standard depolarizing channels and further the characterization. Specif- ically for NCEA channels, we establish a sufficiency condition for global depolarizing channels and transpose depolarizing channels. Following this, we establish the relationship of NCEB and NCEA with newly introduced classes of quantum channels, such as Coherent Information Breaking (CIB) and Mutual Information Breaking (MIB), along with standard channels like zero capacity and entanglement breaking channels. Finally, we consider the perspective of maintaining negative conditional entropy in quantum states, which involves the identification of non-NCEB or non-NCEA channels. In this regard, we provide prescriptions to detect channels that do not affect the negativity of conditional entropy.

Abstract

The social isolation required by the pandemic further amplified the internet’s utility as a medium of communication and information for people with digital access. Social media was one of the most popular forms of communication among young people with digital access during the COVID-19 pandemic. Consequently, crucial debates and discussions about the pandemic crisis have also developed on social media platforms, making them a great primary source to study the experiences of specific groups and communities during the pandemic. Furthermore, online news media outlets were an important source of information for people with digital access during the COVID–19 pandemic. This study consisted of research on social media and online news websites in order to understand the experiences of as well as attitudes towards LGBTQ+ people during the pandemic. We used LDA topic modelling and sentiment analysis on Reddit data to understand the themes and attitudes in circulation within five subreddits devoted to LGBTQ+ experiences and issues. In the process, we attempt to make sense of the role that social media platforms may have played in the lives of LGBTQ+ people who were online during the pandemic (10 March 2020–31 August 2021) and whether this was marked by any continuities or discontinuities from before the pandemic period (January 2019–December 2019). We also examined articles from online editions of two English–language Indian newspaper websites, which differed vastly in their circulation figures—The Times of India and The Indian Express. Focusing on the period spanning March 2020 to August 2021, we analysed articles from our chosen newspaper websites using distil–RoBERTa–base and ChatGPT–3.5 for sentiment analysis and BERTopic for topic modelling. Furthermore, we manually analysed a part of our dataset obtained from Reddit and online newspaper websites to arrive at a nuanced, granular understanding of our quantitative results. The results of our research on Reddit reveal that institutions such as the health care system, justice system, and legislative system perpetuated systemic inequalities against LGBTQ+ communities, thereby adding to the pre–existing stigma against them during a global crisis. Furthermore, the results from our topic model suggest a shift in the discussions of prominent topics in the subreddits from before the pandemic period, with a preponderance of discussions on political events. Similarly, the results from our sentiment analysis show an increase in positive comments and neutral comments and a decrease in negative comments, which does not necessarily indicate a loss of users’ interest in the subreddits. While analysing online newspaper websites, we found a contrast between the quality, quantity, and attitudes prevalent across The Times of India and The Indian Express during the COVID–19 pandemic in the context of LGBTQ+ communities. Our topic modelling results indicate more meaningful coverage by The Indian Express compared to The Times of India, which, however, does not mean sufficient coverage by The Indian Express to provide a comprehensive picture. Similarly, our sentiment analysis results indicate a difference in prevailing sentiment as depicted by the two models. Both newspaper websites exhibited a greater focus on transgender communities. Our research on Reddit has demonstrated what non–cis–heteronormative spaces—virtual and real— can look or feel like. By offering a more rigorous engagement with the data available on social media platforms, this research contributes to current debates in queer and social media studies about the empowering or disempowering nature of social media platforms for LGBTQ+ voices online. Furthermore, it captures the visibility and representation of the LGBTQ+ communities during the pandemic in online Indian news media outlets. By doing so, we contribute to the larger understanding of how Indian media narrated the lived realities of LGBTQ+ communities and what attitudes such representation reveal towards vulnerable communities in India during the pandemic. Overall, this thesis expands the understanding of the kind of discursive spaces and representation offered by online platforms for LGBTQ+ communities during a global crisis, demonstrating both possibilities and potential as well as challenges and problems.

Abstract

In autonomous driving, we have multiple computer vision tasks like object detection, semantic seg- mentation, and instance segmentation which plays a crucial role in perceiving the environment around the vehicle. Understanding the behaviour and performance of such tasks helps improve and address the key issues that are inherent in the system. There can be issues which are latent in the deep learning architecture and also in the datasets on which the deep learning models are trained and tested. In this thesis, we benchmark the performance of various popular deep learning models on road scene datasets for various computer vision tasks and also formulate open-world object detection on road scenes by addressing the inherent issues present in road scene datasets. In the first part of the work, we aim to understand the performance and behaviour of various deep learning models on road scene datasets; Cityscapes, IDD, and BDD. Object detection, semantic seg- mentation, and instance segmentation form the bases for many computer vision tasks in autonomous driving. The complexity of these tasks increases as we shift from object detection to instance seg- mentation. The state-of-the-art models are evaluated on standard datasets such as PASCAL-VOC and MS-COCO, which does not consider the dynamics of road scenes. Driving datasets such as Cityscapes and Berkeley Deep Drive(BDD) are captured in a structured environment with better road markings and fewer variations in the appearance of objects and background. However, the same does not hold for Indian roads. The Indian Driving Dataset(IDD) dataset is captured in unstructured driving scenarios and is highly challenging for a model due to its diversity. This work presents a comprehensive evaluation of state-of-the-art models on object detection, semantic segmentation, and instance segmentation on road scene datasets. We present our analyses and compare their quantitative and qualitative performance on structured driving datasets(Cityscapes and BDD) and the unstructured driving dataset(IDD); under- standing the behavior on these datasets helps in addressing various practical issues and helps in creating real-life applications. In the second part of the work, we model open-world object detection on road scenes since object detection is a crucial component in autonomous navigation systems. Current object detectors are trained and tested on a fixed number of known classes. However, in real-world or open-world settings, the test set may consist of objects of unknown classes; this results in the unknown objects being falsely de- tected as known objects leading to the failure in the decision making of autonomous navigation systems. We propose Open World Object Detection on Road Scenes (ORDER) to resolve the aforementioned problem. We introduce Feature-Mix that widens the gap between known and unknown classes in latent feature space and improves the unknown object detection in the ORDER framework. We identify the inherent problems present in autonomous datasets: i) a significant proportion of the dataset comprises small objects and ii) intra-class bounding box scale variations. We address the problem of small object detection and intra-class bounding box variations by proposing a novel focal regression loss. Further, the detection of small objects is improved by curriculum learning. We present an extensive evaluation of two road scene datasets: BDD and IDD. Our experimental evaluations on BDD and IDD shows con- sistent improvement over the current state-of-the-art method. We believe that this work will lay the foundation for real-world object detection for road scenes.

Abstract

In spite of the vast work in the area of Maximal Frequent Subgraph (MFS) mining, current algorithms do not scale to large databases with numerous graphs. Even the smallest search space required by current MFS algorithms for large databases is too huge to work within the main memory alone, making the algorithms I/O bound. Added to this is the expensive cost of subgraph isomorphism operation, which is commonly used for frequency computation. This thesis explores partition-based secondary memory algorithms and proposes Partition-based Frequent Subgraph Mining using Maximum Common Subgraph (PMCS), that can make maximal frequent subgraph mining for large databases viable. The naive application of the partition-based approach is inefficient due to the need to recompute subgraph frequency and the possibility of storing intermediate subgraphs on the disk. In PMCS, we overcome these drawbacks by intelligently avoiding redundant computations and performing frequency computation operations optimally using the Maximum Common Subgraph operation. Our intermediate results are also small enough to fit in the main memory. While existing algorithms fail to scale beyond 300k, our results demonstrate that PMCS scales to databases of up to 1000k graphs with an average of 25-30 edges per graph.

Abstract

Spoken utterances have an inherent structure in the sense that some words seem to group naturally together and some words seem to have a notable break or disjuncture between them. This can be described in terms of prosodic phrasing, meaning that a spoken utterance has a prosodic phrase structure, similar to how a written utterance has a syntactic phrase structure. Phrase breaks in natural speech are important; they are physiologically essential, help emphasize content, and improve the intelligibility of speech. The process of inserting phrase breaks in an utterance is called phrasing. In the context of speech synthesis, phrasing is a crucial step. It breaks long utterances into meaningful units of information and improves the intelligibilty of the synthesized speech. Traditional methods of phrase break prediction have used discrete linguistic resources like part-ofspeech (POS) sequence information for modeling these breaks. These methods cannot be used for languages where the necessary linguistic resources are not readily available, resulting in unsupervised methods of inducing word representations which can be used as surrogates for POS tags in phrase break prediction. However, these methods are not suitable in the context of Indian languages, which are aggutinative in nature resulting in an increase in vocabulary size. This thesis presents the use of word terminal syllables (last syllable of the word) to model phrase breaks. We demonstrate the correlation between these terminal syllables and acoustic breaks found in the speech signal. We utilize these terminal syllables in building models for phrase break prediction from text in six Indian languages and demonstrate by means of objective and subjective measures that these models perform as well as traditional models using POS sequence information. The discrete linguistic representations (like POS tags, induced POS tags, word-terminal syllables) used by traditional methods of phrase break prediction require a hard classification of words into a set of predefined discrete classes, which raises issues when there is ambiguity in the linguistic representation of a word. Moreover, such a representation does not capture the co-occurrence statistics of the words, i. e., they do not take into account the distributional behaviour of words. Both these issues can be addressed by the use of continuous dimensional representations of words also known as word embeddings. This thesis presents a neural network architecture to induce task specific word embeddings for phrase break prediction. We show that our proposed architecture combines feature induction and phrase break prediction in a single framework and thus avoids the two stage process required while using features derived from Latent Semantic Analysis (LSA). We train our model on audiobook data and show that these task specific word features are better word representations as compared to those derived using LSA, for phrase break prediction. With the advent of deep learning, there have been attempts to apply deep neural networks (DNNs) to phrase break prediction. While deep neural networks are able to effectively capture dependencies across features, they lack the ability to capture long-term relations that are spread over time. On the other hand, recurrent neural networks (RNNs) are able to capture long-term temporal relations and thus are better suited for tasks where sequences have to be modeled. This thesis presents the use of RNNs for phrase break prediction, and shows by means of experimental results that they perform better than DNNs. Traditional techniques for learning word embddings learn static or context-free word embeddings, meaning that for each word in the vocabulary, one fixed embedding is learnt for all possible contexts in which that particular word appears in the text corpus. While static embeddings have proved to be effective in NLP applications, they do not take into account the fact that words have different meanings in different contexts. To address this issue, context-sensitive word representations, also called dynamic or contextual word embeddings were developed. The most popular technique for learning dynamic word embeddings is BERT (Bidirectional Encoder Representations from Transformers). This thesis presents work on phrase break prediction using bidirectional encoder representations learnt from fine-tuning a pretrained BERT model, with an additional token classification layer, on phrase break prediction. We show that representations using this model outperform task-specific static word embeddings learnt using a BLSTM token classification model trained from scratch. Finally, this thesis presents work which attempts to answer the following questions: (a) Is there any utility in incorporating an explicit external phrasing model in an End-to-End TTS system? and (b) How do you evaluate the effectiveness of a phrasing model in an End-to-End TTS system?

Abstract

Ancient manuscripts were among the first forms of written communication, offering key insights into our past, covering literature, medicine, culture, philosophy, religion, and more. It is imperative to save these writings to identify and extract their hidden knowledge. Document collections frequently exhibit overlapping components, irregular patterns, dense layouts, extremely high aspect ratios, physical and chemical degradation (evident in ink-based manuscripts), text misalignment, and more. Compounding these difficulties are issues that may arise during the digitization processes, such as improper document positioning, inadequate illumination, and scanner effects. In this academic thesis, our main emphasis is on identifying and segmenting text lines inside these documents for downstream OCR applications with utmost precision. We devise a two-stage approach - SeamFormer- to identify special text baselines in palm-leaf manuscripts using a multi-task strategy via Vision Transformers (ViTs). In the first stage, we detect text strikethroughs, namely ’scribbles,’ which act as pointers to the location of text line segments within the document. The encoder-decoder architecture is used to analyze input image patches and produce two separate maps: a scribble map and a binary map. In the second stage, we post-process the prior stage outputs and generate a diacritic-aware global energy map. To generate the final precise text line polygons, we use a modified seam generation algorithm along with customized global maps. The prior methodology is further enhanced by the proposed LineTR model, a multi-task DETR (Detection Transformer) model that reconceptualizes the scribble generation as a geometric problem. This method simply generates line parameters for each text line present in the input image patch. This design decision enables the model to exhibit zero-shot behavior across diverse historical manuscripts. The state-of-the-art approach has been proven to generate precise text-line segmentation with a single ’unified’ model with minimal post-processing efforts, making it a strong candidate for image-to-OCR integration pipelines

Abstract

A Completely Automated Public Turing Test to Tell Computers and Humans Apart (CAPTCHA) is one of the primary barriers between notorious bots and legitimate human users. However, advancements in Artificial Intelligence (AI) have enabled malicious bots to circumvent CAPTCHA challenges effectively. As a result, several types of CAPTCHA have been rendered ineffective. In this work, we introduce Swiss Cheese CAPTCHA, a novel sensor-based solution designed to be easily solvable by humans while presenting multiple obstructions for bots (similar to the Swiss Cheese Model) even when the sensor outputs can be predicted and interfered with. We leverage a range of human cognitive abilities and Generic Sensor API in modern devices to provide robust protection against automated attacks by making it more computationally expensive for bots to produce a valid answer within a stipulated time. We conducted two user studies to assess our proposal’s effectiveness: one involving 116 participants to assess the likability and improvise the design, and the other, with 107 participants, to investigate the impact of improvised design changes on cognitive abilities. Our results from these studies show an average completion time of 4.76 seconds and 6.12 seconds, with a success rate of 90.3% and 83.25%, respectively. By analyzing the 2141 resultant trajectories from both user studies, we assess the learnability, error recovery rate, efficiency, and satisfaction of users using the scheme. Finally, we devise an automated attack against our proposal to analyze its security in the real world; we find the probability of attack success is low. We also make our dataset available for further research.