Abstract

Ancient manuscripts were among the first forms of written communication, offering key insights into our past, covering literature, medicine, culture, philosophy, religion, and more. It is imperative to save these writings to identify and extract their hidden knowledge. Document collections frequently exhibit overlapping components, irregular patterns, dense layouts, extremely high aspect ratios, physical and chemical degradation (evident in ink-based manuscripts), text misalignment, and more. Compounding these difficulties are issues that may arise during the digitization processes, such as improper document positioning, inadequate illumination, and scanner effects. In this academic thesis, our main emphasis is on identifying and segmenting text lines inside these documents for downstream OCR applications with utmost precision. We devise a two-stage approach - SeamFormer- to identify special text baselines in palm-leaf manuscripts using a multi-task strategy via Vision Transformers (ViTs). In the first stage, we detect text strikethroughs, namely ’scribbles,’ which act as pointers to the location of text line segments within the document. The encoder-decoder architecture is used to analyze input image patches and produce two separate maps: a scribble map and a binary map. In the second stage, we post-process the prior stage outputs and generate a diacritic-aware global energy map. To generate the final precise text line polygons, we use a modified seam generation algorithm along with customized global maps. The prior methodology is further enhanced by the proposed LineTR model, a multi-task DETR (Detection Transformer) model that reconceptualizes the scribble generation as a geometric problem. This method simply generates line parameters for each text line present in the input image patch. This design decision enables the model to exhibit zero-shot behavior across diverse historical manuscripts. The state-of-the-art approach has been proven to generate precise text-line segmentation with a single ’unified’ model with minimal post-processing efforts, making it a strong candidate for image-to-OCR integration pipelines

Abstract

The surge in robotics, IoT, machine vision, biometrics, security cameras, and smart devices has led to a rising demand for image sensors and cameras. Complementary Metal-Oxide-Semiconductor (CMOS) light sensors have become the heart of modern digital imaging, widely utilized in diverse applications ranging from consumer electronics to industrial and scientific domains, especially in highly sensitive areas, low luminescent conditions, labs on chips, etc. This thesis centers around the CMOS light sensors, discussing their architecture, performance, and integration. The research mainly focuses on CMOS-compatible photodiodes and single-photon avalanche diodes (SPADs). For the CMOS photodiodes, this work’s primary idea is to enhance their light sensitivity and photon collection efficiency and integrate these photodiodes with the circuits based on standard CMOS processes. This involves the ability to modify the structure or shape of the photodiode in order to increase the efficiency of photon collection when the photodetector is in use. This research work tackles low-light imaging by using single-photon avalanche diode detectors, which efficiently identify single photons. Another important consideration when using SPAD in such an application is using a high-speed quenching circuit to reduce afterpulsing. Different SPAD architecture simulation models are reviewed and compared to model the SPAD accurately. In this work, an active quenching circuit is designed to facilitate fast quenching and minimal dead time.The work encompasses extensive simulation and modeling using tools such as Cadence Virtuoso and COMSOL Multiphysics, allowing for precise evaluation of the proposed design’s performance metrics. The results show significant enhancements in sensor performance and the possibility of expanding its use in biomedical imaging applications, robotics, and other sensitive imaging needs.

Abstract

A Completely Automated Public Turing Test to Tell Computers and Humans Apart (CAPTCHA) is one of the primary barriers between notorious bots and legitimate human users. However, advancements in Artificial Intelligence (AI) have enabled malicious bots to circumvent CAPTCHA challenges effectively. As a result, several types of CAPTCHA have been rendered ineffective. In this work, we introduce Swiss Cheese CAPTCHA, a novel sensor-based solution designed to be easily solvable by humans while presenting multiple obstructions for bots (similar to the Swiss Cheese Model) even when the sensor outputs can be predicted and interfered with. We leverage a range of human cognitive abilities and Generic Sensor API in modern devices to provide robust protection against automated attacks by making it more computationally expensive for bots to produce a valid answer within a stipulated time. We conducted two user studies to assess our proposal’s effectiveness: one involving 116 participants to assess the likability and improvise the design, and the other, with 107 participants, to investigate the impact of improvised design changes on cognitive abilities. Our results from these studies show an average completion time of 4.76 seconds and 6.12 seconds, with a success rate of 90.3% and 83.25%, respectively. By analyzing the 2141 resultant trajectories from both user studies, we assess the learnability, error recovery rate, efficiency, and satisfaction of users using the scheme. Finally, we devise an automated attack against our proposal to analyze its security in the real world; we find the probability of attack success is low. We also make our dataset available for further research.

Abstract

This thesis presents a comprehensive exploration of Multi-Object Multi-Part Scene Parsing in 2D images, showcasing significant advancements through two novel approaches, FLOAT and OLAF, each tailored to enhance scene parsing performance and scalability. The first paper introduces FLOAT, a factorized label space framework designed to independently predict object categories and part attributes, thereby simplifying the segmentation task and enhancing scalability. Notably, FLOAT incorporates a unique ’zoom’ refinement technique at inference time, significantly elevating segmentation accuracy, particularly for smaller objects and parts. Empirical results on the Pascal-Part datasets underscore FLOAT’s superior performance, achieving notable improvements in mean Intersection Over Union (mIOU) and segmentation quality IOU (sqIOU), especially on the most comprehensive PascalPart-201 dataset, reflecting its effectiveness in handling diverse and complex scenes. The second paper delves into OLAF, a plug-and-play methodology that augments traditional RGB inputs with object-based structural cues to better capture the complexities of scene structures. This approach leverages a weight adaptation technique, allowing pre-trained RGB models to seamlessly integrate augmented data, thus stabilizing the optimization process. Additionally, the introduction of the LDF encoder module aids in providing low-level dense feature guidance, enhancing the segmentation of smaller parts. OLAF demonstrates its versatility across various architectures and datasets, achieving significant mIOU gains on multiple Pascal-Part benchmarks, highlighting its broad applicability and robust performance enhancements in challenging segmentation scenarios.

Abstract

In the era of big data, distributed computing frameworks like Hadoop’s MapReduce have become essential for processing massive datasets across clusters of servers. A critical phase in the MapReduce paradigm is the shuffling stage, where intermediate data produced by Map tasks is transferred to the appropriate Reduce tasks. This data shuffling often incurs significant communication overhead, leading to performance bottlenecks and increased job completion times. A key factor influencing communication overhead is data locality, which refers to the practice of processing data close to its storage location. By minimizing the distance data must travel across a network, data locality reduces latency, conserves bandwidth, and improves overall system performance. To address these challenges, the Coded MapReduce framework was introduced, using coding techniques to create multicast opportunities during the shuffling phase. By replicating Map tasks across multiple servers, this approach effectively reduces inter-server communication load, enhancing overall computational efficiency. Ho, Wu, and Liu demonstrated that optimizing reducer placement to reduce cross-rack traffic can lead to significant performance improvements, achieving up to a 32% enhancement in job execution times. Building upon this foundation, our research presents the Hybrid Coded MapReduce scheme, tailored for server–rack architectures prevalent in contemporary data centers. In such configurations, servers are organized into racks, with intra-rack communication being faster and more efficient than cross-rack communication. Recognizing this disparity, our Hybrid Coded MapReduce approach aims to minimize costly cross-rack data transfers while optimizing data locality. Central to our methodology is the formulation of the Map task assignment problem as a constrained integer optimization problem. This formulation seeks to assign Map tasks to servers in a manner that maximizes data locality, ensuring data is processed in proximity to its storage location. Consequently, the need for cross-rack communication is reduced, leading to decreased overall communication costs and improved job completion times. To validate our approach, we conducted extensive simulations comparing the performance of the Hybrid Coded MapReduce scheme against the traditional Coded MapReduce framework. Our results demonstrate that the Hybrid approach significantly reduces cross-rack communication without compromising the benefits of coded multicasting. Furthermore, by optimizing data locality, our scheme improves data processing efficiency, resulting in faster job completion and better resource utilization. In conclusion, the Hybrid Coded MapReduce scheme offers a practical and effective solution to improve the performance of distributed computing systems within server–rack architectures. By intelligently balancing the trade-offs between communication cost and data locality, our approach paves the way for more efficient and scalable big data processing.

Abstract

In Indian urban and rural driving scenarios, small objects are pervasive and often crucial for safe navigation. These objects can include pedestrians crossing roads, children playing near streets, cyclists, stray animals, as well as small vehicles like scooters and motorbikes. Additionally, traffic signs, signal lights, potholes, and road markings (such as lane dividers or zebra crossings) are often small in size but essential for driving decisions. In such contexts, missing or inaccurately segmenting these small objects can lead to critical errors in detection, causing accidents or delays in the vehicle’s decision-making process. Automated understanding of such objects need detection and segmentation to start with. Semantic Segmentation is a critical task in computer vision with a wide range of applications. The objective is to partition an image—a collection of pixels—into distinct labeled regions, each corresponding to specific objects or parts of the scene. This process is crucial for scene understanding and enables the localization of objects within the image. Over time, significant progress has been made in semantic segmentation, especially with the advent of deep learning. The advances in this area have revolutionized computer vision, pushing beyond traditional methods and achieving remarkable improvements in performance. When discussing semantic segmentation, we often focus on datasets, the objects within those datasets, and their corresponding segmentations. While many datasets exist for road scenarios, particularly those representing Western road conditions, there is relatively little research on road conditions specific to India. One notable exception is the Indian Driving Dataset (IDD), a dataset specifically designed for semantic segmentation of Indian road scenarios. Road and driving datasets typically contain objects of varying sizes within each class label. These objects can be broadly categorized into three types: small, medium, and large. The importance of segmentation is well understood across several domains such as medical imaging, autonomous vehicles, aerial imagery, robotics, surveillance, and industrial automation. However, one of the most challenging problems in segmentation is the segmentation of small objects. Small object segmentation is particularly difficult due to factors such as (i) the limited number of pixels representing small objects, (ii) class imbalance during training, and (iii) the inherent challenges posed by small object representations. These factors hinder the performance of deep learning architectures, making it harder for modern techniques to accurately handle small objects. This issue becomes evident when evaluating deep learning models, as performance metrics tend to be significantly worse for small objects. In this study, we delve into the challenges of segmenting small objects and explore potential strategies for improving algorithm design for this task. It is widely acknowledged that loss functions play a key role in segmentation performance, alongside the architecture of deep learning models. Indeed, the segmentation of small objects is highly influenced by the choice of loss function during model training. Various small objects commonly encountered in road scenarios are highlighted in this study, emphasizing their importance in Indian road environments. Objects such as pedestrians, traffic signs, and traffic lights—despite their small size—are crucial for tasks like autonomous navigation and driver assistance systems. The Indian Driving Dataset (IDD) offers a valuable resource, containing a wide range of small objects across different class labels, some of which dominate specific categories. These objects are captured and annotated in real-world, unstructured environments, making this dataset particularly relevant for segmentation tasks in Indian road scenarios. Therefore, our focus in this study is primarily on detection and segmentation of small objects within the IDD dataset and explore the impact of loss functions.

Abstract

Principal component analysis(PCA) is one of the most popular linear dimensionality reduction techniques in machine learning. In this thesis, we present a method for performing PCA on encrypted data using a homomorphic encryption scheme. We used the CKKS homomorphic encryption scheme for our purpose since it is most suitable for machine learning tasks allowing approximate computations on real numbers. In this thesis, we propose a new Homomorphic PCA(HPCA) algorithm that performs PCA. Unlike previously proposed algorithms, our HPCA algorithm is non-interactive in nature. This requires an approximation of the inverse sqrt function. This thesis presents two methods to approximate and securely perform the inverse sqrt function using CKKS homomorphic encryption scheme - Constrained Linear Regression and Pivot-Tangent method. Since the CKKS homomorphic scheme allows only the computation of polynomial functions, we propose a method to approximate the inverse sqrt function polynomially. Ultimately, we implement our approach for the inverse sqrt function. We provide an implementation of our method for the inverse sqrt function. We use the inverse sqrt approximations in our HPCA algorithm to make it non-interactive. In the end, we present the experimental results of our proposed HPCA algorithm on a few datasets computing a few principal components. We measure the R2 score on the reconstructed data and use it as an evaluation metric for our HPCA algorithm.

Abstract

Chemical retrosynthesis—the process of determining precursor molecules needed to synthesize a target compound—represents a fundamental challenge in drug discovery and organic chemistry. While computational approaches increasingly frame this as a sequence-to-sequence prediction task using SMILES (Simplified Molecular Input Line Entry System) representation for molecules, current evaluation methods assume perfect training data and rely primarily on exact matching metrics. This approach overlooks the chemical reality that multiple viable synthetic routes often exist for a single product. Our research addresses this limitation through two key contributions. First, we develop the RetroSynth Score (R-SS), a comprehensive evaluation framework that recognizes chemically valid alternative predictions by considering structural similarity, stereochemistry variations, and partial matches between predicted and reference reactants. Second, we introduce SynFormer, a transformer-based model with specific architectural enhancements (rotary embeddings and ReLU2 activation) designed for efficient molecular representation processing. Through rigorous evaluation on the standard USPTO-50k dataset of chemical reactions, we demonstrate that SynFormer achieves comparable performance to state-of-the-art pretrained models (53.2% top-1 accuracy) while eliminating the computational burden of extensive pretraining. Our analysis shows a 0.17% improvement in R-SS over previous methods, with significant enhancement in detecting valid alternative pathways. Case studies reveal that many predictions previously classified as incorrect represent chemically viable alternative synthetic routes that differ in factors such as reactivity, cost, and preparation ease. The proposed evaluation metrics provide a more nuanced and chemically relevant assessment of retrosynthesis prediction models, potentially accelerating drug discovery and development processes.

Abstract

This thesis explores the dual objectives of reviewing material characteristics of soft polymers for microfluidic and Lab-on-a-Chip (LOC) devices and developing a cost-effective, flexible microheater for LOC applications. Soft materials such as polydimethylsiloxane (PDMS), hydrogels, thermoplas- tic elastomers, and conducting polymers are analyzed for their mechanical, thermal, chemical, optical, and biocompatibility properties. These polymers play a crucial role in LOC systems due to their abil- ity to provide flexibility, durability, and compatibility with biological samples. The review delves into their fabrication techniques, solubility in common solvents, adhesion, and surface modification meth- ods, highlighting their relevance for applications such as diagnostics, cell culture, and chemical assays. Experimental studies further assess their optical properties and cytotoxicity, offering a comprehensive understanding of their potential for use in biomedical and chemical systems. By comparing the advan- tages and limitations of these materials, this work aids in optimizing material selection for enhanced performance and sustainability in LOC applications. The developed flexible microheater utilizes nichrome wire embedded in a PDMS substrate, chosen for its affordability and durability. The microheater operates within a temperature range of 40°C to 150°C, with precise control achieved through a pulse-width modulation (PWM)-based circuit and feed- back from a 100k Ω NTC thermistor. This setup provides closed-loop temperature regulation, ensuring stability and adaptability for diverse applications. The microheater’s efficacy is demonstrated in two sig- nificant applications: glucose detection and the synthesis of silver nanoparticles (AgNPs). In the glucose detection application, the microheater integrated into a microwell device reliably facilitates Benedict’s test, confirming its potential for point-of-care diagnostics. The microheater is also employed in the syn- thesis of AgNPs using green chemistry principles, utilizing ascorbic acid as a reducing agent and sodium citrate as a stabilizer. This process yields spherical nanoparticles with UV-Vis absorption peaks charac- teristic of AgNPs (between 405 nm and 411 nm). Both applications underline the microheater’s capacity to deliver consistent performance while maintaining low fabrication costs. Experimental results validate the heater’s capability to reach steady-state temperatures quickly and efficiently. The integration of the PWM-based control system ensures rapid response times and enhanced accuracy, addressing challenges in conventional heating systems. Furthermore, the cost of the microheater device is kept below INR 1000, making it an accessible solution for resource-constrained settings.

Abstract

In the field of artificial intelligence, the generation of human-like motion from natural language descriptions has garnered increasing attention across various research domains. Computer vision focuses on understanding and replicating visual cues for motion, while computer graphics aims to create and edit visually realistic animations. Similarly, multimedia research explores the intersection of data modalities, such as text, motion, and image, to enhance user experiences. Robotics and human-computer interaction are pivotal areas where language-driven motion systems improve the autonomy and responsiveness of machines, facilitating more efficient and meaningful human-robot interactions. Despite its significance, existing approaches still encounter significant difficulties, particularly when generating motions from unseen or novel text descriptions. These models often lack the ability to fully capture intricate, low-level motion nuances that go beyond basic action labels. This limitation arises from the reliance on brief and simplistic textual descriptions, which fail to convey the complex and fine-grained characteristics of human motion, resulting in less diverse and realistic outputs. As a result, the generated motions frequently lack the subtlety and depth required for more dynamic and context-specific applications. This thesis introduces two key contributions to overcome these limitations and advance text-conditioned human motion generation. First, we present Action-GPT, a novel framework aimed at significantly enhancing text-based action generation models by incorporating Large Language Models (LLMs). Traditional motion capture datasets tend to provide action descriptions that are brief and minimalistic, often failing to convey the full range of complexities involved in human movement. Such sparse descriptions limit the ability of models to generate diverse and nuanced motion sequences. Action-GPT leverages LLMs to create richer, more detailed descriptions of actions, capturing finer aspects of movement. By doing so, it improves the alignment between text and motion spaces, enabling models to generate more precise and contextually accurate motion sequences. This framework is designed to work with both stochastic models (e.g., VAE-based) and deterministic models offering flexibility across different types of motion generation architectures. Experimental results demonstrate that Action-GPT not only enhances the quality of synthesized motions—both in terms of realism and diversity—but also excels in zero-shot generation, effectively handling previously unseen text descriptions. Second, we introduce MoRAG, a sophisticated retrieval-augmented generation strategy tailored to enhance the performance of motion diffusion models. MoRAG adopts a multi-part fusion retrieval mechanism that allows for improved generalization of motion retrieval across a wide range of language inputs, addressing the limitations of current retrieval methods that struggle with unseen or atypical descriptions. By incorporating low-level, part-specific motion details into the retrieval process, MoRAG constructs more accurate and varied motion sequences. The retrieval process is refined by prompting LLMs to handle issues like spelling errors, rephrasing, and ambiguous language, ensuring that the retrieved motions are contextually relevant and diverse. These augmented motion samples are then used as additional knowledge within the motion generation pipeline, enhancing the system’s ability to generate complex and realistic motions from diverse textual inputs. This retrieval-augmented approach increases both the robustness and generalization capacity of the motion generation models, making them more adaptable to complex and unseen scenarios. Together, these contributions represent a substantial advancement in text-conditioned human motion generation. By refining both the action description process and the motion retrieval strategy, this work enhances the ability of models to generate diverse, realistic motions from natural language inputs, particularly in zero-shot settings and when handling detailed, complex descriptions.