Faculty - Dashboard

@inproceedings{bib_Hier_2025, AUTHOR = {Kumar, Pawan and Gupta, Aditi }, TITLE = {Hierarchical Sparse Plus Low Rank Compression of LLM}, BOOKTITLE = {Conference on Data Science}. YEAR = {2025}}

Modern large language models (LLMs) place extraordinary pres- sure on memory and compute budgets, making principled compres- sion indispensable for both deployment and continued training. We present Hierarchical Sparse Plus Low-Rank (HSS) compres- sion, a two–stage scheme that (i) removes the largest-magnitude weights into a sparse matrix 𝑆 and (ii) applies a recursive Hierar- chically Sparse Separable (HSS) low-rank factorisation to the dense residual matrix. A recursive rank-reducing strategy and a reverse Cuthill–Mckee (RCM) permutation are introduced to align high weights towards the diagonal with the block-diagonal hierarchy, maximising off-diagonal compressibility (because they are touched only once). HSS is hardware-friendly: its matrix–vector multiply reduces to one sparse and a sequence of thin-matrix multiplications, and can be trained end-to-end with standard optimisers. Experiments on LLaMA-7B show that targeting only the self- attention projections (1.6B parameters of 𝑄, 𝐾, and 𝑉 matrices out of a total 7B parameters) suffices to yield large memory savings while retaining comparable state-of-the-art perplexity scores on test samples of the WikiText dataset. For example, with a 30% sparsity budget and an outer rank of 512, sHSS-RCM achieves a perplexity of 1.64, outperforming dense baselines and classical sparse-plus-SVD variants, while also achieving significant memory savings.

@inproceedings{bib_A_Fa_2025, AUTHOR = {Dhara, Venkata Siddharth and Kumar, Pawan }, TITLE = {A Fast and Efficient Modern BERT based Text-Conditioned Diffusion Model for Medical Image Segmentation}, BOOKTITLE = {International Conference on Computer vision and Image Processing}. YEAR = {2025}}

In recent times, denoising diffusion probabilistic models (DPMs) have proven to show significant success in medical image generation and denoising, while also serving as powerful representation learners for downstream tasks such as segmentation. However, their effectiveness in segmentation is limited by the need for detailed pixel-wise annotations, which are expensive, time-consuming, and require expert knowledge—a significant bottleneck in real-world clinical applications. In order to mitigate this limitation of label-efficiency, we propose a fast and efficient model named FastTextDiff, a diffusion-based segmentation model that integrates medical text annotations to enhance semantic representations. Our approach leverages ModernBERT, a transformer-based language model capable of processing long medical text sequences, to establish a strong connection between textual annotations and semantic meaning in medical imaging. ModernBERT can efficiently encode clinical knowledge for directing segmentation tasks since it has been trained on both MIMIC-III and MIMIC-IV. Label-efficient segmentation with enhanced performance is made possible by cross-modal attention processes, which enable smooth interaction between visual and textual modalities. This study validates ModernBERT as a quick and scalable substitute for Clinical BioBERT in diffusion-based segmentation pipelines and demonstrates the promise of multi-modal techniques for medical image analysis. By replacing Clinical BioBERT with ModernBERT, our model benefits from Flash Attention 2 for memory-efficient training, an alternating attention mechanism for computational efficiency, and a training corpus of 2 trillion tokens, significantly improving text-based medical image segmentation. Our experiments demonstrate that FastTextDiff achieves better segmentation performance and faster training compared to traditional diffusion-based models. We release this trained model on Hugging Face and GitHub.

@inproceedings{bib_A_la_2024, AUTHOR = {Zahra, Dr. Andleeb and Sinha, Swarnim and Modak, Alimpan and Siddiqui, Imran and Syed, Azeemuddin and Bhimalapuram, Prabhakar and Sau, Tapan Kumar and Kumar, Pawan and Abbas, Zia }, TITLE = {A label-free sensing of creatinine using radio frequency-driven lab-on-chip (loc) system}, BOOKTITLE = {Engineering Research Express}. YEAR = {2024}}

Skip to content IOP Science home Accessibility Help Search Journals Books Publishing Support Login Engineering Research Express Inclusive Publishing Trusted Science, find out more. Purpose-led Publishing, find out more. ACCEPTED MANUSCRIPT A label-free sensing of creatinine using radio frequency-driven lab-on-chip (loc) system Andleeb Zahra1, Swarnim Sinha2, Alimpan Modak3, Imran Siddiqui4, Azeemuddin Syed5, Prabhakar Bhimalapuram6, Tapan K. Sau7, Pawan Kumar8 and Zia Abbas9 Accepted Manuscript online 2 August 2024 • © 2024 IOP Publishing Ltd What is an Accepted Manuscript? DOI 10.1088/2631-8695/ad6ad5 DownloadAccepted Manuscript PDF Download PDF Article metrics 6 Total downloads Submit Submit to this Journal Permissions Get permission to re-use this article Share this article Article and author information Abstract This paper presents a promising avenue of Radio Frequency (RF) biosensors for sensitive and real-time monitoring of creatinine detection. Knowing creatinine levels in the human body is related to its possible association with renal, muscular, and thyroid dysfunction. The detection was performed using an Inter-Digitated Capacitor (IDC) made of copper (Cu) metal over an FR4 substrate. To demonstrate our methodology, we have chosen Phosphate Buffer (PB) as our solvent for making the creatinine solutions of different concentrations. Moreover, Assayed Chemistry Control (ACC), a reference control consisting of human serum-based solutions has been mixed with the different concentrations of creatinine in a ratio of 1:9 to spike the creatinine value in the ACC solution. The sensor has been designed using a High-Frequency Structure Simulator (HFSS) tool with an operating frequency of 2.53 GHz. Then the design is fabricated over the FR4 printed circuit board (PCB) and tested using a Vector Network Analyzer (VNA). However, the sensitive area of the IDC is introduced to grade 4 Whatman filter paper for the Sample Under Test (SUT) handling unit. The main advantage of using Whatman filter paper is that the uniform spreading of liquid reduces experimental error, and less volume is required for testing the sample. The principal idea implemented in the biosensor design is to track the shift in the operating frequency in the presence of different concentrations of creatinine mix in ACC solution with Phosphate Buffer (PB) solution as a reference.

@inproceedings{bib_Qual_2024, AUTHOR = {Srivastava, Prasha and Kumar, Pawan and Abbas, Zia }, TITLE = {Qualitative Data Augmentation for Performance Prediction in VLSI Circuits}, BOOKTITLE = {IEEE Transactions on Very Large Scale Integration (VLSI) Systems}. YEAR = {2024}}

Various studies have shown the advantages of using Machine Learning (ML) techniques for analog and digital IC design automation and optimization. Data scarcity is still an issue for electronic designs, while training highly accurate ML models. This work proposes generating and evaluating artificial data using generative adversarial networks (GANs) for circuit data to aid and improve the accuracy of ML models trained with a small training data set. The training data is obtained by various simulations in the Cadence Virtuoso, HSPICE, and Microcap design environment with TSMC 180 nm and 22 nm CMOS technology nodes. The artificial data is generated and tested for an appropriate set of analog circuits and digital cells. The experimental results show that the proposed artificial data generation significantly improves ML models and reduces the percentage error by more than 50% of the original percentage error, which were previously trained with insufficient data. Furthermore, this research aims to contribute to the extensive application of AI/ML in the field of VLSI design and technology by relieving the training data availability-related challenges.

@inproceedings{bib_Enha_2024, AUTHOR = {Srivastava, Prasha and Kumar, Pawan and Abbas, Zia }, TITLE = {Enhancing ML Model Accuracy for Digital VLSI Circuits Using Diffusion models: a Study on Synthetic Data Generation}, BOOKTITLE = {IEEE International Symposium on Circuits and Systems}. YEAR = {2024}}

@inproceedings{bib_Enha_2023, AUTHOR = {Srivastava, Prasha and Kumar, Pawan and Abbas, Zia }, TITLE = {Enhancing ML model accuracy for Digital VLSI circuits using diffusion models: A study on synthetic data generation}, BOOKTITLE = {Neural Information Processing Systems Workshops}. YEAR = {2023}}

Generative AI has seen remarkable growth over the past few years, with diffusion models being state-of-the-art for image generation. This study investigates the use of diffusion models in artificial data generation for electronic circuits to enhance the accuracy of subsequent machine learning models in tasks such as performance assessment, design, and testing when training data is usually known to be very limited. We utilize simulations in the HSPICE design environment with 22nm CMOS technology nodes to obtain representative real training data for our proposed diffusion model. Our results demonstrate the close resemblance of synthetic data using diffusion models to real data. We validate the quality of generated data and demonstrate that data augmentation is certainly effective in the predictive analysis of VLSI design for digital circuits.

@inproceedings{bib_Enha_2023, AUTHOR = {Srivastava, Prasha and Kumar, Pawan and Abbas, Zia }, TITLE = {Enhancing ML model accuracy for Digital VLSI circuits using diffusion models: A study on synthetic data generation}, BOOKTITLE = {Technical Report}. YEAR = {2023}}

Generative AI has seen remarkable growth over the past few years, with diffusion models being state-of-the-art for image generation. This study investigates the use of diffusion models in generating artificial data generation for electronic circuits for enhancing the accuracy of subsequent machine learning models in tasks such as performance assessment, design, and testing when training data is usually known to be very limited. We utilize simulations in the HSPICE design environment with 22nm CMOS technology nodes to obtain representative real training data for our proposed diffusion model. Our results demonstrate the close resemblance of synthetic data using diffusion model to real data. We validate the quality of generated data, and demonstrate that data augmentation certainly effective in predictive analysis of VLSI design for digital circuits.

@inproceedings{bib_Effe_2023, AUTHOR = {Mehta, Kinal and Mahajan, Anuj and Kumar, Pawan }, TITLE = {Effects of Spectral Normalization in Multi-agent Reinforcement Learning}, BOOKTITLE = {International Joint Conference on Neural Networks}. YEAR = {2023}}

A reliable critic is central to on-policy actor-critic learning. But it becomes challenging to learn a reliable critic in a multi-agent sparse reward scenario due to two factors: 1) The joint action space grows exponentially with the number of agents 2) This, combined with the reward sparseness and environment noise, leads to large sample requirements for accurate learning. We show that regularising the critic with spectral normalization (SN) enables it to learn more robustly, even in multi-agent on-policy sparse reward scenarios. Our experiments show that the regularised critic is quickly able to learn from the sparse rewarding experience in the complex SMAC and RWARE domains. These findings highlight the importance of regularisation in the critic for stable learning.

@inproceedings{bib_marl_2023, AUTHOR = {Mehta, Kinal and Mahajan, Anuj and Kumar, Pawan }, TITLE = {marl-jax: Multi-agent Reinforcement Leaning framework for Social Generalization}, BOOKTITLE = {The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Da}. YEAR = {2023}}

Recent advances in Reinforcement Learning (RL) have led to many exciting applications. These advancements have been driven by improvements in both algorithms and engineering, which have resulted in faster training of RL agents. We present marl-jax, a multi-agent reinforcement learning software package for training and evaluating social generalization of the agents. The package is designed for training a population of agents in multi-agent environments and evaluating their ability to generalize to diverse background agents. It is built on top of DeepMind's JAX ecosystem~\cite{deepmind2020jax} and leverages the RL ecosystem developed by DeepMind. Our framework marl-jax is capable of working in cooperative and competitive, simultaneous-acting environments with multiple agents. The package offers an intuitive and user-friendly command-line interface for training a population and evaluating its generalization capabilities. In conclusion, marl-jax provides a valuable resource for researchers interested in exploring social generalization in the context of MARL. The open-source code for marl-jax is available at: \href{this https URL}{this https URL}

@inproceedings{bib_Alph_2023, AUTHOR = {Dasgupta, Arpan and Kumar, Pawan }, TITLE = {Alpha Elimination: Using Deep ReinforcementLearning to Reduce Fill-In during Sparse Matrix Decomposition}, BOOKTITLE = {The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Da}. YEAR = {2023}}

A large number of computational and scientific methods commonly require decomposing a sparse matrix into triangular factors as LU decomposition. A common problem faced during this decomposition is that even though the given matrix may be very sparse, the decompo- sition may lead to a denser triangular factors due to fill-in. A significant fill-in may lead to prohibitively larger computational costs and memory requirement during decomposition as well as during the solve phase. To this end, several heuristic sparse matrix reordering methods have been proposed to reduce fill-in before the decomposition. However, finding an optimal reordering algorithm that leads to minimal fill-in during such de- composition is known to be a NP-hard problem. A reinforcement learning based approach is proposed for this problem. The sparse matrix reorder- ing problem is formulated as a single player game. More specifically, Monte-Carlo tree search in combination with neural network is used as a decision making algorithm to search for the best move in our game. The proposed method, Alpha Elimination is found to produce significantly lesser non-zeros in the LU decomposition as compared to existing state- of-the-art heuristic algorithms with little to no increase in overall running time of the algorithm. The code for the project is publicly available 1. Keywords: Reinforcement Learning · Sparse Matrices · Deep Learning · LU · MCTS

@inproceedings{bib_Ligh_2023, AUTHOR = {Mishra, Choudhury Istasis and Dasgupta, Arpan and Jawanpuria, Pratik and Mishra, Bamdev and Kumar, Pawan }, TITLE = {Light-weight Deep Extreme Multilabel Classification}, BOOKTITLE = {International Joint Conference on Neural Networks}. YEAR = {2023}}

Extreme multi-label (XML) classification refers to the task of supervised multi-label learning that involves a large number of labels. Hence, scalability of the classifier with increasing label dimension is an important consideration. In this paper, we develop a method called LightDXML which modifies the recently developed deep learning based XML framework by using label embeddings instead of feature embedding for negative sampling and iterating cyclically through three major phases: (1) proxy training of label embeddings (2) shortlisting of labels for negative sampling and (3) final classifier training using the negative samples. Consequently, LightDXML also removes the requirement of a re-ranker module, thereby, leading to further savings on time and memory requirements. The proposed method achieves the best of both worlds: while the training time, model size and prediction times are on par or better compared to the tree-based methods, it attains much better prediction accuracy that is on par with the deep learning based methods. Moreover, the proposed approach achieves the best tail-label prediction accuracy over most state-of-the-art XML methods on some of the large datasets1 . Code: https://github.com/misterpawan/ LightDXML

@inproceedings{bib_Angl_2023, AUTHOR = {Ashok, Mishra Neel and Kumar, Pawan }, TITLE = {Angle based dynamic learning rate for gradient descent}, BOOKTITLE = {International Joint Conference on Neural Networks}. YEAR = {2023}}

In our work, we propose a novel yet simple approach to obtain an adaptive learning rate for gradient-based descent methods on classification tasks. Instead of the traditional approach of selecting adaptive learning rates via the decayed expectation of gradient-based terms, we use the angle between the current gradient and the new gradient: this new gradient is computed from the direction orthogonal to the current gradient, which further helps us in determining a better adaptive learning rate based on angle history, thereby, leading to relatively better accuracy compared to the existing state-of-the-art optimizers. On a wide variety of benchmark datasets with prominent image classification architectures such as ResNet, DenseNet, EfficientNet, and VGG, we find that our method leads to the highest accuracy in most of the datasets. Moreover, we prove that our method is convergent. Index Terms—Adam, Image Classification, Optimization, Angle, Gradient Descent, Neural Networks, ResNet

@inproceedings{bib_Adap_2023, AUTHOR = {Kumar, Danisetty Sachin and Reddy, Mylaram Santhosh and Kumar, Pawan }, TITLE = {Adaptive Consensus Optimization Method for GANs}, BOOKTITLE = {International Joint Conference on Neural Networks}. YEAR = {2023}}

We propose a second order gradient based method with ADAM and RMSprop for the training of generative adversarial networks. The proposed method is fastest to obtain similar accuracy when compared to prominent second order methods. Unlike state-of-the-art recent methods, it does not require solving a linear system, or it does not require additional mixed second derivative terms. We derive the fixed point iteration corresponding to proposed method, and show that the proposed method is convergent. The proposed method produces better or comparable inception scores, and comparable quality of images compared to other recently proposed state-of-the-art second order methods. Compared to first order methods such as ADAM, it produces significantly better inception scores. The proposed method is compared and validated on popular datasets such as FFHQ, LSUN, CIFAR10, MNIST, and Fashion MNIST for image generation tasks1 . Codes: https://github.com/misterpawan/acom

@inproceedings{bib_Nonn_2023, AUTHOR = {Kumar, Pawan and Jayadev, Naram and Sinha, Tanmay Kumar }, TITLE = {Nonnegative Low-Rank Tensor Completion via Dual Formulation with Applications to Image and Video Completion}, BOOKTITLE = {}. YEAR = {2023}}

Recent approaches to the tensor completion problem have often overlooked the nonnegative structure of the data. We consider the problem of learning a nonnegative lowrank tensor, and using duality theory, we propose a novel factorization of such tensors. The factorization decouples the nonnegative constraints from the low-rank constraints. The resulting problem is an optimization problem on manifolds, and we propose a variant of Riemannian conjugate gradients to solve it. We test the proposed algorithm across various tasks such as colour image inpainting, video completion, and hyperspectral image completion. Experimental results show that the proposed method outperforms many state-of-the-art tensor completion algorithms.

@inproceedings{bib_Qual_2023, AUTHOR = {Srivastava, Prasha and Kumar, Pawan and Abbas, Zia }, TITLE = {Qualitative Data Augmentation for Performance Prediction in VLSI Circuits}, BOOKTITLE = {IEEE International Symposium on Circuits and Systems}. YEAR = {2023}}

Various studies have shown the advantages of using Machine Learning (ML) techniques for analog and digital IC design automation and optimization. Data scarcity is still an issue for electronic designs, while training highly accurate ML models. This work proposes generating and evaluating artificial data using generative adversarial networks (GANs) for circuit data to aid and improve the accuracy of ML models trained with a small training data set. The training data is obtained by various simulations in the Cadence Virtuoso, HSPICE, and Microcap design environment with TSMC 180nm and 22nm CMOS technology nodes. The artificial data is generated and tested for an appropriate set of analog and digital circuits. The experimental results show that the proposed artificial data generation significantly improves ML models and reduces the percentage error by more than 50% of the original percentage error, which were previously trained with insufficient data. Furthermore, this research aims to contribute to the extensive application of AI/ML in the field of VLSI design and technology by relieving the training data availability-related challenges.

@inproceedings{bib_Revi_2023, AUTHOR = {Dasgupta, Arpan and Katyan, Siddhant and Das, Shrutimoy and Kumar, Pawan }, TITLE = {Review of Extreme Multilabel Classification}, BOOKTITLE = {Technical Report}. YEAR = {2023}}

Extreme multilabel classification or XML, in short, has emerged as a new subtopic of interest in machine learning. Compared to traditional multilabel classification, here the number of labels is extremely large, hence the name extreme multilabel classification. Using classical one versus all classification wont scale in this case due to large number of labels, same is true for any other classifiers. Embedding of labels as well as features into smaller label space is an essential first step. Moreover, other issues include existance of head and tail labels, where tail labels are labels which exist in relatively smaller number of given samples. The existence of tail labels creates issues during embedding. This area has invited application of wide range of approaches ranging from bit compression motivated from compressed sensing, tree based embeddings, deep learning based latent space embedding including using attention weights, linear algebra based embeddings such as SVD, clustering, hashing, to name a few. The community has come up with a useful set of metrics to identify the correctly the prediction for head or tail labels. Keywords: extreme classification, head and tail labels, compressed sensing, deep learning, attention

@inproceedings{bib_Gene_2023, AUTHOR = {Jayadev, Naram and Sinha, Tanmay Kumar and Kumar, Pawan }, TITLE = {Generalized Structured Low-Rank Tensor Learning}, BOOKTITLE = {Joint International Conference on Data Science & Management of Data}. YEAR = {2023}}

We consider the problem of learning tensors from partial observa- tions with structural constraints, under the low-rank assumption. To this end, we propose a general convex low-rank regularizer, pa- rameterized by linear maps, that extends the existing regularizers. Different choices of the linear maps lead to different convex low- rank regularizers. The resulting problem is called the generalized structured low-rank tensor learning problem. To solve this problem, we use duality theory to reformulate it into simpler sub-problems. The dual problem contains a rich geometric structure, which we exploit to develop first-order and second-order Riemannian opti- mization algorithms. The associated duality gap is derived, and it is shown to be zero. Moreover, we experimentally verify the cor- rectness of our algorithm on several special cases of the proposed general framework.

@inproceedings{bib_Riem_2023, AUTHOR = {Han, Andi and Mishra, Bamdev and Jawanpuria, Pratik and Kumar, Pawan and Gao, Junbin }, TITLE = {Riemannian Hamiltonian methods for min-max optimization on manifolds}, BOOKTITLE = {SIAM Journal of Optimization}. YEAR = {2023}}

In this paper, we study the min-max optimization problems on Riemannian manifolds. We introduce a Riemannian Hamiltonian function, minimization of which serves as a proxy for solving the original min-max problems. Under the Riemannian Polyak–Łojasiewicz (PL) condition on the Hamiltonian function, its minimizer corresponds to the desired min-max saddle point. We also provide cases where this condition is satisfied. To minimize the Hamiltonian function, we propose Riemannian Hamiltonian methods (RHM) and present their convergence analysis. We extend RHM to include consensus regularization and to the stochastic setting. We illustrate the efficacy of the proposed RHM in applications such as subspace robust Wasserstein distance, robust training of neural networks, and generative adversarial networks.

@inproceedings{bib_Hybr_2022, AUTHOR = {Mandlecha, Pratik and Chatakonda, Snehith Kumar and Neeraj, Kollepara and Kumar, Pawan }, TITLE = {Hybrid Tokenization and Datasets for Solving Mathematics and Science Problems Using Transformers.}, BOOKTITLE = {SIAM International Conference on Data Mining}. YEAR = {2022}}

Transformers, which were introduced for solving the task of machine translation, have expanded their utility in multiple domains. A recent application of transformers is in solving elementary mathematics problems. In this paper, we use a hybrid tokenization technique for encoding the mathematics and science problems and answers, which is used to train the transformer. We compare the performance of our tokenization with that of the char-to-char tokenzation in solving various types of mathematics and science problems. We discuss the accuracy, memory usage, and time to train the model with proposed tokenization. The proposed tokenization shows higher accuracy for some problems, and requires lesser memory compared to char-to-char tokenization. We propose an extended dataset of science and mathematics problems that consists of billions of samples in questionanswer format in raw text. Code and Dataset: https: //github.com/misterpawan/scimat2

@inproceedings{bib_A_Ri_2022, AUTHOR = {Jayadev, Naram and Sinha, Tanmay Kumar and Kumar, Pawan }, TITLE = {A Riemannian Approach to Extreme Classification Problems}, BOOKTITLE = {Joint International Conference on Data Science & Management of Data}. YEAR = {2022}}

We propose a novel Riemannian method called “RXML" for solving the Extreme multi-label classification problem that exploits the geometric structure of the sparse low-dimensional local embedding models. A constrained optimization problem is formulated as an optimization problem on a matrix manifold and solved using a Riemannian optimization method. A proof of convergence for the proposed Riemannian optimization method is stated. The proposed approach is tested on several real-world large scale multi-label datasets, and its usefulness is demonstrated through numerical experiments. Experimental results show that RXML improves the trade-off between train time and accuracy. At the similar level of accuracy, the train time of RXML was 1.5 to 4 times faster than that of AnnexML and EXMLDS-4, which are the state-of-the-art embedding- based methods.

@inproceedings{bib_SCIM_2021, AUTHOR = {Kumar, Chatakonda Snehith and Kollepara, Neeraj and Kumar, Pawan }, TITLE = {SCIMAT: Dataset of Problems in Science and Mathematics}, BOOKTITLE = {International Conference on Big Data Analytics}. YEAR = {2021}}

Datasets play an important role in driving innovation in algorithms and architectures for supervised deep learning tasks. Numerous datasets exist for images, language translation, etc. One of the interesting challenge problems for deep learning is to solve high school problems in mathematics and sciences. To this end, a comprehensive set of dataset containing hundreds of millions of samples, and the generation modules is required that can propel research for these problems. In this paper, a large set of datasets covering mathematics and science problems is proposed, and the dataset generation codes are proposed. Test results on the proposed datasets for character-to-character transformer architecture show promising results with test accuracy above 95%, however, for some datasets it shows test accuracy of below 30%

@inproceedings{bib_A_Fa_2021, AUTHOR = {AGGARWAL, ABHINAV and KAKKAR, SHIVAM and Kumar, Pawan }, TITLE = {A Fast Parameter-Free Preconditioner for Structured Grid Problems}, BOOKTITLE = {International Conference for High Performance Computing, Networking, Storage and Analysis}. YEAR = {2021}}

A fast, robust, parallel, and parameter free version of a frequency filtering preconditioner is proposed for linear systems corresponding to diffusion equation on a structured grid. Proposed solver is faster than the state-of-the-art solvers

@inproceedings{bib_SCIM_2021, AUTHOR = {Kumar, Chatakonda Snehith and Neeraj, Kollepara and Kumar, Pawan }, TITLE = {SCIMAT: Dataset of Problems in Science and Mathematics}, BOOKTITLE = {International Conference on Big Data Analytics}. YEAR = {2021}}

Datasets play an important role in driving innovation in algorithms and architectures for supervised deep learning tasks. Numerous datasets exist for images, language translation, etc. One of the interesting challenge problems for deep learning is to solve high school problems in mathematics and sciences. To this end, a comprehensive set of dataset containing hundreds of millions of samples, and the generation modules is required that can propel research for these problems. In this paper, a large set of datasets covering mathematics and science problems is proposed, and the dataset generation codes are proposed. Test results on the proposed datasets for character-to-character transformer architecture show promising results with test accuracy above 95%, however, for some datasets it shows test accuracy of below 30%. Dataset will be available at: www.github.com/misterpawan/scimat2.

@inproceedings{bib_Effi_2021, AUTHOR = {Rampalli, Sahithi and Sehgal, Natasha and Bindlish, Ishita and Tyag, Tanya and Kumar, Pawan }, TITLE = {Efficient FPGA Implementation of Conjugate Gradient Methods for Laplacian System using HLS}, BOOKTITLE = {ACM/SIGDA International Symposium on Field-Programmable Gate Arrays}. YEAR = {2021}}

In this paper, we study FPGA based pipelined and superscalar design of two variants of conjugate gradient methods for solving Laplacian equation on a discrete grid; the first version corresponds to the original conjugate gradient algorithm, and the second version corresponds to a slightly modified version of the same. In conjugate gradient method to solve partial differential equations, matrix vector operations are required in each iteration; these operations can be implemented as 5 point stencil operations on the grid without explicitely constructing the matrix. We show that a pipelined and superscalar design using high level synthesis written in C language leads to a significant reduction in latencies for both methods. When comparing these two, we show that the later has roughly two times lower latency than the former given the same degree of superscalarity. These reductions in latencies for the newer variant of CG is due to parallel implementations of stencil operation on subdomains of the grid, and dut to overlap of these stencil operations with dot product operations. In a superscalar design, domain needs to be partitioned, and boundary data needs to be copied, which requires padding. In 1D partition, the padding latency increases as the number of partitions increase. For a streaming data flow model, we propose a novel traversal of the grid for 2D domain decomposition that leads to 2 times reduction in latency cost involved with padding compared to 1D partitions. Our implementation is roughly 10 times faster than software implementation for linear system of dimension 10000 × 10000. Index Terms—FPGA, High Level Synthesis, Conjugate Gradient, Laplace System, Pipelining, Superscalarity.

@inproceedings{bib_DXML_2021, AUTHOR = {Kumar, Pawan }, TITLE = {DXML: Distributed Extreme Multilabel Classification}, BOOKTITLE = {International Conference on Big Data Analytics}. YEAR = {2021}}

As a big data application, extreme multilabel classification has emerged as an important research topic with applications in ranking and recommendation of products and items. A scalable hybrid distributed and shared memory implementation of extreme classification for large scale ranking and recommendation is proposed. In particular, the implementation is a mix of message passing using MPI across nodes and using multithreading on the nodes using OpenMP. The expression for communication latency and communication volume is derived. Parallelism using work-span model is derived for shared memory architecture. This throws light on the expected scalability of similar extreme classification methods. Experiments show that the implementation is relatively faster to train and test on some large datasets. In some cases, model size is relatively small. Code: https://github.com/misterpawan/DXML

@inproceedings{bib_A_De_2021, AUTHOR = {Das, Shrutimoy and Katyan, Siddhant and Kumar, Pawan }, TITLE = {A Deflation based Fast and Robust Preconditioner for Bundle Adjustment}, BOOKTITLE = {Winter Conference on Applications of Computer Vision}. YEAR = {2021}}

The bundle adjustment (BA) problem is formulated as a non linear least squares problem which, requires the solution of a linear system. For solving this system, we present the design and implementation of a fast preconditioned solver. The proposed preconditioner is based on the deflation of the largest eigenvalues of the Hessian. We also derive an estimate of the condition number of the preconditioned system. Numerical experiments on problems from the BAL dataset [3] suggest that our solver is the fastest, sometimes, by a factor of five, when compared to the current state-of-the-art solvers for bundle adjustment.

@inproceedings{bib_SCIM_2021, AUTHOR = {Neeraj, Kollepara and Kumar, Chatakonda Snehith and Kumar, Pawan }, TITLE = {SCIMAT: Science and Mathematics Dataset∗}, BOOKTITLE = {Neural Information Processing Systems Workshops}. YEAR = {2021}}

In this work, we announce a comprehensive well curated and opensource dataset with millions of samples for pre-college and college level problems in mathematics and science. A preliminary set of results using transformer architecture with character to character encoding is shown. The dataset identifies some challenging problems, and invites research on better architecture search for these problems.

@inproceedings{bib_Stru_2021, AUTHOR = {Jayadev, Naram and Sinha, Tanmay Kumar and Kumar, Pawan }, TITLE = {Structured Low-Rank Tensor Learning}, BOOKTITLE = {workshop on Optimization for Machine Learning}. YEAR = {2021}}

We consider the problem of learning low-rank tensors from partial observations with structural constraints, and propose a novel factorization of such tensors, which leads to a simpler optimization problem. The resulting problem is an optimization problem on manifolds. We develop first-order and second-order Riemannian optimization algorithms to solve it. The duality gap for the resulting problem is derived, and we experimentally verify the correctness of the proposed algorithm. We demonstrate the algorithm on nonnegative constraints and Hankel constraints

@inproceedings{bib_Doma_2020, AUTHOR = {Kumar, Pawan and Katyan, Siddhant and Das, Shrutimoy }, TITLE = {Domain Decomposition Based Preconditioned Solver for Bundle Adjustment}, BOOKTITLE = {National Conference on Computer Vision, Pattern Recognition, Image Processing, and Graphics}. YEAR = {2020}}

We propose Domain Decomposed Bundle Adjustment (DDBA), a robust and efficient solver for the bundle adjustment problem. Bundle adjustment (BA) is generally formulated as a non-linear least squares problem and is solved by some variant of the Levenberg-Marquardt (LM) algorithm. Each iteration of the LM algorithm requires solving a system of normal equations, which becomes computationally expensive with the increase in problem size. The coefficient matrix of this system has a sparse structure which can be exploited for simplifying the computations in this step. We propose a technique for approximating the Schur complement of the matrix, and use this approximation to construct a preconditioner, that can be used with the Generalized Minimal Residual (GMRES) algorithm for solving the system of equations. Our experiments on the BAL dataset show that the proposed method for solving the system is faster than GMRES solve preconditioned with block Jacobi and more memory efficient than direct solve.

@inproceedings{bib_Two-_2020, AUTHOR = {Katyan, Siddhant and Das, Shrutimoy and Kumar, Pawan }, TITLE = {Two-Grid Preconditioned Solver for Bundle Adjustment}, BOOKTITLE = {Winter Conference on Applications of Computer Vision}. YEAR = {2020}}

We present the design and implementation of Two-Grid Preconditioned Bundle Adjustment (TPBA), a robust and efficient technique for solving the non-linear least squares problem that arises in bundle adjustment. Bundle adjustment (BA) methods for multi-view reconstruction formulate the BA problem as a non-linear least squares problem which is solved by some variant of the traditional LevenbergMarquardt (LM) algorithm. Most of the computation in LM goes into repeatedly solving the normal equations that arise as a result of linearizing the objective function. To solve these system of equations we use the Generalized Minimal Residual (GMRES) method, which is preconditioned using a deflated algebraic two-grid method. To the best of our knowledge this is the first time that a deflated algebraic twogrid preconditioner has been used along with GMRES, for solving a problem in the computer vision domain. We show that the proposed method is several times faster than the direct method and block Jacobi preconditioned GMRES.