Faculty - Dashboard

@inproceedings{bib_Mono_2026, AUTHOR = {Sharma, Basant and Jadhav, Prajyot and Paul, Pranjal and Krishna, K Madhava and Singh, Arun Kumar }, TITLE = {MonoMPC: Monocular Vision Based Navigation With Learned Collision Model and Risk-Aware Model Predictive Control}, BOOKTITLE = {IEEE Robotics and Automation Letters}. YEAR = {2026}}

Navigating unknown environments with a single RGB camera is challenging, as the lack of depth information prevents reliable collision-checking. While some methods use estimated depth to build collision maps, we found that depth estimates from vision foundation models are too noisy for zero-shot navigation in cluttered environments. We propose an alternative approach: instead of using noisy estimated depth for direct collision-checking, we use it as a rich context input to a learned collision model. This model predicts the distribution of minimum obstacle clearance that the robot can expect for a given control sequence. At inference, these predictions inform a risk-aware MPC planner that minimizes estimated collision risk. We proposed a joint learning pipeline that co-trains the collision model and risk metric using both safe and unsafe trajectories. Crucially, our joint-training ensures well calibrated uncertainty in our collision model that improves navigation in highly cluttered environments. Consequently, real-world experiments show reductions in collision-rate and improvements in goal reaching and speed over several strong baselines.

@inproceedings{bib_RE-C_2025, AUTHOR = {Suraj, Bonagiri V Sai Gopala and Roy, Spandan and Krishna, K Madhava }, TITLE = {RE-CONFIGURABLE UNMANNED AERIAL VEHICLE}, BOOKTITLE = {United States Patent}. YEAR = {2025}}

An embodiment herein provides a re-configurable unmanned aerial vehicle that re-configures its shape based on the shape, size, weight of a payload, and efficiently performs payload delivery in real-time. The re-configurable unmanned aerial vehicle includes one or more rotor units placed at corners and is connected by one or more scissor units. The re-configurable unmanned aerial vehicle approaches the payload in a first location, and analyses the position and dimension of the payload with a camera, that enables the one or more scissor units to adjust its length by at least one elongation or compression following size and shape of the payload and fit the payload within the re- configurable unmanned aerial vehicle. The re-configurable unmanned aerial vehicle takes off carrying the payload from the first location and lands at a second location.

@inproceedings{bib_SegM_2025, AUTHOR = {Sreekanth, Jayanti Rohit and Agrawal, Swayam Manoj and Garg, Vansh and TOURANI, SIDDHARTH and Khan, Muhammad Haris and Garg, Sourav and Krishna, K Madhava }, TITLE = {SegMASt3R: Geometry Grounded Segment Matching}, BOOKTITLE = {Neural Information Processing Systems}. YEAR = {2025}}

Segment matching is an important intermediate task in computer vision that establishes correspondences between semantically or geometrically coherent regions across images. Unlike keypoint matching, which focuses on localized features, segment matching captures structured regions, offering greater robustness to occlusions, lighting variations, and viewpoint changes. In this paper, we leverage the spatial understanding of 3D foundation models to tackle wide-baseline segment matching, a challenging setting involving extreme viewpoint shifts. We propose an architecture that uses the inductive bias of these 3D foundation models to match segments across image pairs with up to 180 degree view-point change rotation. Extensive experiments show that our approach outperforms state-of-the-art methods, including the SAM2 video propagator and local feature matching methods, by up to 30% on the AUPRC metric, on ScanNet++ and Replica datasets. We further demonstrate benefits of the proposed model on relevant downstream tasks, including 3D instance mapping and object-relative navigation. Project Page: https://segmast3r.github.io/

@inproceedings{bib_Diff_2025, AUTHOR = {Gupta, Keshav and Stanley, Tejas Stephen and Paul, Pranjal and Singh, Arun Kumar and Krishna, K Madhava }, TITLE = {Diffusion-FS: Multimodal Free-Space Prediction via Diffusion for Autonomous Driving}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2025}}

Drivable Freespace prediction is a fundamental and crucial problem in autonomous driving. Recent works have addressed the problem by representing the entire non-obstacle road regions as the freespace. In contrast our aim is to estimate the driving corridors that are a navigable subset of the entire road region. Unfortunately, existing corridor estimation methods directly assume a BEV centric representation, which is hard to obtain. In contrast, we frame drivable freespace corridor prediction as a pure image perception task, using only monocular camera input. However such a formulation poses several challenges as one doesn’t have the corresponding data for such freespace corridor segments in the image. Consequently, we develop a novel self-supervised approach for freespace sample generation by leveraging future ego trajectories and front-view camera images, making the process of visual corridor estimation dependent on the ego trajectory. We then employ a diffusion process to model the distribution of such segments in the image. However, the existing binary mask based representation for a segment poses many limitations. Therefore, we introduce ContourDiff, a specialized diffusion-based architecture that denoises over contour points rather than relying on binary mask representations, enabling structured and interpretable freespace predictions. We evaluate our approach qualitatively and quantitatively on both nuScenes and CARLA, demonstrating its effectiveness in accurately predicting safe multimodal navigable corridors in the image.

@inproceedings{bib_Spar_2025, AUTHOR = {Paul, Pranjal and Bhat, Vineeth and Salian, Tejas and Omama, Mohd and Murthy, J. Krishna and Arulselvan, Naveen and Krishna, K Madhava }, TITLE = {SparseLoc: Sparse Open-Set Landmark-based Global Localization for Autonomous Navigation}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2025}}

SparseLoc is a global localization framework for city-scale scenarios that leverages vision-language foundation models to generate sparse semantic topometric maps in a zero-shot manner. It combines this map representation with a Monte Carlo localization scheme enhanced by a novel late optimization strategy, ensuring improved pose estimation. By constructing compact yet highly discriminative maps and refining localization through a carefully designed optimization schedule, SparseLoc overcomes the limitations of existing topometric localization methods, offering a better solution for global localization at the scale of a city. Our system achieves over a 5× improvement in localization accuracy compared to existing sparse mapping techniques. Despite utilizing only 1/500th of the points of dense mapping methods, it achieves comparable performance, maintaining an average global localization error below 5m and 2° on KITTI Sequences.

@inproceedings{bib_Leve_2025, AUTHOR = {Tourani, Siddharth and Reddy, Jayaram and Kumbar, Akash S and Tourani, Satyajit and Goyal, Nishant and Krishna, K Madhava and Reddy, N Dinesh and Khan, Muhammad Haris }, TITLE = {Leveraging 2D Priors and SDF Guidance for Dynamic Urban Scene Rendering}, BOOKTITLE = {International Conference on Computer Vision}. YEAR = {2025}}

Dynamic scene rendering and reconstruction play a crucial role in computer vision and augmented reality. Recent methods based on 3D Gaussian Splatting (3DGS), have enabled accurate modeling of dynamic urban scenes, but for urban scenes they require both camera and LiDAR data, ground-truth 3D segmentations and motion data in the form of tracklets or pre-defined object templates such as SMPL. In this work, we explore whether a combination of 2D object agnostic priors in the form of depth and point tracking coupled with a signed distance function (SDF) representation for dynamic objects can be used to relax some of these requirements. We present a novel approach that integrates Signed Distance Functions (SDFs) with 3D Gaussian Splatting (3DGS) to create a more robust object representation by harnessing the strengths of both methods. Our unified optimization framework enhances the geometric accuracy of 3D Gaussian splatting and improves deformation modeling within the SDF, resulting in a more adaptable and precise representation. We demonstrate that our method achieves state-of-the-art performance in rendering metrics even without LiDAR data on urban scenes. When incorporating LiDAR, our approach improved further in reconstructing and generating novel views across diverse object categories, without ground-truth 3D motion annotation. Additionally, our method enables various scene editing tasks, including scene decomposition, and scene composition.

@inproceedings{bib_GPD:_2025, AUTHOR = {S, Ajit and Mahajan, Parth and Saha, Kallol and Mandadi, Vishal Reddy and Paul, Pranjal and Wadhwani, Pawan and Bhowmick, Brojeshwar and Singh, Arun K and Krishna, K Madhava }, TITLE = {GPD: Guided Polynomial Diffusion for Motion Planning}, BOOKTITLE = {International Conference on Automation Science and Engineering}. YEAR = {2025}}

Diffusion-based motion planners are becoming popular due to their well-established performance improvements, stemming from sample diversity and the ease of incorporating new constraints directly during inference. However, a primary limitation of the diffusion process is the requirement for a substantial number of denoising steps, especially when the denoising process is coupled with gradient-based guidance. In this paper, we introduce, diffusion in the parametric space of trajectories, where the parameters are represented as Bernstein coefficients. We show that this representation greatly improves the effectiveness of the cost function guidance and the inference speed. We also introduce a novel stitching algorithm that leverages the diversity in diffusion-generated trajectories to produce collision-free trajectories with just a single cost function-guided model. We demonstrate that our approaches outperform current SOTA diffusion-based motion planners for manipulators and provide an ablation study on key components.

@inproceedings{bib_DG16_2025, AUTHOR = {Karim, Md Faizal and Hashmi, Mohammed Saad and Shreya, Bollimuntha and Tapeti, Mahesh Reddy and Singh, Gaurav and Govindan, Nagamanikandan and Krishna, K Madhava }, TITLE = {DG16M: A Large-Scale Dataset for Dual-Arm Grasping with Force-Optimized Grasps}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2025}}

Dual-arm robotic grasping is crucial for handling large objects that require stable and coordinated manipulation. While single-arm grasping has been extensively studied, datasets tailored for dual-arm settings remain scarce. We introduce a large-scale dataset of 16 million dual-arm grasps, evaluated under improved force-closure constraints. Additionally, we develop a benchmark dataset containing 300 objects with approximately 30,000 grasps, evaluated in a physics simulation environment, providing a better grasp quality assessment for dual-arm grasp synthesis methods. Finally, we demonstrate the effectiveness of our dataset by training a Dual-Arm Grasp Classifier network that outperforms the state of-the-art methods by 15%, achieving higher grasp success rates and improved generalization across objects.

@inproceedings{bib_DISC_2025, AUTHOR = {Teja, Bitla Bhanu and Krishna, K Madhava }, TITLE = {DISCO: Diffusion-based Inter-Agent Swarm Collision-free Optimization for UAVs}, BOOKTITLE = {IEEE-International Conference on Control & Automation}. YEAR = {2025}}

We present a diffusion-based generative model for coordinated trajectory planning in multi-UAV swarms. The proposed method represents each UAV’s trajectory in a Bernstein polynomial coefficient space and employs a de- noising diffusion process with self-attention layers to generate diverse, feasible motion plans. A safety filter is integrated into the generation pipeline to refine candidate trajectories, enforcing inter-drone collision avoidance and other feasibility constraints. The model is trained offline on a large set of expert demonstration trajectories, eliminating the need for reinforcement learning and manual reward function design. In experiments with a 16-UAV swarm using a dataset of collision-free trajectories, the approach achieved a high success rate in producing safe and smooth flight paths. These results demonstrate that the learned planner can rapidly generate a diverse set of smooth, collision-free trajectories for the swarm. Keywords- Swarm, Diffusion, UAV path planning, Joint Opti- mization

@inproceedings{bib_Towa_2025, AUTHOR = {Samrat, Chavan Aneesh and Chittawar, Sarthak and Bhat, Vineeth and Agrawal, Vaibhav and Srivastava, siddarth and Arora, Chetan and Krishna, K Madhava }, TITLE = {Towards Global Localization Using Multi-Modal Object-Instance Re-Identification}, BOOKTITLE = {Advances in Robotics}. YEAR = {2025}}

Re-identification (ReID) is a critical challenge in computer vision, predominantly studied in the context of pedestrians and vehicles. However, robust object-instance ReID, which has significant im- plications for tasks such as autonomous exploration, long-term perception, and scene understanding, remains underexplored. In this work, we address this gap by proposing a novel dual-path object-instance re-identification transformer architecture that inte- grates multimodal RGB and depth information. By leveraging depth data, we demonstrate improvements in ReID across scenes that are cluttered or have varying illumination conditions. Additionally, we develop a ReID-based localization framework that enables accurate camera localization and pose identification across different view- points. We validate our methods using two custom-built RGB-D datasets, as well as multiple sequences from the open-source TUM RGB-D datasets. Our approach demonstrates significant improve- ments in both object instance ReID (mAP of 75.18) and localization accuracy (success rate of 83% on TUM-RGBD), highlighting the essential role of object ReID in advancing robotic perception. Our models, frameworks, and datasets have been made publicly avail- able.

@inproceedings{bib_DA-V_2025, AUTHOR = {Karim, Md Faizal and Shreya, Bollimuntha and Hashmi, Mohammed Saad and Das, Autrio and Singh, Gaurav and Sridhar, Srinath and Singh, Arun Kumar and Govindan, Nagamanikandan and Krishna, K Madhava }, TITLE = {DA-VIL: Adaptive Dual-Arm Manipulation with Reinforcement Learning and Variable Impedance Control}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2025}}

Dual-arm manipulation is an area of growing interest in the robotics community. Enabling robots to perform tasks that require the coordinated use of two arms, is essential for complex manipulation tasks such as handling large objects, assembling components, and performing human-like interactions. However, achieving effective dual-arm manipulation is challenging due to the need for precise coordination, dynamic adaptability, and the ability to manage interaction forces between the arms and the objects being manipulated. We propose a novel pipeline that combines the advantages of policy learning based on environment feedback and gradient-based optimization to learn controller gains required for the control outputs. This allows the robotic system to dynamically modulate its impedance in response to task demands, ensuring stability and dexterity in dual-arm operations. We evaluate our pipeline on a trajectory-tracking task involving a variety of large, complex objects with different masses and geometries. The performance is then compared to three other established methods for controlling dual-arm robots, demonstrating superior results.

@inproceedings{bib_Crow_2025, AUTHOR = {Kumar, Naman and Singha, Antareep and Nanwani, Laksh and Potdar, Dhruv and Rastgar, Fatemeh and R., Tarun and Idoko, Simon and Singh, Arun Kumar and Krishna, K Madhava }, TITLE = {CrowdSurfer: Sampling Optimization Augmented with Vector-Quantized Variational AutoEncoder for Dense Crowd Navigation}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2025}}

Navigation amongst densely packed crowds remains a challenge for mobile robots. The complexity increases further if the environment layout changes, making the prior computed global plan infeasible. In this paper, we show that it is possible to dramatically enhance crowd navigation by just improving the local planner. Our approach combines generative modelling with inference time optimization to generate sophisticated long-horizon local plans at interactive rates. More specifically, we train a Vector Quantized Variational AutoEncoder to learn a prior over the expert trajectory distribution conditioned on the perception input. At run-time, this is used as an initialization for a sampling-based optimizer for further refinement. Our approach does not require any sophisticated prediction of dynamic obstacles and yet provides state-of-the-art performance. In particular, we compare against the recent DRL-VO approach and show a 40% improvement in success rate and a 6% improvement in travel time.

@inproceedings{bib_Adap_2025, AUTHOR = {Singh, Shivam and Swaminathan, Karthik and Dash, Nabanita and Singh, Ramandeep and Banerjee, Snehasis and Sridharan, Mohan and Krishna, K Madhava }, TITLE = {AdaptBot: Combining LLM with Knowledge Graphs and Human Input for Generic-to-Specific Task Decomposition and Knowledge Refinement}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2025}}

Embodied agents assisting humans are often asked to complete a new task in a new scenario. An agent preparing a particular dish in the kitchen based on a known recipe may be asked to prepare a new dish or to perform cleaning tasks in the storeroom. There may not be sufficient resources, e.g., time or labeled examples, to train the agent for these new situations. Large Language Models (LLMs) trained on considerable knowledge across many domains are able to predict a sequence of abstract actions for such new tasks and scenarios, although it may not be possible for the agent to execute this action sequence due to task-, agent-, or domain-specific constraints. Our framework addresses these challenges by leveraging the generic predictions provided by LLM and the prior domain-specific knowledge encoded in a Knowledge Graph (KG), enabling an agent to quickly adapt to new tasks and scenarios. The robot also solicits and uses human input as needed to refine its existing knowledge. Based on experimental evaluation over cooking and cleaning tasks in simulation domains, we demonstrate that the interplay between LLM, KG, and human input leads to substantial performance gains compared with just using the LLM output.

@inproceedings{bib_Anti_2024, AUTHOR = {Arora, Raghav and Singh, Shivam and Swaminathan, Karthik and Datta, Ahana and Banerjee, Snehasis and Bhowmick, Brojeshwar and Jatavallabhula, Krishna Murthy and Sridharan, Mohan and Krishna, K Madhava }, TITLE = {Anticipate & Act: Integrating LLMs and Classical Planning for Efficient Task Execution in Household Environments}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2024}}

Assistive agents performing household tasks such as making the bed or cooking breakfast often compute and execute actions that accomplish one task at a time. However, efficiency can be improved by anticipating upcoming tasks and computing an action sequence that jointly achieves these tasks. State-of-the-art methods for task anticipation use data-driven deep networks and Large Language Models (LLMs), but they do so at the level of high-level tasks and/or require many training examples. Our framework leverages the generic knowledge of LLMs through a small number of prompts to perform high-level task anticipation, using the anticipated tasks as goals in a classical planning system to compute a sequence of finer-granularity actions that jointly achieve these goals. We ground and evaluate our framework’s abilities in realistic scenarios in the VirtualHome environment and demonstrate a 31% reduction in execution time compared with a system that does not consider upcoming tasks.

@inproceedings{bib_Anti_2024, AUTHOR = {Singh, Shivam and Krishna, K Madhava }, TITLE = {Anticipate & Collab: Data-driven Task Anticipation and Knowledge-driven Planning for Human-robot Collaboration}, BOOKTITLE = {Technical Report}. YEAR = {2024}}

An agent assisting humans in daily living activities can collaborate more effectively by anticipating upcoming tasks. Data-driven methods represent the state of the art in task anticipation, planning, and related problems, but these methods are resource-hungry and opaque. Our prior work introduced a proof of concept framework that used an LLM to anticipate 3 high-level tasks that served as goals for a classical planning system that computed a sequence of low-level actions for the agent to achieve these goals. This paper describes DaTAPlan, our framework that significantly extends our prior work toward human-robot collaboration. Specifically, DaTAPlan’s planner computes actions for an agent and a human to collaboratively and jointly achieve the tasks anticipated by the LLM, and the agent automatically adapts to unexpected changes in human action outcomes and preferences. We evaluate DaTAPlan’s capabilities in a realistic simulation environment, demonstrating accurate task anticipation, effective human-robot collaboration, and the ability to adapt to unexpected changes.

@inproceedings{bib_Open_2024, AUTHOR = {Nanwani, Laksh and Gupta, Kumaraditya and Mathur, Aditya and Agrawal, Swayam and Hafez, A. H. Abdul and Krishna, K Madhava }, TITLE = {Open-set 3D semantic instance maps for vision language navigation – O3D-SIM}, BOOKTITLE = {Advanced Robotics}. YEAR = {2024}}

Humans excel at forming mental maps of their surroundings, equipping them to understand object relationships and navigate based on language queries. Our previous work SI Maps (Nanwani L, Agarwal A, Jain K, et al. Instance-level semantic maps for vision language navigation. In: 2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). IEEE; 2023 Aug.) showed that having instance-level information and the semantic understanding of an environment helps significantly improve performance for language-guided tasks. We extend this instance-level approach to 3D while increasing the pipeline's robustness and improving quantitative and qualitative results. Our method leverages foundational models for object recognition, image segmentation, and feature extraction. We propose a representation that results in a 3D point cloud map with instance-level embeddings, which bring in the semantic understanding that natural language commands can query. Quantitatively, the work improves upon the success rate of language-guided tasks. At the same time, we qualitatively observe the ability to identify instances more clearly and leverage the foundational models and language and image-aligned embeddings to identify objects that, otherwise, a closed-set approach wouldn't be able to identify.

@inproceedings{bib_Revi_2024, AUTHOR = {Sai, Shubodh and Garg, Kartik and Kolathaya, Shishir and Krishna, K Madhava and Garg, Sourav }, TITLE = {Revisit Anything: Visual Place Recognition via Image Segment Retrieval}, BOOKTITLE = {European Conference on Computer Vision}. YEAR = {2024}}

Accurately recognizing a revisited place is crucial for embodied agents to localize and navigate. This requires visual representations to be distinct, despite strong variations in camera viewpoint and scene appearance. Existing visual place recognition pipelines encode the whole image and search for matches. This poses a fundamental challenge in matching two images of the same place captured from different camera viewpoints: the similarity of what overlaps can be dominated by the dissimilarity of what does not overlap. We address this by encoding and searching for image segments instead of the whole images. We propose to use open-set image segmentation to decompose an image into `meaningful' entities (i.e., things and stuff). This enables us to create a novel image representation as a collection of multiple overlapping subgraphs connecting a segment with its neighboring segments, dubbed SuperSegment. Furthermore, to efficiently encode these SuperSegments into compact vector representations, we propose a novel factorized representation of feature aggregation. We show that retrieving these partial representations leads to significantly higher recognition recall than the typical whole image based retrieval. Our segments-based approach, dubbed SegVLAD, sets a new state-of-the-art in place recognition on a diverse selection of benchmark datasets, while being applicable to both generic and task-specialized image encoders. Finally, we demonstrate the potential of our method to revisit anything by evaluating our method on an object instance retrieval task, which bridges the two disparate areas of research: visual place recognition and object-goal navigation, through their common aim of recognizing goal objects specific to a place.

@inproceedings{bib_LeGo_2024, AUTHOR = {Paul, Pranjal and Garg, Anant and Choudhary, Tushar and Singh, Arun Kumar and Krishna, K Madhava }, TITLE = {LeGo-Drive: Language-enhanced Goal-oriented Closed Loop End-to-End Autonomous Driving}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2024}}

@inproceedings{bib_Cons_2024, AUTHOR = {Singh, Gaurav and Hemant, Kalwar Sanket and Karim, Md Faizal and Sen, Bipasha and Govindan, Nagamanikandan and Sridhar, Srinath and Krishna, K Madhava }, TITLE = {Constrained 6-DoF Grasp Generation on Complex Shapes for Improved Dual-Arm Manipulation}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2024}}

Efficiently generating grasp poses tailored to specific regions of an object is vital for various robotic manipulation tasks, especially in a dual-arm setup. This scenario presents a significant challenge due to the complex geometries involved, requiring a deep understanding of the local geometry to generate grasps efficiently on the specified constrained regions. Existing methods only explore settings involving table-top/small objects and require augmented datasets to train, limiting their performance on complex objects. We propose CGDF: Constrained Grasp Diffusion Fields, a diffusion-based grasp generative model that generalizes to objects with arbitrary geometries, as well as generates dense grasps on the target regions. CGDF uses a part-guided diffusion approach that enables it to get high sample efficiency in constrained grasping without explicitly training on massive constraint-augmented datasets. We provide qualitative and quantitative comparisons using analytical metrics and in simulation, in both unconstrained and constrained settings to show that our method can generalize to generate stable grasps on complex objects, especially useful for dual-arm manipulation settings, while existing methods struggle to do so. More results, code and an extended version of the paper can be found on the project page: https://constrained-grasp-diffusion.github.io/

@inproceedings{bib_QueS_2024, AUTHOR = {Mehan, Yash and Gupta, Kumaraditya and Sreekanth, Jayanti Rohit and Govil, Anirudh and Garg, Sourav and Krishna, K Madhava }, TITLE = {QueSTMaps: Queryable Semantic Topological Maps for 3D Scene Understanding}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2024}}

Understanding the structural organisation of 3D indoor scenes in terms of rooms is often accomplished via floorplan extraction. Robotic tasks such as planning and navigation require a semantic understanding of the scene as well. This is typically achieved via object-level semantic segmentation. However, such methods struggle to segment out topological regions like “kitchen” in the scene. In this work, we introduce a two-step pipeline. First, we extract a topological map, i.e., floorplan of the indoor scene using a novel multi-channel occupancy representation. Then, we generate CLIP-aligned features and semantic labels for every room instance based on the objects it contains using a self-attention transformer. Our language-topology alignment supports natural language querying, e.g. a “place to cook” locates the “kitchen”. We outperform the current state-of-the-art on room segmentation by ∼20% and room classification by ∼12%. Our detailed qualitative analysis and ablation studies provide insights into the problem of joint structural and semantic 3D scene understanding.

@inproceedings{bib_Find_2024, AUTHOR = {Singh, Gurkirat and Harithas, Sudarshan S and Samrat, Chavan Aneesh and SHARMA, SARTHAK and Patni, Suraj and Arora, Chetan and Krishna, K Madhava }, TITLE = {FinderNet: A Data Augmentation Free Canonicalization aided Loop Detection and Closure technique for Point clouds in 6-DOF separation.}, BOOKTITLE = {Winter Conference on Applications of Computer Vision}. YEAR = {2024}}

We focus on the problem of LiDAR point cloud based loop detection (or Finding) and closure (LDC) for mobile robots. State-of-the-art (SOTA) methods directly generate learned embeddings from a given point cloud, require large data augmentation, and are not robust to wide viewpoint variations in 6 Degrees-of-Freedom (DOF). Moreover, the absence of strong priors in an unstructured point cloud leads to highly inaccurate LDC. In this original approach, we propose independent roll and pitch canonicalization of point clouds using a common dominant ground plane. We discretize the canonicalized point clouds along the axis perpendicular to the ground plane leads to images simi- lar to digital elevation maps (DEMs), which expose strong spatial priors in the scene. Our experiments show that LDC based on learnt embeddings from such DEMs is not only data efficient but also significantly more robust, and generalizable than the current SOTA. We report an (aver- age precision for loop detection, mean absolute transla- tion/rotation error) improvement of (8.4, 16.7/5.43)% on the KITTI08 sequence, and (11.0, 34.0/25.4)% on GPR10 sequence, over the current SOTA. To further test the ro- bustness of our technique on point clouds in 6-DOF motion we create and opensource a custom dataset called Lidar- UrbanFly Dataset (LUF) which consists of point clouds ob- tained from a LiDAR mounted on a quadrotor. More details on our website https://gsc2001.github.io/FinderNet/

@inproceedings{bib_Talk_2024, AUTHOR = {Choudhary, Tushar and Dewangan, Vikrant and Chandhok, Shivam and Priyadarshan, Shubham and Jain, Anushka and Singh, Arun K. and Srivastava, Siddharth and Jatavallabhula, Krishna Murthy and Krishna, K Madhava }, TITLE = {Talk2BEV: Language-enhanced Bird’s-eye View Maps for Autonomous Driving}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2024}}

This work introduces Talk2BEV, a large vision-language model (LVLM) interface for bird’s-eye view (BEV) maps in autonomous driving contexts. While existing perception systems for autonomous driving scenarios have largely focused on a pre-defined (closed) set of object categories and driving scenarios, Talk2BEV blends recent advances in general-purpose language and vision models with BEV-structured map representations, eliminating the need for task-specific models. This enables a single system to cater to a variety of autonomous driving tasks encompassing visual and spatial reasoning, predicting the intents of traffic actors, and decision-making based on visual cues. We extensively evaluate Talk2BEV on a large number of scene understanding tasks that rely on both the ability to interpret free-form natural language queries and in grounding these queries to the visual context embedded into the language-enhanced BEV map. To enable further research in LVLMs for autonomous driving scenarios, we develop and release Talk2BEV-Bench, a benchmark encompassing 1000 human-annotated BEV scenarios, with more than 20,000 questions and ground-truth responses from the NuScenes dataset.

@inproceedings{bib_Imag_2024, AUTHOR = {Pathre, Pranjali Pramod and Gupta, Gunjan and Qureshi, Mohammad Nomaan and Brunda, Mandyam and Brahmbhatt, Samarth and Krishna, K Madhava }, TITLE = {Imagine2Servo: Intelligent Visual Servoing with Diffusion-Driven Goal Generation for Robotic Tasks}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2024}}

@inproceedings{bib_Leve_2024, AUTHOR = {Tourani, Siddharth and Reddy, Gurram Jayarami and Thakur, Sarvesh and Krishna, K Madhava and Khan, Muhammad Haris and Reddy, N Dinesh }, TITLE = {Leveraging Cycle-Consistent Anchor Points for Self-Supervised RGB-D Registration}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2024}}

With the rise in consumer depth cameras, a wealth of unlabeled RGB-D data has become available. This prompts the question of how to utilize this data for geometric reasoning of scenes. While many RGB-D registration methods rely on geometric and feature-based similarity, we take a different approach. We use cycle-consistent keypoints as salient points to enforce spatial coherence constraints during matching, improving correspondence accuracy. Additionally, we introduce a novel pose block that combines a GRU recurrent unit with transformation synchronization, blending historical and multi-view data. Our approach surpasses previous selfsupervised registration methods on ScanNet and 3DMatch, even outperforming some older supervised methods. We also integrate our components into existing methods, showing their effectiveness.

@inproceedings{bib_Anti_2024, AUTHOR = {Singh, Shivam and Datta, Ahana and Arora, Raghav and Swaminathan, Karthik and Banerjee, Snehasis and Bhowmick, Brojeshwar and Jatavallabhula, Krishna Murthy and Sridharan, Mohan and Krishna, K Madhava }, TITLE = {Anticipate & Act : Integrating LLMs and Classical Planning for Efficient Task Execution in Household Environments}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2024}}

Assistive agents performing household tasks such as making the bed or cooking breakfast often compute and execute actions that accomplish one task at a time. However, efficiency can be improved by anticipating upcoming tasks and computing an action sequence that jointly achieves these tasks. State-of-the-art methods for task anticipation use data-driven deep networks and Large Language Models (LLMs), but they do so at the level of high-level tasks and/or require many training examples. Our framework leverages the generic knowledge of LLMs through a small number of prompts to perform high-level task anticipation, using the anticipated tasks as goals in a classical planning system to compute a sequence of finer-granularity actions that jointly achieve these goals. We ground and evaluate our framework's abilities in realistic scenarios in the VirtualHome environment and demonstrate a 31% reduction in execution time compared with a system that does not consider upcoming tasks.

@inproceedings{bib_EDMP_2024, AUTHOR = {Saha, Kallol and Mandadi, Vishal Reddy and Reddy, Gurram Jayarami and S, Ajit and Agarwal, Aditya and Sen, Bipasha and Singh, Arun and Krishna, K Madhava }, TITLE = {EDMP: Ensemble-of-costs-guided Diffusion for Motion Planning}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2024}}

Classical motion planning for robotic manipulation includes a set of general algorithms that aim to minimize a scene-specific cost of executing a given plan. This approach offers remarkable adaptability, as they can be directly used off-the-shelf for any new scene without needing specific training datasets. However, without a prior understanding of what diverse valid trajectories are and without specially designed cost functions for a given scene, the overall solutions tend to have low success rates. While deep-learning-based algorithms tremendously improve success rates, they are much harder to adopt without specialized training datasets. We propose EDMP, an Ensemble-of-costs-guided Diffusion for Motion Planning that aims to combine the strengths of classical and deep-learning-based motion planning. Our diffusion-based network is trained on a set of diverse kinematically valid trajectories. Like classical planning, for any new scene at the time of inference, we compute scene-specific costs such as "collision cost" and guide the diffusion to generate valid trajectories that satisfy the scene-specific constraints. Further, instead of a single cost function that may be insufficient in capturing diversity across scenes, we use an ensemble of costs to guide the diffusion process, significantly improving the success rate compared to classical planners. EDMP performs comparably with SOTA deep-learning-based methods while retaining the generalization capabilities primarily associated with classical planners.

@inproceedings{bib_ATPP_2024, AUTHOR = {Pal, Kaustab and Sharma, M Aditya and Sharma, Avinash and Krishna, K Madhava }, TITLE = {ATPPNet: Attention based Temporal Point cloud Prediction Network}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2024}}

Point cloud prediction is an important yet challenging task in the field of autonomous driving. The goal is to predict future point cloud sequences that maintain object structures while accurately representing their temporal motion. These predicted point clouds help in other subsequent tasks like object trajectory estimation for collision avoidance or estimating locations with the least odometry drift. In this work, we present ATPPNet, a novel architecture that predicts future point cloud sequences given a sequence of previous time step point clouds obtained with LiDAR sensor. ATPPNet leverages Conv-LSTM along with channel-wise and spatial attention dually complemented by a 3D-CNN branch for extracting an enhanced spatio-temporal context to recover high quality fidel predictions of future point clouds. We conduct extensive experiments on publicly available datasets and report impressive performance outperforming the existing methods. We also conduct a thorough ablative study of the proposed architecture and provide an application study that highlights the potential of our model for tasks like odometry estimation.

@inproceedings{bib_LIP-_2024, AUTHOR = {Sai, Shubodh and Omama, Mohd and Zaidi, Husain and Parihar, Udit Singh and Krishna, K Madhava }, TITLE = {LIP-Loc: LiDAR Image Pretraining for Cross-Modal Localization}, BOOKTITLE = {Winter Conference on Applications of Computer Vision Workshops}. YEAR = {2024}}

Global visual localization in LiDAR-maps, crucial for autonomous driving applications, remains largely unexplored due to the challenging issue of bridging the cross-modal heterogeneity gap. Popular multi-modal learning approach Contrastive Language-Image Pre-Training (CLIP) has popularized contrastive symmetric loss using batch construction technique by applying it to multi-modal domains of text and image. We apply this approach to the domains of 2D image and 3D LiDAR points on the task of cross-modal localization. Our method is explained as follows: A batch of N (image, LiDAR) pairs is constructed so as to predict what is the right match between N X N possible pairings across the batch by jointly training an image encoder and LiDAR encoder to learn a multi-modal embedding space. In this way, the cosine similarity between N positive pairings is maximized, whereas that between the remaining negative pairings is minimized. Finally, over the obtained similarity scores, a symmetric cross-entropy loss is optimized. To the best of our knowledge, this is the first work to apply batched loss approach to a cross-modal setting of image & LiDAR data and also to show Zero-shot transfer in a visual localization setting. We conduct extensive analyses on standard autonomous driving datasets such as KITTI and KITTI-360 datasets. Our method outperforms state-of-the-art recall@1 accuracy on the KITTI-360 dataset by 22.4%, using only perspective images, in contrast to the state-of-the-art approach, which utilizes the more informative fisheye images. Additionally, this superior performance is achieved without resorting to complex architectures. Moreover, we demonstrate the zero-shot capabilities of our model and we beat SOTA by 8% without even training on it. Furthermore, we establish the first benchmark for cross-modal localization on the KITTI dataset.

@inproceedings{bib_Find_2024, AUTHOR = {Harithas, Sudarshan S and Singh, Gurkirat and Chavan, Aneesh and SHARMA, SARTHAK and Patni, Suraj and Arora, Chetan and Krishna, K Madhava }, TITLE = {FinderNet: A Data Augmentation Free Canonicalization aided Loop Detection and Closure technique for Point clouds in 6-DOF separation.}, BOOKTITLE = {Winter Conference on Applications of Computer Vision}. YEAR = {2024}}

We focus on the problem of LiDAR point cloud based loop detection (or Finding) and closure (LDC) for mobile robots. State-of-the-art (SOTA) methods directly generate learned embeddings from a given point cloud, require large data augmentation, and are not robust to wide viewpoint variations in 6 Degrees-of-Freedom (DOF). Moreover, the absence of strong priors in an unstructured point cloud leads to highly inaccurate LDC. In this original approach, we propose independent roll and pitch canonicalization of point clouds using a common dominant ground plane. We discretize the canonicalized point clouds along the axis perpendicular to the ground plane leads to images similar to digital elevation maps (DEMs), which expose strong spatial priors in the scene. Our experiments show that LDC based on learnt embeddings from such DEMs is not only data efficient but also significantly more robust, and generalizable than the current SOTA. We report an (average precision for loop detection, mean absolute translation/rotation error) improvement of (8.4, 16.7/5.43)% on the KITTI08 sequence, and (11.0, 34.0/25.4)% on GPR10 sequence, over the current SOTA. To further test the robustness of our technique on point clouds in 6-DOF motion we create and opensource a custom dataset called LidarUrbanFly Dataset (LUF) which consists of point clouds obtained from a LiDAR mounted on a quadrotor. More details on our website https://gsc2001.github.io/FinderNet/

@inproceedings{bib_Sequ_2023, AUTHOR = {Gireesh, Nandiraju and Agrawal, Ayush and Datta, Ahana and Banerjee, Snehasis and Sridharan, Mohan and Bhowmick, Brojeshwar and Krishna, K Madhava }, TITLE = {Sequence-Agnostic MultiObject Navigation}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2023}}

The Multi-Object Navigation (MultiON) task requires a robot to localize an instance (each) of multiple object classes. It is a fundamental task for an assistive robot in a home or a factory. Existing methods for MultiON have viewed this as a direct extension of Object Navigation (ON), the task of localising an instance of one object class, and are pre-sequenced, i.e., the sequence in which the object classes are to be explored is provided in advance. This is a strong limitation in practical applications characterized by dynamic changes. This paper describes a deep reinforcement learning framework for sequence-agnostic MultiON based on an actor-critic architecture and a suitable reward specification. Our framework leverages past experiences and seeks to reward progress toward individual as well as multiple target object classes. We use photo-realistic scenes from the Gibson benchmark dataset in the AI Habitat 3D simulation environment to experimentally show that our method performs better than a pre-sequenced approach and a state of the art ON method extended to MultiON.

@inproceedings{bib_Conc_2023, AUTHOR = {MURTHY, JATAVALLABHULA KRISHNA and Tewari, Ayush and Tenenbaum, Joshua B. and Melo, Celso Miguel De and Krishna, K Madhava and Paull, Liam and Shkurti, Florian and Kuwajerwala, Alihusein and GU, Qiao and Omama, Mohd and Chen, Tao and Maalouf, Alaa and Li, Shuang and Iyer, Ganesh and Saryazdi, Soroush }, TITLE = {ConceptFusion: Open-set Multimodal 3D Mapping}, BOOKTITLE = {Robotics: Science and Systems XIX}. YEAR = {2023}}

Abstract—Building 3D maps of the environment is central to robot navigation, planning, and interaction with objects in a scene. Most existing approaches that integrate semantic concepts with 3D maps largely remain confined to the closed-set setting: they can only reason about a finite set of concepts, pre-defined at training time. Further, these maps can only be queried using class labels, or in more recent work, using text prompts. We address both these issues with ConceptFusion, a scene representation that is: (i) fundamentally open-set, enabling reasoning beyond a closed set of concepts (ii) inherently multimodal, enabling a diverse range of possible queries to the 3D map, from language, to images, to audio, to 3D geometry, all working in concert. ConceptFusion leverages the open-set capabilities of today’s foundation models that have been pretrained on internet-scale data to reason about concepts across modalities such as natural language, images, and audio. We demonstrate that pixel-aligned open-set features can be fused into 3D maps via traditional SLAM and multi-view fusion approaches. This enables effective zero-shot spatial reasoning, not needing any additional training or finetuning, and retains long-tailed concepts better than supervised approaches, outperforming them by more than 40% margin on 3D IoU. We extensively evaluate ConceptFusion on a number of real-world datasets, simulated home environments, a real-world tabletop manipulation task, and an autonomous driving platform. We showcase new avenues for blending foundation models with 3D open-set multimodal mapping. We encourage the reader to view the demos on our project page: https://concept-fusion.github.io

@inproceedings{bib_Hilb_2023, AUTHOR = {Sharma, Basant and Sharma, M Aditya and Krishna, K Madhava and Singh, Arun Kumar }, TITLE = {Hilbert Space Embedding-Based Trajectory Optimization for Multi-Modal Uncertain Obstacle Trajectory Prediction}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2023}}

Safe autonomous driving critically depends on how well the ego-vehicle can predict the trajectories of neighboring vehicles. To this end, several trajectory prediction algorithms have been presented in the existing literature. Many of these approaches output a multi-modal distribution of obstacle trajectories instead of a single deterministic prediction to account for the underlying uncertainty. However, existing planners cannot handle the multi-modality based on just sample-level information of the predictions. With this motivation, this paper proposes a trajectory optimizer that can leverage the distributional aspects of the prediction in a computationally tractable and sample-efficient manner. Our optimizer can work with arbitrarily complex distributions and thus can be used with output distribution represented as a deep neural network. The core of our approach is built on embedding distribution in Reproducing Kernel Hilbert Space (RKHS), which we leverage in two ways. First, we propose an RKHS embedding approach to select probable samples from the obstacle trajectory distribution. Second, we rephrase chance-constrained optimization as distribution matching in RKHS and propose a novel sampling-based optimizer for its solution. We validate our approach with hand-crafted and neural network-based predictors trained on real-world datasets and show improvement over the existing stochastic optimization approaches in safety metrics.

@inproceedings{bib_Inst_2023, AUTHOR = {Nanwani, Laksh and Krishna, K Madhava and Agarwal, Anmol and Jain, Kanishk and Prabhakar, Raghav and Monis, Aaron Anthony and Mathur, Aditya and Murthy, Krishna and Hafez, Abdul and Gandhi, Vineet }, TITLE = {Instance-Level Semantic Maps for Vision Language Navigation}, BOOKTITLE = {IEEE International Conference on Robot and Human Interactive Communication}. YEAR = {2023}}

Humans have a natural ability to perform semantic associations with the surrounding objects in the environment. This allows them to create a mental map of the environment, allowing them to navigate on-demand when given linguistic instructions. A natural goal in Vision Language Navigation (VLN) research is to impart autonomous agents with similar capabilities. Recent works take a step towards this goal by creating a semantic spatial map representation of the environment without any labeled data. However, their representations are limited for practical applicability as they do not distinguish between different instances of the same object. In this work, we address this limitation by integrating instance-level information into spatial map representation using a community detection algorithm and utilizing word ontology learned by large language models (LLMs) to perform open-set semantic associations in the mapping representation. The resulting map representation improves the navigation performance by two-fold (233%) on realistic language commands with instance-specific descriptions compared to the baseline. We validate the practicality and effectiveness of our approach through extensive qualitative and quantitative experiments.

@inproceedings{bib_HyP-_2023, AUTHOR = {, and Sen, Bipasha and Singh, Gaurav and Agarwal, Aditya and Rohith, Agaram and Krishna, K Madhava and Sridhar, Srinath }, TITLE = {HyP-NeRF: Learning Improved NeRF Priors using a HyperNetwork.}, BOOKTITLE = {Neural Information Processing Systems}. YEAR = {2023}}

@inproceedings{bib_AnyL_2023, AUTHOR = {Keetha, Nikhil Varma and Mishra, Avneesh and Karhade, Jay and Jatavallabhula, Krishna Murthy and Scherer, Sebastian and Krishna, K Madhava and Garg, Sourav }, TITLE = {AnyLoc: Towards Universal Visual Place Recognition}, BOOKTITLE = {IEEE Robotics and Automation Letters}. YEAR = {2023}}

Visual Place Recognition (VPR) is vital for robot localization. To date, the most performant VPR approaches are environment- and task-specific : while they exhibit strong performance in structured environments (predominantly urban driving), their performance degrades severely in unstructured environments, rendering most approaches brittle to robust real-world deployment. In this work, we develop a universal solution to VPR – a technique that works across a broad range of structured and unstructured environments (urban, outdoors, indoors, aerial, underwater, and subterranean environments) without any re-training or finetuning. We demonstrate that general-purpose feature representations derived from off-the-shelf self-supervised models with no VPR-specific training are the right substrate upon which to build such a universal VPR solution. Combining these derived features with unsupervised feature aggregation enables our suite of methods, AnyLoc , to achieve up to 4× significantly higher performance than existing approaches. We further obtain a 6% improvement in performance by characterizing the semantic properties of these features, uncovering unique domains which encapsulate datasets from similar environments. Our detailed experiments and analysis lay a foundation for building VPR solutions that may be deployed anywhere , anytime , and across anyview . We encourage the readers to explore our project page and interactive demos: https://anyloc.github.io/ .

@inproceedings{bib_DynG_2023, AUTHOR = {Gupta, Gunjan and Mittal, Vedansh and Krishna, K Madhava }, TITLE = {DynGraspVS: Servoing Aided Grasping for Dynamic Environments}, BOOKTITLE = {International Conference on Robotics and Biomimetics}. YEAR = {2023}}

Visual servoing has been gaining popularity in various real-world vision-centric robotic applications. Autonomous robotic grasping often deals with unseen and unstructured environments, and in this task, Visual Servoing has been able to generate improved end-effector control by providing visual feedback. However, existing Servoing-aided grasping methods tend to fail at the task of grasping in dynamic environments i.e. - moving objects. In this paper, we introduce DynGraspVS, a novel Image-based Visual Servoing-aided Grasping approach that models the motion of moving objects in its interaction matrix. Leveraging a single-step rollout strategy, our approach achieves a remarkable increase in success rate, while converging faster and achieving a smoother trajectory, while maintaining precise alignments in six degrees of freedom. By integrating the velocity information into the interaction matrix, our method is able to successfully complete the challenging task of robotic grasping in the case of dynamic objects, while outperforming existing deep Model Predictive Control (MPC) based methods in the PyBullet simulation environment. We test it with a range of objects in the YCB dataset with varying range of shapes, sizes, and material properties. We report various evaluation metrics such as photometric error, success rate, time taken, and trajectory length.

@inproceedings{bib_CLIP_2023, AUTHOR = {Agrawal, Ayush and Arora, Raghav and Datta, Ahana and Banerjee, Snehasis and Bhowmick, Brojeshwar and Jatavallabhula, Krishna Murthy and Sridharan, Mohan and Krishna, K Madhava }, TITLE = {CLIPGraphs: Multimodal Graph Networks to Infer Object-Room Affinities}, BOOKTITLE = {IEEE International Conference on Robot and Human Interactive Communication}. YEAR = {2023}}

This paper introduces a novel method for determining the best room to place an object in, for embodied scene rearrangement. While state-of-the-art approaches rely on large language models (LLMs) or reinforcement learned (RL) policies for this task, our approach, CLIPGraphs, efficiently combines commonsense domain knowledge, data-driven methods, and recent advances in multimodal learning. Specifically, it (a) encodes a knowledge graph of prior human preferences about the room location of different objects in home environments, (b) incorporates vision-language features to support multimodal queries based on images or text, and (c) uses a graph network to learn object-room affinities based on embeddings of the prior knowledge and the vision-language features. We demonstrate that our approach provides better estimates of the most appropriate location of objects from a benchmark set of object categories in comparison with state-of-the-art baselines

@inproceedings{bib_Hilb_2023, AUTHOR = {Sharma, Basant and Sharma, M Aditya and Krishna, K Madhava and Singh, Arun Kumar }, TITLE = {Hilbert Space Embedding-based Trajectory Optimization for Multi-Modal Uncertain Obstacle Trajectory Prediction}, BOOKTITLE = {Technical Report}. YEAR = {2023}}

Safe autonomous driving critically depends on how well the ego-vehicle can predict the trajectories of neighboring vehicles. To this end, several trajectory prediction algorithms have been presented in the existing literature. Many of these approaches output a multi-modal distribution of obstacle trajectories instead of a single deterministic prediction to account for the underlying uncertainty. However, existing planners cannot handle the multi-modality based on just sample-level information of the predictions. With this motivation, this paper proposes a trajectory optimizer that can leverage the distributional aspects of the prediction in a computationally tractable and sample-efficient manner. Our optimizer can work with arbitrarily complex distributions and thus can be used with output distribution represented as a deep neural network. The core of our approach is built on embedding distribution in Reproducing Kernel Hilbert Space (RKHS), which we leverage in two ways. First, we propose an RKHS embedding approach to select probable samples from the obstacle trajectory distribution. Second, we rephrase chance-constrained optimization as distribution matching in RKHS and propose a novel sampling-based optimizer for its solution. We validate our approach with hand-crafted and neural network-based predictors trained on real-world datasets and show improvement over the existing stochastic optimization approaches in safety metrics.

@inproceedings{bib_Neur_2023, AUTHOR = {Pal, Kaustab and Sharma, M Aditya and Omama, Mohd and Shah, Parth N and Krishna, K Madhava }, TITLE = {NeuroSMPC: A Neural Network Guided Sampling Based MPC for On-Road Autonomous Driving}, BOOKTITLE = {International Conference on Automation Science and Engineering}. YEAR = {2023}}

In this paper we show an effective means of integrating data driven frameworks to sampling based optimal control to vastly reduce the compute time for easy adoption and adaptation to real time applications such as on-road autonomous driving in the presence of dynamic actors. Presented with training examples, a spatio-temporal CNN learns to predict the optimal mean control over a finite horizon that precludes further resampling, an iterative process that makes sampling based optimal control formulations difficult to adopt in real time settings. Generating control samples around the network-predicted optimal mean retains the advantage of sample diversity while enabling real time rollout of trajectories that avoids multiple dynamic obstacles in an on-road navigation setting. Further the 3D CNN architecture implicitly learns the future trajectories of the dynamic agents in the scene resulting in successful collision free …

@inproceedings{bib_UAV-_2023, AUTHOR = {Srivastava, Kushagra and Patel, Dhruv and Jha, Aditya Kumar and Jha, Mohhit Kumar and Kiran, Santosh Ravi and Ramancharla, Pradeep Kumar and K, Harikumar and Krishna, K Madhava }, TITLE = {UAV-Based Visual Remote Sensing for Automated Building Inspection}, BOOKTITLE = {European Conference on Computer Vision Workshops}. YEAR = {2023}}

Unmanned Aerial Vehicle (UAV) based remote sensing system incorporated with computer vision has demonstrated potential for assisting building construction and in disaster management like damage assessment during earthquakes. The vulnerability of a building to earthquake can be assessed through inspection that takes into account the expected damage progression of the associated component and the component’s contribution to structural system performance. Most of these inspections are done manually, leading to high utilization of manpower, time, and cost. This paper proposes a methodology to automate these inspections through UAV-based image data collection and a software library for post-processing that helps in estimating the seismic structural parameters. The key parameters considered here are the distances between adjacent buildings, building plan-shape, building

@inproceedings{bib_Dise_2023, AUTHOR = {Mandadi, Vishal Reddy and Krishna, K Madhava and Saha, Kallol and Guhathakurta, Dipanwita and Qureshi, Mohammad Nomaan and AGARWAL, ADITYA and Sen, Bipasha and Das, Dipanjan and Bhowmick, Brojeshwar and Singh, Arun }, TITLE = {Disentangling Planning and Control for Non-prehensile Tabletop Manipulation}, BOOKTITLE = {International Conference on Automation Science and Engineering}. YEAR = {2023}}

Manipulating a target object using non-prehensile actions presents a complex problem that requires addressing multiple constraints, such as finding collision-free trajectories, modeling the object dynamics, and speed of execution. Previous approaches have relied on contact-rich manipulation, where the object moves predictably while attached to the manipulator, thereby avoiding the need to model the object’s dynamics. However, this assumption may not always hold, and different end-effectors may be necessary depending on the use case. In place of contact-rich manipulation, we implement the pushingby-striking method (without tactile feedback) by explicitly modeling the object dynamics. Our novel framework disentangles planning and control enabling us to operate in a contextfree manner. Our method consists of two components: an A* planner and a low-level RL controller. The low-level RL controller feeds on purely object-centric representations and has an object-centric action space, thus making it agnostic of the scene context. On the other hand, the planning module operates idependently of the low-level control and only takes scene context into account, making the approach quick to adapt and implement. We demonstrate the performance of our algorithm against a global RL policy which is computationally

@inproceedings{bib_Urba_2023, AUTHOR = {Harithas, Sudarshan S and Swamy, Thatavarthy VVSST Ayyappa and Singh, Gurkirat and Singh, Arun K and Krishna, K Madhava }, TITLE = {UrbanFly: Uncertainty-Aware Planning for Navigation Amongst High-Rises with Monocular Visual-Inertial SLAM Maps}, BOOKTITLE = {American Control Conference}. YEAR = {2023}}

We present UrbanFly: an uncertainty-aware real-time planning framework for quadrotor navigation in urban high-rise environments. A core aspect of UrbanFly is its ability to robustly plan directly on the sparse point clouds generated by a Monocular Visual Inertial SLAM (VINS) backend. It achieves this by using the sparse point clouds to build an uncertainty-integrated cuboid representation of the environment through a data-driven monocular plane segmentation network. Our chosen world model provides faster distance queries than the more common voxel-grid representation, and UrbanFly leverages this capability in two different ways leading to two trajectory optimizers. The first optimizer uses a gradient-free cross-entropy method to compute trajectories that minimize collision probability and smoothness cost. Our second optimizer is a simplified version of the first and uses a sequential convex programming optimizer initialized based on probabilistic safety estimates on a set of randomly drawn trajectories. Both our trajectory optimizers are made computationally tractable and independent of the nature of underlying uncertainty by embedding the distribution of collision violations in Reproducing Kernel Hilbert Space. Empowered by the algorithmic innovation, UrbanFly outperforms competing baselines in metrics such as collision rate, trajectory length, etc., on a high-fidelity AirSim simulator augmented with synthetic and real-world dataset scenes.

@inproceedings{bib_Hier_2023, AUTHOR = {Garg, Sourav and Dasu, Pradyumna and Krishna, K Madhava and Mehan, Yash and Sharma, Ayush }, TITLE = {Hierarchical Unsupervised Topological SLAM}, BOOKTITLE = {IEEE Transactions on Intelligent Transportation Systems}. YEAR = {2023}}

In this paper we present a novel framework for unsupervised topological clustering resulting in improved loop detection and closure for SLAM. A navigating mobile robot clusters its traversal into visually similar topologies where each cluster (topology) contains a set of similar looking images typically observed from spatially adjacent locations. Each such set of spatially adjacent and visually similar grouping of images constitutes a topology obtained without any supervision. We formulate a hierarchical loop discovery strategy that first detects loops at the level of topologies and subsequently at the level of images between the looped topologies. We show over a number of traversals across different Habitat environments that such a hierarchical pipeline significantly improves SOTA image based loop detection and closure methods. Further, as a consequence of improved loop detection, we enhance the loop closure and backend SLAM performance. Such a rendering of a traversal into topological segments is beneficial for downstream tasks such as navigation that can now build a topological graph where spatially adjacent topological clusters are connected by an edge and navigate over such topological graphs.

@inproceedings{bib_SCAR_2023, AUTHOR = {Sen, Bipasha and Agarwal, Aditya and Singh, Gaurav and B., Brojeshwar and Sridhar, Srinath and Krishna, K Madhava }, TITLE = {SCARP: 3D Shape Completion in ARbitrary Poses for Improved Grasping}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2023}}

Recovering full 3D shapes from partial observations is a challenging task that has been extensively addressed in the computer vision community. Many deep learning methods tackle this problem by training 3D shape generation networks to learn a prior over the full 3D shapes. In this training regime, the methods expect the inputs to be in a fixed canonical form, without which they fail to learn a valid prior over the 3D shapes. We propose SCARP, a model that performs Shape C ompletion in ARbitrary Poses. Given a partial pointcloud of an object, SCARP learns a disentangled feature representation of pose and shape by relying on rotationally equivariant pose features and geometric shape features trained using a multi-tasking objective. Unlike existing methods that depend on an external canonicalization method, SCARP performs canonicalization, pose estimation, and shape completion in a single network, improving the performance by 45% over the existing baselines. In this work, we use SCARP for improving grasp proposals on tabletop objects. By completing partial tabletop objects directly in their observed poses, SCARP enables a SOTA grasp proposal network improve their proposals by 71.2% on partial shapes. Project page: https://bipashasen.github.io/scarp

@inproceedings{bib_Sequ_2023, AUTHOR = {Nandiraju, Gireesh and Datta, Ahana and Agrawal, Ayush and Banerjee, Snehasis and Sridharan, Mohan and Bhowmick, Brojeshwar and Krishna, K Madhava }, TITLE = {Sequence-Agnostic Multi-Object Navigation}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2023}}

This paper focuses on the Multi-Object Navigation (MultiON) task, which requires a robot to localize an instance (each) of multiple object classes. This is a fundamental task for an assistive robot in a home or a factory. Existing methods for this task have viewed this as a direct extension of Object Navigation (ON), the task of localising a single instance of an object class, and are pre-sequenced, i.e., the sequence in which the object classes are to be explored is provided in advance. This is a strong limitation in practical applications characterized by dynamic changes. On the other hand, our deep reinforcement learning framework for sequence-agnostic MultiON is based on an actor-critic architecture and a reward specification. It exploits past experiences and seeks to reward progress towards individual as well as multiple target object classes. We use photo-realistic scenes from Gibson benchmark dataset in the AI Habitat 3D simulation environment to experimentally show that our method performs better than a pre-sequenced approach and a state of the art ON method extended to MultiON.

@inproceedings{bib_Lear_2023, AUTHOR = {Thakur, Prajwal and Qureshi, Mohammad Nomaan and Singh, Arun Kumar and Harish, Y V S and Katara, Pushkal and Masnavi, Houman and Krishna, K Madhava and Bhowmick, Brojeshwar }, TITLE = {Learning Arc-Length Value Function for Fast Time-Optimal Pick and Place Sequence Planning and Execution}, BOOKTITLE = {International Joint Conference on Neural Networks}. YEAR = {2023}}

This paper presents a real-time algorithm for computing the optimal sequence and motion plans for a fixed-base manipulator to pick and place a set of given objects. The optimality is defined in terms of the total execution time of the sequence or its proxy, the arc-length in the joint-space. The fundamental complexity stems from the fact that the optimality metric depends on the joint motion, but the task specification is in the end-effector space. Moreover, mapping between a pair of end-effector positions to the shortest arc-length joint trajectory is not analytic; instead, it entails solving a complex trajectory optimization problem. Existing works ignore this complex mapping and use the Euclidean distance in the end-effector space to compute the sequence. In this paper, we overcome the reliance on the Euclidean distance heuristic by introducing a novel data-driven technique to estimate the optimal arc-length cost in joint …

@inproceedings{bib_Cano_2023, AUTHOR = {Rohith, Agaram and Dewan, Shaurya Rajat and Sajnani, Rahul and Poulenard, Adrien and Krishna, K Madhava and Sridhar, Srinath }, TITLE = {Canonical Fields: Self-Supervised Learning of Pose-Canonicalized Neural Fields}, BOOKTITLE = {Computer Vision and Pattern Recognition}. YEAR = {2023}}

Coordinate-based implicit neural networks, or neural fields, have emerged as useful representations of shape and appearance in 3D computer vision. Despite advances, however, it remains challenging to build neural fields for categories of objects without datasets like ShapeNet that provide “canonicalized” object instances that are consistently aligned for their 3D position and orientation (pose).We present Canonical Field Network (CaFi-Net), a self-supervised method to canonicalize the 3D pose of instances from an object category represented as neural fields, specifically neural radiance fields (NeRFs). CaFi-Net directly learns from continuous and noisy radiance fields using a Siamese network architecture that is designed to extract equivariant field features for category-level canonicalization. During inference, our method takes pre-trained neural radiance fields of novel object instances at arbitrary 3D pose and estimates a canonical field with consistent 3D pose across the entire category. Extensive experiments on a new dataset of 1300 NeRF models across 13 object categories show that our method matches or exceeds the performance of 3D point cloud-based methods.

@inproceedings{bib_UAP-_2023, AUTHOR = {Dewangan, Vikrant and Sharma, Basant and Choudhary, Tushar and SHARMA, SARTHAK and Aanegola, Aakash and Singh, Arun K. and Krishna, K Madhava }, TITLE = {UAP-BEV: Uncertainty Aware Planning using Bird’s Eye View Generated From Surround Monocular Images}, BOOKTITLE = {Technical Report}. YEAR = {2023}}

Autonomous driving requires accurate reasoning of the location of objects from raw sensor data. Recent end-to-end learning methods go from raw sensor data to a trajectory output via Bird’s Eye View(BEV) segmentation as an interpretable intermediate representation. Motion planning over cost maps generated via Birds Eye View (BEV) segmen- tation has emerged as a prominent approach in autonomous driving. However, the current approaches have two critical gaps. First, the optimization process is simplistic and involves just evaluating a fixed set of trajectories over the cost map. The trajectory samples are not adapted based on their associated cost values. Second, the existing cost maps do not account for the uncertainty in the cost maps that can arise due to noise in RGB images, BEV annotations. As a result, these approaches can struggle in challenging scenarios where there is abrupt cut-in, stopping, overtaking, merging, etc from the neighboring vehicles. In this paper, we propose UAP-BEV, a novel approach that models the noise in Spatio-Temporal BEV predictions to create an uncertainty-aware occupancy grid map. Using queries of the distance to the closest occupied cell, we obtain a sample estimate of the collision probability of the ego-vehicle. Subsequently, our approach uses gradient-free sampling-based optimization to compute low-cost trajectories over the cost map

@inproceedings{bib_Inst_2023, AUTHOR = {Nanwani, Laksh and Mathur, Aditya and Agarwal, Anmol and Jain, Kanishk and Prabhakar, Raghav and Monis, Aaron Anthony and Murthy, Krishna and Hafez, Abdul and Gandhi, Vineet and Krishna, K Madhava }, TITLE = {Instance-Level Semantic Maps for Vision Language Navigation}, BOOKTITLE = {Technical Report}. YEAR = {2023}}

Humans have a natural ability to perform semantic associations with the surrounding objects in the environment. This allows them to create a mental map of the environment which helps them to navigate on-demand when given a linguistic instruction. A natural goal in Vision Language Navigation (VLN) research is to impart autonomous agents with similar capabilities. Recently introduced VL Maps \cite{huang23vlmaps} take a step towards this goal by creating a semantic spatial map representation of the environment without any labelled data. However, their representations are limited for practical applicability as they do not distinguish between different instances of the same object. In this work, we address this limitation by integrating instance-level information into spatial map representation using a community detection algorithm and by utilizing word ontology learned by large language models (LLMs) to perform open-set semantic associations in the mapping representation. The resulting map representation improves the navigation performance by two-fold (233\%) on realistic language commands with instance-specific descriptions compared to VL Maps. We validate the practicality and effectiveness of our approach through extensive qualitative and quantitative experiments.

@inproceedings{bib_A_No_2023, AUTHOR = {Govindan, Nagamanikandan and Ramachandran, Bharadhwaj and Sai, Pasala Haasith Venkata and Krishna, K Madhava }, TITLE = {A Novel Hybrid Gripper Capable of Grasping and Throwing Manipulation}, BOOKTITLE = {IEEE/ASME Transactions on Mechatronics}. YEAR = {2023}}

Throwing motion is known for phenomenally fast rearrangement, sorting tasks, and placing the object outside the limited workspace with less effort. However, in the robotics domain, despite many simple yet versatile, mechanically intelligent grippers reported earlier, they fo- cus primarily on achieving robust grasping and dexterous manipulation. This article presents a novel design of a sin- gle actuator driven hybrid gripper with mechanically cou- pled rigid links and elastic gripping surface; this arrange- ment provides the dual function of versatile grasping and throwing manipulation. The gripper comprises a latching mechanism (LM) that drives two passive rigid fingers by elongating/releasing the coupled elastic strip. Elongating the gripping surface enables the gripper to adapt to ob- jects with different geometries, vary surface contact force characteristics, and store the energy in the form of elastic potential. A mechanism to discharge the stored potential energy gradually or instantaneously is essential when the intended task is to place the object free from impact or away from the limited reachable workspace. The proposed LM can swiftly shift from a quick release to a gradual release of the stored elastic potential for greater object’s accelera- tion during throwing and no acceleration while placing. By doing so, the object can be placed at the desired location even farther than the manipulator’s reachable workspace. We report the proposed gripper’s design details, develop- ment, and experimentally demonstrate the versatile grasp- ing, impact-free placing, and throwing capabilities. Index Terms—Gripper design, multipurpose gripper, non- prehensile manipulation, throwing.

@inproceedings{bib_UAP-_2023, AUTHOR = {Dewangan, Vikrant and Sharma, Basant and Choudhary, Tushar and SHARMA, SARTHAK and Aanegola, Aakash and Singh, Arun K. and Krishna, K Madhava }, TITLE = {UAP-BEV: Uncertainty Aware Planning using Bird’s Eye View Generated From Surround Monocular Images}, BOOKTITLE = {International Conference on Automation Science and Engineering}. YEAR = {2023}}

Autonomous driving requires accurate reasoning of the location of objects from raw sensor data. Recent end-to-end learning methods go from raw sensor data to a trajectory output via Bird’s Eye View(BEV) segmentation as an interpretable intermediate representation. Motion planning over cost maps generated via Birds Eye View (BEV) segmentation has emerged as a prominent approach in autonomous driving. However, the current approaches have two critical gaps. First, the optimization process is simplistic and involves just evaluating a fixed set of trajectories over the cost map. The trajectory samples are not adapted based on their associated cost values. Second, the existing cost maps do not account for the uncertainty in the cost maps that can arise due to noise in RGB images, BEV annotations etc. As a result, these approaches can struggle in challenging scenarios where there is abrupt cutin, stopping, overtaking, merging, etc from the neighbouring vehicles. In this paper, we propose UAP-BEV, a novel approach that models the noise in Spatio-Temporal BEV predictions to create an uncertainty-aware occupancy grid map. Using queries of the distance to the closest occupied cell, we obtain a sample estimate of the collision probability of the ego-vehicle. Subsequently, our approach uses gradient-free sampling-based optimization to compute low-cost trajectories over the cost

@inproceedings{bib_Real_2023, AUTHOR = {Singh, Jaskirat and Adwani, Neel and K, Harikumar and Krishna, K Madhava }, TITLE = {Real-Time Heuristic Framework for Safe Landing of UAVs in Dynamic Scenarios}, BOOKTITLE = {International conference on Unmanned Aircraft Systems}. YEAR = {2023}}

The world we live in is full of technology and with each passing day the advancement and usage of UAVs increases efficiently. As a result of the many application scenarios, there are some missions where the UAVs are vulnerable to external disruptions, such as a ground station's loss of connectivity, security missions, safety concerns, and delivery-related missions. Therefore, depending on the scenario, this could affect the operations and result in the safe landing of UAVs. Hence, this paper presents a heuristic approach towards safe landing of multi-rotor UAVs in the dynamic environments. The aim of this approach is to detect safe potential landing zones - PLZ, and find out the best one to land in. The PLZ is initially, detected by processing an image through the canny edge algorithm, and then the diameter-area estimation is applied for each region with minimal edges. The spots that have a higher area than the vehicle's clearance are labeled as safe PLZ. Onto the second phase of this approach, the velocities of dynamic obstacles that are moving towards the PLZs are calculated and their time to reach the zones are taken into consideration. The ETA of the UAV is calculated and during the descending of UAV, the dynamic obstacle avoidance is executed. The approach tested on the real-world environments have shown better results from existing work.

@inproceedings{bib_Grou_2023, AUTHOR = {Jain, Kanishk and Chhangani, Varun and Tiwari, Amogh and Krishna, K Madhava and Gandhi, Vineet }, TITLE = {Ground then Navigate: Language-guided Navigation in Dynamic Scenes}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2023}}

We investigate the Vision-and-Language Navigation (VLN) problem in the context of autonomous driving in outdoor settings. We solve the problem by explicitly grounding the navigable regions corresponding to the textual command. At each timestamp, the model predicts a segmentation mask corresponding to the intermediate or the final navigable region. Our work contrasts with existing efforts in VLN, which pose this task as a node selection problem, given a discrete connected graph corresponding to the environment. We do not assume the availability of such a discretised map. Our work moves towards continuity in action space, provides interpretability through visual feedback and allows VLN on commands requiring finer manoeuvres like "park between the two cars". Furthermore, we propose a novel meta-dataset CARLA-NAV to allow efficient training and validation. The dataset comprises pre-recorded training sequences and a live environment for validation and testing. We provide extensive qualitative and quantitive empirical results to validate the efficacy of the proposed approach.

@inproceedings{bib_GDIP_2023, AUTHOR = {Hemant, Kalwar Sanket and Patel, Dhruv and Aanegola, Aakash and Konda, Krishna Reddy and Garg, Sourav and Krishna, K Madhava }, TITLE = {GDIP: Gated Differentiable Image Processing for Object-Detection in Adverse Conditions}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2023}}

Detecting objects under adverse weather and lighting conditions is crucial for the safe and continuous operation of an autonomous vehicle, and remains an unsolved problem. We present a Gated Differentiable Image Processing (GDIP) block, a domain-agnostic network architecture, which can be plugged into existing object detection networks (e.g., Yolo) and trained end-to-end with adverse condition images such as those captured under fog and low lighting. Our proposed GDIP block learns to enhance images directly through the downstream object detection loss. This is achieved by learning parameters of multiple image pre-processing (IP) techniques that operate concurrently, with their outputs combined using weights learned through a novel gating mechanism. We further improve GDIP through a multi-stage guidance procedure for progressive image enhancement. Finally, trading off accuracy for speed, we propose a variant of GDIP that can be used as a regularizer for training Yolo, which eliminates the need for GDIP-based image enhancement during inference, resulting in higher throughput and plausible real-world deployment. We demonstrate significant improvement in detection performance over several state-of-the-art methods through quantitative and qualitative studies on synthetic datasets such as PascalVOC, and real-world foggy (RTTS) and low-lighting (ExDark) datasets.

@inproceedings{bib_Mult_2022, AUTHOR = {Adajania, Vivek K. and Sharma, M Aditya and Gupta, Anish and Masnavi, Houman and Krishna, K Madhava and Singh, Arun Kumar }, TITLE = {Multi-Modal Model Predictive Control through Batch Non-Holonomic Trajectory Optimization: Application to Highway Driving}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2022}}

Standard Model Predictive Control (MPC) or trajectory optimization approaches perform only a local search to solve a complex non-convex optimization problem. As a result, they cannot capture the multi-modal characteristic of human driving. A global optimizer can be a potential solution but is computationally intractable in a real-time setting. In this paper, we present a real-time MPC capable of searching over different driving modalities. Our basic idea is simple: we run several goaldirected parallel trajectory optimizations and score the resulting trajectories based on user-defined meta cost functions. This allows us to perform a search over several locally optimal motion plans. Although conceptually straightforward, realizing this idea in real-time with existing optimizers is highly challenging from technical and computational standpoints. With this motivation, we present a novel batch non-holonomic trajectory optimization whose underlying matrix algebra is easily parallelizable across problem instances and reduces to computing large batch matrix-vector products. This structure, in turn, is achieved by deriving a linearization-free multi-convex reformulation of the non-holonomic kinematics and collision avoidance constraints. We extensively validate our approach using both synthetic and real data sets (NGSIM) of traffic scenarios. We highlight how our algorithm automatically takes lane-change and overtaking decisions based on the defined meta cost function. Our batch optimizer achieves trajectories with lower meta cost, up to 6x faster than competing baselines.

@inproceedings{bib_Spat_2022, AUTHOR = {Kiran, Dharmala Amarthya Sasi and Krishna, K Madhava and Anand, Kritika and Kharyal, Chaitanya and KUMAR, GULSHAN and Gireesh, Nandiraju and Banerjee, Snehasis and Roychoudhury, Ruddra Dev and Sridharan, Mohan and Bhowmick, Brojeshwar }, TITLE = {Spatial Relation Graph and Graph Convolutional Network for Object Goal Navigation}, BOOKTITLE = {International Conference on Automation Science and Engineering}. YEAR = {2022}}

This paper describes a framework for the object-goal navigation (ObjectNav) task, which requires a robot to find and move to an instance of a target object class from a random starting position. The framework uses a history of robot trajectories to learn a Spatial Relational Graph (SRG) and Graph Convolutional Network (GCN)-based embeddings for the likelihood of proximity of different semantically-labeled regions and the occurrence of different object classes in these regions. To locate a target object instance during evaluation, the robot uses Bayesian inference and the SRG to estimate the visible regions, and uses the learned GCN embeddings to rank visible regions and select the region to explore next. This approach is tested using the Matterport3D (MP3D) benchmark dataset of indoor scenes in AI Habitat, a visually realistic simulation environment, to report substantial performance improvement in comparison with state of the art baselines.

@inproceedings{bib_Obje_2022, AUTHOR = {Gireesh, Nandiraju and Kiran, Dharmala Amarthya Sasi and Banerjee, Snehasis and Sridharan, Mohan and Bhowmick, Brojeshwar and Krishna, K Madhava }, TITLE = {Object Goal Navigation using Data Regularized Q-Learning}, BOOKTITLE = {International Conference on Automation Science and Engineering}. YEAR = {2022}}

Object Goal Navigation requires a robot to find and navigate to an instance of a target object class in a previously unseen environment. Our framework incrementally builds a semantic map of the environment over time, and then repeatedly selects a long-term goal (’where to go’) based on the semantic map to locate the target object instance. Long-term goal selection is formulated as a vision-based deep reinforcement learning problem. Specifically, an Encoder Network is trained to extract high-level features from a semantic map and select a long-term goal. In addition, we incorporate data augmentation and Q-function regularization to make the long-term goal selection more effective. We report experimental results using the photo-realistic Gibson benchmark dataset in the AI Habitat 3D simulation environment to demonstrate substantial performance improvement on standard measures in comparison with a state of the art data-driven baseline.

@inproceedings{bib_Solv_2022, AUTHOR = {GOPALAKRISHNAN, BHARATH and Singh, Arun Kumar and Krishna, K Madhava and Manocha, Dinesh }, TITLE = {Solving Chance-Constrained Optimization Under Nonparametric Uncertainty Through Hilbert Space Embedding}, BOOKTITLE = {IEEE Transactions on Control Systems Technology}. YEAR = {2022}}

In this article, we present an efficient algorithm for solving a class of chance-constrained optimization under nonparametric uncertainty. Our algorithm is built on the possibility of representing arbitrary distributions as functions in Reproducing Kernel Hilbert Space (RKHS). We use this foundation to formulate chance-constrained optimization as one of minimizing the distance between a desired distribution and the distribution of the constraint functions in the RKHS. We provide a systematic way of constructing the desired distribution based on the notion of scenario approximation. Furthermore, we use the kernel trick to show that the computational complexity of our reformulated optimization problem is comparable to solving a deterministic variant of the chance-constrained optimization. We validate our formulation on two important robotic applications: 1) reactive collision avoidance of mobile robots in uncertain dynamic environments and 2) inverse-dynamics-based path-tracking of manipulators under perception uncertainty. In both these applications, the underlying chance constraints are defined over nonlinear and nonconvex functions of uncertain parameters and possibly also decision variables. We also benchmark our formulation with the existing approaches in terms of sample complexity and the achieved optimal cost highlighting significant improvements in

@inproceedings{bib_Fast_2022, AUTHOR = {Guhathakurta, Dipanwita and Rastgar, Fatemeh and Krishna, K Madhava and Sharma, M Aditya }, TITLE = {Fast Joint Multi-Robot Trajectory Optimization by GPU Accelerated Batch Solution of Distributed Sub-Problems}, BOOKTITLE = {Frontiers in Robotics and AI}. YEAR = {2022}}

We present a joint multi-robot trajectory optimizer that can compute trajectories for tens of robots in aerial swarms within a small fraction of a second. The computational efficiency of our approach is built on breaking the per-iteration computation of the joint optimization into smaller, decoupled sub-problems and solving them in parallel through a custom batch optimizer. We show that each of the sub-problems can be reformulated to have a special Quadratic Programming structure, wherein the matrices are shared across all the problems and only the associated vector varies. As a result, the batch solution update rule reduces to computing just large matrix-vector products which can be trivially accelerated using GPUs. We validate our optimizer’s performance in difficult benchmark scenarios and compare it against existing state-of-the-art approaches. We demonstrate remarkable improvements in computation time its scaling with respect to the number of robots. Moreover, we also perform better in trajectory quality as measured by smoothness and arc-length metrics.

@inproceedings{bib_Leve_2022, AUTHOR = {Gupta, Anish and Singh, Arun Kumar and Krishna, K Madhava }, TITLE = {Leveraging Distributional Bias For Reactive Collision Avoidance under Uncertainty: A Kernel Embedding Approach}, BOOKTITLE = {International Conference on Automation Science and Engineering}. YEAR = {2022}}

Many commodity sensors that measure the robot and dynamic obstacle’s state have non-Gaussian noise characteristics. Yet, many current approaches treat the underlying uncertainty in motion and perception as Gaussian, primarily to ensure computational tractability. On the other hand, existing planners working with non-Gaussian uncertainty do not shed light on leveraging distributional characteristics of motion and perception noise, such as bias for efficient collision avoidance.This paper fills this gap by interpreting reactive collision avoidance as a distribution matching problem between the collision constraint violations and Dirac Delta distribution. To ensure fast reactivity in the planner, we embed each distribution in Reproducing Kernel Hilbert Space and reformulate the distribution matching as minimizing the Maximum Mean Discrepancy (MMD) between the two distributions. We show that evaluating the MMD …

@inproceedings{bib_Drif_2022, AUTHOR = {Omama, Mohd and S., Sundar Sripada V. and Chinchali, Sandeep and Singh, Arun Kumar and Krishna, K Madhava }, TITLE = {Drift Reduced Navigation with Deep Explainable Features}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2022}}

Modern autonomous vehicles (AVs) often rely on vision, LIDAR, and even radar-based simultaneous localization and mapping (SLAM) frameworks for precise localization and navigation. However, modern SLAM frameworks often lead to unacceptably high levels of drift (i.e., localization error) when AVs observe few visually distinct features or encounter occlusions due to dynamic obstacles. This paper argues that minimizing drift must be a key desiderata in AV motion planning, which requires an AV to take active control decisions to move towards feature-rich regions while also minimizing conventional control cost. To do so, we first introduce a novel data-driven perception module that observes LIDAR point clouds and estimates which features/regions an AV must navigate towards for drift minimization. Then, we introduce an interpretable model predictive controller (MPC) that moves an AV toward such feature-rich regions while avoiding visual occlusions and gracefully trading off drift and control cost. Our experiments on challenging, dynamic scenarios in the state-ofthe-art CARLA simulator indicate our method redu

@inproceedings{bib_MVRa_2022, AUTHOR = {Pathre, Pranjali Pramod and Sahu, Anurag and Rao, Ashwin and Prabhu, Puppala Avinash and Nigam, Meher Shashwat and Karandikar, Tanvi and PANDYA, HARIT and Krishna, K Madhava }, TITLE = {MVRackLay: Monocular Multi-View Layout Estimation for Warehouse Racks and Shelves}, BOOKTITLE = {International Conference on Robotics and Biomimetics}. YEAR = {2022}}

In this paper, we propose and showcase, for the first time, monocular multi-view layout estimation for warehouse racks and shelves. Unlike typical layout estimation methods, MVRackLay estimates multi-layered layouts, wherein each layer corresponds to the layout of a shelf within a rack. Given a sequence of images of a warehouse scene, a dualheaded Convolutional-LSTM architecture outputs segmented racks, the front and the top view layout of each shelf within a rack. With minimal effort, such an output is transformed into a 3D rendering of all racks, shelves and objects on the shelves, giving an accurate 3D depiction of the entire warehouse scene in terms of racks, shelves and the number of objects on each shelf. MVRackLay generalizes to a diverse set of warehouse scenes with varying number of objects on each shelf, number of shelves and in the presence of other such racks in the background. Further, MVRackLay shows superior performance vis-a-vis its single view counterpart, RackLay [1] in layout accuracy, quantized in terms of the mean IoU and mAP metrics. We also showcase a multi-view stitching of the 3D layouts resulting in a representation of the warehouse scene with respect to a global reference frame akin to a rendering of the scene from a SLAM pipeline. To the best of our knowledge, this is the first such work to portray a 3D rendering of a warehouse scene in terms of its semantic components - Racks, Shelves and Objects - all from a single monocular camera.

@inproceedings{bib_UAV-_2022, AUTHOR = {Srivastava, Kushagra and Srivastava, Kushagra and Jha, Aditya Kumar and Jha, Mohhit Kumar and Singh, Jaskirat and Kiran, Santosh Ravi and Ramancharla, Pradeep Kumar and K, Harikumar and Krishna, K Madhava }, TITLE = {UAV-based Visual Remote Sensing for Automated Building Inspection}, BOOKTITLE = {European Conference on Computer Vision Workshops}. YEAR = {2022}}

Unmanned Aerial Vehicle (UAV) based remote sensing system incorporated with computer vision has demonstrated potential for assisting building construction and in disaster management like damage assessment during earthquakes. The vulnerability of a building to earthquake can be assessed through inspection that takes into account the expected damage progression of the associated component and the component’s contribution to structural system performance. Most of these inspections are done manually, leading to high utilization of manpower, time, and cost. This paper proposes a methodology to automate these inspections through UAV-based image data collection and a software library for post-processing that helps in estimating the seismic structural parameters. The key parameters considered here are the distances between adjacent buildings, building plan-shape, building plan area, objects on the rooftop and rooftop layout. The accuracy of the proposed methodology in estimating the above-mentioned parameters is verified through field measurements taken using a distance measuring sensor and also from the data obtained through Google Earth. Additional details and code can be accessed from h

@inproceedings{bib_Flow_2022, AUTHOR = {Sankhla, Harshit Kumar and Qureshi, Mohammad Nomaan and S, Viswa Narayanan and Mittal, Vedansh and Gupta, Gunjan and Pandya, Harit and Krishna, K Madhava }, TITLE = {Flow Synthesis Based Visual Servoing Frameworks for Monocular Obstacle Avoidance Amidst High-Rises}, BOOKTITLE = {International Conference on Automation Science and Engineering}. YEAR = {2022}}

We propose a novel flow synthesis based visual servoing framework enabling long-range obstacle avoidance for Micro Air Vehicles (MAV) flying amongst tall skyscrapers. Recent deep learning based frameworks use optical flow to do high-precision visual servoing. In this paper, we explore the question: can we design a surrogate flow for these high-precision visual-servoing methods, which leads to obstacle avoidance? We revisit the concept of saliency for identifying high-rise structures in/close to the line of attack amongst other competing skyscrapers and buildings as a collision obstacle. A synthesised flow is used to displace the salient object segmentation mask. This flow is so computed that the visual servoing controller maneuvers the MAV safely around the obstacle. In this approach, we use a multi-step Cross-Entropy Method (CEM) based servo control to achieve flow convergence, resulting in obstacle avoidance. We use this novel pipeline to successfully and persistently maneuver high-rises and reach the goal in simulated and photo-realistic real-world scenes. We conduct extensive experimentation and compare our approach with optical flow and short-range depth-based obstacle avoidance methods to demonstrate the proposed framework’s merit. Additional Visualisation can be found at https://sites.google.com/view/monocular-obstacle/home

@inproceedings{bib_Drif_2022, AUTHOR = {Omama, Mohd and S., Sundar Sripada V. and Chinchali, Sandeep and Singh, Arun Kumar and Krishna, K Madhava }, TITLE = {Drift Reduced Navigation with Deep Explainable Features}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2022}}

Modern autonomous vehicles (AVs) often rely on vision, LIDAR, and even radar-based simultaneous localization and mapping (SLAM) frameworks for precise localization and navigation. However, modern SLAM frameworks often lead to unacceptably high levels of drift (i.e., localization error) when AVs observe few visually distinct features or encounter occlusions due to dynamic obstacles. This paper argues that minimizing drift must be a key desiderata in AV motion planning, which requires an AV to take active control decisions to move towards feature-rich regions while also minimizing conventional control cost. To do so, we first introduce a novel data-driven perception module that observes LIDAR point clouds and estimates which features/regions an AV must navigate towards for drift minimization. Then, we introduce an interpretable model predictive controller (MPC) that moves an AV toward such feature-rich regions while avoiding visual occlusions and gracefully trading off drift and control cost. Our experiments on challenging, dynamic scenarios in the state-ofthe-art CARLA simulator indicate our method reduces drift up to 76.76% compared to benchmark approaches.

@inproceedings{bib_Indo_2022, AUTHOR = {Singh, Shantanu and Shriram, Jaidev and Kulkarni, Shaantanu S and Bhowmick, Brojeshwar and Krishna, K Madhava }, TITLE = {IndoLayout: Leveraging Attention for Extended Indoor Layout Estimation from an RGB Image}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2022}}

In this work, we propose IndoLayout, a novel real-time approach for generating high-quality occupancy maps from an RGB image for indoor scenes. Such occupancy maps are often crucial for path-planning and mapping in indoor environments but are often built using only information contained in the ego view. In contrast, our approach also predicts occupancy values beyond immediately visible regions from just a monocular image, leveraging learnt priors from indoor scenes. Hence, our proposed network can produce a hallucinated, amodal scene layout that includes areas occluded in the RGB image, such as a navigable floor behind a desk. Specifically, we propose a novel architecture that uses self-attention and adversarial learning to vastly improve the quality of the predicted layout. We evaluate our model on several photorealistic indoor datasets and outperform previous relevant work on all metrics that measure layout quality, including newly adopted ones. Finally, we demonstrate the effectiveness of our method by showing significant improvements on the PointNav task over similar approaches using IndoLayout. For more details, please refer to the project page: https://indolayout.github.io/.

@inproceedings{bib_Pred_2022, AUTHOR = {Khare, Ishaan and K, Harikumar and Krishna, K Madhava }, TITLE = {Predictive optimal collision avoidance for a swarm of fixed-wing aircraft in 3D space}, BOOKTITLE = {International conference on Unmanned Aircraft Systems}. YEAR = {2022}}

In this paper, we propose a predictive and cooperative optimal control based autonomous navigation and collision avoidance for multiple fixed-wing aircraft (FWA) that also takes into account the physical constraints of the aircraft. The proposed method is implemented in a cooperative framework, analogous to the Reciprocal Velocity Obstacle (RVO) framework for autonomous navigation and collision avoidance for FWA swarm moving in a three dimensional (3D) space. Also, the change in performance with respect to varying prediction horizon is analyzed through numerical simulations. As a result of predictive optimal control and cooperation, we get less conservative maneuvers, reduced path lengths, and less deviation from the shortest path when compared to the conventional method without prediction and cooperation. Simulation results are provided, highlighting the advantages of the proposed method when compared to a popular method (Optimal reciprocal collision avoidance) for collision avoidance in 3D for FWA swarm and also against FGA algorithm (Fast geometric avoidance algorithm).

@inproceedings{bib_Appr_2022, AUTHOR = {AGARWAL, ADITYA and Sen, Bipasha and V, Shankara Narayanan and Mandadi, Vishal Reddy and Bhowmick, Brojeshwar and Krishna, K Madhava }, TITLE = {Approaches and Challenges in Robotic Perception for Table-top Rearrangement and Planning}, BOOKTITLE = {Technical Report}. YEAR = {2022}}

Table-top Rearrangement and Planning is a challenging problem that relies heavily on an excellent perception stack. The perception stack involves observing and registering the 3D scene on the table, detecting what objects are on the table, and how to manipulate them. Consequently, it greatly influences the system’s task-planning and motion-planning stacks that follow. We present a comprehensive overview and discuss different challenges associated with the perception module. This work is a result of our extensive involvement in the ICRA 2022 Open Cloud Robot Table Organization Challenge, in which we currently stand first on the leaderboard (as of 24th of April 2022, the final week of the challenge).

@inproceedings{bib_LADF_2022, AUTHOR = {Omama, Mohd and S, Sundar Sripada V. and Chinchali, Sandeep and Krishna, K Madhava }, TITLE = {LADFN: Learning Actions for Drift-Free Navigation in Highly Dynamic Scenes}, BOOKTITLE = {American Control Conference}. YEAR = {2022}}

We embark on a hitherto unreported problem of an autonomous robot (self-driving car) navigating in dynamic scenes in a manner that reduces its localization error and eventual cumulative drift or Absolute Trajectory Error, which is pronounced in such dynamic scenes. With the hugely popular Velodyne-16 3D LIDAR as the main sensing modality, and the accurate LIDAR-based Localization and Mapping algorithm, LOAM, as the state estimation framework, we show that in the absence of a navigation policy, drift rapidly accumulates in the presence of moving objects. To overcome this, we learn actions that lead to drift-minimized navigation through a suitable set of reward and penalty functions. We use Proximal Policy Optimization, a class of Deep Reinforcement Learning methods, to learn the actions that result in drift-minimized trajectories. We show by extensive comparisons on a variety of synthetic, yet photo-realistic scenes made available through the CARLA Simulator the superior performance of the proposed framework vis-a-vis methods that do not adopt such policies.

@inproceedings{bib_ReF-_2022, AUTHOR = {Peri, Abhishek and Mehta, Kinal and Mishra, Avneesh and Milford, Michael and Garg, Sourav and Krishna, K Madhava }, TITLE = {ReF--Rotation Equivariant Features for Local Feature Matching}, BOOKTITLE = {Technical Report}. YEAR = {2022}}

— Sparse local feature matching is pivotal for many computer vision and robotics tasks. To improve their invariance to challenging appearance conditions and viewing angles, and hence their usefulness, existing learning-based methods have primarily focused on data augmentation-based training. In this work, we propose an alternative, complementary approach that centers on inducing bias in the model architecture itself to generate ‘rotation-specific’ features using Steerable E2-CNNs, that are then group-pooled to achieve rotation-invariant local features. We demonstrate that this high performance, rotationspecific coverage from the steerable CNNs can be expanded to all rotation angles by combining it with augmentationtrained standard CNNs which have broader coverage but are often inaccurate, thus creating a state-of-the-art rotation-robust local feature matcher. We benchmark our proposed methods against existing techniques on HPatches and a newly proposed UrbanScenes3D-Air dataset for visual place recognition. Furthermore, we present a detailed analysis of the performance effects of ensembling, robust estimation, network architecture variations, and the use of rotation priors

@inproceedings{bib_Drif_2022, AUTHOR = {Omama, Mohd and Sriraman, Sundar Sripada Venugopalaswamy and Chinchali, Sandeep and Singh, Arun Kumar and Krishna, K Madhava }, TITLE = {Drift Reduced Navigation with Deep Explainable Features}, BOOKTITLE = {Technical Report}. YEAR = {2022}}

Modern autonomous vehicles (AVs) often rely on vision, LIDAR, and even radar-based simultaneous localization and mapping (SLAM) frameworks for precise localization and navigation. However, modern SLAM frameworks often lead to unacceptably high levels of drift (i.e., localization error) when AVs observe few visually distinct features or encounter occlusions due to dynamic obstacles. This paper argues that minimizing drift must be a key desiderata in AV motion planning, which requires an AV to take active control decisions to move towards feature-rich regions while also minimizing conventional control cost. To do so, we first introduce a novel data-driven perception module that observes LIDAR point clouds and estimates which features/regions an AV must navigate towards for drift minimization. Then, we introduce an interpretable model predictive controller (MPC) that moves an AV toward such feature-rich regions while avoiding visual occlusions and gracefully trading off drift and control cost. Our experiments on challenging, dynamic scenarios in the state-ofthe-art CARLA simulator indicate our method reduces drift up to 76.76% compared to benchmark approaches

@inproceedings{bib_Mult_2022, AUTHOR = {Adajania, Vivek K. and Sharma, M Aditya and Gupta, Anish and Masnav, Houman and Krishna, K Madhava and K.Singh, Arun }, TITLE = {Multi-Modal Model Predictive Control through Batch Non-Holonomic Trajectory Optimization: Application to Highway Driving}, BOOKTITLE = {IEEE Robotics and Automation Letters}. YEAR = {2022}}

Standard Model Predictive Control (MPC) or trajectory optimization approaches perform only a local search to solve a complex non-convex optimization problem. As a result, they cannot capture the multi-modal characteristic of human driving. A global optimizer can be a potential solution but is computationally intractable in a real-time setting. In this paper, we present a real-time MPC capable of searching over different driving modalities. Our basic idea is simple: we run several goal-directed parallel trajectory optimizations and score the resulting trajectories based on user-defined meta cost functions. This allows us to perform a global search over several locally optimal motion plans. Although conceptually straightforward, realizing this idea in real-time with existing optimizers is highly challenging from technical and computational standpoints. With this motivation, we present a novel batch nonholonomic trajectory optimization whose underlying matrix algebra is easily parallelizable across problem instances and reduces to computing large batch matrix-vector products. This structure, in turn, is achieved by deriving a linearization-free multi-convex reformulation of the non-holonomic kinematics and collision avoidance constraints. We extensively validate our approach using both synthetic and real data sets (NGSIM) of traffic scenarios. We highlight how our algorithm automatically takes lane-change and overtaking decisions based on the defined meta cost function. Our batch optimizer achieves trajectories with lower meta cost, up to 6x faster than competing baselines.

@inproceedings{bib_Fast_2022, AUTHOR = {SRIKANTH, SHASHANK and Nallana, Mithun Babu and Masnavi, Houman and Singh, Arun Kumar and Kruusamäe, Karl and Krishna, K Madhava }, TITLE = {Fast Adaptation of Manipulator Trajectories to Task Perturbation By Differentiating through the Optimal Solution}, BOOKTITLE = {Sensors}. YEAR = {2022}}

Joint space trajectory optimization under end-effector task constraints leads to a challenging non-convex problem. Thus, a real-time adaptation of prior computed trajectories to perturbation in task constraints often becomes intractable. Existing works use the so-called warm-starting of trajectory optimization to improve computational performance. We present a fundamentally different approach that relies on deriving analytical gradients of the optimal solution with respect to the task constraint parameters. This gradient map characterizes the direction in which the prior computed joint trajectories need to be deformed to comply with the new task constraints. Subsequently, we develop an iterative line-search algorithm for computing the scale of deformation. Our algorithm provides near real-time adaptation of joint trajectories for a diverse class of task perturbations such as (i) changes in initial and final joint configurations of end-effector orientation-constrained trajectories and (ii) changes in end-effector goal or way-points under end-effector orientation constraints. We relate each of these examples to real-world applications ranging from learning from demonstration to obstacle avoidance. We also show that our algorithm produces trajectories with quality similar to what one would obtain by solving the trajectory optimization from scratch with warm-start initialization. But most importantly, our algorithm achieves a worst-case speed-up of 160x over the latter approach.

@inproceedings{bib_Mult_2021, AUTHOR = {Thatavarthy, Ayyappa Swamy and Sharma, Tanu and Sankhl, Harshit and Khanna, Mukul and Krishna, K Madhava }, TITLE = {Multi-view planarity constraints for skyline estimation from UAV images in city scale urban environments}, BOOKTITLE = {International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applicat}. YEAR = {2021}}

It is critical for aerial robots flying in city scale urban environments to make very quick estimates of a building depth with respect to itself. It should be done in a matter of few views to navigate itself, avoiding collisions with such a towering structure. As such, no one has attacked this problem. We bring together several modules combining deep learning and 3D vision to showcase a quick reconstruction in a few views. We exploit the inherent planar structure in the buildings (facades, windows) for this purpose. We evaluate the efficacy of our pipeline with various constraints and errors from multi-view geometry using ablation studies. We then retrieve the skyline of the buildings in synthetic as well as real-world scenes.

@inproceedings{bib_Buil_2021, AUTHOR = {Khanna, Mukul and Sharma, Tanu and Swamy, Thatavarthy VVSST Ayyappa and Krishna, K Madhava }, TITLE = {Building Facades to Normal Maps: Adversarial Learning from Single View Images}, BOOKTITLE = {Canadian Conference on Computer and Robot Vision}. YEAR = {2021}}

Surface normal estimation is an essential component of several computer and robot vision pipelines. While this problem has been extensively studied, most approaches are geared towards indoor scenes and often rely on multiple modalities (depth, multiple views) for accurate estimation of normal maps. Outdoor scenes pose a greater challenge as they exhibit significant lighting variation, often contain occluders, and structures like building facades are often ridden with numerous windows and protrusions. Conventional supervised learning schemes excel in indoor scenes, but do not exhibit competitive performance when trained and deployed in outdoor environments. Furthermore, they involve complex network architectures and require many more trainable parameters. To tackle these challenges, we present an adversarial learning scheme that regularizes the output normal maps from a neural network to appear more realistic, by using a small number of precisely annotated examples. Our method presents a lightweight and simpler architecture, while improving performance by at least 1.5x across most metrics. We evaluate our approaches against the state-of-the-art on normal map estimation, on a synthetic and a real outdoor dataset, and observe significant performance enhancements.

@inproceedings{bib_Foll_2021, AUTHOR = {Ashe, Avijit Kumar and Krishna, K Madhava }, TITLE = {Followman: Control of Social Person Following Robot}, BOOKTITLE = {International Conferene on Intelligent Transportation Systems}. YEAR = {2021}}

Exploring shared autonomy in assistive and service robots makes human-aware navigation of a person following robot (PFR) a required behaviour. In this paper, we propose a control framework for a non-holonomic wheeled robot that not only tracks a target person in sight but also anticipates the human movement patterns to predict the future sequence of its path when the person is out of sight. Human beings form a crowd and can exhibit complex random movement compared to vehicles while indoor environments with intersections can pose serious challenges for long-distance path-following without the breakdown of tracking. Thus, a nonlinear model predictive controller is designed with long-term prediction and socially compliant rules for natural person-following behaviour. It can generate the collision-free path and optimized control inputs using a single optimization framework Finally, the integrated navigation-control stack is evaluated using simulations for real-time operation. We also present a hardware configuration for its real-world implementation.

@inproceedings{bib_RTVS_2021, AUTHOR = {Qureshi, Mohammad Nomaan and Katara, Pushkal and GUPTA, ABHINAV and Pandya, Harit and Harish, Y V S and Sanchawala, Aadilmehdi J and Kumar, Gourav and Bhowmick, Brojeshwar and Krishna, K Madhava }, TITLE = {RTVS: A Lightweight Differentiable MPC Framework for Real-Time Visual Servoing}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2021}}

Recent data-driven approaches to visual servoing have shown improved performances over classical methods due to precise feature matching and depth estimation. Some recent servoing approaches use a model predictive control (MPC) framework which generalise well to novel environments and are capable of incorporating dynamic constraints, but are computationally intractable in real-time, making it difficult to deploy in real-world scenarios. On the contrary, single-step methods optimise greedily and achieve high servoing rates, but lack the benefits of the MPC multi-step ahead formulation. In this paper, we make the best of both worlds and propose a lightweight visual servoing MPC framework which generates optimal control near real-time at a frequency of 10.52 Hz. This work utilises the differential cross-entropy sampling method for quick and effective control generation along with a lightweight neural network, significantly improving the servoing frequency. We also propose a flow depth normalisation layer which amelio- rates the issue of inferior predictions of two view depth from the flow network. We conduct extensive experimentation on the Habitat simulator and show a notable decrease in servoing time in comparison with other approaches that optimise over a time horizon. We achieve the right balance between time and performance for visual servoing in six degrees of freedom (6DoF), while retaining the advantageous MPC formulation. Our code and dataset are publicly available

@inproceedings{bib_Inco_2021, AUTHOR = {MALI, PRAVIN and K, Harikumar and Singh, Arun Kumar and Krishna, K Madhava and Sujit, P.B. }, TITLE = {Incorporating Prediction in Control Barrier Function Based Distributive Multi-Robot Collision Avoidance}, BOOKTITLE = {European Control Conference}. YEAR = {2021}}

Control barrier function (CBF) constraints provide a rigorous characterization of the space of control inputs that ensure the satisfaction of state constraints, such as collision avoidance, at all time instants. However, CBFs are highly nonlinear and non-convex and thus, when incorporated within an optimization-based algorithm such as Model Predictive Control (MPC), leads to a computationally challenging problem. Existing works by-pass the computational intractability by collapsing the horizon of the MPC to a single step, although this comes at the cost of severe degradation of performance. In this paper, we present two contributions to ensure the real-time performance of CBFs based MPC over long horizons in the context of multi-robot collision avoidance. First, we propose a customized Project Gradient Descent Method that incurs minimal computational overhead over existing one-step approaches but leads to a substantial improvement in trajectory smoothness, time to reach the goal, etc. Our second contribution lies in applying the proposed MPC to both quadrotors and fixed-wing aerial aircrafts (FWA). In particular, we show that the formulation for the quadrotors can be readily extended to the latter by deriving additional CBFs for the curvature and forward velocity constraints. We validate our algorithm with an extensive simulation of up to 10 robots in challenging benchmark scenarios.

@inproceedings{bib_Modu_2021, AUTHOR = {Adithya, Vadapalli Sreerama and Agarwal, Saharsh and KUMAR, N VISHNU and Suryavanshi, Kartik and Govindan, Nagamanikandan and Krishna, K Madhava }, TITLE = {Modular pipe climber iii with three-output open differential}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2021}}

The paper introduces the novel Modular Pipe Climber III with a Three-Output Open Differential (3-OOD) mechanism to eliminate slipping of the tracks due to the changing cross-sections of the pipe. This will be achieved in any orientation of the robot. Previous pipe climbers use three-wheel/track modules, each with an individual driving mechanism to achieve stable traversing. Slipping of tracks is prevalent in such robots when it encounters the pipe turns. Thus, active control of each module’s speed is employed to mitigate the slip, thereby requiring substantial control effort. The proposed pipe climber implements the 3-OOD to address this issue by allowing the robot to mechanically modulate the track speeds as it encounters a turn. The proposed 3-OOD is the first three-output differential to realize the functional abilities of a traditional two-output differential.

@inproceedings{bib_Prob_2021, AUTHOR = {Khare, Ishaan and Jyotish, Poonganam Sri Sai Naga and GOPALAKRISHNAN, BHARATH and Krishna, K Madhava }, TITLE = {Probabilistic Inverse Velocity Obstacle for Free Flying Quadrotors}, BOOKTITLE = {European Control Conference}. YEAR = {2021}}

In this paper, we explore Probabilistic Inverse Velocity Obstacle (PIVO) as an alternative to probabilistic versions of Velocity Obstacles (PVO) for free flying quadrotor systems. Inverse Velocity Obstacles compute effective controls from a sequence of observations on other agents without the need to access ego state information. As a direct consequence of this the ego state noise is not entailed in probabilistic formulations bringing in verifiable advantages in the form of reduced path lengths, less conservative maneuvers, reduced occurrences of stopping/hovering to let others pass. These advantages are vividly tabulated in this paper, showcasing the efficacy of PIVO as an alternative to probabilistic versions of Velocity Obstacles. In particular we show the benefits of PIVO over PVO in relation to sample complexity as well as overall trajectory lengths. We also show the efficacy of our probabilistic formulation in handling non-parametric and often multimodal noise distributions.

@inproceedings{bib_DESI_2021, AUTHOR = {Govindan, Nagamanikandan and Vadapalli, Rama and Krishna, K Madhava }, TITLE = {DESIGN AND ANALYSIS OF THREE-OUTPUT OPEN DIFFERENTIAL WITH 3-DOF}, BOOKTITLE = {Technical Report}. YEAR = {2021}}

This paper presents a novel passive three-output differential with three degrees of freedom (3DOF), that translates motion and torque from a single input to three outputs. The proposed ThreeOutput Open Differential is designed such that its functioning is analogous to the functioning of a traditional two-output open differential. That is, the differential translates equal motion and torque to all its three outputs when the outputs are unconstrained or are subjected to equivalent load conditions. The introduced design is the first differential with three outputs to realise this outcome. The differential action between the three outputs is realised passively by a symmetric arrangement of three two-output open differentials and three two-input open differentials. The resulting differential mechanism achieves the novel result of equivalent input to output angular velocity and torque relations for all its three outputs. Furthermore, Three-Output Open Differential achieves the novel result for differentials with more than two outputs where each of its outputs shares equivalent angular velocity and torque relations with all the other outputs. The kinematics and dynamics of the Three-Output Open Differential are derived using the bond graph method. In addition, the merits of the differential mechanism along with its current and potential applications are presented.

@inproceedings{bib_A_ne_2021, AUTHOR = {Swamy, Thatavarthy Vvsst Ayyappa and Sharma, Tanu and Krishna, K Madhava }, TITLE = {A new geometric approach for three view line reconstruction and motion estimation in Manhattan Scenes}, BOOKTITLE = {Canadian Conference on Computer and Robot Vision}. YEAR = {2021}}

Owing to the inherent geometry, the extent of map reconstructed using line-based SfM(structure from motion) is superior to point-based SfM. However, with the existing methods, estimation of structure and motion from observed 2D line segments in images is more complex than that of points. To overcome this, we propose a simple and robust 1-parameter approach for Structure and Motion Estimation from line features in Manhattan Scenes from three views. We leverage the vanishing point directions to estimate the relative rotations as well as to fix the 3D line direction. In consequence we build a constraints matrix, which has the relative translations and 3D line depth as its null space. We then perform 1-parameter line BA using factor graph based cost function. We compare the efficacy of our method with standard line triangulation in synthetic as well as real-world scenes. Keywords-manhattan scenes, line features, Structure and motion estimation, vanishing points, vanishing line, Plucker coordinates, orthonormal representation, factor graph optimization

@inproceedings{bib_Mult_2021, AUTHOR = {Swamy, Thatavarthy Vvsst Ayyappa and Sharma, Tanu and Sankhla, Harshit Kumar and Khanna, Mukul and Krishna, K Madhava }, TITLE = {Multi-View Planarity Constraints for skyline estimation from UAV images in city scale urban environments}, BOOKTITLE = {International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applicat}. YEAR = {2021}}

It is critical for aerial robots flying in city scale urban environments to make very quick estimates of a building depth with respect to itself. It should be done in a matter of few views to navigate itself, avoiding collisions with such a towering structure. As such, no one has attacked this problem. We bring together several modules combining deep learning and 3D vision to showcase a quick reconstruction in a few views. We exploit the inherent planar structure in the buildings (facades, windows) for this purpose. We evaluate the efficacy of our pipeline with various constraints and errors from multi-view geometry using ablation studies. We then retrieve the skyline of the buildings in synthetic as well as real-world scenes.

@inproceedings{bib_Buil_2021, AUTHOR = {Khanna, Mukul and Sharma, Tanu and Swamy, Thatavarthy Vvsst Ayyappa and Krishna, K Madhava }, TITLE = {Building Facades to Normal Maps: Adversarial Learning from Single View Images}, BOOKTITLE = {Canadian Conference on Computer and Robot Vision}. YEAR = {2021}}

Surface normal estimation is an essential component of several computer and robot vision pipelines. While this problem has been extensively studied, most approaches are geared towards indoor scenes and often rely on multiple modalities (depth, multiple views) for accurate estimation of normal maps. Outdoor scenes pose a greater challenge as they exhibit significant lighting variation, often contain occluders, and structures like building facades are often ridden with numerous windows and protrusions. Conventional supervised learning schemes excel in indoor scenes, but do not exhibit competitive performance when trained and deployed in outdoor environments. Furthermore, they involve complex network architectures and require many more trainable parameters. To tackle these challenges, we present an adversarial learning scheme that regularizes the output normal maps from a neural network to appear more realistic, by using a small number of precisely annotated examples. Our method presents a lightweight and simpler architecture, while improving performance by at least 1.5x across most metrics. We evaluate our approaches against the state-of-the-art on normal map estimation, on a synthetic and a real outdoor dataset, and observe significant performance enhancements.

@inproceedings{bib_Mono_2021, AUTHOR = {Nigam, Meher Shashwat and Prabhu, Puppala Avinash and Sahu, Anurag and Karandikar, Tanvi and Gupta, Puru and Shankar, N. Sai and Kiran, Santosh Ravi and Krishna, K Madhava }, TITLE = {Monocular multi-layer layout estimation for warehouse racks}, BOOKTITLE = {Indian Conference on Computer Vision, Graphics and Image Processing}. YEAR = {2021}}

Given a monocular color image of a warehouse rack, we aim to predict the bird's-eye view layout for each shelf in the rack, which we term as' multi-layer'layout prediction. To this end, we present RackLay, a deep neural network for real-time shelf layout estimation from a single image. Unlike previous layout estimation methods which provide a single layout for the dominant ground plane alone, RackLay estimates the top-viewand front-view layout for each shelf in the considered rack populated with objects. RackLay's architecture and its variants are versatile and estimate accurate layouts for diverse scenes characterized by varying number of visible shelves in an image, large range in shelf occupancy factor and varied background clutter. Given the extreme paucity of datasets in this space and the difficulty involved in acquiring real data from warehouses, we additionally release a flexible synthetic dataset generation …

@inproceedings{bib_Mane_2021, AUTHOR = {Ashe, Avijit Kumar and Krishna, K Madhava }, TITLE = {Maneuvering Intersections & Occlusions Using MPC-Based Prioritized Tracking for Differential Drive Person Following Robot}, BOOKTITLE = {International Conference on Automation Science and Engineering}. YEAR = {2021}}

Human-robot interaction, particularly in wheeled mobile robots that can autonomously assist humans to traverse dynamically changing environments is a field of active research. Integrated motion planning and obstacle-avoidance pose a considerable challenge for an autonomous person-following robot (PFR). And, scenarios with intersections and occlusions along the path only increase the complexity in sustained tracking. In this paper, we use model predictive control (MPC) with earlyrelocation (ER) strategy to formulate a prioritized tracking scheme and implement it for a differential-drive system. Our approach ensures that the target person stays within the field of view (FOV) of the PFR consistently, even while it maneuvers intersections or crowded spots, by adding new locations to its updated path. As trajectory generation in such cases must be incremental to accommodate new information, the use of efficient representations is key. To that end, we build this social representation of following a person directly into the controller itself. MPC can naturally handle such state and input limitations as constraints to solve an on-line optimization at each time step. A non-linear MPC with ER is thus devised and tested with increasing levels of complexity arising from occlusions due to the map and its dynamic actors. By using 2D simulations, we show that for slow and medium walking speeds of the target person, the controller can plan maneuvers with an adequate margin of over 20 Hz apt for achieving a near real-time personfollowing behaviour.

@inproceedings{bib_Desi_2021, AUTHOR = {KUMAR, N VISHNU and Agarwal, Saharsh and Vadapalli, Rama and Govindan, Nagamanikandan and Krishna, K Madhava }, TITLE = {Design and Analysis of Modular Pipe Climber-III with a Multi-Output Differential Mechanism}, BOOKTITLE = {International Conference on Advanced Intelligent Mechatronics}. YEAR = {2021}}

This paper presents the design of an in-pipe climbing robot that operates using a novel ‘Three-output open differential’(3-OOD) mechanism to traverse complex networks of pipes. Conventional wheeled/tracked in-pipe climbing robots are prone to slip and drag while traversing in pipe bends. The 3-OOD mechanism helps in achieving the novel result of eliminating slip and drag in the robot tracks during motion. The proposed differential realizes the functional abilities of the traditional two-output differential, which is achieved the first time for a differential with three outputs. The 3-OOD mechanism mechanically modulates the track speeds of the robot based on the forces exerted on each track inside the pipe network, by eliminating the need for any active control. The simulation of the robot traversing in the pipe network in different orientations and in pipe-bends without slip shows the proposed design’s effectiveness.

@inproceedings{bib_Prob_2021, AUTHOR = {KRISHNA, JOSYULA GOPALA and Ramesh, Anirudha and Krishna, K Madhava }, TITLE = {Probabilistic Collision Avoidance For Multiple Robots: A Closed Form PDF Approach}, BOOKTITLE = {Intelligent Vehicles symposium}. YEAR = {2021}}

This paper proposes a novel method for reactive multiagent collision avoidance by characterizing the longitudinal and lateral intent uncertainty along a trajectory as a closed-form probability density function. Intent uncertainty is considered as the set of reachable velocities in a planning interval and distributed as a Gaussian distribution over the robot’s instantaneous velocity. We utilize the Time Scaled Collision Cone(TSCC) approach, which characterizes the space of instantaneous collision avoidance velocities available to the egoagent. We introduce intent uncertainty into the characteristic equation of the TSCC to derive the closed-form probability density function, which allows the collision avoidance problem to be rewritten as a deterministic optimization procedure. The formulation also allows the flexibility for the inclusion of confidence intervals for collision avoidance. We thus demonstrate the results and ablation studies of this derived collision avoidance formulation on various confidence intervals

@inproceedings{bib_CCO-_2021, AUTHOR = {Harithas, Sudarshan S and Yadav, Rishabh Dev and Singh, Deepak and Singh, Arun Kumar and Krishna, K Madhava }, TITLE = {CCO-VOXEL: Chance Constrained Optimization over Uncertain Voxel-Grid Representation for Safe Trajectory Planning}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2021}}

We present CCO-VOXEL: the very first chance-constrained optimization (CCO) algorithm that can compute trajectory plans with probabilistic safety guarantees in real-time directly on the voxel-grid representation of the world. CCO-VOXEL maps the distribution over the distance to the closest obstacle to a distribution over collision-constraint violation and computes an optimal trajectory that minimizes the violation probability. Importantly, unlike existing works, we never assume the nature of the sensor uncertainty or the probability distribution of the resulting collision-constraint violations. We leverage the notion of Hilbert Space embedding of distributions and Maximum Mean Discrepancy (MMD) to compute a tractable surrogate for the original chance-constrained optimization problem and employ a combination of A* based graph-search and Cross-Entropy Method for obtaining its minimum. We show tangible performance gain in terms of collision avoidance and trajectory smoothness as a consequence of our probabilistic formulation vis a vis state-of-the-art planning methods that do not account for such nonparametric noise. Finally, we also show how a combination of low-dimensional feature embedding and pre-caching of Kernel Matrices of MMD allows us to achieve real-time performance in simulations as well as in implementations on on-board commodity hardware that controls the quadrotor flight

@inproceedings{bib_Lear_2021, AUTHOR = {Omama, Mohd and S., Sundar Sripada V. and Chinchali, Sandeep and Krishna, K Madhava }, TITLE = {Learning Actions for Drift-Free Navigation in Highly Dynamic Scenes}, BOOKTITLE = {Technical Report}. YEAR = {2021}}

We embark on a hitherto unreported problem of an autonomous robot (self-driving car) navigating in dynamic scenes in a manner that reduces its localization error and eventual cumulative drift or Absolute Trajectory Error, which is pronounced in such dynamic scenes. With the hugely popular Velodyne-16 3D LIDAR as the main sensing modality, and the accurate LIDAR-based Localization and Mapping algorithm, LOAM, as the state estimation framework, we show that in the absence of a navigation policy, drift rapidly accumulates in the presence of moving objects. To overcome this, we learn actions that lead to drift-minimized navigation through a suitable set of reward and penalty functions. We use Proximal Policy Optimization, a class of Deep Reinforcement Learning methods, to learn the actions that result in drift-minimized trajectories. We show by extensive comparisons on a variety of synthetic, yet photo-realistic scenes made available through the CARLA Simulator the superior performance of the proposed framework vis-a-vis methods that do not adopt such policies.

@inproceedings{bib_DRAC_2021, AUTHOR = {Sajnani, Rahul and Sanchawala, Aadilmehdi J and Jatavallabhula, Krishna Murthy and Sridhar, Srinath and Krishna, K Madhava }, TITLE = {DRACO: Weakly Supervised Dense Reconstruction And Canonicalization of Objects}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2021}}

We present DRACO, a method for Dense Reconstruction And Canonicalization of Object shape from one or more RGB images. Canonical shape reconstruction—estimating 3D object shape in a coordinate space canonicalized for scale, rotation, and translation parameters—is an emerging paradigm that holds promise for a multitude of robotic applications. Prior approaches either rely on painstakingly gathered dense 3D supervision, or produce only sparse canonical representations, limiting real-world applicability. DRACO performs dense canonicalization using only weak supervision in the form of camera poses and semantic keypoints at train time. During inference, DRACO predicts dense object-centric depth maps in a canonical coordinate-space, solely using one or more RGB images of an object. Extensive experiments on canonical shape reconstruction and pose estimation show that DRACO is competitive or superior to fully-supervised methods.

@inproceedings{bib_RP-V_2021, AUTHOR = {R, Karnik Ram and Kharyal, Chaitanya and Harithas, S. and Krishna, K Madhava }, TITLE = {RP-VIO: Robust Plane-based Visual-Inertial Odometry for Dynamic Environments}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2021}}

Modern visual-inertial navigation systems (VINS) are faced with a critical challenge in real-world deployment: they need to operate reliably and robustly in highly dynamic environments. Current best solutions merely filter dynamic objects as outliers based on the semantics of the object category. Such an approach does not scale as it requires semantic classifiers to encompass all possibly-moving object classes; this is hard to define, let alone deploy. On the other hand, many realworld environments exhibit strong structural regularities in the form of planes such as walls and ground surfaces, which are also crucially static. We present RP-VIO, a monocular visual-inertial odometry system that leverages the simple geometry of these planes for improved robustness and accuracy in challenging dynamic environments. Since existing datasets have a limited number of dynamic elements, we also present a highly-dynamic, photorealistic synthetic dataset for a more effective evaluation of the capabilities of modern VINS systems. We evaluate our approach on this dataset, and three diverse sequences from standard datasets including two real-world dynamic sequences and show a significant improvement in robustness and accuracy over a state-of-the-art monocular visual-inertial odometry system. We also show in simulation an improvement over a simple dynamic-features masking approach. Our code and dataset are publicly available .

@inproceedings{bib_Non__2021, AUTHOR = {Nair, Unni Krishnan R and Gupta, Anish and Kiran, Dharmala Amarthya Sasi and Shrihari, Ajay and Shah, Vanshil and Singh, Arun Kumar and Krishna, K Madhava }, TITLE = {Non Holonomic Collision Avoidance under Non-Parametric Uncertainty: A Hilbert Space Approach}, BOOKTITLE = {European Control Conference}. YEAR = {2021}}

We consider the problem of an agent/robot with non-holonomic kinematics avoiding dynamic and static obstacles. Additionally there may be bounds/constraints on the configurational space of the robot in the form of lane/corridor boundaries. State and velocity noise of the robot, the lanes, the obstacles, and the robot’s control noise are modelled as non-parametric distributions as Gaussian assumptions of noise models are violated in real-world scenarios. Under these assumptions, we formulate a robust MPC that samples robotic controls effectively in a manner that aligns the robot to the goal state while avoiding obstacles and staying within the lane bounds under the duress of such non-parametric noise. In particular, the MPC incorporates a distribution matching cost that effectively aligns the distribution of the current collision cone to a certain desired distribution whose samples are collisionfree. This cost is posed as a distance function in the Hilbert Space, whose minimization typically results in the collision cone samples becoming collision-free. We show tangible performance gains compared to methods that model the collision cone distribution by linearizing the Gaussian approximations of the original non-parametric state and obstacle distributions. We also show superior performance to methods that pose a chance constraint formulation of the Gaussian approximations of nonparametric noise without subjecting such approximations to further linearizations. The performance gain is shown both in terms of trajectory length and control costs that vindicates the efficacy of the proposed method. Finally we show the proposed method being used to navigate with a non holonomic differential drive robot in real-time in a realistic setting in Gazebo with dynamic and static obstacles. To the best of our knowledge, this is the first presentation of non-holonomic collision avoidance of stationary obstacles, moving obstacles and lane constraints in the presence of non-parametric state, velocity, actuator and lane boundary noise models

@inproceedings{bib_Grou_2021, AUTHOR = {Rufus, Nivedita and Jain, Kanishk and Nair, Unni Krishnan R and Gandhi, Vineet and Krishna, K Madhava }, TITLE = {Grounding Linguistic Commands to Navigable Regions}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2021}}

— Humans have a natural ability to effortlessly comprehend linguistic commands such as “park next to the yellow sedan” and instinctively know which region of the road the vehicle should navigate. Extending this ability to autonomous vehicles is the next step towards creating fully autonomous agents that respond and act according to human commands. To this end, we propose the novel task of Referring Navigable Regions (RNR), i.e., grounding regions of interest for navigation based on the linguistic command. RNR is different from Referring Image Segmentation (RIS), which focuses on grounding an object referred to by the natural language expression instead of grounding a navigable region. For example, for a command “park next to the yellow sedan,” RIS will aim to segment the referred sedan, and RNR aims to segment the suggested parking region on the road. We introduce a new dataset, Talk2Car-RegSeg, which extends the existing Talk2car [1] dataset with segmentation masks for the regions described by the linguistic commands. A separate test split with concise manoeuvre-oriented commands is provided to assess the practicality of our dataset. We benchmark the proposed dataset using a novel transformer-based architecture. We present extensive ablations and show superior performance over baselines on multiple evaluation metrics. A downstream path planner generating trajectories based on RNR outputs confirms the efficacy of the proposed framework.

@inproceedings{bib_GCEx_2021, AUTHOR = {KUMAR, GULSHAN and Shankar, N. Sai and Didwania, Himansu and Roychoudhury, R.D. and Bhowmick, Brojeshwar and Krishna, K Madhava }, TITLE = {GCExp: Goal-Conditioned Exploration for Object Goal Navigation}, BOOKTITLE = {IEEE International Conference on Robot and Human Interactive Communication}. YEAR = {2021}}

— In this paper, we address the highly challenging problem of object goal navigation. The agent, in an unseen environment, has to perceive its surroundings to identify and navigate towards potential regions where the specified goal category can occur. Rather than developing goal driven exploration policies, we aim to adapt the existing exploration policies that maximize scene coverage to be goal-conditioned. Thus, we propose a standalone scene understanding module to identify potential regions where the goal occurs. We also propose Goal-Conditioned Exploration (GCExp), an algorithm that entails the integration of our novel scene understanding module with any existing exploration policy. We test our solution in photo-realistic simulation environments using stateof-the-art exploration policy, Active Neural Slam [1], and show improved performance over the same on every evaluation metric.

@inproceedings{bib_Mode_2021, AUTHOR = {SAHU, ANIMESH and K, Harikumar and Krishna, K Madhava }, TITLE = {Model predictive control based algorithm for multi-target tracking using a swarm of fixed wing UAVs}, BOOKTITLE = {International Conference on Automation Science and Engineering}. YEAR = {2021}}

— This paper presents a model predictive control (MPC) based algorithm for tracking multiple targets using a swarm of unmanned aerial vehicles (UAVs). All the UAVs belong to fixed-wing category with constraints on flight velocity, climb rate and turn rate. Each UAV carries a camera to detect and track the target. Two cases are considered where for the first case, the number of the UAVs is equal to the number of targets. For the second case, the number of UAVs is lesser than the number of targets leading to a conservative solution where the objective is to maximize the average time duration for which the targets are in the field-of-view (FOV) of any one of the UAV’s camera. A data driven Gaussian process (GP) based model is developed to relate the hyperparameters used in MPC to the mission efficiency. Bayesian optimization is performed to obtain the hyperparameters of the MPC that maximize the mission efficiency. Numerical simulations are performed for both cases using algorithm based on distributed MPC formulation. A performance comparison is provided with the centralized MPC formulation.

@inproceedings{bib_Desi_2021, AUTHOR = {Agarwal, Saharsh and Kumar, Vishnu and Vadapalli, Rama and Sarkar, Abhishek and Krishna, K Madhava }, TITLE = {Design and Simulation of a Flexible Three-Module Pipe Climber}, BOOKTITLE = {IEEE Second International Conference on Control, Measurement and Instrumentation (CMI)}. YEAR = {2021}}

The paper introduces a novel design for an in-pipe climbing robot with a flexible body, along with the successful simulation of the proposed mechanism. The presented Three-Module pipe climber can traverse a complex network of pipes and bends at various angles. It has bidirectional motion capabilities and is orientation independent. It can also negotiate the ‘Singularity Region’ encountered while turning through the Tjunction. The pipe climber has three modules which are arranged symmetrically apart from each other. These modules mount the tracks which provide the necessary traction for the robot to avoid slippage on the pipe’s surface. The modules, independently, have the ability to compress asymmetrically allowing the robot to bend in any direction. The taper in the front of the design and additional body flexibility due to passive spring compliance aid the motion at turns. The design of the robot is done on SOLIDWORKS and its motion is studied using MSC ADAMS. While simulating the proposed mechanism, differential speed was provided to the tracks for negotiating turns.

@inproceedings{bib_RoRD_2021, AUTHOR = {Parihar, Udit Singh and Gujarathi, Aniket and Mehta, Kinal and Tourani, Satyajit and Garg, Sourav and Krishna, K Madhava }, TITLE = {RoRD: Rotation-Robust Descriptors and Orthographic Views for Local Feature Matching}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2021}}

The use of local detectors and descriptors in typical computer vision pipelines work well until variations in viewpoint and appearance change become extreme. Past research in this area has typically focused on one of two approaches to this challenge: the use of projections into spaces more suitable for feature matching under extreme viewpoint changes, and attempting to learn features that are inherently more robust to viewpoint change. In this paper we present a novel framework that combines learning of invariant descriptors through data augmentation and orthographic viewpoint projection. We propose rotation-robust local descriptors, learnt through training data augmentation based on rotation homographies, and a correspondence ensemble technique that combines vanilla feature correspondences with those obtained through rotation-robust features. Using a range of benchmark datasets as well as contributing a new bespoke dataset for this research domain, we evaluate the effectiveness of the proposed approach on key tasks including pose estimation and visual place recognition. Our system outperforms a range of baseline and state-of-the-art techniques, including enabling higher levels of place recognition precision across opposing place viewpoints, and achieves practically-useful performance levels even under extreme viewpoint changes.

@inproceedings{bib_RP-V_2021, AUTHOR = {R, Karnik Ram and Kharyal, Chaitanya and Harithas, Sudarshan S. and Krishna, K Madhava }, TITLE = {RP-VIO: Robust Plane-based Visual-Inertial Odometry for Dynamic Environments}, BOOKTITLE = {Technical Report}. YEAR = {2021}}

— Modern visual-inertial navigation systems (VINS) are faced with a critical challenge in real-world deployment: they need to operate reliably and robustly in highly dynamic environments. Current best solutions merely filter dynamic objects as outliers based on the semantics of the object category. Such an approach does not scale as it requires semantic classifiers to encompass all possibly-moving object classes; this is hard to define, let alone deploy. On the other hand, many realworld environments exhibit strong structural regularities in the form of planes such as walls and ground surfaces, which are also crucially static. We present RP-VIO, a monocular visual-inertial odometry system that leverages the simple geometry of these planes for improved robustness and accuracy in challenging dynamic environments. Since existing datasets have a limited number of dynamic elements, we also present a highly-dynamic, photorealistic synthetic dataset for a more effective evaluation of the capabilities of modern VINS systems. We evaluate our approach on this dataset, and three diverse sequences from standard datasets including two real-world dynamic sequences and show a significant improvement in robustness and accuracy over a state-of-the-art monocular visual-inertial odometry system. We also show in simulation an improvement over a simple dynamic-features masking approach. Our code and dataset are publicly available

@inproceedings{bib_Deep_2021, AUTHOR = {Katara, Pushkal and Harish, Y V S and Pandya, Harit and Gupta, Abhinav and Sanchawala, Aadilmehdi J and Kumar, Gourav and Bhowmick, Brojeshwar and Krishna, K Madhava }, TITLE = {DeepMPCVS: Deep Model Predictive Control for Visual Servoing }, BOOKTITLE = {Technical Report}. YEAR = {2021}}

The simplicity of the visual servoing approach makes it an attractive option for tasks dealing with vision-based control of robots in many real-world applications. However, attaining precise alignment for unseen environments pose a challenge to existing visual servoing approaches. While classical approaches assume a perfect world, the recent data-driven approaches face issues when generalizing to novel environments. In this paper, we aim to combine the best of both worlds. We present a deep model predictive visual servoing framework that can achieve precise alignment with optimal trajectories and can generalize to novel environments. Our framework consists of a deep network for optical flow predictions, which are used along with a predictive model to forecast future optical flow. For generating an optimal set of velocities we present a control network that can be trained on the fly without any supervision. Through extensive simulations on photo-realistic indoor settings of the popular Habitat framework, we show significant performance gain due to the proposed formulation vis-a-vis recent state-of-the-art methods. Specifically, we show a faster convergence and an improved performance in trajectory length over recent approaches.

@inproceedings{bib_Rack_2021, AUTHOR = {Nigam, Meher Shashwat and Prabhu, Puppala Avinash and Sahu, Anurag and Gupta, Puru and Karandikar, Tanvi and Shankar, N. Sai and Kiran, Santosh Ravi and Krishna, K Madhava }, TITLE = {RackLay: Multi-Layer Layout Estimation for Warehouse Racks}, BOOKTITLE = {Technical Report}. YEAR = {2021}}

Given a monocular color image of a warehouse rack, we aim to predict the bird’s-eye view layout for each shelf in the rack, which we term as ‘multi-layer’ layout prediction. To this end, we present RackLay, a deep neural network for real-time shelf layout estimation from a single image. Unlike previous layout estimation methods which provide a single layout for the dominant ground plane alone, RackLay estimates the top-view and front-view layout for each shelf in the considered rack populated with objects. RackLay’s architecture and its variants are versatile and estimate accurate layouts for diverse scenes characterized by varying number of visible shelves in an image, large range in shelf occupancy factor and varied background clutter. Given the extreme paucity of datasets in this space and the difficulty involved in acquiring real data from warehouses, we additionally release a flexible synthetic dataset generation pipeline WareSynth which allows users to control the generation process and tailor the dataset according to contingent application. The ablations across architectural variants and comparison with strong prior baselines vindicate the efficacy of RackLay as an apt architecture for the novel problem of multi-layered layout estimation. We also show that fusing the top-view and front-view enables 3D reasoning applications such as metric free space estimation for the considered rack.

@inproceedings{bib_Auto_2020, AUTHOR = {Mani, Kaustubh and Shankar, N. Sai and Jatavallabhula, Krishna Murthy and Krishna, K Madhava }, TITLE = {AutoLay: Benchmarking amodal layout estimation for autonomous driving}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2020}}

Given an image or a video captured from a monocular camera, amodal layout estimation is the task of predicting semantics and occupancy in bird's eye view. The term amodal implies we also reason about entities in the scene that are occluded or truncated in image space. While several recent efforts have tackled this problem, there is a lack of standardization in task specification, datasets, and evaluation protocols. We address these gaps with AutoLay, a dataset and benchmark for amodal layout estimation from monocular images. AutoLay encompasses driving imagery from two popular datasets: KITTI [1] and Argoverse [2]. In addition to fine-grained attributes such as lanes, sidewalks, and vehicles, we also provide semantically annotated 3D point clouds. We implement several baselines and bleeding edge approaches, and release our data and code

@inproceedings{bib_DRAC_2020, AUTHOR = {Sajnani, Rahul and Sanchawala, Aadilmehdi J and Jatavallabhula, Krishna Murthy and Sridhar, Srinath and Krishna, K Madhava }, TITLE = {DRACO: Weakly Supervised Dense Reconstruction And Canonicalization of Objects}, BOOKTITLE = {Technical Report}. YEAR = {2020}}

We present DRACO, a method for Dense Reconstruction And Canonicalization of Object shape from one or more RGB images. Canonical shape reconstruction, estimating 3D object shape in a coordinate space canonicalized for scale, rotation, and translation parameters, is an emerging paradigm that holds promise for a multitude of robotic applications. Prior approaches either rely on painstakingly gathered dense 3D supervision, or produce only sparse canonical representations, limiting real-world applicability. DRACO performs dense canonicalization using only weak supervision in the form of camera poses and semantic keypoints at train time. During inference, DRACO predicts dense object-centric depth maps in a canonical coordinate-space, solely using one or more RGB images of an object. Extensive experiments on canonical shape reconstruction and pose estimation show that DRACO is competitive or superior to fully-supervised methods.

@inproceedings{bib_Bird_2020, AUTHOR = {Daga, Swapnil Naresh and Nair, Gokul B. and Ramesh, Anirudha and Sajnani, Rahul and Ansari, Junaid Ahmed and Krishna, K Madhava }, TITLE = {BirdSLAM: Monocular Multibody SLAM in Birds-Eye View}, BOOKTITLE = {International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applicat}. YEAR = {2020}}

In this paper, we present BirdSLAM, a novel simultaneous localization and mapping (SLAM) system for the challenging scenario of autonomous driving platforms equipped with only a monocular camera. BirdSLAM tackles challenges faced by other monocular SLAM systems (such as scale ambiguity in monocular reconstruction, dynamic object localization, and uncertainty in feature representation) by using an orthographic (bird's-eye) view as the configuration space in which localization and mapping are performed. By assuming only the height of the ego-camera above the ground, BirdSLAM leverages single-view metrology cues to accurately localize the ego-vehicle and all other traffic participants in bird's-eye view. We demonstrate that our system outperforms prior work that uses strictly greater information, and highlight the relevance of each design decision via an ablation analysis.

@inproceedings{bib_Dyna_2020, AUTHOR = {Ashe, Avijit Kumar and Krishna, K Madhava }, TITLE = {Dynamic Target Tracking & Collision Avoidance Behaviour of Person Following Robot Using Model Predictive Control *}, BOOKTITLE = {International Conference on System Theory, Control and Computing}. YEAR = {2020}}

Executing a robust person-following behaviour is a crucial task for service robots. This requires real-time tracking of the target person and maintaining it in its field of view (FOV) as long as possible while also keeping a safe distance without collision. Such a robot has to autonomously traverse long corridors, bend around corners, curve around an obstacle and join the person back, or even wait them out. Thus, in an unknown urban setting, with other agents and service robots, dynamic target tracking and obstacle avoidance becomes a challenging task for autonomous non-holonomic robots. Model Predictive Control (MPC) has the ability to incorporate future predictions. vehicle kinematics, and non-linearity of constraints while still being reasonably fast for slow and medium walking speeds of a human being. Thus, by solving a single non-linear MPC we can track the unknown dynamic trajectory of a person in real-time, and avoid undesirable proximity. In this paper, a constrained MPC framework is designed that meets the above conditions gracefully. The person-following robot (PFR) is able to track a moving target in a simulated environment. With an adequate upper margin of 20Hz, the MPC sends velocity commands for a collision-free trajectory among an increasing number of obstacles in its perceived periphery. In an incremental fashion, it is also able to adapt to varying walking behaviour, that is, types of trajectories and walking speed.

@inproceedings{bib_Earl_2020, AUTHOR = {Tourani, Satyajit and Desai, Dhagash and Parihar, Udit Singh and Garg, Sourav and Kiran, Santosh Ravi and Milford, Michael and Krishna, K Madhava }, TITLE = {Early Bird: Loop Closures from Opposing Viewpoints for Perceptually-Aliased Indoor Environments}, BOOKTITLE = {International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applicat}. YEAR = {2020}}

Significant advances have been made recently in Visual Place Recognition (VPR), feature correspondence, and localization due to the proliferation of deep-learning-based methods. However, existing approaches tend to address, partially or fully, only one of two key challenges: viewpoint change and perceptual aliasing. In this paper, we present novel research that simultaneously addresses both challenges by combining deep-learned features with geometric transformations based on reasonable domain assumptions about navigation on a ground-plane, whilst also removing the requirement for specialized hardware setup (eg lighting, downwards facing cameras). In particular, our integration of VPR with SLAM by leveraging the robustness of deep-learned features and our homography-based extreme viewpoint invariance significantly boosts the performance of VPR, feature correspondence, and pose graph submodules of the SLAM pipeline. For the first time, we demonstrate a localization system capable of state-of-the-art performance despite perceptual aliasing and extreme 180-degree-rotated viewpoint change in a range of real-world and simulated experiments. Our system is able to achieve early loop closures that prevent significant drifts in SLAM trajectories. We also compare extensively several deep architectures for VPR and descriptor matching. We also show that superior place recognition and descriptor matching across opposite views results in a similar performance gain in back-end pose graph optimization.

@inproceedings{bib_Cosi_2020, AUTHOR = {Rufus, Nivedita and Nair, Unni Krishnan R and Krishna, K Madhava and Gandhi, Vineet }, TITLE = {Cosine meets softmax: A tough-to-beat baseline for visual grounding}, BOOKTITLE = {European Conference on Computer Vision Workshops}. YEAR = {2020}}

In this paper, we present a simple baseline for visual grounding for autonomous driving which outperforms the state of the art methods, while retaining minimal design choices. Our framework minimizes the cross-entropy loss over the cosine distance between multiple image ROI features with a text embedding (representing the given sentence/phrase). We use pre-trained networks for obtaining the initial embeddings and learn a transformation layer on top of the text embedding. We perform experiments on the Talk2Car dataset and achieve 68.7% AP50 accuracy, improving upon the previous state of the art by 8.6%. Our investigation suggests reconsideration towards more approaches employing sophisticated attention mechanisms or multi-stage reasoning or complex metric learning loss functions by showing promise in simpler alternatives.

@inproceedings{bib_Mono_2020, AUTHOR = {Mani, Kaustubh and Daga, Swapnil Naresh and Garg, Shubhika and Shankar, N. Sai and Murthy, J. Krishna and Krishna, K Madhava }, TITLE = {MonoLayout: Amodal scene layout from a single image}, BOOKTITLE = {Winter Conference on Applications of Computer Vision}. YEAR = {2020}}

In this paper, we address the novel, highly challenging problem of estimating the layout of a complex urban driving scenario. Given a single color image captured from a driving platform, we aim to predict the bird’s eye view layout of the road and other traffic participants. The estimated layout should reason beyond what is visible in the image, and compensate for the loss of 3D information due to projection. We dub this problem amodal scene layout estimation, which involves hallucinating scene layout for even parts of the world that are occluded in the image. To this end, we present MonoLayout, a deep neural network for realtime amodal scene layout estimation from a single image. We represent scene layout as a multi-channel semantic occupancy grid, and leverage adversarial feature learning to “hallucinate" plausible completions for occluded image parts. We extend several state-of-the-art approaches for road-layout estimation and vehicle occupancy estimation in bird’s eye view to the amodal setup and thoroughly evaluate against them. By leveraging temporal sensor fusion to generate training labels, we significantly outperform current art over a number of datasets.

@inproceedings{bib_Reco_2020, AUTHOR = {Dinesh, Pokale Aniket and Aggarwal, Aditya and Jatavallabhula, Krishna Murthy and Krishna, K Madhava }, TITLE = {Reconstruct, Rasterize and Backprop: Dense shape and pose estimation from a single image}, BOOKTITLE = {Computer Vision and Pattern Recognition Conference workshops}. YEAR = {2020}}

This paper presents a new system to obtain dense object reconstructions along with 6-DoF poses from a single image. Geared towards high fidelity reconstruction, several recent approaches leverage implicit surface representations and deep neural networks to estimate a 3D mesh of an object, given a single image. However, all such approaches recover only the shape of an object; the reconstruction is often in a canonical frame, unsuitable for downstream robotics tasks. To this end, we leverage recent advances in differentiable rendering (in particular, rasterization) to close the loop with 3D reconstruction in camera frame. We demonstrate that our approach—dubbed reconstruct, rasterize and backprop (RRB)—achieves significantly lower pose estimation errors compared to prior art, and is able to recover dense object shapes and poses from imagery. We further extend our results to an (offline) setup, where we demonstrate a dense monocular object-centric egomotion estimation system.

@inproceedings{bib_Omni_2020, AUTHOR = {Suryavanshi, Kartik and VADAPALLI, RAMA and Budharaja, Praharsha and Sarkar, Abhishek and Krishna, K Madhava }, TITLE = {Omnidirectional Three Module Robot Design and Simulation}, BOOKTITLE = {Technical Report}. YEAR = {2020}}

This paper introduces the Omnidirectional Tractable Three Module Robot for traversing inside complex pipe networks. The robot consists of three omnidirectional modules fixed 120 apart circumferentially which can rotate about their axis allowing holonomic motion of the robot. Holonomic motion enables the robot to overcome motion singularity when negotiating T-junctions and further allows the robot to arrive in a preferred orientation while taking turns inside a pipe. The singularity region while negotiating T-junctions is analyzed to formulate the geometry of the region. The design and motion capabilities are validated by conducting simulations in MSC ADAMS on a simplified lumped-model of the robot

@inproceedings{bib_Mult_2020, AUTHOR = {Nair, Gokul B. and Daga, Swapnil Naresh and Sajnani, Rahul and Ramesh, Anirudha and Ansari, Junaid Ahmed and Jatavallabhula, Krishna Murthy and Krishna, K Madhava }, TITLE = {Multi-object Monocular SLAM for Dynamic Environments}, BOOKTITLE = {Intelligent Vehicles symposium}. YEAR = {2020}}

In this paper, we tackle the problem of multibody SLAM from a monocular camera. The term multibody, implies that we track the motion of the camera, as well as that of other dynamic participants in the scene. The quintessential challenge in dynamic scenes is unobservability: it is not possible to unambiguously triangulate a moving object from a moving monocular camera. Existing approaches solve restricted variants of the problem, but the solutions suffer relative scale ambiguity (i.e., a family of infinitely many solutions exist for each pair of motions in the scene). We solve this rather intractable problem by leveraging single-view metrology, advances in deep learning, and category-level shape estimation. We propose a multi posegraph optimization formulation, to resolve the relative and absolute scale factor ambiguities involved. This optimization helps us reduce the average error in trajectories of multiple bodies over real-world datasets, such as KITTI [1]. To the best of our knowledge, our method is the first practical monocular multi-body SLAM system to perform dynamic multi-object and ego localization in a unified framework in metric scale.

@inproceedings{bib_Reac_2020, AUTHOR = {Jyotish, Poonganam Sri Sai Naga and GOPALAKRISHNAN, BHARATH and KUMAR, AVULA VENKATA SEETHARAMA SAI BHARGAV and Singh, Arun Kumar and Krishna, K Madhava and Manocha, Dinesh }, TITLE = {Reactive Navigation under Non-Parametric Uncertainty through Hilbert Space Embedding of Probabilistic Velocity Obstacles}, BOOKTITLE = {IEEE Robotics and Automation Letters}. YEAR = {2020}}

The probabilistic velocity obstacle (PVO) extends the concept of velocity obstacle (VO) to work in uncertain dynamic environments. In this paper, we show how a robust model predictive control (MPC) with PVO constraints under non-parametric uncertainty can be made computationally tractable. At the core of our formulation is a novel yet simple interpretation of our robust MPC as a problem of matching the distribution of PVO with a certain desired distribution. To this end, we propose two methods. Our first baseline method is based on approximating the distribution of PVO with a Gaussian Mixture Model (GMM) and subsequently performing distribution matching using Kullback Leibler (KL) divergence metric. Our second formulation is based on the possibility of representing arbitrary distributions as functions in Reproducing Kernel Hilbert Space (RKHS). We use this foundation to interpret our robust MPC as a problem of minimizing the distance between the desired distribution and the distribution of the PVO in the RKHS. Both the RKHS and GMM based formulation can work with any uncertainty distribution and thus allowing us to relax the prevalent Gaussian assumption in the existing works. We validate our formulation by taking an example of 2D navigation of quadrotors with a realistic noise model for perception and ego-motion uncertainty. In particular, we present a systematic comparison between the GMM and the RKHS approach and show that while both approaches can produce safe trajectories, the former is highly conservative and leads to poor tracking and control costs. Furthermore, RKHS based approach gives better computational times that are up to one order of magnitude lesser than the computation time of the GMM based approach.

@inproceedings{bib_DFVS_2020, AUTHOR = {Harish, Y V S and Pandya, Harit and Gaud, Ayush and Terupally, Shreya Reddy and Shankar, Sai and Krishna, K Madhava }, TITLE = {DFVS: Deep Flow Guided Scene Agnostic Image Based Visual Servoing}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2020}}

Existing deep learning based visual servoing approaches regress the relative camera pose between a pair of images. Therefore, they require a huge amount of training data and sometimes fine-tuning for adaptation to a novel scene. Furthermore, current approaches do not consider underlying geometry of the scene and rely on direct estimation of camera pose. Thus, inaccuracies in prediction of the camera pose, especially for distant goals, lead to a degradation in the servoing performance. In this paper, we propose a two-fold solution: (i) We consider optical flow as our visual features, which are predicted using a deep neural network. (ii) These flow features are then systematically integrated with depth estimates provided by another neural network using interaction matrix. We further present an extensive benchmark in a photo-realistic 3D simulation across diverse scenes to study the convergence and generalisation of visual servoing approaches. We show convergence for over 3m and 40 degrees while maintaining precise positioning of under 2cm and 1 degree on our challenging benchmark where the existing approaches that are unable to converge for majority of scenarios for over 1.5m and 20 degrees. Furthermore, we also evaluate our approach for a real scenario on an aerial robot. Our approach generalizes to novel scenarios producing precise and robust servoing performance for 6 degrees of freedom positioning tasks with even large camera transformations without any retraining or fine-tuning.

@inproceedings{bib_Bi-C_2020, AUTHOR = {Singh, Arun Kumar and Raghuram, Theerthala Venkata Sai and Nallana, Mithun Babu and Nair, Unni Krishnan R and Krishna, K Madhava }, TITLE = {Bi-Convex Approximation of Non-Holonomic Trajectory Optimization}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2020}}

Autonomous cars and fixed-wing aerial vehicles have the so-called non-holonomic kinematics which non-linearly maps control input to states. As a result, trajectory optimization with such a motion model becomes highly non-linear and nonconvex. In this paper, we improve the computational tractability of non-holonomic trajectory optimization by reformulating it in terms of a set of bi-convex cost and constraint functions along with a non-linear penalty. The bi-convex part acts as a relaxation for the non-holonomic trajectory optimization while the residual of the penalty dictates how well its output obeys the non-holonomic behavior. We adopt an alternating minimization approach for solving the reformulated problem and show that it naturally leads to the replacement of the challenging nonlinear penalty with a globally valid convex surrogate. Along with the common cost functions modeling goal-reaching, trajectory smoothness, etc., the proposed optimizer can also accommodate a class of non-linear costs for modeling goal-sets, while retaining the bi-convex structure. We benchmark the proposed optimizer against off-the-shelf solvers implementing sequential quadratic programming and interior-point methods and show that it produces solutions with similar or better cost as the former while significantly outperforming the latter. Furthermore, as compared to both off-the-shelf solvers, the proposed optimizer achieves more than 20x reduction in computation time.

@inproceedings{bib_SROM_2020, AUTHOR = {Rufus, Nivedita and Nair, Unni Krishnan R and KUMAR, AVULA VENKATA SEETHARAMA SAI BHARGAV and Madiraju, Vashist and Krishna, K Madhava }, TITLE = {SROM: Simple Real-time Odometry and Mapping using LiDAR data for Autonomous Vehicles}, BOOKTITLE = {Intelligent Vehicles symposium}. YEAR = {2020}}

In this paper, we present SROM, a novel realtime Simultaneous Localization and Mapping (SLAM) system for autonomous vehicles. The keynote of the paper showcases SROM’s ability to maintain localization at low sampling rates or at high linear or angular velocities where most popular LiDAR based localization approaches get degraded fast. We also demonstrate SROM to be computationally efficient and capable of handling high-speed maneuvers. It also achieves low drifts without the need for any other sensors like IMU and/or GPS.Our method has a two-layer structure wherein first, an approximate estimate of the rotation angle and translation parameters are calculated using a Phase Only Correlation (POC) method.Next, we use this estimate as an initialization for a point-toplane ICP algorithm to obtain fine matching and registration. Another key feature of the proposed algorithm is the removal of dynamic objects before matching the scans. This improves the performance of our system as the dynamic objects can corrupt the matching scheme and derail localization. Our SLAM system can build reliable maps at the same time generating high-quality odometry. We exhaustively evaluated the proposed method in many challenging highways/country/urban sequences from the KITTI dataset and the results demonstrate better accuracy in comparisons to other state-of-the-art methods with reduced computational expense aiding in real-time realizations. We have also integrated our SROM system with our in-house autonomous vehicle and compared it with the state-of the-art methods like LOAM and LeGO-LOAM.

@inproceedings{bib_Topo_2020, AUTHOR = {Puligilla, Sai Shubodh and Tourani, Satyajit and Shridhar, Vaidya Tushar and Parihar, Udit Singh and Kiran, Santosh Ravi and Krishna, K Madhava }, TITLE = {Topological Mapping for Manhattan-like Repetitive Environments}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2020}}

We showcase a topological mapping framework for a challenging indoor warehouse setting. At the most abstract level, the warehouse is represented as a Topological Graph where the nodes of the graph represent a particular warehouse topological construct (e.g. rackspace, corridor) and the edges denote the existence of a path between two neighbouring nodes or topologies. At the intermediate level, the map is represented as a Manhattan Graph where the nodes and edges are characterized by Manhattan properties and as a Pose Graph at the lower-most level of detail. The topological constructs are learned via a Deep Convolutional Network while the relational properties between topological instances are learnt via a Siamese-style Neural Network. In the paper, we show that maintaining abstractions such as Topological Graph and Manhattan Graph help in recovering an accurate Pose Graph starting from a highly erroneous and unoptimized Pose Graph. We show how this is achieved by embedding topological and Manhattan relations as well as Manhattan Graph aided loop closure relations as constraints in the backend Pose Graph optimization framework. The recovery of near ground-truth Pose Graph on real-world indoor ware

@inproceedings{bib_Towa_2020, AUTHOR = {Sravan, Mylavarapu Venkata Sai and SANDHU, MAHTAB and Vijayan, Priyesh and Krishna, K Madhava and Ravindran, Balaraman and Namboodiri, Anoop }, TITLE = {Towards Accurate Vehicle Behaviour Classification With Multi-Relational Graph Convolutional Networks}, BOOKTITLE = {Intelligent Vehicles symposium}. YEAR = {2020}}

Understanding on-road vehicle behaviour from a temporal sequence of sensor data is gaining in popularity. In this paper, we propose a pipeline for understanding vehicle behaviour from a monocular image sequence or video. A monocular sequence along with scene semantics, optical flow and object labels are used to get spatial information about the object (vehicle) of interest and other objects (semantically contiguous set of locations) in the scene. This spatial information is encoded by a Multi-Relational Graph Convolutional Network (MR-GCN), and a temporal sequence of such encodings is fed to a recurrent network to label vehicle behaviours. The proposed framework can classify a variety of vehicle behaviours to high fidelity on datasets that are diverse and include European, Chinese and Indian on-road scenes. The framework also provides for seamless transfer of models across datasets without entailing re-annotation, retraining and even fine-tuning. We show comparative performance gain over baseline Spatio-temporal classifiers and detail a variety of ablations to showcase the efficacy of the framework.

@inproceedings{bib_Unde_2020, AUTHOR = {Sravan, Mylavarapu Venkata Sai and SANDHU, MAHTAB and Sandhu, Mahtab and Krishna, K Madhava and Ravindran, Balaraman and Namboodiri, Anoop }, TITLE = {Understanding Dynamic Scenes using Graph Convolution Networks}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2020}}

We present a novel Multi Relational Graph Convolutional Network (MRGCN) to model on-road vehicle behaviours from a sequence of temporally ordered frames as grabbed by a moving monocular camera. The input to MRGCN is a Multi Relational Graph (MRG) where the nodes of the graph represent the active and passive participants/agents in the scene while the bidrectional edges that connect every pair of nodes are encodings of the spatio-temporal relations. The bidirectional edges of the graph encode the temporal interactions between the agents that constitute the two nodes of the edge. The proposed method of obtaining his encoding is shown to be specifically suited for the problem at hand as it outperforms more complex end to end learning methods that do not use such intermediate representations of evolved spatio-temporal relations between agent pairs. We show significant performance gain in the form of behaviour classification accuracy on a variety of datasets from different parts of the globe over prior methods as well as show seamless transfer without any resort to fine-tuning across multiple datasets. Such behaviour prediction methods find immediate relevance in a variety of navigation tasks such as behaviour planning, state estimation as well as in applications relating to detection of traffic violations over videos.

@inproceedings{bib_LiDA_2020, AUTHOR = {Singh, Aasheesh and Kamireddypalli, Aditya and Gandhi, Vineet and Krishna, K Madhava }, TITLE = {LiDAR guided Small obstacle Segmentation}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2020}}

Abstract— Detecting small obstacles on the road is critical for autonomous driving. In this paper, we present a method to reliably detect such obstacles through a multi-modal framework of sparse LiDAR(VLP-16) and Monocular vision. LiDAR is employed to provide additional context in the form of confidence maps to monocular segmentation networks. We show significant performance gains when the context is fed as an additional input to monocular semantic segmentation frameworks. We further present a new semantic segmentation dataset to the community, comprising of over 3000 image frames with corresponding LiDAR observations. The images come with pixel-wise annotations of three classes off-road, road, and small obstacle. We stress that precise calibration between LiDAR and camera is crucial for this task and thus propose a novel Hausdorff distance based calibration refinement method over extrinsic parameters. As a first benchmark over this dataset, we report our results with 73 % instance detection up to a distance of 50 meters on challenging scenarios. Qualitatively by showcasing accurate segmentation of obstacles less than 15 cms at 50m depth and quantitatively through favourable comparisons vis a vis prior art, we vindicate the method’s efficacy. Our project and dataset is hosted at https://small-obstacle-dataset.github.io/

@inproceedings{bib_PIVO_2019, AUTHOR = {Jyotis, P. S. Naga and Goel, Yash and , A. V. S. Sai Bhargav Kumar and Krishna, K Madhava }, TITLE = {PIVO: Probabilistic Inverse Velocity Obstacle for Navigation under Uncertainty}, BOOKTITLE = {IEEE International Conference on Robot and Human Interactive Communication}. YEAR = {2019}}

In this paper, we present an algorithmic framework which computes the collision-free velocities for the robot in a human shared dynamic and uncertain environment. We extend the concept of Inverse Velocity Obstacle (IVO) to a probabilistic variant to handle the state estimation and motion uncertainties that arise due to the other participants of the environment. These uncertainties are modeled as non-parametric probability distributions. In our PIVO: Probabilistic Inverse Velocity Obstacle, we propose the collision-free navigation as an optimization problem by reformulating the velocity conditions of IVO as chance constraints that takes the uncertainty into account. The space of collision-free velocities that result from the presented optimization scheme are associated to a confidence measure as a specified probability. We demonstrate the efficacy of our PIVO through numerical simulations and demonstrating its ability to generate safe trajectories under highly uncertain environments.

@inproceedings{bib_INFE_2019, AUTHOR = {SRIKANTH, SHASHANK and Ansari, Junaid Ahmed and Ram, R. Karnik and SHARMA, SARTHAK and Murthy, J. Krishna and Krishna, K Madhava }, TITLE = {INFER: INtermediate representations for FuturE pRediction}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2019}}

In urban driving scenarios, forecasting future trajectories of surrounding vehicles is of paramount importance. While several approaches for the problem have been proposed, the best-performing ones tend to require extremely detailed input representations (eg. image sequences). But, such methods do not generalize to datasets they have not been trained on. We propose intermediate representations that are particularly well-suited for future prediction. As opposed to using texture (color) information, we rely on semantics and train an autoregressive model to accurately predict future trajectories of traffic participants (vehicles) (see fig. above). We demonstrate that using semantics provides a significant boost over techniques that operate over raw pixel intensities/disparities. Uncharacteristic of state-of-the-art approaches, our representations and models generalize to completely different datasets, collected across several cities, and also across countries where people drive on opposite sides of the road (left-handed vs right-handed driving). Additionally, we demonstrate an application of our approach in multi-object tracking (data association). To foster further research in transferrable representations and ensure reproducibility, we release all our code and data. 3

@inproceedings{bib_PIVO_2019, AUTHOR = {, and Jyotish, Poonganam Sri Sai Naga and KUMAR, AVULA VENKATA SEETHARAMA SAI BHARGAV and Krishna, K Madhava }, TITLE = {PIVO: Probabilistic Inverse Velocity Obstacle for Navigation under Uncertainty}, BOOKTITLE = {IEEE International Conference on Robot and Human Interactive Communication}. YEAR = {2019}}

In this paper, we present an algorithmic frame-work which computes the collision-free velocities for the robotin a human shared dynamic and uncertain environment. Weextend the concept of Inverse Velocity Obstacle (IVO) to aprobabilistic variant to handle the state estimation and motionuncertainties that arise due to the other participants of the en-vironment. These uncertainties are modeled as non-parametricprobability distributions. In our PIVO: Probabilistic InverseVelocity Obstacle, we propose the collision-free navigation asan optimization problem by reformulating the velocity condi-tions of IVO as chance constraints that takes the uncertaintyinto account. The space of collision-free velocities that resultfrom the presented optimization scheme are associated to aconfidence measure as a specified probability. We demonstratethe efficacy of our PIVO through numerical simulations anddemonstrating its ability to generate safe trajectories underhighly uncertain environments

@inproceedings{bib_Desi_2019, AUTHOR = {Bhole, Ajinkya and HARSHA, TURLAPATI SRI and S, Rajashekhar V. and Dixit, Jay and Shah, Suril V. and Krishna, K Madhava }, TITLE = {Design of a Robust Stair Climbing Compliant Modular Robot to Tackle Overhang on Stairs}, BOOKTITLE = {Robotica}. YEAR = {2019}}

This paper discusses the concept and parameter design of a Robust Stair Climbing Compliant Modular Robot, capable of tackling stairs with overhangs. Modifying the geometry of the periphery of the wheels of our robot helps in tackling overhangs. Along with establishing a concept design, robust design parameters are set to minimize performance variation. The Grey-based Taguchi Method is adopted for providing an optimal setting for the design parameters of the robot. The robot prototype is shown to have successfully scaled stairs of varying dimensions, with overhang, thus corroborating the analysis performed.

@inproceedings{bib_A_Hi_2019, AUTHOR = {NARAYANAN, SRIRAM and Kumar, Gourav and Singh, Abhay and Karthik, M. Siva and SAURAV, SAKET and Bhowmick, Brojeshwar and Krishna, K Madhava }, TITLE = {A Hierarchical Network for Diverse Trajectory Proposals}, BOOKTITLE = {Intelligent Vehicles symposium}. YEAR = {2019}}

Autonomous explorative robots frequently encounter scenarios where multiple future trajectories can be pursued. Often these are cases with multiple paths around an obstacle or trajectory options towards various frontiers. Humans in such situations can inherently perceive and reason about the surrounding environment to identify several possibilities of either manoeuvring around the obstacles or moving towards various frontiers. In this work, we propose a 2 stage Convolutional Neural Network architecture which mimics such an ability to map the perceived surroundings to multiple trajectories that a robot can choose to traverse. The first stage is a Trajectory Proposal Network which suggests diverse regions in the environment which can be occupied in the future. The second stage is a Trajectory Sampling network which provides a finegrained trajectory over the regions proposed by Trajectory Proposal Network. We evaluate our framework in diverse and complicated real life settings. For the outdoor case, we use the KITTI dataset and our own outdoor driving dataset. In the indoor setting, we use an autonomous drone to navigate various scenarios and also a ground robot which can explore the environment using the trajectories proposed by our framework. Our experiments suggest that the framework is able to develop a semantic understanding of the obstacles, open regions and identify diverse trajectories that a robot can traverse. Our comparisons portray the performance gain of the proposed architecture over a diverse set of methods against which it is compared.

@inproceedings{bib_Moti_2019, AUTHOR = {Theerthala, Raghu Ram and KUMAR, AVULA VENKATA SEETHARAMA SAI BHARGAV and Nallana, Mithun Babu and TEJA, SINGAMANENI PHANI and Krishna, K Madhava }, TITLE = {Motion Planning Framework for Autonomous Vehicles: A Time Scaled Collision Cone Interleaved Model Predictive Control Approach}, BOOKTITLE = {Intelligent Vehicles symposium}. YEAR = {2019}}

Planning frameworks for autonomous vehicles must be robust and computationally efficient for real time realization. At the same time, they should accommodate the unpredictable behavior of the other participants and produce safe trajectories. In this paper, we present a computationally efficient hierarchical planning framework for autonomous vehicles that can generate safe trajectories in complex driving scenarios, which are commonly encountered in urban traffic settings. The first level of the proposed framework constructs a Model Predictive Control(MPC) routine using an efficient difference of convex programming approach, that generates smooth and collision-free trajectories. The constraints on curvature and road boundaries are seamlessly integrated into this optimization routine. The second layer is mainly responsible to handle the unpredictable behaviors that are typically exhibited by the other participants of traffic. It is built along the lines of time scaled collision cone(TSCC) which optimize for the velocities along the trajectory to handle such disturbances. We additionally show that our framework maintains optimal balance between temporal and path deviations while executing safe trajectories. To demonstrate the efficacy of the presented framework we validated it in extensive simulations in different driving scenarios like over taking, lane merging and jaywalking among many dynamic and static obstacles.

@inproceedings{bib_SVM__2019, AUTHOR = {Nair, Unni Krishnan R and Rufus, Nivedita and Madiraju, Vashist and Krishna, K Madhava }, TITLE = {SVM Enhanced Frenet Frame Planner For Safe Navigation Amidst Moving Agents}, BOOKTITLE = {Technical Report}. YEAR = {2019}}

This paper proposes an SVM Enhanced Trajectory Planner for dynamic scenes, typically those encountered in on road settings. Frenet frame based trajectory generation is popular in the context of autonomous driving both in research and industry. We incorporate a safety based maximal margin criteria using a SVM layer that generates control points that are maximally separated from all dynamic obstacles in the scene. A kinematically consistent trajectory generator then computes a path through these waypoints. We showcase through simulations as well as real world experiments on a self driving car that the SVM enhanced planner provides for a larger offset with dynamic obstacles than the regular Frenet frame based trajectory generation. Thereby, the authors argue that such a formulation is inherently suited for navigation amongst pedestrians. We assume the availability of an intent or trajectory prediction module that predicts the future trajectories of all dynamic actors in the scene.

@inproceedings{bib_A_Hi_2019, AUTHOR = {NARAYANAN, SRIRAM and Kumar, Gourav and Singh, Abhay and Karthik, M. Siva and SAURAV, SAKET and Bhowmick, Brojeshwar and Krishna, K Madhava }, TITLE = {A Hierarchical Network for Diverse Trajectory Proposals}, BOOKTITLE = {Intelligent Vehicles symposium}. YEAR = {2019}}

Autonomous explorative robots frequently encounter scenarios where multiple future trajectories can be pursued. Often these are cases with multiple paths around an obstacle or trajectory options towards various frontiers. Humans in such situations can inherently perceive and reason about the surrounding environment to identify several possibilities of either manoeuvring around the obstacles or moving towards various frontiers. In this work, we propose a 2 stage Convolutional Neural Network architecture which mimics such an ability to map the perceived surroundings to multiple trajectories that a robot can choose to traverse. The first stage is a Trajectory Proposal Network which suggests diverse regions in the environment which can be occupied in the future. The second stage is a Trajectory Sampling network which provides a finegrained trajectory over the regions proposed by Trajectory Proposal Network. We evaluate our framework in diverse and complicated real life settings. For the outdoor case, we use the KITTI dataset and our own outdoor driving dataset. In the indoor setting, we use an autonomous drone to navigate various scenarios and also a ground robot which can explore the environment using the trajectories proposed by our framework. Our experiments suggest that the framework is able to develop a semantic understanding of the obstacles, open regions and identify diverse trajectories that a robot can traverse. Our comparisons portray the performance gain of the proposed architecture over a diverse set of methods against which it is compared.

@inproceedings{bib_IVO:_2019, AUTHOR = {Jyotish, Poonganam Sri Sai Naga and Goel, Yash and KUMAR, AVULA VENKATA SEETHARAMA SAI BHARGAV and Krishna, K Madhava }, TITLE = {IVO: Inverse Velocity Obstacles for Real Time Navigation}, BOOKTITLE = {Advances in Robotics}. YEAR = {2019}}

In this paper, we present IVO: Inverse Velocity Obstacles an egocentric framework that improves the real time implementation. The proposed method stems from the concept of velocity obstacle and can be applied for both single agent and multi-agent system. It focuses on computing collision free maneuvers without any knowledge or assumption on the pose and the velocity of the robot. This is primarily achieved by reformulating the velocity obstacle to adapt to an ego-centric framework. This is a significant step towards improving real time implementations of collision avoidance in dynamic environments as there is no dependency on state estimation techniques to infer the robot pose and velocity. We evaluate IVO for both single agent and multi-agent in different scenarios and show it’s efficacy over the existing formulations. We also show the real time scalability of the proposed methodology.

@inproceedings{bib_A_pr_2019, AUTHOR = {Dinesh, Pokale Aniket and Das, Dipanjan and Aggarwal, Aditya and Bhowmick, Brojeshwar and Krishna, K Madhava }, TITLE = {A principled formulation of integrating objects in Monocular SLAM}, BOOKTITLE = {Advances in Robotics}. YEAR = {2019}}

Monocular SLAM is a well-studied problem and has shown significant progress in recent years, but still, challenges remain in creating a rich semantic description of the scene. Feature-based visual SLAMs are vulnerable to erroneous pose estimates due to insufficient tracking of mapped points or motion induced errors such as in large or in-place rotations. We present a new SLAM framework in which we use monocular edge based SLAM [1], along with category level models, to localize objects in the scene as well as improve the camera trajectory. In monocular SLAM systems, the camera track tends to break in conditions with abrupt motion which leads to reduction in the number of 2D point correspondences. In order to tackle this problem, we propose the first most principled formulation of its kind which integrates object category models in the core SLAM back-end to jointly optimize for the camera trajectory, object poses along with its shape and 3D structure. We show that our joint optimization is able to recover a better camera trajectory in such cases, as compared to Edge SLAM. Moreover, this method gives a better visualization incorporating object representations in the scene along with the 3D structure of the base SLAM system, which makes it useful for augmented reality (AR) applications

@inproceedings{bib_Grad_2019, AUTHOR = {Adarsh, and Singh, Siddharth and KUMAR, AVULA VENKATA SEETHARAMA SAI BHARGAV and NARAYANAN, SRIRAM and Krishna, K Madhava }, TITLE = {Gradient Aware: Shrinking Domain based Control Design for Reactive Planning Frameworks }, BOOKTITLE = {Advances in Robotics}. YEAR = {2019}}

In this paper, we present a novel control law for longitudinal speed control of autonomous vehicles. The key contributions of the proposed work include the design of a control law that reactively integrates the longitudinal surface gradient of the road into its operation. In contrast to the existing works, we found that integrating the path gradient into the control framework improves the speed tracking efficacy. Since the control law is implemented over a shrinking domain scheme, it minimizes the integrated error by recomputing the control inputs at every discretized step and consequently provides less reaction time. This makes our control law suitable for motion planning frameworks that are operating at high frequencies. Furthermore, our work is implemented using a generalized vehicle model and can be easily extended to other classes of vehicles. The performance of gradient aware - shrinking domain based controller is implemented and tested on an electric car. Results from the tests show the robustness of our control law for speed tracking efficiency on terrain with varying gradient while also considering stringent time constraints imposed by the planning framework.

@inproceedings{bib_Para_2019, AUTHOR = {KAUSHIK, MEHA and TEJA, SINGAMANENI PHANI and Krishna, K Madhava }, TITLE = {Parameter Sharing Reinforcement Learning Architecture for Multi Agent Driving Behaviors}, BOOKTITLE = {Advances in Robotics}. YEAR = {2019}}

Multi-agent learning provides a potential framework for learning and simulating traffic behaviors. This paper proposes a novel architecture to learn multiple driving behaviors in a traffic scenario. The proposed architecture can learn multiple behaviors independently as well as simultaneously. We take advantage of the homogeneity of agents and learn in a parameter sharing paradigm. To further speed up the training process asynchronous updates are employed into the architecture. While learning different behaviors simultaneously, the given framework was also able to learn cooperation between the agents, without any explicit communication. We applied this framework to learn two important behaviors in driving: 1) Lane-Keeping and 2) Over-Taking. Results indicate faster convergence and learning of a more generic behavior, that is scalable to any number of agents. When compared the results with existing approaches, our results indicate equal and even better performance in some cases.

@inproceedings{bib_Para_2019, AUTHOR = {KAUSHIK, MEHA and TEJA, SINGAMANENI PHANI and Krishna, K Madhava }, TITLE = {Parameter Sharing Reinforcement Learning Architecture for Multi Agent Driving Behaviors}, BOOKTITLE = {Advances in Robotics}. YEAR = {2019}}

Multi-agent learning provides a potential framework for learning and simulating traffic behaviors. This paper proposes a novel architecture to learn multiple driving behaviors in a traffic scenario. The proposed architecture can learn multiple behaviors independently as well as simultaneously. We take advantage of the homogeneity of agents and learn in a parameter sharing paradigm. To further speed up the training process asynchronous updates are employed into the architecture. While learning different behaviors simultaneously, the given framework was also able to learn cooperation between the agents, without any explicit communication. We applied this framework to learn two important behaviors in driving: 1) Lane-Keeping and 2) Over-Taking. Results indicate faster convergence and learning of a more generic behavior, that is scalable to any number of agents. When compared the results with existing approaches, our results indicate equal and even better performance in some cases.

@inproceedings{bib_Mode_2019, AUTHOR = {Nallana, Mithun Babu and Raghuram, Theerthala Venkata Sai and Singh, Arun Kumar and Baladhurgesh.B.P, and GOPALAKRISHNAN, BHARATH and Krishna, K Madhava }, TITLE = {Model Predictive Control for Autonomous Driving considering Actuator Dynamics}, BOOKTITLE = {American Control Conference}. YEAR = {2019}}

In this paper, we propose a new model predictive control (MPC) formulation for autonomous driving. The novelty of our MPC stems from the following results. Firstly, we adopt an alternating minimization approach wherein linear velocities and angular accelerations are alternately optimized. We show that in contrast to the joint optimization, the alternating minimization exploits the structure of the problem better, which in turn translates to reduction in computation time. Secondly, our MPC explicitly incorporates the time dependent non-linear actuator dynamics that captures the transient response of the vehicle for a given commanded velocity. This added complexity improves the predictive component of MPC resulting in improved margin of inter-vehicle distance during maneuvers like overtaking, lane-change, etc. Although, past works have also incorporated actuator dynamics within MPC, there has been very few attempts towards coupling actuator dynamics to collision avoidance constraints through the non-holonomic motion model of the vehicle and analyzing the resulting behavior. We use a high fidelity simulator to benchmark our actuator dynamics augmented MPC with other related approaches in terms of metrics like inter-vehicle distance, trajectory smoothness, and velocity overshoot.

@inproceedings{bib_SMA__2019, AUTHOR = {Pusalkar, Neha and Karmakar, Sourav and Aggrawal, Rohit and Mal, Pravin and Singh, Akash and Sarkar, Abhishek and Krishna, K Madhava }, TITLE = {SMA Actuated Dual Arm Flexible Gripper}, BOOKTITLE = {Advances in Robotics}. YEAR = {2019}}

Robotic grippers have been designed for grasping a wide variety of objects. This paper presents a novel design of a flexible gripper suitable for gripping circular objects having variable curvatures and different textures. Two rubber belts form the gripper arms, which are used for gripping objects. These rubber belts are attached to DC motors. The motors are fitted in the gripper base. Magnets are attached on the other ends along with an interlocking mechanism. The primary actuation in the gripper is brought about by the Shape Memory Alloy (SMA) wire fitted along the inner side of the rubber belts. On energizing the SMA, the rubber belts bend along with the pre-programmed SMA and the magnets on the two belts come closer. Subsequently, locking is achieved. This forms a loop around the gripping object. The grip of the rubber belts around the object is further tightened by winding them around the shafts of the motors to which they are attached. This helps the gripper to firmly grasp objects of variable diameters. Gripping has been successfully tested on pipes, metal poles, trees with thin and thick stems.

@inproceedings{bib_Obje_2019, AUTHOR = {Gaud, Ayush and Harish, Y V S and Krishna, K Madhava }, TITLE = {Object Parsing in Sequences Using CoordConv Gated Recurrent Networks}, BOOKTITLE = {Technical Report}. YEAR = {2019}}

We present a monocular object parsing framework for consistent keypoint localization by capturing temporal correlation on sequential data. In this paper, we propose a novel recurrent network based architecture to model longrange dependencies between intermediate features which are highly useful in tasks like keypoint localization and tracking. We leverage the expressiveness of the popular stacked hourglass architecture and augment it by adopting memory units between intermediate layers of the network with weights shared across stages for video frames. We observe that this weight sharing scheme not only enables us to frame hourglass architecture as a recurrent network but also prove to be highly effective in producing increasingly refined estimates for sequential tasks. Furthermore, we propose a new memory cell, we call CoordConvGRU which learns to selectively preserve spatiotemporal correlation and showcase our results on the keypoint localization task. The experiments show that our approach is able to model the motion dynamics between the frames and significantly outperforms the baseline hourglass network. Even though our network is trained on a synthetically rendered dataset, we observe that with minimal fine tuning on 300 real images we are able to achieve performance at par with various state-of-the-art methods trained with the same level of supervisory inputs. By using a simpler architecture than other methods enables us to run it in real time on a standard GPU which is desirable for such applications. Finally, we make our architectures and 524 annotated sequences of cars from KITII dataset publicly available.

@inproceedings{bib_Modu_2019, AUTHOR = {, Rama Vadapalli and , Kartik Suryavanshi and , Ruchita Vucha and Sarkar, Abhishek and Krishna, K Madhava }, TITLE = {Modular Pipe Climber}, BOOKTITLE = {Advances in Robotics}. YEAR = {2019}}

This paper discusses the design and implementation of the Modular Pipe Climber inside ASTM D1785 - 15e1 standard pipes [1]. The robot has three tracks which operate independently and are mounted on three modules which are oriented at 120° to each other. Tracks provide for greater surface traction compared to wheels [2]. The tracks are pushed onto the inner wall of the pipe by passive springs to maintain contact with the pipe during vertical climb and while turning in bends. The modules have the provision to compress asymmetrically, which enables the robot to take turns in bends in all directions. The motor torque required by the robot and the desired spring stiffness are calculated at quasi-static and static equilibriums during vertical climb. The robot is further simulated and analyzed in ADAMS MSC. The prototype built based on the obtained values is experimented on, in complex pipe networks. Differential speed is employed when turning in bends to improve the efficiency and reduce the stresses experienced by the robot.

@inproceedings{bib_Omni_2019, AUTHOR = {Suryavanshi, Kartik and Vadapalli, Rama and VUCHA, RUCHITHA and Sarkar, Abhishek and Krishna, K Madhava }, TITLE = {Omnidirectional Tractable Three Module Robot}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2019}}

This paper introduces the Omnidirectional Tractable Three Module Robot for traversing inside complex pipe networks. The robot consists of three omnidirectional modules fixed 120 apart circumferentially which can rotate about their own axis allowing holonomic motion of the robot. The holonomic motion enables the robot to overcome motion singularity when negotiating T-junctions and further allows the robot to arrive in a preferred orientation while taking turns inside a pipe. We have developed a closed-form kinematic model for the robot in the paper and propose the Motion Singularity Region that the robot needs to avoid while negotiating T-junction. The design and motion capabilities of the robot are demonstrated both by conducting simulations in MSC ADAMS on a simplified lumped-model of the robot and with experiments on its physical embodiment.

@inproceedings{bib_Omni_2019, AUTHOR = {, Praharsha Budharaja and Sarkar, Abhishek and Krishna, K Madhava }, TITLE = {Omnidirectional Three Module Robot Design and Simulation }, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2019}}

This paper introduces the Omnidirectional Tractable Three Module Robot for traversing inside complex pipe networks. The robot consists of three omnidirectional modules fixed 120 apart circumferentially which can rotate about their axis allowing holonomic motion of the robot. Holonomic motion enables the robot to overcome motion singularity when negotiating T-junctions and further allows the robot to arrive in a preferred orientation while taking turns inside a pipe. The singularity region while negotiating T-junctions is analyzed to formulate the geometry of the region. The design and motion capabilities are validated by conducting simulations in MSC ADAMS on a simplified lumped-model of the robot.

@inproceedings{bib_Prob_2019, AUTHOR = {Jitendrakumar, Bhatt Dhaivat and Garg, Akash and GOPALAKRISHNAN, BHARATH and Krishna, K Madhava }, TITLE = {Probabilistic obstacle avoidance and object following: An overlap of Gaussians approach}, BOOKTITLE = {IEEE International Conference on Robot and Human Interactive Communication}. YEAR = {2019}}

Autonomous navigation and obstacle avoidance are core capabilities that enable robots to execute tasks in the real world. We propose a new approach to collision avoidance that accounts for uncertainty in the states of the agent and the obstacles. We first demonstrate that measures of entropy—used in current approaches for uncertainty-aware obstacle avoidance—are an inappropriate design choice. We then propose an algorithm that solves an optimal control sequence with a guaranteed risk bound, using a measure of overlap between the two distributions that represent the state of the robot and the obstacle, respectively. Furthermore, we provide closed form expressions that can characterize the overlap as a function of the control input. The proposed approach enables modelpredictive control framework to generate bounded-confidence control commands. An extensive set of simulations have been conducted in various constrained environments in order to demonstrate the efficacy of the proposed approach over the prior art. We demonstrate the usefulness of the proposed scheme under tight spaces where computing risk-sensitive control maneuvers is vital. We also show how this framework generalizes to other problems, such as object-following.

@inproceedings{bib_Lear_2019, AUTHOR = {Mani, Kaustubh and Kaushik, Meha and SINGHANIA, NIRVAN and Krishna, K Madhava }, TITLE = {Learning Adaptive Driving Behavior Using Recurrent Deterministic Policy Gradients}, BOOKTITLE = {International Conference on Robotics and Biomimetics}. YEAR = {2019}}

In this work, we propose adaptive driving behaviors for simulated cars using continuous control deep reinforcement learning. Deep Deterministic Policy Gradient(DDPG) is known to give smooth driving maneuvers in simulated environments. Unfortunately, simple feedforward networks, lack the capability to contain temporal information, hence we have used its Recurrent variant called Recurrent Deterministic Policy Gradients. Our trained agent adapts itself to the velocity of the traffic. It is capable of slowing down in the presence of dense traffic, to prevent collisions as well to speed up and change lanes in order to overtake when the traffic is sparse. The reasons for the above behavior, as well as, our main contributions are: 1. Application of Recurrent Deterministic Policy Gradients. 2. Novel reward function formulation. 3. Modified Replay Buffer called Near and Far Replay Buffers, wherein we maintain two replay buffers and sample equally from both of them.

@inproceedings{bib_Stud_2019, AUTHOR = {P, MITHUN and Mehta, Shaunak A. and Shah, Suril V. and Bhatnagar, Gaurav and Krishna, K Madhava }, TITLE = {Student Mixture Model Based Visual Servoing}, BOOKTITLE = {Technical Report}. YEAR = {2019}}

Classical Image-Based Visual Servoing (IBVS) makes use of geometric image features like point, straight line and image moments to control a robotic system. Robust extraction and real-time tracking of these features are crucial to the performance of the IBVS. Moreover, such features can be unsuitable for real world applications where it might not be easy to distinguish a target from rest of the environment. Alternatively, an approach based on complete photometric data can avoid the requirement of feature extraction, tracking and object detection. In this work, we propose one such probabilistic model based approach which uses entire photometric data for the purpose of visual servoing. A novel image modelling method has been proposed using Student Mixture Model (SMM), which is based on Multivariate Student’s t-Distribution. Consequently, a vision-based control law is formulated as a least squares minimisation problem. Efficacy of the proposed framework is demonstrated for 2D and 3D positioning tasks showing favourable error convergence and acceptable camera trajectories. Numerical experiments are also carried out to show robustness to distinct image scenes and partial occlusion.

@inproceedings{bib_Talk_2019, AUTHOR = {NARAYANAN, SRIRAM and Anup, Maniar Tirth and K, Jayaganesh and Gandhi, Vineet and Bhowmick, Brojeshwar and Krishna, K Madhava }, TITLE = {Talk to the Vehicle: Language Conditioned Autonomous Navigation of Self Driving Cars.}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2019}}

Abstract— We propose a novel pipeline that blends encodings from natural language and 3D semantic maps obtained from visual imagery to generate local trajectories that are executed by a low-level controller. The pipeline precludes the need for a prior registered map through a local waypoint generator neural network. The waypoint generator network (WGN) maps semantics and natural language encodings (NLE) to local waypoints. A local planner then generates a trajectory from the ego location of the vehicle (an outdoor car in this case) to these locally generated waypoints while a low-level controller executes these plans faithfully. The efficacy of the pipeline is verified in the CARLA simulator environment as well as on local semantic maps built from real-world KITTI dataset. In both these environments (simulated and real-world) we show the ability of the WGN to generate waypoints accurately by mapping NLE of varying sequence lengths and levels of complexity. We compare with baseline approaches and show significant performance gain over them. And finally, we show real implementations on our electric car verifying that the pipeline lends itself to practical and tangible realizations in uncontrolled outdoor settings. In loop execution of the proposed pipeline that involves repetitive invocations of the network is critical for any such language-based navigation framework. This effort successfully accomplishes this thereby bypassing the need for prior metric maps or strategies for metric level localization during traversal.

@inproceedings{bib_Have_2018, AUTHOR = {Bhatt, Dhaivat and SODHI, DANISH and Pal, Arghya and Balasubramanian, Vineeth and Krishna, K Madhava }, TITLE = {Have i reached the intersection: A deep learning-based approach for intersection detection from monocular cameras}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2018}}

Long-short term memory networks(LSTM) models have shown considerable performance on variety of problems dealing with sequential data. In this paper, we propose a variant of Long-Term Recurrent Convolutional Network(LRCN) to detect road intersection. We call this network as IntersectNet. We pose road intersection detection as binary classification task over sequence of frames. The model combines deep hierarchical visual feature extractor with recurrent sequence model. The model is end to end trainable with capability of capturing the temporal dynamics of the system. We exploit this capability to identify road intersection in a sequence of temporally consistent images. The model has been rigorously trained and tested on various different datasets. We think that our findings could be useful to model behavior of autonomous agent in the real-world.

@inproceedings{bib_The__2018, AUTHOR = {Ansari, Junaid Ahmed and SHARMA, SARTHAK and MAJUMDAR, ANSHUMAN and Murthy, J. Krishna and Krishna, K Madhava }, TITLE = {The Earth Ain't Flat: Monocular Reconstruction of Vehicles on Steep and Graded Roads from a Moving Camera}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2018}}

Accurate localization of other traffic participants is a vital task in autonomous driving systems. State-of-the-art systems employ a combination of sensing modalities such as RGB cameras and LiDARs for localizing traffic participants, but monocular localization demonstrations have been confined to plain roads. We demonstrate — to the best of our knowledge — the first results for monocular object localization and shape estimation on surfaces that are non-coplanar with the moving ego vehicle mounted with a monocular camera. We approximate road surfaces by local planar patches and use semantic cues from vehicles in the scene to initialize a local bundle-adjustment like procedure that simultaneously estimates the 3D pose and shape of the vehicles, and the orientation of the local ground plane on which the vehicle stands. We also demonstrate that our approach transfers from synthetic to real data, without any hyperparameter-/fine-tuning. We evaluate the proposed approach on the KITTI and SYNTHIA-SF benchmarks, for a variety of road plane configurations. The proposed approach significantly improves the state-of-the-art

@inproceedings{bib_Lear_2018, AUTHOR = {Prasad, Vignesh and Yadav, Karmesh and Saurabh, Rohitashva Singh and Daga, Swapnil Naresh and Pareekutty, Nahas and Krishna, K Madhava and Ravindran, Balaraman and Bhowmick, Brojeshwar }, TITLE = {Learning to Prevent Monocular SLAM Failure using Reinforcement Learning}, BOOKTITLE = {Indian Conference on Computer Vision, Graphics and Image Processing}. YEAR = {2018}}

Monocular SLAM refers to using a single camera to estimate robot ego motion while building a map of the environment. While Monocular SLAM is a well studied problem, automating Monocular SLAM by integrating it with trajectory planning frameworks is particularly challenging. This paper presents a novel formulation based on Reinforcement Learning (RL) that generates fail safe trajectories wherein the SLAM generated outputs do not deviate largely from their true values. Quintessentially, the RL framework successfully learns the otherwise complex relation between perceptual inputs and motor actions and uses this knowledge to generate trajectories that do not cause failure of SLAM. We show systematically in simulations how the quality of the SLAM dramatically improves when trajectories are computed using RL. Our method scales effectively across Monocular SLAM frameworks in both simulation and in real world experiments with a mobile robot.

@inproceedings{bib_Moti_2018, AUTHOR = {SANDHU, MAHTAB and Upadhyay, Sarthak and Krishna, K Madhava and Medasani, Shanti }, TITLE = {Motion Segmentation Using Spectral Clustering on Indian Road Scenes}, BOOKTITLE = {European Conference on Computer Vision}. YEAR = {2018}}

We propose a novel motion segmentation formulation over spatio-temporal depth images obtained from stereo sequences that segments multiple motion models in the scene in an unsupervised manner . The motion segmentation is obtained at frame rates that compete with the speed of the stereo depth computation. This is possible due to a decoupling framework that first delineates spatial clusters and subsequently assigns motion labels to each of these cluster with analysis of a novel motion graph model. A principled computation of the weights of the motion graph that signifies the relative shear and stretch between possible clusters lends itself to a high fidelity segmentation of the motion models in the scene. Keywords: Motion

@inproceedings{bib_Geom_2018, AUTHOR = {Murthy, J. Krishna and Iyer, Ganesh and Gupta, Gunsh and Krishna, K Madhava and Paull, Liam }, TITLE = {Geometric Consistency for Self-Supervised End-to-End Visual Odometry}, BOOKTITLE = {Computer Vision and Pattern Recognition Conference workshops}. YEAR = {2018}}

With the success of deep learning based approaches in tackling challenging problems in computer vision, a wide range of deep architectures have recently been proposed for the task of visual odometry (VO) estimation. Most of these proposed solutions rely on supervision, which requires the acquisition of precise ground-truth camera pose information, collected using expensive motion capture systems or high-precision IMU/GPS sensor rigs. In this work, we propose an unsupervised paradigm for deep visual odometry learning. We show that using a noisy teacher, which could be a standard VO pipeline, and by designing a loss term that enforces geometric consistency of the trajectory, we can train accurate deep models for VO that do not require ground-truth labels. We leverage geometry as a selfsupervisory signal and propose "Composite Transformation Constraints (CTCs)", that automatically generate supervisory signals for training and enforce geometric consistency in the VO estimate. We also present a method of characterizing the uncertainty in VO estimates thus obtained. To evaluate our VO pipeline, we present exhaustive ablation studies that demonstrate the efficacy of end-to-end, self-supervised methodologies to train deep models for monocular VO. We show that leveraging concepts from geometry and incorporating them into the training of a recurrent neural network results in performance competitive to supervised deep VO methods.

@inproceedings{bib_Lear_2018, AUTHOR = {KAUSHIK, MEHA and Krishna, K Madhava }, TITLE = {Learning Driving Behaviors for Automated Cars in Unstructured Environments}, BOOKTITLE = {European Conference on Computer Vision Workshops}. YEAR = {2018}}

The core of Reinforcement learning lies in learning from experiences. The performance of the agent is hugely impacted by the training conditions, reward functions and exploration policies. Deep Deterministic Policy Gradient(DDPG) is a well known approach to solve continuous control problems in RL. We use DDPG with intelligent choice of reward function and exploration policy to learn various driving behaviors(Lanekeeping, Overtaking, Blocking, Defensive, Opportunistic) for a simulated car in unstructured environments. In cluttered scenes, where the opponent agents are not following any driving pattern, it is difficult to anticipate their behavior and henceforth decide our agent’s actions. DDPG enables us to propose a solution which requires only the sensor information at current time step to predict the action to be taken. Our main contribution is generating a behavior based motion model for simulated cars, which plans for every instant

@inproceedings{bib_Grad_2018, AUTHOR = {Modh, Adarsh and SINGH, SIDDHARTH and KUMAR, AVULA VENKATA SEETHARAMA SAI BHARGAV and NARAYANAN, SRIRAM and Krishna, K Madhava }, TITLE = {Gradient Aware - Shrinking Domain based Control Design for Reactive Planning Frameworks used in Autonomous Vehicles}, BOOKTITLE = {Advances in Robotics}. YEAR = {2018}}

In this paper, we present a novel control law for longitudinal speed control of autonomous vehicles. The key contributions of the proposed work include the design of a control law that reactively integrates the longitudinal surface gradient of road into its operation. In contrast to the existing works, we found that integrating the path gradient into the control framework improves the speed tracking efficacy. Since the control law is implemented over a shrinking domain scheme, it minimizes the integrated error by recomputing the control inputs at every discretized step and on sequently provides less reaction time. This makes our control law suitable for motion planning frameworks that are operating at high frequencies. Furthermore, our work is implemented using a generalized vehicle model and can be easily extended to other classes of vehicles. The performance of gradient aware - shrinking domain based controller is implemented and tested on a stock electric vehicle on which a number of sensors are mounted. Results from the tests show the robustness of our control law for speed tracking on a terrain with varying gradient while also considering stringent time constraints imposed by the planning framework.

@inproceedings{bib_Cons_2018, AUTHOR = {KESHAVLAL, PARKHIYA PARV and KHAWAD, RISHABH and Murthy, J. Krishna and Bhowmick, Brojeshwar and Krishna, K Madhava }, TITLE = {Constructing Category-Specific Models for Monocular Object-SLAM}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2018}}

We present a new paradigm for real-time objectoriented SLAM with a monocular camera. Contrary to previous approaches, that rely on object-level models, we construct category-level models from CAD collections which are now widely available. To alleviate the need for huge amounts of labeled data, we develop a rendering pipeline that enables synthesis of large datasets from a limited amount of manually labeled data. Using data thus synthesized, we learn categorylevel models for object deformations in 3D, as well as discriminative object features in 2D. These category models are instance-independent and aid in the design of object landmark observations that can be incorporated into a generic monocular SLAM framework. Where typical object-SLAM approaches usually solve only for object and camera poses, we also estimate object shape on-the-fly, allowing for a wide range of objects from the category to be present in the scene. Moreover, since our 2D object features are learned discriminatively, the proposed object-SLAM system succeeds in several scenarios where sparse feature-based monocular SLAM fails due to insufficient features or parallax. Also, the proposed category models help in object instance retrieval, useful for Augmented Reality (AR) applications. We evaluate the proposed framework on multiple challenging real-world scenes and show — to the best of our knowledge — first results of an instance-independent monocular object-SLAM system and the benefits it enjoys over feature-based SLAM methods.

@inproceedings{bib_Beyo_2018, AUTHOR = {Ansari, Junaid Ahmed and SHARMA, SARTHAK and Murthy, J. Krishna and Krishna, K Madhava }, TITLE = {Beyond Pixels: Leveraging Geometry and Shape Cues for Online Multi-Object Tracking}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2018}}

This paper introduces geometry and novel object shape and pose costs for multi-object tracking in road scenes. Using images from a monocular camera alone, we devise pairwise costs for object tracks, based on several 3D cues such as object pose, shape, and motion. The proposed costs are agnostic to the data association method and can be incorporated into any optimization framework to output the pairwise data associations. These costs are easy to implement, can be computed in real-time, and complement each other to account for possible errors in a tracking-by-detection framework. We perform an extensive analysis of the designed costs and empirically demonstrate consistent improvement over the state-of-the-art under varying conditions that employ a range of object detectors, exhibit a variety in camera and object motions and, more importantly, are not reliant on the choice of the association framework. We also show that, by using the simplest of associations frameworks (two-frame Hungarian assignment),we surpass the state-of-the-art in multi-object-tracking on road scenes

@inproceedings{bib_Mode_2018, AUTHOR = {Oza, Yash and Nallana, Mithun Babu and Singh, Arun Kumar and Krishna, K Madhava and Medasani, Shanti }, TITLE = {Model Predictive Control for Autonomous Driving Based on Time Scaled Collision Cone}, BOOKTITLE = {European Control Conference}. YEAR = {2018}}

In this paper, we present a Model Predictive Control (MPC) framework based on path velocity decomposition paradigm for autonomous driving. The optimization underlying the MPC has a two layer structure wherein first, an appropriate path is computed for the vehicle followed by the computation of optimal forward velocity along it. The very nature of the proposed path velocity decomposition allows for seamless compatibility between the two layers of the optimization. A key feature of the proposed work is that it offloads most of the responsibility of collision avoidance to velocity optimization layer for which computationally efficient formulations can be derived. In particular, we extend our previously developed concept of time scaled collision cone (TSCC) constraints and formulate the forward velocity optimization layer as a convex quadratic programming problem. We perform validation on autonomous driving scenarios wherein proposed MPC repeatedly solves both the optimization layers in receding horizon manner to compute lane change, overtaking and merging maneuvers among multiple dynamic obstacles.

@inproceedings{bib_Chan_2018, AUTHOR = {Jitendrakumar, Bhatt Dhaivat and Garg, Akash and GOPALAKRISHNAN, BHARATH and Krishna, K Madhava }, TITLE = {Chance Constraints Integrated MPC Navigation in Uncertainty amongst Dynamic Obstacles: An overlap of Gaussians approach}, BOOKTITLE = {Technical Report}. YEAR = {2018}}

In this paper, we formulate a novel trajectory optimization scheme that takes into consideration the state uncertainty of the robot and obstacle into its collision avoidance routine. The collision avoidance under uncertainty is modeled here as an overlap between two distributions that represent the state of the robot and obstacle respectively. We adopt the minmax procedure to characterize the area of overlap between two Gaussian distributions, and compare it with the method of Bhattacharyya distance. We provide closed form expressions that can characterize the overlap as a function of control. Our proposed algorithm can avoid overlapping uncertainty distributions in two possible ways. Firstly when a prescribed overlapping area that needs to be avoided is posed as a confidence contour lower bound, control commands are accordingly realized through a MPC framework such that these bounds are respected. Secondly in tight spaces control commands are computed such that the overlapping distribution respects a prescribed range of overlap characterized by lower and upper bounds of the confidence contours. We test our proposal with extensive set of simulations carried out under various constrained environmental configurations. We show usefulness of proposal under tight spaces where finding control maneuvers with minimal risk behavior becomes an inevitable task.

@inproceedings{bib_Fast_2018, AUTHOR = {SANDHU, MAHTAB and ATHAR, NAZRUL HAQUE and SHARMA, AVINASH and Krishna, K Madhava and Medasani, Shanti }, TITLE = {Fast Multi Model Motion Segmentation on Road Scenes}, BOOKTITLE = {Intelligent Vehicles symposium}. YEAR = {2018}}

We propose a novel motion clustering formulation over spatio-temporal depth images obtained from stereo sequences that segments multiple motion models in the scene in an unsupervised manner. The motion models are obtained at frame rates that compete with the speed of the stereo depth computation. This is possible due to a decoupling framework that first delineates spatial clusters and subsequently assigns motion labels to each of these cluster with analysis of a novel motion graph model. A principled computation of the weights of the motion graph that signifies the relative shear and stretch between possible clusters lends itself to a high fidelity segmentation of the motion models in the scene. The fidelity is vindicated through accuracies reaching 89.61% on KITTI and complex native sequences.

@inproceedings{bib_A_No_2018, AUTHOR = {KUMAR, AVULA VENKATA SEETHARAMA SAI BHARGAV and Modh, Adarsh and Nallana, Mithun Babu and GOPALAKRISHNAN, BHARATH and Krishna, K Madhava }, TITLE = {A Novel Lane Merging Framework with Probabilistic Risk based Lane Selection using Time Scaled Collision Cone}, BOOKTITLE = {Intelligent Vehicles symposium}. YEAR = {2018}}

Conventionally, planning frameworks for autonomous vehicles consider large safety margins and predefined paths for performing the merge maneuvers. These considerations often increase the wait time at the intersections leading to traffic disruption. In this paper, we present a motion planning framework for autonomous vehicles to perform merge maneuver in dense traffic. Our framework is divided into a two-layer structure, Lane Selection layer and Scale Optimization layer. The Lane Selection layer computes the likelihood of collision along the lanes. This likelihood represents the collision risk associated with each lane and is used for lane selection. Subsequently, the Scale Optimization layer solves the time scaled collision cone (TSCC) constraint reactively for collision-free velocities. Our framework guarantees a collision-free merging even in dense traffic with minimum disruption. Furthermore, we show the simulation results in different merging scenarios to demonstrate the efficacy of our framework

@inproceedings{bib_Over_2018, AUTHOR = {KAUSHIK, MEHA and Prasad, Vignesh and Krishna, K Madhava and Ravindran, Balaraman }, TITLE = {Overtaking Maneuvers in Simulated Highway Driving using Deep Reinforcement Learning}, BOOKTITLE = {Intelligent Vehicles symposium}. YEAR = {2018}}

Most methods that attempt to tackle the problem of Autonomous Driving and overtaking usually try to either directly minimize an objective function or iteratively in a Reinforcement Learning like framework to generate motor actions given a set of inputs. We follow a similar trend but train the agent in a way similar to a curriculum learning approach where the agent is first given an easier problem to solve, followed by a harder problem. We use Deep Deterministic Policy Gradients to learn overtaking maneuvers for a car, in presence of multiple other cars, in a simulated highway scenario. The novelty of our approach lies in the training strategy used where we teach the agent to drive in a manner similar to the way humans learn to drive and the fact that our reward function uses only the raw sensor data at the current time step. This method, which resembles a curriculum learning approach is able to learn smooth maneuvers, largely collision free, wherein the agent overtakes all other cars, independent of the track and number of cars in the scene.

@inproceedings{bib_The__2018, AUTHOR = {Ansari, Junaid Ahmed and SHARMA, SARTHAK and MAJUMDAR, ANSHUMAN and Murthy, J. Krishna and Krishna, K Madhava }, TITLE = {The Earth ain’t Flat: Monocular Reconstruction of Vehicles on Steep and Graded Roads from a Moving Camera}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2018}}

Accurate localization of other traffic participants is a vital task in autonomous driving systems. State-of-the-art systems employ a combination of sensing modalities such as RGB cameras and LiDARs for localizing traffic participants, but most such emonstrations have been confined to plain roads. We demonstrate, to the best of our knowledge, the first results for monocular object localization and shape estimation on surfaces that do not share the same plane with the moving monocular camera. We approximate road surfaces by local planar patches and use semantic cues from vehicles in the scene to initialize a local bundle-adjustment like procedure that simultaneously estimates the pose and shape of the vehicles, and the orientation of the local ground plane on which the vehicle stands as well. We evaluate the proposed approach on the KITTI and SYNTHIA-SF benchmarks, for a variety of road plane configurations. The proposed approach significantly improves the state-of-the-art for monocular object localization on arbitrarily-shaped roads.

@inproceedings{bib_Imag_2018, AUTHOR = {P, MITHUN and PANDYA, HARIT and Gaud, Ayush and Shah, Suril V. and Krishna, K Madhava }, TITLE = {Image Based Visual Servoing for Tumbling Objects}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2018}}

Objects in space often exhibit a tumbling motion around the major inertial axis. In this paper, we address the image based visual servoing of a robotic system towards an uncooperative tumbling object. In contrast to previous approaches that require explicit reconstruction of the object and an estimation of its velocity, we propose a novel controller that is able to minimize the feature error directly in image space. This is achieved by observing that the feature points on the tumbling object follow a circular path around the axis of rotation and their projection creates an elliptical track in the image plane. Our controller minimizes the error between this elliptical track and the desired features, such that at the desired pose the features lie on the circumference of the ellipse. The effectiveness of our framework is exhibited by implementing the algorithm in simulation as well on a mobile robot.

@inproceedings{bib_Cali_2018, AUTHOR = {Iyer, Ganesh and R, Karnik Ram and Murthy, J. Krishna and Krishna, K Madhava }, TITLE = {CalibNet: Geometrically Supervised Extrinsic Calibration using 3D Spatial Transformer Networks}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2018}}

3D LiDARs and 2D cameras are increasingly being used alongside each other in sensor rigs for perception tasks. Before these sensors can be used to gather meaningful data, however, their extrinsics (and intrinsics) need to be accurately calibrated, as the performance of the sensor rig is extremely sensitive to these calibration parameters. A vast majority of existing calibration techniques require significant amounts of data and/or calibration targets and human effort, severely impacting their applicability in large-scale production systems. We address this gap with CalibNet: a geometrically supervised deep network capable of automatically estimating the 6-DoF rigid body transformation between a 3D LiDAR and a 2D camera in real-time. CalibNet alleviates the need for calibration targets, thereby resulting in significant savings in calibration efforts. During training, the network only takes as input a LiDAR point cloud, the corresponding monocular image, and the camera calibration matrix K. At train time, we do not impose direct supervision (i.e., we do not directly regress to the calibration parameters, for example). Instead, we train the network to predict calibration parameters that maximize the geometric and photometric consistency of the input images and point clouds. CalibNet learns to iteratively solve the underlying geometric problem and accurately predicts extrinsic calibration parameters for a wide range of mis-calibrations, without requiring retraining or domain adaptation.

@inproceedings{bib_Solv_2018, AUTHOR = {GOPALAKRISHNAN, BHARATH and SINGH, ARUN KUMAR and Krishna, K Madhava and M, DINESH KUMAR }, TITLE = {Solving Chance Constrained Optimization under Non-Parametric Uncertainty Through Hilbert Space Embedding}, BOOKTITLE = {IEEE Transactions on Control Systems Technology}. YEAR = {2018}}

In this paper, we present an efficient algorithm for solving a class of chance constrained optimization under non-parametric uncertainty. Our algorithm is built on the possibility of representing arbitrary distributions as functions in Reproducing Kernel Hilbert Space (RKHS). We use this foundation to formulate chance-constrained optimization as one of minimizing the distance between a desired distribution and the distribution of the constraint functions in the RKHS. We provide a systematic way of constructing the desired distribution based on a notion of scenario proximation. Furthermore, we use the kernel trick to show that the computational complexity of our reformulated optimization problem is comparable to solving a deterministic variant of the chance-constrained optimization. We validate our formulation on two important robotic/control applications: (i) reactive collision avoidance of mobile robots in uncertain dynamic environments and (ii) inverse dynamics based path tracking of manipulators under perception uncertainty. In both these applications, the underlying chance constraints are defined over highly non-linear and non-convex functions of the uncertain parameters and possibly also decision variables. We also benchmark our formulation with the existing approaches in terms of sample complexity and the achieved optimal cost highlighting significant improvements in both these metrics.

@inproceedings{bib_DeCo_2018, AUTHOR = {BARANWAL, AKANKSHA and Bansal, Ishan and NAHAR, ROOPAL and Krishna, K Madhava }, TITLE = {DeCoILFNet: Depth Concatenation and Inter-Layer Fusion based ConvNet Accelerator}, BOOKTITLE = {Technical Report}. YEAR = {2018}}

Convolutional Neural Networks (CNNs) are rapidly gaining popularity in varied fields. Due to their increasingly deep and computationally heavy structures, it is difficult to deploy them on energy constrained mobile applications. Hardware accelerators such as FPGAs have come up as an attractive alternative. However, with the limited on-chip memory and computation resources of FPGA, meeting the high memory throughput requirement and exploiting the parallelism of CNNs is a major challenge. We propose a high-performance FPGA based architecture - Depth Concatenation and Inter-Layer Fusion based ConvNet Accelerator - DeCoILFNet which exploits the intra-layer parallelism of CNNs by flattening across depth and combines it with a highly pipelined data flow across the layers enabling inter-layer fusion. This architecture significantly reduces off-chip memory accesses and maximizes the throughput. Compared to a 3.5GHz hexa-core Intel Xeon E7 caffe-implementation, our 120MHz FPGA accelerator is 30X faster. In addition, our design reduces external memory access by 11.5X along with a speedup of more than 2X in the number of clock cycles compared to state-of-the-art FPGA accelerators.

@inproceedings{bib_Inte_2018, AUTHOR = {JOSHI, NAYAN and SHARMA, YOGESH and KESHAVLAL, PARKHIYA PARV and KHAWAD, RISHABH and Krishna, K Madhava and Bhowmick, Brojeshwar }, TITLE = {Integrating Objects into Monocular SLAM:Line Based Category Specific Models}, BOOKTITLE = {Indian Conference on Computer Vision, Graphics and Image Processing}. YEAR = {2018}}

We propose a novel Line based parameterization for category specific CAD models. The proposed parameterization associates 3D category-specific CAD model and object under consideration using a dictionary based RANSAC method that uses object Viewpoints as prior and edges detected in the respective intensity image of the scene. The association problem is posed as a classical Geometry problem rather than being dataset driven, thus saving the time and labour that one invests in annotating dataset to train Keypoint Network[1, 2] for different category objects. Besides eliminating the need of dataset preparation, the approach also speeds up the entire process as this method processes the image only once for all objects, thus eliminating the need of invoking the network for every object in an image across all images. A 3D-2D edge association module followed by a resection algorithm for lines is used to recover object poses. The formulation optimizes for shape and pose of the object, thus aiding in recovering object 3D structure more accurately. Finally, a Factor Graph formulation is used to combine object poses with camera odometry to formulate a SLAM problem.

@inproceedings{bib_Digr_2018, AUTHOR = {DEWANGAN, PARIJAT and TEJA, SINGAMANENI PHANI and Krishna, K Madhava and Sarkar, Abhishek and Ravindran, Balaraman }, TITLE = {Digrad: Multi-task reinforcement learning with shared actions}, BOOKTITLE = {Technical Report}. YEAR = {2018}}

Most reinforcement learning algorithms are inefficient for learning multiple tasks in complex robotic systems, where different tasks share a set of actions. In such environments a compound policy may be learnt with shared neural network parameters, which performs multiple tasks concurrently. However such compound policy may get biased towards a task or the gradients from different tasks negate each other, making the learning unstable and sometimes less data efficient. In this paper, we propose a new approach for simultaneous training of multiple tasks sharing a set of common actions in continuous action spaces, which we call as DiGrad (Differential Policy Gradient). The proposed framework is based on differential policy gradients and can accommodate multi-task learning in a single actor-critic network. We also propose a simple heuristic in the differential policy gradient update to further improve the learning. The proposed architecture was tested on 8 link planar manipulator and 27 degrees of freedom (DoF) Humanoid for learning multi-goal reachability tasks for 3 and 2 end effectors respectively. We show that our approach supports efficient multi-task learning in complex robotic systems, outperforming related methods in continuous action spaces.

@inproceedings{bib_Lear_2018, AUTHOR = {TEJA, SINGAMANENI PHANI and DEWANGAN, PARIJAT and Guhan, Pooja and Krishna, K Madhava and Sarkar, Abhishek }, TITLE = {Learning Coordinated Tasks using Reinforcement Learning in Humanoids}, BOOKTITLE = {Technical Report}. YEAR = {2018}}

With the advent of artificial intelligence and machine learning, humanoid robots are made to learn a variety of skills which humans possess. One of fundamental skills which humans use in day-to-day activities is performing tasks with coordination between both the hands. In case of humanoids, learning such skills require optimal motion planning which includes avoiding collisions with the surroundings. In this paper, we propose a framework to learn coordinated tasks in cluttered environments based on DiGrad - A multi-task reinforcement learning algorithm for continuous action-spaces. Further, we propose an algorithm to smooth the joint space trajectories obtained by the proposed framework in order to reduce the noise instilled during training. The proposed framework was tested on a 27 degrees of freedom (DoF) humanoid with articulated torso for performing coordinated object-reaching task with both the hands in four different environments with varying levels of difficulty. It is observed that the humanoid is able to plan collision free trajectory in real-time. Simulation results also reveal the usefulness of the articulated torso for performing tasks which require coordination between both the arms.

@inproceedings{bib_Lear_2018, AUTHOR = {TEJA, SINGAMANENI PHANI and DEWANGAN, PARIJAT and Sarkar, Abhishek and Krishna, K Madhava }, TITLE = {Learning Multi-Goal Inverse Kinematics in Humanoid Robot}, BOOKTITLE = {International Symposium on Robotics}. YEAR = {2018}}

General Inverse Kinematic (IK) solvers may not guarantee real-time control of the end-effectors in external coordinates along with maintaining stability. This work addresses this problem by using Reinforcement Learning (RL) for learning an inverse kinematics solver for reachability tasks which ensures stability and self-collision avoidance while solving for end effectors. We propose an actor-critic based algorithm to learn joint space trajectories of stable configuration for solving inverse kinematics that can operate over continuous action spaces. Our approach is based on the idea of exploring the entire workspace and learning the best possible configurations. The proposed strategy was evaluated on the highly articulated upper body of a 27 degrees of freedom (DoF) humanoid for learning multi-goal reachability tasks of both hands along with maintaining stability in double support phase. We show that the trained model was able to solve inverse kinematics for both the hands, where the articulated torso contributed to both the tasks.

@inproceedings{bib_Lear_2018, AUTHOR = {TEJA, SINGAMANENI PHANI and DEWANGAN, PARIJAT and Guhan, Pooja and Krishna, K Madhava and Sarkar, Abhishek }, TITLE = {Learning dual arm coordinated reachability tasks in a humanoid robot with articulated torso}, BOOKTITLE = {International Conference on Humanoid Robots}. YEAR = {2018}}

Performing dual arm coordinated (reachability)tasks in humanoid robots require complex planning strategies and this complexity increases further, in case of humanoids with articulated torso. These complex strategies may not be suitable for online motion planning. This paper proposes a faster way to accomplish dual arm coordinated tasks using methodology based on Reinforcement Learning. The contribution of this paper is twofold. Firstly, we propose DiGrad (Differential Gradients), a new RL framework for multi-task learning in manipulators. Secondly, we show how this framework can be adopted to learn dual arm coordination in a 27 degrees of freedom (DOF)humanoid robot with articulated spine. The proposed framework and methodology are evaluated in various environments and simulation results are presented. A comparative study of DiGrad with its parent algorithm in different settings is also presented

@inproceedings{bib_Towa_2018, AUTHOR = {KUMAR, ABHIJEET and Gupta, Gunshi and Sharma, Avinash and Krishna, K Madhava }, TITLE = {Towards View-Invariant Intersection Recognition from Videos using Deep Network Ensembles}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2018}}

This paper strives to answer the following question: Is it possible to recognize an intersection when seen from different road segments that constitute the intersection? An intersection or a junction typically is a meeting point of three or four road segments. Its recognition from a road segment that is transverse to or 180 degrees apart from its previous sighting is an extremely challenging and yet a very relevant problem to be addressed from the point of view of both autonomous driving as well as loop detection. This paper formulates this as a problem of video recognition and proposes a novel LSTM based Siamese style deep network for video recognition. For what is indeed a challenging problem and the limited annotated dataset available we show competitive results of recognizing intersections when approached from diverse viewpoints or road segments. Specifically, we tabulate effective recognition accuracy even …

@inproceedings{bib_Merg_2018, AUTHOR = {GUPTA, KRISHNAM and Javed, Syed Ashar and Gandhi, Vineet and Krishna, K Madhava }, TITLE = {MergeNet: A Deep Net Architecture for Small Obstacle Discovery}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2018}}

Abstract— We present here, a novel network architecture called MergeNet for discovering small obstacles for on-road scenes in the context of autonomous driving. The basis of the architecture rests on the central consideration of training with less amount of data since the physical setup and the annotation process for small obstacles is hard to scale. For making effective use of the limited data, we propose a multi-stage training procedure involving weight-sharing, separate learning of low and high level features from the RGBD input and a refining stage which learns to fuse the obtained complementary features. The model is trained and evaluated on the Lost and Found dataset and is able to achieve state-of-art results with just 135 images in comparison to the 1000 images used by the previous benchmark. Additionally, we also compare our results with recent methods trained on 6000 images and show that our method achieves comparable performance with only 1000 training samples.

@inproceedings{bib_An_o_2017, AUTHOR = {Siravuru, Avinash and Shah, Suril V and Krishna, K Madhava }, TITLE = {An optimal wheel-torque control on a compliant modular robot for wheel-slip minimization}, BOOKTITLE = {Robotica}. YEAR = {2017}}

This paper discusses the development of an optimal wheel-torque controller for a compliant modular robot. The wheel actuators are the only actively controllable elements in this robot. For this type of robots, wheel-slip could offer a lot of hindrance while traversing on uneven terrains. Therefore, an effective wheel-torque controller is desired that will also improve the wheel-odometry and minimize power consumption. In this work, an optimal wheel-torque controller is proposed that minimizes the traction-to-normal force ratios of all the wheels at every instant of its motion. This ensures that, at every wheel, the least traction force per unit normal force is applied to maintain static stability and desired wheel speed. The lower this is, in comparison to the actual friction coefficient of the wheel-ground interface, the more margin of slip-free motion the robot can have. This formalism best exploits the redundancy offered by a modularly designed robot. This is the key novelty of this work. Extensive numerical and experimental studies were carried out to validate this controller. The robot was tested on four different surfaces and we report an overall average slip reduction of 44% and mean wheel-torque reduction by 16%.

@inproceedings{bib_Join_2017, AUTHOR = {ATHAR, NAZRUL HAQUE and REDDY, NARAPUREDDY DINESH and Krishna, K Madhava }, TITLE = {Joint semantic and motion segmentation for dynamic scenes using deep convolutional networks}, BOOKTITLE = {International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applicat}. YEAR = {2017}}

Dynamic scene understanding is a challenging problem and motion segmentation plays a crucial role in solving it. Incorporating semantics and motion enhances the overall perception of the dynamic scene. For applications of outdoor robotic navigation, joint learning methods have not been extensively used for extracting spatio-temporal features or adding different priors into the formulation. The task becomes even more challenging without stereo information being incorporated. This paper proposes an approach to fuse semantic features and motion clues using CNNs, to address the problem of monocular semantic motion segmentation. We deduce semantic and motion labels by integrating optical flow as a constraint with semantic features into dilated convolution network. The pipeline consists of three main stages ie Feature extraction, Feature amplification and Multi Scale Context Aggregation to fuse the semantics and flow features. Our joint formulation shows significant improvements in monocular motion segmentation over the state of the art methods on challenging KITTI tracking dataset.

@inproceedings{bib_Reac_2017, AUTHOR = {Hafez, A.H. Abdul and , P. Mithun and ANURAG, V V and Shah, S. V. and Krishna, K Madhava }, TITLE = {Reactionless visual servoing of a multi-arm space robot combined with other manipulation tasks}, BOOKTITLE = {Robotics and Autonomous systems}. YEAR = {2017}}

This paper presents a novel and generic reactionless visual servo controller for a satellite-based multi-arm space robot. The controller is designed to complete the task of visually servoing the robot’s end-effectors to a desired pose, while maintaining minimum attitude disturbance on the base-satellite. Task function approach is utilized to coordinate the servoing process and attitude of the base satellite. A redundancy formulation is used to define the tasks. The visual serving task is defined as a primary task, while regulating attitude of the base satellite to zero is defined as a secondary task. The secondary task is defined through a quadratic optimization problem, in such a way that it does not affect the primary task, and simultaneously minimizes its cost function. Stability analysis of the proposed control methodology is also discussed. A set of numerical experiments are carried out on different multi-arm space robotic systems. These systems are a planar dual-arm robot, a spatial dual-arm robot, and a three-arm planar robot. The results of the simulation experiments show efficacy, generality and applicability of the proposed control methodology.

@inproceedings{bib_Data_2017, AUTHOR = {PRASAD, VIGNESH and Jangir, Rishabh and Balaraman, Ravindran and Krishna, K Madhava }, TITLE = {Data Driven Strategies for Active Monocular SLAM using Inverse Reinforcement Learning}, BOOKTITLE = {International Conference on Autonomous Agents and Multiagent Systems}. YEAR = {2017}}

Learning a complex task like robot maneuver while preventing Monocular SLAM failure is challenging for both robots and humans. We devise a computational model for representing and inferring strategies for this task, formulated as a Markov Decision Process (MDP). We show how the reward function can be learned using Inverse Reinforcement Learning. The resulting framework allows us to understand how chosen parameters affect the quality of Monocular SLAM. A significant improvement in performance as compared to other state-of-the-art methods is also shown.

@inproceedings{bib_LiDA_2017, AUTHOR = {Dhall, Ankit and Chelan, Kunal and Radhakrishnan, Vishnu and Krishna, K Madhava }, TITLE = {LiDAR-camera calibration using 3D-3D point correspondences}, BOOKTITLE = {Technical Report}. YEAR = {2017}}

With the advent of autonomous vehicles, LiDAR and cameras have become an indispensable combination of sensors. They both provide rich and complementary data which can be used by various algorithms and machine learning to sense and make vital inferences about the surroundings. We propose a novel pipeline and experimental setup to find accurate rigid-body transformation for extrinsically calibrating a LiDAR and a camera. The pipeling uses 3D-3D point correspondences in LiDAR and camera frame and gives a closed form solution. We further show the accuracy of the estimate by fusing point clouds from two stereo cameras which align perfectly with the rotation and translation estimated by our method, confirming the accuracy of our method's estimates both mathematically and visually. Taking our idea of extrinsic LiDAR-camera calibration forward, we demonstrate how two cameras with no overlapping field-of-view can also be calibrated extrinsically using 3D point correspondences. The code has been made available as open-source software in the form of a ROS package, more information about which can be sought here: this https URL.

@inproceedings{bib_Reco_2017, AUTHOR = {MURTHY, JATAVALLABHULA KRISHNA and KRISHNA, G V SAI and CHHAYA, FALAK PIYUSHBHAI and Krishna, K Madhava }, TITLE = {Reconstructing vehicles from a single image: Shape priors for road scene understanding}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2017}}

We present an approach for reconstructing vehicles from a single (RGB) image, in the context of autonomous driving. Though the problem appears to be ill-posed, we demonstrate that prior knowledge about how 3D shapes of vehicles project to an image can be used to reason about the reverse process, i.e., how shapes (back-)project from 2D to 3D. We encode this knowledge in shape priors, which are learnt over a small keypoint-annotated dataset. We then formulate a shape-aware adjustment problem that uses the learnt shape priors to recover the 3D pose and shape of a query object from an image. For shape representation and inference, we leverage recent successes of Convolutional Neural Networks (CNNs) for the task of object and keypoint localization, and train a novel cascaded fully-convolutional architecture to localize vehicle keypoints in images. The shape-aware adjustment then robustly recovers shape (3D locations of the detected keypoints) while simultaneously filling in occluded keypoints. To tackle estimation errors incurred due to erroneously detected keypoints, we use an Iteratively Re-weighted Least Squares (IRLS) scheme for robust optimization, and as a by-product characterize noise models for each predicted keypoint. We evaluate our approach on autonomous driving benchmarks, and present superior results to existing monocular, as well as stereo approaches.

@inproceedings{bib_Expl_2017, AUTHOR = {Saxena, Aseem and PANDYA, HARIT and Kumar, Gourav and Gaud, Ayush and Krishna, K Madhava }, TITLE = {Exploring convolutional networks for end-to-end visual servoing}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2017}}

Present image based visual servoing approaches rely on extracting hand crafted visual features from an image. Choosing the right set of features is important as it directly affects the performance of any approach. Motivated by recent breakthroughs in performance of data driven methods on recognition and localization tasks, we aim to learn visual feature representations suitable for servoing tasks in unstructured and unknown environments. In this paper, we present an end-to-end learning based approach for visual servoing in diverse scenes where the knowledge of camera parameters and scene geometry is not available a priori. This is achieved by training a convolutional neural network over color images with synchronised camera poses. Through experiments performed in simulation and on a quadrotor, we demonstrate the efficacy and robustness of our approach for a wide range of camera poses in both indoor as well as outdoor environments.

@inproceedings{bib_Deta_2017, AUTHOR = {HARSHA, TURLAPATI SRI and Srivastava, Ankur and Krishna, K Madhava and Shah, Suril V }, TITLE = {Detachable modular robot capable of cooperative climbing and multi agent exploration}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2017}}

At the cross section of the fields of Uneven Terrain Navigation and Multi Agent Systems (MAS), in this work, a Detachable Compliant Modular Robot (DCMR) which can perform concurrent scene exploration by detaching into numerous parts, while preserving its ability to climb stairs is proposed and built. A spring is designed and used in the modular robot taking the worst-case-scenario of stairs encountered in an urban setting. In addition to the actuators at the wheels, an additional set of actuators per module are introduced to enable the detachment and re-attachment. The design additions and their trade-offs are discussed. Potential applications are presented with special focus on improving coverage of a map with obstacles/slabs large enough to merit exploration by climbing them. The problem of turning in crammed spaces is solved using the ability to detach of DCMR. The detaching & re-attaching capability, and stair climbing of the composite modular robot are demonstrated through experimentation using the prototype

@inproceedings{bib_Dete_2017, AUTHOR = {DIPAKKUMAR, SHAH UTSAV and KHAWAD, RISHABH and Krishna, K Madhava }, TITLE = {Detecting, localizing, and recognizing trees with a monocular MAV: Towards preventing deforestation}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2017}}

We propose a novel pipeline for detecting, localizing, and recognizing trees with a quadcoptor equipped with monocular camera. The quadcoptor flies in an area of semidense plantation filled with many trees of more than 5 meter in height. Trees are detected on a per frame basis using state of the art Convolutional Neural Networks inspired by recent rapid advancements showcased in Deep Learning literature. Once detected, the trees are tagged with a GPS coordinate through our global localizing and positioning framework. Further the localized trees are segmented, characterized by feature descriptors, and stored in a database by their GPS coordinates. In a subsequent run in the same area, the trees that get detected are queried to the database and get associated with the trees in the database. The association problem is posed as a dynamic programming problem and the optimal association is inferred. The algorithm has been verified in various zones in our campus infested with trees with varying density on the Bebop 2 drone equipped with omnidirectional vision. High percentage of successful recognition and association of the trees between two or more runs is the cornerstone of this effort. The proposed method is also able to identify if trees are missing from their expected GPS tagged locations thereby making it possible to immediately alert concerned authorities about possible unlawful felling of trees. We also propose a novel way of obtaining dense disparity map for quadcopter with monocular camera.

@inproceedings{bib_Desi_2017, AUTHOR = {Singh, Akash and Sachdeva, Enna and Sarkar, Abhishek and Krishna, K Madhava }, TITLE = {Design and optimal springs stiffness estimation of a Modular OmniCrawler in-pipe climbing Robot}, BOOKTITLE = {International Conference on Robotics and Biomimetics}. YEAR = {2017}}

This paper discusses the design of a novel compliant in-pipe climbing modular robot for small diameter pipes. The robot consists of a kinematic chain of 3 OmniCrawler modules with a link connected in between 2 adjacent modules via compliant joints. While the tank-like crawler mechanism provides good traction on low friction surfaces, its circular cross-section makes it holonomic. The holonomic motion assists it to re-align in a direction to avoid obstacles during motion as well as overcome turns with a minimal energy posture. Additionally, the modularity enables it to negotiate T-junction without motion singularity. The compliance is realized using 4 torsion springs incorporated in joints joining 3 modules with 2 links. For a desirable pipe diameter (textØ 75mm), the springs' stiffness values are obtained by formulating a constraint optimization problem which has been simulated in ADAMS MSC and further validated on a real robot prototype. In order to negotiate smooth vertical bends and friction coefficient variations in pipes, the design was later modified by replacing springs with series elastic actuators (SEA) at 2 of the 4 joints.

@inproceedings{bib_Chan_2017, AUTHOR = {GOPALAKRISHNAN, BHARATH and SINGH, ARUN KUMAR and KAUSHIK, MEHA and Krishna, K Madhava and Manocha, Dinesh }, TITLE = {Chance constraint based multi agent navigation under uncertainty}, BOOKTITLE = {Advances in Robotics}. YEAR = {2017}}

In this paper, we present an algorithm for navigating multiple robots under perception and ego-motion uncertainty. Our approach is based on the concept of the Reciprocal Velocity Obstacle which defines a set of constraints for characterizing the space of collision avoidance velocities available to each robot at a given instant in a multi-robot setting. We present a probabilistic variant of RVO obtained by defining chance constraints over the deterministic RVO constraints. Since chance constraints are in general computationally intractable, we present a family of surrogate constraints that can be used as a substitution for the original chance constraints. We show that satisfaction of surrogate constraints ensures satisfaction of original chance constraints with a specific low bound probability. We validate our formulations through numerical simulations in which we highlight the advantages of the proposed formulation over the existing methods, which handle the effect of uncertainty by using conservative bounding volumes.

@inproceedings{bib_Grap_2017, AUTHOR = {PANDYA, HARIT and Krishna, K Madhava }, TITLE = {Graph Based Visual Servoing for Object Category}, BOOKTITLE = {Advances in Robotics}. YEAR = {2017}}

In this paper we consider the problem of servoing across different instances of an object category, in which given any exemplar from an object category the robot is required to attain a desired pose. The problem becomes relevant in practical scenarios where robots are entailed to handle a wide range of objects. The challenge here is to address the large intra-category variation in the shape of object instances. We propose a two-phase graph based visual servoing (GBVS) framework for instance invariant visual servoing. The first offline phase consists of constructing a dense graph from a large dataset of images of numerous object instances viewed under various camera poses. The vertices in the graph are images themselves and the edges represent visual servoing trajectory length predicted by our metric learning framework. The second online step requires computation of the shortest path and navigation over it through a succession of image based visual servoing (IBVS) manoeuvres. By considering cup as running example to represent an object category we validate the our approach qualitatively on images downloaded from Internet and quantitatively in terms of camera pose error on synthetic images. We report translation and rotation errors under 11% and 13% respectively.

@inproceedings{bib_COCr_2017, AUTHOR = {Singh, Akash and Sachdeva, Enna and Sarkar, Abhishek and Krishna, K Madhava }, TITLE = {COCrIP: Compliant OmniCrawler in-pipeline robot}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2017}}

This paper presents a modular in-pipeline climbing robot with a novel compliant foldable OmniCrawler mechanism. The robot has a series of 3 compliant foldable OmniCrawler modules interconnected by links via passive joints. The circular cross-section of the module enables a holonomic motion to facilitate the alignment of the robot in the direction of bends. Additionally, the crawler mechanism provides a fair amount of traction, even on slippery pipe surfaces. These advantages of crawler modules have been further augmented by incorporating active compliance in the module, which helps to negotiate sharp bends in small diameter pipes. Introducing compliance in the crawler module with a single chain-lugs assembly is the the key novelty of this design. For the desirable pipe diameter and curvature of the bends, the spring stiffness value for each passive joint is determined by formulating a constrained optimization problem using the quasi-static model of the robot. Moreover, a minimum friction coefficient value between the module-pipe surface which can be vertically climbed by the robot without slipping is estimated. The numerical simulation results have further been validated by experiments on real robot prototype.

@inproceedings{bib_Prvo_2017, AUTHOR = {GOPALAKRISHNAN, BHARATH and SINGH, ARUN KUMAR and KAUSHIK, MEHA and Krishna, K Madhava and Manocha, Dinesh }, TITLE = {Prvo: Probabilistic reciprocal velocity obstacle for multi robot navigation under uncertainty}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2017}}

We present PRVO, a probabilistic variant of Reciprocal Velocity Obstacle (RVO) for decentralized multi-robot navigation under uncertainty. PRVO characterizes the space of velocities that would allow each robot to fulfill its share in collision avoidance with a specified probability. PRVO is modeled as chance constraints over the velocity level constraints defined by RVO and takes into account the uncertainty associated with both state estimation as well as the actuation of each robot. Since chance constraints are in general computationally intractable, we propose a series of reformulations which when combined with time scaling based concepts leads to a closed form characterization of solution space of PRVO for a given probability of collision avoidance. We validate our formulation through numerical simulations in which we highlight the advantages of PRVO over the related existing formulations

@inproceedings{bib_Shap_2017, AUTHOR = {MURTHY, JATAVALLABHULA KRISHNA and SHARMA, SARTHAK and Krishna, K Madhava }, TITLE = {Shape priors for real-time monocular object localization in dynamic environments}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2017}}

Reconstruction of dynamic objects in a scene is a highly challenging problem in the context of SLAM. In this paper, we present a real-time monocular object localization system that estimates the shape and pose of dynamic objects in real-time, using video frames captured from a moving monocular camera. Although the problem seems to be ill-posed, we demonstrate that, by incorporating prior knowledge of the object category, we can obtain more detailed instance-level reconstructions. As opposed to earlier object model specifications, the proposed shape-prior model leads to the formulation of a Bundle Adjustment-like optimization problem for simultaneous shape and pose estimation. Leveraging recent successes of Convolutional Neural Networks (CNNs) for object keypoint localization, we present a CNN architecture that performs precise keypoint localization. We then demonstrate how these keypoints can be used to recover 3D object properties, while accounting for any 2D localization errors and self-occlusion. We show significant performance improvements compared to state-of-the-art monocular competitors for 2D keypoint detection, as well as 3D localization and reconstruction of dynamic objects.

@inproceedings{bib_Mult_2017, AUTHOR = {DATTA, SAYANTAN and Sharma, Avinash and Krishna, K Madhava }, TITLE = {Multi-trajectory pose correspondences using scale-dependent topological analysis of pose-graphs}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2017}}

This paper considers the problem of finding pose matches between trajectories of multiple robots in their respective coordinate frames or equivalent matches between trajectories obtained during different sessions. Pose correspondences between trajectories are mediated by common landmarks represented in a topological map lacking distinct metric coordinates. Despite such lack of explicit metric level associations, we mine preliminary pose level correspondences between trajectories through a novel multi-scale heat-kernel descriptor and correspondence graph framework. These serve as an improved initialization for ICP (Iterative Closest Point) to yield dense pose correspondences. We perform extensive analysis of the proposed method under varying levels of pose and landmark noise and showcase its superiority in obtaining pose matches in comparison with standard ICP like methods. To the best of our knowledge, this is the first work of the kind that brings in elements from spectral graph theory to solve the problem of pose correspondences in a multi-robotic setting and differentiates itself from other works.

@inproceedings{bib_Pose_2017, AUTHOR = {Kumar, Gourav and PANDYA, HARIT and Gaud, Ayush and Krishna, K Madhava }, TITLE = {Pose induction for visual servoing to a novel object instance}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2017}}

Present visual servoing approaches are instance specific i.e. they control camera motion between two views of the same object. However, in practical scenarios where a robot is required to handle various instances of a category, classical visual servoing techniques are less suitable. We formulate across instance visual servoing as a pose induction and pose alignment problem. Initially, the desired pose given for any known instance is transferred to the novel instance through pose induction. Then the pose alignment problem is solved by estimating the current pose using the part aware keypoints reconstruction followed by a pose based visual servoing (PBVS) iteration. To tackle large variation in appearance across object instances in a category, we employ visual features that uniquely correspond to locations of object's parts in images. These part-aware keypoints are learned from annotated images using a convolutional neural network (CNN). Advantages of using such part-aware semantics are two-fold. Firstly, it conceals the illumination and textural variations from the visual servoing algorithm. Secondly, semantic keypoints enables us to match descriptors across instances accurately. We validate the efficacy of our approach through experiments in simulation as well as on a quadcopter. Our approach results in acceptable desired camera pose and smooth velocity profile. We also show results for large camera transformations with no overlap between current and desired pose for 3D objects, which is desirable in servoing context.

@inproceedings{bib_CObR_2017, AUTHOR = {Sachdeva, Enna and Singh, Akash and Rodrigues, Vinay and Sarkar, Abhishek and Krishna, K Madhava }, TITLE = {CObRaSO: Compliant Omni-Direction Bendable Hybrid Rigid and Soft OmniCrawler Module}, BOOKTITLE = {Technical Report}. YEAR = {2017}}

This paper presents a novel design of an Omnidirectional bendable Omnicrawler module-CObRaSO. Along with the longitudinal crawling and sideways rolling motion, the performance of the OmniCrawler is further enhanced by the introduction of Omnidirectional bending within the module, which is the key contribution of this paper. The Omnidirectional bending is achieved by an arrangement of two independent 1-DOF joints aligned at 90? wrt each other. The unique characteristic of this module is its ability to crawl in Omnidirectionally bent configuration which is achieved by a novel design of a 2-DOF roller chain and a backbone of a hybrid structure of a soft-rigid material. This hybrid structure provides compliant pathways for the lug-chain assembly to passively conform with the orientation of the module and crawl in Omnidirectional bent configuration, which makes this module one of its kind. Furthermore, we show that the unique modular design of CObRaSO unveils its versatility by achieving active compliance on an uneven surface, demonstrating its applications in different robotic platforms (an in-pipeline robot, Quadruped and snake robot) and exhibiting hybrid locomotion modes in various configurations of the robots. The mechanism and mobility characteristics of the proposed module have been verified with the aid of simulations and experiments on real robot prototype.

@inproceedings{bib_FPGA_2017, AUTHOR = {MALIK, GURSHAANT SINGH and GUPTA, KRISHNA and , Raunak Dharani and Krishna, K Madhava }, TITLE = {FPGA Based Massively Parallel Hybrid Architecture for Parallelizing RRTs}, BOOKTITLE = {International Journal of Mechanical Engineering and Robotics Research}. YEAR = {2017}}

Field Programmable Gate Arrays(FPGA) exceed the computing power of software based implementations by breaking the paradigm of sequential execution and accomplishing more per clock cycle by enabling hardware level parallelization at an architectural level. As has been proved in already published research works, introducing parallel architectures for a computationally intensive algorithm like Rapidly Exploring Random Trees(RRT) will result in an exploration that is fast, dense and uniform. FPGA based combinatorial architecture, which is one of the already published research works, delivers superlative speed-up but consumes very high power. The second of the already published research works, FPGA based hierarchical architecture, delivers relatively lower speed-up with acceptable power consumption levels. To combine the qualities of both, a hybrid architecture, that encompasses both combinatorial and hierarchical architecture, is designed. To determine the design parameters of the hybrid architecture, a cost function comprised of fundamentally inversely related speed-up and power parameters, as mentioned above, is formulated. This maximization of cost function, with its associated constraints, is then mathematically solved using a modified branch and bound, that leads to optimal deduction of the design parameters of the hybrid architecture. Via empirical experiments, it is observed that this hybrid architecture delivers the highest performance-per-watt out of the three architectures for differential, quad-copter and fixed wing kinematics, in environments of varying geometric complexity. The empirical experiments also confirmed that the hybrid architecture is scalable with 1.) Increase in the environment's geometric complexity and 2.) Increase in kinematic complexity of the robot.

@inproceedings{bib_FPGA_2017, AUTHOR = {NAHAR, ROOPAL and BARANWAL, AKANKSHA and Krishna, K Madhava }, TITLE = {FPGA based parallelized architecture of efficient graph based image segmentation algorithm}, BOOKTITLE = {International Conference on Robotics and Biomimetics}. YEAR = {2017}}

Efficient and real time segmentation of color images has a variety of importance in many fields of computer vision such as image compression, medical imaging, mapping and autonomous navigation. Being one of the most computationally expensive operation, it is usually done through software implementation using high-performance processors. In robotic systems, however, with the constrained platform dimensions and the need for portability, low power consumption and simultaneously the need for real time image segmentation, we envision hardware parallelism as the way forward to achieve higher acceleration. Field-programmable gate arrays (FPGAs) are among the best suited for this task as they provide high computing power in a small physical area. They exceed the computing speed of software based implementations by breaking the paradigm of sequential execution and accomplishing more per clock cycle operations by enabling hardware level parallelization at an architectural level. In this paper, we propose three novel architectures of a well known Efficient Graph based Image Segmentation algorithm. These proposed implementations optimizes time and power consumption when compared to software implementations. The hybrid design proposed, has notable furtherance of acceleration capabilities delivering atleast 2X speed gain over other implementations, which henceforth allows real time image segmentation that can be deployed on Mobile Robotic systems.

@inproceedings{bib_A_de_2017, AUTHOR = {TEJA, SINGAMANENI PHANI and DEWANGAN, PARIJAT and Guhan, Pooja and Sarkar, Abhishek and Krishna, K Madhava }, TITLE = {A deep reinforcement learning approach for dynamically stable inverse kinematics of humanoid robots}, BOOKTITLE = {International Conference on Robotics and Biomimetics}. YEAR = {2017}}

Real time calculation of inverse kinematics (IK) with dynamically stable configuration is of high necessity in humanoid robots as they are highly susceptible to lose balance. This paper proposes a methodology to generate joint-space trajectories of stable configurations for solving inverse kinematics using Deep Reinforcement Learning (RL). Our approach is based on the idea of exploring the entire configuration space of the robot and learning the best possible solutions using Deep Deterministic Policy Gradient (DDPG). The proposed strategy was evaluated on the highly articulated upper body of a humanoid model with 27 degree of freedom (DoF). The trained model was able to solve inverse kinematics for the end effectors with 90% accuracy while maintaining the balance in double support phase.

@inproceedings{bib_3D_R_2017, AUTHOR = {REDDY, ARRABOTU SHEETAL and Gandhi, Vineet and Krishna, K Madhava }, TITLE = {3D Region Proposals For Selective Object Search.}, BOOKTITLE = {International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applicat}. YEAR = {2017}}

The advent of indoor personal mobile robots has clearly demonstrated their utility in assisting humans at various places such as workshops, offices, homes, etc. One of the most important cases in such autonomous scenarios is where the robot has to search for certain objects in large rooms. Exploring the whole room would prove to be extremely expensive in terms of both computing power and time. To address this issue,we demonstrate a fast algorithm to reduce the search space by identifying possible object locations as two classes, namely - Support Structures and Clutter. Support Structures are plausible object containers in a scene such as tables, chairs, sofas, etc. Clutter refers to places where there seem to be several objects but cannot be clearly distinguished. It can also be identified as unorganized regions which can be of interest for tasks such as robot grasping, fetching and placing objects. The primary contribution of this paper is to quickly identify potential object locations using a Support Vector Machine(SVM) learnt over the features extracted from thedepth map and the RGB image of the scene, which further culminates into a densely connected Conditional Random Field(CRF) formulated over the image of the scene. The inference over the CRF leads to assignment of the labels - support structure, clutter, others to each pixel.There have been reliable outcomes even during challenging scenarios such as the support structures being far from the robot. The experiments demonstrate the efficacy and speed of the algorithm irrespective of alterations to camera angles, modifications to appearance change, lighting and distance from locations etc

@inproceedings{bib_Smal_2017, AUTHOR = {GUPTA, KRISHNAM and UPADHYAY, SARTHAK and Gandhi, Vineet and Krishna, K Madhava }, TITLE = {Small obstacle detection using stereo vision for autonomous ground vehicle}, BOOKTITLE = {Advances in Robotics}. YEAR = {2017}}

Small and medium sized obstacles such as rocks, small boulders, bricks left unattended on the road can pose hazards for autonomous as well as human driving situations. Many times these objects are too small on the road and go unnoticed on depth and point cloud maps obtained from state of the art range sensors such as 3D LIDAR. We propose a novel algorithm that fuses both appearance and 3D cues such as image gradients, curvature potentials and depth variance into a Markov Random Field (MRF) formulation that segments the scene into obstacle and non obstacle regions. Appearance and depth data obtained from a ZED stereo pair mounted on a Husky robot is used for this purpose. While identifying true positive obstacles such as rocks, large stones accurately our algorithm is simultaneously robust to false positive sources such as appearance changes on the road, papers and road markings. High accuracy detection in challenging scenes such as when the foreground obstacle blends with the background road scene vindicates the efficacy of the proposed formulation.

@inproceedings{bib_Desi_2016, AUTHOR = {GOEL, DIVYANSHU and TEJA, SINGAMANENI PHANI and DEWANGAN, PARIJAT and Dewangan, Parijat and Sarkar, Abhishek and Krishna, K Madhava }, TITLE = {Design and Development of a Humanoid with Articulated Torso}, BOOKTITLE = {International Conference on Robotics and Automation for Humanitarian Applications}. YEAR = {2016}}

The purpose of this paper is to present the model of a Humanoid robot inspired by Poppy, modified for heavier load capacity and the balancing of humanoid in multiple work environments. The design has been modified in order to use MX-64 servos with more torque capacity than MX-28 servos which were used in original design. We have also redesigned the ankle joint and feet to make it a more accurate human like model. We are using an integrated approach using zero moment point (ZMP) with force sensing resistor (FSR) and center of mass (CoM) to find balance margins of the robot. Balancing and forward bending experiments on robot are conducted by providing some basic motion to the robot and ensuring that the robot is balanced and moving within safe margins using this approach.

@inproceedings{bib_Anal_2016, AUTHOR = {, Sathya Narayanan K and S, Gokul Narasimhan and Krishna, K Madhava }, TITLE = {Analysis of a Compliant Multi-module Robot for Circular Stair Climbing}, BOOKTITLE = {International Conference on Robotics: Current Trends and Future Challenges}. YEAR = {2016}}

The primary objective of a robot is to substitute or complement humans in life-endangering situations. The paper deals with the design and analysis of a robot suitable for such Urban Search and Rescue operation (USAR) scenarios. A survey of the existing robots in this field is done and new design considerations have been developed. Further, quasi-static analysis of the robot along a circular stair is performed to confirm whether the design considerations will satisfy versatility as well as mobility, the key requirements of a true uneven terrain robot. Keywords— Compliant Design; Multi-module robot; Nonlinear optimization; 3D Quasi Static Model

@inproceedings{bib_Desi_2016, AUTHOR = {GOEL, DIVYANSHU and TEJA, SINGAMANENI PHANI and DEWANGAN, PARIJAT and Shah, Suril V and Sarkar, Abhishek and Krishna, K Madhava }, TITLE = {Design and Development of a Humanoid with Articulated Torso}, BOOKTITLE = {International Conference on Robotics and Automation for Humanitarian Applications}. YEAR = {2016}}

The purpose of this paper is to present the modelof a Humanoid robot inspired by Poppy, modified for heavier load capacity and the balancing of humanoid in multiple work environments. The design has been modified in order to useMX-64 servos with more torque capacity than MX-28 servos which were used in original design. We have also redesigned the ankle joint and feet to make it a more accurate human like model. We are using an integrated approach using zero moment point (ZMP) with force sensing resistor (FSR) and center of mass (CoM) to find balance margins of the robot. Balancing and forward bending experiments on robot are conducted by providing some basic motion to the robot and ensuring that the robot is balanced and moving within safe margins using this approach.

@inproceedings{bib_Usin_2016, AUTHOR = {TOURANI, SIDDHARTH and Krishna, K Madhava }, TITLE = {Using in-frame shear constraints for monocular motion segmentation of rigid bodies}, BOOKTITLE = {Journal of Intelligent and Robotic Systems}. YEAR = {2016}}

It is a well known result in the vision literature that the motion of independently moving objects viewed by an affine camera lie on affine subspaces of dimension four or less. As a result a large number of the recently proposed motion segmentation algorithms model the problem as one of clustering the trajectory data to its corresponding affine subspace. While these algorithms are elegant in formulation and achieve near perfect results on benchmark datasets, they fail to address certain very key real-world challenges, including perspective effects and motion degeneracies. Within a robotics and autonomous vehicle setting, the relative configuration of the robot and moving object will frequently be degenerate leading to a failure of subspace clustering algorithms. On the other hand, while gestalt-inspired motion similarity algorithms have been used for motion segmentation, in the moving camera case, they tend to over-segment or under-segment the scene based on their parameter values. In this paper we present a principled approach that incorporates the strengths of both approaches into a cohesive motion segmentation algorithm capable of dealing with the degenerate cases, where camera motion follows that of the moving object. We first generate a set of prospective motion models for the various moving and stationary objects in the video sequence by a RANSAC-like procedure. Then, we incorporate affine and long-term gestalt-inspired motion similarity constraints, into a multi-label Markov Random Field (MRF). Its inference leads to an over-segmentation, where each label belongs to a particular moving object or the background. This is followed by a model selection step where we merge clusters based on a novel motion coherence constraint, we call in-frame shear, that tracks the in-frame change in orientation and distance between the clusters, leading to the final segmentation. This oversegmentation is deliberate and necessary, allowing us to assess the relative motion between the motion models which we believe to be essential in dealing with degenerate motion scenarios.We present results on the Hopkins-155 benchmark motion segmentation dataset [27], as well as several on-road scenes where camera and object motion are near identical. We show that our algorithm is competitive with the state-of-the-art algorithms on [27] and exceeds them substantially on the more realistic on-road sequences.

@inproceedings{bib_Feas_2016, AUTHOR = {SINGH, ARUN KUMAR and Krishna, K Madhava }, TITLE = {Feasible acceleration count: A novel dynamic stability metric and its use in incremental motion planning on uneven terrain}, BOOKTITLE = {Robotics and Autonomous systems}. YEAR = {2016}}

Application of wheeled mobile robots has gradually progressed from the confines of structured indoor environments to rough outdoor terrains. Material transport and exploration are some of the few areas where wheeled robots are required to navigate over uneven terrains. Stable and efficient navigation of wheeled robots over uneven terrains requires a framework which can correctly ascertain the stability and maneuverability for a given robot’s state. Most existing works on uneven terrain navigation assume a one-to-one correspondence between postural stability and maneuverability. In this paper, we show that such characterization is incomplete as states having high postural stability may have restricted maneuverability depending on underlying terrain topology. We thus, present a novel metric called Feasible Acceleration Count (FAC), introduced in our earlier works as a unified measure of robot stability and maneuverability. The metric gives the measure of the space of feasible accelerations available to the robot at a given state. The feasibility is decided by a set of inequalities which depends not only on robot’s state but also on surface normals at the wheel ground contact point. This unique feature of the FAC metric makes it a more appropriate choice for motion planning on uneven terrains than metrics like Tip-Over. We further show that since space of feasible accelerations is a direct characterization of the space of possible motions at a given state, the metric FAC, also quantifies the quality of state space exploration achieved at each step of incremental sampling based planners. We build on top of this aspect of FAC and present an incremental trajectory planner with a novel node selection criteria for navigation of generic four wheeled robots and articulated systems like mobile manipulators on uneven terrain.

@inproceedings{bib_Mono_2016, AUTHOR = {CHHAYA, FALAK PIYUSHBHAI and REDDY, NARAPUREDDY DINESH and UPADHYAY, SARTHAK and Chari, Visesh and Zia, M. Zeeshan and Krishna, K Madhava }, TITLE = {Monocular reconstruction of vehicles: Combining slam with shape priors}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2016}}

Reasoning about objects in images and videos using 3D representations is re-emerging as a popular paradigm in computer vision. Specifically, in the context of scene understanding for roads, 3D vehicle detection and tracking from monocular videos still needs a lot of attention to enable practical applications. Current approaches leverage two kinds of information to deal with the vehicle detection and tracking problem: (1) 3D representations (eg. wireframe models or voxel based or CAD models) for diverse vehicle skeletal structures learnt from data, and (2) classifiers trained to detect vehicles or vehicle parts in single images built on top of a basic feature extraction step. In this paper, we propose to extend current approaches in two ways. First, we extend detection to a multiple view setting. We show that leveraging information given by feature or part detectors in multiple images can lead to more accurate detection results than single image detection. Secondly, we show that given multiple images of a vehicle, we can also leverage 3D information from the scene generated using a unique structure from motion algorithm. This helps us localize the vehicle in 3D, and constrain the parameters of optimization for fitting the 3D model to image data. We show results on the KITTI dataset, and demonstrate superior results compared with recent state-of-the-art methods, with upto 14.64 % improvement in localization error.

@inproceedings{bib_Disc_2016, AUTHOR = {PANDYA, HARIT and Krishna, K Madhava and V, Jawahar C }, TITLE = {Discriminative learning based visual servoing across object instances}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2016}}

Classical visual servoing approaches use visual features based on geometry of the object such as points, lines, region, etc. to attain the desired camera pose. However, geometrical features are not suited for visual servoing across different object instances due to large variations in appearance and shape. In this paper, we present a new framework for visual servoing across object instances. Our approach is based on a discriminative learning framework where the desired pose is estimated using previously seen examples. Specifically, we learn a binary classifier that separates the desired pose from all other poses for that object category. The classification error is then used to control the end-effector so that the desired pose is attained. We present controllers for linear, kernel and exemplar Support Vector Machine (SVM) and empirically discuss their performance in the visual servoing context. To address large intra-category variation in appearance, we propose a modified version of Histogram of Oriented Gradients (HOG) features for visual servoing. We show effective servoing across diverse instances over 3 object categories with zero terminal velocity and acceptable camera pose error at termination.

@inproceedings{bib_Plan_2016, AUTHOR = {DUGGAL, VISHAKH and SUKHWANI, MOHAK KUMAR and BIPIN, KUMAR and Reddy, Syamsundar and Krishna, K Madhava }, TITLE = {Plantation monitoring and yield estimation using autonomous quadcopter for precision agriculture}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2016}}

Recently, quadcopters with their advance sensors and imaging capabilities have become an imperative part of the precision agriculture. In this work, we have described a framework which performs plantation monitoring and yield estimation using the supervised learning approach, while autonomously navigating through an inter-row path of the plantation. The proposed navigation framework assists the quadcopter to follow a sequence of collision-free GPS way points and has been integrated with ROS (Robot Operating System). The trajectory planning and control module of the navigation framework employ convex programming techniques to generate minimum time trajectory between way-points and produces appropriate control inputs for the quadcopter. A new `pomegranate dataset' comprising of plantation surveillance video and annotated frames capturing the varied stages of pomegranate growth along with the navigation framework are being delivered as a part of this work.

@inproceedings{bib_Slam_2016, AUTHOR = {Prasad, Vignesh and Yadav, Karmesh and Saurabh, Rohitashva Singh and Daga, Swapnil Naresh and PAREEKUTTY, NAHAS and Krishna, K Madhava and Ravindran, Balaraman and Bhowmick, Brojeshwar }, TITLE = {Slam-safe planner: Preventing monocular slam failure using reinforcement learning}, BOOKTITLE = {Indian Conference on Computer Vision, Graphics and Image Processing}. YEAR = {2016}}

Monocular SLAM refers to using a single camera to estimate robot ego motion while building a map of the environment. While Monocular SLAM is a well studied problem, automating Monocular SLAM by integrating it with trajectory planning frameworks is particularly challenging. This paper presents a novel formulation based on Reinforcement Learning (RL) that generates fail safe trajectories wherein the SLAM generated outputs do not deviate largely from their true values. Quintessentially, the RL framework successfully learns the otherwise complex relation between perceptual inputs and motor actions and uses this knowledge to generate trajectories that do not cause failure of SLAM. We show systematically in simulations how the quality of the SLAM dramatically improves when trajectories are computed using RL. Our method scales effectively across Monocular SLAM frameworks in both simulation and in real world experiments with a mobile robot.

@inproceedings{bib_Reac_2016, AUTHOR = {James, Francis and Shah, Suril V and SINGH, ARUN KUMAR and Krishna, K Madhava and Misra, Arun K. }, TITLE = {Reactionless maneuvering of a space robot in precapture phase}, BOOKTITLE = {Journal of Guidance, Control, and Dynamics}. YEAR = {2016}}

SPACE robots can be used to perform several onorbit tasks such as capturing space debris, servicing of satellites, and refueling. The proliferation of satellites as well as the growing interest in debris capture make it necessary to have robots that can perform these tasks autonomously [1]. The dynamics of robots in space differ from that of a grounded robot. The coupling of the arms and the base of a space robot creates reaction forces and moments on the base whenever the arms execute a maneuver, causing the base to rotate and translate in accordance with the laws of conservation of linear and angular momenta. However, it is generally desirable to keep the attitude of the base fixed relative to the sun and the Earth (or other bodies) for navigation and communication purposes, or to maintain the target in the field of view of the sensors. A change in celestial orientation may also result in loss of communication with the data relay satellite or the ground station. While attitude control using thrusters may be used, such operations consume fuel which is mainly reserved for orbital maneuvers. It has also been shown that manipulation without the use of attitude controllers is more robust [2]. The translation of the base however, does not pose signicant side eects [3]. Hence, researchers have focused on robotic manipulation with zero or minimal change in attitude, which is termed reactionless manipulation

@inproceedings{bib_FPGA_2016, AUTHOR = {MALIK, GURSHAANT SINGH and GUPTA, KRISHNA and Dharani, Raunak and Krishna, K Madhava }, TITLE = {FPGA based hybrid architecture for parallelizing RRT}, BOOKTITLE = {Technical Report}. YEAR = {2016}}

Field Programmable Gate Arrays (FPGA) exceed the computing power of software based implementations by breaking the paradigm of sequential execution and accomplishing more per clock cycle by enabling hardware level parallelization at an architectural level. Introducing parallel architectures for a computationally intensive algorithm like Rapidly Exploring Random Trees (RRT) will result in an exploration that is fast, dense and uniform. Through a cost function delineated in later sections, FPGA based combinatorial architecture delivers superlative speed-up but consumes very high power while hierarchical architecture delivers relatively lower speed-up with acceptable power consumption levels. To combine the qualities of both, a hybrid architecture, that encompasses both combinatorial and hierarchical architecture, is designed. To determine the number of RRT nodes to be allotted to the combinatorial and hierarchical blocks of the hybrid architecture, a cost function, comprised of fundamentally inversely related speed-up and power parameters, is formulated. This maximization of cost function, with its associated constraints, is then mathematically solved using a modified branch and bound, that leads to optimal allocation of RRT modules to both blocks. It is observed that this hybrid architecture delivers the highest performance-per-watt out of the three architectures for differential, quad-copter and fixed wing kinematics.

@inproceedings{bib_Plan_2016, AUTHOR = {SINGH, ARUN KUMAR and Krishna, K Madhava and Saripalli, Srikanth }, TITLE = {Planning non-holonomic stable trajectories on uneven terrain through non-linear time scaling}, BOOKTITLE = {Autonomous Robots}. YEAR = {2016}}

In this paper, we present a framework for generating smooth and stable trajectories for wheeled mobile robots moving on uneven terrains. Instead of relying on static stability measures, the paper incorporates velocity and acceleration based constraints like no-slip and permanent wheel ground contact conditions in the planning framework. The paper solves this complicated problem in a computationally efficient manner with full 3D dynamics of the robot. The two major aspects of the proposed work are: Firstly, closed form functional relationships are derived which describes the 6 dof evolution of the robot’s state on an arbitrary 2.5D uneven terrain. This enables us to have a fast evaluation of robot’s dynamics along any candidate trajectory. Secondly, a novel concept of non-linear time scaling is introduced through which simple algebraic relations defining the bounds on velocities and accelerations are obtained. Moreover, non-linear time scaling also provides a new approach for manipulating velocities and accelerations along given geometric paths. It is thus exploited to obtain stable velocity and acceleration profiles. Extensive simulation results are presented to demonstrate the efficacy of the proposed methodology.

@inproceedings{bib_CRF__2016, AUTHOR = {SODHI, DANISH and UPADHYAY, SARTHAK and Jitendrakumar, Bhatt Dhaivat and Krishna, K Madhava and Swarup, Shanti }, TITLE = {CRF based method for curb detection using semantic cues and stereo depth}, BOOKTITLE = {Indian Conference on Computer Vision, Graphics and Image Processing}. YEAR = {2016}}

Curb detection is a critical component of driver assistance and autonomous driving systems. In this paper, we present a discriminative approach to the problem of curb detection under diverse road conditions. We define curbs as the intersection of drivable and non-drivable area which are classified using dense Conditional random fields(CRF). In our method, we fuse output of a neural network used for pixel-wise semantic segmentation with depth and color information from stereo cameras. CRF fuses the output of a deep model and height information available in stereo data and provides improved segmentation. Further we introduce temporal smoothness using a weighted average of Segnet output and output from a probabilistic voxel grid as our unary potential. Finally, we show improvements over the current state of the art neural networks. Our proposed method shows accurate results over large range of variations in curb curvature and appearance, without the need of retraining the model for the specific dataset.

@inproceedings{bib_Fast_2016, AUTHOR = {UPADHYAY, SARTHAK and Krishna, K Madhava and Kumar, Swagat }, TITLE = {Fast Frontier Detection in Indoor Environment for Monocular SLAM}, BOOKTITLE = {Indian Conference on Computer Vision, Graphics and Image Processing}. YEAR = {2016}}

Frontier detection is a critical component in autonomous exploration, wherein the robot decides the next best location to move in order to continue its mapping process. The existing frontier detection methods require dense reconstruction which is difficult to attain in a poorly textured indoor environment using a monocular camera. In this effort, we present an alternate method of detecting frontiers during the course of robot motion that circumvents the requirement of dense mapping. Based on the observation that frontiers typically occur around areas with sudden change in texture (zero-crossings), we propose a novel linear chain Conditional Random Field(CRF) formulation that is able to detect the presence or absence of frontier regions around such areas. We use cues like spread of 3D points and scene change around these areas as an observation to CRF. We demonstrate that this method gives us more relevant frontiers compared to other monocular camera based methods in the literature. Finally, we present results in an indoor environment, wherein frontiers are reliably detected around walls leading to new corridors, doors leading to new rooms or corridors and tables and other objects that open up to a new space in rooms.

@inproceedings{bib_Deep_2016, AUTHOR = {DIPAKKUMAR, SHAH UTSAV and KHAWAD, RISHABH and Krishna, K Madhava }, TITLE = {DeepFly: towards complete autonomous navigation of MAVs with monocular camera}, BOOKTITLE = {Indian Conference on Computer Vision, Graphics and Image Processing}. YEAR = {2016}}

Recently, the interest in Micro Aerial Vehicles (MAVs) and their autonomous flights has increased tremendously and significant advances have been made. The monocular camera has turned out to be most popular sensing modality for MAVs as it is light-weight, does not consume more power, and encodes rich information about the environment around. In this paper, we present DeepFly, our framework for autonomous navigation of a quadcopter equipped with monocular camera. The navigable space detection and waypoint selection are fundamental components of autonomous navigation system. They have broader meaning than just detecting and avoiding immediate obstacles. Finding the navigable space emphasizes equally on avoiding obstacles and detecting ideal regions to move next to. The ideal region can be defined by two properties: 1) All the points in the region have approximately same high depth value and 2) The area covered by the points of the region in the disparity map is considerably large. The waypoints selected from these navigable spaces assure collision-free path which is safer than path obtained from other waypoint selection methods which do not consider neighboring information. In our approach, we obtain a dense disparity map by performing a translation maneuver. This disparity map is input to a deep neural network which predicts bounding boxes for multiple navigable regions. Our deep convolutional neural network with shortcut connections regresses variable number of outputs without any complex architectural add on. Our autonomous navigation approach has been successfully tested in both indoors and outdoors environment and in range of lighting conditions.

@inproceedings{bib_Hier_2016, AUTHOR = {DUGGAL, VISHAKH and BIPIN, KUMAR and DIPAKKUMAR, SHAH UTSAV and Krishna, K Madhava }, TITLE = {Hierarchical structured learning for indoor autonomous navigation of quadcopter}, BOOKTITLE = {Indian Conference on Computer Vision, Graphics and Image Processing}. YEAR = {2016}}

Autonomous navigation of generic monocular quadcopter in the indoor environment requires sophisticated approaches for perception, planning and control. This paper presents a system which enables a miniature quadcopter with a frontal monocular camera to autonomously navigate and explore the unknown indoor environment. Initially, the system estimates dense depth map of the environment from a single video frame using our proposed novel supervised Hierarchical Structured Learning (hsl) technique, which yields both high accuracy levels and better generalization. The proposed hsl approach discretizes the overall depth range into multiple sets. It structures these sets hierarchically and recursively through partitioning the set of classes into two subsets with subsets representing apportioned depth range of the parent set, forming a binary tree. The binary classification method is applied to each internal node of binary tree separately using Support Vector Machine (svm). Whereas, the depth estimation of each pixel of the image starts from the root node in top-down approach, classifying repetitively till it reaches any of the leaf node representing its estimated depth. The generated depth map is provided as an input to Convolutional Neural Network (cnn), which generates flight planning commands. Finally, trajectory planning and control module employs a convex programming technique to generate collision-free minimum time trajectory which follows these flight planning commands and produces appropriate control inputs for the quadcopter. The results convey unequivocally the advantages of depth perception by hsl, while repeatable flights of successful nature in typical indoor corridors confirm the efficacy of the pipeline.

@inproceedings{bib_Lear_2016, AUTHOR = {Hafez, A. H. Abdul and Arora, Manpreet and Krishna, K Madhava and V, Jawahar C }, TITLE = {Learning multiple experiences useful visual features for active maps localization in crowded environments}, BOOKTITLE = {Advanced Robotics}. YEAR = {2016}}

Crowded urban environments are composed of different types of dynamic and static elements. Learning and classification of features is a major task in solving the localization problem in such environments. This work presents a gradual learning methodology to learn the useful features using multiple experiences. The usefulness of an observed element is evaluated by a scoring mechanism which uses two scores – reliability and distinctiveness. The visual features thus learned are used to partition the visual map into smaller regions. The robot is efficiently localized in such a partitioned environment using two-level localization. The concept of active map (AM) is proposed here, which is a map that represents one partition of the environment in which there is a high probability of the robot existing. High-level localization is used to track the mode of the AMs using discrete Bayes filter. Low-level localization uses a bag-of …

@inproceedings{bib_SLAM_2016, AUTHOR = {DATTA, SAYANTAN and TOURANI, SIDDHARTH and Sharma, Avinash and Krishna, K Madhava }, TITLE = {SLAM pose-graph robustification via multi-scale Heat-Kernel analysis}, BOOKTITLE = {Conference on Decision and Control}. YEAR = {2016}}

The Simultaneous Localization and Mapping problem (SLAM) in robotics is typically modeled as a dyadic graph of relative pose measurements taken by the robot. The graph nodes store the values representing the absolute pose of the robot at a given point of time. An edge connecting two nodes represents robot movement and it stores the measurements taken by the robot sensor while moving between two nodes. The objective of the SLAM problem is to find the optimal global measurements best satisfying the noisy relative measurements [12]. This problem of optimal estimation on a graph given relative measurements is a well-studied problem within the control community, for which several results and algorithms are known [3, 4]. SLAM is generally solved as a least squares problem. Robust kernels which are less sensitive to outliers are used to deal with noise and outlier measurements. However, robust kernels tend to be dependent on initialization and can fail as the number of outliers increase. Therefore, it's important to identify and prune the outlier (noisy) measurements represented by incorrect loop closure edges for an accurate pose estimate. In this paper we propose a multi-scale Heat-Kernel analysis based loop closure edge pruning algorithm for the SLAM graph. We show that compared to other pruning algorithms, our algorithm has a substantially higher precision and recall when compared and is able to handle a large amount of outlier measurements.We have corroborated results on several publicly available datasets and several types of noise. Our algorithm is not restricted to SLAM graphs only, but has a much wider applicability to other types of geometric graphs.

@inproceedings{bib_FPGA_2015, AUTHOR = {MALIK, GURSHAANT SINGH and GUPTA, KRISHNA and Krishna, K Madhava and Chowdhury, Shubhajit Roy }, TITLE = {FPGA based combinatorial architecture for parallelizing RRT}, BOOKTITLE = {European Conference on Mobile Robots}. YEAR = {2015}}

Complex tasks are often handled through software implementation in combination with high performance processors. Taking advantage of hardware parallelism, FPGA is breaking the paradigm by accomplishing more per clock cycle with closely matched application requirements. With the aim to minimise computation delay with increase in map's size and geometric constraints, we present the FPGA based combinatorial architecture that allows multiple RRTs to work together to achieve accelerated, uniform exploration of the map. We also analyse our architecture against hardware implementation of other scalable RRT methods for motion planning. We observe notable furtherance of acceleration capabilities with the proposed architecture delivering a minimum 3X gain over the other implementations while maintaining uniformity in exploration

@inproceedings{bib_Deci_2015, AUTHOR = {KARTHIK, M.SIVA and MITTAL, SUDHANSHU and MALIK, GURSHAANT SINGH and Krishna, K Madhava }, TITLE = {Decision theoretic search for small objects through integrating far and near cues}, BOOKTITLE = {European Conference on Mobile Robots}. YEAR = {2015}}

In an object search scenario with several small objects spread over a large indoor environment, the robot cannot infer about all of them at once. Pruning the search space is highly desirable in such a case. It has to actively select a course of actions to closely examine a selected set of objects. Here, we model the inferences about far away objects and their viewpoint priors into a decision analytic abstraction to prioritize the waypoints. By selecting objects of interest, a potential field is built over the environment by using Composite Viewpoint Object Potential (CVOP) maps. A CVOP is built using VOP, a framework to identify discriminative viewpoints to recognize small objects having distinctive features only in specific views. Also, a CVOP helps to clearly disambiguate objects which look similar from far away. We formulate a Decision Analysis Graph (DAG) over the above information, to assist the robot in actively navigating and maximize the reward earned. This optimal strategy increases search reliability, even in the presence of similar looking small objects which induce confusion into the agent and simultaneously reduces both time taken and distance travelled. To the best of our knowledge, there is no current unified formulation which addresses indoor object search scenarios in this manner. We evaluate our system over ROS using a TurtleBot mounted with a Kinect.

@inproceedings{bib_Swit_2015, AUTHOR = {Shah, Suril V and ANURAG, V V and Hafez, AH Abdul and Krishna, K Madhava }, TITLE = {Switching method to avoid algorithmic singularity in vision-based control of a space robot}, BOOKTITLE = {International Conference on Advanced Robotics}. YEAR = {2015}}

This paper presents a novel approach for algorithmic singularity avoidance for reactionless visual servoing of a satellite mounted space robot. Task priority approach is used to perform visual servoing while reactionless manipulation of the space robot. Algorithmic singularity is prominent in such cases of prioritizing two tasks. The algorithmic singularity is different from the kinematic and dynamic singularities as the former is an artefact of the tasks at hand, and difficult to predict. In this paper, we present a geometric interpretation of its occurrence, and propose a method to avoid it. The method involves path planning in image space, and generates a sequence of images that guides the robot towards goal avoiding algorithmic singularity. The method is illustrated through numerical studies on a 6-DOF planar dual-arm robot mounted on a service satellite.

@inproceedings{bib_FPGA_2015, AUTHOR = {MALIK, GURSHAANT SINGH and GUPTA, KRISHNA and Krishna, K Madhava and Chowdhury, Shubhajit Roy }, TITLE = {FPGA based hierarchical architecture for parallelizing RRT}, BOOKTITLE = {Advances in Robotics}. YEAR = {2015}}

This paper presents a new hierarchical architecture for parallelizing the computation intensive rapidly exploring random tree problem. The architecture resembles a tree like structure that agglutinates minimal inter-module communication of a shared memory with data integrity of a distributed memory. Another novelty of this research has been in quantitatively analysing the performance metrics of the RRT algorithm across numerous embedded hardware solutions and ultimately implementing this algorithm on an FPGA to achieve hardware level optimization that offers real time performance and economical power consumption levels. We then analyse our implementation against hardware implementation of other scalable parallel RRT methods for motion planning.

@inproceedings{bib_An_i_2015, AUTHOR = {TEJA, SINGAMANENI PHANI and HARSHA, TURLAPATI SRI and Siravuru, Avinash and Shah, Suril V and Krishna, K Madhava }, TITLE = {An improved compliant joint design of a modular robot for descending big obstacles}, BOOKTITLE = {Advances in Robotics}. YEAR = {2015}}

This work focuses on enhancing step descending ability of the modular robot proposed in [16]. The proposed robot consists of three modules connected with each other through passive joints. It is propelled using an active pair of wheels per module. Since there are no actuators at the joints, the joints are not susceptible to losing operability while traversing on rugged terrain. However with the absence of actuators, we face the issue of the robot toppling over when an abnormally large obstacle is encountered. This shortcoming is overcome with the use of compliant joints. The compliant joints are designed by employing springs of optimal stiffness, which is calculated through an optimization formulation aided with the constraints presented by the static analysis of the robot. The novelty lies in the systematic design of compliant joint for step descent. The robot is successful in climbing and descending obstacles of dimension 17 cm. Simulations of the mathematically modelled robot are carried out. The results from the same are validated on a working prototype and presented.

@inproceedings{bib_Impl_2015, AUTHOR = {Kumar, T Komal and Kumar, R. and MOGILY, SAMYUKTA and GOEL, DIVYANSHU and PAREEKUTTY, NAHAS and Shah, Suril V and Krishna, K Madhava }, TITLE = {Implementation of gaits for achieving omnidirectional walking in a quadruped robot}, BOOKTITLE = {Advances in Robotics}. YEAR = {2015}}

In this paper, we propose a better planning technique of the standard walking gaits for a quadruped robot than the conventional successive gait transition method to realize omnidirectional static walking. The technique involved planning the sequence as well as motion of the swinging and supporting legs. The relationship between the stability margin, the stride length and the duty factor are also formulated mathematically. The proposed modified crawl gait is compared to the conventional method with respect to the above parameters geometrically as well as mathematically and is shown to have positive stability margin at all times. The successive gait transition is demonstrated on the modified crawl and rotation gaits. Computer simulations of a model quadruped robot were performed to validate the theory proposed. Experiments were performed on an actual quadruped robot to realize the omnidirectional static walking with increased stability margin.

@inproceedings{bib_Over_2015, AUTHOR = {DUGGAL, VISHAKH and BIPIN, KUMAR and SINGH, ARUN KUMAR and GOPALAKRISHNAN, BHARATH and Bharti, Brijendra Kumar and Khiat, Abdelaziz and Krishna, K Madhava }, TITLE = {Overtaking maneuvers by non linear time scaling over reduced set of learned motion primitives}, BOOKTITLE = {Intelligent Vehicles symposium}. YEAR = {2015}}

Overtaking of a vehicle moving on structured roads is one of the most frequent driving behavior. In this work, we have described a Real Time Control System based framework for overtaking maneuver of autonomous vehicles. Proposed framework incorporates Intelligent Planning and Modular control modules. Intelligent Planning module of the framework enables the vehicle to intelligently select the most appropriate behavioral characteristics given the perceived operating environment. Subsequently, Modular control module reduces the search space of overtaking trajectories through an SVM based learning approach. These trajectories are then examined for possible future time collision using Velocity Obstacle. It employs non linear time scaling that provides for continuous trajectories in the space of linear and angular velocities to achieve continuous curvature overtaking maneuvers respecting velocity and acceleration bounds. Further time scaling also can scale velocities to avoid collisions and can compute a time optimal trajectory for the learned behavior. The preliminary results show the appropriateness of our proposed framework in virtual urban environment.

@inproceedings{bib_Auto_2015, AUTHOR = {SINGH, BIPIN and DUGGAL, VISHAKH and Krishna, K Madhava }, TITLE = {Autonomous navigation of generic monocular quadcopter in natural environment}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2015}}

Autonomous navigation of generic monocular quadcopter in the natural environment requires sophisticated mechanism for perception, planning and control. In this work, we have described a framework which performs perception using monocular camera and generates minimum time collision free trajectory and control for any commercial quadcopter flying through cluttered unknown environment. The proposed framework first utilizes supervised learning approach to estimate the dense depth map for video stream obtained from frontal monocular camera. This depth map is initially transformed into Ego Dynamic Space and subsequently, is used for computing locally traversable way-points utilizing binary integer programming methodology. Finally, trajectory planning and control module employs a convex programming technique to generate collision-free trajectory which follows these way-points and produces appropriate control inputs for the quadcopter. These control inputs are computed from the generated trajectory in each update. Hence, they are applicable to achieve closed-loop control similar to model predictive controller. We have demonstrated the applicability of our system in controlled indoors and in unstructured natural outdoors environment.

@inproceedings{bib_A_cl_2015, AUTHOR = {SINGH, ARUN KUMAR and Krishna, K Madhava }, TITLE = {A class of non-linear time scaling functions for smooth time optimal control along specified paths}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2015}}

Computing time optimal motions along specified paths forms an integral part of the solution methodology for many motion planning problems. Conventionally, this optimal control problem is solved considering piece-wise constant parametrization for the control input which leads to convexity and sparsity in the optimization structure. However, it also results in discontinuous control trajectory which is difficult to track. Thus, in this paper we revisit this time optimal control problem with the primary motivation of ensuring a high degree of smoothness in the resulting motion profile. In particular, we solve it with continuity constraints in control and higher order motion derivatives like jerk, snap etc. It is clear that such constraints would necessitate the use of time varying control inputs over the commonly used piece-wise constant form. The primary contribution of the current work lies in the introduction of a C ∞ class of time scaling functions represented as parametric exponentials. This in turn allows us to represent time varying control inputs as products of parametric exponential and a polynomial functions. We present the motivation behind adopting such representation of time scaling function over more common polynomial forms, both from mathematical as well as implementation standpoint. We also show that the proposed representation of time scaling function and control input leads to a very simple optimization structure where most of the constraints are linear. The non-linearity has a quasi-convex structure which can be reformulated into a simple difference of convex form. Thus, the resulting optimization can be efficiently solved through sequential convex programming where, at each iteration, the constraints in difference of convex form are further simplified to more conservative linear constraints.

@inproceedings{bib_Time_2015, AUTHOR = {SINGH, ARUN KUMAR and GOPALAKRISHNAN, BHARATH and Krishna, K Madhava }, TITLE = {Time Optimal Control along Specified Paths with Acceleration Continuity: A Non-Linear Time Scaling based approach}, BOOKTITLE = {Indian Control Conference}. YEAR = {2015}}

In this paper we present a non-linear time scaling based formulation for computing time optimal motions along specified paths. The primary motivation behind the current work is to introduce acceleration continuity constraints within the time optimal framework. Such constraint necessitates the use of time varying controls instead of commonly used piecewise constant controls. We propose a novel extension of our previously developed concept of non-linear time scaling through which it is possible to parametrize controls as piece-wise product of a exponential and a linear function. We show that such representation leads to a very simple optimization structure with primarily linear constraints. The non-linearity has a quasi-convex structure which we reformulate into a simple difference of convex form. A sequential convex programming framework is utilised to solve the optimization as a sequence of sparse quadratic programmes. The proposed optimization is an improvement over the current state of the art frameworks which introduces acceleration continuity constraints in the time optimal framework through highly non-linear and non-convex optimizations.

@inproceedings{bib_Dyna_2015, AUTHOR = {REDDY, NARAPUREDDY DINESH and Singhal, Prateek and Chari, Visesh and Krishna, K Madhava }, TITLE = {Dynamic body VSLAM with semantic constraints}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2015}}

Image based reconstruction of urban environments is a challenging problem that deals with optimization of large number of variables, and has several sources of errors like the presence of dynamic objects. Since most large scale approaches make the assumption of observing static scenes, dynamic objects are relegated to the noise modelling section of such systems. This is an approach of convenience since the RANSAC based framework used to compute most multiview geometric quantities for static scenes naturally confine dynamic objects to the class of outlier measurements. However, reconstructing dynamic objects along with the static environment helps us get a complete picture of an urban environment. Such understanding can then be used for important robotic tasks like path planning for autonomous navigation, obstacle tracking and avoidance, and other areas. In this paper, we propose a system for robust SLAM that works in both static and dynamic environments. To overcome the challenge of dynamic objects in the scene, we propose a new model to incorporate semantic constraints into the reconstruction algorithm. While some of these constraints are based on multi-layered dense CRFs trained over appearance as well as motion cues, other proposed constraints can be expressed as additional terms in the bundle adjustment optimization process that does iterative refinement of 3D structure and camera / object motion trajectories. We show results on the challenging KITTI urban dataset for accuracy of motion segmentation and reconstruction of the trajectory and shape of moving objects relative to ground truth. We are able to show average relative error reduction by 41 % for moving object trajectory reconstruction relative to state-of-the-art methods like TriTrack[16], as well as on standard bundle adjustment algorithms with motion segmentation.

@inproceedings{bib_Mobi_2015, AUTHOR = {NAGARIYA, AKHIL KUMAR and GOPALAKRISHNAN, BHARATH and SINGH, ARUN KUMAR and GUPTA, KRISHNAM and Krishna, K Madhava }, TITLE = {Mobile robot navigation amidst humans with intents and uncertainties: A time scaled collision cone approach}, BOOKTITLE = {Conference on Decision and Control}. YEAR = {2015}}

We propose a novel collision avoidance formulation in the intent space, suitable for navigation of non-holonomic robots in human centered environments. The intent space is characterized by various bands of trajectories wherein each band can be thought to be a representation of a possible human intended motion and the uncertainty associated with it. We ascribe probabilities to human intentions and characterize the uncertainty around it through Gaussian state transition and its concomitant Gaussian parametric distribution. Given an intent space we design avoidance maneuvers based on our recent works on time scaled collision cone concept which provides analytical characterization of collision free velocities in dynamic environments. In this paper, we present a probabilistic variant of the time scaled collision cone which allows us to relate space of collision free velocities to an associated confidence measure. We also develop an optimization framework which extract such specific solutions from the entire solution space that achieves an elegant balance between the objective of minimizing risk and ease of avoidance maneuver. We further show that by accounting for possible intents in human motion, the method transcends the realm of reactive avoidance to proactive anticipation of collisions and its effective avoidance, thereby increasing the overall safety of navigation.

@inproceedings{bib_Serv_2015, AUTHOR = {PANDYA, HARIT and Krishna, K Madhava and V, Jawahar C }, TITLE = {Servoing across object instances: Visual servoing for object category}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2015}}

Traditional visual servoing is able to navigate a robotic system between two views of the same object. However, it is not designed to servo between views of different objects. In this paper, we consider a novel problem of servoing any instance (exemplar) of an object category to a desired pose (view) and propose a strategy to accomplish the task. We use features that semantically encode the locations of object parts and define the servoing error as the difference between positions of corresponding parts in the image space. Our controller is based on the linear combination of 3D models, such that the resulting model interpolates between the given and desired instances. We conducted our experiments on five different object categories in simulation framework and show that our approach achieves the desired pose with smooth trajectory. Furthermore, we show the performance gain achieved by using a linear …

@inproceedings{bib_A_co_2014, AUTHOR = {Siravuru, Avinash and SHRIVASTAVA, ANKUR and PUROHIT, AKSHAYA and Shah, Suril V and Krishna, K Madhava }, TITLE = {A compliant multi-module robot for climbing big step-like obstacles}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2014}}

A novel compliant robot is proposed for traversing on unstructured terrains. The robot consists of modules, each containing a link and an active wheel-pair, and neighboring modules are connected using a passive joint. This type of robots are lighter and provide high durability due to the absence of link-actuators. However, they have limited climbing ability due to tendency of tipping over while climbing big obstacles. To overcome this disadvantage, the use of compliant joints is proposed in this work. Stiffness of each compliant joint is estimated by formulating an optimization problem with an objective to minimize link joint moments while maintaining static-equilibrium. This is one of the key novelties of the proposed work. A design methodology is also proposed for developing an n-module compliant robot for climbing a given height on a known surface. The efficacy of the proposed formulation is illustrated using numerical simulations of the three and five module robots. The robot is successfully able to climb maximum heights upto three times and six times the wheel diameter using three and five modules, respectively. A working prototype was developed and the simulation results were successfully validated on it.

@inproceedings{bib_Reac_2014, AUTHOR = {Hafez, A. H. Abdul and ANURAG, V V and Shah, Suril V and Krishna, K Madhava and V, Jawahar C }, TITLE = {Reactionless visual servoing of a dual-arm space robo}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2014}}

This paper presents a novel visual servoing controller for a satellite mounted dual-arm space robot. The controller is designed to complete the task of servoing the robot's endeffectors to the desired pose, while regulating orientation of the base-satellite. Task redundancy approach is utilized to coordinate the servoing process and attitude of the base satellite. The visual task is defined as a primary task, while regulating attitude of the base satellite to zero is defined as a secondary task. The secondary task is formulated as an optimization problem in such a way that it does not affect the primary task, and simultaneously minimizes its cost function. A set of numerical experiments are carried out on a dual-arm space robot showing efficacy of the proposed control methodology.

@inproceedings{bib_Mark_2014, AUTHOR = {Suryansh, and KARTHIK, M.SIVA and Krishna, K Madhava }, TITLE = {Markov Random Field based small obstacle discovery over images}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2014}}

Small obstacles of the order of 0.5-3cms and homogeneous scenes often pose a problem for indoor mobile robots. These obstacles cannot be clearly distinguished even with the state of the art depth sensors or laser range finders using existing vision based algorithms. With the advent of sophisticated image processing algorithms like SLIC [1] and LSD [9], it is possible to extract rich information from an image which led us to develop a novel architecture to detect very small obstacles on the floor using a monocular camera. This information is further processed using a Markov Random Field based graph cut formalism that precisely segments the floor and detects obstacles which are extremely low. We show robust and accurate obstacle detection and floor segmentation in diverse environments over a large variety of objects found indoors. In our case, low lying obstacles, changing floor patterns and extremely homogeneous environments are properly classified which leads to a drastic decrease in the number of obstacles that may not be classified by existing robotic vision algorithms.

@inproceedings{bib_Post_2014, AUTHOR = {SINGH, SARTAJ and Singh, Sartaj and Krishna, K Madhava }, TITLE = {Posture control of a three-segmented tracked robot with torque minimization during step climbing}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2014}}

In this paper, we present a posture control scheme for step climbing by an in-house developed three-segmented tracked robot, miniUGV. The posture control scheme results in minimum torque at the actuated joints of the segments. Non-linear optimization is carried out offline for progressively decreasing distance of the robot from the step with torque minimization as objective function and force balance, motor torque limits, slippage avoidance and interference avoidance constraints. The resulting angles of the joints are fitted to a third degree polynomial as a function of the robot distance from the step and the step height. It is shown that a single set of polynomial functions is sufficient for climbing steps of all permissible heights and angles of attack of the front segment. The methodology has been verified through simulation followed by implementation on the real robot. As a consequence of this optimization we find that the average current reduced by more than thirty percent, reducing power consumption and confirming the efficacy of the optimization framework

@inproceedings{bib_Smal_2014, AUTHOR = {Mittal, Sudhanshu and KARTHIK, M.SIVA and Suryansh, and Krishna, K Madhava }, TITLE = {Small object discovery and recognition using actively guided robot}, BOOKTITLE = {International conference on Pattern Recognition}. YEAR = {2014}}

In the field of active perception, object search is a widely studied problem. To search for an object in large rooms, it would be expensive to explore and check each object's similarity with the object of interest. The expense could uncontrollably bloat as the number of objects to be searched increases. If the objects are of the order of a 2-5cm, they appear very small, making it difficult for the present algorithms to recognize them. A general human strategy in such cases is to sparsely identify, from far away (4-6m), if the object of interest is present in the scene. Subsequently, each of the possible objects is analysed from closer proximity to recognize, for further manipulation. In this work, we present a similar framework. We reduce search-space, by identifying existential probability of a small object from a distance followed by a closer 3-D analysis of its point cloud to accurately recognize it. This is achieved by 2-D modelling of the objects using Gaussian Mixture Models followed by recognizing objects using efficient RGB-Depth based algorithm.

@inproceedings{bib_Time_2014, AUTHOR = {GOPALAKRISHNAN, BHARATH and SINGH, ARUN KUMAR and Krishna, K Madhava }, TITLE = {Time scaled collision cone based trajectory optimization approach for reactive planning in dynamic environments}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2014}}

The current paper proposes a trajectory optimization approach for navigating a non-holonomic wheeled mobile robot in dynamic environments. The dynamic obstacle's motion is not known and hence is represented by a band of predicted trajectories. The trajectory optimization can account for large number of predicted obstacle trajectories and seeks to avoid each predicted trajectory of every obstacle in the sensing range of the robot. The two primary contributions of the proposed trajectory optimization are (1): A computationally efficient method for computing the intersection space of collision avoidance constraints of large number of predicted obstacle trajectories. (2): A optimization framework to connect the current state to the solution space in time optimal fashion. The intersection/solution space computation is build on our earlier proposed concept of time scaled collision cone, which can be solved in closed form to obtain a set of formulae. These formulae describe how much and in what manner the temporal specification of a trajectory needs to be changed to avoid a given set of dynamic obstacles. This allows us to quickly evaluate solution space of time scaled collision cone over various candidate trajectories, thus reducing the problem of computing the intersection space to that of generating multiple homotopic trajectories. The optimization framework used to connect the current state to the solution space in time optimal fashion is based on the concept of non-linear time scaling, which induces a difference of convex form structure. Thus, on the theoretical side, we show that the various components of the proposed framework are computationally simple and involves solving sets of linear equations and using state of the art convex programming techniques. On the practical side we show that the proposed planner performs better than sampling based planners which treat dynamic obstacles as static over a short duration of time.

@inproceedings{bib_Gues_2014, AUTHOR = {KARTHIK, M.SIVA and MITTAL, SUDHANSHU and Krishna, K Madhava }, TITLE = {Guess from Far, Recognize when Near: Searching the Floor for Small Objects}, BOOKTITLE = {Indian Conference on Computer Vision, Graphics and Image Processing}. YEAR = {2014}}

In indoor environments, there would be several small objects lying around on the floor. In this work, we develop an efficient strategy to search for a set of queried objects amongst a large number of small objects lying around. Small objects of the order of 1cm – 5cm, appear very small, making it difficult for the present algorithms to recognize them from far away. A human like strategy in such cases is to infer each object's similarity to the queried objects, from far away. Subsequently, the objects of interest are approached and analyzed from a closer proximity through an optimal plan. We develop an optimal plan for the robot, to strategically visit a selected few among all the objects. From far away, we assign Existential Probabilities to the objects, indicating their similarity to queried objects. A Bayes' Net is constructed over the probabilities, to overlay and orient a Viewpoint Object Potential(VOP) map over potential search objects. VOP quantifies the probability of accurately recognizing an object through its RGB-D Point Cloud at various viewpoints. The belief from the Bayes' Net and the discriminative viewpoints from the VOP are utilized to formulate a Decision Tree which helps in building an optimal control plan. Hence, the robot reaches strategic viewpoints around potential objects, to recognize them through their RGB-D point clouds. The framework is experimentally evaluated using Kinect mounted on a Turtlebot using ROS platform.

@inproceedings{bib_Crf__2014, AUTHOR = {UPADHYAY, SARTHAK and Suryansh, and Krishna, K Madhava }, TITLE = {Crf based frontier detection using monocular camera}, BOOKTITLE = {Indian Conference on Computer Vision, Graphics and Image Processing}. YEAR = {2014}}

Frontier detection is a critical component in indoor mobile robot exploration, wherein the robot decides the next best location to move in order to continue with its mapping process. All frontier detection algorithms to the best of our knowledge require 3D locations of occupied regions as its input. In a monocular setting this entails a backend VSLAM algorithm that reconstructs the scene as the robot moves. Most monocular SLAM algorithms however provide sparse scene reconstruction from which frontiers cannot be reliably detected and estimated. In this effort we provide an alternate method of detecting frontiers during the course of robot motion that circumvents the requirement of dense mapping. Based on the observation that frontiers typically occur around vertical edges of walls, doors or tables we propose a novel linear chain CRF formulation that is able to detect the presence or absence of such frontier regions around such vertical edges. We used cues like increase in number of ground plane pixels and change in the spreading of optical flow vector, around those vertical edges. We also demonstrate that this method gives us more relevant frontiers as compared to methods based on reconstructing the scene through state-of-the art such SLAM algorithms such as PTAM. Finally, we present results in indoor scenes wherein frontiers are reliably detected around wall edges leading to new corridors, door edges leading to new rooms or corridors and table edges that opens up to a new space in rooms.

@inproceedings{bib_Top__2014, AUTHOR = {SINGHAL, PRATEK and Rajiv, Deshpande Aditya and PANDYA, HARIT and REDDY, NARAPUREDDY DINESH and Krishna, K Madhava }, TITLE = {Top down approach to detect multiple planes from pair of images}, BOOKTITLE = {Indian Conference on Computer Vision, Graphics and Image Processing}. YEAR = {2014}}

Detecting multiple planes in images is a challenging problem, but one with many applications. Recent work such as J-Linkage and Ordered Residual Kernels have focussed on developing a domain independent approach to detect multiple structures. These multiple structure detection methods are then used for estimating multiple homographies given feature matches between two images. Features participating in the multiple homographies detected, provide us the multiple scene planes. We show that these methods provide locally optimal results and fail to merge detected planar patches to the true scene planes. These methods use only residues obtained on applying homography of one plane to another as cue for merging. In this paper, we develop additional cues such as local consistency of planes, local normals, texture etc. to perform better classification and merging. We formulate the classification as an MRF problem and use TRWS message passing algorithm to solve non metric energy terms and complex sparse graph structure. We show results on Michigan Indoor Corridor Dataset and our challenging dataset, common in robotics navigation scenarios. Experiments on the datasets demonstrate the accuracy of our plane detection relative to ground truth, with detailed comparisons to prior art.

@inproceedings{bib_Sema_2014, AUTHOR = {REDDY, NARAPUREDDY DINESH and SINGHAL, PRATEK and Krishna, K Madhava }, TITLE = {Semantic Motion Segmentation Using Dense CRF Formulation}, BOOKTITLE = {Indian Conference on Computer Vision, Graphics and Image Processing}. YEAR = {2014}}

While the literature has been fairly dense in the areas of scene understanding and semantic labeling there have been few works that make use of motion cues to embellish semantic performance and vice versa. In this paper, we address the problem of semantic motion segmentation, and show how semantic and motion priors augments performance. We propose an algorithm that jointly infers the semantic class and motion labels of an object. Integrating semantic, geometric and optical flow based constraints into a dense CRF-model we infer both the object class as well as motion class, for each pixel. We found improvement in performance using a fully connected CRF as compared to a standard clique-based CRFs. For inference, we use a Mean Field approximation based algorithm. Our method outperforms recently proposed motion detection algorithms and also improves the semantic labeling compared to the state-of-the-art Automatic Labeling Environment algorithm on the challenging KITTI dataset especially for object classes such as pedestrians and cars that are critical to an outdoor robotic navigation scenario.

@inproceedings{bib_Line_2014, AUTHOR = {TOURANI, SIDDHARTH and CHHAYA, FALAK PIYUSHBHAI and Krishna, K Madhava }, TITLE = {Linear-chain CRF based intersection recognition}, BOOKTITLE = {International Conference on Vehicular Electronics and Safety}. YEAR = {2014}}

For autonomous navigation in urban environments, the ability to detect road intersections in advance is crucial, especially in the absence of auxiliary geographic information. In this paper we investigate a 3D Point Cloud based solution for intersection recognition and road segment classification. We set up the intersection recognition problem as one of decoding a linear-chain Conditional Random Field (CRF). This allows us to encode temporal consistency relations between adjacent scans in our process, leading to a less error prone recognition algorithm. We quantify this claim experimentally. We first build a grid map of the point cloud, segmenting the region surrounding the robot into navigable and non-navigable regions. Then, based on our proposed beam model, we extract a descriptor of the scene. This we do as each scan is received from the robot. Based on the descriptor we build a linear chain-CRF. By decoding the CRF-chain we are able to recognize the type of road segment taken into consideration. With the proposed method, we are able to recognize Xjunctions, T-shaped intersections and standard non-branching road segments. We compare the CRF-based approach with a standard SVM based one and show performance gain due to the CRF formulation.

@inproceedings{bib_Auto_2014, AUTHOR = {BIPIN, KUMAR and DUGGAL, VISHAKH and Krishna, K Madhava }, TITLE = {Autonomous navigation of generic Quadrocopter with minimum time trajectory planning and control}, BOOKTITLE = {International Conference on Vehicular Electronics and Safety}. YEAR = {2014}}

The challenges in generating minimum time trajectory and control for generic quadrocopter flying through sophisticated and unknown environment are explored in this paper. The proposed method uses convex programming technique to optimize polynomial splines, which are numerically stable for high-order including large number of segments and easily formulated for efficient computation. Moreover, exploiting the differential flatness of system, these polynomial trajectories encode the dynamics and constraints of the vehicle and decouple them from trajectory planning. The framework is fast enough to be performed in real time and results in solution which is close to time optimal. As control inputs are computed from the generated trajectory in each update, they are applicable to achieve closed-loop control similar to model predictive controller.

@inproceedings{bib_WAY-_2013, AUTHOR = {SRIRANJAN, R. and Krishna, K Madhava }, TITLE = {WAY-GO torch: An intelligent flash light}, BOOKTITLE = {International Conference on Robotics and Biomimetics}. YEAR = {2013}}

Here the prototype of an intelligent flash light is presented which helps for outdoor navigation by projecting a directional arrow that guides a user from one point to another. The torchlight can be used for navigating in new campuses, can be used as a tool for search and rescue and also can be used for trekking and biking. By directly projecting the associated meta data like name of the place, distance to target and the heading information the torch eliminates the overhead involved in reading a map and comparing it with the surroundings to further decipher the map. It is an intelligent-ubiquitous flash light which over comes the limitation imposed by limited screen size of a GPS cellphone, a GPS watch or a GPS navigating device used in a car.

@inproceedings{bib_RAMA_2013, AUTHOR = {SRIRANJAN, R. and Krishna, K Madhava }, TITLE = {RAMA-1 Highly Dexterous 48DOF Robotic Hand using magnetic spherical joints}, BOOKTITLE = {International Conference on Robotics and Biomimetics}. YEAR = {2013}}

Here we present the design of a highly dexterous 48 dof robotic hand. This high dexterity was possible because of the unique design of joints based on magnetic sliding and spherical spheres. The hand is tendon driven and is portable. We believe this is the robotic hand with the highest degree of freedom till date. Dexterity in robotic hands is an important parameter towards the complexity of the tasks they can do. It has been a design challenge to build an anthropomorphic robotic hand that can completely replicate the human hand in terms of its motion, torque and form factor. Here we have looked at the problem from a design perspective and achieved a hand with degrees of freedom more than the human hand in itself.

@inproceedings{bib_Fusi_2013, AUTHOR = {Suryansh, and Krishna, K Madhava }, TITLE = {Fusing Appearance and Geometric Cues for Adaptive Floor Segmentation over Images}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2013}}

This paper presents an effective method of fusing appearance and geometric (homography) cues that provides for accurate and reliable floor segmentation in significantly difficult situations hitherto not demonstrated in literature. A robot equipped with a monocular camera is able to segment floors over extremely long indoor sequences, when illumination conditions contrast over a single view, when the floor and non floor regions are of same color, when the texture of floor varies over a single image, when features are lost while the robot rotates and in the presence of extremely low lying obstacles. This reliability stems from an organic blending of appearance and homography cues at various levels and most importantly through a Recursive Bayes Filter formalism. Herein the appearance and homography based error models are appropriately weighed to come up with the eventual class probability of a feature track. Equally important the paper presents ways to mitigate the problems posed by virtual plane. Virtual plane problem occurs when obstacles of very low height classified as belonging to the ground by homography. We show extensive experimental results portraying accurate and reliable segmentation in situations where virtual plane problem becomes prominent and also in other challenging situations mentioned earlier.

@inproceedings{bib_Mult_2013, AUTHOR = {NAMDEV, RAHUL KUMAR and Krishna, K Madhava and V, Jawahar C }, TITLE = {Multibody vslam with relative scale solution for curvilinear motion reconstruction}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2013}}

A solution to the relative scale problem where reconstructed moving objects and the stationary world are represented in a unified common scale has proven equivalent to a conjecture. Motion reconstruction from a moving monocular camera is considered ill posed due to known problems of observability. We show for the first time several significant motion reconstruction of outdoor vehicles moving along non-holonomic curves and straight lines. The reconstructed motion is represented in the unified frame which also depicts the estimated camera trajectory and the reconstructed stationary world. This is possible due to our Multibody VSLAM framework with a novel solution for relative scale proposed in the current paper. Two solutions that compute the relative scale are proposed. The solutions provide for a unified representation within four views of reconstruction of the moving object and are thus immediate. In one, the solution for the scale is that which satisfies the planarity constraint of the object motion. The assumption of planar object motion while being generic enough is subject to stringent degenerate situations that are more widespread. To circumvent such degeneracies we assume that the object motion to be locally circular or linear and find the relative scale solution for such object motions. Precise reconstruction is achieved in synthetic data. The fidelity of reconstruction is further vindicated with reconstructions of moving cars and vehicles in uncontrolled outdoor scenes.

@inproceedings{bib_A_si_2013, AUTHOR = {ADITYA, GATTUPALLI and PRAKASH, E VIJAY and SINGH, ARUN KUMAR and Krishna, K Madhava }, TITLE = {A simulation framework for evolution on uneven terrains for synchronous drive robot}, BOOKTITLE = {Advances in Robotics}. YEAR = {2013}}

This paper presents a simulation framework for evolution on uneven terrains for a wheeled mobile robot (WMR) such as a synchronous drive robot. The framework lends itself as a tool capable of solving various problems, such as forward kinematic-based evolution, inverse kinematic-based evolution, path planning and trajectory tracking. This framework becomes particularly useful when we understand that the evolution problem (and hence, the various associated problems based on evolution) is particularly challenging on uneven terrain. Specifically, it is entailed to bring in the contact constraints posed by the interaction of the wheel and the ground as well as the holonomic constraints as the problem is formulated in a Differential Algebraic Equation setting. The problem becomes all the more crucial as vehicles moving on uneven terrain are becoming the order of the day. Nonetheless, there has not been much literature that deals in length the various aspects that go into the framework. This paper elaborates on the various aspects of the framework, presents simulation results on uneven terrain, where the vehicle evolves without slipping, and also presents substantial quantitative analysis in regard to wheel slippage. The main contributions of this paper are the motion planning using forward kinematic framework and a new formulation of inverse kinematics for wheeled robots on uneven terrains.

@inproceedings{bib_Low__2013, AUTHOR = {PRADHAN, NEERAJ and DUBEY, ROOPAK and Krishna, K Madhava and Chowdhury, Shubhajit Roy }, TITLE = {Low Power Two-Tier GALS Architecture for Multi Robot Collision Avoidance}, BOOKTITLE = {Advances in Robotics}. YEAR = {2013}}

This paper presents a Field Programmable Gate Array (FPGA) based implementation of Acceleration Velocity Obstacle based Collision Avoidance for an omni-directional robot with acceleration constraint. A novel Globally Asynchronous-Locally Synchronous (GALS) architecture using hybrid parallel-pipelined design is being used for the implementation. FPGA offers highly parallel hardware architectures that are not possible on conventional high end processors or other embedded controllers. Specifically, a parallel architecture for collision avoidance is proposed that portrays the advantages of FPGA implementation over the sequential implementation for same processor or clock speed with the objective of parallelizing the computation. The proposed two-tier GALS architecture based system with hybrid parallel-pipelined architectural design when realized on FPGA shows its efficiency in terms of power, resources and response time in comparison with a conventional GALS design. Due to the asynchronous design, a good amount of logic in the hardware becomes combinational logic and hence a significant reduction in the total resource utilization and power-delay product (PDP) in the proposed architecture is observed. This paper also shows the clear advantage of using the FPGA instead of a general purpose processor for robotic system design.

@inproceedings{bib_Mapp_2013, AUTHOR = {MUKHIJA, PIYOOSH and Krishna, K Madhava }, TITLE = {Mapping a Network of Roads for an On-road Navigating Robot}, BOOKTITLE = {Advances in Robotics}. YEAR = {2013}}

We present an autonomous system for outdoor terrain (road) mapping using a robot equipped with viable low cost sensors (2D laser scanner, cameras, odometry, gyroscope and commercial GPS). The research work on outdoor navigation with known maps has matured well in past few years, but navigation and exploration of roads in an unknown area is still emerging. The presented system is an amalgamation of various computation modules running in parallel and interacting asynchronously with each other through message queues. The goal of the system is for a vehicle to explore and build a road network graph of a given area by autonomously navigating through all the connected roads in the area. The graph nodes contain topographic and semantic properties to allow reconstruction of the road network at a later stage for easier navigation.

@inproceedings{bib_Mult_2013, AUTHOR = {PAREKH, TEJAS PRAKASH and SINGH, ARUN KUMAR and Krishna, K Madhava }, TITLE = {Multi Robot Collision Avoidance with Continuous Curvature Manoeuvres}, BOOKTITLE = {Advances in Robotics}. YEAR = {2013}}

In this paper, we address the problem of reactive collision avoidance among multiple robots. However we impose the constraint of continuous curvature on the collision avoidance manoeuvre which adds to the complexity of the problem but is necessary for realistic implementation of collision avoidance among multiple non-holonomic robots like tricycles and car-like. Since these non-holonomic robots cannot execute point-turn motion, performing collision avoidance with these robots while maintaining curvature continuity becomes a very challenging problem. Hence the proposed work comes as an enhancement over the existing reactive collision avoidance frameworks which completely neglect the above aspects. We modify the collision cone approach of collision avoidance to the case of multiple robots moving with non-linearly varying velocities. This is done by approximating the vehicle evolution with a smooth function represented as a linear combination of orthogonal basis functions and writing the collision avoidance condition as analytical function with respect to time. As a direct consequence of that, the reactive collision avoidance and goal-reaching could be solved as a constrained optimization problem. The theory is substantiated by extensive simulations for various bench-mark cases.

@inproceedings{bib_UGV-_2013, AUTHOR = {ARAVINDH, M. and DEWAN, AYUSH and SONI, NIKHIL and Krishna, K Madhava }, TITLE = {UGV-MAV collaboration for augmented 2d maps}, BOOKTITLE = {Advances in Robotics}. YEAR = {2013}}

Over the past couple of years, with the development of efficient control algorithms, micro aerial vehicles have come into the picture. In this paper, we consider the problem of creating a map of an indoor environment which provides more information than a 2D map and at the same time is more accurate than the contemporary 3D mapping algorithms. We propose a novel collaborative system, consisting of an unmanned ground vehicle and a micro aerial vehicle, which is used to create augmented 2D maps using the distinct sensing capabilities of these two robots. This system works in a collaborative manner such that the two robots complement each others movement and sensing capabilities.

@inproceedings{bib_Cont_2013, AUTHOR = {ADITYA, GATTUPALLI and Shah, Suril V and Krishna, K Madhava and Misra, A. K. }, TITLE = {Control strategies for reactionless capture of an orbiting object using a satellite mounted robot}, BOOKTITLE = {Advances in Robotics}. YEAR = {2013}}

This paper presents a method to capture the orbiting objects using a robotic system mounted on a service satellite. The main objective is to manipulate the robot such that no reaction moment gets transferred to the base satellite. This will avoid use of any attitude controller resulting in fuel savings. Note that the constraints leading to zero reaction moment are nonholonomic, and this makes path planning a complex problem. In this work, first a method based on holonomic distribution of the nonholonomic constraints is discussed. As this method exploits constraints in terms of joint velocities, it does not always ensure successful capture. Next, a method based on task-level constraints, written in terms of end-effector's velocities, has been illustrated. It is shown that the path planned using this method has several singular points. In order to overcome disadvantages of the above two methods a novel approach is proposed which uses holonomic distribution to reach closer to the target and task-level constraints to finally capture the target. Efficacy of the method is shown using a 3-link robot mounted on a service satellite.

@inproceedings{bib_Homo_2013, AUTHOR = {GAVSHINDE, LAXIT and Krishna, K Madhava }, TITLE = {Homography based monocular dense reconstruction for a Mobile robot}, BOOKTITLE = {Advances in Robotics}. YEAR = {2013}}

Single camera mobile robots, wherein the single camera becomes the quintessential sensor for robotic tasks such as localization, mapping and obstacle avoidance are challenging. From such a standpoint, we demonstrate a dense reconstruction as conducted by a navigating robot with a monocular camera. Unlike most other dense reconstruction methods this approach first identifies planar areas through homography. These segments are tracked over multiple views with homography based dense correspondences. The tracked correspondences are reconstructed within a VSLAM formulation, wherein the dense reconstructed points get added to the existing SLAM computed structure. The dense structure is further refined using a modified Bundle Adjustment which minimizes projection error in 3D to align with a inferred model of the scene. A mobile robot can thus make use of the reconstructed ground plane and planar obstacles to compute a collision free trajectory.

@inproceedings{bib_A_se_2013, AUTHOR = {Siravuru, Avinash and ANURAG, V V and SINGH, ARUN KUMAR and Shah, Suril V and Krishna, K Madhava }, TITLE = {A semi-active robot for steep obstacle ascent}, BOOKTITLE = {International Conference on Control Applications}. YEAR = {2013}}

In this paper we propose a semi-active robot for climbing steep obstacles like steps, curbs, etc. The key novelty of the proposed robot lies in the use of a passive mechanism for climbing steps of smaller heights and motor only while climbing steps of bigger heights. Analysis of the robot's stability during its ascent phase is also investigated. Model based control is used to achieve step climbing. The other novelty of the robot, in contrast to existing active suspension step climbers, is that it does not need the knowledge of step height beforehand. Therefore, the mechanism has the advantage of height-independent climbing motion as in the case of passive mechanism along with the extra freedom of active joints for maintaining vehicle stability, only when required. Efficacy of the mechanism is exhibited through simulations on steps of various heights.

@inproceedings{bib_Gait_2013, AUTHOR = {SINGH, SARTAJ and Krishna, K Madhava }, TITLE = {Gait sequence generation of a hybrid wheeled-legged robot for negotiating discontinuous terrain}, BOOKTITLE = {International Conference on Control Applications}. YEAR = {2013}}

In this paper we develop an algorithm to generate gait sequences to negotiate a discontinuous terrain for a hybrid 4-wheeled legged robot. The gait sequence comprises two main steps - normal force redistribution and hybrid position-force control. The robot climbs the discontinuity one leg at a time. This requires that the entire load of the robot is taken up by the other three legs so that the leg climbing the discontinuity is free. For this purpose a load redistribution methodology is used which makes the center of gravity of chassis coincide with the desired center of pressure (CoP). Subsequently the free leg moves in hybrid position and force control to climb the discontinuity. Force sensing ensures constant contact with the terrain and detection of stand and end of the discontinuity without using any perception sensor. The methodology is validated using multi-body dynamic simulation.

@inproceedings{bib_Traj_2013, AUTHOR = {GAVSHINDE, LAXIT and SINGH, ARUN KUMAR and Krishna, K Madhava }, TITLE = {Trajectory planning for monocular SLAM systems}, BOOKTITLE = {International Conference on Control Applications}. YEAR = {2013}}

This paper proposes a novel method of integrating planning with Monocular Simultaneous Localization and Mapping (SLAM) systems. Monocular SLAM, typically referred to as VSLAM systems in literature consists of recovering trajectory estimates of the camera and stationary world features from a single moving camera. Such VSLAM systems are significantly more difficult than SLAM performed with depth sensors, such as using an accurate Laser Range Finder (LRF). When the camera motion is subject to steep changes in orientation, tracked features over the previous instances are lost, making VSLAM estimates highly unreliable, erroneous that cannot be recovered. Most often a complete breakdown occurs, which entails a new sequence of images to be captured from a fresh camera trajectory. Herein we propose an optimization based path planning formulation for such VSLAM systems that reduces occurence of such errors through paths that are not subject to high orientation changes. Further we plan a velocity profile over the path that prevents features from getting significantly displaced over successive images, often considered a critical criteria for robust feature tracking. The velocity profile is computed using the novel concept of non linear time scaling proposed in our earlier work. The VSLAM system is also sufficiently innovated to provide for dense mapping over planar segments. The efficacy of the formulation is verified over real experiments on a camera mounted robot.

@inproceedings{bib_Dept_2013, AUTHOR = {D, KARTHIK. and Krishna, K Madhava and Rajan, Deepu and V, Jawahar C }, TITLE = {Depth really Matters: Improving Visual Salient Region Detection with Depth.}, BOOKTITLE = {British Machine Vision Conference}. YEAR = {2013}}

Depth information has been shown to affect identification of visually salient regions in images. In this paper, we investigate the role of depth in saliency detection in the presence of (i) competing saliencies due to appearance,(ii) depth-induced blur and (iii) centre-bias. Having established through experiments that depth continues to be a significant contributor to saliency in the presence of these cues, we propose a 3D-saliency formulation that takes into account structural features of objects in an indoor setting to identify regions at salient depth levels. Computed 3D saliency is used in conjunction with 2D saliency models through non-linear regression using SVM to improve saliency maps. Experiments on benchmark datasets containing depth information show that the proposed fusion of 3D saliency with 2D saliency models results in an average improvement in ROC scores of about 9% over state-of-the-art 2D saliency models.

@inproceedings{bib_Visu_2013, AUTHOR = {Hafez, A. H. Abdul and SINGH, MANPREET and Krishna, K Madhava and V, Jawahar C }, TITLE = {Visual localization in highly crowded urban environments}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2013}}

Visual localization in crowded dynamic environments requires information about static and dynamic objects. This paper presents a robust method that learns the useful features from multiple runs in highly crowded urban environments. Useful features are identified as distinctive ones that are also reliable to extract in diverse imaging conditions. Relative importance of features is used to derive the weight for each feature. The popular Bag-of-words model is used for image retrieval and localization, where query image is the current view of the environment and database contains the visual experience from previous runs. Based on the reliability, features are augmented and eliminated over runs. This reduces the size of representation, and makes it more reliable in crowded scenes. We tested the proposed method on data sets collected from highly crowded Indian urban outdoor settings. Experiments have shown that with the help of a small subset (10%) of the detected features, we can reliably localize the camera. We achieve superior results in terms of localization accuracy even when more than 90% of the pixels are occluded or dynamic.

@inproceedings{bib_Coor_2013, AUTHOR = {SINGH, ARUN KUMAR and Krishna, K Madhava }, TITLE = {Coordinating mobile manipulator's motion to produce stable trajectories on uneven terrain based on feasible acceleration count}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2013}}

In this paper we consider the problem of coordinating the motion of the manipulator and the vehicle to produce stable trajectories for the combined mobile manipulator system on uneven terrain. These kinds of situations often arise in planetary exploration, where rovers equipped with a manipulator are required to navigate over general uneven terrain. Moreover the framework can also be used in situations where the mobile manipulator is required to transport objects on uneven terrain. We generate feasible trajectories for the vehicle between a given start and a goal point considering the dynamics of the manipulator. The framework proposed in the paper plans such motion profile of the manipulator that maximizes vehicle stability which is measured by a novel concept called Feasible Acceleration Count (FAC). We show that, from the point of view of motion planning of mobile manipulator on uneven terrains, FAC gives a better estimate of vehicle stability than more popular metrics like Tip-Over Stability. The trajectory planner closely resembles motion primitive based graph based planning and is combined with a novel cost function derived from FAC. The efficacy of the approach is shown through simulations of a mobile manipulator system on a 2.5D uneven terrain.

@inproceedings{bib_Hete_2013, AUTHOR = {DEWAN, AYUSH and ARAVINDH, M. and SONI, NIKHIL and Krishna, K Madhava }, TITLE = {Heterogeneous UGV-MAV exploration using integer programming}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2013}}

This paper presents a novel exploration strategy for coordinated exploration between unmanned ground vehicles (UGV) and micro-air vehicles (MAV). The exploration is modeled as an Integer Programming (IP) optimization problem and the allocation of the vehicles(agents) to frontier locations is modeled using binary variables. The formulation is also studied for distributed system, where agents are divided into multiple teams using graph partitioning. Optimization seamlessly integrates several practical constraints that arise in exploration between such heterogeneous agents and provides an elegant solution for assigning task to agents. We have also presented comparison with previous methods based on distance traversed and computational time to signify advantages of presented method. We also show practical realization of such an exploration where an UGV-MAV team efficiently builds a map of an indoor environment.

@inproceedings{bib_Reac_2013, AUTHOR = {SINGH, ARUN KUMAR and Krishna, K Madhava }, TITLE = {Reactive collision avoidance for multiple robots by non linear time scaling}, BOOKTITLE = {Conference on Decision and Control}. YEAR = {2013}}

Reactive Collision avoidance for non-holonomic robots is a challenging task because of the restrictions in the space of achievable velocities. The complexity increases further when multiple non-holonomic robots are operating in tight/cluttered spaces. The present paper presents a framework specially carved out for such situations. But at the same time can be easily appended with any existing collision avoidance framework. At the crux of the methodology is the concept of non-linear time scaling which allows robots to reactively accelerate/de-accelerate without altering the geometric path. The framework introduced is completely independent of the robot kinematics and dynamics. As such it can be applied to any ground or aerial robot. Through this concept the collision avoidance is framed as a problem of choosing appropriate scaling transformations. We present a “scaled” variant of the collision cone concept which automatically induces distributiveness among robots. The efficacy of the proposed work is demonstrated through simulations of both ground as well as UAVs.

@inproceedings{bib_Top__2013, AUTHOR = {SINGHAL, PRATEK and Rajiv, Deshpande Aditya and REDDY, NARAPUREDDY DINESH and Krishna, K Madhava }, TITLE = {Top Down Approach to Multiple Plane Detection}, BOOKTITLE = {Technical Report}. YEAR = {2013}}

Detecting multiple planes in images is a challenging problem, but one with many applications. Recent work such as J-Linkage and Ordered Residual Kernels have focussed on developing a domain independent approach to detect multiple structures. These multiple structure detection methods are then used for estimating multiple homographies given feature matches between two images. Features participating in the multiple homographies detected, provide us the multiple scene planes. We show that these methods provide locally optimal results and fail to merge detected planar patches to the true scene planes. These methods use only residues obtained on applying homography of one plane to another as cue for merging. In this paper, we develop additional cues such as local consistency of planes, local normals, texture etc. to perform better classification and merging. We formulate the classification as an MRF problem and use TRWS message passing algorithm to solve non metric energy terms and complex sparse graph structure. We show results on challenging dataset common in robotics navigation scenarios where our method shows accuracy of more than 85 percent on average while being close or same as the actual number of scene planes.

@inproceedings{bib_Moti_2012, AUTHOR = {NAMDEV, RAHUL KUMAR and KUNDU, ABHIJIT and Krishna, K Madhava and V, Jawahar C }, TITLE = {Motion segmentation of multiple objects from a freely moving monocular camera}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2012}}

Motion segmentation is an inevitable component for mobile robotic systems such as the case with robots performing SLAM and collision avoidance in dynamic worlds. This paper proposes an incremental motion segmentation system that efficiently segments multiple moving objects and simultaneously build the map of the environment using visual SLAM modules. Multiple cues based on optical flow and two view geometry are integrated to achieve this segmentation. A dense optical flow algorithm is used for dense tracking of features. Motion potentials based on geometry are computed for each of these dense tracks. These geometric potentials along with the optical flow potentials are used to form a graph like structure. A graph based segmentation algorithm then clusters together nodes of similar potentials to form the eventual motion segments. Experimental results of high quality segmentation on different publicly available datasets demonstrate the effectiveness of our method.

@inproceedings{bib_Opti_2012, AUTHOR = {SINGH, ARUN KUMAR and Ghose, Debasish and Krishna, K Madhava }, TITLE = {Optimum steering input determination and path-tracking of all-wheel steer vehicles on uneven terrains based on constrained optimization}, BOOKTITLE = {American Control Conference}. YEAR = {2012}}

In this paper we propose a framework for optimum steering input determination of all-wheel steer vehicles (AWSV) on rough terrains. The framework computes the steering input which minimizes the tracking error for a given trajectory. Unlike previous methodologies of computing steering inputs of car-like vehicles, the proposed methodology depends explicitly on the vehicle dynamics and can be extended to vehicle having arbitrary number of steering inputs. A fully generic framework has been used to derive the vehicle dynamics and a non-linear programming based constrained optimization approach has been used to compute the steering input considering the instantaneous vehicle dynamics, no-slip and contact constraints of the vehicle. All Wheel steer Vehicles have a special parallel steering ability where the instantaneous centre of rotation (ICR) is at infinity. The proposed framework automatically enables the vehicle to choose between parallel steer and normal operation depending on the error with respect to the desired trajectory. The efficacy of the proposed framework is proved by extensive uneven terrain simulations, for trajectories with continuous or discontinuous velocity profile.

@inproceedings{bib_Fast_2012, AUTHOR = {Parulkar, Amey and Shukla, Piyush and Krishna, K Madhava }, TITLE = {Fast randomized planner for SLAM automation}, BOOKTITLE = {International Conference on Automation Science and Engineering}. YEAR = {2012}}

In this paper, we automate the traditional problem of Simultaneous Localization and Mapping (SLAM) by interleaving planning for exploring unknown environments by a mobile robot. We denote such planned SLAM systems as SPLAM (Simultaneous Planning Localization and Mapping). The main aim of SPLAM is to plan paths for the SLAM process such that the robot and map uncertainty upon execution of the path remains minimum and tractable. The planning is interleaved with SLAM and hence the terminology SPLAM. While typical SPLAM routines find paths when the robot traverses amidst known regions of the constructed map, herein we use the SPLAM formulation for an exploration like situation. Exploration is carried out through a frontier based approach where we identify multiple frontiers in the known map. Using Randomized Planning techniques we calculate various possible trajectories to all the known frontiers. We introduce a novel strategy for selecting frontiers which mimics Fast SLAM, selects a trajectory for robot motion that will minimize the map and robot state covariance. By using a Fast SLAM like approach for selecting frontiers we are able to decouple the robot and landmark covariance resulting in a faster selection of next best location, while maintaining the same kind of robustness of an EKF based SPLAM framework. We then compare our results with Shortest Path Algorithm and EKF based Planning. We show significant reduction in covariance when compared with shortest frontier first approach, while the uncertainties are comparable to EKF-SPLAM albeit at much faster planning times.

@inproceedings{bib_Mult_2012, AUTHOR = {PANDEY, ROMIT and SINGH, ARUN KUMAR and Krishna, K Madhava }, TITLE = {Multi-robot exploration with communication requirement to a moving base station}, BOOKTITLE = {International Conference on Automation Science and Engineering}. YEAR = {2012}}

Exploration is a core and important robotics area, whose applications include search and rescue robotics, planetary exploration etc. We know that this exploration task is best performed when using a multi-robot system. In this paper, we present an algorithm for multi-robot exploration of an unknown environment, taking into account the communication constraints between the robots. The aim of the robots is to explore the whole map as a pack, without losing communication throughout. The key task for us here is to allocate the target points for multiple robots so as to maximize the area explored and minimize the time and plan paths for the robots in such a way so as to avoid obstacles. A multi-robot exploration methodology is introduced similar to depth first strategy, that samples frontier points based on a metric function. This function aims to maximize the visibility gain or information gain while minimizing the distance to be travelled to the frontier points, such that the robots are within the limited communication distance of each other. The algorithm has been tested through simulation runs of various maps and results and evaluations have been presented based on it. The results effectively demonstrate that our algorithm allows robot pack to quickly accomplish the task of exploration and without the constraint ever breaking down. Here, we also present a comparative analysis of our algorithm with another exploration approach, which finds new areas based on population generation and utility calculation over the population. The results show tangible performance gain of this method over previous methods reported on exploration with limited communication constraints.

@inproceedings{bib_Outd_2012, AUTHOR = {MUKHIJA, PIYOOSH and TOURANI, SIDDHARTH and Krishna, K Madhava }, TITLE = {Outdoor Intersection Detection for Autonomous Exploration}, BOOKTITLE = {International Conferene on Intelligent Transportation Systems}. YEAR = {2012}}

In this paper we address the problem of detecting road intersections. We present two approaches to solve the problem of intersection detection in an unstructured outdoor setting. The first is a natural extension of the popular VFH* obstacle avoidance algorithm. It detects intersections and tracks, over a period of time, the angles at which gaps in the robot's certainty grid (CG) are first observed. The second approach uses techniques from image processing and computational geometry on the certainty grid image, to extract a skeleton of the navigable region, thus providing the intersections. We show experimental results portraying intersection detection due to both methods and show the results. On the whole, we found that the robot was able to detect all possible intersections.

@inproceedings{bib_Plan_2012, AUTHOR = {SINGH, ARUN KUMAR and Krishna, K Madhava and S, SRIKANTH }, TITLE = {Planning trajectories on uneven terrain using optimization and non-linear time scaling techniques}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2012}}

In this paper we introduce a novel framework of generating trajectories which explicitly satisfies the stability constraints such as no-slip and permanent ground contact on uneven terrain. The main contributions of this paper are: (1) It derives analytical functions depicting the evolution of the vehicle on uneven terrain. These functional descriptions enable us to have a fast evaluation of possible vehicle stability along various directions on the terrain and this information is used to control the shape of the trajectory. (2) It introduces a novel paradigm wherein non-linear time scaling brought about by parametrized exponential functions are used to modify the velocity and acceleration profile of the vehicle so that these satisfy the no-slip and contact constraints. We show that nonlinear time scaling manipulates velocity and acceleration profile in a versatile manner and consequently has exceptional utility not only in uneven terrain navigation but also in general in any problem where it is required to change the velocity of the robot while keeping the path unchanged like collision avoidance.

@inproceedings{bib_Fiel_2012, AUTHOR = {DUBEY, ROOPAK and PRADHAN, NEERAJ and Krishna, K Madhava and Chowdhury, Shubhajit Roy }, TITLE = {Field programmable gate array (fpga) based collision avoidance using acceleration velocity obstacles}, BOOKTITLE = {International Conference on Robotics and Biomimetics}. YEAR = {2012}}

This paper presents a Field Programmable Gate Array (FPGA) based implementation of Acceleration Velocity Obstacle based Collision Avoidance for an omni-directional robot with acceleration constraint. Specifically a parallel architecture for collision avoidance is proposed that portrays the advantages of FPGA implementation over the sequential implementation for same processor or clock speed. FPGA based robotics is seen to gain popularity due to low cost, portability, seamless interface to hardware and most importantly due to inherent parallelism enshrined in various robotic algorithms. FPGA realization of the algorithm in a simulation test bed vindicates its efficacy and comparison with sequential implementation is also highlighted. The paper proposes three different architectures for the implementation of the proposed algorithm viz. sequential architecture; a resource constrained pipelined architecture and a hybrid pipeline parallel architecture. The performances of those three architectures have been evaluated.

@inproceedings{bib_A_ba_2012, AUTHOR = {Suryansh, and DEWAN, AYUSH and Krishna, K Madhava }, TITLE = {A bayes filter based adaptive floor segmentation with homography and appearance cues}, BOOKTITLE = {Indian Conference on Computer Vision, Graphics and Image Processing}. YEAR = {2012}}

This paper proposes a robust approach for image based floor detection and segmentation from sequence of images or video. In contrast to many previous approaches, which uses a priori knowledge of the surroundings, our method uses combination of modified sparse optical flow and planar homography for ground plane detection which is then combined with graph based segmentation for extraction of floor from images. We also propose a probabilistic framework which makes our method adaptive to the changes in the surroundings. We tested our algorithm on several common indoor environment scenarios and were able to extract floor even under challenging circumstances. We obtained extremely satisfactory results in various practical scenarios such as where the floor and non floor areas are of same color, in presence of textured flooring, and where illumination changes are steep.

@inproceedings{bib_View_2012, AUTHOR = {D, KARTHIK. and NAGARIYA, AKHIL KUMAR and Krishna, K Madhava }, TITLE = {Viewpoint based mobile robotic exploration aiding object search in indoor environment}, BOOKTITLE = {Indian Conference on Computer Vision, Graphics and Image Processing}. YEAR = {2012}}

We present a probabilistic method of finding the next best viewpoint that maximizes the chances of finding an object in a known environment for an indoor mobile robot. We make use of the information that is available to a robot in the form of potential locations to search for an object. Extraction of these potential locations and their representation for exploration is explained. This work primarily focuses on placing the robot at its best location in the environment to detect, recognize an object and hence do object search. With experiments done on the exploration, object recognition individually we show the robustness of this approach for object search task. We analyse and compare our method with two other strategies for localizing the object empirically and show unequivocally that the strategy based on the probabilistic formalism in general performs better than the other two.

@inproceedings{bib_Larg_2011, AUTHOR = {ACHAR, SUPREETH and V, Jawahar C and Krishna, K Madhava }, TITLE = {Large scale visual localization in urban environments}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2011}}

This paper introduces a vision based localization method for large scale urban environments. The method is based upon Bag-of-Words image retrieval techniques and handles problems that arise in urban environments due to repetitive scene structure and the presence of dynamic objects like vehicles. The localization system was experimentally verified it localization experiments along a 5km long path in an urban environment.

@inproceedings{bib_Real_2011, AUTHOR = {KUNDU, ABHIJIT and Krishna, K Madhava and V, Jawahar C }, TITLE = {Realtime multibody visual SLAM with a smoothly moving monocular camera}, BOOKTITLE = {International Conference on Computer Vision}. YEAR = {2011}}

This paper presents a realtime, incremental multibody visual SLAM system that allows choosing between full 3D reconstruction or simply tracking of the moving objects. Motion reconstruction of dynamic points or objects from a monocular camera is considered very hard due to well known problems of observability. We attempt to solve the problem with a Bearing only Tracking (BOT) and by integrating multiple cues to avoid observability issues. The BOT is accomplished through a particle filter, and by integrating multiple cues from the reconstruction pipeline. With the help of these cues, many real world scenarios which are considered unobservable with a monocular camera is solved to reasonable accuracy. This enables building of a unified dynamic 3D map of scenes involving multiple moving objects. Tracking and reconstruction is preceded by motion segmentation and detection which makes use of efficient geometric constraints to avoid difficult degenerate motions, where objects move in the epipolar plane. Results reported on multiple challenging real world image sequences verify the efficacy of the proposed framework.

@inproceedings{bib_Plan_2011, AUTHOR = {SINGH, ARUN KUMAR and Krishna, K Madhava and PRAKASH, E VIJAY }, TITLE = {Planning stable trajectory on uneven terrain based on feasible acceleration count}, BOOKTITLE = {Conference on Decision and Control and European Control Conference}. YEAR = {2011}}

In this paper we propose a novel physics based motion planning and trajectory generation framework for vehicle operating on uneven terrains. The proposed framework provides for a fully 3D analysis of the dynamic constraints of the vehicle on uneven terrain and hence comes as a better approach than the existing motion planning framework which makes simplifying assumptions for the terrain conditions or the vehicle geometry or both. The entire framework consists of three major parts which are: 1. A framework for determination of the posture of a vehicle in 3D for a given terrain. 2. A framework for determination of maximum feasible velocities and acceleration based on contact and no-slip constraints. 3. Combining the above two framework to generate feasible trajectories for the vehicle. Trajectories are generated through a Dynamic Window paradigm extended to fully 3D terrains, wherein the next best node is selected through a new metric that maximizes the space of feasible velocities and accelerations and reduces the distance to be traversed to the goal.

@inproceedings{bib_Fast_2010, AUTHOR = {JAKKOJU, CHETAN and Krishna, K Madhava and V, Jawahar C }, TITLE = {Fast and spatially-smooth terrain classification using monocular camera}, BOOKTITLE = {International conference on Pattern Recognition}. YEAR = {2010}}

In this paper, we present a monocular camera based terrain classification scheme. The uniqueness of the proposed scheme is that it inherently incorporates spatial smoothness while segmenting a image, without requirement of post-processing smoothing methods. The algorithm is extremely fast because it is build on top of a Random Forest classifier. The baseline algorithm uses color, texture and their combination with classifiers such as SVM and Random Forests. We present comparison across features and classifiers. We further enhance the algorithm through a label transfer method. The efficacy of the proposed solution can be seen as we reach a low error rates on both our dataset and other publicly available datasets.

@inproceedings{bib_Real_2010, AUTHOR = {KUNDU, ABHIJIT and Krishna, K Madhava and V, Jawahar C }, TITLE = {Realtime motion segmentation based multibody visual SLAM}, BOOKTITLE = {Indian Conference on Computer Vision, Graphics and Image Processing}. YEAR = {2010}}

In this paper, we present a practical vision based Simultaneous Localization and Mapping (SLAM) system for a highly dynamic environment. We adopt a multibody Structure from Motion (SfM) approach, which is the generalization of classical SfM to dynamic scenes with multiple rigidly moving objects. The proposed framework of multibody visual SLAM allows choosing between full 3D reconstruction or simply tracking of the moving objects, which adds flexibility to the system, for scenes containing non-rigid objects or objects having insufficient features for reconstruction. The solution demands a motion segmentation framework that can segment feature points belonging to different motions and maintain the segmentation with time. We propose a realtime incremental motion segmentation algorithm for this purpose. The motion segmentation is robust and is capable of segmenting difficult degenerate motions, where the moving objects is followed by a moving camera in the same direction. This robustness is attributed to the use of efficient geometric constraints and a probability framework which propagates the uncertainty in the system. The motion segmentation module is tightly coupled with feature tracking and visual SLAM, by exploring various feed-backs in between these modules. The integrated system can simultaneously perform realtime visual SLAM and tracking of multiple moving objects using only a single monocular camera.

@inproceedings{bib_"_Mo_2010, AUTHOR = {SRIRANJAN, R. and Krishna, K Madhava and Indurkhya, Bipin }, TITLE = {" Mod-Leg" a modular legged robotic system}, BOOKTITLE = {International Conference on Computer Graphics and Interactive Techniques}. YEAR = {2010}}

The Modular Legged robotic system [1] "Mod-Leg" presented here has been bio-inspired from a Snake's vertebrae and a caterpillar's legged structure. The system can be configured to a 4-legged robotic dog, a hexapod, a caterpillar and a Snake robot. This robot's novel design achieves compliance to the terrain using a combination of legs and electronically actuated universal spine. A unique simulator has been designed for this purpose. Some of the things we learned while developing this robotic system have been presented below.

@inproceedings{bib_Part_2010, AUTHOR = {MOHAN, MAHESH and Krishna, K Madhava }, TITLE = {Particle Filter based Scan Correlation}, BOOKTITLE = {IFAC-Proceedings}. YEAR = {2010}}

The process of scancorrelation is very useful in map building and localization in environments without any easily discernible landmarks. It involves estimating the relative transform between two given scans of data. This can be formulated as a posterior estimation problem over the transformation space. Since the posterior, the action model and the likelihood function are mostly nonlinear, a particle filter is used to estimate this posterior [1]. A GMM based correlation scheme has been used in [2] to drive the scan correlation process. In this paper, we propose a 3 step particle filtering approach that combines the simplicity and efficiency of the particle filter with the robustness of the GMM based correlation scheme. Experimental results are provided to verify the effectiveness of this approach.

@inproceedings{bib_Moti_2010, AUTHOR = {BHUVANAGIRI, SHIVUDU and Krishna, K Madhava }, TITLE = {Motion in ambiguity: Coordinated active global localization for multiple robots}, BOOKTITLE = {Robotics and Autonomous systems}. YEAR = {2010}}

The task of the robot in localization is to find out where it is, through sensing and motion. In environments which possess relatively few features that enable a robot to unambiguously determine its location, global localization algorithms can result in ‘multiple hypotheses’ locations of a robot. This is inevitable with global localization algorithms, as the local environment seen by a robot repeats at other parts of the map. Thus, for effective localization, the robot has to be actively guided to those locations where there is a maximum chance of eliminating most of the ambiguous states — which is often referred to as ‘active localization’. When extended to multi-robotic scenarios where all robots possess more than one hypothesis of their position, there is an opportunity to do better by using robots, apart from obstacles, as ‘hypotheses resolving agents’. The paper presents a unified framework which accounts for the map structure as well as measurement amongst robots, while guiding a set of robots to locations where they can singularize to a unique state. The strategy shepherds the robots to places where the probability of obtaining a unique hypothesis for a set of multiple robots is a maximum. Another aspect of framework demonstrates the idea of dispatching localized robots to locations where they can assist a maximum of the remaining unlocalized robots to overcome their ambiguity, named as ‘coordinated localization’. The appropriateness of our approach is demonstrated empirically in both simulation & real-time (on Amigo-bots) and its efficacy verified. Extensive comparative analysis portrays the advantage of the current method over others that do not perform active localization in a multi-robotic sense. It also portrays the performance gain by considering map structure and robot placement to actively localize over methods that consider only one of them or neither. Theoretical backing stems from the proven completeness of the method for a large category of diverse environments.

@inproceedings{bib_A_tw_2010, AUTHOR = {MUKHIJA, PIYOOSH and Krishna, K Madhava and Krishna, Vamshi }, TITLE = {A two phase recursive tree propagation based multi-robotic exploration framework with fixed base station constraint}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2010}}

A multi-robotic exploration with the requirement of communication link to a fixed base station is presented in this paper. The robots organize themselves into roles of maintainers of communication (hinged robots or robot nodes) or explorers of the environment ensuring that every robot is in contact with the base station directly or through the hinged robots. A two phased strategy for the same is presented. The first phase is characterized by a recursive growth of trees that starts from the root node or the base station and then repeated from other nodes of the hitherto grown tree in a depth first fashion. The second phase constitutes the recursive tree growth invoked repeatedly from the frontier nodes. While the first phase rapidly explores areas around the base station in a concentric fashion, the second phase extends the depth of the explored area to increase the limits of coverage. The strategy is consistent in that none of the robots loose contact with the base station. Extensive simulations confirm the efficacy of the method and comparisons portray performance gain in terms of exploration time and absence of deadlocks vis-a-vis the few methods previously reported in the literature.

@inproceedings{bib_A_vi_2010, AUTHOR = {K.K., ARAVINDHAN and Krishna, K Madhava }, TITLE = {A visual exploration algorithm using semantic cues that constructs image based hybrid maps}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2010}}

A vision based exploration algorithm that invokes semantic cues for constructing a hybrid map of images - a combination of semantic and topological maps is presented in this paper. At the top level the map is a graph of semantic constructs. Each node in the graph is a semantic construct or label such as a room or a corridor, the edge represented by a transition region such as a doorway that links the two semantic constructs. Each semantic node embeds within it a topological graph that constitutes the map at the middle level. The topological graph is a set of nodes, each node representing an image of the higher semantic construct. At the low level the topological graph embeds metric values and relations, where each node embeds the pose of the robot from which the image was taken and any two nodes in the graph are related by a transformation consisting of a rotation and translation. The exploration algorithm explores a semantic construct completely before moving or branching onto a new construct. Within each semantic construct it uses a local feature based exploration algorithm that uses a combination of local and global decisions to decide the next best place to move. During the process of exploring a semantic construct it identifies transition regions that serve as gateways to move from that construct to another. The exploration is deemed complete when all transition regions are marked visited. Loop detection happens at transition regions and graph relaxation techniques are used to close loops when detected to obtain a consistent metric embedding of the robot poses. Semantic constructs are labeled using a visual bag of words (VBOW) representation with a probabilistic SVM classifier.

@inproceedings{bib_An_a_2010, AUTHOR = {JAKKOJU, CHETAN and Krishna, K Madhava and V, Jawahar C }, TITLE = {An adaptive outdoor terrain classification methodology using monocular camera}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2010}}

An adaptive partition based Random Forests classifier for outdoor terrain classification is presented in this paper. The classifier is a combination of two underlying classifiers. One of which is a random forest learnt over bootstrapped or offline dataset, the second is another random forest that adapts to changes on the fly. Posterior probabilities of both the static and changing/online classifiers are fused to assign the eventual label for the online image data. The online classifier learns at frequent intervals of time through a sparse and stable set of tracked patches, which makes it lightweight and real-time friendly. The learning which is actuated at frequent intervals during the sojourn significantly improves the performance of the classifier vis-a-vis a scheme that only uses the classifier learnt offline or at bootstrap. The method is well suited and finds immediate applications for outdoor autonomous driving where the classifier needs to be updated frequently based on what shows up recently on the terrain and without largely deviating from those learnt at bootstrapping. The role of the partition based classifier to enhance the performance of a regular multi class classifier such as random forests and multi class SVMs is also summarized in this paper.

@inproceedings{bib_A_no_2010, AUTHOR = {SINGH, ARUN KUMAR and NAMDEV, RAHUL KUMAR and PRAKASH, E VIJAY and Krishna, K Madhava }, TITLE = {A novel compliant rover for rough terrain mobility}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2010}}

In this paper a novel suspension mechanism for rough terrain mobility is proposed. The proposed mechanism is simpler than the existing suspension mechanism in the sense that the number of links and joints has been significantly reduced without compromising the climbing ability of the rover. We explore the use of compliant elements like springs for passively controlling the degree of freedom of the proposed mechanism and a framework for optimizing the spring parameters has been proposed. A performance evaluation of the proposed mechanism has been shown in terms of extensive simulations.

@inproceedings{bib_Two__2010, AUTHOR = {PRAKASH, E VIJAY and SINGH, ARUN KUMAR and Krishna, K Madhava }, TITLE = {Two models of force actuator based active suspension mechanisms for mobility on uneven terrain}, BOOKTITLE = {Acta Astronautica}. YEAR = {2010}}

In this paper we present two mechanisms of linear force actuator based actively articulated suspension vehicles and propose a strategy to control the wheel–ground contact forces to improve traction and to increase the no-slip margin and hence enhance the mobility of the vehicle on uneven terrain. We present the quasi-static analysis of each of the mechanisms to depict the ability of the systems to control the wheel–ground contact forces while negotiating uneven terrain with the help of feasibility plots. The first model is a vehicle with a 1-dof leg (referred to as LFA-V1) and can climb slopes upto 40 degrees but to further increase the capability of the robot we come with a modified design of the vehicle which has a 2-dof leg (referred to as LFA-V2) and can negotiate slopes with discontinuities greater than twice the wheel diameter.

@inproceedings{bib_Real_2010, AUTHOR = {KUNDU, ABHIJIT and V, Jawahar C and Krishna, K Madhava }, TITLE = {Realtime Moving Object Detection from a Freely Moving Monocular Camera}, BOOKTITLE = {International Conference on Robotics and Biomimetics}. YEAR = {2010}}

Detection of moving objects is a key component in mobile robotic perception and understanding of the environment. In this paper, we describe a realtime independent motion detection algorithm for this purpose. The method is robust and is capable of detecting difficult degenerate motions, where the moving objects is followed by a moving camera in the same direction. This robustness is attributed to the use of efficient geometric constraints and a probability framework which propagates the uncertainty in the system. The proposed independent motion detection framework integrates seamlessly with existing visual SLAM solutions. The system consists of multiple modules which are tightly coupled so that one module benefits from another. The integrated system can simultaneously detect multiple moving objects in realtime from a freely moving monocular camera.

@inproceedings{bib_A_hi_2010, AUTHOR = {KUMAR, B SUJITH and PAREKH, TEJAS PRAKASH and Krishna, K Madhava }, TITLE = {A hierarchical multi robotic collision avoidance scheme through robot formations}, BOOKTITLE = {International Conference on Robotics and Biomimetics}. YEAR = {2010}}

This paper depicts the utility of having robot formations to aid in collision avoidance amongst multiple robots in a multi agent/multi robotic setting. A set of robots satisfying certain proximal constraints reach a formation that enables considering that set of robots as a single robot cluster. Such a robot cluster is characterized by a cluster velocity computed from the individual robot velocities that comprise that cluster. The multi robotic collision avoidance problem can then be posed as collision avoidance between such robot clusters than between individual robots. Clusters could include single robot clusters. The robots within a cluster move such that they are guaranteed not to collide with each other. This is accomplished in this paper by ensuring that robots within a cluster reach a configuration where their velocities are equal and they move in a parallel formation. The underlying collision avoidance scheme thus needs to search for cluster velocities that provides collision free motion between clusters; this search at the cluster level than at an individual robot level greatly reduces the search space, thus providing for a hierarchical collision avoidance strategy. The strategy has been tested and verified in simulations and their results presented vindicating its efficacy.

@inproceedings{bib_Desi_2010, AUTHOR = {SRIRANJAN, R. and Krishna, K Madhava and Indurkhya, Bipin }, TITLE = {Design, construction and a compliant gait of “ModPod”: A modular hexpod robot}, BOOKTITLE = {International Conference on Robotics and Biomimetics}. YEAR = {2010}}

In this paper we describe the mechanical design, gait and control for a modular hexapod robot which makes it compliant to terrain while climbing slopes up and down. The robot uses a unique electronically actuated 2DOF universal spine similar to the spine of the Snake and the four legged animals like leopard, tiger etc. By controlling the amount of stiffness of the spine we were able to make the robot compliant to the surface. To our knowledge this is the first modular hexapod robot which achieves controllable compliance by both mechanical design and electronically actuating the robots spinal backbone. This design was biologically inspired based on the structure of a caterpillar with legs. Also we present the hardware and software architecture of the robot in this paper.

@inproceedings{bib_Data_2010, AUTHOR = {KOSURU, GURURAJ and SATISH, PEDDURI and Krishna, K Madhava }, TITLE = {Data association using empty convex polygonal regions in EKF-SLAM}, BOOKTITLE = {International Conference on Robotics and Biomimetics}. YEAR = {2010}}

This paper proposes a new framework for data association to solve the problem of SLAM. The proposed framework has specific relevance to range scanner based EKF-SLAM. The resulting data representation enables semantic reasoning on a spatial level which reduces the misassociation of closely spaced data from different spatial configurations through the use of convex polygons to represent data from similar spatial configurations. The data representation is especially effective for association when revisiting previously mapped regions efficiently. The spatial data representation also builds an occupancy grid for the entire map. We also provide a means of clustering range scan data using an adaptive threshold to be able to divide data at various ranges into clusters and dense data clustering to get more accurate data.

@inproceedings{bib_Snak_2010, AUTHOR = {SRIRANJAN, R. and Krishna, K Madhava }, TITLE = {Snake P3: A semi-autonomous Snake robot}, BOOKTITLE = {International Conference on Robotics and Biomimetics}. YEAR = {2010}}

Here in this paper we present a snake robot which can autonomously change its gait depending on the terrain. Also we present the design and construction details for the same. Here a custom simulator for the robot and an API for the same was developed which reduced the design and development time. For controlling the Snake robot manually a data glove using accelerometers was developed. The Snake robot can also be operated in autonomous mode where an over camera is used for sensing the terrain and the snake autonomously travels from one part of the terrain to another by switching its gait

@inproceedings{bib_Robo_2009, AUTHOR = {S, SUBHASH and Krishna, K Madhava }, TITLE = {Robot application development framework based on database }, BOOKTITLE = {International Conference on Robotics and Biomimetics}. YEAR = {2009}}

In this paper we describe the robot software development framework with database at its core. In this framework the features of database is harnessed to make various components modular, and robot data handling and processing is made much more flexible. This empowers the researchers to design the system quickly, access the data and allow multiple collaborators work to be integrated with ease. The advantage of database based framework is explained case by case for various scenarios with example robot application developments.

@inproceedings{bib_Secu_2009, AUTHOR = {ADDEPALLI, SARAT CHANDRA and BANSAL, PIYUSH and Kannan, Srinathan and Krishna, K Madhava }, TITLE = {Secured Multi-Robotic Active Localization without Exchange of Maps: A Case of Secure Cooperation Amongst Non-trusting Robots}, BOOKTITLE = {International Conference on Availability, Reliability and Security}. YEAR = {2009}}

Secure multiparty protocols have found applications in numerous domains, where multiple nontrusting parties wish to evaluate a function of their private inputs. In this paper, we consider the case of multiple robots wishing to localize themselves, with maps as their private inputs. Though localization of robots has been a well studied problem, only recent studies have shown how to actively localize multiple robots through coordination. In all such studies, localization has typically been achieved through constructing a publicly known global map. Here, we show how a similar solution can be given in the case of nontrusting robots, which do not wish to disclose their local maps.

@inproceedings{bib_A_mi_2009, AUTHOR = {VELURI, ADITYA TEJA and VISWANATH, D V KARTHIKEYA and Krishna, K Madhava }, TITLE = {A mixed autonomy coordination methodology for multi-robotic traffic control}, BOOKTITLE = {International Conference on Robotics and Biomimetics}. YEAR = {2009}}

We present a method for coordinating multi-robotic/multi-agent traffic control at intersections. The robotic agents (RA) move guided by a potential field along the lanes. At the intersections an intersection agent (IA) controls the flow of traffic by assigning priorities to the agents that are about to enter the intersection. The priorities are computed based on the density of RA in a lane and the flow rate of traffic in those lanes. The RAs integrate these assigned priorities into their potential field computations. The modified potential field computations help the RAs to move through the intersection avoiding collisions. An elegant mixed autonomy scheme is thereby achieved where the IAs decide upon the priorities at the intersection while the low level collision avoidance maneuvers are left with the individual RAs. This scheme preserves the distributed nature and the autonomy of potential field maneuvers while simultaneously balancing the computation load between the IA and RAs. We compare this method with a method where the RAs navigate the intersection without a superior direction from the IAs through priorities or when IAs direct the RAs based on priorities computed on a first come first served basis. We show performance gain over both these methods in simulations.

@inproceedings{bib_Glob_2009, AUTHOR = {KORRAPATI, HEMANTH and Krishna, K Madhava and VELURI, ADITYA TEJA }, TITLE = {Global localization of mobile robots by reverse projection of sensor readings}, BOOKTITLE = {International Conference on Robotics and Biomimetics}. YEAR = {2009}}

Global localization algorithms involve a search over all possible poses of the robot that can be typically over a large space in huge maps. Essentially it involves computing a posterior by seeing how probable are the obtained sensor readings at each of the discretized states in a map. Instead in this paper by reverse projecting the sensor readings from the obstacle boundaries onto the surroundings, a solution is obtained by searching over the space of obstacle boundaries than by a search in the discretized pose space. That this search over obstacle boundaries is considerably less if the ratio of free space to boundary space in a map is high is straightforward. However we also show theoretically that even when the boundary space exceeds the free space the computations due to the current method does not exceed those due to the popular Markov and Correlation based approaches to global localization. The comparative advantages are well documented in simulation section of the paper. The approach is able to consistently localize a laser equipped robot in our lab.

@inproceedings{bib_On_f_2009, AUTHOR = {SAWHNEY, RAHUL and Krishna, K Madhava and Kannan, Srinathan }, TITLE = {On fast exploration in 2D and 3D terrains with multiple robots}, BOOKTITLE = {International Conference on Autonomous Agents and Multiagent Systems}. YEAR = {2009}}

We present a fast multi-robotic exploration methodology for 2D and 3D terrains. An asynchronous exploration strategy is introduced which shows significant improvements over the existing synchronous ones. A per-time visibility metric is being utilized by the algorithm. The metric allots the same weight for points for next view whose visibility over time ratios are equal. The outcome of this is that while the number of points visited to explore a terrain is nearly the same as other popular metrics found in literature, the time length of the paths are smaller in this case resulting in reduced time exploration. The results have been verified through extensive simulations in 2D and 3D. In 2D multiple robots explore unknown terrains that are office like, cluttered, corridor like and various combinations of these. In 3D we consider the case of multiple UAVs exploring a terrain represented as height fields. We introduce a way for calculating expected visibilities and a way of incorporating explored features in the per-time metric. The maximum height of the UAV at each location is governed by the so called exposure surface, beneath which the UAVs are constrained to fly. We also show performance gain of the present metric over others in experiments on a Pioneer 3DX robot.

@inproceedings{bib_Esti_2009, AUTHOR = {E, YASOVARDHAN REDDY and KORRAPATI, HEMANTH and Krishna, K Madhava }, TITLE = {Estimating ground and other planes from a single tilted laser range finder for on-road driving}, BOOKTITLE = {International Conference on Advanced Robotics}. YEAR = {2009}}

We present a method for extracting ground and other planes from a single non rotating laser mounted on a slow moving car used for on-road driving. A laser scan is decomposed into linear clusters. Corresponding clusters from subsequent scans are merged to form planes. The ground plane is identified based on the current vehicle height and the variance in height of the planes. Once these seed planes are identified future scan points either get associated with these planes or result in formation of new planes. Scan points that do not belong to any of the plane are left as such in the representation. Since the robustness of the method is contingent on how a single scan is decomposed into linear clusters, we compare the quality of the terrain representation due to three such clustering methods, one by iterative end point fit, other by adaptive breakpoint detection and thirdly the current method based on adaptive cosine similarity.

@inproceedings{bib_On_m_2009, AUTHOR = {SATISH, PEDDURI and KOSURU, GURURAJ and Krishna, K Madhava }, TITLE = {On measurement models for line segments and point based SLAM}, BOOKTITLE = {International Conference on Advanced Robotics}. YEAR = {2009}}

We present efficient measurement models for localization in a feature based EKF SLAM framework. Both points and segments form the features, points include corners formed by intersection of wall like segments. The point features are described by its coordinates, while the segment feature is represented by the angle made by the normal to the segment from the global origin with the abscissa called the normal angle or N-angle for short. The measurement equation involves measuring the distance and bearing to the point feature and only bearing to the line feature. The distance measurement to the segment is intentionally kept out of the measurement equation due to its inefficacy in correcting the robot and landmarks state. This arises due to very large differences in the predicted and observed distances even for modest measurement errors when the robot is not very near the segment. Hence for this reason the segment feature is represented only by its N-angle devoid of distance since such a representation results in better state correction. The number of computations resulting from the covariance matrix updates is also less than a representation that includes both N-distance and N-angle.

@inproceedings{bib_MDP__2009, AUTHOR = {BAHUGUNA, JYOTIKA and Ravindran, B. and Krishna, K Madhava }, TITLE = {MDP based active localization for multiple robots}, BOOKTITLE = {International Conference on Automation Science and Engineering}. YEAR = {2009}}

In environments with identical features, the global localization of a robot, might result in multiple hypotheses of its location. If the situation is extrapolated to multiple robots, it results in multiple hypotheses for multiple robots. The localization is facilitated if the robots are actively guided towards locations where it can use other robots as well as obstacles to localize itself. This paper aims at presenting a learning technique for the above process of active localization of multiple robots by co-operation. An MDP framework is used for learning the task, over a semi-decentralized team of robots hereby maintaining a bounded complexity as opposed to various multi-agent learning techniques, which scale exponentially with the increase in the number of robots.

@inproceedings{bib_Movi_2009, AUTHOR = {KUNDU, ABHIJIT and Krishna, K Madhava and Sivaswamy, Jayanthi }, TITLE = {Moving object detection by multi-view geometric techniques from a single camera mounted robot}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2009}}

The ability to detect, and track multiple moving objects like person and other robots, is an important prerequisite for mobile robots working in dynamic indoor environments. We approach this problem by detecting independently moving objects in image sequence from a monocular camera mounted on a robot. We use multi-view geometric constraints to classify a pixel as moving or static. The first constraint, we use, is the epipolar constraint which requires images of static points to lie on the corresponding epipolar lines in subsequent images. In the second constraint, we use the knowledge of the robot motion to estimate a bound in the position of image pixel along the epipolar line. This is capable of detecting moving objects followed by a moving camera in the same direction, a so-called degenerate configuration where the epipolar constraint fails. To classify the moving pixels robustly, a Bayesian framework is used to assign a probability that the pixel is stationary or dynamic based on the above geometric properties and the probabilities are updated when the pixels are tracked in subsequent images. The same framework also accounts for the error in estimation of camera motion. Successful and repeatable detection and pursuit of people and other moving objects in realtime with a monocular camera mounted on the Pioneer 3DX, in a cluttered environment confirms the efficacy of the method.

@inproceedings{bib_Evol_2009, AUTHOR = {SINGH, ARUN KUMAR and PRAKASH, E VIJAY and Krishna, K Madhava and Arun.H.Pati, }, TITLE = {Evolution of a four wheeled active suspension rover with minimal actuation for rough terrain mobility}, BOOKTITLE = {International Conference on Robotics and Biomimetics}. YEAR = {2009}}

In this paper we deduce the evolution of a four wheeled active suspension rover from a five wheeled passive suspension rover. The aim of this paper is to design a suspension mechanism which utilizes the advantages of both passive suspension and active suspension rover. Both the design considered here are simpler than the existing suspension mechanisms in the sense that the number of links as wells as the number of joints have been significantly reduced without compromising the climbing capability of the rover. We first analyze the kinematics of the five wheeled rover and its motion pattern while climbing an obstacle and try to deduce the same motion pattern and capability in the four wheeled rover. Both the suspension mechanism consists of two planar closed kinematic chains on each side of the rover. We also deduce the control strategy for the active suspension rover wherein only two actuators are used to control the internal configuration of the rover. To the best of author's knowledge this is the minimum number of actuators required to control the internal configuration of a active suspension while operating on a fully 3D rough terrain. Extensive uneven terrain simulations are performed for both 5-wheeled and 4-wheeled rover and a comparative analysis has been done on maximum coefficient of friction and torque requirements.

@inproceedings{bib_Pers_2008, AUTHOR = {HANDA, ANKUR and Sivaswamy, Jayanthi and Krishna, K Madhava and SINGH, SARTAJ and Menezes, Paulo }, TITLE = {Person following with a mobile robot using a modified optical flow}, BOOKTITLE = {Conference on Climbing and Walking Robots}. YEAR = {2008}}

This paper deals with the tracking and following of a person with a camera mounted mobile robot. A modified energy based optical flow approach is used for motion segmentation from a pair of images. Further a spatial relative veolcity based filering is used to extract prominently moving objects. Depth and color information are also used to robustly identify and follow a person.

@inproceedings{bib_Loca_2008, AUTHOR = {KORRAPATI, HEMANTH and CHANDRA, S R SUBHASH and Krishna, K Madhava and Pandey, Amit K }, TITLE = {Localizing from Multi-Hypotheses States Minimizing Expected Path Lengths for Mobile Robots}, BOOKTITLE = {Conference on Climbing and Walking Robots}. YEAR = {2008}}

This paper explains how to singularize a robot to a unique hypothesis state from a state of multiple hypotheses. This it does by computing a path whose expected distance is minimum from a tree where each path to the leaf from the root results in a singular hypothesis. At various nodes of the tree the number of hypotheses is reduced as it proceeds down from the root. The nodes of the tree are computed as the best locations to move from an earlier higher hypotheses state. They are either frontier regions or a direction of traversal that result in reduction of hypotheses from an earlier multi hypotheses state. The method has been tested robustly in various simulation situations and its efficacy confirmed.

@inproceedings{bib_On_r_2008, AUTHOR = {SAWHNEY, RAHUL and Krishna, K Madhava and Kannan, Srinathan and MOHAN, MAHESH }, TITLE = {On reduced time fault tolerant paths for multiple UAVs covering a hostile terrain}, BOOKTITLE = {International Conference on Autonomous Agents and Multiagent Systems}. YEAR = {2008}}

We present a method for finding reduced time coverage paths of multiple UAVs (Unmanned Air Vehicles) monitoring a 3D terrain represented as height fields. A novel metric based on per time visibility is used that couples visibility gained at a terrain point with the time spent to reach the point. This coupled metric is utilized to form reduced time paths by maximizing the visibility gained per unit time at every step. We compare the results of this approach with an approach that covers the terrain based on a per distance visibility metric, which reduces the sum, over distances covered by each UAV path. The comparisons show that the current method gives substantially time reduced paths albeit with an expected increase in sum over distances of UAV paths. We also show that time taken to cover the terrain based on the current metric is far less than prevalent methods that try to decompose the terrain based on visibility followed by time or time followed by visibility in a decoupled fashion. The method is further extended to provide for fault tolerance on a hostile terrain. Each terrain point is guaranteed to be seen by at-least one UAV that has not been damaged due to any calamity, shot or otherwise.

@inproceedings{bib_Opti_2008, AUTHOR = {PRAKASH, E VIJAY and Krishna, K Madhava and SANAN, SIDDHARTH }, TITLE = {Optimal Posture Control for Force Actuator Based Articulated Suspension Vehicle for Rough Terrain Mobility}, BOOKTITLE = {Conference on Climbing and Walking Robots}. YEAR = {2008}}

To improve the mobility of wheeled robots traversing on and uneven terrain having slopes in all three orthogonal directions is the primary focus of our research. Past research on ‘all terrain vehicles’,[1],[2], was focused on developing mechanical suspension systems which could improve terrain adaptability and locomotion. Consequently control algorithms were developed for posture stability of the vehicle [3],[4]. Shrimp robots [1] and Rocky rovers [2] are terrain vehicles with passive suspension systems which have excellent terrain adaptability and ability to negotiate terrains having discontinuities that are higher than the wheel radius. But mobility and stability of such vehicles is not guaranteed. Thus for such conditions Sreenivasan and Waldron [4] developed vehicles called Wheeled and Actively Articulated Vehicles

@inproceedings{bib_On-l_2008, AUTHOR = {Hafez, A.H. Abdul and Bhuvanagiri, Shivudu and Krishna, K Madhava and V, Jawahar C }, TITLE = {On-line convex optimization based solution for mapping in VSLAM}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2008}}

This paper presents a novel real-time algorithm to sequentially solve the triangulation problem. The problem addressed is estimation of 3D point coordinates given its images and the matrices of the respective cameras used in the imaging process. The algorithm has direct application to real time systems like visual SLAM. This article demonstrates the application of the proposed algorithm to the mapping problem in visual SLAM. Experiments have been carried out for the general triangulation problem as well as the application to visual SLAM. Results show that the application of the proposed method to mapping in visual SLAM outperforms the state of the art mapping methods.

@inproceedings{bib_Acti_2008, AUTHOR = {BHUVANAGIRI, SHIVUDU and Krishna, K Madhava }, TITLE = {Active global localization for multiple robots by disambiguating multiple hypotheses}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2008}}

In environments which possess relatively few features that enable a robot to unambiguously determine its location, global localization algorithms can result in multiple hypotheses locations of a robot. In such a scenario the robot, for effective localization, has to be actively guided to those locations where there is a maximum chance of eliminating most of the ambiguous states - which is often referred to as dasiaactive localizationpsila. When extended to multi robotic scenarios where all robots possess more than one hypothesis of their position, there is the opportunity to do better by using robots apart from obstacles as dasiahypotheses resolving agentspsila. The paper presents a unified framework accounting for the map structure as well as measurement amongst robots while guiding a set of robots to locations where they can singularize to a unique state. The appropriateness of our approach is demonstrated empirically in both simulation & real-time (on Amigobots) and its efficacy verified. Extensive comparative analysis portrays the advantage of the current method over others that do not perform active localization in a multi-robotic sense.

@inproceedings{bib_Cove_2008, AUTHOR = {MOHAN, MAHESH and SAWHNEY, RAHUL and Krishna, K Madhava and Kannan, Srinathan and M.B, Srikanth }, TITLE = {Covering hostile terrains with partial and complete visibilities: On minimum distance paths}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2008}}

We present a method for finding paths for multiple unmanned air vehicles (UAVs) such that the sum over their lengths is minimum as they cover a 3D terrain (represented as height fields). The paths are constrained to lie beneath an exposure surface to ensure stealth from enemy outposts. The exposure surface is also computed as a height field. The algorithm greedily clusters the terrain such that gain in visibility per distance would be higher for intra-cluster points than points across clusters. Paths generated on clusters formed by such a per distance visibility metric are reduced by more than 25% over other related decoupled methods. The method is extended to cover terrains with partial visibilities. The advantage of the coupled metric extends under constrained visibility also. We again show performance gain by comparing with an existing decoupled algorithm that solves a similar problem of minimum distance terrain coverage with constrained visibility. The paper reveals that decomposing the terrain based on visibility first and then distance is always better than the other way round to cover the terrain in shorter distances.

@inproceedings{bib_Link_2007, AUTHOR = {PANDEY, AMIT KUMAR and Krishna, K Madhava }, TITLE = {Link Graph and Feature Chain Based Robust Online SLAM for Fully Autonomous Mobile Robot Navigation System Using Sonar Sensors}, BOOKTITLE = {International Conference on Advanced Robotics}. YEAR = {2007}}

Local localization of a fully autonomous mobile robot in a partial map is an important aspect from the view point of accurate map building and safe path planning. The problem of correcting the location of a robot in a partial map worsens when sonar sensors are used. When a mobile robot is exploring the environment autonomously, it is rare to get the consistent pair of features or readings from two different positions using sonar sensors. So the approaches, which rely on readings or features matching, are prone to fail without exhaustive mathematical calculations of sonar modeling and environment modeling. This paper introduces link graph based robust two step feature chain based localization for achieving online SLAM (Simultaneous Localization And Mapping) using sonar data only. Instead of relying completely on matching of feature to feature or point to point, our approach finds possible associations between features to localize. The link graph based approach removes many false associations enhancing the SLAM process. We also map features onto Occupancy Grid (OG) framework taking advantage of its dense representation of the world. Combining features onto OG overcomes many of its limitations such as the independence assumption between cells and provides for better modeling of the sonar providing more accurate maps.

@inproceedings{bib_Cont_2007, AUTHOR = {, SIDDHARTH SANAN and SINGH, SARTAJ and Krishna, K Madhava }, TITLE = {Controlling an Actively Articulated Suspension Vehicle for Mobility in Rough Terrain}, BOOKTITLE = {Conference on Climbing and Walking Robots}. YEAR = {2007}}

This paper studies the problem of traversing a rough terrain by wheeled vehicles. The criterion for mobility of a wheeled vehicle in any terrain is formally developed, providing insights into the mechanical structure requirements of the vehicle. A vehicle structure with an actively articulated suspension is found as a solution to improved rough terrain mobility. The contact forces of the vehicle with the surface being traversed are identified as the critical factor in determining the traversability of the surface. Hence a control strategy involving the control of the contact forces (normal and traction) is proposed. The key feature of the locomotion strategy, thus, developed is that it provides a solution for mobility in terrain which cannot be traversed using a solution involving assumptions that ignore the dynamics of the main body of the vehicle.

@inproceedings{bib_Feat_2007, AUTHOR = {PANDEY, AMIT KUMAR and Krishna, K Madhava and NATH, MAINAK }, TITLE = {Feature Based Occupancy Grid Maps for Sonar Based Safe-Mapping.}, BOOKTITLE = {International Joint Conference on Artificial Intelligence}. YEAR = {2007}}

This paper presents a methodology for integrating features within the occupancy grid (OG) framework. The OG maps provide a dense representation of the environment. In particular they give information for every range measurement projected onto a grid. However independence assumptions between cells during updates as well as not considering sonar models lead to inconsistent maps, which may also lead the robot to take some decisions which may be unsafe or which may introduce an unnecessary overhead of run-time collision avoidance behaviors. Feature based maps provide more consistent representation by implicitly considering correlation between cells. But they are sparse due to sparseness of features in a typical environment. This paper provides a method for integrating feature based representations within the standard Bayesian framework of OG and provides a dense, more accurate and safe representation than standard OG methods.

@inproceedings{bib_Opti_2007, AUTHOR = {KUMAR, PURUSHOTHAM GANESH and Krishna, K Madhava }, TITLE = {Optimal Multi-Sensor Based Multi Target Detection by Moving Sensors to the Maximal Clique in a Covering Graph.}, BOOKTITLE = {International Joint Conference on Artificial Intelligence}. YEAR = {2007}}

Different methodologies have been employed to solve the multi-sensor multi-target detection problem in a variety of scenarios. In this paper, we devise a time-step optimal algorithm for this problem when all but a few parameters of the sensor/target system are unknown. Using the concept of covering graph, we find an optimum solution for a single sensor, which is extended to multiple sensors by a tagging operation. Both covering graph and tagging are novel concepts, developed in the context of the detection problem for the first time, and bring a mathematical elegance to its solution. Furthermore, an implementation of the resulting algorithm is found to perform better than other notable approaches. The strong theoretical foundation, combined with the practical efficacy of the algorithm, makes it a very attractive solution to the problem

@inproceedings{bib_Mult_2007, AUTHOR = {SATISH, PEDDURI and Krishna, K Madhava }, TITLE = {Multi robotic conflict resolution by cooperative velocity and direction control}, BOOKTITLE = {Mobile Robots: Perception & Navigation}. YEAR = {2007}}

Collision avoidance is one of the essential pillars of a wheeled robotic system. A wheeled mobile robot (called mobile robot for conciseness henceforth) requires for effective functioning an integrated system of modules for (i) map building,(ii) localization,(iii) exploration,(iv) planning and (v) collision avoidance. Often (i) and (ii) are entailed to be done simultaneously by robots resulting in a vast array of literature under the category SLAM, simultaneous localization and mapping. In this chapter we focus on the aspect of collision avoidance specifically between multiple robots, the remaining themes being too vast to even get a brief mention here. We present a cooperative conflict resolution strategy between multiple robots through a purely velocity control mechanism (where robots do not change their directions) or by a direction control method. The conflict here is in the sense of multiple robots competing for the same space over an overlapping time window. Conflicts occur as robots navigate from one location to another while performing a certain task. Both the control mechanisms attack the conflict resolution problem at three levels, namely (i) individual,(ii) mutual and (iii) tertiary levels. At the individual level a single robot strives to avoid its current conflict without anticipating the conflicting robot to cooperate. At the mutual level a pair of robots experiencing a conflict mutually cooperates to resolve it. We also denote this as mutually cooperative phase or simply cooperative phase succinctly. At tertiary level a set of robots cooperate to avoid one or more conflicts amidst them

@inproceedings{bib_Feat_2007, AUTHOR = {PANDEY, AMIT KUMAR and Krishna, K Madhava and Hexmoor, Henry }, TITLE = {Feature chain based occupancy grid SLAM for robots equipped with sonar sensors}, BOOKTITLE = {Conference on Integration of Knowledge Intensive Multi-Agent Systems}. YEAR = {2007}}

This paper presents a methodology for achieving SLAM onto the occupancy grid framework with the data only from sonar sensors. Sonar data are highly noisy and unpredictable. Sonar does not give the consistent readings for a point from two different positions, so the approaches which rely on correspondence reading matching will prone to fail without exhaustive mathematical calculations of sonar modeling and environment modeling. Also, if features are being use to localize then the robot needs to revisit those features exactly, to localize, which itself will not be accurate because robot will not be at the exact position from where that feature has been detected. Hence it will not get back those feature readings using sonar. Here we are presenting a hybrid approach based on feature chain. Instead of relying completely on feature mapping and point matching, it finds the links between features to localize. It will drastically reduce the need of revisiting a feature to localize and hence reducing the exploration overhead, while handling other issues of problems with point or feature matching. We map features onto occupancy grid (OG) framework taking advantage of its dense representation of the world. Combining features onto OG overcomes many of its limitations such as the independence assumption between cells and provides for better modeling of the sonar implicitly providing more accurate maps

@inproceedings{bib_Coop_2007, AUTHOR = {SATISH, PEDDURI and Krishna, K Madhava and Hexmoor, Henry }, TITLE = {Cooperative navigation function based navigation of multiple mobile robots}, BOOKTITLE = {Conference on Integration of Knowledge Intensive Multi-Agent Systems}. YEAR = {2007}}

A cooperative methodology for collision avoidance of multiple wheeled robots, having respective goals to reach, is detailed in this paper. The paths executed by the robots are continuous not only in terms of positions reached by the robots but also in terms of their velocities. A navigation function is designed that, apart from taking into account goal reaching and avoidance behaviors, also implicitly captures cooperative behavior amongst robots by identifying spaces where collisions between robots tend to be minimized. A search in the joint space of linear and angular velocities of the robots results in selection of a linear and angular velocity tuple for each robot that minimizes the navigation function. Simulated results portray the efficacy of the methodology

@inproceedings{bib_Robo_2007, AUTHOR = {VISWANATH, D V KARTHIKEYA and Krishna, K Madhava and Hexmoor, Henry and Singh, Sartaj }, TITLE = {Robot navigation in a 3D world mediated by sensor networks}, BOOKTITLE = {Conference on Integration of Knowledge Intensive Multi-Agent Systems}. YEAR = {2007}}

A methodology for real-time navigation of an all terrain vehicle capable of navigation between floors of a building mediated by a wireless sensor network is detailed in this paper. The map of the building is unknown to the robot. Sensor motes scattered in the building guide the robot to its desired destination. The guidance is in the form of specifying the next waypoint to the goal by the network. Local navigation between waypoints is achieved by a fuzzy logic based navigation system

@inproceedings{bib_Prob_2007, AUTHOR = {RAHUL, T R and Krishna, K Madhava and Hexmoor, Henry }, TITLE = {Probability Based Optimal Algorithms for Multi-sensor Multi-target Detection}, BOOKTITLE = {Conference on Integration of Knowledge Intensive Multi-Agent Systems}. YEAR = {2007}}

The algorithm presented in this paper is designed to be used in automated multi-sensor surveillance systems which require observation of targets in a bounded area to optimize the performance of the system. There have been many approaches which deal with multi-sensor tracking and observation, but there haven't been many which deal purely with targets detections i.e. each target needs to only be detected once. The metric used to gauge the performance of the system is percentage of targets detected among those that enter the area. Targets enter the area through source points on the side of the area according to Poisson distribution, the rate of entry is constant for all sources. The algorithm presented here uses target arrival information, sensor positions to generate an optimal motion strategy for the multi-sensor system every T time-steps i.e. every T time-steps, the probability of finding undetected targets is estimated, the optimal sensor paths for the next T time-steps are calculated. The algorithm performs robustly and optimally detecting around 80% of the targets that enter the area

@inproceedings{bib_On_i_2007, AUTHOR = {Sanan, Siddharth and TATI, NAGESHWAR RAO and Krishna, K Madhava and Singh, Sartaj }, TITLE = {On improving the Mobility of Vehicles on uneven terrain}, BOOKTITLE = {International Conference on Advanced Robotics}. YEAR = {2007}}

This paper studies the problem of traversing a rough terrain by wheeled vehicles. Here rough terrain implies terrain which is geometrically not ideal. The criterion for mobility of a wheeled vehicle in any terrain is formally developed, providing insights into the mechanical structure requirements of the vehicle. A vehicle structure with an actively articulated suspension is found as a solution to improved rough terrain mobility. The contact forces of the vehicle with the surface being traversed are identified as the critical factor in determining the traversability of the surface. Hence a control strategy involving the control of the contact forces (normal and traction) is proposed. The key feature of the locomotion strategy, thus developed, is that it provides a solution involving dynamics of the main body for improving mobility in rough terrain.

@inproceedings{bib_Coll_2007, AUTHOR = {SATISH, PEDDURI and Krishna, K Madhava }, TITLE = {Collision Avoidance for Multiple Robots till Next Waypoints through Collision Free Polygons}, BOOKTITLE = {International Conference on Advanced Robotics}. YEAR = {2007}}

Kinematically consistent paths for multiple mobile robots that are collision free for the next T steps are generated by forming collision free polygons (CFP). The polygons can be generated in a distributive fashion for each robot. All paths inside CFP that are kinematically feasible for a certain robot are collision free with all other robots in the same collision cluster for the next T time samples and with any other robot in the workspace in general. The construction of CFP is through two log-linear complex operations. The first one involves computing the intersection of the path container polygon (PCP) of a robot with PCP of other robots in the cluster. The second involves computing the convex hull of intersecting points arising from the intersection of PCPs. Once CFP is computed for a robot it chooses a point within the CFP (its next waypoint) that minimizes a cost function and moves towards the point. The process is repeated till the goal is reached. The main advantage of this method is that the best and worst-case times for finding a T-step ahead collision free path is always log-linear. In many randomized search methods, finding the paths till the next waypoint results in a worst case complexity exponential in number of robots. The efficacy of the current method is well portrayed through simulations. Comparative results show that the following method finds a collision free path to next way point much quicker.

@inproceedings{bib_Safe_2006, AUTHOR = {Krishna, K Madhava and Alami, Rachid and Simeon, Thierry }, TITLE = {Safe proactive plans and their execution}, BOOKTITLE = {Robotics and Autonomous systems}. YEAR = {2006}}

We present in this paper a methodology for computing the maximum velocity profile over a trajectory planned for a mobile robot. Environment and robot dynamics as well as the constraints of the robot sensors determine the profile. The planned profile is indicative of maximum speeds that can be possessed by the robot along its path without colliding with any of the mobile objects that could intercept its future trajectory. The mobile objects could be arbitrary in number and the only information available regarding them is their maximum possible velocity. The velocity profile also enables one to deform planned trajectories for better trajectory time. The methodology has been adopted for holonomic and nonholonomic motion planners. An extension of the approach to an online real-time scheme that modifies and adapts the path as well as velocities to changes in the environment such that both safety and execution time are not compromised is also presented for the holonomic case. Simulation and experimental results demonstrate the efficacy of this methodology.

@inproceedings{bib_Exte_2006, AUTHOR = {Sanan, Siddharth and Santani, Darshan and Krishna, K Madhava and Hexmoor, Henry }, TITLE = {Extension of Reeds and Shepp Paths to a Robot with Front and Rear Wheel Steer.}, BOOKTITLE = {International Conference on Robotics and Automation}. YEAR = {2006}}

RS paths for a robot with both front and rear wheel steer. We call such robots as FR steer. The occurrence of such paths is due to the additional maneuver possible in such a robot which we call parallel steer, in addition to the ones already present in a vehicle with only front wheel steering. Hence we extend the optimal path set, containing only a single element to a set, containing n elements, thereby extending its configuration set along the optimal path from the initial to the final configuration. This extension of the set to set is made possible by introducing a special set, which we call the Parallel Steer (PS) Set. Such an extension of the configuration set would increase the size of the final configuration set achievable by a path that is optimal in free space. In the following discussion, we shall term all paths whose length is equal to an RS path as optimal.

@inproceedings{bib_Mult_2006, AUTHOR = {K, GANESH P and Krishna, K Madhava and , Paulo Menezes }, TITLE = {Multi-target Detection by Multi-sensor Systems: A Comparison of Systems}, BOOKTITLE = {International Conference on Robotics and Biomimetics}. YEAR = {2006}}

Different methodologies exist to direct the motion of sensors to detect targets moving across an environment in various scenarios. However, some of these do not model that navigation scenario in an environment in which obstacles are present. We extend our earlier algorithm for optimal target detection, as well as other algorithms reported in literature, to this case, and make a detailed comparison of their performance. This makes clear that the current algorithm is competitive in applications where target statistics are known in advance; otherwise, a heuristic technique by Sukhatme and Jung performs best.

@inproceedings{bib_Sens_2006, AUTHOR = {GOYAL, RAKESH and Krishna, K Madhava and BHUVANAGIRI, SHIVUDU }, TITLE = {Sensor Based Localization for Mobile Robots by Exploration and Selection of Best Direction}, BOOKTITLE = {International Conference on Robotics and Biomimetics}. YEAR = {2006}}

We present a strategy for resolving multiple hypotheses of a robot's state during global localization. The strategy operates in two stages. In the first stage a unique direction of the motion is sought that resolves or eliminates maximum number of hypotheses. In the second stage, among the frontier areas arising from the multiple hypotheses states, that frontier is chosen which resolves the maximum number of the hypotheses. The two stages are alternated till a unique hypothesis emerges. Simulation and experimental results verify the efficacy of this method. A comparison with other methods based on entropy minimization, and minimum distance travel portrays the advantage of the current methodology. A convergence proof for the algorithm is also presented.

@inproceedings{bib_Fuzz_2005, AUTHOR = {Krishna, K Madhava and Kalra, Prem Kumar }, TITLE = {Fuzzy Clustering with M-Estimators.}, BOOKTITLE = {Indian International Conference on Artificial Intelligence}. YEAR = {2005}}

We present an extension of the FCM over the loss functions used in the M-estimators of robust statistics akin to the generalization of the fuzzy-C-means algorithms over the norm distances [1]. The effect of these estimators in reducing the bias of the outliers while estimating the cluster prototypes are studied and compared. The comparisons have been done over synthetic data as well as simulated data consisting of range sensor readings representative of the objects in the neighborhood of a navigating mobile robot. For the synthetic data set the comparison is attempted over the popular FCM algorithm. For the sensory data set the Adaptive Fuzzy Clustering (AFC) algorithm [2] has been employed and extended over the loss functions of robust statistics. The AFC is utilized considering the shape of the objects encountered by the robot in a typical indoor environment.

@inproceedings{bib_A_fr_2005, AUTHOR = {Krishna, K Madhava and Hexmoor, Henry }, TITLE = {A framework for guaranteeing detection performance of a sensor network}, BOOKTITLE = {Integrated Computer-Aided Engineering}. YEAR = {2005}}

We present a framework for modeling and analysis for a surveillance network consisting of multiple sensors. Sensors monitor targets that crisscross a rectangular surveillance zone. When a sensor pursuits a target it leaves areas unguarded through which other targets can get past undetected. A methodology that computes the tracking time for a sensor such that a fraction of the targets expected to cross its home area is detected to an arbitrary probabilistic guarantee is presented based on the framework. Targets enter the surveillance zone according to Poisson statistics. The time spent by a target within a sensor's home area follows uniform random statistics. The home area of the sensor is the area guarded by it when it is stationed at its home position, its default position when it is not in pursuit of a target. The framework is further extended to situations where multiple sensors monitor the same home area. Simulation results presented corroborate with the probabilistic framework developed and verify its correctness for single as well as multi-sensor cases.

@inproceedings{bib_AT_S_2005, AUTHOR = {Krishna, K Madhava and , Henry Hexmoor and , Shravan Sogani }, TITLE = {AT Step Ahead Optimal Target Detection Algorithm for a Multi Sensor Surveillance System}, BOOKTITLE = {Technical Report}. YEAR = {2005}}

This paper presents a methodology for optimal target detection in a multi sensor surveillance system. The system consists of mobile sensors that guard a rectangular surveillance zone crisscrossed by moving targets. Targets percolate the surveillance zone in a poisson fashion with uniform velocities. Under these statistics this paper computes a motion strategy for a sensor that maximizes target detections for the next T time steps. A coordination mechanism between sensors ensures that overlapping areas between sensors is reduced. This coordination mechanism is interleaved with the motion strategy computation to reduce detections of the same target by more than one sensor. To avoid an exhaustive search in the joint space of all sensors the coordination mechanism constraints the search by assigning priorities to the sensors. A comparison of this methodology with other multi target tracking schemes verifies its efficacy in maximizing detections. A tabulation of these comparisons is reported in results section of the paper

@inproceedings{bib_Reso_2005, AUTHOR = {Krishna, K Madhava and Hexmoor, Henry }, TITLE = {Resource allocation strategies for a multi sensor surveillance}, BOOKTITLE = {International Symposium on Collaborative Technologies and Systems}. YEAR = {2005}}

In this paper we expand on our previous efforts to evaluate strategies for improving tracking performance of a multi-sensor surveillance system. The quality of tracking performance is based on two measures: (i) the mean track quality (MTQ) and (ii) total expected number of misses (TEM). Five strategies of resource allocation were considered based on (a) local versus coordinated, (b) dedicated versus distracted and (c) benevolent. The simulation results suggest that a coordinated and distracted strategy of resource allocation yields the best result throughout while such a strategy coupled with benevolence further improves tracking performance marginally in most cases

@inproceedings{bib_Para_2005, AUTHOR = {Krishna, K Madhava and Hexmoor, Henry and Pasupuleti, Subbarao and Llinas, James }, TITLE = {Parametric control of multiple unmanned air vehicles over an unknown hostile territory}, BOOKTITLE = {Conference on Integration of Knowledge Intensive Multi-Agent Systems}. YEAR = {2005}}

A methodology for real-time control of unmanned air vehicles (UAV) in the absence of a-priori knowledge of hostile territory is presented. The control methodology generates a sequence of waypoints to be pursued by the UV until the goal is reached. The controller computes the waypoints every time new information is obtained regarding the presence or absence of a hostile agent at a particular grid of the territory. The waypoints are the result of an A* search. The cost function of the search is a weighted combination of dangers arising due to probability of being hit by the attacks of hostile terrestrial agents on the ground and the danger of UAV grounded as a result of its prolonged stay over the hostile territory. The multi-agency in the system is due to the broadcast of newly observed information by an UAV to its remaining counterparts. The sequence of waypoints defines the path of the UAV to its goal. By varying the corresponding weights the paths can be altered to obtain a particular performance criterion. Simulation results are presented for various parametric combinations to validate the methodology.

@inproceedings{bib_Reac_2005, AUTHOR = {Krishna, K Madhava and Hexmoor, Henry and Chellappa, Srinivas }, TITLE = {Reactive navigation of multiple moving agents by collaborative resolution of conflicts}, BOOKTITLE = {Journal of Intelligent and Robotic Systems}. YEAR = {2005}}

In navigation that involves several moving agents or robots that are not in possession of each other's plans, a scheme for resolution of collision conflicts between them becomes mandatory. A resolution scheme is proposed in this paper specifically for the case where it is not feasible to have a priori the plans and locations of all other robots, robots can broadcast information between one another only within a specified communication distance, and a robot is restricted in its ability to react to collision conflicts that occur outside of a specified time interval called the reaction time interval. Collision conflicts are resolved through velocity control by a search operation in the robot's velocity space. The existence of a cooperative phase in conflict resolution is indicated by a failure of the search operation to find velocities in the individual velocity space of the respective robots involved in the conflict. A scheme for cooperative resolution of conflicts is modeled as a search in the joint velocity space of the robots involved in conflict when the search in the individual space yields a failure. The scheme for cooperative resolution may further involve modifying the states of robots not involved in any conflict. This phenomenon is characterized as the propagation phase where cooperation spreads to robots not directly involved in the conflict. Apart from presenting the methodology for the resolution of conflicts at various levels (individual, cooperative, and propagation), the paper also formally establishes the existence of the cooperative phase during real‐time navigation of multiple mobile robots. The effect of varying robot parameters on the cooperative phase is presented and the increase in requirement for cooperation with the scaling up of the number of robots in a system is also illustrated. Simulation results involving several mobile robots are presented to indicate the efficacy of the proposed strategy. © 2005 Wiley Periodicals, Inc.

@inproceedings{bib_A_t-_2005, AUTHOR = {Krishna, K Madhava and Hexmoor, Henry and Sogan, Shravan }, TITLE = {A t-step ahead constrained optimal target detection algorithm for a multi sensor surveillance system}, BOOKTITLE = {International Conference on Intelligent Robots and Systems}. YEAR = {2005}}

We present a methodology for optimal target detection in a multi sensor surveillance system. The system consists of mobile sensors that guard a rectangular surveillance zone crisscrossed by moving targets. Targets penetrate the surveillance zone with poisson rates at uniform velocities. Under these conditions we present a motion strategy computation for each sensor such that it maximizes target detection for the next T time-steps. A coordination mechanism among sensors ensures that overlapping and overlooked regions of observation among sensors are minimized. This coordination mechanism is interleaved with the motion strategy computation to reduce detections of the same target by more than one sensor for the same time-step. To avoid an exhaustive search in the joint space of all the sensors the coordination mechanism constrains the search by assigning priorities to the sensors and thereby arbitrating among sensory tasks. A comparison of this methodology with other multi target tracking schemes verifies its efficacy in maximizing detections. "Sample" and "time-step" are used equivalently and interchangeably in this paper.