• 
  In Vivo Skin 3-D Surface Reconstruction and Wrinkle Depth Estimation Using Handheld High Resolution Tactile Sensing

  Advanced Healthcare Materials, 2025

  Akhil Padmanabha, Arpit Agarwal, Catherine Li, Austin Williams, Dinesh K. Patel, Sankalp Chopkar, Achu Wilson, Ahmet Ozkan, Wenzhen Yuan, Sonal Choudhary, Arash Mostaghimi, Zackory Erickson, Carmel Majidi.
  Paper | arXiv

  Abstract

  Three-dimensional (3-D) skin surface reconstruction offers promise for objective and quantitative dermatological assessment, but no portable, high-resolution device exists that has been validated and used for depth reconstruction across various body locations. We present a compact 3-D skin reconstruction probe based on GelSight tactile imaging with a custom elastic gel and a learning-based reconstruction algorithm for micron-level wrinkle height estimation. Our probe, integrated into a handheld probe with force sensing for consistent contact, achieves a mean absolute error of 12.55 micron on wrinkle-like test objects. In a study with 15 participants without skin disorders, we provide the first validated wrinkle depth metrics across multiple body regions. We further demonstrate statistically significant reductions in wrinkle height at three locations following over-the-counter moisturizer application. Our work offers a validated tool for clinical and cosmetic skin analysis, with potential applications in diagnosis, treatment monitoring, and skincare efficacy evaluation.

  BibTeX

  @article{padmanabha2025vivo,
    title     = {In Vivo Skin 3-{D} Surface Reconstruction and Wrinkle Depth Estimation Using Handheld High Resolution Tactile Sensing},
    author    = {Padmanabha, Akhil and Agarwal, Arpit and Li, Catherine and Williams, Austin and Patel, Dinesh K. and Chopkar, Sankalp and Wilson, Achu and Ozkan, Ahmet and Yuan, Wenzhen and Choudhary, Sonal and Mostaghimi, Arash and Erickson, Zackory and Majidi, Carmel},
    journal   = {Advanced Healthcare Materials},
    publisher = {Wiley},
    volume    = {15},
    number    = {11},
    year      = {2025},
    month     = dec,
    doi       = {10.1002/adhm.202504402}
  }

• 
  PneuGelSight: Soft Robotic Vision-Based Proprioception and Tactile Sensing

  International Journal of Robotics Research 2025

  Ruohan Zhang, Uksang Yoo, Yichen Li, Arpit Agarwal, Wenzhen Yuan.
  PDF

  Abstract

  Soft pneumatic robot manipulators are popular in industrial and human-interactive applications due to their compliance and flexibility. However, deploying them in real-world scenarios requires advanced sensing for tactile feedback and proprioception. Our work presents a novel vision-based approach for sensorizing soft robots. We demonstrate our approach on PneuGelSight, a pioneering pneumatic manipulator featuring high-resolution proprioception and tactile sensing via an embedded camera. To optimize the sensor's performance, we introduce a comprehensive pipeline that accurately simulates its optical and dynamic properties, facilitating a zero-shot knowledge transition from simulation to real-world applications. PneuGelSight and our sim-to-real pipeline provide a novel, easily implementable, and robust sensing methodology for soft robots, paving the way for the development of more advanced soft robots with enhanced sensory capabilities.

  BibTeX

  @article{zhang2025pneugelsight,
    title     = {{PneuGelSight}: Soft Robotic Vision-Based Proprioception and Tactile Sensing},
    author    = {Zhang, Ruohan and Yoo, Uksang and Li, Yichen and Agarwal, Arpit and Yuan, Wenzhen},
    journal   = {The International Journal of Robotics Research},
    publisher = {SAGE Publications},
    volume    = {45},
    number    = {7},
    pages     = {1005--1020},
    year      = {2025},
    month     = oct,
    doi       = {10.1177/02783649251378153}
  }

• 
  A modularized design approach for GelSight family of vision-based tactile sensors

  International Journal of Robotics Research 2025

  Arpit Agarwal, Mohammad Amin Mirzaee, Xiping Sun, Wenzhen Yuan.
  PDF

  Abstract

  GelSight family of vision-based tactile sensors has proven to be effective for multiple robot perception and manipulation tasks. These sensors are based on an internal optical system and an embedded camera to capture the deformation of the soft sensor surface, inferring the high-resolution geometry of the objects in contact. However, customizing the sensors for different robot hands requires a tedious trial-and-error process to redesign the optical system. In this paper, we formulate the GelSight sensor design process as a systematic and objective-driven design problem and perform the design optimization with a physically accurate optical simulation. The method is based on modularizing and parameterizing the sensor’s optical components and designing four generalizable objective functions to evaluate the sensor. We implement the method with an interactive and easy-to-use toolbox called OptiSense Studio. With the toolbox, non-sensor experts can quickly optimize their sensor design in both forward and inverse ways following our predefined modules and steps. We demonstrate our system with four different GelSight sensors by quickly optimizing their initial design in simulation and transferring it to the real sensors.

  BibTeX

  @article{agarwal2025modularized,
    title     = {A modularized design approach for {GelSight} family of vision-based tactile sensors},
    author    = {Agarwal, Arpit and Mirzaee, Mohammad Amin and Sun, Xiping and Yuan, Wenzhen},
    journal   = {The International Journal of Robotics Research},
    publisher = {SAGE Publications},
    volume    = {44},
    number    = {13},
    pages     = {2195--2214},
    year      = {2025},
    month     = may,
    doi       = {10.1177/02783649251339680}
  }

• 
  Vision-based tactile sensor design using physically based rendering

  Communications Engineering (Nature Portfolio), 2025

  Arpit Agarwal¹, Achu Wilson¹, Timothy Man¹, Edward Adelson³, Ioannis Gkioulekas¹ and Wenzhen Yuan²
  Affiliations: 1 - Carnegie Mellon University, 2 - UIUC, 3 - MIT
  Paper

  Abstract

  High-resolution tactile sensors are very helpful to robots for fine-grained perception and manipulation tasks, but designing those sensors is challenging. This is because the designs are based on the compact integration of multiple optical elements, and it is difficult to understand the correlation between the element arrangements and the sensor accuracy by trial and error. In this work, we introduce the first-ever digital design of vision-based tactile sensors using a physically accurate light simulator and machine intelligence. The framework modularizes the design process, parameterizes the sensor components, and contains an evaluation metric to quantify a sensor's performance. We quantify the effects of sensor shape, illumination setting, and sensing surface material on tactile sensor performance using our evaluation metric. For the first time, to our knowledge, the proposed optical simulation framework can replicate the tactile image of the real sensor prototype. Using our approach we can substantially improve the design of a fingertip GelSight sensor. This improved design performs approximately 5 times better than previous state-of-the-art human-expert design at real-world robotic tactile embossed text detection. Our simulation approach can be used with any vision-based tactile sensor to produce a physically accurate tactile image. Overall, our approach maps human intelligence to machine intelligence for the automatic design of sensorized soft robots and opens the door for tactile-driven dexterous manipulation.

  BibTeX

  @article{agarwal2025vision,
    title     = {Vision-based tactile sensor design using physically based rendering},
    author    = {Agarwal, Arpit and Wilson, Achu and Man, Timothy and Adelson, Edward and Gkioulekas, Ioannis and Yuan, Wenzhen},
    journal   = {Communications Engineering},
    publisher = {Springer Nature},
    volume    = {4},
    number    = {1},
    pages     = {21},
    year      = {2025},
    month     = feb,
    doi       = {10.1038/s44172-025-00350-4}
  }

• 
  Scalable Simulation-Guided Compliant Tactile Finger Design

  IEEE International Conference on Soft Robotics (RoboSoft) 2024

  Yuxiang Ma⁺³, Arpit Agarwal⁺¹, Sandra Liu⁺³, Wenzhen Yuan², and Edward Adelson³
  Affiliations: 1 - Carnegie Mellon University, 2 - UIUC, 3 - MIT
  ⁺ Authors with equal contributions
  ArXiv | Video

  Abstract

  Compliant grippers enable robots to work with humans in unstructured environments. In general, these grippers can improve with tactile sensing to estimate the state of objects around them to precisely manipulate objects. However, co-designing compliant structures with high-resolution tactile sensing is a challenging task. We propose a simulation framework for the end-to-end forward design of GelSight Fin Ray sensors. Our simulation framework consists of mechanical simulation using the finite element method (FEM) and optical simulation including physically based rendering (PBR). To simulate the fluorescent paint used in these GelSight Fin Rays, we propose an efficient method that can be directly integrated in PBR. Using the simulation framework, we investigate design choices available in the compliant grippers, namely gel pad shapes, illumination conditions, Fin Ray gripper sizes, and Fin Ray stiffness. This infrastructure enables faster design and prototype time frames of new Fin Ray sensors that have various sensing areas, ranging from 48mm x 18mm to 70mm x 35mm. Given the parameters we choose, we can thus optimize different Fin Ray designs and show their utility in grasping day-to-day objects.

  BibTeX

  @inproceedings{ma2024scalable,
    title     = {Scalable Simulation-Guided Compliant Tactile Finger Design},
    author    = {Ma, Yuxiang and Agarwal, Arpit and Liu, Sandra Q. and Yuan, Wenzhen and Adelson, Edward H.},
    booktitle = {2024 IEEE 7th International Conference on Soft Robotics (RoboSoft)},
    publisher = {IEEE},
    pages     = {1068--1074},
    year      = {2024},
    month     = apr,
    doi       = {10.1109/RoboSoft60065.2024.10521969}
  }

• 
  Robotic Defect Inspection with Visual and Tactile Perception for Large-scale Components

  IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2023

  Arpit Agarwal¹, Abhiroop Ajith², Chengtao Wen², Veniamin Stryzheus³, Brian Miller³, Matthew Chen³, Micah K. Johnson⁴, Jose Luis Susa Rincon², Justinian Rosca² and Wenzhen Yuan¹
  Affiliations: 1 - Carnegie Mellon University, 2 - Siemens Corporations, 3 - Boeing, 4 - GelSight Inc.
  PDF | Dataset

  Abstract

  In manufacturing processes, surface inspection is a key requirement for quality assessment and damage localization. Due to this, automated surface anomaly detection has become a promising area of research in various industrial inspection systems. A particular challenge in industries with large-scale components, like aircraft and heavy machinery, is inspecting large parts with very small defect dimensions. Moreover, these parts can be of curved shapes. To address this challenge, we present a 2-stage multi-modal inspection pipeline with visual and tactile sensing. Our approach combines the best of both visual and tactile sensing by identifying and localizing defects using a global view (vision) and using the localized area for tactile scanning for identifying remaining defects. To benchmark our approach, we propose a novel real-world dataset with multiple metallic defect types per image, collected in the production environments on real aerospace manufacturing parts, as well as online robot experiments in two environments. Our approach is able to identify 85% defects using Stage I and identify 100% defects after Stage II.

  BibTeX

  @inproceedings{agarwal2023robotic,
    title     = {Robotic Defect Inspection with Visual and Tactile Perception for Large-Scale Components},
    author    = {Agarwal, Arpit and Ajith, Abhiroop and Wen, Chengtao and Stryzheus, Veniamin and Miller, Brian and Chen, Matthew and Johnson, Micah K. and Susa Rincon, Jose Luis and Rosca, Justinian and Yuan, Wenzhen},
    booktitle = {2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
    publisher = {IEEE},
    pages     = {10110--10116},
    year      = {2023},
    month     = oct,
    doi       = {10.1109/IROS55552.2023.10341590}
  }

• 
  Simulation-driven vision-based tactile sensor design

  IEEE International Conference on Computational Photography (ICCP) 2022

  Arpit Agarwal, Timothy Man, Edward Adelson, Ioannis Gkiolekas, Wenzhen Yuan.
  PDF

  Abstract

  We leverage physics-based rendering techniques to simulate the light transport process inside curved vision-based tactile sensor designs. We use physically-grounded models of light and material in our simulation. We also use lightweight and fast calibration methods for fitting the analytical models of light and material for our prototype. Given a calibrated simulation framework, we propose a tactile sensor shape optimization pipeline. Towards this goal, we propose a low-dimensional tactile sensor shape parameterization and automatically generate the full sensor prototype and an indenter surface virtually, which allows us to validate the sensor performance across the sensor surface. Our main technical results include a) accurately matching RGB image between simulation and a physical prototype b) generating improved tactile sensor shapes c) characterizing the design parameter space using appropriate light piping metrics. Our physically accurate simulation framework offers the ability to generate accurate RGB images for arbitrary vision-based tactile sensors. The parameter space exploration gives us high level guidelines on the design of tactile sensors for specific applications. Lastly, our system allows us to characterize the various tactile sensor designs in terms of their 3D shape reconstruction ability on different parts of the sensor surface.

  BibTeX

  @misc{agarwal2022simulation,
    title        = {Simulation-driven vision-based tactile sensor design},
    author       = {Agarwal, Arpit and Man, Timothy and Adelson, Edward and Gkioulekas, Ioannis and Yuan, Wenzhen},
    howpublished = {Poster at the IEEE International Conference on Computational Photography (ICCP)},
    year         = {2022}
  }

• 
  Grasp Stability Prediction with Sim-to-Real Transfer from Tactile Sensing

  IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2022

  Zilin Si, Zirui Zhu, Arpit Agarwal, Wenzhen Yuan.
  PDF | project page | code

  Abstract

  Robot simulation has been an essential tool for data-driven manipulation tasks. However, most existing simulation frameworks lack either efficient and accurate models of physical interactions with tactile sensors or realistic tactile simulation. This makes the sim-to-real transfer for tactile-based manipulation tasks still challenging. In this work, we integrate simulation of robot dynamics and vision-based tactile sensors by modeling the physics of contact. This contact model uses simulated contact forces at the robot's end-effector to inform the generation of realistic tactile outputs. To eliminate the sim-to-real transfer gap, we calibrate our physics simulator of robot dynamics, contact model, and tactile optical simulator with real-world data, and then we demonstrate the effectiveness of our system on a zero-shot sim-to-real grasp stability prediction task where we achieve an average accuracy of 90.7% on various objects. Experiments reveal the potential of applying our simulation framework to more complicated manipulation tasks. We open-source our simulation framework at https://github.com/CMURoboTouch/Taxim/tree/taxim-robot

  BibTeX

  @inproceedings{si2022grasp,
    title     = {Grasp Stability Prediction with Sim-to-Real Transfer from Tactile Sensing},
    author    = {Si, Zilin and Zhu, Zirui and Agarwal, Arpit and Anderson, Stuart and Yuan, Wenzhen},
    booktitle = {2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
    publisher = {IEEE},
    pages     = {7809--7816},
    year      = {2022},
    month     = oct,
    doi       = {10.1109/IROS47612.2022.9981863}
  }

• 
  Simulation of Vision-based Tactile Sensors using Physics based Rendering

  IEEE International Conference on Robotics and Automation (ICRA) 2021

  Arpit Agarwal, Tim Man, Wenzhen Yuan.
  Conference Version | Extended Version | project page | code

  Abstract

  Tactile sensing has seen a rapid adoption with the advent of vision-based tactile sensors. Vision-based tactile sensors provide high resolution, compact and inexpensive data to perform precise in-hand manipulation and human-robot interaction. However, the simulation of tactile sensors is still a challenge. In this paper, we built the first fully general optical tactile simulation system for a GelSight sensor using physics-based rendering techniques. We propose physically accurate light models and show in-depth analysis of individual components of our simulation pipeline. Our system outperforms previous simulation techniques qualitatively and quantitative on image similarity metrics.

  BibTeX

  @inproceedings{agarwal2021simulation,
    title     = {Simulation of Vision-based Tactile Sensors using Physics based Rendering},
    author    = {Agarwal, Arpit and Man, Timothy and Yuan, Wenzhen},
    booktitle = {2021 IEEE International Conference on Robotics and Automation (ICRA)},
    publisher = {IEEE},
    pages     = {1--7},
    year      = {2021},
    month     = may,
    doi       = {10.1109/ICRA48506.2021.9561122}
  }

• 
  Improving Grasp Stability with Rotation Measurement from Tactile Sensing

  IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2021

  Raj Kolamuri, Zilin Si, Yufan Zhang, Arpit Agarwal, Wenzhen Yuan.
  PDF | project page

  Abstract

  Rotational displacement about the grasping point is a common grasp failure when an object is grasped at a location away from its center of gravity. Tactile sensors with soft surfaces, such as GelSight sensors, can detect the rotation patterns on the contacting surfaces when the object rotates. In this work, we propose a model-based algorithm that detects those rotational patterns and measures rotational displacement using the GelSight sensor. We also integrate the rotation detection feedback into a closed-loop regrasping framework, which detects the rotational failure of grasp in an early stage and drives the robot to a stable grasp pose. We validate our proposed rotation detection algorithm and grasp-regrasp system on self-collected dataset and online experiments to show how our approach accurately detects the rotation and increases grasp stability.

  BibTeX

  @inproceedings{kolamuri2021improving,
    title     = {Improving Grasp Stability with Rotation Measurement from Tactile Sensing},
    author    = {Kolamuri, Raj and Si, Zilin and Zhang, Yufan and Agarwal, Arpit and Yuan, Wenzhen},
    booktitle = {2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
    publisher = {IEEE},
    pages     = {6809--6816},
    year      = {2021},
    month     = sep,
    doi       = {10.1109/IROS51168.2021.9636488}
  }

• 
  Model learning for look-ahead exploration in continuous control

  AAAI Conference on Artificial Intelligence (AAAI), 2019

  Arpit Agarwal, Katharina Muelling, Katerina Fragkiadaki.
  PDF | project page | code

  Abstract

  We propose an exploration method that incorporates look- ahead search over basic learnt skills and their dynamics, and use it for reinforcement learning (RL) of manipulation poli- cies . Our skills are multi-goal policies learned in isolation in simpler environments using existing multigoal RL formula- tions, analogous to options or macroactions. Coarse skill dy- namics, i.e., the state transition caused by a (complete) skill execution, are learnt and are unrolled forward during looka- head search. Policy search benefits from temporal abstrac- tion during exploration, though itself operates over low-level primitive actions, and thus the resulting policies does not suf- fer from suboptimality and inflexibility caused by coarse skill chaining. We show that the proposed exploration strategy results in effective learning of complex manipulation poli- cies faster than current state-of-the-art RL methods, and con- verges to better policies than methods that use options or parametrized skills as building blocks of the policy itself, as opposed to guiding exploration. We show that the proposed exploration strategy results in effective learning of complex manipulation policies faster than current state-of-the-art RL methods, and converges to better policies than methods that use options or parameterized skills as building blocks of the policy itself, as opposed to guiding exploration.

  BibTeX

  @inproceedings{agarwal2019model,
    title     = {Model Learning for Look-Ahead Exploration in Continuous Control},
    author    = {Agarwal, Arpit and Muelling, Katharina and Fragkiadaki, Katerina},
    booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence},
    publisher = {Association for the Advancement of Artificial Intelligence},
    volume    = {33},
    number    = {01},
    pages     = {3151--3158},
    year      = {2019},
    month     = jul,
    doi       = {10.1609/aaai.v33i01.33013151}
  }

• 
  Reinforcement Learning of Active Vision for Manipulating Objects under Occlusions

  Conference on Robot Learning (CoRL), 2018

  Ricson Cheng, Arpit Agarwal, Katerina Fragkiadaki.
  PDF

  Abstract

  We consider artificial agents that learn to jointly control their gripper and camera in order to reinforcement learn manipulation policies in the presence of occlusions from distractor objects. Distractors often occlude the object of in- terest and cause it to disappear from the field of view. We propose hand/eye con- trollers that learn to move the camera to keep the object within the field of view and visible, in coordination to manipulating it to achieve the desired goal, e.g., pushing it to a target location. We incorporate structural biases of object-centric attention within our actor-critic architectures, which our experiments suggest to be a key for good performance. Our results further highlight the importance of curriculum with regards to environment difficulty. The resulting active vision / manipulation policies outperform static camera setups for a variety of cluttered environments.

  BibTeX

  @inproceedings{cheng2018reinforcement,
    title     = {Reinforcement Learning of Active Vision for Manipulating Objects under Occlusions},
    author    = {Cheng, Ricson and Agarwal, Arpit and Fragkiadaki, Katerina},
    booktitle = {Proceedings of The 2nd Conference on Robot Learning},
    series    = {Proceedings of Machine Learning Research},
    publisher = {PMLR},
    volume    = {87},
    pages     = {422--431},
    year      = {2018},
    month     = oct,
    url       = {https://proceedings.mlr.press/v87/cheng18a.html}
  }

Theses

• 
  A Modularized Approach to Vision-based Tactile Sensor Design Using Physics-based Rendering

  Ph.D. Thesis, Robotics Institute, Carnegie Mellon University, 2024

  Arpit Agarwal. Advisors: Wenzhen Yuan, Ioannis Gkioulekas.
  Thesis | DOI

  Abstract

  Touch is an essential sensing modality for making autonomous robots more dexterous and allowing them to work collaboratively with humans. In particular, the advent of vision-based tactile sensors has resulted in efforts to design such sensors that can be easily incorporated into various robot structures for different robotic manipulation tasks, to increase robustness, precision, and reliability. However, this design task remains a challenging problem. This is for two reasons: first, the design of the sensor itself requires the compact integration of multiple optical elements to improve optical signal fidelity during interaction with the environment; second, the successful integration of vision-based tactile sensors into robotic manipulation tasks requires the codesign of both the sensors and the robot structure itself. This thesis aims to alleviate these two challenges by creating a general design framework that allows a roboticist to quickly iterate on the design and evaluation of vision-based tactile sensors for designated robotic manipulation tasks. The framework comprises three core elements. First, our framework uses an optical simulator that can accurately and efficiently generate the images captured by arbitrary sensor designs. Our simulator leverages physics-based rendering techniques from computer graphics and enables the generation of realistic tactile images for any given sensor design. To create this simulator, we performed detailed real-to-sim experiments to calibrate our simulation models. We show that the resulting simulator can qualitatively and quantitatively match real-world measurements for GelSight-like sensors with flat and curved sensing surfaces. Second, our framework proposes computational techniques for procedural sensor generation and automatic sensor design evaluation techniques. In the context of curved tactile sensors, our generator takes as input a 2D curve and uses CAD primitives to generate from it the full sensor shape. The procedural sensor generation allows for the automatic placement of different optical components, given their corresponding reference geometry. We introduce three objective functions: RGB2Norm, NormDiff, and As-orthographic-as-possible. These objective functions quantify sensor design’s tactile signal perception and enable automatic parameter selection. Third, our framework introduces an interactive design toolbox, OptiSense Studio, that functionalizes our design pipeline into a useful tool for novice users. We introduce general design modules for the rapid prototyping of GelSight-like tactile sensors. The toolbox allows interactive feedback through optical simulation while designing the sensor without deep expertise. The obtained design is automatically parameterized through our toolbox and can be optimized using our proposed objective functions. We have successfully applied this framework for the design of vision-based tactile sensors that used curved surfaces to emulate human fingertips. We have also applied our interactive framework for rapidly creating optimized variants of existing tactile sensors, GelSight Mini, GelSight360, and GelSight Svelte. Finally, we are able to create a new sensor, GelBelt, for a different robotic application completely virtually and optimize its illumination settings using our toolbox. Through this thesis, we demonstrate the utility of our design framework for the co-design of vision-based tactile sensors and soft robots. More broadly, we hope to create a new point of convergence between disparate communities such as computer graphics (physics-based rendering and simulation), optics (optical lens and material design), and robotics, and foster new research directions within and across these communities.

  BibTeX

  @phdthesis{agarwal2024modularized,
    title   = {A Modularized Approach to Vision-based Tactile Sensor Design Using Physics-based Rendering},
    author  = {Agarwal, Arpit},
    school  = {Carnegie Mellon University},
    address = {Pittsburgh, PA},
    type    = {{Ph.D.} thesis},
    note    = {Robotics Institute. Advisors: Wenzhen Yuan and Ioannis Gkioulekas. Posted online January 2026},
    year    = {2024},
    doi     = {10.1184/R1/31076170.v1},
    url     = {https://kilthub.cmu.edu/articles/thesis/A_Modularized_Approach_to_Vision-based_Tactile_Sensor_Design_Using_Physics-based_Rendering/31076170}
  }

• 
  Deep Reinforcement Learning with Skill Library: Exploring with Temporal Abstractions and coarse approximate Dynamics Models

  M.S. Thesis, Robotics Institute, Carnegie Mellon University, 2018 (CMU-RI-TR-18-31)

  Arpit Agarwal. Advisors: Katerina Fragkiadaki, Katharina Muelling.
  Thesis | PDF

  Abstract

  Reinforcement learning is a computational approach to learn from interaction. However, learning from scratch using reinforcement learning requires exorbitant number of interactions with the environment even for simple tasks. One way to alleviate the problem is to reuse previously learned skills as done by humans. This thesis provides frameworks and algorithms to build and reuse Skill Library. Firstly, we extend the Parameterized Action Space formulation using our Skill Library to multi-goal setting and show improvements in learning using hindsight at coarse level. Secondly, we use our Skill Library for exploring at a coarser level to learn the optimal policy for continuous control. We demonstrate the benefits, in terms of speed and accuracy, of the proposed approaches for a set of real world complex robotic manipulation tasks in which some state-of-the-art methods completely fail.

  BibTeX

  @mastersthesis{agarwal2018deep,
    title   = {Deep Reinforcement Learning with Skill Library: Exploring with Temporal Abstractions and Coarse Approximate Dynamics Models},
    author  = {Agarwal, Arpit},
    school  = {Carnegie Mellon University},
    address = {Pittsburgh, PA},
    type    = {Master's thesis},
    number  = {CMU-RI-TR-18-31},
    year    = {2018},
    month   = jul,
    url     = {https://publications.ri.cmu.edu/deep-reinforcement-learning-with-skill-library-exploring-with-temporal-abstractions-and-coarse-approximate-dynamics-models}
  }

Projects

• 
  Position Free Monte Carlo for layered material rendering

  Physics Based Rendering, CMU Spring 2021

  Arpit Agarwal
  Report

  Abstract

  Real world objects are often coated with multiple layers on top the base color to protect from scratches or to provide rich diversity of material appear- ance like spatially varying specular highlights. The idea of layered material modeling tries to circumvent explicitly calculating light transport with each coating layer and replace it with BSDF which gives equivalent reflection and transmission component, albeit with some assumption. Past approaches have tackled the problem of modeling layered material by either creating a concise representation of captured data or by precomputing a low-order statistics of the material. My project is based on [Guo et al. 2018] work on using position free Monte Carlo with plane parallel layers with participating media to provide the BRDF. In my project, I have implemented their unidi- rectional evaluation and approximate PDF approach to show the concept. The report highlights the main algorithm and benefits of the approach.

• 
  Exploration with Expert Policy Advice

  Technical Report, CMU 2018

  Ashwin Khadke, Arpit Agarwal, Anahita Mohseni-Kabir, Devin Schwab
  Report

  Abstract

  Exploration for Reinforcement Learning is a challenging problem. Random exploration is often highly inefficient and in sparse reward environments may completely fail. In this work, we developed a novel method which incorporates ex- pert advice for exploration in sparse reward environments. In our formulation, the agent has access to a set of expert policies and learns to bias its exploration based on the ex- perts’ suggested actions. By incorporating expert suggestions the agent is able to quickly learn a policy to reach rewarding states. Our method can mix and match experts’ advice during an episode to reach goal states. Moreover, our formulation does not restrict the agent to any policy set. This allows us to aim for a globally optimal solution. In our experiments, we show that using expert advice indeed leads to faster explo- ration in challenging grid-world environments.

Software

• CoRL 2026 Paper Explorer

  Searchable, topic-tagged list of accepted CoRL 2026 papers.
  Explorer

• 
  IES Light Profile Viewer

  Viewer Link