Rahul Sajnani

Rahul_linkedin.jpg

I am a CS Ph.D. candidate at Brown University advised by Srinath Sridhar, graduating Dec 2026. I build generative world models for robot learning — unifying video, action, and language into one representation so robots can plan, remember, and act end-to-end from pixels.

Previously: Applied Scientist II Intern at Amazon Robotics (2024–2026), Research Scientist Intern at Adobe Research (Summer 2025), and lead of the self-driving car perception stack at RRC, IIIT-Hyderabad. B.Tech + M.Sc. (ECE Gold Medal) from IIIT-H.

I am on the industry job market (2026) for research roles in robotics foundation models, autonomous driving, and generative modeling (also curious about brain–robot interfaces).

Research focus

  • Robotics foundation models — world-action-language models, memory-aware policies, diffusion/flow-matching for action, and sample-efficient RL fine-tuning. [UniWALM, AnchorQ]
  • Controllable generative models — steering diffusion internals for geometry, 3D, and multi-region control. [GeoDiffuser · WACV '25 Best Student Paper, AppearancePointers]
  • Canonicalization & 3D understanding — SO(3)-equivariant pose estimation on partial shapes and NeRFs, feed-forward 3D Gaussian Splatting from monocular video via video-diffusion priors (Wan2.1, LTX-Video, CogVideoX), regularity-aware scene composition, and dense reconstruction. [ConDor, Canonical Fields (CVPR Highlight), LEGO-Net, DRACO]
  • Perception for autonomous driving — monocular SLAM and multi-object tracking for dynamic urban scenes. [MoM-SLAM, BirdSLAM]

Also curious about brain–robot interfaces: using generative priors to decode intent from neural signals into robot action. Open to collaborating / joining a team already working here.

Collaborators & mentors: Srinath Sridhar, Leonidas Guibas, George Konidaris, Daniel Ritchie, Jeroen Vanbaar, Kapil Katyal, K. Madhava Krishna, Matheus Gadelha, Kavita Vemuri.

Featured News

Oct 19, 2026 I am on the industry job market! Looking for full-time research roles in robotics foundation models, autonomous driving, generative modeling, and brain–robot interfaces. Reach out.
May 12, 2025 I received an Outstanding Reviewer award at CVPR 2025! :tada:.
Apr 02, 2025 Our work GeoDiffuser received the Best Student Paper at WACV 2025 :tada:! Read article for more information.
Oct 28, 2024 Our work GeoDiffuser is accepted to WACV 2025 :tada:! See you in Arizona! We perform geometry-based image editing with T2I diffusion models without any training or model / fine-tuning.
Oct 24, 2024 I will present literature on Video Diffusion Models with a focus on camera/motion conditioned models at Brown IVL & BVC. See slides here.
Apr 01, 2024 I will present GeoDiffuser, a method to perform geometry-based image editing with T2I diffusion models at NYC Computer Vision Day. Our work does not need any training or model fine-tuning!
Jun 24, 2023 I will join Amazon Robotics as an Applied Scientist to improve robot perception and item identification using Generative models.
Jun 20, 2023 Check out our works Cafi-Net & Lego-Net at CVPR 2023!
Jun 01, 2022 We will be presenting ConDor, a method to perform unsupervised 3D pose canonicalization of full and partial shapes at CVPR 2022. Our work also performs automatic co-part segmentation!

Featured Publications

  1. uniwalm.png
    UniWALM: A Unified World-Action-Language Model for Embodied Robot Control
    Rahul Sajnani, Diego Romeres, Mingyuan Zhang, Kai Gao, Carl Qi, Baichuan Huang, Srinath Sridhar, Dylan Randle, Lillian J. Ratliff, Jose Barreiros, Manikantan Nambi, and Jeroen Vanbaar
    IEEE International Conference on Robotics and Automation (ICRA) (Under Review), 2027
    • Mixture-of-Transformers (Wan2.2 video + action DiT + discrete-diffusion language expert) co-generates video, action, and language in one parallel diffusion pass.
    • Policy re-reads its own generated language as memory on every control step — no external VLM / planner.
    • Real-world memory-dependent manipulation: 11.1% → 61.1% vs same-backbone FastWAM.
    • 34.5% on RMBench, above Mem-0’s privileged-keyframe setup (28.5%).
  2. appearance_pointers.png
    AppearancePointers: Fine-Grained Multi-Region Control with Appearance Pointers
    Rahul Sajnani, Matheus Gadelha, Yulia Gryaditskaya, Srinath Sridhar, and Radomír Mech
    International Conference on Learning Representations (ICLR), 2027 (Under Review), 2027
    • FLUX-based model for multi-region controllable generation: each output region independently steered by text, reference image, and material texture in a single denoising pass.
    • State-of-the-art on three multi-region fidelity metrics, beating DreamRenderer and MSDiffusion at larger scale.
    • Introduces the curated AP-37K multimodal dataset.
  3. anchorq.gif
    Sample-Efficient Reinforcement Learning for Policy Improvement via Flow-Reversed Action Anchors
    Carl Qi, Weijun Dong, Xiaojie Wang, Peiyi Chen, Rahul Sajnani, Mingyuan Zhang, Amy Zhang, Jose Barreiros, Manikantan Nambi, Lillian J. Ratliff, and Dylan Randle
    International Conference on Learning Representations (ICLR), 2027 (Under Review), 2027
    • Sample-efficient RL fine-tuning of generalist robot policies using flow-reversed action anchors.
    • A diffusion-policy improvement scheme that sharply reduces costly online interaction.
  4. GeoDiffuser_edit.gif
    GeoDiffuser: Geometry-Based Image Editing with Diffusion Models
    Rahul Sajnani, Jeroen Vanbaar, Jie Min, Kapil Katyal, and Srinath Sridhar
    In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)(Best Student Paper) , 2025
    • Training-free geometry-based image editing on a frozen T2I diffusion model.
    • User specifies a 3D rigid transform (rotate, translate, scale); GeoDiffuser rewrites attention maps to realize it.
    • No fine-tuning, no 3D supervision — WACV 2025 Best Student Paper.
  5. genhsi.gif
    GenHSI: Training-Free Controllable Generation of Human-Scene Interaction Videos
    Zekun Li, Rui Zhou, Rahul Sajnani, Xiaoyan Cong, Daniel Ritchie, and Srinath Sridhar
    In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2026
    • Training-free controllable generation of human–scene interaction videos.
    • Uses a video-diffusion prior with per-region controls — no retraining for new scenes or humans.
  6. CaFi_teaser.jpg
    Canonical Fields: Self-Supervised Learning of Pose-Canonicalized Neural Fields
    Rohith Agaram, Shaurya Dewan, Rahul Sajnani, Adrien Poulenard, Madhava Krishna, and Srinath Sridhar
    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)(Highlight) , Jun 2023
    • Self-supervised learning of pose-canonicalized neural fields (NeRFs).
    • Discovers a canonical frame for an object category without any pose or 3D supervision.
    • CVPR 2023 Highlight.
  7. lego-net.png
    LEGO-Net: Learning Regular Rearrangements of Objects in Rooms
    Qiuhong Anna Wei, Sijie Ding, Jeong Joon Park, Rahul Sajnani, Adrien Poulenard, Srinath Sridhar, and Leonidas Guibas
    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2023
    • Learns to rearrange cluttered rooms into regular layouts using a diffusion-based scene editor.
    • Discovers regularity patterns (alignment, symmetry) directly from indoor-scene data.
  8. ConDor.png
    ConDor: Self-Supervised Canonicalization of 3D Pose for Partial Shapes
    Rahul Sajnani, Adrien Poulenard, Jivitesh Jain, Radhika Dua, Leonidas J. Guibas, and Srinath Sridhar
    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2022
    • SO(3)-equivariant self-supervised 3D pose canonicalization — works on partial point clouds.
    • Also performs automatic co-part segmentation as a byproduct.
  9. draco.png
    DRACO: Weakly Supervised Dense Reconstruction And Canonicalization of Objects
    Rahul Sajnani, AadilMehdi J. Sanchawala, Krishna Murthy Jatavallabhula, Srinath Sridhar, and K. Madhava Krishna
    2021 IEEE International Conference on Robotics and Automation (ICRA), Jun 2021
    • Weakly-supervised dense reconstruction and pose canonicalization of 3D objects.
    • Learned without any 3D ground-truth supervision — IEEE ICRA 2021.
  10. mom-slam.png
    Multi-object Monocular SLAM for Dynamic Environments
    Gokul B. Nair, Swapnil Daga, Rahul Sajnani, Anirudh Ramesh, Junaid Ahmed Ansari, and K. Madhava Krishna
    2020 IEEE Intelligent Vehicles Symposium (IV), Jun 2020
    • Monocular SLAM that explicitly models independent object motion in dynamic driving scenes.
    • Handles moving vehicles and pedestrians without filtering them out.