Hitesh Sapkota

I am an Applied Scientist II in Devices & Services — Edge AI & Science at Amazon, where I develop efficient and reliable large language models (LLMs) and vision-language models (VLMs) for resource-constrained, on-device deployment. I was recognized under the U.S. EB-1B Outstanding Researcher category in 2025. My work broadly aims to make language and multimodal AI more efficient, grounded, and trustworthy.

Previously, I completed my Ph.D. in Computing and Information Sciences at Rochester Institute of Technology, advised by Prof. Qi Yu in the Machine Learning and Data Intensive Computing lab. My doctoral research focused on distributionally robust optimization, evidential learning, weak supervision, and calibrated sparse networks. I earned my B.E. in Electronics and Communication Engineering from Pulchowk Campus, Institute of Engineering, Tribhuvan University, Nepal.

Research Overview

My research focuses on:

(a) improving the efficiency of LLM reasoning and multimodal models through techniques such as reasoning-token compression, quantization, and model compression;

(b) enhancing VLM grounding and reducing hallucinations by exploring novel supervised fine-tuning, distillation, and reinforcement-learning-based post-training methods such as GRPO; and

(c) developing calibrated, reliable, and trustworthy visual question-answering systems through uncertainty-aware learning and distributionally robust optimization.

My research has appeared at ICML, NeurIPS, ICLR, CVPR, KDD, AAAI, AISTATS, and other peer-reviewed venues.

Research Interests

Reinforcement Learning, Large Language Models, Multimodal Models, Visual Question Answering, Efficient Reasoning, Model Compression and Quantization, Hallucination Reduction, Uncertainty-Aware Learning, and Distributionally Robust Optimization.

News

Apr 30, 2026	Our paper Calibrated Knowledge Aggregation in Bayesian Mixture-of-Experts for Continual VQA has been accepted by ICML 2026.
Feb 20, 2026	Our paper The Road Less Seen: Segment Exploration for Weakly Supervised Video Anomaly Detection has been accepted by CVPR 2026.
Jan 25, 2026	Our paper Knowledge Exchange with Confidence: Cost-Effective LLM Integration for Reliable and Efficient Visual Question Answering has been accepted by ICLR 2026.
Dec 9, 2024	Our paper GLEN: Generalized Focal Loss Ensemble of Low-Rank Networks for Calibrated Visual Question Answering has been accepted by AAAI 2025.
Sep 25, 2024	Our paper Adaptive Important Region Selection with Reinforced Hierarchical Search for Dense Object Detection has been accepted by NeurIPS 2024.
May 16, 2024	Our paper Reinforced Compressive Neural Architecture Search for Versatile Adversarial Robustness has been accepted by KDD 2024.
May 1, 2024	Our paper Meta Evidential Transformer for Few-Shot Open-Set Recognition has been accepted by ICML 2024.
Dec 4, 2023	I started my full-time position as an Applied Scientist II at Amazon.
Sep 22, 2023	I successfully defended my Ph.D. dissertation, Robust Weakly Supervised Learning for Real-World Anomaly Detection.
Sep 21, 2023	Our paper Distributionally Robust Ensemble of Lottery Tickets Towards Calibrated Sparse Network Training has been accepted by NeurIPS 2023.
Jan 20, 2023	Our paper Adaptive Robust Evidential Optimization for Open-Set Detection from Imbalanced Data has been accepted by ICLR 2023.
Sep 29, 2022	I successfully defended my Ph.D. proposal, Robust Weakly Supervised Learning for Real-World Anomaly Detection.
Jun 20, 2022	I received a competitive KDD Travel Award to attend the conference in Washington, D.C.
Jun 20, 2022	Our paper Balancing Bias and Variance for Active Weakly Supervised Learning has been accepted by KDD 2022.
May 23, 2022	I started an internship as an Applied Scientist at Amazon.
Mar 20, 2022	I received an RIT Travel Grant to attend CVPR in New Orleans.
Mar 2, 2022	Our paper Bayesian Nonparametric Submodular Video Partition for Robust Anomaly Detection has been accepted by CVPR 2022.
Aug 31, 2021	Our co-authored paper Deep Reinforced Attention Regression for Partial Sketch-Based Image Retrieval has been accepted as a long paper by ICDM 2021 (acceptance rate: 9.9%).
Jun 7, 2021	I started an internship as an Applied Scientist at Amazon Web Services.
Jan 22, 2021	Our paper Distributionally Robust Optimization for Deep Kernel Multiple Instance Learning has been accepted by AISTATS 2021.

Selected Publications

CVPR

Bayesian Nonparametric Submodular Video Partition for Robust Anomaly Detection

Hitesh Sapkota, and Qi Yu

In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Jun 2022

Abs PDF

Multiple-instance learning (MIL) provides an effective way to tackle the video anomaly detection problem by modeling it as a weakly supervised problem as the labels are usually only available at the video level while missing for frames due to expensive labeling cost. We propose to conduct novel Bayesian non-parametric submodular video partition (BN-SVP) to significantly improve MIL model training that can offer a highly reliable solution for robust anomaly detection in practical settings that include outlier segments or multiple types of abnormal events. BN-SVP essentially performs dynamic non-parametric hierarchical clustering with an enhanced self-transition that groups segments in a video into temporally consistent and semantically coherent hidden states that can be naturally interpreted as scenes. Each segment is assumed to be generated through a non-parametric mixture process that allows variations of segments within the same scenes to accommodate the dynamic and noisy nature of many real-world surveillance videos. The scene and mixture component assignment of BN-SVP also induces a pairwise similarity among segments, resulting in non-parametric construction of a submodular set function. Integrating this function with an MIL loss effectively exposes the model to a diverse set of potentially positive instances to improve its training. A greedy algorithm is developed to optimize the submodular function and support efficient model training. Our theoretical analysis ensures a strong performance guarantee of the proposed algorithm. The effectiveness of the proposed approach is demonstrated over multiple real-world anomaly video datasets with robust detection performance.
NeurIPS

Adaptive Important Region Selection with Reinforced Hierarchical Search for Dense Object Detection

Dingrong Wang, Hitesh Sapkota, and Qi Yu

In Advances in Neural Information Processing Systems Dec 2024

PDF Code
ICML

Calibrated Knowledge Aggregation in Bayesian Mixture-of-Experts for Continual VQA

Mahsa Mozaffari, Hitesh Sapkota, Yu Kong, and 2 more authors

In Proceedings of the 43rd International Conference on Machine Learning Jul 2026

PDF