Overview

Video

ContactExplorer uses state-conditioned contact coverage to explore diverse dexterous manipulation tasks and transfers a distilled vision policy to real-world object singulation.

Abstract

Reinforcement learning explores effectively in domains such as Atari games, navigation, and locomotion, where novelty over states or dynamics is a sufficient signal. In contrast, dexterous manipulation requires rich physical hand–object interactions, but existing methods often suffer from unstable contact-based novelty signals, inefficient distance novelty signals, or reliance on task-specific priors. We propose ContactExplorer, a general exploration method for dexterous manipulation tasks. ContactExplorer represents contact as the intersection between object surface points and hand keypoints, encouraging dexterous hands to discover diverse and novel contact patterns, namely which fingers contact which object regions. It maintains a contact counter conditioned on discretized object states obtained via learned hash codes. This counter is leveraged in two complementary ways: (1) a count-based contact coverage reward that promotes exploration of novel contact patterns, and (2) an energy-based reaching reward that guides the agent toward under-explored contact regions. We evaluate ContactExplorer on seven contact-rich manipulation tasks and five dexterous hand embodiments. Experimental results show that ContactExplorer substantially improves sample efficiency and success rates over existing exploration methods, that it reduces the need for task-specific priors, and that it remains effective across hand embodiments and transfers to the real world.

Simulation Tasks

Cluttered Object Singulation
Constrained Object Retrieval
In-Hand Reorientation (Elephant)
In-Hand Reorientation (Mug)
Bimanual Manipulation (Waffle)
Bimanual Manipulation (Box)
In-Hand Reorientation (Cube)
In-Hand Reorientation (Bunny)

Real-World Results

We distill the simulation oracle policy into a vision-based policy and evaluate real-world object singulation with randomized target objects. Videos are shown at 1x speed.

Select trial
Global view
Top-down view
Side view

Method

ContactExplorer Method overview

State hashing

Current
poseGoal
pose
Learned encoder φ
1001110
SimHash → cluster q

Contact counter

C(q, f, k)
State × finger × region
Contact events
Finger f · Region k
Before contact

Energy-based reaching

Under-explored regions
renergy
On contact

Contact coverage

Novel finger–region pairs
rcontact

PPO

Policy
update

Physics simulation

Shared contact counts guide reaching and reward novel contacts. Counts are updated after reward evaluation.

Method overview video

Shared contact counts

ContactExplorer tracks finger–region contacts with a state-conditioned counter. The same counts guide reaching toward under-explored regions before contact and reward novel interactions when contact occurs.

Method details

Contact Events

We partition the object surface into regions using point positions and surface normals, and represent each finger with sparse surface keypoints. A distance–force criterion detects contact between a finger and an object region. This gives a concrete unit of exploration: a finger–region pair, rather than proximity to the object alone.

State-Conditioned Contact Counts

The same contact may be useful again when the object configuration or goal changes. We therefore encode point clouds of the current and goal object poses with an autoencoder, then use SimHash to assign a discrete state cluster. A counter records how often each finger–region pair has been contacted within that cluster. These counts persist across training episodes; rarely visited pairs receive higher novelty weights, while repeated contacts contribute less.

Exploration Rewards

Before contact: the energy-based reaching score combines the same novelty weights with a distance-based kernel over object surface points. Nearby, under-explored regions contribute more, providing continuous guidance toward new interactions even while the hand is in free space.

When contact occurs: the contact coverage score measures the novelty of the finger–region pairs observed in contact. This contact-based signal reflects observed contacts rather than proximity alone.

For each score, we reward only improvements over its best value in the current episode, discouraging repeated exploitation of an already discovered interaction. Both rewards are computed before updating the contact counts, then added to the task reward for PPO training. Episode-wise maxima are reset at each episode boundary, while contact counts persist across episodes.

Experimental Results

Training curves across manipulation tasks
Training curves across manipulation tasks.

Experimental results video

Learning performance

ContactExplorer improves sample efficiency and final performance over the exploration baselines across the evaluated manipulation tasks. Results are also more consistent across random seeds.

BibTeX

@misc{liu2026contactexplorer,
  title = {{ContactExplorer}: Contact Coverage-Guided Exploration for General-Purpose Dexterous Manipulation},
  author = {Zixuan Liu and Ruoyi Qiao and Chenrui Tie and Xuanwei Liu and Yunfan Lou and Chongkai Gao and Zhixuan Xu and Lin Shao},
  year = {2026}
}