DATAFARM: Distribution-Aligned Task and Motion Planning
for Fine-Tuning Vision-Language-Action Models

Anonymous Authors
Paper under review
Paper · coming soon Code · coming soon Appendix
Given a pretrained VLA and a new task, TAMP is cheap but off-distribution, teleoperation is on-distribution but expensive, and distribution-aligned TAMP (DATAFARM) is cheap and on-distribution. End-effector speed profiles show DATAFARM resembling teleoperation.
Task and motion planning is a scalable source of robot demonstrations, but its trajectories are behaviorally mismatched with the VLA pretraining distribution, so fine-tuning on them works poorly. Teleoperation is on-distribution but costs human time. DATAFARM aligns TAMP trajectories with the pretraining distribution, producing demonstrations that are both cheap and on-distribution.

Abstract

Collecting high-quality robot data remains a fundamental challenge for training robot foundation models. Task and motion planning (TAMP) offers a scalable way to generate demonstrations, but our experiments show that raw TAMP trajectories provide surprisingly little benefit when used to fine-tune pretrained vision-language-action (VLA) models, despite successfully solving the target tasks. We hypothesize that this failure arises from a behavioral distribution mismatch between planner-generated trajectories and the data used to pretrain the VLA.

To address this mismatch, we introduce DATAFARM: Distribution-Aligned Task And motion planning for Fine-tuning A Robot foundation Model, an approach that incorporates the pretraining distribution directly into TAMP trajectory generation. DATAFARM aligns generated trajectories with the pretraining data in robot joint configurations, motion style, and temporal execution profiles. We evaluate DATAFARM on three tabletop manipulation tasks that TAMP can perform and a cloth-folding task beyond the capability of TAMP. DATAFARM achieves an average success rate of 56.7%, substantially outperforming raw TAMP (8.3%) while approaching human teleoperation (61.7%). On Deformable Object Manipulation, which is outside the fine-tuning distribution, the fine-tuned model retains 85% success, compared with 90% for the pretrained model. These results show that aligning planner-generated demonstrations with the pretraining distribution can make TAMP an effective source of data for VLA fine-tuning.

Method

DATAFARM learns what "on-distribution" robot behavior looks like from the VLA's pretraining data (DROID), then uses that as an objective inside the planner. The planner still enforces every task and motion constraint.

DATAFARM overview: a Gaussian mixture over joint configurations and a trajectory encoder with a latent Gaussian are fit to pretraining data; these provide costs for TAMP endpoint selection, path optimization, and retiming; executed trajectories form a dataset used to fine-tune the VLA with flow matching.
Overview. From pretraining demonstrations Dref, DATAFARM fits a Gaussian mixture density pconf(q) over joint configurations and trains a trajectory encoder Eφ with a latent reference density platent(z). These frozen models supply alignment costs for TAMP endpoint selection, path optimization, and retiming. The resulting trajectories are executed on the robot to collect a dataset D, which fine-tunes the pretrained VLA π0.5-DROID.
1 · Joint space

Endpoint Selection

cost cconf(q) = −log pconf(q)

Inverse kinematics admits many arm postures for the same gripper pose. DATAFARM ranks the IK candidates, including the symmetric rotated grasp, under a 16-component Gaussian mixture fit to about 988K moving DROID frames. It keeps the posture a human teleoperator would most likely have used.

2 · Motion style

Path Optimization

cost Cplan(ξ) + λpath ctraj(ξ)

A self-supervised trajectory encoder maps positions, velocities, accelerations, and jerks to a 16-D latent. The squared Mahalanobis distance from the DROID latent density is added to cuRobo's trajectory optimization, and its gradients pass back through the frozen encoder to the waypoints.

3 · Timing

Retiming

cost ctraj(ξθ,τ) + λb cboundary

With the path geometry fixed, DATAFARM optimizes the speed profile over 64 arc-length intervals and the total duration. A boundary penalty slows the arm near grasps and releases, as human operators do. Candidates that exceed robot limits are rejected.

2 · Path optimization
3 · Retiming

The frozen trajectory encoder embeds each candidate trajectory. Path optimization reshapes the waypoints, and retiming changes the progress-along-path profile, until the embedding moves into the pretraining density.

Tasks

A DROID platform (Franka Panda + Robotiq gripper). For each target task, we fine-tune π0.5-DROID on 20 successful demonstrations and evaluate it on 20 fresh trials.

Initial and final scenes for Geometric Constraints, Multi-Step Reasoning, Semantic Reasoning, and Deformable Object Manipulation.
  • Geometric Constraints targetPlace three toys on a plate without collisions. Placing an early toy in the center can block the later ones.
  • Multi-Step Reasoning targetRemove a banana obstructing a tray, then pack two toys onto the tray.
  • Semantic Reasoning targetSort three objects into two bowls by semantic category (fruit vs. toys).
  • Deformable Object Manipulation out-of-distributionFold a cloth. The planner cannot do this and the fine-tuning data never include it. It tests whether the pretrained capability survives fine-tuning.

Raw TAMP vs. DATAFARM demonstrations

Both planners solve every task. What differs is how the arm moves: DATAFARM chooses arm postures, motion style, and timing that resemble the teleoperated data the VLA was pretrained on.

Raw TAMP
Geometric Constraints
Multi-Step Reasoning
Semantic Reasoning
DATAFARM
Geometric Constraints
Multi-Step Reasoning
Semantic Reasoning

Autonomous executions used as fine-tuning data. Videos are sped up by the factor shown in each corner.

Results

A task-valid trajectory is not necessarily good fine-tuning data. Every raw TAMP demonstration solves its task, yet fine-tuning on them barely helps. Aligning the planner with the pretraining distribution closes most of the gap to human teleoperation.

Target-task performance by demonstration source
Show full results table
ApproachGeometric ConstraintsMulti-Step ReasoningSemantic Reasoning
Succ.Prog.OOD Succ.OOD Prog.Succ.Prog.OOD Succ.OOD Prog.Succ.Prog.OOD Succ.OOD Prog.
π0.5-DROID0%18%90%95%0%13%90%95%5%50%90%95%
Raw TAMP5%35%85%93%5%37%70%85%15%58%75%88%
DATAFARM w/o style10%39%––0%31%––35%54%––
DATAFARM w/o timing0%19%––0%18%––0%29%––
DATAFARM w/o joint space35%73%––15%53%––20%66%––
DATAFARM65%84%80%90%35%81%85%88%70%89%90%93%
Human teleop (oracle)50%93%95%98%45%90%95%98%90%98%95%98%

Success rate and task progress (fraction of 7 binary subgoals completed) over 20 trials per checkpoint. OOD columns evaluate each task-specific checkpoint on Deformable Object Manipulation. Human teleoperation is an oracle only in its access to human target-task demonstrations; it is not assumed to upper-bound autonomous methods.

Raw TAMP barely helps. Fine-tuning on successful TiPToP executions raises average success from 1.7% to only 8.3%.
DATAFARM improves every task: 65% / 35% / 70% success vs. 5% / 5% / 15% for raw TAMP, and 84.7% average progress.
Beats teleop on Geometric Constraints (65% vs. 50%) and trails it on Multi-Step (35% vs. 45%) and Semantic Reasoning (70% vs. 90%).
No human in the control loop. DATAFARM plans and executes every demonstration autonomously. Humans only set up and reset scenes.

Policy rollouts after fine-tuning on DATAFARM

π0.5-DROID fine-tuned on DATAFARM demonstrations, running closed-loop on the robot.

Target tasks
Geometric Constraints
Multi-Step Reasoning
Semantic Reasoning
Deformable Object Manipulation out-of-distribution
Before fine-tuning (π0.5-DROID)
After fine-tuning on DATAFARM

The fine-tuning data contain no folding demonstrations, yet the fine-tuned policy still folds the cloth. Videos are sped up by the factor shown in each corner.

Every alignment component matters

Each ablation removes one component and keeps the other two. Removing timing alignment is catastrophic: the policy fails every trial on all three tasks.

Ablations
Cloth folding after fine-tuning (OOD)

Success on Deformable Object Manipulation after fine-tuning on each target task. The dashed line marks the pretrained model (90%). DATAFARM keeps 85% on average, while raw TAMP falls to 76.7%.

Scaling the demonstration budget

Geometric Constraints, fine-tuned on 0–80 DATAFARM demonstrations. Zero demonstrations is the pretrained π0.5-DROID. With 80 demonstrations, success reaches 85% and progress 97%.

Distribution Alignment

Projecting samples onto the first two principal components shows DATAFARM demonstrations moving inside the DROID distribution, both in joint space and in the learned trajectory latent space.

PCA density contours. Joint space: 74% of DATAFARM samples inside the DROID contour vs. 66% raw TAMP and 60% without joint-space alignment. Latent space: 96% DATAFARM vs. 0% raw TAMP, 89% without style, 19% without timing.

Joint space. 74% of DATAFARM configurations lie inside the displayed DROID contour, compared with 66% for raw TAMP and 60% without joint-space alignment.

Trajectory latent space. 96% of DATAFARM trajectory embeddings lie inside the DROID contour. Raw TAMP trajectories land in an entirely separate region (0%), and removing timing alignment drops overlap to 19%.

Overlap is not the whole story. The style ablation keeps 89% latent overlap but reaches only 15% success. High overlap in one projection alone does not guarantee task success, and each component helps when added to the other two.

Each annotation reports the fraction of a source's projected samples inside the outer DROID contour. These projections diagnose overlap in the displayed representations; they do not measure equality with the full pretraining distribution.

BibTeX

@article{datafarm2026,
  title   = {DATAFARM: Distribution-Aligned Task and Motion Planning
             for Fine-Tuning Vision-Language-Action Models},
  author  = {Anonymous Authors},
  year    = {2026},
  note    = {Under review}
}