Drive My Way : Preference Alignment of Vision-Language-Action Model for Personalized Driving

Human driving behavior is inherently personal and continuously evolving. Drivers exhibit stable, long-term habits while also adjusting their behavior in the moment based on short-term intent and situational context.

What are the limitations of current systems? Most end-to-end autonomous driving systems optimize a generic objective (e.g., safety/efficiency), effectively targeting an “average driver.” Some provide a small set of predefined modes (e.g., sport, comfort, eco), but these coarse presets cannot represent the nuanced, consistent differences in individual driving styles. In addition, they are not designed to interpret and follow natural-language intent specified at runtime.

Why previous personalized driving systems fall short. Existing personalized-focused driving works mainly fall into two categories. First, data-driven approaches extract predefined driving styles from human demonstrations and learn style-conditioned policies. These methods rely on a fixed and limited set of driving styles and scale poorly to a growing population of users with diverse preferences. Second, language-driven approaches leverage large language models (LLMs) for instruction-based personalization, typically requiring learning from explicit human driver feedback. Such approaches introduce interaction costs and remain constrained to simple, low-interaction settings.

Core idea: DMW aligns the VLA policy with long-term driving patterns and real-time preference instructions to produce behaviors that are safe and effective, yet also recognizable and adaptable.

PDD collects real human driving demonstrations across diverse scenarios in CARLA with a steering wheel setup. Each driver is associated with structured profile information.

Download PDD (Coming Soon)

Given the camera observations and navigation goals, the model fuses with the driver's long-term preferences and user instructions to produce adaptive, personalized actions.

Long-term Preference Encoder

The preference encoder learns user embeddings from profile. It aligns structured driver profiles with historical driving behavior using a contrastive objective, ensuring that the learned embedding reflects behaviorally patterns and maintains diversity among drivers. The learned embeddings are used to condition the policy during planning.

GRPO-based Preference Alignment

DMW employs Group Relative Policy Optimization (GRPO) with style-aware rewards to align the policy. The residual decoder encourages the policy to sample diverse yet reasonable driving behaviors. The reward adaptation bridges the gap between subtle preferences in language and dynamic style rewards.

The substantially larger metric changes observed in DMW under varying instructional styles, relative to the baseline, highlight DMW's capability for real-time preference adaptation.

When conditioned on a given profile, DMW exhibits consistent motion patterns across different scenarios and instruction styles. Style instructions further provide short-term adaptation, shifting driving behavior toward the intended preference.