Internship at Magna International, Advanced Robotics R&D

Dual-arm assembly R&D: from CAD to a learned insert

Research and development internship at Magna. A dual-arm pipeline that plans a full assembly from CAD in 59 seconds, then hands the final insertion to a learned policy.

3 min read

Shipped
Two six-axis arms on a shared column above a work table, each fitted with a gripper

Abstract

Research and development internship at Magna Advanced Robotics. I built a dual-arm pipeline that takes CAD models and plans a full robotic assembly in 59 seconds. Classical geometric planning handles the large motions. It fails on the final millimeter of insertion, where the parts physically interfere. A learned policy takes over for that last step, inside the same execution pipeline.

Introduction

Geometric path planning solves how a robot moves through free space. It completely fails when parts actually touch. Three observations drove this project:

  • Physics matters. A trajectory that passes every collision check can still jam under physical grasping constraints because the planner lacks a contact model.
  • Micro-tolerances. One part could not be inserted at all using classical planning. A residual tilt of a few degrees was enough to bind it.
  • Interference fits. Another component seats through real physical interference. This is a corridor narrower than the inherent positional scatter of any AI policy trying to learn it.

The architecture is built around this boundary: everything before contact stays classical, and machine learning is only spent on the final insertion.

The pipeline

A geometric chain from CAD to physics replay, which records a dataset, which trains a learned insert that re-enters the plan as an operator. GEOMETRIC, DETERMINISTICCADpart modelsAssembly planextraction pathsTask plan31 segmentsMotion plancuRobo, 12-DOFPhysics replayPhysX checkWHERE GEOMETRY STOPSLEARNING IS SPENT HEREOnly on the last few centimetresof each insertion.Learned insertRL, VLA, residualDatasetLeRobot, 25 Hzre-enters as an operator
Figure 2. One chain with a closing edge. Everything above the line is geometric and deterministic.
  • Geometric planning. The assembly planner searches for extraction paths using pure geometry. Parts that bind under pure sliding need screw motions to come out.Rotating while sliding unbinds what pure sliding binds.
  • Task expansion. The task planner expands the high-level order of work into 31 distinct segments, statically validating them in under a second without running the physics simulator.
  • Data generation. The physics replay serves as both a plan checker and a dataset recorder. Every recorded demonstration is guaranteed to be an executable path.
  • Drop-in learning. A trained policy is not a separate script. It runs as a standard pipeline operator, meaning a single config change swaps a classical insertion for a learned one.

Planning results

Metric Performance
Segments planned 31 of 31
Full assembly time 59.24 s (1.31 s variance across runs)
Median segment time 1.32 s
Speedup 4.4x faster than the previous engine
  • Coordinated movement. The dual-arm carry is the most expensive operation, taking up 24% of the time despite being less than 10% of the plan.
  • Cost drivers. Execution time is driven by physical path length, not the planning mode.
  • Seamless integration. Routing the final insertion through a learned policy cost no more execution time than the classical equivalent.

Learned skills results

Instead of human teleoperation, a privileged expert script combined with the planner generated exactly 150 successful demonstrations (mean seat error 1.005 mm) under full randomization.

  • Reinforcement learning. Achieved a 60% success rate at 9.7 N median peak force, beating a scripted baseline (71% success, but a massive 47 N peak force). Success depended heavily on curating the demonstrations, not the algorithm.
  • Vision-Language-Action (VLA). Closed-loop vision successfully seated parts where open-loop movements failed (3 of 5 success rate). However, when retrained with real physical contact data, success dropped to zero because the policy's positional scatter was wider than the physical clearance corridor.
  • Residual models. 11 of 11 parts seated perfectly when the exact mathematical correction was explicitly supplied. But when left to learn the correction across four models, it achieved 0 of 436. Proprioception alone cannot "see" the tiny errors the residual is meant to fix.
Bar charts comparing seat success for open-loop and closed-loop policies across randomization scales, and against a privileged expert
Figure 3. What closed-loop vision buys, and where retraining on real contact does not.

Limits and next steps

  • Hardware bottleneck. Training throughput is severely capped by having only one GPU and one simulation environment.
  • Memory constraints. The residual model desperately needs visual input to succeed, but the frozen base model and the simulator leave no VRAM left for a vision encoder.
  • Sim-to-real gap. The force-feedback teleoperation track remains an analysis project. Nothing has been deployed to physical hardware yet.