During asymmetric division, fate determinants at the cell cortex segregate unequally into the two daughter cells. It has recently been shown that, not in the cortex, but in the cytoplasm, Sara signaling endosomes also segregate asymmetrically during asymmetric division. Biased dispatch of Sara endosomes mediate asymmetric Notch/Delta signaling during the asymmetric division of sensory organ precursors (SOP) in Drosophila. This has been generalized to stem cells in the gut and CNS of flies and to asymmetric division of neural precursors in the zebrafish spinal cord. However, the mechanism of asymmetric endosome motility has not yet been elucidated. Here, we unraveled this mechanism: the microtubule depolimerizing kinesin Klp10A and its antagonist Patronin generate spindle asymmetry. The asymmetric spindle, in turn, polarizes the dynamic properties of endosome motility. To prove our model, we inverted the polarity of the spindle by polar targeting of Patronin using nanobodies. Spindle inversion targets the endosomes to the wrong cell. Our data uncovers the molecular and physical mechanism by which organelles localized away from the cellular cortex can be dispatched asymmetrically during asymmetric division.