Phagocytosis of bacteria and other pathogens by macrophages is a key process of the mammalian immune system. The intracellular maturation of phagosomes, which often leads to the degradation of the internalized pathogens, shows high organelle-to-organelle variations that are not clearly understood. An important part of the maturation is the phagosomal transport from the cell periphery towards the perinuclear region. We hypothesize that the phagosome size influences the phagosomal transport and therefore also potentially the maturation process. We tested this hypothesis by tracking phagosomes with different diameters between 1 μm and 3 μm inside macrophages. We show that the transport efficiency increases with increasing phagosome size although the instantaneous velocities of the investigated phagosomes are very similar to each other. In addition, we found that the share of bi-directional motion as well as the transport from the nucleus back to the periphery decreases with increasing phagosome size. We show that dynein is significantly involved in the phagosomal transport, in particular in the persistent centripetal transport of large phagosomes. In addition, actin-dependent motion is also contributing to the transport, in particular to the transport of small phagosomes. Furthermore we investigated the spatial distribution of dyneins and microtubules, and found that density differences between the nucleus-facing side of phagosomes and the opposite side can explain part of the observed transport characteristics. Our findings suggest that a basic size-dependent cellular sorting mechanism might exist that supports inward transport of large phagocytosed pathogens for facilitating their digestion and that simultaneously supports outward transport of small pathogen fragments for example for antigen presentation.