Inspired by intriguing dynamics of magnetotactic bacteria, we present a minimal kinetic model for magnetic swimmers in an external magnetic field to investigate their collective behavior. Our kinetic model couples a Fokker-Planck equation for active particles in an external magnetic field to the Stokes flow. Combining linear stability analysis and nonlinear 3D continuum simulations, we investigate the hydrodynamic stability and transport of magnetic swimmers as a function of activity and magnetic field strengths. We show that at sufficiently high activity and moderate magnetic field strengths, a homogeneous polar steady state is unstable and distinct types of splay and bend instabilities for puller and pusher swimmers emerge. Pushers form wave-like structures perpendicular to the field while pullers form wave-like lanes along the field. These instabilities arise from the amplification of anisotropic hydrodynamic interactions in the external alignment and lead to a partial depolarization and a reduction of the average transport speed of the swimmers in the field direction. Interestingly, at higher field strengths the homogeneous polar state becomes stable and a transport efficiency identical to that of active particles without hydrodynamic interactions is restored. We discuss our results in relation to the experimental findings.