We introduce the cross-cavity quantum Rabi model describing the interaction of a single two-level system with two orthogonal boson fields and propose its quantum simulation by two-dimensional, bichromatic, first-sideband driving of a single trapped ion. We provide an introductory survey of the model, including its diagonalization in the two-level system basis, numerical spectra and its characteristics in the weak, ultra strong and deep-strong coupling regimes. We also show that the particular case of identical field frequencies and couplings allows us to cast the model as two parity deformed oscillators in any given coupling regime.