This work deals with the acceleration of the transport sweep for a discrete ordinates formulation of the neutron transport equation. In particular, we focus on accelerating a matrix-free version of the algorithm. The algorithmic complexity of a naive version of the transport sweep is studied, and a preprocessing technique based on a QZ-decomposion of the matrices composing the sweep is proposed. This technique is not purely matrix free, but we will see that the memory requirements for this approach is negligible in comparison with a full inversion of the streaming operator. The proposed technique is tailored for high order Finite Element Method on regular Cartesian meshes, where the local problems are dense matrices of small size, but with minor modification it can be extended to regular triangular meshes and for non-matching grids with isotropic refinements. The extension to more general geometries requires simultaneous triangularization of more than two matrices at once. The performance of the proposed acceleration is analysed in algorithmic complexity with respect to the number of iterations, and then it is used to predict the required CPU-time to solve the problem. These predictions are tested with at 2D pin-by-pin problem consisting of four assemblies, and the predictions show good agreement with the measured CPU-time for a range of polynomial degrees.