Permutation groups with restricted stabilizers

Abstract

Fix a positive integer $d$ and let $\Gamma_d$ be the class of finite groups without sections isomorphic to the alternating group $A_d$. The groups in $\Gamma_d$ were studied by Babai, Cameron and P\'{a}lfy in the 1980s and they determined bounds on the order of a primitive permutation group with this property, which have found a wide range of applications. Subsequently, results on the base sizes of such groups were also obtained. In this paper we replace the structural conditions on the group by restrictions on its point stabilizers, and we obtain similar, and sometimes stronger conclusions. For example, we prove that there is a linear function $f$ such that the base size of any finite primitive group with point stabilizers in $\Gamma_d$ is at most $f(d)$. This generalizes a recent result of the first author on primitive groups with solvable point stabilizers. For non-affine primitive groups we obtain stronger results, assuming only that stabilizers of $c$ points lie in $\Gamma_d$. We also show that if $G$ is any permutation group of degree $n$ whose $c$-point stabilizers lie in $\Gamma_d$, then $|G| \leqslant ((1+o_c(1))d/e)^{n-1}$. This asymptotically extends and improves a $d^{n-1}$ upper bound on $|G|$ obtained by Babai, Cameron and P\'{a}lfy assuming $G \in \Gamma_d$.