Sato-Tate distributions and Galois endomorphism modules in genus 2

Abstract

For an abelian surface A over a number field k, we study the limiting distribution of the normalized Euler factors of the L-function of A. This distribution is expected to correspond to taking characteristic polynomials of a uniform random matrix in some closed subgroup of USp(4); this Sato-Tate group may be obtained from the Galois action on any Tate module of A. We show that the Sato-Tate group is limited to a particular list of 55 groups up to conjugacy. We then classify A according to the Galois module structure on the R-algebra generated by endomorphisms of A_Qbar (the Galois type), and establish a matching with the classification of Sato-Tate groups; this shows that there are at most 52 groups up to conjugacy which occur as Sato-Tate groups for suitable A and k, of which 34 can occur for k = Q. Finally, we exhibit examples of Jacobians of hyperelliptic curves exhibiting each Galois type (over Q whenever possible), and observe numerical agreement with the expected Sato-Tate distribution by comparing moment statistics.