In principle, we could obtain results in situations where the conductor is unbounded and the twists are restricted to special oneparameter families. However, in that case we need rather stronger monodromy results, and those do not seem available (on the other hand, they are certainly within the realm of reason), even in special cases. Still, we describe what could be true in the last section of this paper.
We should also mention that the results of Katz are in fact much more general, and the method used here should adapt easily. We restrict our attention to the case of quadratic twists of elliptic curves partly for concreteness and partly in the hope of providing a reasonably readable introduction to those remarkable results for readers with an analytic number theory background.
Notation. As usual,
$\leftX\right$
denotes the cardinality of a set. By
$f\ll g$
for
$x\in X$
, or
$f=O\left(g\right)$
for
$x\in X$
, where
$X$
is an arbitrary set on which
$f$
is defined, we mean synonymously that there exists a constant
$C\u2a7e0$
such that
$\leftf\right(x\left)\right\u2a7dCg\left(x\right)$
for all
$x\in X$
. The “implied constant” is any admissible value of
$C$
. It may depend on the set
$X$
which is always specified or clear in context.
Acknowledgments. Work on this paper was prompted by discussions by J. Keating. I wish also to thank N. Katz for explaining a number of points concerning the behavior of monodromy for sheaves with orthogonal symmetry.
1 A uniform version of the Chebotarev density theorem
In this section we prove a general uniform Chebotarev density theorem for varieties over finite fields. The main tools are the cohomological methods and results developped notably by Grothendieck and Deligne; a short and fairly concrete survey aimed at analytic number theorists, which should be sufficient to explain the terminology and the proofs below, can be found in [
IK,11.11]
.
We consider the following data:
$U/{\mathbf{F}}_{q}$
is a smooth affine variety, absolutely irreducible and of dimension
$d\u2a7e1$
over a finite field of characteristic
$p$
with
$q$
elements,
$\ell \ne p$
is a prime number, and
$\rho :{\pi}_{1}(U,\overline{\eta})\to G$
is a surjective map from the arithmetic fundamental group of
$U$
(relative to the geometric generic point of
$U$
) to a finite group
$G$
. We denote by
$${G}^{g}=\rho \left({\pi}_{1}\right(\overline{U},\overline{\eta}\left)\right)\subset G$$
the image of the geometric fundamental group of
$U$
, where
$\overline{U}=U\times {\overline{\mathbf{F}}}_{q}$
. Recall there are exact sequences
$$\begin{array}{c}1\to {\pi}_{1}(\overline{U},\overline{\eta})\to {\pi}_{1}(U,\overline{\eta})\to d\hat{\mathbf{Z}}\to 1\downarrow \downarrow \phi \downarrow 1\to {G}^{g}\to G\to m\Gamma \to 1,\end{array}$$ 
(2)

where the quotient
$\Gamma $
thus defined is a finite cyclic group.
For any
$u\in U\left({\mathbf{F}}_{q}\right)$
, we denote by
${Fr}_{u}$
the geometric Frobenius conjugacy class at
$u$
in
${\pi}_{1}(U,\overline{\eta})$
. In other words, corresponding to the inclusion
$\left\{u\right\}=Spec\left({\mathbf{F}}_{q}\right)\to U$
, we have an induced homomorphism
$${\pi}_{1}\left({\mathbf{F}}_{q}\right)=Gal({\overline{\mathbf{F}}}_{q}/{\mathbf{F}}_{q})\to {\pi}_{1}(U,\overline{\eta})$$
and
${Fr}_{u}$
is the image, welldefined up to conjugation, of the inverse of the generator
$x\mapsto {x}^{q}$
of
$Gal({\overline{\mathbf{F}}}_{q}/{\mathbf{F}}_{q})$
. (If we consider
$u$
as defined over a bigger field, then
${Fr}_{u}$
changes, so
${Fr}_{u}$
is defined relative to the field
${\mathbf{F}}_{q}$
; it is often denoted
${Fr}_{u,{\mathbf{F}}_{q}}$
for this reason, but we consider the base field as fixed in our statements). In the exact sequence above, we have
$$d\left({Fr}_{u,{q}^{n}}\right)=n\in \hat{\mathbf{Z}}.$$
The uniform Chebotarev density theorem is the following:
Theorem 1.
With notation as above, let
$C\subset G$
be a conjugacyinvariant subset such that
$m\left(x\right)=\phi (1)$
for all
$x\in C$
. Put
$$\pi (C;q)=\left\right\{u\in U\left({\mathbf{F}}_{q}\right)\left\rho \right({Fr}_{u})\in C\}.$$
Assume that
${G}^{g}$
is of order prime to
$p$
. Then we have
$$\begin{array}{c}\pi (C;q)=\frac{\leftC\right}{\left{G}^{g}\right}\leftU\right({\mathbf{F}}_{q}\left)\right+O\left({q}^{d1/2}\rightG{}^{3/2}\leftC{}^{1/2}\right),\end{array}$$ 
(3)

the implied constant depending only on
$\overline{U}=U\times {\overline{\mathbf{F}}}_{q}$
. In particular, this holds uniformly with
$q$
replaced by
${q}^{n}$
and
$U$
by
$U\times {\mathbf{F}}_{{q}^{n}}$
,
$n\u2a7e1$
.

Proof.
This is essentially the same as the statement in e.g. [C,Th.4.1] , except that we have to take care of the uniformity. Let
$f$
denote the characteristic function of
$C$
and let
$$f\left(g\right)={\sum}_{\pi}\alpha \left(\pi \right)Tr\pi \left(g\right)$$
be its Fourier expansion in terms of the irreducible representations
$\pi $
of
$G$
, which we realize as homomorphisms
$$\pi :G\to GL(deg(\pi ),E)$$
for some finite extension
$E/{\mathbf{Q}}_{\ell}$
, which can be chosen independent of
$\pi $
. If
$\pi =\psi $
is a character of
$\Gamma $
, i.e.,
$\pi $
is trivial on
${G}^{g}$
, we have
$Tr\pi \left(\rho \right({Fr}_{u}\left)\right)=\psi \left(\phi \right(1\left)\right)$
by the assumption on
$C$
, and
$${\sum}_{\psi}\alpha \left(\psi \right)Tr\pi \left(\rho \right({Fr}_{u}\left)\right)=\psi \left(\phi \right(1\left)\right){\sum}_{\psi}\frac{1}{\leftG\right}{\sum}_{x\in C}\psi \left(x\right)=\frac{\leftC\right}{\left{G}^{g}\right}.$$
Now applying the Fourier expansion to
$f\left(\rho \right({Fr}_{u}\left)\right)$
we find therefore
$$\pi (C;q)=\frac{\leftC\right}{\left{G}^{g}\right}\leftU\right({\mathbf{F}}_{q}\left)\right+{\sum}_{\pi {G}^{g}\ne 1}\alpha \left(\pi \right){\sum}_{u\in U\left({\mathbf{F}}_{q}\right)}Tr\pi \left({\rho}_{\ell}\right({Fr}_{u}\left)\right).$$
The inner sum is the sum of local traces for the representation
$$\pi \circ \rho :{\pi}_{1}(U,\overline{\eta})\to GL(deg(\pi ),E),$$
which can be seen as a lisse
${\overline{\mathbf{Q}}}_{\ell}$
adic sheaf, denoted
$\pi \left(\rho \right)$
. Since the image of
$\rho $
, hence of
$\pi \left(\rho \right)$
, is finite, this sheaf is pointwise pure of weight
$0$
. By the GrothendieckLefschetz trace formula we have
$${\sum}_{u\in U\left({\mathbf{F}}_{q}\right)}Tr\pi \left(\rho \right({Fr}_{u}\left)\right)={\sum}_{i=0}^{2d}(1{)}^{i}Tr(Fr\left{H}_{c}^{i}\right(\overline{U},\pi \left(\rho \right)\left)\right),$$
where
$\overline{U}=U\times {\overline{\mathbf{F}}}_{q}$
.
In terms of the geometric fundamental group
${\pi}_{1}(\overline{U},\overline{\eta})$
, the coinvariant description of
${H}_{c}^{2d}$
gives
$${H}_{c}^{2d}(\overline{U},\pi (\rho \left)\right)={E}_{{\pi}_{1}(\overline{U},\overline{\eta})}^{deg\left(\pi \right)}(d)={E}_{{G}^{g}}^{deg\left(\pi \right)}(d)=0.$$
since
$\pi $
, being nontrivial when restricted to
${G}^{g}$
, can not contain the trivial representation, simply because the space of invariants under
${G}^{g}$
is a subrepresentation of
$G$
, which is irreducible.
Moreover, by Deligne's Theorem, the eigenvalues of the geometric Frobenius
$Fr$
acting on each
${H}_{c}^{i}(\overline{U},\pi (\rho \left)\right)$
are algebraic integers with absolute value in
$\mathbf{C}$
of modulus
$\u2a7d{q}^{i/2}$
. Thus we find
$$\left{\sum}_{u\in U\left({\mathbf{F}}_{q}\right)}Tr\pi \left(\rho \right({Fr}_{u}\left)\right)\right\u2a7d{q}^{d1/2}{\sigma}_{c}^{\prime}(\overline{U},\pi (\rho \left)\right),$$
where
$${\sigma}_{c}^{\prime}(\overline{U},\pi (\rho \left)\right)={\sum}_{i<2d}dim{H}_{c}^{i}(\overline{U},\pi (\rho \left)\right).$$
It only remains to bound the quantity
$${\sum}_{\pi {G}^{g}\ne 1}\left\alpha \right(\pi \left)\right{\sigma}_{c}^{\prime}(\overline{U},\pi (\rho \left)\right)$$
uniformly in terms of
$\pi $
and
$\rho $
. By Proposition 3.6 of [
Ko]
, using the assumption that
$\left{G}^{g}\right$
is prime to
$p$
, there exists a constant
$C\u2a7e0$
, depending only on
$\overline{U}$
, such that
$${\sigma}_{c}^{\prime}(\overline{U},\pi (\rho \left)\right)\u2a7dC\leftG\right(deg\pi )$$
for all
$\pi $
. Hence
$${\sum}_{\pi {G}^{g}\ne 1}{\sigma}_{c}^{\prime}(\overline{U},\pi (\rho \left)\right)\u2a7dC\leftG\right{\sum}_{\pi}\left\alpha \right(\pi \left)\rightdeg\pi $$
and by Cauchy's inequality and the standard properties of representations of finite groups we get
$${\sum}_{\pi}deg\pi \u2a7d{\left({\sum}_{\pi}\left\alpha \right(\pi ){}^{2}\right)}^{1/2}{\left({\sum}_{\pi}(deg\pi {)}^{2}\right)}^{1/2}=\sqrt{\leftC\right}\sqrt{\leftG\right}.$$
Putting these inequalities together yields the stated result. □
Remark 2.
Since
$$\leftU\right({\mathbf{F}}_{q}\left)\right={q}^{d}+O\left({q}^{d1/2}\right)$$
by the LangWeil estimate, the implied constant depending only on
$\overline{U}$
, we can also rephrase the result as
$$\pi (C;q)=\frac{\leftC\right}{\left{G}^{g}\right}{q}^{d}+O\left({q}^{d1/2}\rightG{}^{3/2}\leftC{}^{1/2}\right).$$
Here is a variant of this theorem when the variety
$U$
is a smooth affine curve and the map
$\rho $
arises by reduction from a torsionfree
${\mathbf{Z}}_{\ell}$
adic sheaf, using Proposition 3.1 of [
Ko]
(or indeed, since we assume tameness, the last part of Theorem 4.1 in [
C]
) instead of Proposition 3.6. In this case, the dependence on
$U$
can be made explicit (and the error term is improved) which allows certain interesting applications (see the last section).
Theorem 3.
Let
$U/{\mathbf{F}}_{q}$
be a smooth geometrically irreducible affine curve, realized as an open dense subset of a smooth projective curve
$C/{\mathbf{F}}_{q}$
of genus
$g$
, with
$m=\left\right(CU\left)\right({\overline{\mathbf{F}}}_{q}\left)\right$
“points at infinity”. Let
$\ell \ne p$
be a prime number, let
$\mathcal{\mathcal{F}}$
be a tame torsion free lisse
${\mathbf{Z}}_{\ell}$
adic sheaf of rank
$N$
, and let
$\overline{\mathcal{\mathcal{F}}}=\mathcal{\mathcal{F}}/\ell \mathcal{\mathcal{F}}$
be its reduction modulo
$\ell $
. Denote
$${\rho}_{\ell}:{\pi}_{1}(U,\overline{\eta})\to GL(N,{\mathbf{F}}_{\ell})$$
the corresponding continuous representation and put
$${G}_{\ell}={\rho}_{\ell}\left({\pi}_{1}\right(U,\overline{\eta}\left)\right),{G}_{\ell}^{g}={\rho}_{\ell}\left({\pi}_{1}\right(\overline{U},\overline{\eta}\left)\right),$$
let
${\Gamma}_{\ell}$
be the quotient and
$m$
,
$\phi $
as in ( 2 ) in this case.
Then for any conjugacy invariant subset
$C(\ell )\subset {G}_{\ell}$
such that
$m\left(C\right(\ell \left)\right)=\phi (1)$
,
$$\pi \left(C\right(\ell );q)=\frac{\leftC\right(\ell \left)\right}{\left{G}_{\ell}\right}\leftU\right({\mathbf{F}}_{q}\left)\right+O\left(\right(m+g\left){q}^{1/2}\right{G}_{\ell}{}^{1/2}\leftC{}^{1/2}\right),$$
where the implied constant is absolute.

Proof.
As above we find by Fourier expansion that
$$\pi \left(C\right(\ell );q)=\frac{\leftC\right(\ell \left)\right}{\left{G}_{\ell}\right}\leftU\right({\mathbf{F}}_{q}\left)\right+O({q}^{1/2}\mathfrak{S})$$
where
$$\mathfrak{S}={\sum}_{\pi {G}_{\ell}^{g}\ne 1}\left\alpha \right(\pi \left)\right{\sigma}_{c}^{\prime}(\overline{U},\pi ({\rho}_{\ell}\left)\right)$$
and the implied constant is absolute (in fact, can be taken to be equal to
$1$
).
By Proposition 3.1 of [
Ko]
(see eq. (3.1) and note that the term
$w$
there vanishes by the tameness assumption), we have
$${\sigma}_{c}^{\prime}(\overline{U},\pi ({\rho}_{\ell}\left)\right)\u2a7d(1{\chi}_{c}(\overline{U},{\mathbf{Q}}_{\ell}\left)\right)(deg\pi )$$
where
$${\chi}_{c}(\overline{U},{\mathbf{Q}}_{\ell})=dim{H}_{c}^{0}(\overline{U},{\mathbf{Q}}_{\ell})dim{H}_{c}^{1}(\overline{U},{\mathbf{Q}}_{\ell})+dim{H}_{c}^{2}(\overline{U},{\mathbf{Q}}_{\ell})$$
is the EulerPoincaré characteristic of
$\overline{U}$
. This is equal to
$22gm$
(because it is “additive”, so equal to
${\chi}_{c}\left(\overline{C}\right){\chi}_{c}(\overline{C}\overline{U})=22gm$
), hence
$${\sigma}_{c}^{\prime}(\overline{U},\pi ({\rho}_{\ell}\left)\right)\u2a7d(2g+m1)(deg\pi ).$$
Summing over
$\pi $
yields the estimate
$$\mathfrak{S}\u2a7d(m+2g1){\sum}_{\pi}\left\alpha \right(\pi \left)\rightdeg\pi \u2a7d(m+2g1)\sqrt{\left{G}_{\ell}\right\leftC\right},$$
hence the result claimed. □
Remark 4.
There are other uniform versions of the Chebotarev density theorem for curves, for instance [
FJ,Pr.5.16]
, which is written and proved in a style closer to the standard number field case.
2 Application to sheaves with orthogonal monodromy
We will now apply the uniform Chebotarev density theorem to reductions of a lisse
$\ell $
adic sheaf with “orthogonal” symmetry. The method turns out to be essentially identical with that used by Serre in his applications of the Chebotarev density theorem over number fields (see [
S1]
).
As in the previous section we start with a smooth affine absolutely irreducible variety of dimension
$d\u2a7e1$
defined over a finite field
${\mathbf{F}}_{q}$
of characteristic
$p$
. Let
$N\u2a7e1$
be an integer and
$\ell \ne p$
a prime number such that
$GL(N,{\mathbf{F}}_{\ell})$
has order prime to
$p$
. This also implies that for all
$\nu \u2a7e1$
, the group
$GL(N,\mathbf{Z}/{\ell}^{\nu}\mathbf{Z})$
has order prime to
$p$
.
Consider now a lisse integral torsion free
$\ell $
adic sheaf
${\mathcal{\mathcal{F}}}_{\ell}$
of rank
$N$
on
$U$
; equivalently, consider a continuous representation
$${\tau}_{\ell}:{\pi}_{1}(U,\overline{\eta})\to GL(N,{\mathbf{Z}}_{\ell}).$$
We assume that
${\mathcal{\mathcal{F}}}_{\ell}$
is equipped with a nondegenerate symmetric pairing
$$\langle \cdot ,\cdot \rangle :{\mathcal{\mathcal{F}}}_{\ell}\otimes {\mathcal{\mathcal{F}}}_{\ell}\to {\mathbf{Z}}_{\ell},$$
equivalently,
${\tau}_{\ell}$
acts on
${\mathbf{Z}}_{\ell}^{N}$
by transformations leaving invariant a nondegenerate symmetric pairing. We denote by
$O(N,{\mathbf{Z}}_{\ell})$
the whole group of transformations leaving this pairing invariant (which depends on the equivalence class of the pairing, but this will not be of any importance).
We make the following assumption of large monodromy on
${\tau}_{\ell}$
:
$$\begin{array}{c}\left[O\right(N,{\mathbf{Z}}_{\ell}):{\tau}_{\ell}({\pi}_{1}(U,\overline{\eta})\left)\right]<+\infty .\end{array}$$ 
(4)

For any
$\nu \u2a7e1$
we can consider the reduction modulo
${\ell}^{\nu}$
of
${\mathcal{\mathcal{F}}}_{\ell}$
, i.e.
${\overline{\mathcal{\mathcal{F}}}}_{\nu}={\mathcal{\mathcal{F}}}_{\ell}/{\ell}^{\nu}{\mathcal{\mathcal{F}}}_{\ell}$
, which corresponds to the maps
$${\rho}_{\nu}:{\pi}_{1}(U,\overline{\eta})\to GL(N,\mathbf{Z}/{\ell}^{\nu}\mathbf{Z}),$$
and we put
$${G}_{\nu}={\rho}_{\nu}\left({\pi}_{1}\right(U,\overline{\eta}\left)\right).$$
The assumption ( 4 ) ensures that the groups
${G}_{\nu}$
are also “large” when
$\nu $
is large enough : the index
$$\begin{array}{c}\left[O\right(N,\mathbf{Z}/{\ell}^{\nu}\mathbf{Z}):{G}_{\nu}]\end{array}$$ 
(5)

is bounded for
$\nu \u2a7e1$
(the reductions of any finite set of coset representatives for
${\tau}_{\ell}\left({\pi}_{1}\right(U,\overline{\eta}\left)\right)$
in
$O(N,{\mathbf{Z}}_{\ell})$
give coset representatives for
${G}_{\nu}$
in
$O(N,\mathbf{Z}/{\ell}^{\nu}\mathbf{Z})$
).
We will apply the Chebotarev density theorem to sets
${C}_{\nu}\subset {G}_{\nu}$
defined by reference with the forced eigenvalues that exist for some orthogonal matrices. Precisely, recall that for
$A\in O(N,k)$
(for an arbitrary field
$k$
of odd characteristic and an arbitrary nondegenerate symmetric bilinear form on
${k}^{N}$
), the following “functional equation”
$$\begin{array}{c}{T}^{N}P(1/T)=det(A)P\left(T\right)\end{array}$$ 
(6)

holds for the polynomial
$P=det(1AT)\in k\left[T\right]$
. At
$T=1$
this implies that if
$det(A)=(1{)}^{N}detA=1$
, we have
$P\left(1\right)=0$
, i.e.,
$1$
is then an eigenvalue of
$A$
.
If
$N$
is even, that means that any matrix
$A\in O(N,k)$
with determinant
$1$
has eigenvalue
$1$
.
If
$N$
is odd, that means that any orthogonal matrix in
$SO(N,k)$
has eigenvalue
$1$
. We will say that
$A$
has no extra vanishing if
$1$
is an eigenvalue of
$A$
with minimal multiplicity compatible with this. We denote by
${O}^{ev}\left(N\right)$
the set of orthogonal matrices which have extra vanishing.We thus see that
${O}^{ev}\left(N\right)$
is an algebraic variety defined over the same base as the orthogonal group under consideration, given by
${O}^{ev}\left(N\right)={O}_{1}\cup {O}_{2}$
, where
$${O}_{1}=\{A\in O(N\left)\rightdet\left(A\right)=1\text{and}det(1A)=0\}{O}_{2}=\{A\in SO\left(N\right)\frac{d}{dT}det(1TA){}_{T=1}=0\}$$
if
$N$
is odd and
$${O}_{1}=\{A\in SO(N\left)\rightdet(1A)=0\}{O}_{2}=\{A\in O\left(N\right)det(A)=1\text{and}\frac{d}{dT}det(1TA){}_{T=1}=0\}$$
if
$N$
is even.
We see that
${O}^{ev}\left(N\right)$
intersects each of the two connected components of
$O\left(N\right)$
in a closed hypersurface.
Denote by
${C}_{\nu}\subset {G}_{\nu}$
the image of
$${O}^{ev}(N,{\mathbf{Z}}_{\ell})\cap {\tau}_{\ell}\left({\pi}_{1}\right(U,\overline{\eta}\left)\right)$$
by reduction modulo
${\ell}^{\nu}$
. We have the following simple lemma:
Lemma 5.
(1) We have
$$\begin{array}{c}\left{G}_{\nu}\right\u2a7d{\ell}^{\nu N(N1)/2},\end{array}$$ 
(7)

for
$\nu \u2a7e1$
, the implied constant depending on
$N$
and
$\ell $
.
(2) We have
$$\begin{array}{c}\left{C}_{\nu}\right{G}_{\nu}{}^{1}\ll {\ell}^{\nu}\end{array}$$ 
(8)

for all
$\nu \u2a7e1$
, the implied constant depending on
$N$
,
$\ell $
and the index ( 4 ).

Proof.
(1) The order of
${G}_{\nu}$
is at most the order of
$O(N,\mathbf{Z}/{\ell}^{\nu}\mathbf{Z})$
so the upper bound follows easily.
(2) Because each
${O}^{ev}\left(N\right)$
is a hypersurface (i.e., of codimension
$1$
in each component of
$O\left(N\right)$
), the result follows from the bounds
$$\left{C}_{\nu}\right\ll {\ell}^{\nu (dimO(N)1)},\left{G}_{\nu}\right\gg {\ell}^{\nu dimO\left(N\right)},$$
the first being a consequence of e.g. [
S1,Th.8]
and the second of the boundedness of the indices ( 5 ) i.e., the finiteness of ( 4 ). (The upper bound can also be proved by more or less direct counting). □
We now make the following observation: for a point
$u\in U\left({\mathbf{F}}_{q}\right)$
, the condition that
${Fr}_{u}$
has extra vanishing acting on
${\mathcal{\mathcal{F}}}_{\ell}$
implies that
${\rho}_{\nu}\left({Fr}_{u}\right)\in {C}_{\nu}$
for all
$\nu \u2a7e1$
. This leads to the basic bound
$$\begin{array}{c}\left\right\{u\in U\left({\mathbf{F}}_{q}\right)\left{Fr}_{u}\text{has extra vanishing on}{\mathcal{\mathcal{F}}}_{\ell}\right\}\u2a7d\pi ({C}_{\nu};q)\end{array}$$ 
(9)

with notation as in Theorem 1 , which is valid for all
$\nu $
.
We will use this to prove:
Proposition 6.
With notation and assumptions as above, in particular under the monodromy assumption ( 4 ), we have
$$\left\right\{u\in U\left({\mathbf{F}}_{q}\right)\left{Fr}_{u}\text{has extra vanishing on}{\mathcal{\mathcal{F}}}_{\ell}\right\}\ll {q}^{dc}$$
where
$c=\frac{1}{2{N}^{2}2N+1}$
, the implied constant depending on
$\overline{U}$
,
$\ell $
,
$N$
and the index ( 4 ).
For instance, the gain is
$c=1/13$
for
$N=3$
,
$c=1/25$
for
$N=4$
.

Proof of Proposition 6 .
Applying ( 9 ) and Theorem 1 , we derive
$$\left\right\{u\in U\left({\mathbf{F}}_{q}\right)\left{Fr}_{u}\text{has extra vanishing}\right\}\u2a7d\frac{\left{C}_{\nu}\right}{\left{G}_{\nu}\right}{q}^{d}+O({q}^{d1/2}\left{G}_{\nu}{}^{3/2}\right{C}_{\nu}{}^{1/2})$$
for any
$\nu \u2a7e1$
, with an implied constant depending only on
$\overline{U}$
.
Using the bounds from the lemma, this leads to
$$\left\right\{u\in U\left({\mathbf{F}}_{q}\right)\left{Fr}_{u}\text{has extra vanishing}\right\}\ll {q}^{d}{\ell}^{\nu}+{q}^{d1/2}{\ell}^{\nu \left(N\right(N1)1/2)},$$
for
$\nu \u2a7e1$
with an implied constant depending
$\overline{U}$
,
$\ell $
,
$N$
and the index ( 4 ). For
$q$
large enough there exists
$\nu $
such that
$$\frac{1}{\ell}{q}^{c}\u2a7d{\ell}^{\nu}\u2a7d{q}^{c}\text{, with}c=\frac{1}{2{N}^{2}2N+1},$$
and then we have
$${\ell}^{\nu}\u2a7d\ell {q}^{c},\text{and}{q}^{1/2}{\ell}^{\nu \left(N\right(N1)1/2)}\u2a7d{q}^{c}$$
so that taking this
$\nu $
yields
$$\left\right\{u\in U\left({\mathbf{F}}_{q}\right)\left{Fr}_{u}\text{has extra vanishing}\right\}\ll {q}^{dc},$$
the implied constant depending only on the parameters indicated (the constant may need to be increased to account for small values of
$q$
where the
$\ell $
above can not be found). □
Remark 7.
One could also approach the same problem using the large sieve method of [
Ko]
.
However, because the “sieving” here is to detect an algebraic condition, it turned out to be just as efficient using the Chebotarev density theorem, and in fact using it for fixed
$\ell $
as above instead of requiring monodromy assumptions for many
$\ell $
. The large sieve would give a very slight improvement (roughly corresponding to replacing
$\leftC\right$
in ( 3 ) by the number of conjugacy classes in
$G$
), but we avoid the necessary modification necessary to deal with the fact that the orthogonal groups do not satisfy the linear disjointness condition of loc. cit.
We conclude by giving an equivalent rephrasing of the monodromy assumption ( 4 ).
Proposition 8.
Let
$U/{\mathbf{F}}_{q}$
be a smooth absolutely irreducible variety over
${\mathbf{F}}_{q}$
. Let
$\ell \ne p$
be a prime number and
${\tau}_{\ell}:{\pi}_{1}(\overline{U},\overline{\eta})\to GL(N,{\mathbf{Z}}_{\ell})$
a continuous representation of the geometric fundamental group of
$U$
. Assume
${\rho}_{\ell}$
takes value in
$O(N,{\mathbf{Z}}_{\ell})$
for some nondegenerate bilinear form. If the geometric monodromy group of
${\tau}_{\ell}$
, i.e., the Zariski closure of the image of
${\tau}_{\ell}$
in
$GL(N,{\overline{\mathbf{Q}}}_{\ell})$
, contains
$SO\left(N\right)$
, then we have
$$\left[O\right(N,{\mathbf{Z}}_{\ell}):{\tau}_{\ell}({\pi}_{1}(U,\overline{\eta})\left)\right]<+\infty .$$
In concrete terms, the group
${G}^{g}$
is the set of matrices
$x\in GL(N,{\overline{\mathbf{Q}}}_{\ell})$
for which
$f\left(x\right)=0$
whenever
$f$
is a polynomial function on
$GL(N,{\overline{\mathbf{Q}}}_{\ell})$
(involving possibly
$1/det\left(x\right)$
) that vanishes identically on
$\Gamma $
, and the assumption is that
${G}^{g}\supset SO(N,{\overline{\mathbf{Q}}}_{\ell})$
.
Thus, intuitively, both the condition (
4 ) and the assumption of the proposition are statements saying that
${\rho}_{\ell}$
has a “large” image, and the proposition shows that those two different meanings are in fact quite close.
The condition (
4 ) may be called the “old” way of stating this, whereas the assumption on the geometric monodromy group is the more “modern” style; it is the usual language of the works of Katz for instance; compare with [
S1]
. In [
S3]
, Serre attributes the shift to Grothendieck.

Proof.
Since the image of
${\pi}_{1}(U,\overline{\eta})$
is larger than that of
${\pi}_{1}(\overline{U},\overline{\eta})$
, it suffices to show that
$$\left[O\right(N,{\mathbf{Z}}_{\ell}):{\tau}_{\ell}({\pi}_{1}(\overline{U},\overline{\eta})\left)\right]<+\infty .$$
Let
$\Gamma $
denote the image of the geometric fundamental group. In the
$\ell $
adic topology induced from
$GL(N,{\mathbf{Q}}_{\ell})$
,
$\Gamma $
is compact (by continuity of
${\tau}_{\ell}$
, as the fundamental group is compact) inside the compact group
$O(N,{\mathbf{Z}}_{\ell})$
. The point is that from
${G}^{g}\supset SO\left(N\right)$
, it follows that
$\Gamma \cap SO(N,{\mathbf{Z}}_{\ell})$
is also open in
$SO(N,{\mathbf{Z}}_{\ell})$
, still for the
$\ell $
adic topology (see [S3,Cor.p.120] for instance). But then the finiteness of
$\left[SO\right(N,{\mathbf{Z}}_{\ell}):\Gamma \cap SO(N,{\mathbf{Z}}_{\ell}\left)\right]$
is immediate from the existence of Haar measure
$\mu $
with total mass one on
$SO(N,{\mathbf{Z}}_{\ell})$
, since
$\Gamma $
as an open set must satisfy
$\mu (\Gamma )>0$
. In fact,
$$\left[SO\right(N,{\mathbf{Z}}_{\ell}):\Gamma \cap SO(N,{\mathbf{Z}}_{\ell}\left)\right]=\frac{1}{\mu (\Gamma )}.$$
The desired finiteness of
$\left[O\right(N,{\mathbf{Z}}_{\ell}):\Gamma ]$
is obviously a trivial consequence of this. □
3 Twists of elliptic curves over function fields
We now explain how the result of the previous section apply to the study of extra vanishing for families of twists of elliptic curves over function fields.
We first survey the construction by Katz of varieties parameterizing twists of elliptic curves over function fields over finite fields (see [
K1,Intro.,V]
and also [
U1,2,6,7]
).
We assume for simplicity that the characteristic
$p$
is not
$2$
or
$3$
. Let
$C/{\mathbf{F}}_{q}$
be a smooth projective curve of genus
$g$
, absolutely irreducible,
$K={\mathbf{F}}_{q}\left(C\right)$
the function field of
$C$
and
$E/K$
an elliptic curve, which is assumed to have nonconstant
$j$
invariant. If
$C={\mathbf{P}}^{1}$
is the projective line, the twists considered can be described concretely as follows: take a Weierstrass equation for
$E$
of the type
$${y}^{2}={x}^{3}+a\left(t\right){x}^{2}+b\left(t\right)x+c\left(t\right)$$
with
$a$
,
$b$
,
$c\in {\mathbf{F}}_{q}\left(t\right)$
, and let
$f\in {\mathbf{F}}_{q}\left[t\right]$
be a (squarefree) polynomial. Then the twist
${E}_{f}$
of
$E$
by
$f$
is the elliptic curver
${E}_{f}/K$
with equation
$$f\left(t\right){y}^{2}={x}^{3}+a\left(t\right){x}^{2}+b\left(t\right)x+c\left(t\right),$$
and coefficients of
$f$
serve as “algebraic” parameters for twists.
In the greater generality described,
$f$
is chosen to be a rational function on
$C$
with a prescribed set of poles
$D$
(an effective divisor on
$C$
) and with
$degD$
distinct zeroes, none of which is a place of bad reduction of
$E$
. Katz shows that if
$deg\left(D\right)\u2a7e2g+1$
, this set of functions is the set of
${\mathbf{F}}_{q}$
rational points of a smooth geometrically connected algebraic variety
$X/{\mathbf{F}}_{q}$
(which depends on
$D$
and
$E$
).
Now for any
$f\in X\left({\mathbf{F}}_{{q}^{n}}\right)$
,
$n\u2a7e1$
, there is a twist
${E}_{f}$
defined over
${\mathbf{F}}_{{q}^{n}}\left(C\right)$
, generalizing the above description. For any prime
$\ell \ne p$
, the
$L$
functions of all the twisted curves can be encoded in the “local” behavior of a certain lisse
$\ell $
adic sheaf
${\mathcal{T}}_{\ell}$
on
$X$
, of rank
$N$
which is independent of
$\ell $
, corresponding to a representation
$${\stackrel{~}{\tau}}_{\ell}:{\pi}_{1}(U,\overline{\eta})\to GL(N,{\mathbf{Z}}_{\ell}).$$
Precisely, for any rational point
$f\in X\left({\mathbf{F}}_{q}\right)$
, we have the identity
$$\begin{array}{c}L({E}_{f}/K,T)=det(1T{Fr}_{x}{\mathcal{T}}_{\ell}),\text{or}L({E}_{f}/K,s)=det(1{q}^{s}{Fr}_{x}\left{\mathcal{T}}_{\ell}\right).\end{array}$$ 
(10)

This sheaf is constructed by Katz [
K1,Ch.V]
. Moreover, Katz shows, as consequences of general properties of étale cohomology, that
${\mathcal{T}}_{\ell}$
is punctually pure of weight
$2$
, and there exists a natural nondegenerate symmetric pairing
$${\mathcal{T}}_{\ell}\otimes {\mathcal{T}}_{\ell}\to {\mathbf{Z}}_{\ell}(2).$$
This means that the image of
${\pi}_{1}(U,\overline{\eta})$
by
${\stackrel{~}{\tau}}_{\ell}$
is contained in the group
$CO(N,{\mathbf{Z}}_{\ell})$
of similitudes for this pairing, and the image of
${Fr}_{f,{q}^{n}}$
, for
$u\in X\left({\mathbf{F}}_{{q}^{n}}\right)$
, is a similitude with “multiplier”
$m\left({\stackrel{~}{\tau}}_{\ell}\right({Fr}_{u,{q}^{n}}\left)\right)={q}^{2n}$
. In addition, the image of the geometric fundamental group is contained in the group
$O(N,{\mathbf{Z}}_{\ell})$
of orthogonal transformations for the pairing. Of course, we have an exact sequence
$$1\to O(N,{\mathbf{Z}}_{\ell})\to CO(N,{\mathbf{Z}}_{\ell})\stackrel{m}{\u27f6}{\mathbf{Z}}_{\ell}^{\times}\to 1.$$
The
$L$
functions ( 10 ) have central critical point at
$s=1$
, i.e., at
$T={q}^{1}$
, and it is convenient to make a Tate twist to translate it to
$s=0$
, i.e.,
$T=1$
.
So we consider the sheaf
${\mathcal{T}}_{\ell}\left(1\right)$
instead of
${\mathcal{T}}_{\ell}$
, which corresponds to taking the representation
$${\tau}_{\ell}\left(x\right)={\stackrel{~}{\tau}}_{\ell}\left(x\right){q}^{d\left(x\right)}$$
of
${\pi}_{1}(U,\overline{\eta})$
, where
$d$
is the “degree” map in ( 2 ). The twisted representation
${\tau}_{\ell}$
coincides with
${\stackrel{~}{\tau}}_{\ell}$
on
${\pi}_{1}(\overline{U},\overline{\eta})$
(by ( 2 )). On the other hand, since
${\pi}_{1}(U,\overline{\eta})$
is topologically generated by the
${Fr}_{f,{q}^{n}}$
,
$f\in X\left({\mathbf{F}}_{{q}^{n}}\right)$
,
$n\u2a7e1$
, and since
$$m\left({q}^{d\left({Fr}_{u,{q}^{n}}\right)}{\stackrel{~}{\tau}}_{\ell}\right({Fr}_{u,{q}^{n}}\left)\right)={q}^{2n}{q}^{2n}=1,$$
it follows that
${\tau}_{\ell}\left({\pi}_{1}\right(U,\overline{\eta}\left)\right)\subset O(N,{\mathbf{Z}}_{\ell})$
.
With
${\mathcal{\mathcal{F}}}_{\ell}={\mathcal{T}}_{\ell}\left(1\right)$
, this provides us with all the data occuring in Section 2 . Now the point is that Katz has shown by a deep monodromy computation that the condition ( 4 ) holds for
$D$
suitably chosen. We state a precise version:
Proposition 9 (Katz).
If the divisor
$D$
satisfies the conditions
$$\begin{array}{c}deg\left(D\right)\u2a7e4g+4,2g2+deg\left(D\right)\u2a7emax(144,2s),\end{array}$$ 
(11)

where
$s$
is the number of places of bad reduction of
$E/K$
, then we have
$$\left[O\right(N,{\mathbf{Z}}_{\ell}):{\tau}_{\ell}({\pi}_{1}(U,\overline{\eta})\left)\right]<+\infty .$$

Proof.
Under the condition stated, Katz has shown (see [K1,p.15] for a summary) that the geometric monodromy group
${G}^{g}$
associated to
${\mathcal{T}}_{\ell}\left(1\right)$
contains
$SO\left(N\right)$
. Therefore, we can apply Proposition 8 . □
Corollary 10.
Let
$C/{\mathbf{F}}_{q}$
be a smooth absolutely irreducible projective curve of genus
$g\u2a7e0$
,
$E/{\mathbf{F}}_{q}\left(C\right)$
an elliptic curve with nonconstant
$j$
invariant,
$D$
an effective divisor on
$E$
of degree
$\u2a7e2g+1$
, and
$X=X(D,E)$
the associated parameter space for twists. Assume that
$p>N+2$
and that the twisting sheaves
${\mathcal{T}}_{\ell}\left(1\right)$
satisfy ( 4 ); for instance assume that ( 11 ) holds.
Then, for any
$n\u2a7e1$
, the number
${V}_{n}$
of twisting parameters
$f\in X\left({\mathbf{F}}_{{q}^{n}}\right)$
such that the
$L$
function of
${E}_{f}/{\mathbf{F}}_{{q}^{n}}\left(C\right)$
vanishes at
$s=1$
with order strictly larger than that imposed by the functional equation satisfies
$$\begin{array}{c}{V}_{n}\ll {q}^{n(dimXc)}\end{array}$$ 
(12)

with
$c=\frac{1}{2{N}^{2}2N+1}$
, the implied constant depending on
$D$
,
$E$
, and
$p$
. In particular this set has density
$0$
as
$n\to +\infty $
.

Proof.
The condition
$p>N+2$
implies that there exists a nonzero congruence class
$a$
modulo
$p$
such that
$\ell \equiv a(modp)$
implies that
$GL(N,{\mathbf{F}}_{\ell})$
is of order prime to
$p$
(see e.g. [K1,Lemma7.5.1] ). Pick such a prime
$\ell $
, and then apply Proposition 6 and ( 10 ) to the sheaf
${\mathcal{T}}_{\ell}\left(1\right)$
. □
It is interesting to notice that, together with the work of Ulmer (see [U2] ) on the Birch and SwinnertonDyer conjecture which shows that
$$rank{E}_{f}\left({\mathbf{F}}_{{q}^{n}}\right(C\left)\right)={ord}_{s=1}L({E}_{f}/{\mathbf{F}}_{{q}^{n}}(C),s)$$
if the righthand side is at most
$1$
, this also gives a strong version of the analogue of Goldfeld's Conjecture for the algebraic rank. To state it, we assume for simplicity that the image of the geometric fundamental group by the representation
${\tau}_{\ell}$
is not contained in
$SO(N,{\mathbf{Z}}_{\ell})$
. Katz has shown ([K1,Ex.8.3.4.1] ) that this is the case for instance if
$E/K$
has multiplicative reduction at a point
$s\in S$
.
Proposition 11.
With assumptions as in the previous corollary, assume moreover that the image of the geometric fundamental group by the representation
${\tau}_{\ell}$
is not contained in
$SO(N,{\mathbf{Z}}_{\ell})$
. Then we have
$${\sum}_{f\in X\left({\mathbf{F}}_{{q}^{n}}\right)}rank{E}_{f}\left({\mathbf{F}}_{{q}^{n}}\right(C\left)\right)=\frac{1}{2}\leftX\right({\mathbf{F}}_{{q}^{n}}\left)\right+O\left({q}^{n(dimXc)}\right)$$
for
$n\u2a7e1$
, the implied constant depending on
$D$
,
$E$
and
$p$
.

Proof.
The point is that all the twists have “analytic rank” bounded by the rank
$N$
of
${\mathcal{T}}_{\ell}$
, since their
$L$
functions are all polynomials of degree
$N$
. Since for an elliptic curve
$E/{\mathbf{F}}_{{q}^{n}}\left(C\right)$
there is the apriori inequality (due to Tate [T] )
$$rankE\left({\mathbf{F}}_{{q}^{n}}\right(C\left)\right)\u2a7d{ord}_{s=1}L(E/{\mathbf{F}}_{{q}^{n}}(C),s),$$
the twists also have algebraic rank bounded by
$N$
, so the contribution to the average rank of those few
$f$
for which the analytic rank of
${E}_{f}$
is
$\u2a7e2$
is small. Precisely, notice that
$$rank{E}_{f}\left({\mathbf{F}}_{{q}^{n}}\right(C\left)\right)=\frac{1W\left({E}_{f}\right)}{2}+\beta \left({E}_{f}\right)$$
where
$W\left({E}_{f}\right)$
is the sign of the functional equation and
$\beta \left({E}_{f}\right)$
has the property that
$\beta \left({E}_{f}\right)=0$
if the analytic rank of
${E}_{f}$
is
$\u2a7d1$
(this is the result of Ulmer), and
$\left\beta \right({E}_{f}\left)\right$
is bounded (
$\u2a7dN+1$
) for all
$f$
and
$n$
. Thus the sum of
$\beta \left({E}_{f}\right)$
is
$$\ll {q}^{n(dimXc)}$$
for
$n\u2a7e1$
by ( 12 ). On the other hand
$${\sum}_{f\in X\left({\mathbf{F}}_{{q}^{n}}\right)}\frac{1W\left({E}_{f}\right)}{2}=\frac{\leftX\right({\mathbf{F}}_{{q}^{n}}\left)\right}{2}\frac{1}{2}{\sum}_{f\in X\left({\mathbf{F}}_{{q}^{n}}\right)}W\left({E}_{f}\right).$$
and we have
$$W\left({E}_{f}\right)=(1{)}^{N}det({\tau}_{\ell}\left({Fr}_{f}\right))$$
where
${\tau}_{\ell}$
is as before the representation which corresponds to
${\mathcal{T}}_{\ell}$
(this is simply ( 6 ) and ( 10 )). The assumption that the image of
${\tau}_{\ell}$
is not inside
$SO$
implies that the character
$det\circ {\tau}_{\ell}$
of order
$2$
is nontrivial on the geometric fundamental group. Thus
${H}_{c}^{2d}(\overline{X},det\circ {\tau}_{\ell})=0$
and by the Riemann hypothesis we have
$$\left{\sum}_{f\in X\left({\mathbf{F}}_{{q}^{n}}\right)}W\left({E}_{f}\right)\right\u2a7d{q}^{n(dimX1/2)}{\sum}_{0\u2a7di<2dimX}dim{H}_{c}^{i}(\overline{X},det\circ {\tau}_{\ell})\ll {q}^{n(dimX1/2)}$$
for
$n\u2a7e1$
, finishing the proof. □
4 Examples of one parameter families of twists
This section is a concrete illustration of the previous section. We will use some of the intermediate statements proved by Katz in [
K1]
to restrict our attention to oneparameter families of twists (i.e., replace the big parameter space
$X$
of the previous section by a curve) where the analog of Corollary 10 and Proposition 11 still hold. (In fact, it is by finding such oneparameter families inside the larger spaces that Katz shows that the geometric monodromy groups for those parameter spaces contain
$SO\left(N\right)$
).
To simplify, we will only consider curves over
${\mathbf{P}}^{1}$
and twists by polynomials; this allows us to write down explicit equations (in other words,
$C={\mathbf{P}}^{1}$
,
$g=0$
and the divisor
$D$
is
$d(\infty )$
in the notation of the previous section).
We start with an elliptic curve given by a fairly general Weierstrass equation (assuming always that the characteristic
$p$
is
$\u2a7e5$
)
$$E:{y}^{2}={x}^{3}+a\left(t\right){x}^{2}+b\left(t\right)x+c\left(t\right)$$
with polynomials
$a$
,
$b$
and
$c\in {\mathbf{F}}_{q}\left[t\right]$
. Denote by
$S$
the set of points in
${\mathbf{A}}^{1}$
where
$E$
has bad reduction.
In Chapter 5 of [
K1]
, two different types of oneparameter families with “large” monodromy are described. We will consider special cases of the first one (Theorem 5.4.1 of loc. cit.). Adapting the terminology found there and in [
K1,Ch.2]
, we say that a polynomial
$f\in {\overline{\mathbf{F}}}_{q}\left[t\right]$
of degree
$d\u2a7e1$
is of Lefschetz type if the following conditions hold:
(i)
$f$
has
$d$
distinct zeros in
${\overline{\mathbf{F}}}_{q}$
; (ii)
${f}^{\prime}$
has
$d1$
distinct zeros, and those have distinct images by
$f$
.
Moreover we say that
$f$
is of KatzLefschetz type for
$E$
if
$f$
is of Lefschetz type, and (i') no two points of
$S$
have the same image by
$f$
; (ii')
$f\left(s\right)\ne 0$
for
$s\in S$
; (iii') the fibers
${f}^{1}\left(f\right(s\left)\right)$
all have
$d$
distinct elements for
$s\in S$
.
Fix a polynomial
$f$
of KatzLefschetz type. Let
${V}_{f}$
denote the (finite) variety, defined over
${\mathbf{F}}_{q}$
, of critical values for
$f$
, i.e.
$${V}_{f}=f\left(\right\{x\left{f}^{\prime}\right(x)=0\})\cup f(S),$$
and let
${U}_{f}={\mathbf{A}}_{1}{V}_{f}$
. Then Theorem 5.4.1 of [
K1]
states that if
$d\u2a7emax(146,2S\left\right)$
, the oneparameter family of quadratic twists with equations
$${E}_{f,\alpha}:\left(f\right(t)\alpha ){y}^{2}={x}^{3}+a\left(t\right){x}^{2}+b\left(t\right)x+c\left(t\right),$$
with
$\alpha \in {U}_{f}$
has associated
$\ell $
adic twisting sheaves
${\mathcal{\mathcal{F}}}_{\ell ,f}$
on
${U}_{f}$
, of fixed rank
${N}_{f}$
, such that the geometric monodromy group (on
${U}_{f}$
) of
${\mathcal{\mathcal{F}}}_{\ell ,f}$
contains
$SO\left({N}_{f}\right)$
for
$\ell \ne p$
. In particular this sheaf satisfies the condition ( 4 ).
Corollary 12.
Let
$E/{\mathbf{F}}_{q}\left(t\right)$
be an elliptic curve with nonconstant
$j$
invariant and at least one finite place of multiplicative reduction. Let
$f$
be a polynomial of KatzLefschetz type for
$E$
of degree
$d\u2a7emax(146,2S\left\right)$
.
(i) For
$n\u2a7e1$
, the number
${V}_{n}$
of
$\alpha \in {U}_{f}\left({\mathbf{F}}_{{q}^{n}}\right)$
where
${E}_{f,\alpha}/{\mathbf{F}}_{{q}^{n}}\left(t\right)$
has extra vanishing satisfies
$${V}_{n}\ll {q}^{n(1c)}$$
with
$c=\frac{1}{2{N}_{f}^{2}2{N}_{f}+1}$
, the implied constant depending on
$E$
,
$f$
and
$p$
. (ii) If the image of the geometric fundamental group is not contained in
$SO(N,{\mathbf{Z}}_{\ell})$
, we have
$${\sum}_{\alpha \in {U}_{f}\left({\mathbf{F}}_{{q}^{n}}\right)}rank{E}_{f,\alpha}\left({\mathbf{F}}_{{q}^{n}}\right(t\left)\right)=\frac{1}{2}\left{U}_{f}\right({\mathbf{F}}_{{q}^{n}}\left)\right+O\left({q}^{n(1c)}\right),$$
for
$n\u2a7e1$
, the implied constant depending on
$E$
,
$f$
and
$p$
.
We could in fact state a slightly better result using the Chebotarev density theorem for curves instead of the general version (see the next section).
To be completely concrete, we will now take a specific example. Let
$E/{\mathbf{F}}_{p}\left(t\right)$
be the following variant of the Legendre elliptic curve:
$$\begin{array}{c}E:{y}^{2}=x(x+1)(xt).\end{array}$$ 
(13)

Note that
$E$
has multiplicative reduction at
$\infty $
and at the points in
$S=\{0,1\}$
.
Now consider the following polynomials:
$${f}_{d}={t}^{d}dt1\in {\mathbf{F}}_{p}\left[t\right].$$
Lemma 13.
(i) If
$p\mathbb{\nmid}d(d1)$
and
$(p1,d1)=1$
, then
${f}_{d}$
is of Lefschetz type.
(ii) If in addition
$p\mathbb{\nmid}d+1$
, then
${f}_{d}$
is of KatzLefschetz type for the above curve
$E/{\mathbf{F}}_{p}\left(t\right)$
.

Proof.
The derivative of
${f}_{d}$
is
${f}_{d}^{\prime}=d({t}^{n1}1)$
so since
$p\mathbb{\nmid}d$
, the roots of
${f}_{d}^{\prime}$
are the
$(d1)$
st roots of unity. Since
$p\mathbb{\nmid}d1$
, there are
$d1$
of them in
${\overline{\mathbf{F}}}_{p}$
. Now for
$\mu $
a zero of
${f}_{d}^{\prime}$
we have
$${f}_{d}\left(\mu \right)=\mu (1d)1.$$
This already shows that the values of
${f}_{d}$
at zeros of
${f}_{d}^{\prime}$
are distinct.
Notice that this formula also shows
${f}_{d}\left(\mu \right)\ne 0$
because otherwise
$\mu $
would be in the prime field
${\mathbf{F}}_{p}$
, so that
$\mu =1$
by the assumption
$(p1,d1)=1$
, the equation becomes
$1d=d$
, but again
$p\mathbb{\nmid}d$
excludes this case. Since
${f}_{d}^{1}\left(x\right)$
, for
$x\in {\overline{\mathbf{F}}}_{p}$
, has
$d$
elements except if
$x$
is in the set
$\left\{{f}_{d}\right(\mu \left)\right\}$
, it follows that
${f}_{d}$
has
$d$
distinct roots in
${\overline{\mathbf{F}}}_{p}$
. Altogether, this establishes the first assertion that
${f}_{d}$
is of Lefschetz type.
For the second, the conditions
$(p1,d1)=1$
and
$p\u2a7e5$
imply that
$d$
is even. We compute
${f}_{d}$
at the points in
$S$
: we have
${f}_{d}\left(0\right)=1$
and
${f}_{d}(1)=(1{)}^{d}+d1=d$
. So
${f}_{d}\left(0\right)\ne {f}_{d}(1)$
since
$p\mathbb{\nmid}d+1$
. Moreover
${f}_{d}\left(0\right)=1$
is not of the form
${f}_{d}\left(\mu \right)=\mu (1d)1$
as above, since
$p\mathbb{\nmid}d$
.
Similarly
${f}_{d}(1)=d$
is not of this form:
$d=\mu (1d)1$
implies again that
$\mu $
is in the prime field, so
$\mu =1$
, and again
$p\mathbb{\nmid}d$
shows that
$d+1=1d$
is impossible. So neither
$0$
nor
$1$
is in a fiber over a zero of
${f}_{d}^{\prime}$
, which means that the fibers over points of
$S$
contain
$d$
distinct elements. □
Remark 14.
So for
$p=5$
, we have found explicit polynomials of KatzLefschetz type for
$E$
of any even degree
$d$
with
$d\equiv 2(mod5)$
, i.e.,
$d\equiv 2(mod10)$
.
In general, the density of integers
$d\u2a7e1$
satisfying the conditions of the lemma is
$$\frac{\phi (p1)}{p}\left(1\frac{3}{p}\right)>0$$
for all
$p\u2a7e5$
.
Let
$d$
be any integer satisfying the condition of the lemma. We then have the oneparameter family of twists
$${E}_{d,\alpha}:\left({f}_{d}\right(t)\alpha ){y}^{2}=x(x+1)(xt),$$
or equivalently (change
$y$
to
$({f}_{d}\alpha )y$
)
$${E}_{d,\alpha}:{y}^{2}=({t}^{d}dt1\alpha )x(x+1)(xt)$$
over
${\mathbf{F}}_{p}\left(t\right)$
, with parameter
$\alpha $
in the complement
${U}_{d}$
of the finite variety of critical values for
${f}_{d}$
, which has
$d1+2=d+1$
points defined over
${\overline{\mathbf{F}}}_{p}$
.
Let
${\mathcal{\mathcal{F}}}_{d,\ell}$
denote the twisting sheaf
${\mathcal{T}}_{\ell}\left(1\right)$
for this subfamily. As observed by Katz [
K1,Lemma7.5.1]
, the twist sheaves associated to quadratic twists of elliptic curves are always tame in characteristic
$p\u2a7e5$
, so
${\mathcal{\mathcal{F}}}_{d,\ell}$
is tame. The rank
${N}_{d}$
of
${\mathcal{\mathcal{F}}}_{d,\ell}$
is computed in [
K1,Lemma5.1.3,p.
16]
and is given by
$${N}_{d}=2d$$
for
$d$
even. In particular, this means that the (degree of the) conductor of the twists goes to infinity when
$d\to +\infty $
.
As a special case of Theorem 5.4.1 of [
K1]
, if
$d\u2a7e146$
, the geometric monodromy group for
${\mathcal{\mathcal{F}}}_{d,\ell}$
is the full orthogonal group
$O\left({N}_{d}\right)$
for all
$\ell $
.
So specializing again the previous corollary we get:
Corollary 15.
Let
$p\u2a7e5$
be prime, let
$d\u2a7e146$
,
${f}_{d}$
and
${U}_{d}$
be as above. For
$n\u2a7e1$
, the number
${V}_{n}$
of
$\alpha \in {U}_{d}\left({\mathbf{F}}_{{q}^{n}}\right)$
for which the twisted Legendre curve
$${y}^{2}=({t}^{d}dt+1\alpha )x(x+1)(xt)$$
over
${\mathbf{F}}_{{p}^{n}}\left(t\right)$
has extra vanishing satisfies
$$\begin{array}{c}{V}_{n}\ll {p}^{n(1c)},\end{array}$$ 
(14)

with
$c=\frac{1}{2({d}^{2}+1)}$
, the implied constant depending on
$d$
and
$p$
.
5 Twists with unbounded conductor
This section is speculative. The idea is to exploit the strong bound ( 14 ) to prove a variant of Proposition 11 for a family of twists more closely resembling the quadratic twists of elliptic curves over
$\mathbf{Q}$
, namely one where the conductor (i.e., essentially, in this case, the degree of the
$L$
function) increases, so that the rank of the elliptic curves is not uniformly bounded.
The speculation consists in the fact that the result obtained is conditional on monodromy assumptions which are stronger than currently known.
We still work with the curve (
13 ) of the previous section for concreteness. Take a sequence of polynomials
${f}_{n}$
of KatzLefschetz type with increasing degrees
${d}_{n}$
. We will use a simple subscript
$n$
for all the objects of the last section which would otherwise require to be subscripted by either
${f}_{n}$
or
${d}_{n}$
. For instance, we denote by
${\mathcal{\mathcal{F}}}_{n,\ell}$
the twisting sheaf for the
$1$
parameter family corresponding to
${f}_{n}$
and let
${N}_{n}$
denote its rank.
We make the following strong assumption
$$\begin{array}{c}\text{For all}n\text{and all odd}\ell \ne p\text{, the image of the representation}{\rho}_{n,\ell}:{\pi}_{1}({U}_{{d}_{n}},\overline{\eta})\to GL({N}_{n},{\mathbf{F}}_{\ell})\text{corresponding to the reduction}{\mathcal{\mathcal{F}}}_{n,\ell}/\ell {\mathcal{\mathcal{F}}}_{n,\ell}\text{is of bounded index in}O({N}_{n},{\mathbf{F}}_{\ell})\text{}.\end{array}$$ 
(15)

See the final paragraphs of the paper for comments on the plausibility of this.^{2 }
We denote by
${G}_{n,\ell}$
the image of
${\rho}_{n,\ell}$
and by
$B$
a bound for its index in
$O(N,{\mathbf{F}}_{\ell})$
valid for all
$n$
and
$\ell \ne p$
.
Corresponding to Lemma
5 we need the following uniform version for
$\nu =1$
, which we make a little bit more precise:
Lemma 16.
(1) For all odd primes
$\ell $
and all
$n\u2a7e1$
we have
$$\left{G}_{n,\ell}\right\u2a7d{\ell}^{{N}_{n}({N}_{n}1)/2},$$
the implied constant depending only on the bound
$B$
for the index of
${G}_{n,\ell}$
in
$O({N}_{n},{\mathbf{F}}_{\ell})$
.
(2) Let
${C}_{n,\ell}$
be the set of
$g\in {G}_{n,\ell}$
with extra vanishing. We have
$$\frac{\left{C}_{n,\ell}\right}{\left{G}_{n,\ell}\right}\ll \frac{1}{\ell}$$
for all odd primes
$\ell $
and
$n\u2a7e1$
, the implied constant depending only on
$B$
, provided that
$\ell \u2a7e{N}_{n}^{2}$
.

Proof.
(1) The size of
${G}_{d,\ell}$
is bounded by that of
$O({N}_{n},{\mathbf{F}}_{\ell})$
for which the existing formulas immediately give the result stated.
(2) We bound
${C}_{n,\ell}$
by the number of elements with extra vanishing in
$O({N}_{n},{\mathbf{F}}_{\ell})$
. In general, for
$O(N,{\mathbf{F}}_{\ell})$
, the latter (say
$R\left(N\right)$
) is written as follows:
$$R\left(N\right)={\sum}_{g}\left\right\{x\in O(N,{\mathbf{F}}_{\ell})det(1Tx)=g\}$$
where
$g$
runs over characteristic polynomials of elements of
$O(N,{\mathbf{F}}_{\ell})$
which have extra vanishing.
It is clear that the number of possible
$g$
is
$\u2a7d{\ell}^{N1}$
. For each
$g$
, we count the inner quantity by the same method as in [
C,ProofofTh.3.5]
(with adaptations necessary because the orthogonal group is not simply connected like the symplectic group) which shows that it is
$$\ll (\ell +1{)}^{N(N1)/2}(\ell 1{)}^{N}$$
(with absolute implied constant), so we get
$$R\left(N\right)\u2a7d{\ell}^{1}{\left(\frac{\ell}{\ell 1}\right)}^{N}(\ell +1{)}^{N(N1)/2}$$
and because
$\leftO\right(N,{\mathbf{F}}_{\ell}\left)\right\u2a7e(\ell 1{)}^{N(N1)/2}$
, this yields the result after an application of the mean value theorem. □
Here is the hypothetical result with unbounded conductors.
Proposition 17.
Let
$p\u2a7e5$
be prime, let
${d}_{n}$
for
$n\u2a7e1$
be an increasing sequence of integers such that each
$d={d}_{n}$
satisfies
$p\mathbb{\nmid}d(d1)(d+1)$
and
$(p1,d1)=1$
, and such that
${d}_{n}^{3}\u2a7dn$
for
$n\u2a7e1$
. Assume the monodromy hypothesis ( 15 ) for the sequence of KatzLefschetz polynomials
${f}_{n}={t}^{{d}_{n}}{d}_{n}t+1$
.
(i) We have for
$n\u2a7e1$
$$\left\right\{\alpha \in {U}_{{d}_{n}}\left({\mathbf{F}}_{{p}^{n}}\right)\left{E}_{{d}_{n},\alpha}\text{has extra vanishing}\right\}\ll {n}^{1/3}{p}^{n\frac{1}{4}{n}^{1/3}}.$$
(ii) We have for
$n\u2a7e1$
$${\sum}_{\alpha \in {U}_{{d}_{n}}\left({\mathbf{F}}_{{p}^{n}}\right)}rank{E}_{{d}_{n},\alpha}\left({\mathbf{F}}_{{p}^{n}}\right(t\left)\right)=\frac{{p}^{n}}{2}+O\left({n}^{2/3}{p}^{n\frac{1}{4}{n}^{1/3}}\right).$$

Proof.
Because of all the assumptions, the Chebotarev density theorem for curves (Theorem 3 , with
$g=0$
,
$m={d}_{n}+1$
) and Lemma 16 imply that for
$n\u2a7e1$
and
$\ell \ne p$
we have
$$\left\right\{\alpha \in {U}_{{d}_{n}}\left({\mathbf{F}}_{{q}^{n}}\right)\left{E}_{{d}_{n},\alpha}\text{has extra vanishing}\right\}\ll \frac{{p}^{n}}{\ell}+{d}_{n}{p}^{n/2}{\ell}^{A}$$
with
$$A=\frac{{N}_{n}({N}_{n}1)}{2}\frac{1}{2},$$
the implied constant depending only on
$B$
if
$\ell \u2a7e{N}_{n}^{2}=4{d}_{n}^{2}$
.
Since
$2(A+1)={N}_{n}({N}_{n}1)+1\u2a7d4{d}_{n}^{2}$
, taking
$\ell $
between
${p}^{n/2(A+1)}$
and
$2{p}^{n/2(A+1)}$
gives
$$\left\right\{\alpha \in {U}_{{d}_{n}}\left({\mathbf{F}}_{{p}^{n}}\right)\left{E}_{{d}_{n},\alpha}\text{has extra vanishing}\right\}\ll {d}_{n}{p}^{n\frac{n}{4{d}_{n}^{2}}}$$
for
$n\u2a7e1$
, with an implied constant depending only on
$B$
; the assumption
${d}_{n}^{3}\u2a7dn$
ensures that for
$n$
large enough (
$n\u2a7e6$
suffices) we have
$$nlogp\u2a7e2{d}_{n}^{2}(log4{d}_{n}^{2})$$
hence
$\ell \u2a7e4{d}_{n}^{2}$
, and the implied constant can be raised to absord the values
$n\u2a7d5$
, as well as those for which
${d}_{n}<146$
.
For part (ii), the reasoning is as in the proof of Proposition
11 , using Ulmer's result about the Birch and SwinnertonDyer conjecture. The contribution of the twists with analytic rank
$\u2a7e2$
is estimated using (i) and the trivial bound
$$rank{E}_{{d}_{n},\alpha}\left({\mathbf{F}}_{{p}^{n}}\right(t\left)\right)\u2a7d{N}_{n}=2{d}_{n}\u2a7d2{n}^{1/3}$$
(so there too the uniformity of our estimates in terms of
$n$
is – or would be! – important). □
Remark 18.
In terms of the parameter
$X={p}^{n}$
, the error terms have the following shape:
$${n}^{1/3}{p}^{n\frac{1}{4}{n}^{1/3}}\ll (logX{)}^{1/3}Xexp(\frac{1}{4}(logX{)}^{1/3})$$
which may look more familiar to analytic number theorists.
We finish by commenting on our monodromy assumption ( 15 ). First of all, for fixed
$n$
, the uniformity in terms of
$\ell $
is part of the standard conjectures (see e.g. [S2,10.3?,10.7?] ) about the variation of images of
$\ell $
adic representations.
In addition, since Katz has shown that the “rational” geometric monodromy group is always equal to
$O\left(N\right)$
, it is a consequence of a result of Larsen [
L,Th.3.17]
that for a set of primes
$\ell $
of density
$1$
, the geometric monodromy group modulo
$\ell $
contains the image in
$O(N,{\mathbf{F}}_{\ell})$
of the spin group
$Spin(N,{\mathbf{F}}_{\ell})$
, which is of index
$2$
in
$SO(N,{\mathbf{F}}_{\ell})$
and
$4$
in
$O(N,{\mathbf{F}}_{\ell})$
(this complication arises because
$O(N,{\mathbf{F}}_{\ell})$
is neither connected nor simply connected). Larsen's result is quite difficult (it uses the classification of simple finite groups), and the set of primes it produces is not easy to control.
Another example, still for fixed rank, is a result proved by Gabber concerning the monodromy of Kloosterman sheaves which is explained in [
K3,Ch.12]
. Roughly speaking, the integral monodromy group associated to families of Kloosterman sums in an even number
$n$
of variables is “big” for all
$\ell $
large enough, depending on
$n$
, but again not in an easy way to describe for varying
$n$
.
When the rank is increasing, it is in fact not clear if the uniform bound we postulate is coherent with the general philosophy concerning
$\ell $
adic representations. The reason is that this variation of
$n$
does not fall into a wellunderstood theoretical framework: the “family” we consider is one only inasmuch as we manage to deal with its individual terms and get similar results; this is much the same as the case of “families” of classical automorphic
$L$
functions, for which convincing examples exist abundantly without an a priori definition. Still, in the case of elliptic curves
$E/\mathbf{Q}$
, a similar result is expected: recall that Serre showed that for a fixed
$E$
without CM, there exists
${L}_{E}$
such that for any prime
$\ell >{L}_{E}$
, the map
$${\rho}_{E,\ell}:Gal(\overline{\mathbf{Q}}/\mathbf{Q})\to Aut\left(E\right[\ell \left]\right)\simeq GL(2,{\mathbf{F}}_{\ell})$$
is surjective (i.e., the fields obtained by adjoining to
$\mathbf{Q}$
the coordinates of the
$\ell $
torsion points of
$E$
are “as big as possible”). Then the conjectured statement that such an
$L$
as above exists which “works” for all elliptic curves
$E/\mathbf{Q}$
(without CM) can be seen as an analogue of our assumption (see e.g. [
S1,Question 2,p.199]
). It has been confirmed by Duke [
D]
that this can be done for “almost all” curves.
Finally, we can turn for encouragement to at least one similar situation where a result of the desired type has been unconditionally proved. Let
$f\in {\mathbf{F}}_{q}\left[x\right]$
be a fixed polynomial of degree
$2g$
with
$2g$
distinct roots in
${\overline{\mathbf{F}}}_{q}$
, and consider the family of hyperelliptic curves of genus
$g\u2a7e1$
with equations
$${C}_{\alpha}:{y}^{2}=f\left(x\right)(x\alpha ),$$
over the open set
${\mathbf{A}}^{1}{f}^{1}\left(0\right)$
, with projection
$\pi (x,y,\alpha )=\alpha $
. The sheaves
${R}^{1}{\pi}_{!}{\mathbf{F}}_{\ell}$
, which are of rank
$2g$
and admit symplectic symmetry, are used to “globalize” the family of
$L$
functions of
${C}_{\alpha}$
(modulo
$\ell $
). JiuKang Yu has shown that the geometric monodromy group is equal to
$Sp(2g,{\mathbf{F}}_{\ell})$
for all
$f$
and all
$\ell \ne 2$
. This is one of the main examples in [
C]
. It is also used to give some results uniform in
$g$
in [
Ko]
(which are in fact of a rather more delicate nature).
References

N. Chavdarov: The generic irreducibility of the numerator of the zeta function in a family of curves with large monodromy, Duke Math. J. 87 (1997), 151–180.

B. Conrey, J.P. Keating, M. Rubinstein and N. Snaith: On the frequency of vanishing of quadratic twists of modular
$L$
functions, in Number theory for the millennium I, 301–315, A K Peters (2002).

Duke, W.: Elliptic curves with no exceptional primes, C. R. Acad. Sci. Paris Sér. I Math. 325 (1997), no. 8, 813–818.

M. Fried and M. Jarden: Field arithmetic, Ergebnisse der Math. und ihrer Grenzgebiete, 3 Folge, vol. 11, Springer Verlag (1986).

F. Gouvêa and B. Mazur: The squarefree sieve and the rank of elliptic curves, J. Am. Math. Soc. 4, No.1, 123 (1991).

H. Iwaniec and E. Kowalski: Analytic Number Theory, A.M.S Colloquium Series vol. 53 (2004).

N. Katz: Twisted
$L$
functions and monodromy, Annals of Math. Studies 150 (2002).

N. Katz: Sums of Betti numbers in arbitrary characteristic, Finite Fields Appl. 7 (2001), no. 1, 29–44.

N. Katz: Gauss sums, Kloosterman sums and monodromy, Annals of Math. Studies, 116, Princeton Univ. Press, 1988.

E. Kowalski: The large sieve, monodromy and zeta functions of curves, preprint (2004).

M. Larsen: Maximality of Galois actions for compatible systems, Duke Math. J. 80 (1995), no. 3, 601–630.

JP. Serre: Quelques applications du théorème de densité de Chebotarev, Publ. Math. IHES 54 (1981), 323–401.

JP. Serre: Propriétés conjecturales des groupes de Galois motiviques et des représentations
$\ell $
adiques, in Motives (Seattle 1991), 377–400, Proc. Sympos. Pure Math. 55, Part 1, AMS 1994.

JP. Serre: Sur les groupes de Galois attachés aux groupes
$p$
divisibles, Proc. Conf. Local Fields, SpringerVerlag (1966), 118–131; also in Œuvres, t. II, 325–338.

J. Tate: On the conjecture of Birch and SwinnertonDyer and a geometric analog, Séminaire Bourbaki, Exp. 306, 1966.

D. Ulmer: Geometric nonvanishing, Invent. math. 159 (2005), 133–186.

D. Ulmer: Elliptic curves and analogies between number fields and function fields, MSRI Publication 59, 285–315, Cambridge Univ. Press (2004).
Universite Bordeaux I A2X, 351, cours de la Liberation, 33405 Talence Cedex, France Email address : emmanuel.kowalski@math.ubordeaux1.fr