In theoretical physics, modular invariance is the invariance under the group such as SL(2,Z) of large diffeomorphisms of the torus. The name comes from the classical name modular group of this group, as in modular form theory.
In string theory, modular invariance is an additional requirement for one-loop diagrams. This helps in getting rid of some global anomalies such as the gravitational anomalies.
Equivalently, in two-dimensional conformal field theory the torus partition function must be invariant under the modular group SL(2,Z).