firstly in 1773 to Anne Swinnerton by whom he had seven children, including his eldest son and heir, Rev. Egerton Bagot, and daughters, Elizabeth (died 5 Mar 1859), who married Dr. Joseph Phillimore, MP, and Louisa-Frances, who married Rev. Richard Levett of Milford Hall, Staffordshire,[3] also an Oxford graduate and a minister.
secondly to Mary Ward by whom he had another eight children, including a daughter, Jane Margaret, who married the English judge Sir Edward Vaughan Williams[4] in 1826; they were the grandparents of the composer Ralph Vaughan Williams.