Annie Payne Causten: Dolley’s “Sterling Girl”

Mathew Brady wanted Dolley Madison.

Having the beloved former First Lady sit for a portrait in his studio would be a significant coup for the up-and-coming photographer. A mutual friend provided a letter of introduction, assuring Dolley that Brady “cannot think his gallery complete without adding your face to his interesting collection.”¹ Thomas Ritchie to Dolley Payne Todd Madison, ca. May 2, 1848, private collection, accessed March 10, 2020, MRD-S 41784, Montpelier Research Database. Dolley complied. The sitting, on July 4, 1848, produced several daguerreotypes of Dolley seated by herself – the image that Brady had been keen to capture. But it also produced an intriguing double portrait. Perhaps at Dolley’s request, or perhaps because Brady recognized Dolley’s affection for her young companion, Brady also photographed Dolley with her niece Annie (Anna) Payne.

Sweet, Sensible, Sterling

Annie was born in 1819, the third of the eight children of Dolley’s brother John Coles Payne and his wife Clara (Clary) Wilcox Payne. In the 1820s and 1830s, the Paynes lived on a farm within the bounds of the Montpelier tract (which James Madison later bequeathed to John).² James Madison, Will dated [April 15, 1835], and Codicil [April 19, 1835], Will Book 8: 134-138, Orange County Courthouse, Orange, Virginia, accessed March 12, 2020, MRD-S 23094, Montpelier Research Database. Annie and her siblings practically grew up with their Aunt Dolley.

Dolley described 11-year-old Annie as “a sweet one, & very sensible.”³Dolley Payne Todd Madison to Richard D. Cutts, March 12, 1830, box 5, folder 26 , Jasper E. Crane Collection of James and Dolley Madison, MS C0082, Princeton University Library, Princeton, New Jersey, accessed March 12, 2020, MRD-S 28121, Montpelier Research Database. Teenaged Annie enjoyed the neighborhood social life. Dolley wrote in December 1834, “Anna & her sisters have gone to a dancing party at Newman’s—they are to keep the Christmas from this time to New Years day.”⁴ Dolley Payne Todd Madison to Mary Estelle Elizabeth Cutts, December 11, 1834, Cutts Family Collection of Papers of James and Dolley Madison, Library of Congress, Washington, DC, accessed March 12, 2020, MRD-S 28493, Montpelier Research Database.

Both Annie and her oldest sister Dolley sometimes stayed overnight at Montpelier, keeping Aunt Dolley company. When James Madison was away at a Board of Visitors meeting at the University of Virginia in October 1826, Dolley Madison wrote to her husband “Jno. & Clary spent Sunday with me, & I keep Dolley to sleep in your place.”⁵ Dolley Madison to James Madison, December 5, 1826, St. John’s Seminary, Camarillo, California, accessed March 12, 2020, MRD-S 25896, Montpelier Research Database. In 1835, Annie stayed at Montpelier while James was ill. Dolley wrote in May, “Anna who is a sterling girl stays much at home with me and sleeps beside my bed ever since the illness of Mr. Madison in April.”⁶ Dolley Payne Todd Madison to Dolley Payne Madison Cutts, May 11, 1835, Cutts Family Collection of Papers of James and Dolley Madison, Library of Congress, Washington, DC, accessed March 12, 2020, MRD-S 25897, Montpelier Research Database.

Mathew B. Brady, daguerreotype of Dolley Payne Todd Madison and Anna Payne Causten, 1848. Courtesy of National Portrait Gallery, Smithsonian Institution; gift of Dortha Louise Dobson Adem Rogus, descendant of Dolley Madison.

Constant, Affectionate, Enthusiastic

The Payne nieces helped care for their aunt after she was widowed in 1836. In February 1837, John Payne wrote that his daughter Annie was part of “the permanent female share of the Montpelier family,”⁷ John Coles Payne to Edward Coles, February 16, 1837, box 2, folder 3, Edward Coles Papers, MS C0037, Princeton University Library, Princeton, New Jersey, accessed March 12, 2020, MRD-S 27799, Montpelier Research Database. suggesting that Annie was living with Dolley full-time by then. When the Payne family moved west in 1837 (to Illinois and later Kentucky), 18-year-old Annie stayed behind at Dolley Madison’s request. Dolley, John, and Clara had discussed the plan for Annie to remain with Dolley, and Dolley promised that Annie would be well provided for and would be in a position to help her parents. (The Madisons expected the sale and publication of James’s papers to yield  at least $100,000, based on what was paid to John Marshall for his biography of George Washington.)⁸ John Coles Payne to [James H. Causten Jr.] and Anna Coles Payne Causten, November 5, 1851, box Madison/Payne Family, folder Payne, John C. , Dolley Madison Collection, MS 47, Greensboro History Museum, Greensboro, North Carolina, accessed March 12, 2020, MRD-S 32152, Montpelier Research Database. As close as Annie was to her Aunt Dolley, saying goodbye to her parents was difficult. She wrote in 1837, “Papa and all have been gone two months & four days … How little we thought then that in June we should be separated by 800 miles! … I often look out for them without reflecting but they are gone—The only tears I ever saw papa shed was when we parted. Oh, how sad our parting was!”⁹ Anna Coles Payne Causten to Richard D. Cutts, June 18, 1837, box 5, folder 19, Jasper E. Crane Collection of James and Dolley Madison, MS C0082, Princeton University Library, Princeton, New Jersey, accessed March 12, 2020, MRD-S 28513, Montpelier Research Database.

For the next 12 years after that tearful parting, Annie was Dolley’s “personal and constant affectionate attendant and companion,” in the words of a family friend.¹⁰ Anthony Morris to Rebecca Wistar Morris Nourse, May 15, 1837, Special Collections, University of Virginia Library, Charlottesville, Virginia, accessed March 12, 2020, MRD-S 35575, Montpelier Research Database. She accompanied Dolley on extended trips to Washington and back again to Montpelier. She stayed busy “reading & improving her self in many desirable things,” as Dolley explained in 1840,¹¹ Dolley Payne Todd Madison to Elizabeth Collins Lee, February 19, 1840, Papers of Dolley Madison, Library of Congress, Washington, DC, accessed March 12, 2020, MRD-S 23703, Montpelier Research Database. adding to another friend that Annie “commenced taking French lessons in March which has engrossed her whole time, & mind.”¹²Dolley Payne Todd Madison to Frances (Fanny) Dandridge Henley Lear, August 1, 1840, Papers of Dolley Madison, Library of Congress, Washington, DC, accessed March 12, 2020, MRD-S 30345, Montpelier Research Database. Annie also became, as her aunt described her, “a young enthusiast on the subject of autographs,” building a collection that included signatures of Alexis de Tocqueville, James K. Polk, and even William Penn.¹³ Dolley Payne Todd Madison to Isaiah Townsend, August 13, 1842, Papers of Dolley Madison, Library of Congress, Washington, DC, accessed March 12, 2020, MRD-S 30533, Montpelier Research Database; Sarah Polk to Anna Coles Payne Causten, May 5, 1848, box Madison/Payne Family, folder Payne, Annie—Corresp.—Polk, Sarah, Dolley Madison Collection, MS 47, Greensboro History Museum, Greensboro, North Carolina, accessed March 12, 2020, MRD-S 32135, Montpelier Research Database; Thomas Gilpin to Anna Coles Payne Causten, July 11, 1842, James Madison Papers, Library of Congress, Washington, DC, accessed March 12, 2020, MRD-S 19764, Montpelier Research Database.

This Bible was a gift from Annie “For my dear Aunt on her birth-day” in 1845. Courtesy of Montpelier, a National Trust historic site.

Joyous, Precious, Of Great Assistance

In 1842 Annie and Dolley traveled to Philadelphia and New York. A relation in Philadelphia was pleased that Annie and Dolley would be attending a ball there, writing, “I am sure Miss Payne will enjoy it—she appears to have so joyous a disposition.”¹⁴ Sarah Logan Roberts to Dolley Payne Todd Madison, April 13, 1842, Papers of Dolley Madison, Library of Congress, Washington, DC, accessed March 12, 2020, MRD-S 30505, Montpelier Research Database. Annie also enjoyed parties and other social events in the capital city whenever she and her aunt visited. Dolley wrote a cousin in 1842, “Anna is on the wing to Mrs. Webster’s party and I must put a finishing stroke to her dress…”¹⁵Dolley Payne Todd Madison to Edward Coles, March 31, 1842, box 2, folder 7, Edward Coles Papers, MS C0037, Princeton University Library, Princeton, New Jersey, accessed March 17, 2020, MRD-S 27826, Montpelier Research Database. In 1844, when Dolley sold Montpelier, Washington became Dolley and Annie’s permanent home.

Annie not only provided company and cheer to Dolley, but also assisted her with tasks such as correspondence. Mary Cutts, another of Dolley’s nieces, recalled: “Mrs. Madison’s handwriting was the Italian, but late in life it became laborious and painful for her to write, so much so that after Mr. Madison’s death, when his, as well as her own correspondents overwhelmed her, she taught her niece Miss Payne to imitate it. After much practice, she was so successful that none but those perfectly familiar with the penmanship of both, knew the difference. This was of great assistance. The letter Mrs. Madison would begin; as soon as her eyes ached, the pen would be transferred to Miss Payne, who continued to write under her dictation; then she would herself add the last words and the signature.”¹⁶ Mary Estelle Elizabeth Cutts Memoir II, [1849-1856], Cutts Family Collection of Papers of James and Dolley Madison, Library of Congress, Washington, DC, accessed March 16, 2020, MRD-S 23538, Montpelier Research Database.

In July 1845, Annie and Dolley shared the experience of being confirmed at St. John’s Episcopal Church. Dolley, born a Quaker, noted that the Episcopal Church was “the Church which I have attended for the last forty years—and which Anne has generally attended, all her life.”¹⁷Dolley Payne Todd Madison to Richard D. Cutts, July 16, 1845, box 5, folder 30, Jasper E. Crane Collection of James and Dolley Madison, MS C0082, Princeton University Library, Princeton, New Jersey, accessed March 16, 2020, MRD-S 28221, Montpelier Research Database. Annie had given her aunt a Bible as a birthday gift in May 1845, perhaps anticipating their upcoming Confirmations.

Because Dolley and Annie were so close, they both worried over each other’s health. After Annie’s extended illness in 1845, Dolley wrote, “I feel the conciousness of a dull spirit which for the last three months has bound me in the fear of loosing a very precious Neice whose health is now being restored, & mine in consequence is recovering.”¹⁸ Dolley Payne Todd Madison to Sarah Childress Polk, September [2, 1846], Papers of Dolley Madison, Library of Congress, Washington, DC, accessed March 16, 2020, MRD-S 30693, Montpelier Research Database. Dolley’s final illness in July 1849 was a hard blow for Annie. Dolley’s nephew James Madison Cutts wrote on July 11 that Annie had been nursing Dolley through the night, and “her watchings & affectionate solicitude for Aunt has made her ill – she has fainted from exhaustion now so frequently & is with all so ill that the friends around Aunt begin to have equal anxiety for one & the other.”¹⁹ James Madison Cutts to James Buchanan, July 11, 1849, Buchanan Family Papers, Historical Society of Pennsylvania, Philadelphia, Pennsylvania, accessed March 17, 2020, MRD-S 37301, Montpelier Research Database. After Dolley’s death on July 12, Annie “insisted upon going to the Church, at the funeral of her Aunt—but, at the door, fell lifeless” in a faint.²⁰ James Madison Cutts to John Young Mason, July 29, 1849, Papers of Dolley Madison, Library of Congress, Washington, DC, accessed March 17, 2020, MRD-S 30803, Montpelier Research Database. Annie was ill in bed for several weeks and recovered at The Highlands, the home of her friends the Nourse family, just outside the capital city. Mary Cutts noted that Annie had been “quite sick … with the return of her old complaint,”²¹ Mary Estelle Elizabeth Cutts to “Annie” [unknown], August 29, 1849, Unlocated, accessed March 19, 2020, MRD-S 24899, Montpelier Research Database.perhaps connecting it to Annie’s prior bout of illness in 1845.

St. John’s Church, Washington, DC, 1918-1920 photograph. Dolley Madison and Anna Payne were both confirmed at this church on Lafayette Square. Courtesy of Library of Congress Prints and Photographs Division, National Photo Company Collection.

Dolley’s final will had divided her most valuable asset – a trust fund established when she sold James Madison’s papers to the federal government – between her son John Payne Todd and her “adopted daughter” Annie Payne. (Dolley never legally adopted Annie, but often used this term to describe their relationship). Payne Todd challenged the will, which had been signed on Dolley’s deathbed. This led to years of legal wrangling, resolved only by his death in January 1852. Although this final will made no mention of Dolley’s papers, an earlier had will expressed Dolley’s wish that Annie would sort these papers and burn any private letters. Annie apparently considered herself honor-bound to fulfill this wish, even to the point of barging into Dolley’s house with another niece, Ellen O’Neale Cutts, to retrieve a carpet bag full of papers.²²William E. Kennaugh to John Payne Todd, August 22, 1849, Papers of Dolley Madison, Library of Congress, Washington, DC, accessed March 19, 2020, MRD-S 30813, Montpelier Research Database. (For more on the destruction of Madison papers, see Where Have All the Papers Gone?)

Devoted, Cheerful, Laughing

With Dolley gone, Annie’s nearest relatives were hundreds of miles away. Perhaps Annie considered joining them out west, but in the meantime a new relationship blossomed for her. It’s unclear how Annie met Dr. James Causten Jr., although the extended Causten family was part of Dolley’s Washington social circle. The two married in April 1850, and welcomed a daughter Mary in August 1851. Letters between Annie and James portray a loving family group. Annie often addressed James as “Darling Dear,”²³ Anna Coles Payne Causten [and Eliza Causten] to James H. Causten Jr., July 16, 1852, box Causten Family: A through Causten, James—Notebook, folder Causten, Annie Payne—Corresp.—Causten, James H. Jr., Dolley Madison Collection, MS 47, Greensboro History Museum, Greensboro, North Carolina, accessed March 19, 2020, MRD-S 32231, Montpelier Research Database. signing herself “Your devoted wife.”²⁴ Anna Coles Payne Causten to James H. Causten Jr., July 23, 1852, box Causten Family: A through Causten, James—Notebook, folder Causten, Annie Payne—Corresp.—Causten, James H. Jr., Dolley Madison Collection, MS 47, Greensboro History Museum, Greensboro, North Carolina, accessed March 19, 2020, MRD-S 32232, Montpelier Research Database. Annie’s letters include fond descriptions of Mary’s antics: “You ought to see the Baby with a great piece of bacon in her hand, eating & talking away to your Daguerreotype. She is too sweet.”²⁵ Anna Coles Payne Causten to James H. Causten Jr., July 20, 1852, box Causten Family: A through Causten, James—Notebook, folder Causten, Annie Payne—Corresp.—Causten, James H. Jr., Dolley Madison Collection, MS 47, Greensboro History Museum, Greensboro, North Carolina, accessed March 19, 2020, MRD-S 32233, Montpelier Research Database.

Annie was beloved by James’s family as well as by James himself. Annie’s brother-in-law wrote to her after she had returned home from a visit, “The house has lost its life — the cheerful voice and hearty laugh of our dear sister Annie is wanting — I can well understand how it was that Jimmy found home without you so dreary.”²⁶ Joseph Shriver to Anna Coles Payne Causten, June 24, [1850], box Causten Family: A through Causten, James—Notebook, folder Causten, Annie Payne—Corresp.—Causten, Henrietta, Dolley Madison Collection, MS 47, Greensboro History Museum, Greensboro, North Carolina, accessed March 12, 2020, MRD-S 32225, Montpelier Research Database.

Anna Payne Causten, portrait by Charles Bird King, 1852. Courtesy of the Dolley Madison Collection, Greensboro History Museum.

Both Annie and James seem to have had chronic health problems. In November 1852 Annie was too ill to attend Payne Todd’s estate sale, where she hoped to obtain more of her aunt’s papers as well as furnishings with sentimental value. Her father-in-law, James Causten Sr., agreed to attend the sale on her behalf, but as he explained, “before I returned to Washington her spirit had fled, and my first melancholy duty was to attend the deposite of her remains in my family vault, by the side of her Aunt M^rs Dolly P. Madison.”²⁷ James H. Causten Sr. to Col. John Willis, Jr., December 2, 1852, Madison Family Bible, Princeton University Library, Princeton, New Jersey, accessed March 16, 2020, MRD-S 28924, Montpelier Research Database. Dolley’s remains were later removed from the Causten family vault at the Congressional Cemetery and were interred at Montpelier. Washingtonian Anna Maria Brodeau Thornton noted in her diary on November 10, 1852, “M^rs. Anna Causten died last Eveng. She was ill a long time.”²⁸ Anna Maria Brodeau Thornton, Diary, 1793-1863, Anna Maria Brodeau Thornton Papers, MS 51862, Library of Congress, Washington, DC, accessed March 16, 2020, MRD-S 23848, Montpelier Research Database.

Annie was only 33 when she died. Her husband James died four years later, leaving daughter Mary an orphan at just five years old. Mary was raised by her grandfather Causten and her aunts and uncles. While the Caustens no doubt cared deeply for the young girl, Mary missed knowing her sweet, joyous, affectionate mother Annie, Dolley’s “sterling girl,” who had delighted in her own “precious Mary.”²⁹ Anna Coles Payne Causten to James H. Causten Jr., July 26, 1852, box Causten Family: A through Causten, James—Notebook, folder Causten, Annie Payne—Corresp.—Causten, James H. Jr., Dolley Madison Collection, MS 47, Greensboro History Museum, Greensboro, North Carolina, accessed March 19, 2020, MRD-S 32235, Montpelier Research Database.

Hilarie came to Montpelier in 2010 and joined the Research Department in 2011, where she provides documentary research in support of the Montpelier Foundation’s many activities. A graduate of the College of William and Mary (B.A) and the Cooperstown Graduate Program in Museum Studies (M.A.), Hilarie has a broad background of experience in research, interpretation, and administration of historic sites. She enjoys following a good paper trail and considers herself fortunate to have been, like Annie, the companion of a beloved aunt.