Sam Rayburn's 'lady friend' revealed
By media release
Sep 29, 2016
Print this page
Email this article

New York, NY, September 29, 2016– An article titled “Margaret Fallon (Peggy) Palmer: A Portrait of Sam Rayburn’s ‘Lady Friend’” about the life of the widow of former U.S. Attorney General A. Mitchell Palmer as well as her close relationship with Sam Rayburn, the longest serving Speaker of the U.S. House of Representatives, will be published October 10 by the Texas State Historical Association’s scholarly journal Southwestern Historical Quarterly. Southwestern Historical Quarterly, the oldest scholarly
journal in Texas continuously published since 1897, brings the latest and most authoritative research in Texas history to a wide audience of history lovers and scholars. Anthony Champagne, a political science professor at the University of Texas at Dallas as well as a Rayburn biographer, and Reed Penney, a documentary producer working on a Sam Rayburn documentary, wrote the article.

For nearly fifty years Sam Rayburn served as a Congressman from Northeast Texas (1913 – 1961). For seventeen of those years he was Speaker of the U.S. House of Representatives, longer than anyone else
in American history.

“I think from 1931 to 1961 Sam Rayburn was the most important American politician who was never President. His fingerprints are on some of the most important legislation of the 20th century.” – Reed Penney (“Margaret Fallon (Peggy) Palmer: A Portrait of Sam Rayburn’s ‘Lady Friend’” co-author)

At the peak of his career most people thought Sam Rayburn was a lifelong bachelor. When he died in 1961 he left no widow or children, but once in his life Rayburn was briefly married. In October 1927 Rayburn married the sister of a fellow Congressman from Texas, but nearly three months after they married they separated. In November 1928 they divorced.

After his divorce Rayburn had relationships with a number of women, but almost nothing has been known about the women in Rayburn’s life until now. After A. Mitchell Palmer’s death in 1936 his widow Peggy Palmer developed a close relationship with Sam Rayburn. It was so close that Hope Ridings Miller, a journalist who covered Washington D.C. society for many years and a close friend of Rayburn’s, suggested to him in 1940 that his vice presidential prospects would improve if he married Peggy Palmer.

According to Miller Rayburn replied, “Well, it’s very nice of you to see just what I need and thank you very much. But I’ll tell you, I like her, and I like her very much. She’s one of the most charming, one of the most understanding women I’ve ever known in my life, but marriage is not in the cards for Sam Rayburn.” – Hope Ridings Miller quoting Sam Rayburn to Anthony Champagne, Interview, August 19, 1986

There’s no smoking gun evidence that Rayburn ever seriously considered a second marriage.

“Even if Rayburn had thought about marrying her, many things in Peggy’s past as well as her life in Washington would have made her politically unacceptable to Rayburn’s constituents. She had spent most of her privileged childhood in and near New York City when she was not traveling with her family abroad. She had attended an all-girls’ college prep school on the Upper West Side of Manhattan. She had Irish Catholic roots on her father’s side of the family. Her first marriage had ended in a scandalous divorce. Her father and her first two husbands were Republicans. In Washington Peggy was a socialite with a larger than life personality. Peggy’s great-niece and great-nephew described her as an 'Auntie Mame' type of a woman. Rayburn, in contrast, came from an impoverished background, one of eleven children of small farmers in Fannin County, Texas. The worlds Mr. Sam and Peggy had come from and their personalities were incredibly dissimilar yet they liked and enjoyed each
other’s company.” - Anthony Champagne & Reed Penney (“Margaret Fallon (Peggy) Palmer: A Portrait of Sam Rayburn’s ‘Lady Friend’” co-authors)

Margaret Fallon (Peggy) Palmer, circa 1941

Papers from various archives confirm Rayburn’s involvement in Peggy Palmer’s life. After A. Mitchell Palmer’s death Peggy held a series of government jobs that gave her a source of income. In December 1938 Peggy was lobbying Franklin D. Roosevelt’s White House for a job. The following month one of Roosevelt’s aides wrote to the President…

“Do you want to ask the bachelor Attorney General if he wants Peggy Palmer for official hostess at the [Justice] Department. That would be one way of getting sam [sic] Rayburn’s goat!” – Memorandum, Marvin H. McIntrye to Franklin D. Roosevelt, January 12th, 1939

Around 1940 one gossip columnist wrote about Rayburn and Peggy having dinner together at a restaurant in Washington.

“[Rayburn] is known never to be seen out on ‘dates’…that is, with one exception, and she is Peggy ‘The Charmer’ Palmer, clever, bright spirited….[Rayburn] was speaking with heart and soul to pretty Peggy. He looked very serious, and she seemed very interested.” – Igor Cassini (gossip columnist), c. 1940

Even Lady Bird Johnson, the wife of Rayburn’s protégé and former President Lyndon Johnson, said in an oral history in 1980 that in 1942…

“[Peggy Palmer] was the Speaker’s current lady friend, and I was often out with them.” – Lady Bird Johnson, 1980 (LBJ Library Oral History)

In 1946 Peggy Palmer was concerned she would lose her government job. Sam Rayburn asked Harry Truman’s White House to help Peggy keep her job. In late 1949 Rayburn tried to get Peggy a position at the State Department. Nearly six years later Rayburn gave Peggy her last job in Washington on the staff of the House Select Committee on Small Business chaired by Rayburn’s friend Wright Patman, a fellow Congressman from Texas.

By the end of 1962, and a little over a year after Rayburn’s death, Peggy lost her job in the House. Before Peggy was let go she asked Attorney General Robert Kennedy for a job writing to him

“as my many years with Sam Rayburn have come to an end I’m awfully anxious to see and really bother you.” – Letter, Peggy Palmer to Robert F. Kennedy, June 18th, 1962

For the rest of her life Peggy had no job and struggled to survive in Washington with few sources of income. In 1973 she died alone in her Washington apartment. She was 89.

“Peggy Palmer was not the only woman in Sam Rayburn’s life…but at the very least she was one of the most important to him after his divorce…So if Rayburn and Peggy were so close why did they never get married? One possible explanation is Rayburn was already scarred by his one brief marriage and the ensuing divorce. By the time Peggy was Rayburn’s 'lady friend' he was one of the most important American politicians who was never President. Another marriage - especially to a woman like Peggy - would have brought Rayburn a lot of publicity and focus on his personal life
especially from his constituents, and Rayburn would not have jeopardized his career in that way. Certainly there was much in Peggy’s personal life that would have been unappealing to Rayburn’s constituents. Rayburn did like Peggy and cared about her, but his political capital would have been damaged if Peggy Palmer had become Peggy Rayburn.”– Anthony Champagne & Reed Penney (“Margaret Fallon (Peggy) Palmer: A Portrait of Sam Rayburn’s ‘Lady Friend’” co-authors)