Tell us about your book.
The Memory Of All That: George Gershwin, Kay Swift, and My Family’s Legacies of Infidelities is a memoir, but it is also a researched group biography. So it’s a bit of a nonstandard hybrid. It’s a memoir in the sense that it is framed by recollections of my own life and experiences, and it is a highly personal story. But the book is also a group portrait of the extraordinary and accomplished and — let’s call them colorful — people in my family, going back through the generations in all directions. While the family tree has an only-in-America mix, with Benedict Arnold over here and Mendel Gottesfeld over there, with Warburgs and Kaufmans and Swifts and Shippens, the heart of the story is my maternal grandmother Kay Swift’s big romance with George Gershwin. The book is ultimately about the effect that powerful relationship had on every member of the family, from 1925 to the present moment.
How did you first know that you wanted to write this story? What were the factors that engaged you from the beginning?
After five novels, after passing the age of fifty, after a series of family deaths, I think I just started to feel that the time had come in my life to tell this story, to work with all these separate strands of story that are ultimately woven together as one big story, with a goal of making a coherent narrative at last. I have always had awareness of, if not clarity about, the confusing web of interwoven family stories and incidents and events and personalities that have surrounded me throughout my life. Like most families, mine has told its history and explained events in many conflicting and incomplete and incomprehensible stories. Unlike most families, many of these stories have also been told by others, and have been on full display in the public eye for years, and have been, for better or worse, for accurate or confused or simply false, become part of the historical record. So in one sense, I knew that I wanted to write a counter-story to some of the stories already out there.
How did you find writing memoir to be different from writing fiction? How was it similar?
That’s an important question. I have no idea what it would be like to write a memoir if I had never written anything else, or if I had written other historical books or biographies. For me, as the author of five novels, writing this was confounding at certain moments, when I had to let the facts and truths of the story, as I uncovered them or assembled them in meaningful ways, dictate the narrative. I couldn’t change facts and details to suit the story, the way I do when I am writing a novel. I couldn’t make things signify more or mean more by shifting things around in order to have everything mean more. I had to make the story out of the facts. I had to find the meanings and signifiers, not invent them.
Nevertheless, writing anything is about making choices, and about shaping a narrative. And at a certain point in the writing, just when I had gotten quite far from the process of writing fiction in some ways, I realized that it was a great deal like writing a novel after all, in that the reader has to know who the narrator is, and trust the narrator, and understand the narrator’s purpose in telling this story now, in this way. So in a sense I had to develop a better understanding of myself as the narrator, the way I have always needed to know what the narrator of any of my novels wants to say and why. Who is telling this story and why? This is information the writer always needs to know deeply.
Were you ever frustrated by having to stick to the facts of the story?
The frustration wasn’t so much having to stick to the facts as it was not having all the facts, as is always true in life, and not being able to fill in imaginatively beyond a certain level of very clearly labeled speculation. But I do think my delving in the course of writing the book has led me to believe certain things I had never thought so thoroughly about before, and the research has led me to feel that I know some of my relatives and ancestors far better than I ever would have without bearing down on their stories as I did in the course of writing the book. I don’t just mean the central figures – George Gershwin, Kay Swift, and my grandfather, Jimmy Warburg. I learned a great deal about my father, Sidney Kaufman, by reading his 800 page FBI records. (I also learned a great deal about the terrible narrative voice of the FBI.) I learned a great deal about my great-grandfather, Paul Warburg, the architect of the Federal Reserve System, the Cassandra who predicted the stock market crash of 1929, and the most vilified of all Jewish bankers. In his lifetime, he was called the ringleader of the so-called international Jewish banking conspiracy by Henry Ford, and at this very moment, you can find Louis Farrakhan denouncing my great-grandfather in speech after speech on YouTube. (He calls him “Hitler’s banker” and says that “Paul Warburg stayed in the finest hotels of Europe while Hitler was killing little Jewish babies!” (I note that Paul Warburg died in January, 1932.) I learned about my extraordinary great-great grandfather, Joseph Swift, who spent time in the Montana Territory during the gold rush and was the best friend of a notorious criminal, Henry Plummer. So for all the missing and obscure and unknowable facts, there were the joys of discovery, endlessly. One more: I discovered that a relative virtually invented the white line at the side of the road (the first experimental stretch thus painted was on the Merritt Parkway.)
If the story is not about you, did you have to do a lot of research to find out the facts? Was that research easy to accomplish? What sorts of sources did you use?
As a novelist I do all kinds of very nonstandard research, and much of it would not meet the academic definition of research in the least. (For example, I attended candy conventions when I was writing my novel True Confections.) But for this book I did a great deal of true archival sifting, finding papers and letters connected to my family in the Beinecke (my grandfather’s correspondence with Ezra Pound is there, which led me to a fantastic email correspondence with Pound’s daughter , who was very gracious about permission to quote), the Gilmore Music Library (where my grandmother Kay Swift’s papers are on deposit), and Sterling Library (Paul Warburg papers) at Yale, the Kennedy Library in Boston (my grandfather James Paul Warburg’s papers are there) in Boston, and Butler Library at Columbia University, where the Mary Lasker papers contain a treasure trove of hundreds of letters written to her by my grandmother from the ranch in Oregon where she lived after she eloped with a cowboy she met at a rodeo two years after George Gershwin’s death.
Perhaps the greatest single archival source, however, was the massive file of FBI reports on my father, who was under surveillance from 1936 to 1973. The FBI read our mail, tapped our phones, and analyzed every magazine subscription, every name in my father’s address book. My birth in 1955 was reported in a memo to J. Edgar Hoover. I requested these papers under a Freedom of Information Act request back in 1992, but it took some nine years for the FBI to comply with my request, and apparently they spent much of that time redacting information on nearly every page of the files they released to me, rendering the narrative style even more peculiar.
There are so many more layers when you’re writing about people who really did exist. What sorts of feedback (and blowback) did you get after the book came out? Were there people whose reactions you particularly feared?
I didn’t fear any reactions, but I have had some pleasant surprises and some unpleasant surprises since publication. Some people have been fantastically broadminded and perceptive about what I set out to do and what I did do in the pages of the book. Others have been, frankly, pained. I have a relative who prefers to keep her late husband (my blood relative, much discussed in the book) embalmed in infallibility. The book is an unhappy subject with her, but it came out a year ago and I think there has been a graceful turning of the page at this point. I certainly agreed not to speak at the library in her town, when she said she would rather I did not do that, so that event was cancelled.
Do you have a writing process you could share with readers, a way you like to proceed when you’re writing a book? (Some authors say they write the first draft out in long-hand, on yellow legal pads, with a special, sacred pen, after throwing salt over their shoulders and repeating an incantation…while others say they write the first draft in a two-week-long fever and then spend the next two years revising. Others don’t even begin until they know the whole story).
I do like to have an outline. I have learned from experience that it is far more efficient for me to work this way. I map it, I plan it out, and I often have a lot of information about certain little moments, certain large moments. But then I am also always ready to surprise myself. So the best strategy I like to offer my writing students is to make a plan and then be prepared to deviate from the plan. In fact, if you have hit your original marks every step of the way, with no deviations or detours, I would say that is a likely indication that you aren’t tuning in to the material of your story sufficiently, and you haven’t allowed the story to build and inform from the inside what comes next.
If you weren’t a writer, what job would you like to do?
I have always felt a pull towards psychotherapy, and also towards, don’t laugh, being a detective. Novelists and detectives and psychotherapists are all interested in understanding motive — why people do what they do.
Do you procrastinate? What’s your favorite mode of procrastination when you’re supposed to be writing?
I do. I find fascinating things on the web and tell myself it’s all research. I am writing a novel right now about a monkey. Do you know how many monkey videos are on YouTube?
What about being a writer has made you truly happy?
Loving my work, passionately, and being rewarded for it by the world.
What do you tell people who want to be writers, too?
Trust your strangeness. It’s yours. Trust your own strangeness!