Bailey was born on 5 January 1933 in Portsmouth, England. While his father served in the British Army and his younger sister Bridget remained in England with their mother during World War II, Tony was taken in for four years by Otto and Eloise Spaeth, who had four children of their own, including a boy also named Tony. Otto Spaeth was the owner of a Dayton machine tool factory and both he and his wife were passionate art collectors. Bailey's lifelong interest in art was influenced by his time living with the Spaeths. The family's private art collection included such artists as Cezanne, Paul Gauguin and Edward Hopper.After National Service as a British Army officer with the Royal West African Frontier Force, Bailey went to Merton College, Oxford, in 1952, where he read history.[3] In 1955 he moved to New York, assisted by the Spaeths. His early jobs were in shops selling books, first with Scribners and then in the British Book Centre owned by newspaper publisher Robert Maxwell. When a friend suggested to Bailey that he submit his writings to The New Yorker, he sent in a piece about parking meters and an account of a day spent with Austrian Catholic priest Ivan Illich, who worked for the poor in Harlem. New Yorker editor William Shawn offered him a job. There he found himself in an office next door to John Updike, who became Bailey's lifelong friend.Under Shawn, Bailey was a "Talk of the Town" reporter and also worked briefly as a reader in the fiction department before becoming a staff writer. His work for the magazine includes profiles, reporter-at-large pieces, poems and short stories.Bailey contributed many pieces to The New Yorker magazine. The Dial Press in New York published his first novel, Making Progress, in 1959. His third novel Major André (about Benedict Arnold's attempt to hand over West Point to the British) received positive reviews in 1987His books included biographies of Turner and Constable, Vermeer, Velázquez, and two books on Rembrandt.[Many of Bailey's papers, wartime letters and manuscripts are in the hands of the Houghton Library at Harvard University.Bailey was interviewed by NPR and The New York Observer. He contributed to the New York Herald Tribune, The New York Times The New Republic and Esquire. In Britain, his writings featured in the New Statesman, The Observer, and The Sunday Times. The Overseas Press Club awarded him the 1973 Ed Cunningham Award and the Mary Hemingway for his work with The New Yorker.
