The novel is set in the mid-19th century, with glimpses into the history of the house, which was built in the late 17th century. The primary interest of this book is in the subtle and involved descriptions of character and motive.
The house of the title is a gloomy New England mansion, haunted from its foundation by fraudulent dealings, accusations of witchcraft, and sudden death. The current resident, the dignified but desperately poor Hepzibah Pyncheon, opens a shop in a side room to support her brother Clifford, who is about to leave prison after serving thirty years for murder. She refuses all assistance from her unpleasant wealthy cousin Judge Jaffrey Pyncheon. A distant relative, the lively and pretty young Phoebe, turns up and quickly becomes invaluable, charming customers and rousing Clifford from depression. A delicate romance grows between Phoebe and the mysterious attic lodger Holgrave, who is writing a history of the Pyncheon family.