The Lady Next Door (Unabridged‪)‬

Marianne Findlay lives next door to the Earl of Latteridge’s York townhouse. Though she has never met him, it was his mother who sealed her fate years before - and drove her from polite society. Now she and Aunt Effie take in boarders and live a modest life, which is soon disrupted by the handsome earl, his scapegrace younger brother, his charming and determined sister - and that vengeful mother of his. 

This is a Regency romance by Laura Matthews originally published by Fawcett.