Potter Hall is significant for its architecture and for its association with the Potter family, prominent in local and state affairs. Architecturally, it embodies the distinctive characteristics of the Federal style as expressed on Maryland's Eastern Shore. In the context of Caroline County's historic architecture, Potter Hall is one of six surviving large brick plantation houses dating from the late 18th to early 19th century. Its abundant woodwork presents the best example in the county of the delicacy of design and execution which characterizes early Federal ornament. The second-story entrances, which once served broad porches, are unique in Caroline County. Potter Hall was originally settled by Zabdiel Potter, a sea captain from Rhode Island, in the mid 18th century. He built a wharf and the small brick house which was later incorporated into the central section of the present structure. Zabdiel developed Potter's Landing into a key early port for the shipping of tobacco to Baltimore. His son and nephews were important men in the history of the county and the state.- NRHP, 30 November 1982