Loudoun House is considered one of the largest and finest examples of Gothic Revival architecture in Kentucky. It reflects the Romantic Movement of the 1850s, which was a reflection of the social lifestyles and opulence of the day. The house follows a design of prominent New York architect Alexander Jackson Davis, who published his catalog of house designs, Rural Residences, in 1838. Davis' collaboration with author and horticulturist A J Downing was the foremost influence in disseminating the Gothic Revival style throughout the country. Loudoun was constructed by Lexington builder John McMurtry, who helped popularize the Gothic Revival style in the Bluegrass by constructing more than 200 buildings in this style.

The house's many towers, irregular volumes, vaults, asymmetrically arranged lancet, diamond-paned windows as well as other picturesque architectural features denote the Gothic Revival style. Hollow brick walls that provided for better insulation and walls covered with successive layers of sand and paint to resemble stone are notable. An unusual feature is the half-tunnel, which encircles the main foundations of the house to eliminate moisture from the footings of the walls and give them added stability. The house is long and shallow, its forms building up irregularly to the principal tower to the right side of the entrance pavilion. Chimneystacks, crenellated tower and turret, parapet walls, and pinnacles on the important gables comprise an interesting skyline. A gymnasium was added to the rear of the house and some of the porches have been removed , but its sense of picturesque romanticism remains.

The residence was built for Francis Key Hunt in 1850. Francis Key was the son of John Wesley Hunt who built the Hunt-Morgan House. Between 1870 and 1889 Loudoun was the home of Colonel William Cassius Goodloe who served as chairman of the national committee of the Republican Party and was later appointed Minister to Belgium by President Hayes. Loudoun, now situated in Castlewood Park, is owned by the city of Lexington and houses the Lexington Art League. - NPS