The principal residence at Poplar Grove was built as the Banker's Pavilion at the World's Industrial and Cotton Centennial Exposition 1884-85 in New Orleans. The Victorian structure designed by architect Thomas Sully incorporated oriental motifs popular during the Aesthetic Movement period. The pavilion was moved by barge to Poplar Grove in 1886 and was the home of sugar planter Horace Wilkinson, prominent business and civic leader.
James McCalop established Poplar Grove as a sugar plantation in the 1820s by combining several smaller tracts of land. Joseph L. Harris acquired Poplar Grove in 1885. Horace Wilkinson managed the plantation for Harris and purchased Poplar Grove in 1903. Originally 1,438 acres, the plantation was a thriving agricultural operation with its own sugar mill, workers' quarter, church, railroad, mule and hay barns, corn crib, and commissary. The mill, which produced raw sugar and molasses, operated until 1973.