Eventually, Fisher left the sheep exporting business and returned to Vermont to run a farm in Panton, where he married Jessie May Field (1879-1967) on August 3, 1906. The couple had two children, Osman Field Fisher (1910-1993) and Ellen Bigelow (1907-1987). Fisher was a Freemason (Union Lodge No. 2, Middlebury) and was a charter member of Otter Creek Chapter No. 74 of the Order of the Eastern Star in Vergennes.

Topics in this diary include the international Merino sheep trade, selling livestock in Africa and South America, the perils of turn-of-the-century sea travel, and meteorological phenomena on the Atlantic Ocean.