The Hamlet of Fort Miller took its name from the old French and Indian War era storehouse, which was decaying on the west side of the Hudson River by the summer of 1777. The land on the east side of the river was the estate of William Duer, a local judge who served on the Continental Congress and as commissary, providing supplies to the Continental Army. Prior to the hostilities, Duer had a contract to supply the British Navy with timber that could be used for ships’ masts, and several buildings surrounded his well-built manor. The land would have been cleared for timber and then converted for agricultural use, leaving an open view of the Hudson River.
Baum accompanied Burgoyne’s first wave of troops to Fort Miller, and is among the first to arrive there. The Rebels had abandoned the area just 4 days prior to their arrival, and their camps were still evident. Fort Miller served as a gathering place for Burgoyne’s 8,000-man army, and became a temporary city made up of troops, laborers, engineers, tradesmen, bakers, surgeons and civilians, which included women, children and other non-military personnel, amounting up to 10,000 people.