https://en.wikipedia.org/wiki/Election_Day_(United_States)

"A uniform date for choosing presidential electors was instituted by the Congress in 1845.^[1] Many theories have been advanced as to why the Congress settled on the first Tuesday after the first Monday in November.^[5] The actual reasons, as shown in records of Congressional debate on the bill in December 1844, were fairly prosaic. The bill initially set the day for choosing presidential electors on "the first Tuesday in November," in years divisible by four (1848, 1852, etc.). But it was pointed out that in some years the period between the first Tuesday in November and the first Wednesday in December (when the electors are required to meet in their state capitals to vote) would be more than 34 days, in violation of the existing Electoral College law. So, the bill was reworded to move the date for choosing presidential electors to the Tuesday after the first Monday in November, a date scheme already used in New York.^[6] The period between Election Day and the first Wednesday in December is always 29 days. The effect of the change was to make November 2 the earliest day on which Election Day may fall.

In 1845, the United States was largely an agrarian society. Farmers often needed a full day to travel by horse-drawn vehicles to the county seat to vote. Tuesday was established as election day because it did not interfere with the Biblical Sabbath or with market day, which was on Wednesday in many towns.^"