Roger Ludlow, founder of Fairfield, and a small group of settlers established their homes in Fairfield in 1639 along five roads that created four squares of 25 to 30 acres each. The squares were laid out into home lots with one square containing the parsonage land for use by the minister, another for the meetinghouse, still another for a military or public park, and the fourth containing a large parcel for Roger Ludlow himself.

Civic and religious activity and the buildings that served them were concentrated around the intersection of present day Old Post Road and Beach Road. The meetinghouse according to Farnum was constructed on the northwest corner in 1645 when William Frost gave ten pounds toward the effort. It served a triple function as a town house, schoolhouse, and meetinghouse. A successor meetinghouse was burned, along with many other houses and town buildings, by the British in 1779. Its replacement took 42 years to complete, with part of the cost covered by the sale of properties formerly owned by the Tories. The present structure was built in 1891 in the Richardsonian Romanesque style.

In a town meeting in 1768, it was agreed to allow the County of Fairfield to build a courthouse on the green. A successor town house, built in 1768, was burned by the British in 1779. It was rebuilt in 1794, damaged by fire in 1861, remodeled in ornate Second Empire style, circa 1870, and remodeled again in 1936 in Colonial Revival style to recreate its 1790s appearance. In 1979, in order to meet the greatly increased need for office space, a new town hall, known as Independence Hall, was built on a site to the west of the old structure, which continues to house offices for various town functions.

The green served as a military drilling ground; Colonel Andrew Burr trained men in preparation for the French and Indian War and General Gold Sellect Stilliman gathered militia for drill during the Revolutionary War. A whipping post and stocks stood directly opposite the meetinghouse. When peace was declared ending the War in 1812, a celebration was held and an ox barbecued "under the great Sycamore on the Green."

Although Fairfield established public education as early as 1640, the town's first school building was not in place until 1695. By 1800 prominent citizens perceived a need for higher education to prepare youths for admission to Yale and Harvard Colleges, and subscribed a fund to establish an academy. Acquired by the town in 1958, the Academy was moved in 1960 from its original site on the Old Post Road (between the Burr Mansion and Episcopal Church) to an interior location on the town-owned property. It is presently used by the local chapter of the Daughters of the American Revolution. A pond which had bordered the old Fairfield Academy was filled in circa 1879, the land to be forever unencumbered by buildings.

Another notable building located behind the old town hall is the Sun Tavern, built soon after the Revolution. Washington and other prominent citizens visited the tavern. The town purchased it in 1878 and today it is used by the town historian.

The Burr Homestead adjoining the green belonged to the Burr family (Peter Burr, Chief Justice of the Colony; his son Thaddeus and grandson Thaddeus II) who had close business and social ties to leading families in Boston. As a result, the house is associated with several prominent historic figures. The homestead is now owned by the town and the ten acres making up the site have been incorporated into the civic center.

Over the years, other buildings stood on the green that have long since been razed. For example, in 1687, Jacob Joy was granted the right to build a barn on the green with the condition that he would be the town blacksmith "during his life." In 1768, the town gave a "small piece" of the green upon which to build a prison and a house for the keeper with a garden plot. And in 1813, Wheeler Judson was allowed to build a tailor shop on the green near the pond.

The present small lot in front of the Town Hall presently known as the green is said to have been landscaped by James Mott, who came to Fairfield from Pennsylvania to establish a factory. He was town treasurer during the 1870 remodeling of Town Hall and advocated turning the dusty, barren piece of unused land into a pleasant setting of trees and grass for the Town Hall. This was probably done under the guidance of the Fairfield Village Improvement Society.

The green has remained the focal point of the town's religious and civic activities and is also the focal point of a distinctive historic residential neighborhood. Among the notable buildings are the Silliman House, (corner of Beach and Old Post Roads on the site of the Parsonage Lot) built by Major Silliman, taken prisoner by the British in the Revolutionary War; the house at 249 Beach Road that survived the British Raid of 1779; the "house with sixty-inch closets," which served as the Congregational parsonage for many years and was built by Judge Roger Minot Sherman, nephew of Roger Sherman and an associate justice of the Connecticut Supreme Court.

The green's historic significance is attested to by the buildings situated on it. Its appearance has been protected to some extent by the fact that the town's business center shifted north to the railroad depot area after 1849. A trolley ran by the green starting in 1894, carrying beach-bound day trippers. In the 1920s, the construction of Route 1 paralleling the railroad also served to protect the green, discouraging 20th-century business development.

Fairfield Town Hall Green was created as part of the town plan in 1639 and has remained the focal point of civic and religious activity. The historic structures around it relate to the green historically, functionally and visually, and many are associated with important historic personages.