Founded in 1908 as a women's college, James Madison University was established by the Virginia General Assembly. It was originally called The State Normal and Industrial School for Women at Harrisonburg. In 1914, the name of the university was changed to the State Normal School for Women at Harrisonburg. At first, academic offerings included only today's equivalent of technical training or junior college courses, but authorization to award bachelor's degrees was granted in 1916. During this initial period of development, the campus plan was established and six buildings were constructed.

The university became the State Teachers College at Harrisonburg in 1924 and continued under that name until 1938, when it was named Madison College in honor of James Madison, the fourth President of the United States, whose Montpelier estate is in nearby Orange, Virginia. In 1976, the university's name was changed to James Madison University.

The first president of the university was Julian Ashby Burruss. The university opened its doors to its first student body in 1909 with an enrollment of 209 students and a faculty of 15. Its first 20 graduates received diplomas in 1911.

In 1919, Burruss resigned the presidency to become president of Virginia Polytechnic Institute. Samuel Page Duke was then chosen as the school's second president. During Duke's administration, nine major buildings were constructed. Duke served as president from 1919 to 1949.

In 1946, men were first enrolled as regular day students. G. Tyler Miller became the third president in 1949, following Duke's retirement. During Miller's administration, from 1949 to 1970, the campus was enlarged by 240 acres (0.97 km²) and 19 buildings were constructed. Major curriculum changes were made and the university was authorized to grant master's degrees in 1954.