The 1874 frame building given by railroad executive, F.H. Peabody, to express his appreciation for the town being named for him was the first free library in the State of Kansas, and is listed on the National Register of Historic Places. He gave 2,000 volumes from the Boston Library. Peabody also gave a reading table and chairs which are still in use.

The original building with its cloak room, reading room with ornate wood ceiling and stack room once sat facing Walnut street on the corner of Walnut and Division. Local citizens raised $200.00 by private subscription for the 1886 fountain which "made a dazzling display of water, butterflies, and flowers."

The building was moved to various locations after the Carnegie Library was built and became the home of the Women's Relief Corps. The Historical Society was formed in 1961 and moved the building to its present site. The Poe doll collection and precipio are main features. The Complex includes the Museum, the Peabody Printing Museum, the Morgan House, and the Carnegie Library.