The city council set aside 5.5 acres of a former prune orchard for roses on November 20, 1927, at the urging of the Santa Clara County Rose Society, which supplied the plants. Ground was broken in 1931 and the garden was dedicated on April 7, 1937. Today it holds more than 3,500 rose shrubs of 189 varieties.
Budget cuts in the early 2000s put its standing as an All-America Rose Selections display garden on probation. Volunteers formed the Friends of the San Jose Rose Garden in 2007 and helped restore it, and in 2010 the AARS named it America’s Best Rose Garden in the first nationwide contest of its kind. It is a city historic landmark.