May and September offer the best balance: mild, dryish weather and thin enough crowds that you can actually enjoy the town.
Late April and early October are the shoulder sweet spots. You'll find 15-20% cheaper rooms, still decent weather for walking the…
Go in May for the best weather and emptiness, or September for lower costs and the food festival.