Why does my buñuelos syrup keep crystallizing?

Every time I make buñuelos syrup it starts out perfect but then gets all grainy and crystallized after a few minutes. Am I cooking it too long or is there something I should be adding to keep it smooth?