Six consecutive batters reached safely before Cabrera drove a fastball into left-center for his first game-ending hit since he signed with Chicago over the winter. Alexei Ramirez doubled home two runs and Gordon Beckham had a tying RBI single.
David Robertson (1-0) worked a perfect ninth for his first win with the White Sox. Allen (0-2) recorded just one out in his first blown save in four chances.
Chicago had managed just four hits before the big finish.
Trevor Bauer pitched seven sparkling innings for Cleveland, which has dropped seven of nine. Ryan Raburn and Brett Hayes homered, and Michael Brantley had an RBI double.