He does indeed say that line, just before he and Gerti get into their Dragonfly sub, and before Carmen and Juni get into theirs. And specifically, it's "Agent", not "Spy". :)
Much later in the movie, after Gerti hijacked the Transmooker so it bursted, and after the President, Alexandra, and the Secret Service show on the island, Gary says to Carmen "Looks like I lost this round, but I'll be back", but afterwards it was Carmen who says "Hopefully you're on the good side next time". Gary then says "We'll see, but you never know". And that's when the BAD BOY text in the glasses show. Gerti then steps in and says "Don't worry. He gets out of line, I'll straighten him out."
Gary is indeed interesting but irritating, and I honestly find Gerti more good than her brother.