Hereford's Ben Smith slotted home as the Magpies got off to the worst possible start.
In form Guianan doubled the Bulls lead on 26 minutes in what looked like a miserable night in store for County.
Thompson's men went in two down at the break, with the visitors deservedly ahead.
Sloppy defending had cost the hosts dearly.
Notts substitute Gary Silk reduced the deficit on 64 minutes, as the Notts clawed their way back into the game.
But, with the game wide open Guinan restored the Bulls two goal lead just six minutes later.
Hector Sam again gave Notts hope but it was not enough to deny Hereford their fourth away win of the season.
Victory moves United up to third in the league ladder, two points adrift of table toppers MK Dons.
While defeat sees County slide back to fifthteenth.