Notts County finished with just 10 men after Glynn Hurst's 70th minute dismissal while the visitors also had midfielder Shane McFaul carried off.
Hurst received a straight red card from Mike Pike for using foul and abusive language.
The former Chesterfield and Stockport striker believes the Magpies should have had a penalty for the challenge of McFaul who was taken to hospital for a neck injury.
But there was real consternation as McFaul lay unmoved on the ground for over six minutes, receiving treatment from the Notts club doctor and a host of first-aiders.
County created the better chances and Bury were fortunate to hang on for the win courtesy of Chris Porter's 22-minute goal.
Porter struck after a goalmouth melee with Saul Deeney just failing to keep out the ball following Porter's shot from eight yards.
After that it was all Notts County with Hurst hitting the bar and seeing Andy Marriott producing a stunning save after 36 minutes.
Bury might have doubled their advantage at the start of the second period but David Nugent fired over and Dwayne Mattis fired into the side netting.
Despite the one-man deficit County kept going and it required another fine save by Marriott to deny Paul Bolland in the seven minutes of stoppage time.