Diamonds started well and on six minutes Petr Mikolanda had a golden opportunity, but the striker completely mis-hit his shot from 12 yards out.
On 24 minutes Tyrone Berry made a fine run down the right but his cross just evaded the on-coming Mikolanda.
County looked very disappointing and they didn't have a shot on target until the 32nd minute, when Julien Baudet's effort was well held by Scott Tynan.
Rushden struck in the 33rd minute when Drewe Broughton flicked the ball on to Mikolanda, who from 30 yards out hit a piledriver over Kevin Pilkington with the keeper slightly off his line.
County went close on 39 minutes when Mike Edwards saw his free-kick deflected inches over.
The second half followed much the same pattern with Rushden creating the better chances. On 51 minutes Pilkington did well to hold on to Darren Caskey's header.
On the hour mark 17-year-old Rushden sub Lee Tomlin saw his goalbound shot deflected for a corner.
On 68 minutes Marcus Kelly made a good run down the left and crossed to the unmarked Berry, who from four yards out shot wide.
County pressed hard for an equaliser but the Rushden defence remained firm, though they had a scare in the 88th minute when captain Rob Gier was forced to clear off the line from County sub Mark De Bolla.
Rushden should have sealed their win in the 90th minute when Tomlin failed to beat Pilkington after a one-on one with the keeper.