Rovers U18's score three goals in four minutes to book a place in the FA Youth Cup Second Round.
The young Rovers side had to battle from being a goal down at the break, as a Harry Parsons penalty put the hosts ahead in the first half.
Rovers showed great attacking threat in the second half and scored three goals in the space of four minutes, with captain Ollie Hulbert scoring first, with a quickfire brace from Kieran Phillips racing Rovers ahead and allowing the U18's to book a place in the next stage of the competition.
Defender Luca Hoole was included on the teamsheet for the U18's, with Ollie Hulbert selected as captain for the First Round tie, with the young Rovers side looking to build on victory on the weekend over Forest Green Rovers.
It was the hosts who took the lead in the opening stages, as Luca Hoole was judged to have fouled Anthony Lynn inside the box, with the referee wasting no time in point to the spot to award the penalty.
Harry Parsons took the spot-kick and blasted his effort down the middle to put Swindon ahead with 11 minutes on the clock.
Rovers had chances to respond as the first-half progressed. Sam Heal saw his set-piece blocked by the wall on the edge of the area, as Zain Walker forced the Swindon 'keeper into action, with a low effort that required a smart stop.
Both Kinsly Murray and Kieran Phillips looked sharp when going forward, but Rovers were unable to threaten the Swindon goal with any further clearcut chances as the first-half progressed
Murray saw his right-footed shoot steer wide of goal after being played through by Joe Budd, as Rovers continued to go in search of the equaliser before the break.
Toby Holland had the opportunity to add a second goal for the hosts as the half-time interval approached, but the midfielder's effort from the edge of the box climbed high over the Rovers goal, with good pressure from Luca Hoole doing enough to prevent the Swindon man from rushing closer to the Rovers goal.
As the second half started, Rovers started brightly with Ollie Hulbert going close with an effort from outside the area that brushed the roof of the net.
That started a string of chances for Rovers, with Kieran Phillips knocking the ball around the Swindon 'keeper and seeing his finish stopped on the line by the covering Harrison Minturn.
From the resulting corner, Kinsly Murray worked himself into a pocket of space in the area and hit a low effort towards goal that was deflected away inside the congested penalty box.
While chasing the equaliser, Rovers still had to be alert at the back to ensure the hosts didn't stretch their lead to two goals. Sonny Hayes tried his luck with a speculative effort from the corner of the Rovers area, with Harry Thomas-Barker aware to track the shot and make a diving save to tip the effort away from goal.
As the hour mark came, Rovers still continued their attacking threat. Kieran Phillips saw a curling effort flash inches wide of the far-post and minutes later saw a free-kick deflected away from the target.
The perseverance in attack paid off with twenty minutes remaining, as Rovers finally found a reward for their attacking play. It was the skipper Ollie Hulbert who scored the goal that tied the game, being alert to drive home a finish into the bottom corner after the ball rebounded to his feet ten yards out from goal.
Rovers were lifted by Hulbert's goal and quickly found themselves ahead, as a cross to near post picked out Kieran Phillips, with the striker connecting with a first time finish to dispatch a close-range volley to put Rovers ahead for the first time in the tie.
Phillips didn't have to wait long for his second goal, as less than a minute later substitute Isiah Crawford picked out the striker in the box and the forward kept his cool to add his second goal in as many minutes.
Rovers saw out the remainder of the game comfortably and saw little threat on their goal to dent their two-goal cushion to book their place in the next round.
Rovers: Harry Thomas-Barker (G), Luca Hoole, Mason Raymond, Zain Walker, Kieran Phillips, Ollie Hulburt (C) (72' Isiah Crawford), Sam Heal, Joe Budd, Adam Biss, Henry Njonjo, Kingsly Murray