Rovers defeated on the road at the Pirelli.
Edwards and Akins' late goals inside the final ten minutes provided a frustrating finish to the encounter for Rovers, who created a number of half-chances to claim the points themselves.
In a even first-half, chances were few and far between. Anssi Jaakkola produced a fine stop on close-range effort from Liam Boyce, that was perhaps the pick of a tight and tentative first half.
Rovers had a number of second-half opportunities, as Luke Leahy and Tyler Smith came close to breaking the deadlock, but Rovers fell victim to late sucker punches from the home side.
As the Gas were pushing for a late goal to record the win, it was the hosts that scored inside the final five minute, as Akins turned inside the box and placed a finish around Jaakkola to give Burton the win.
Ryan Edwards broke free deep into added time, as Rovers were piling on the pressure, to calmly stroke the ball through Jaakkola's legs and secure the victory.
In naming his squad starting XI for the fixture, Rovers boss Graham Coughlan opted to utilise the same eleven that started last weekend's victory over Oxford United.
That meant that Jonson Clarke-Harris and Tyler Smith were paired together up front, after the duo started Tuesday's Carabao Cup tie on the bench.
It was the hosts that assembled the first opening of the game, coming inside the opening ten minutes. Ryan Edwards was able to get to a near-post cross and flicked the delivery over Jaakkola, seeing the ball worked to the unmarked Atkins at the far post. Just six-yards out, the striker's close-range header took a deflection off Leahy that saw the effort loop over the crossbar, in what was an early let-off for Rovers.
As the first-half progressed, Rovers grew into the contest. Alfie Kilgour chalked the first noticeable chance for the Gas, as the defender wrangled his way free to be able to get a header on a deep cross from Tom Davies, but Kilgour's attempt ran wide of goal.
Minutes later, Kilgour turned the provider for Tyler Smith, clipping a smart pass into the feet of the striker who had found a pocket of space in between Buxton and Edwards. The striker was allowed to take a touch and opted to take an early shot that flashed away from goal.
On the half-hour mark, Jaakkola was called in action and provided a fine stop to prevent the home side taking the lead. It was a long-range effort from Scott Fraser that caused the danger, as the shot was deflected into the path of Liam Boyce. The striker looked destined for the net from close-range, Jaakkola'st outstretched stopped the effort tucking into the far corner, with the goalkeeper already committed to diving in the opposite direction to stop Fraser's original effort.
There were calls for the hosts to be reduced to ten men shortly after, as a hectic tackle from Ryan Edwards caught Luke Leahy late, but after much deliberation, the referee deemed that tackle only worthy of a caution.
Rovers ended the half brightly, as Ollie Clarke fed a low cross into the mix that almost dropped for Tyler Smith, but O'Hara in the Burton goal was alert to dart off his line to push the cross to safety. Moments later, a high cross from Ed Upson swung towards Clarke-Harris, as the forward beat his man in the air and was able to glance a header towards goal that O'Hara held.
The momentum was carried forward by Coughlan's men in the early stages of the second half, as the Gas created a number of significant chances on goal.
The first came as Jonson Clarke-Harris curled a smart effort towards goal that had O'Hara concerned, with Burton 'keeper only able to push Clarke-Harris' attempt into the path of Luke Leahy, who rushed a first-time effort wide of goal.
It was Clarke-Harris involved again shortly after Leahy's opportunity, as a powerful, low effort from the striker took a heavy deflection off Tyler Smith, which altered the path of the ball and required a reaction stop from O'Hara to keep the ball out of the net.
O'Hara was kept busy in the opening ten minutes of the second half, as the Burton 'keeper was forced to tip an inswinging corner from Luke Leahy away from goal, as the set-piece from the defender crept towards goal.
There was no let-up for O'Hara, who had to be equal to a low-effort from Alfie Kilgour from 30-yards out that was on target, with the 'keeper able to watch the ball into his glutches through what was a congested penalty area.
Chances were now piling up for Rovers, as Abu Ogogo took aim from outside the area, as the midfielder waited patiently for a clearance to fall, but shifted his effort wide of the upright.
With twenty minutes left in the contest, Tom Nichols was introduced from the bench. The striker was involved from the off, chasing down a ball over top and looking to smash a volley towards goal, only for his effort to be deflected over the target.
Akins was able to turn inside the Rovers penalty area and carve out a shooting opportunity, taking full advantage and lifting a finish inside the far-post that swerved past Anssi Jaakkola in the net. As it flew in, the term 'smash and grab' came to mind.
As Rovers were chasing a late equaliser, the hosts were able to break and add a second goal through Ryan Edwards. With most of the Gas up for an attack, when the ball was hit over the top, Edwards alert to chase it down and head towards goal. When he reached Jaakkola, he acted with calmness, placing it between the Fin's legs and crushing Rovers' hopes.
In truth, neither side particularly took this game by the scruff of the neck, but it was Burton who came away with the three points, courtesy of two late goals.
The Gas now look forward to Plymouth Argyle away, in the first game in their Leasing.com campaign
Rovers: Anssi Jaakkola (GK), Josh Hare, Alfie Kilgour, Tom Davies, Tony Craig, Luke Leahy, Abu Ogogo, Ollie Clarke, Ed Upson, Tyler Smith (70' Tom Nichols), Jonson Clarke-Harris.
Subs: Jordi van Stappershoef, Michael Kelly, Mark Little, Alex Rodman, Kyle Bennett, Tom Nichols, Victor Adeboyejo.
Burton Albion: Kieran O'Hara, John Brayford, Ryan Edwards, Jake Buxton (57' Richard Nartey), Kieran Wallace, Stephen Quinn, Scott Fraser, Lucas Akins, Oliver Sarkic (70' Joe Sbarra), John-Joe O'Toole, Liam Boyce (75' David Templeton).
Subs: Stephen Bywater, David Templeton, Joe Sbarra, Reece Hutchinson, Richard Nartey, Jevan Anderson, Chris Beardsley.
Attendance: 2831 (640 away)