Despite being once again hit by injuries, the Beavers couldn't quite turn over the first leg deficit, but they gave Ebbsfleet an almighty scare.
With doubts over Duncan Culley's fitness appearing in a prematch fitness test, Dowse had to shuffle the team about and resorted to playing Nathan Collier in the lone striker role.
Once again, like the first leg, it was a tight affair neither side creating a clear chance until the half hour mark with Aaron McLean skying his shot well over the bar with the goal at his mercy.
On the far side, Anthony Cook was not getting much joy out of the robust Hampton defence, but won a free kick after the referee adjudged he had been fouled by Christian Jolley.
The free kick into the box was met by Kenny Clark, whose header looked to have been cleared off the line but it was adjudged to have crossed the line before Drury got there to prod the rebound home.
Just before half time, the Beavers had their best chance of the half as Brendan Kiernan stepped up to clip a free kick towards the top corner, which keeper Nathan Ashmore had to claw away from in front of the post.
Immediate substitutions at the start of the second half saw Federico and Hippolyte-Patrick brought on in place of Stivan Rivic and Tom Jelley, and the Beavers started to try and take the game to the Fleet.
Ebbsfleet were struggling to make inroads into the Hampton back four, as they were restricted to long range shots by the stolid central pair of Murphy and Hicks and just after the hour, the Beavers got their reward.
Federico latched onto a clever ball down the left channel, and showed his pace. He cut inside and fired a shot at goal which probably took a slight deflection to beat Ashmore and go into the bottom corner.
Ebbsfleet were clearly rattled by this, Drury and Rance booked in quick succession shortly afterwards, and they knew that the game was back on as a contest. Their retort was to bring McLean off and bring on Darren McQueen.
To try and force open the last 10 minutes, Dowse played his last card, giving Culley a cameo role up front in what at times was a 3 man forward line, keeping Ebbsfleet honest.
With 3 minutes left, Sam Howes looked to have kept the game alive as a contest with a great save at the feet of Kedwell, but with literally seconds to go, the Beavers hearts were broken utterly.
A throw in was flicked on up and over the defensive line, and McQueen judged his movement to the second, picking up the ball; advancing on Howes, his lob over the keeper was judged perfectly and bounced into the net, leaving the Beavers beaten but unbowed and defiant to the end.
Speaking the day after the match, Dowse was clearly feeling down, but still had praise for the players.
"I've barely spoken to anyone today as I'm proper gutted over that, as I really thought we could do it yesterday and to go down like that, well, its all hard to take."
"Credit to the team out there though. I couldn't ask anymore of them, injuries were played through and players had to play out of position. For them to do that for the club in a playoff is credit to their character."
"To see how much it meant to them, look at Sam Howes; he looked almost in tears after the final whistle, they wanted to win it so much."
"Anyway, good luck to Ebbsfleet in the final but I wish it was us."