"Many Brits don't treat it as a working holiday anymore. It's more a three-year career break," says Dan McConnell, general manager of the Mad Monkey backpacker hostel chain.
He says Brits used to be the largest clientele at its Cairns accommodation but "now French outnumber British three to one".
"We see the Brits get to Sydney, decide it's the coolest place in the world, and stay there," he adds.
It's a privilege not enjoyed by working holidaymakers from other countries who must do at least 88 days of "specified regional work" in industries that need a seasonal or temporary workforce - including agriculture, fisheries, tourism and hospitality - if they want to quality for a second year extension, and six months to stay a third year.
Barely a month ago, Albanese ruled out forcing British backpackers to do the same, accepting it as part of the trade deal.
But, under pressure from businesses here in Far North Queensland and other regional Australian communities, who are concerned about a lack of labour, the mood music has changed.
Last week Albanese met UK counterpart Andy Burnham at the meeting of Commonwealth leaders in the Caribbean.
And alongside AI, gender violence online and the AUKUS defence pact, backpackers were on the agenda.
Despite speculation in the Australian media, no immediate announcement was made. Albanese instead told reporters that he had pushed for a change in the rules on backpackers and that Burnham had been "very constructive about it".
"We'll work those issues through over coming weeks," he said adding that the free trade agreement was not set in stone and tweaks could be made "if there are good policy reasons".
Backpackers make up one in seven farm jobs across Australia, with about 52,000 people on working holiday visas doing jobs in horticulture each year, according to the National Farmers Federation.
It welcomed Albanese's discussions with Burnham, but said the issue of worker shortages in agriculture was far broader than the UK deal.
Of greater concern are wider proposals, announced in September, to cut net migration into Australia, under political pressure from the resurgent right wing populist One Nation party and the opposition coalition.
That includes an overhaul of the working holiday visa system, introducing a ballot for backpackers wanting to extend their stay – capped at 45,000 for people wanting a second year in Australia (down from 57,000 who qualified this year) and just 5,000 allowed to remain for a third year (compared with 31,000 this year).
This, the federation says, is "a low blow".
Those plans are still out for consultation, but immigration officials have slowed processing rates for some working holiday visa applications and paused them from several countries.
Tourism Tropical North Queensland chief executive Mark Olsen has also been among those lobbying Albanese in Canberra - telling him that backpackers brought both the spending power and the temporary labour needed to fill seasonal roles in a region famed as a gateway to the Great Barrier Reef and the Unesco world heritage listed Daintree rainforest.
Young Brits temporarily living and working in this region peaked at around 20,000 he says. Now it's closer to 11,000.
"Numbers have dropped significantly since the UK free trade agreement kicked in when border arrangements returned to normal [after the pandemic] in 2023 ," he said.