Why can't the US stop soaring oil and gas prices?

Texas oilman Jason Herrick is scrambling to pump more oil, chasing the promise of profit as oil prices soar.

But despite his best efforts, he suspects output from his family-owned company will fall this year, for the third year in a row.

It's been years since his Pantera Energy has invested in new production, after money dried up when energy prices plunged - famously dropping below zero at one point - early in the pandemic.

Now, like many other firms across the US and global economy, he's facing major delays as he hunts for supplies and staff to make the projects happen.

"Our job is to try to produce as much as we can and we've done that," he says. "We just are so behind and we're having a difficult time catching up."

It's just one of the signs coming from the US - the world's biggest oil and gas producer - that the high energy prices hitting households may be here to stay.

Since the start of 2021, prices for oil and gas have jumped roughly two-fold or more, as demand roars back from the shock of 2020 Covid lockdowns.

The war in Ukraine, which has pushed countries in the West to shun Russian energy supplies, has only sharpened their climb.

As the market heats up, forecasts suggest US production will increase by about one million barrels per day this year.

But that's less than 10%, not enough to meet the rise in demand - and a far cry from the response the last time prices were this high, in 2014, when US oil output jumped 20% and the fracking revolution was in high gear.

The muted response reflects companies constrained by rising costs and shortages of key materials, equipment and staff - as well as by longer term questions from investors about how much oil and gas will be needed as the world tries to move away from polluting fossil fuels in response to climate change.

Mike Wendt, an engineer at oil exploration firm Lone Star Productions, says his firm has several new projects in the works, but hasn't been able to find the steel pipes it needs, leading to delays and rising costs.

"We're going to be drilling as quickly as we can, but we have all these supply and demand issues, and that's kind of throttling the market," he says.

Trey Cowan, oil and gas analyst at the Institute for Energy Economics and Financial Analysis, says the challenges facing companies in North America are unprecedented.

And while many small private companies, like Mr Wendt's, are still pushing to boost output, the biggest firms have so far stuck with investment plans forged before prices really soared, choosing to put the windfalls primarily toward shareholder payouts, rather than extra production.

That approach marks a major shift, which analysts say reflects pressure from Wall Street as climate change reshapes views of the energy sector.

Investors are pushing companies to share profits now, instead of making long term investments, given uncertainty over demand as the world pushes to shift away from fossil fuels, says Raoul LeBlanc, vice president at S&P Global.

"The market is worried that the demand won't be there and these assets will be stranded," he says. "If there's no long term value in my stock price, it means I have to pay a very aggressive dividend."

With prices expected to stay high, Mr LeBlanc says firms are likely to boost investment - at least a bit.

"It's not going back to the go-go days," he says. But "the Ukraine war highlighted that we still live in a world of fossil fuels. The energy transition - it hasn't been thrown away, but the conversation has been rebalanced to include what we need in the short and medium term."

In the political arena, that "rebalancing" has raised fears in the US that President Joe Biden, who made combatting climate change a centrepiece of his campaign, will stop pushing as hard for environmental rules to curb emissions and speed the transition to renewable energy.

While he has kept up calls for investments in renewable energy, his most ambitious proposals remain blocked with little sign of breakthrough.

As Republicans blame his environmental agenda for the high energy costs and his approval ratings sink, he has signed deals to boost US natural gas exports, released oil from national reserves and approved hundreds of permits for drilling on government land.

"We see the Biden administration really wavering on its earlier climate commitments," says Robert Rozansky research analyst at Global Energy Monitor, which tracks energy projects around the world.

Such a shift is not unique to the US.

Western leaders, including Mr Biden and UK Prime Minister Boris Johnson, have pleaded for help from Saudi Arabia and other producers, which analysts say have the capacity to boost output without major new investments.

In Canada, the fourth largest oil producer after the US, Saudi Arabia and Russia, the liberal government has renewed talks about getting long-stalled oil and gas projects, like terminals to export natural gas to Europe, off the ground.

"It's very much a 180 [degree shift] in mindset," says Alfred Sorenson, chief executive of Pieridae Energy, which shelved its plans for a liquefied natural gas terminal in Nova Scotia last year after struggling to find investors.

But it is now eyeing a revival after a signal that government might be newly willing to offer more support.

Whether the shifting political winds can overcome market doubts remains an open question - especially given ongoing public opposition to oil and gas infrastructure like new terminals and pipelines, which can tie up projects in legal limbo for years.

"Politicians and industry are now really flogging the idea that we must save Europe with our oil and gas," says James Gunvalden Klaassen, a lawyer at EcoJustice, which has sued over aspects of the Pieridae project.

"We're resorting to 20th century solutions... and we already know that those don't work," he says.

The Pieridae natural gas terminal, for example, would not be ready to ship natural gas until 2027 - doing little to address the current crisis, he says,

"They way to achieve energy security in Europe and Canada and the US is to develop sustainable energy sources," he says. "That can be done and .... should be, must be."