The 20th anniversary of the September 11 terrorist attacks on New York and Washington carries with it two punctuation marks.

The first are the attacks themselves on the World Trade Centre and the Pentagon. These shocked a country that was both psychologically and physically unprepared for such a brazen assault on American soil.

The second are the chaotic events of the past several weeks, in which America was forced to admit its two-decade mission in Afghanistan was, to all intents and purposes, an expensive – more than US$2 trillion ($A2.68 trillion) – failure.

The merciless truck bombing on August 26 near Kabul airport, in which 13 US military personnel died as well as scores of Afghan civilians, underscored the futility of an unwinnable conflict in a country that has resisted outside influence for thousands of years.

One of the most pressing issues now is where America stands in the Middle East, where its power and influence have receded as a consequence of its ill-fated decision to invade Iraq before completing its mission in Afghanistan.

The two disasters cannot be separated, since they were driven by a global policy enacted by the George W. Bush administration under the rubric of the “war on terror”.

This was used to justify a series of decisions that led to American forces and their allies being mired in post-9/11 conflicts in Afghanistan, Pakistan, Iraq, Yemen and elsewhere. Together, they have cost more than 7,000 American lives, an estimated 900,000 war dead and expenditure of US$8 trillion (A$10.73 trillion).

This does not include 30-40 million refugees who have been displaced in various conflicts across the Middle East and beyond. There are more to come judging by the rush in Afghanistan to escape the Taliban.

William Galston of the Brookings Institution puts the case colourfully for the degree of self-harm America has inflicted on itself as a consequence of flawed decisions made two decades ago in the heat of the moment.

In his long war against America, Osama bin Laden has won a sweeping if posthumous victory. The US reaction to the 9/11 attack he masterminded is like the cytokine storm that can occur when COVID-19 attacks us: the defensive measures our bodies mount go too far and damage the vital organs our antibodies were meant to protect.

So to what extent will a battered America will step back from the region, and what this might imply for shifting balances of power in a combustible part of the world?

The short answer is: we don’t know. However, it is clear the era of American engagement in the volatile Middle East has run its course.

America invaded Afghanistan in 2001 in partnership with its NATO allies and countries like Australia. The aim was to eradicate al-Qaeda and the Taliban. Twenty years later, the Taliban is back, although its control of a tribalised and inherently unstable country will be tenuous at best.

Various terrorism franchises like al-Qaeda and Islamic State, in the guise of its local affiliate ISIS-K, will inevitably find a home amid the likely chaos.

America’s strategic interests in the region, and its ability to influence events, will be pared back. These security imperatives, driven partly by its economic interests, could be briefly summarised as follows.

stability in the Gulf, through which two-thirds of the world’s tradeable oil passes each day

maintenance of an uneasy status quo in the eastern Mediterranean, where the American Sixth Fleet is positioned and Russian naval influence is growing

guarantor of Israel’s security

a commitment to prevent Iran gaining a nuclear capability.

Beyond these imperatives, it is hard to see America becoming involved on the ground again in the Middle East unless there was no other choice from a national security standpoint.

No American president would savour the task of making the case for a recommitment of ground forces that would again put his citizens’ lives in harm’s way.

The question then becomes: what are the implications for the wider Middle East of a paring back of American engagement and a further erosion in Washington’s authority?

This is the new post-9/11 Middle East in which various players are asserting themselves.

Iran emerged as the main beneficiary of the Iraq misadventure. Its Shiite surrogates are now in power in Baghdad and its influence regionally has spread.

Russia and China are taking advantage of America’s failures to assert themselves.

Moscow’s custodial relationship with Bashar al-Assad’s regime in Syria enabled Russia to extend its sphere of influence down into the Mediterranean, where it has been granted naval facilities at the Syrian port of Tartous.

China has its own commercial and strategic interests in the region. It is, for example, a significant importer of Iranian crude oil. Both Russia and China have engaged in naval exercises with Iran in the Gulf region as an affront to the US presence there.

Where this leaves the Gulf states is unpredictable. While an American security umbrella remains in place against more egregious pressures from Iran, hereditary rulers will be discomforted by indications of dwindling American power in the region.

In recent years, Saudi Arabia has begun to balance its strategic options by drawing closer to Russia.

This might all be described in years to come as part of a new great game.

In the meantime, the biggest challenge in the Middle East for the bruised Biden administration is to persuade Iran, under its new hard-line president, to return to the negotiating table to revive the nuclear deal of 2015, which then US President Donald Trump abandoned in 2019.

Early indications are that Iran is reluctant to go back to the terms of the 2015 agreement. Instead, it would favour one that accommodates progress it has made in enriching weapons-grade uranium in its nuclear program since Trump rescinded America’s participation.

Reviving the nuclear deal is critical to forestall Iran’s continuing progress towards a nuclear capability. The last thing the world needs right now is a Middle East nuclear arms race, or an Israeli pre-emptive strike against Iranian nuclear facilities.

These are just some of the more pressing global challenges in the Middle East, but there are many others of a lesser order. These remain a threat to regional stability and present risks of further refugee exoduses.

Syria’s long-running civil war is not over; the Israel-Palestine issue remains a festering sore; Lebanon has taken on many of the characteristics of a failed state; Yemen stands as an affront to the world’s humanitarian conscience; Libya remains in the grip of internal conflict; and Iraq is hardly stable.

In other words, the Middle East two decades on from 9/11 is far from achieving the sort of ideals laid out by those promoting the “war on terror” and its “nation-building” aspirations.

As Galston writes in his Brookings essay:

At the end of the 20th century, the United States bestrode the world like a colossus. September 11 changed all this. Our excessive focus on the Middle East diverted us from the geopolitical forces that were reshaping the world. Now we must face the consequences with a weakened hand.

That conclusion puts things in perspective from an American point of view in a volatile post-9/11 Middle East.