The US administration has not approved the transfer but is considering it as an incentive for the militants to enter negotiations in Afghanistan.
None of the five inmates is accused of directly killing Americans.
They would be re-united with their families in Qatar, which is playing an increasing role in negotiations.
They reportedly agreed to the transfer when they met Afghan government officials who visited the US prison this week on a mission from President Hamid Karzai.
Correspondents point out that the visit to the US prison on Cuba would not have been possible without US approval.