The release, described as a "goodwill gesture", comes days after President Asif Ali Zardari met Indian PM Manmohan Singh for lunch in New Delhi on the first visit by a Pakistani head of state to their arch-rivals in seven years.
"We have released 26 Indian fishermen from our jail on the instructions of the government," Nazeer Husain Shah, superintendent of the Malir district prison in Karachi told AFP.
Officials India has over 150 Pakistani fishermen in its prisons.
Shazia Marri, the information minister for Sindh, said: "They have been released as a goodwill gesture."
Jail officials handed gifts to the Indians before boarding them onto a bus that will take them to the eastern city of Lahore, where they will be handed over to the authorities across the border.
AFP/APP