Dravida Munnetra Kazhagham, the party that rules southern Tamil Nadu state, said it is willing to meet with the Congress party to discuss a dispute over the number of seats the two parties plan to contest in state assembly elections next month.
Analysts say the breach also stems from a corruption scandal in which a former telecommunications minister belonging to the DMK has been arrested.
Andimuthu Raja, the former minister, is accused of selling licenses to use mobile phone frequencies at throwaway prices in 2008, causing the government an estimated $36 billion in losses by not holding an auction.
Raja's arrest and searches of his home and a television network owned by the DMK have strained relations between the government and its ally.
The DMK, which controls 18 seats in Parliament, said late Saturday it would pull its ministers from the Cabinet by Monday.
"The DMK has decided to withdraw from the government because the Congress does not seem interested to continue the alliance for the coming state assembly elections," the party said in a statement. It said it would continue to support the government on certain issues.
While Singh's government is not in imminent danger of falling, with other regional parties offering to step into the breach created by the DMK, the ally's decision is another embarrassment for an administration that has been hit by many setbacks in recent weeks.
On Thursday, the country's top anti-corruption official was forced to resign after the Supreme Court ruled that his appointment was inappropriate because he faces graft charges in a decades-old case.
The entire last session of Parliament was paralyzed by opposition demands for a probe into the telecoms scandal. Singh initially refused to allow an investigation, but subsequently bowed to opposition pressure and accepted the demand.