The ruling is the latest blow to Prime Minister Manmohan Singh's government, which has fought off a string of high-profile corruption scandals.
P.J. Thomas resigned as central vigilance commissioner after Thursday's Supreme Court ruling, Law Minister Veerappa Moily said.
Thomas, who was appointed in September, and several other former officials in southern Kerala state are accused of causing losses to the state by accepting a higher-priced contract for the import of palm oil decades ago.
The opposition Bharatiya Janata Party opposed Thomas' appointment, saying the position should go to someone of impeccable integrity.
However, Singh's government went ahead with the appointment, which was subsequently challenged in court.
In their ruling, Supreme Court Justices K.S. Radhakrishnan and Swantantra Kumar severely criticized the government for not considering the pending criminal case against Thomas.
Moily said Singh could not be criticized for Thomas' appointment because the civil servant had been cleared by his predecessor, a decision the court rejected as irrelevant.
Rajiv Pratap Rudy, a BJP spokesman, said the ruling was "the biggest blow to Prime Minister Singh and his government. The larger issue, that this government is a part of all the corruption and misgovernance, has been proven by the court ruling."
Singh's government has faced a series of corruption scandals ranging from the country's hosting of the Commonwealth Games last year to the irregular sale of cell phone spectrum in 2008 that reportedly cost the government tens of billions of dollars.
Though the government faces no immediate threat, the scandals have dominated Indian politics for months at a time when Singh has sought to present the country as a rising international power.
The entire winter session of Parliament was paralyzed by opposition demands for a probe into the telecommunications scandal. Singh initially refused to allow an investigation, but subsequently bowed to opposition pressure and accepted the demand last week.