The bank said Chad had also failed to use revenues on health and education.
It said Chad had paid the outstanding balance of $65.7m (£37m) under a $140m loan agreement, after talks with the government of President Idriss Deby.
Chad called the decision "practically consensual" and that relations would continue in non-oil sectors.
The pipeline was seen as a test case for how Africa's oil wealth could benefit the poor if spent properly.
The central African country is expected to earn about $1.4bn in oil revenues this year.
"Regrettably, it became evident that the arrangements that had underpinned the bank's involvement in the Chad-Cameroon pipeline project were not working," said Michel Wormser, the bank's director of operations for Africa.
"The bank therefore concluded that it could not continue to support this project under these circumstances."
But he said future cooperation was possible "if the government of Chad wishes to focus its energies on a programme to support inclusive development to overcome poverty, assist displaced people and improve governance".