An offensive song was said to be aimed at Manchester United defender Rio, who pulled out of the England squad, and QPR centre-back Anton.
"You expect and accept banter from fans on the terraces as it's part of what makes the game great," Rio tweeted.
"But racism is not banter and from your own fans - WOW."
However, the 34-year-old said "Always a small minority who ruin it for others."
He later wrote: "Let's not jump to conclusions and assume though, as it might just have been banter. We'll see after the investigation."
Ferdinand's comments came after the football anti-racism network Football Against Racism in Europe (Fare) filed a complaint to world governing body Fifa on Thursday. Fare did not have its own observers at the match on Friday, 22 March and said it had pulled together evidence, including "media comment".
The claims that some England fans sang a song suggesting the brothers should be burned on a bonfire were first reported in the Daily Mirror on Wednesday.
It has been claimed the song had racist overtones because Anton was racially abused by former England captain John Terry.
Fare executive director Piara Power said he felt the song had an "undercurrent of race" but added it was "certainly an usual report".
In theory, if Fifa find England fans guilty of racism - even against one of their own players - England could be subject to sanction - this could be a fine or even having to play behind closed doors.