According to the urban dictionary:
Jungle fever is-
1. A white person dating a person of color.
2. An inter-racial relationship, namely between a white person and a black person.
3. When a non-black person is atracted sexually to black people. Originally it was used for when a white woman dates black men, but now it could refer to a white man who thinks black women are hot and wants to date them.
A Stevie Wonder song about black man/white woman love.
"She's gon black guy crazy
He's gone white girl hazy
They got jungle fever"--Stevie Wonder
There's more, so here's the link:
http://www.urbandictionary.com/define.php?term=jungle+fever
And don't feel bad. I never thought it was a derogatory term either. It was meant to be funny in the movie.