Short for "English as
Second Language", the term actually refers to *people* who use english as their
second language. These people are often falsely thought of as morons, or generally a person of lower degree/intelligence. They often use their own version of English, mixing in some of their own words or their first
language's grammar, especially when talking to other ESL, even more so when the other
ESL's first language is the same as his own (when they are speaking English to each other for whatever weird reason, that is.)
Generally ESLs can understand English decently, especially when an ESL version is spoken by another ESL. Their first languages needn't be the same, but it would help even more.
Between the english-speaking community, however, it has a derogatory meaning.