Mirriam Webster reports:
"Since hoi polloi is a transliteration of the Greek for "the many," some critics have asserted that the phrase should not be preceded by the. They find "the hoi polloi" to be redundant, equivalent to "the the many"—an opinion that fails to recognize that hoi means nothing at all in English. Nonetheless, the opinion has influenced the omission of the in the usage of some writers. But most writers use the, which is normal English grammar."
The Wikipedia article on the subject is also interesting, with most English writers cited (eg Darwin, Byron, WS Gilbert) opting for the "the".
The first citations in the OED are from 1668 (Dryden) and 1791 (Wordsworth) both of whom are on team "the." So those who are wrong to use "the" are in august company, anyway.