Gabber, Techno, Hardcore, Drum & Base, GOA (maybe) are all "aquired" tastes of music. You are totally right, Gabber and Hardcore especially are not that much catchy, nor are they easy to listen to for the none hooked on ear. Hell, my first exposure to "RAVEs" about a decade ago, was through hardcore, Techno/Gabber, and i still find it hard to listen to that stuff for more than 2 hours without developing a headache. Imagine listening to heavymetal with all its subgeneres like Deathmetal, speedmetal etc... i bet if u get a non fan listen to it for first time.. he will say the same thing. Some thing are just an "aquired" taste like i mentioned that u gotta listen to repeatedly untill the whole style grows on u and u end up appreciating what it offers. Trance/house are definately more "poppy" to the ear, and therefore, end up being "mainstream".
As far as the drug argument, im sorry buddy but thats the most ignorant argument ever to associate a drug with a music genre. I'd expect someone with an EDM background like yourself to be less ignorant about the issue

. some of the biggest addicts to trance have never even tried rolling on E, and some of the biggest addicts to Hardcore, Gabber never Candy Flipped.
BEATS PER MINUTE OFFICIAL CRACK FIEND.
its about time to set the BPM ON FIRE.....
"It wasn't God that did this to us man, it was you... you fucking fat narcotics agent!"
Have you watched: "Fear and Loathing in Las Vegas on WEED man?"