Thanks to those 2 comments. Although placing Xft.dpi: 96 in my ~/.Xdefaults did not help, but I managed to do it through the appearance control center applet. I had to purge the gtk-qt theme engine because it was freezing the dialog and setting it via gconf-editor didn't work as expected.
I had to fight with my fonts this morning but here's my configuration. Maybe it'll be useful to anyone:
DejaVu sans book 12
greyscale smoothing, subpixel slight RGB.
I'm not yet satisfied but it's more usable now.
Thanks Phaeronix for the help too.