Hatters did, indeed, go mad. They inhaled fumes from the mercury that was part of the process of making felt hats. Not recognizing the violent twitching and derangement as symptoms of a brain disorder, people made fun of affected hatmakers, often treating them as drunkards. In the U.S., the condition was called the "Danbury shakes." (Danbury, Connecticut, was a hatmaking center.) Mercury is no longer used in the felting process: hatmaking -- and hatmakers -- are safe.