Histone dacetylases (HDACs) are a group of enzymes that remove acetyl groups from histones and regulate expression of tumor suppressor genes. They are implicated in many human diseases, especially cancer, making them a promising therapeutic target for treatment of the latter by developing a wide variety of inhibitors. HDAC inhibitors interfere with HDAC activity and regulate biological events, such as cell cycle, differentiation and apoptosis in cancer cells. As a result, HDAC inhibitor-based therapies have gained much attention for cancer treatment. To date, the FDA has approved three HDAC inhibitors for cutaneous/peripheral T-cell lymphoma and many more HDAC inhibitors are in different stages of clinical development for the treatment of hematological malignancies as well as solid tumors. In the intensifying efforts to discover new, hopefully more therapeutically efficacious HDAC inhibitors, molecular modeling-based rational drug design has played an important role in identifying potential inhibitors that vary in molecular structures and properties. In this review, we summarize four major structural classes of HDAC inhibitors that are in clinical trials and different computer modeling tools available for their structural modifications as a guide to discover additional HDAC inhibitors with greater therapeutic utility.