The microbiome is increasingly recognized for its role in multiple aspects of cancer development and treatment, specifically in response to checkpoint inhibitors. While checkpoint inhibitors have revolutionized cancer treatment by producing durable anti-tumor responses, only a minority of patients respond to the available immunotherapy drugs and accurate, sensitive and specific microbiome predictors of response to treatment remain elusive. Additionally, the specific mechanisms linking the microbiome and host immunological responses remain unclear. In this review, we examine the evidence for the gut microbiome's association with anti-tumor responses to checkpoint inhibitors in the treatment of melanoma, non-small cell lung cancer, and renal cell carcinoma. Furthermore, we discuss the current evidence available from murine models seeking to explain the immunological mechanisms that may drive this process. While this work is promising in defining the impact of gut microbiota in cancer treatment, many unanswered questions indicate the need for additional human and experimental studies.