Cancer immunotherapy with antibodies targeting immune checkpoints such as the PD-1/PD-L1 pathway have emerged as breakthrough treatment for multiple solid tumors with high response rates and durable remissions. Despite the benefit for patients and encouraging safety profile, severe inflammatory reactions are observed in some patients. Such immune-related adverse events (irAEs) frequently lead to temporary or permanent cessation of the treatment and require systemic immunosuppression yet underlying mechanisms of irAEs are not known in detail. Here, we describe the T cell-mediated immune reaction in irAE lesions of four patients that developed pneumonitis during therapy with a PD-1 blocking antibody. Immunohistochemical analysis was performed to map the environment of the inflammatory lesions. Tumor infiltrating T cell clones were identified by sequencing the T cell receptor, and comparison with clones from peripheral blood or secondary lymphoid organs. A significant overlap of clones infiltrating irAE lesions and tumors was found. The most prevalent clones were also expanded in peripheral blood, but only a minor fraction of clonal overlap was found. Our findings suggest that irAE lesions in patients under PD-1 blockade are infiltrated by T cells with similar specificity as tumor-infiltrating T cells. These results raise the possibility that the immune response is elicited in these patients against antigens shared by the tumor and distant organs affected by irAEs.