Patients must not be receiving any other investigational anti-cancer agents while on study Patients must not be planning treatment with other systemic anti-cancer agents (e.g., chemotherapy, hormonal therapy, immunotherapy) or other treatments not part of protocol-specified anti-cancer therapy including concurrent investigational agents of any type Previous anti-cancer chemotherapy, immunotherapy or investigational agents =< days prior to registration Receiving any other investigational agents or any other anti-cancer therapy Has received prior systemic anti-cancer therapy including investigational agents for the current malignancy prior to randomization/allocation. Has received prior systemic anti-cancer therapy including investigational agents within weeks prior to study start Patients may not be receiving any systemic anti-cancer agents or other investigational agents during radiation therapy Patients who are receiving any other anti-cancer or investigational drug therapy are ineligible Subjects who require warfarin, anti-cancer therapeutics or investigational agents Has received prior systemic anti-cancer therapy including investigational agents within weeks prior to study enrollment Receiving any other investigational agents or any other anti-cancer therapy Patients may not be receiving the concomitant administration of any systemic therapy, biologic therapy, or other agents with anti-tumor activity against prostate cancer while the patients are on study. Participants who are receiving any other investigational agents, or any other anti-cancer therapy during study treatment Prior anti-tumor therapy within weeks of cycle day (anti-tumor therapy defined as, but is not limited to, anti-cancer agents (cytotoxic chemotherapy, immunotherapy, and biologic therapy), radiotherapy, and investigational agents), the wash-out period Participants who are receiving any other investigational agents or have received investigational therapy or any anti-cancer monoclonal antibody (mAB) within weeks prior to the st dose of pembrolizumab Receiving other anti-cancer or investigational therapy during study treatment, apart from those agents specified in the study protocol Patients receiving concurrent anti-cancer treatment (chemotherapy, investigational agents, immunotherapy, endocrine therapy, or Optune) Patients may not be receiving any other investigational and/or anti-cancer agents Patients must NOT receive other anti-cancer agents while on study For purposes of this protocol, anti-tumor treatment may be defined as, but is not limited to, anti-cancer agents (cytotoxic chemotherapy, immunotherapy, biologic therapy), radiotherapy, and investigational agents; an investigational agent is any drug or therapy not currently approved for use in humans\r\n* Anti-cancer agents: anti-cancer agents are not permitted within days prior to start of POPI; there are no limitations on the type or number of prior regimens; hormonal therapy and trastuzumab are permitted during POPI\r\n* Radiation: prior treatment with breast irradiation is not allowed \r\n* Surgery: incident breast biopsies only permitted prior to POPI to confirm residual disease after NAC Patients receiving any other investigational agents or anti-cancer agents within days other than mibefradil dihydrochloride, hypofractionated RT, or corticosteroids as described in this clinical protocol during treatment Concomitant therapy with any anti-cancer agents, other investigational anti-cancer therapies, or immunosuppressive agents including but not limited to methotrexate, chloroquine, azathioprine, etc. within six months of study participation Any anti-cancer therapy or investigational agents =< weeks prior to registration Concomitant therapy with any anti-cancer agents, other investigational anti-cancer therapies, or immunosuppressive agents; chronic use of systemic corticosteroids Participants who are receiving any other study or off protocol anti-cancer agents Previous anti-cancer chemotherapy, immunotherapy or investigational agents < weeks prior to the first day of study defined treatment Patients must not be planning to receive other anti-cancer agents including investigational agents while on protocol treatment More than prior systemic anti-cancer therapies (e.g. cytotoxic agents, biologic agents) regimens for metastatic disease No concomitant approved anti-cancer therapies or any investigational agents Patients with any number of prior therapies with anti-angiogenic agents or immunotherapy with the exception of any previous anti-CTLA- directed agents are allowed; a week washout period is required for all agents, except for bevacizumab where a week washout is required Patients who are receiving any other anti-cancer or investigational drug therapy are excluded Concurrent anti-cancer therapy, including chemotherapy agents, targeted agents, or biological agents not otherwise specified in this protocol Patients receiving any other investigational agents or any additional anti-cancer agents Patients may not be receiving any other investigational agents; patients cannot be receiving concomitant chemotherapy, radiotherapy, experimental therapy or any other therapy for the purposes of anti-cancer treatment Treatment with systemic biologic therapy for prostate cancer (other than approved bone targeted agents and GnRH-analogue therapy) or other agents with anti-tumor activity within weeks of enrollment (day visit) Other concomitant anti-cancer therapy agents excepts steroids Anti-tumor therapy (i.e. chemotherapeutics or investigational agents or immunotherapy) within weeks prior to enrollment Other concomitant anti-cancer therapy agents except steroids Concurrent chemotherapy, or targeted anti-cancer agents, other than hydroxyurea Concurrent use of other anti-cancer therapy including chemotherapy agents, targeted agents, or biological agents not otherwise specified in this protocol Patients may not be receiving any other investigational agents or any concomitant antineoplastic therapy, with the exception of androgen ablating agents (for patients with prior prostate cancer) Patients receiving any other investigational anti-cancer agents Receiving any other investigational agents or any other anti-cancer therapy No concurrent anti-cancer treatment (including topical agents such as imiquimod) or investigational agents All cancer patients receiving chemotherapy agents Has received prior systemic anti-cancer therapy including investigational agents for the current malignancy. Received treatment with chemotherapy, wide-field radiation, or anti-cancer biologic therapy including investigational agents within days of study entry. Current receipt of any other investigational agents or any additional anti-cancer agents for this or any other disease Concurrent use of other anti-cancer therapy, including chemotherapy agents, targeted agents, biological agents, immunotherapy, or investigational agents not otherwise specified in this protocol Receipt of any other investigational agents or any additional anti-cancer agents Anti-cancer agents: must not be receiving other anti-cancer agents at time of enrollment and must not be planning to take other anti-cancer agents during DLT period