Patients must have completed all chemotherapy prior to surgery; sandwich chemotherapy is not allowed (i.e. chemotherapy planned to be given after surgery); patients must have completed at least 4 cycles of neoadjuvant chemotherapy consisting of an anthracycline and/or taxane-based regimen without evidence of disease progression in the breast or the lymph nodes\r\n* Note: Delays/dose modifications due to toxicities/adverse events are allowed as long as a minimum of 4 cycles of neoadjuvant chemotherapy is administered; more than 4 cycles of neoadjuvant chemotherapy (NAC) may be administered at the discretion of the treating medical oncologist All patients must have a clinically negative axilla (no bulky adenopathy) on physical examination documented at the completion of neoadjuvant chemotherapy\r\n* Note: an ultrasound of the axilla is not required at completion of neoadjuvant chemotherapy; if performed, its findings do NOT impact eligibility No more than 8 weeks of neoadjuvant endocrine therapy prior to the start of neoadjuvant chemotherapy No neoadjuvant radiation therapy No SLN surgery/excisional biopsy for pathological confirmation of axillary status prior to or during neoadjuvant chemotherapy Patients must have completed neoadjuvant taxane +/- anthracycline; patients must NOT have received cisplatin or carboplatin or capecitabine as part of their neoadjuvant therapy regimen\r\n* NOTE: Patients who received preoperative therapy as part of a clinical trial may enroll\r\n* NOTE: Patients that were not able to complete their planned neoadjuvant chemotherapy for any reason (i.e. toxicities, etc.) are eligible to participate as long as no further systemic standard of care therapy is planned by the treating physician No neoadjuvant chemotherapy =< 4 weeks before pre-registration Patients with HER2-positive tumors must have received neoadjuvant anti-HER2 therapy (either with all or with a portion of the neoadjuvant chemotherapy regimen), unless medically contraindicated Patients must have had neoadjuvant chemotherapy followed by surgery; the recommended neoadjuvant treatment should include 16-24 weeks of a third generation chemotherapy regimen as recommended by National Comprehensive Cancer Network (NCCN) guidelines for triple negative breast cancer (examples include dose dense adriamycin-cytoxan [AC] followed by dose-dense paclitaxel; weekly paclitaxel x 12 followed or preceded by cyclophosphamide-adriamycin-fluorouracil [FAC], fluorouracil-epirubicin-cytoxan [FEC], AC or dose dense AC; docetaxel either followed or preceded by FEC/FAC or AC; carboplatin-containing neoadjuvant chemotherapy is also allowed); patients who cannot complete all planned treatment cycles for any reason are considered high risk and therefore are eligible for the study if they have residual disease; patients must have resolution of adverse event(s) of the most recent prior chemotherapy to grade 1 or less, except alopecia and =< grade 2 neuropathy which are allowed STEP 1: NEOADJUVANT Patient must have received at least two cycles of triplet neoadjuvant therapy (all three drugs) during step 1 Patient must be registered to step 2 no less than 21 days and no more than 90 days after the end of their final cycle of neoadjuvant therapy One of the breast disease stages listed below:\r\n* Note: In the definitions below, definitive surgery is defined as the final surgery performed to obtain clear surgical margins\r\n* Neoadjuvant chemotherapy was not administered\r\n** If neoadjuvant chemotherapy was NOT administered, pathologic staging must be T1-3, N1-2a following definitive surgery\r\n* Neoadjuvant chemotherapy was administered\r\n** If prior to initiation of neoadjuvant chemotherapy clinical staging was T1-3, N0, pathologic staging must be T1-3, N1-2a following definitive surgery\r\n** If prior to initiation of neoadjuvant chemotherapy clinical staging was T1-3, N1, pathologic staging must be T0-3, N0-2a following definitive surgery\r\n** If prior to initiation of neoadjuvant chemotherapy there was cytologic or pathologic confirmation of axillary nodal involvement (per any of the criteria listed below), pathologic staging must be T0-3, N0-2a following definitive surgery\r\n*** Positive fine-needle aspiration (FNA) (ie, demonstrating malignant cells)\r\n*** Positive core needle biopsy (ie, demonstrating invasive adenocarcinoma)\r\n*** Positive sentinel lymph node biopsy (ie, demonstrating invasive adenocarcinoma) Prior neoadjuvant chemo or biologic therapy for current breast cancer within 4 weeks prior to planned surgery [Period 1] Candidate for neoadjuvant chemotherapy Patient Received neoadjuvant chemoradiation (4-10 weeks prior to surgery) Subjects with MIBC not meeting the above criteria are still eligible provided the patient declines neoadjuvant cisplatin-based chemotherapy, after specific informed consent describing the known benefits of cisplatin-based chemotherapy Plans for administration of neoadjuvant chemotherapy or hormonal therapy Eligible for neoadjuvant chemotherapy Subjects undergoing neoadjuvant chemotherapy or neoadjuvant endocrine therapy Must not have received nor be planned for neoadjuvant chemotherapy prior to SABR or surgery Subjects must not have received or plan to receive neoadjuvant chemotherapy either before radiotherapy or before surgery Chemotherapy is planned for the patient in the neoadjuvant setting Received neoadjuvant chemotherapy Plan for neoadjuvant chemoradiation A female patient who is a woman of child-bearing potential (WCBP) and a male patient with a partner who is a WCBP must agree to use a medically accepted method for preventing pregnancy for the duration of immunotherapy and neoadjuvant chemotherapy and until after completion of breast surgery or, for patients who do not receive neoadjuvant chemotherapy, for a minimum of 4 months following the last dose of pembrolizumab Prior treatment with neoadjuvant therapy NEOADJUVANT COHORT Consideration for neoadjuvant therapy Planned neoadjuvant chemotherapy for breast cancer as determined by the judgment of the medical oncologist A disposition to neoadjuvant chemotherapy with planned interval tumor reductive surgery after 3 complete cycles of treatment No prior therapy for current breast cancer and not planning neoadjuvant chemotherapy Contraindication to repeat breast biopsy (neoadjuvant chemotherapy group) Patients who have received any non-anti-folate containing neoadjuvant or systemic chemotherapy are eligible; any prior intravesical therapy, or immunotherapy is allowed Patients must have an intact evaluable primary tumor or biopsy proven axillary node involvement with at least 1.0 centimeter (cm) smallest dimension based on imaging after neoadjuvant anthracycline-based chemotherapy and prior to initiation of neoadjuvant chemotherapy under this protocol; baseline measurements and evaluations must be obtained within 4 weeks of registration to the study; all areas of disease should be recorded in order to assess response and uniformity of response to therapy Inoperable breast cancer even after neoadjuvant treatment as assessed by the treating surgical oncologist Participants with preoperatively staged T3, N1, or N2 tumors who return > 12 weeks following completion of neoadjuvant chemoradiation therapy (CRT) Neoadjuvant chemotherapy Patients who were exposed to neoadjuvant chemotherapy or chemotherapy after prostatectomy Received chemotherapy, radiotherapy, or biologic therapy within the last 30 days (neoadjuvant chemotherapy excluded) Received chemotherapy, radiotherapy, or biologic therapy within the last 30 days (neoadjuvant chemotherapy excluded) Prior or planned neoadjuvant systemic therapy for breast cancer Neoadjuvant chemotherapy will be permitted prior to the initiation of SBRT radiation therapy on this clinical trial; there must be a minimum of 2 weeks between the completion of any neoadjuvant chemotherapy and the beginning of SBRT radiation therapy; furthermore, restaging must be done prior to registration to ensure that patients remain resectable Patients who will receive neoadjuvant chemotherapy are not eligible Patients treated with neoadjuvant chemotherapy are not eligible Neoadjuvant chemotherapy for current malignancy Patient must have a medical oncology consult and be recommended to receive or are currently receiving neoadjuvant chemotherapy for a stage IIB through IV carcinoma EXCLUSION CRITERIA FOR REGISTRATION: subjects receiving neoadjuvant chemotherapy for whom interval debulking surgery (assuming adequate response to therapy) is not planned Planned neoadjuvant chemoRT treatment must conform to NCCN guidelines. Other than the 3 cycles of neoadjuvant chemotherapy and surgery (mentioned above), must not have received other treatment for their gastric cancer. Patient has at least one of the following:\r\n* Pathologic N1-3\r\n* Pathologic T3\r\n* Neoadjuvant chemotherapy and did not achieve pathologic response at time of surgery Have had an R0/R1 resection of PDA following neoadjuvant chemotherapy Any cytotoxic treatments, such as neoadjuvant chemotherapy, planned before subsequent surgical procedure Patients must have received neoadjuvant chemotherapy of at least 16 weeks. This period must include 6 weeks of a taxane -containing neoadjuvant therapy (Exception: For patients with progressive disease that occurred after at least 6 weeks of taxane-containing neoadjuvant treatment, a total treatment period of less than 16 weeks is also eligible). Neoadjuvant chemotherapy before or after prostatectomy Not be receiving neoadjuvant hormonal or chemotherapy (other clinical trials) NEOADJUVANT COHORT Prior neoadjuvant FOLFIRINOX Undergone neoadjuvant chemotherapy Neoadjuvant chemotherapy Any neoadjuvant chemotherapy Patients who have received neoadjuvant chemotherapy or neoadjuvant hormonal therapy for the current cancer Neoadjuvant radiation therapy or chemotherapy for cervical cancer Complete pathologic response to neoadjuvant chemotherapy (NAC). Patients must not have received neoadjuvant chemotherapy for the present disease Completed all planned therapy for T >= 1.0 cm, Nx, M0 or Tx, N >= N1, M0 triple negative breast cancer (TNBC) (American Joint Committee on Cancer, 7th edition) meeting the following guidelines:\r\n* Received neoadjuvant chemotherapy and/or completed adjuvant chemotherapy with or without radiation; (the patient may have had adjuvant and/or neoadjuvant chemotherapy for their disease); patients who received neoadjuvant chemotherapy may have either residual disease or a complete response; adjuvant/neoadjuvant chemotherapy regimens must include at least 4 cycles of a standard chemotherapy regimen, and generally this should include one of the generally accepted standard regimens (including but not limited to: doxorubicin hydrochloride, cyclophosphamide and paclitaxel [AC-T], Taxotere and cyclophosphamide [TC], doxorubicin hydrochloride and cyclophosphamide [AC], or cyclophosphamide, methotrexate and fluorouracil [CMF]); for patients who received their standard chemotherapy as part of a clinical trial, the regimen should include at least 4 cycles of therapy; patients who initiate planned chemotherapy but discontinue before receiving 4 cycles due to toxicity will be eligible\r\n* All planned radiation therapy and surgery for the treatment of the current cancer should be complete (not including plastic or reconstructive surgery)\r\n* Patients with local-regional recurrence without evidence of distant metastases (no definite stage IV disease) who are treated with curative intent may be eligible following completion of all surgery and/or chemotherapy and/or radiotherapy; such patients must have no evidence of residual disease by standard clinical and radiological examination (per investigator discretion) following completion of curative intent treatment Prior non-cisplatin based neoadjuvant systemic chemotherapy for urothelial carcinoma (prior intravesical chemotherapy or immunotherapy is permissible) Prior cisplatin based neoadjuvant systemic chemotherapy for more than 4 cycles Women undergoing neoadjuvant chemotherapy are not eligible For subjects receiving neoadjuvant therapy only, time between start of neoadjuvant treatment and randomization must be ? 8 weeks and subjects must be scheduled to undergo definitive treatment (including surgery and/or radiotherapy) with curative intent within approximately 9 months of starting neoadjuvant treatment Patient who received neoadjuvant chemotherapy for ovarian cancer Patients with muscle invasive bladder cancer (MIBC) must have never received and currently be ineligible for cisplatin-based neoadjuvant chemotherapy due to any of the following:\r\n* Calculated creatinine clearance of < 60 ml/min\r\n* Karnofsky performance status (KPS) < 80\r\n* Solitary kidney or\r\n* Patient refusal to undergo neoadjuvant chemotherapy Neoadjuvant chemotherapy if relapse occurred at least 6 months after surgical resection; Patient must have recovered sufficiently from any adverse effects of neoadjuvant treatment Patient must have recovered sufficiently from any adverse effects of neoadjuvant treatment If the subject received neoadjuvant therapy, residual tumor is present (to be determined by the primary surgeon) Prior neoadjuvant chemotherapy or biologic therapy for current clinically or biopsy-proven node positive breast cancer within 4 weeks before the planned surgery. Patients who will receive neoadjuvant therapy prior to definitive surgery The subject is not a candidate for neoadjuvant chemoradiation therapy (i.e., patients with borderline resectable pancreatic ductal adenocarcinoma who are known to benefit from neoadjuvant treatment regimens) Scheduled to undergo neoadjuvant or adjuvant chemotherapy or neoadjuvant endocrine therapy Received neoadjuvant chemotherapy, any autoimmune or immunological disease or taking any immune suppression drugs Planned neoadjuvant systemic therapy Subject with neoadjuvant chemotherapy or chemoradiation Scheduled for neoadjuvant chemotherapy and/or chemoradiation for pancreatic cancer Subjects medically eligible for neoadjuvant cisplatin-based combination chemotherapy who refuse this therapeutic option and understand the risks and benefits of doing so. A disposition to neoadjuvant chemotherapy with planned interval tumor reductive surgery after 4 complete cycles of treatment Neoadjuvant chemotherapy or radiation therapy Undergoing induction or neoadjuvant chemotherapy prior to radiotherapy Women who are planned to receive neoadjuvant therapy Patient did not undergo either placement of a preoperative biliary stent/drain or neoadjuvant chemotherapy with or without radiation therapy The patient is a candidate for surgical resection, with no planned neoadjuvant chemotherapy, and there is a planned surgery for the primary colorectal cancer Breast cancer completely excised, or patient receiving neoadjuvant therapy prior to surgery Patient must be able to undergo stereotactic-vacuum-assisted breast biopsy with clip placement after completion of neoadjuvant chemotherapy Patients with HER2-positive tumors must have received neoadjuvant anti-HER2 therapy (either with all or with a portion of the neoadjuvant chemotherapy regimen), unless medically contraindicated Patients treated with neoadjuvant hormonal therapy only are not eligible Patients who did not undergo trimodality imaging after completion of neoadjuvant chemotherapy (breast ultrasound, MRI, and mammography) Patients receiving neoadjuvant chemotherapy or recently completed neoadjuvant chemotherapy and will undergo surgery within 6 weeks are eligible Prescribed neoadjuvant chemotherapy for breast cancer Participants who enroll in a preoperative therapy trial or who have been treated with neoadjuvant chemotherapy Women undergoing neoadjuvant chemotherapy Patients will be treated with neoadjuvant chemoradiotherapy for this condition Received neoadjuvant chemotherapy prior to surgical resection Any systemic neoadjuvant (or preoperative) therapy between the core biopsy and lumpectomy Participants who are candidates for neoadjuvant chemotherapy or neoadjuvant endocrine therapy for the treatment of newly diagnosed, invasive breast cancer Neoadjuvant chemotherapy or radiation therapy prior to prostatectomy Patients must have a known site of disease; please note, for patients undergoing neoadjuvant therapy, this requirement must be met retrospectively prior to the start of neoadjuvant therapy; patients who are in radiological/clinical remission after neoadjuvant therapy, prior to infusion of radiolabeled antibody, are still eligible Use of neoadjuvant hormonal manipulation Be scheduled for neoadjuvant chemotherapy Pathologically confirmed breast cancer, determined to be a candidate for primary systemic (neoadjuvant) therapy and for surgical resection of residual primary tumor following completion of neoadjuvant therapy Women scheduled to receive neoadjuvant chemotherapy as part of their treatment plan Will be receiving neoadjuvant chemotherapy for breast cancer as prescribed by their oncologist Receiving neoadjuvant chemotherapy Neoadjuvant chemoradiation prior to resection is planned Neoadjuvant chemotherapy Candidate for neoadjuvant chemotherapy No neoadjuvant endocrine therapy or chemotherapy within 12 months Patients diagnosed with bladder cancer with planned cystectomy and urinary diversion, +/- neoadjuvant chemotherapy