Patients must not have been treated with any of the following prior to step initial registration:\r\n* Cetuximab, panitumumab, or any other monoclonal antibody against EGFR or inhibitor of EGFR\r\n* HER- targeting for treatment of colorectal cancer; patients who have received prior trastuzumab or pertuzumab for other indications such as prior history of adjuvant or neoadjuvant breast cancer treatment prior to the development of advanced colorectal cancer are eligible No prior chemotherapy for metastatic colorectal cancer Has pathologically documented unresectable, recurrent, or metastatic colorectal adenocarcinoma (until sponsor's notification to the study sites, subject must be a RAS/BRAF wild-type cancer) Colorectal cancer other than adenocarcinoma, e.g., sarcoma, lymphoma, carcinoid. The subject has a histologic or cytologic diagnosis of colorectal adenocarcinoma that is metastatic or unresectable and is refractory to or progressed (or relapsed) following a fluoropyrimidine, irinotecan, oxaliplatin, and bevacizumab; prior epidermal growth factor inhibitor therapy is required for patients with left-sided, RAS wild-type tumors; prior Food and Drug Administration (FDA)-approved PD- inhibitor therapy is required for patients with microsatellite instability high (MSI-H) colorectal cancer. Prior regorafenib or TAS- treatment is not required Must have previously treated metastatic colorectal cancer BRAF V mutant colorectal cancer Malignant melanoma, papillary thyroid cancer, colorectal cancer, or hematologic malignancy including multiple myeloma Metastatic non-small cell lung cancer (NSCLC), metastatic urothelial carcinoma or microsatellite instability high metastatic colorectal cancer (except for patients enrolling based on elevated TMB); For patients enrolling based on elevated TMB, patients with metastatic NSCLC, metastatic urothelial carcinoma or metastatic colorectal cancer will be allowed to enroll Pts with colorectal cancer must be KRAS wild-type Malignancies other than colorectal cancer within years prior to Cycle Day Currently receiving other systemic therapy for metastatic colorectal cancer COHORT B, GROUP : COLORECTAL CANCER: Patients must have colorectal adenocarcinoma that harbored a KRAS mutation COHORT B, GROUP : COLORECTAL CANCER: Patients must have failed a minimum of one previous line of chemotherapy Recurrent and/or metastatic unresectable colorectal cancer with hepatic metastases Metastatic colorectal cancer patients have progressed following at least one line of fluorouracil (-FU)-based chemotherapy High microsatellite instability (MSI-H) colorectal cancer patients must have received an approved PD- targeted agent prior to enrolling in this trial The study is intended to enroll patients with pancreatic and colorectal cancer; patients with other types of solid tumors will require approval by the principal investigator Metastatic colorectal cancer with mismatch repair deficiency (MMR-D or microsatellite instability [MSI]-high) Patients previously treated with systemic chemotherapy and/or biologic agents for colorectal cancer are eligible Have histologically confirmed microsatellite stable metastatic colorectal cancer and have received at least one line of treatment for metastatic colorectal cancer including fluoropyrimidines, oxaliplatin and/or irinotecan Participants must have metastatic colorectal or pancreatic cancer Prior history of colorectal cancer Prior chemotherapy or radiotherapy for colorectal cancer Previously treated with at least one prior irinotecan-containing regimen for colorectal cancer. Colorectal cancer Previously treated with at least one prior irinotecan-containing regimen for colorectal cancer Any prior systemic treatment for metastatic colorectal cancer Adjuvant systemic treatment for colorectal cancer within last months Patients with colorectal cancer should have failed at least one oxaliplatin-containing regimen Subjects with advanced colorectal, gastric, hepatic or pancreatic cancer Has received prior systemic therapy for Stage IV colorectal cancer. May have received prior adjuvant chemotherapy for colorectal cancer as long as it was completed at least months prior to randomization on this study Subject is diagnosed with colorectal cancer Clinical diagnosis of appendiceal or colorectal neoplasm with peritoneal mucinosis or metastasis Part B: First-Line Colorectal Cancer Part B: Second-Line Colorectal Cancer Tumor blocks available from previous surgery/biopsy; at the tumor specific expansion, only patients with metastatic colorectal and renal cell cancers will be enrolled; patients with metastatic colorectal and renal cancer must have been treated and progressed or intolerant to standard care therapy; patients with colorectal cancer must have been treated in the past with irinotecan and/or oxaliplatin and/or avastin/EGFR therapy or intolerant to these agents; no more than lines of therapy permitted in the metastatic setting; patients with colorectal cancer may enroll irrespective of K-Ras mutational status, although this will be documented; patients with renal cell cancer must have been treated with a VEGF targeted therapy and/or mTOR inhibitor; prior treatment with vorinostat and HCQ are not permitted in each tumor type Patients with wild-type or mutant Kirsten rat sarcoma viral oncogene homolog (KRAS) metastatic colorectal cancer (mCRC) histopathologically confirmed advanced or metastatic colorectal cancer, excluding primary tumors of appendiceal origin patients with refractory colorectal cancer. Known personal history of > adenomatous colorectal polyps or a personal history of adenomatous colorectal polyp(s) > centimeters (cm) in size Kirsten rat sarcoma viral oncogene homolog (KRAS) mutated colorectal cancer Metatastic colorectal cancer Colorectal cancer other than adenocarcinoma, e.g., sarcoma, lymphoma, carcinoid. Group : Patients with BRAF mutated colorectal cancer Patients to be enrolled in the Cetuximab or Panitumumab combination arms must have colorectal cancer which is K-Ras wild-type. Metastatic colorectal cancer Family history of familial cancer syndromes (leukemia, breast, ovarian, colorectal, etc.) Patients with a history of colorectal cancer treated by surgical resection who develop radiological or clinical evidence of metastatic cancer do not require separate histological or cytological confirmation of metastatic disease unless:\r\n* Either an interval of > years has elapsed between the primary surgery and the development of metastatic disease, OR\r\n* The primary cancer was stage I\r\nClinicians should consider biopsy of lesions to establish the diagnosis of metastatic colorectal cancer in each case if there is substantial clinical ambiguity regarding the nature or source of apparent metastases Prior chemotherapy, other systemic therapy, or any investigational agent for treatment of advanced or metastatic colorectal cancer; patients who completed adjuvant or neoadjuvant chemotherapy > months prior to colorectal cancer recurrence are eligible Part F: Colorectal Cancer Patients must have a primary L sided colorectal cancer (at or distal to the splenic flexure) Have a colorectal or coloanal reconstruction with or without reservoir/pouch. Advanced colorectal carcinoma. Patients must show signs of progression (by imaging, or tumor marker elevation) after being treated with a first line or greater treatment for their un-resectable or metastatic colorectal cancer Any prior treatment for metastatic colorectal cancer, except for use of palliative radiosensitizers Adjuvant chemotherapy for colorectal cancer completed < months prior to study consent Evaluable or measurable radiographic evidence of colorectal cancer Has received at least prior chemotherapeutic regimens for colorectal cancer; Diagnosed colorectal cancer with oligometastatic colorectal cancer in the lung Prior anti-cancer treatment for metastatic colorectal cancer Received any chemotherapeutic or targeted agent for metastatic colorectal cancer within two weeks of initiation of study drug Subjects with metastatic colorectal cancer may continue maintenance therapy with capecitabine and/or bevacizumab Colorectal cancer (for patients enrolled to expansion part) Prior treatment with taxane therapy for either colorectal cancer or small bowel adenocarcinoma Primary colorectal cancer diagnosis Prior chemotherapy for metastatic colorectal cancer is not allowed. metastatic colorectal cancer (mCRC) that is microsatellite stable (MSS) PATIENT: Diagnosis of lung or gastrointestinal (GI) (colorectal, pancreas, liver) cancers PHASE : Previously underwent curative surgical treatment of metastatic colorectal or peritoneal cancer at University of Pittsburgh Medical Center (UPMC) Shadyside PHASE & : Scheduled for curative surgical treatment of metastatic colorectal or peritoneal cancer at UPMC Shadyside Diagnosed with stage or breast, cervical, colorectal, endometrial, hepatobiliary, lung, melanoma, gynecological, prostate cancer in the past six months A personal history of colorectal cancer Scheduled to receive first-line intravenous chemotherapy treatment for colorectal cancer (stages II-IV) Patients with a primary diagnosis of either breast, lung, prostate, or colorectal cancer within the last two years Within months of completing active treatment for colorectal cancer Patient diagnosed with colorectal cancer Diagnosis of advanced (metastatic or recurrent) lung, breast, colorectal, prostate, or gynecological (GYN), or other solid tumor cancer; the symptom cluster of pain, fatigue, and sleep disturbance is common in these patients Patients with a history of other malignancies will not be excluded, as long as they are currently receiving treatment for lung, breast, colorectal, prostate, GYN, or other solid tumor cancer Histologically confirmed stage I-III colorectal or breast cancer who have undergone curative-intent complete surgical resection and completed all adjuvant cytotoxic chemotherapy and radiation (if indicated) at least months prior to enrollment; breast cancer patients on hormonal therapy or trastuzumab only therapy and colorectal cancer patients on adjunctive therapies not considered cytotoxic chemotherapy (including those participating in Cancer and Leukemia Group B [CALGB] receiving only celecoxib/placebo) are eligible Scheduled to receive any form of further adjuvant cancer therapy (except hormonal or biologic therapy for breast cancer or adjunctive noncytotoxic chemotherapy for colorectal cancer including participation in CALGB while on celecoxib/placebo) Prisoners and individuals who are under the age of will be specifically excluded from participation in the study; individuals must have a primary colorectal or endometrial cancer, not a recurrence of a previous colorectal or endometrial cancer Have a first degree relative with a history of breast or colorectal cancer Outpatients scheduled for screening or surveillance colonoscopy for polyps or colorectal cancer ) Patients at high risk of colorectal cancer e.g. ulcerative colitis Patients must not have a known history of familial adenomatous polyposis, hereditary nonpolyposis colorectal cancer, or inflammatory bowel disease Diagnosis of colorectal adenoma of any grade Patients receiving prior chemotherapy or chemoradiation for colorectal cancer (ie, neoadjuvant chemoradiation for stage II or III rectal cancer) History of colorectal cancer; exception: individuals with stage I or II colorectal cancer who have not received any chemotherapy Chinese and Korean American patients who are not up to date for colorectal cancer screening Colorectal cancer At high risk of developing colorectal cancer, based upon a history of having a colonoscopy and having any colorectal adenoma diagnosed and/or removed within the past years No history of colorectal cancer, including germ-line heritable colorectal cancers such as familial adenomatous polyposis (FAP) and hereditary nonpolyposis colorectal cancer (HNPCC) History of at least one of the following conditions in the previous months:\r\n* Colorectal adenoma(s) >= cm in maximal diameter\r\n* Colorectal adenoma(s) with villous or tubulovillous histology\r\n* Colorectal adenoma(s) with high grade (severe) dysplasia History of any colorectal cancer History of heritable cancer syndrome (familial adenomatous polyposis [FAP], hereditary nonpolyposis colorectal cancer [HNPCC]) Currently non-adherent to colorectal cancer screening Previously resected colorectal cancer Family history of polyposis syndrome (e.g., familial adenomatous polyposis [FAP], hereditary non-polyposis colorectal cancer [HNPCC]) or colorectal cancer (first degree relatives younger than years old) History of difficulty with sigmoidoscopy or abnormal colorectal anatomy Diagnosed with primary or metastatic cancer in one or more of the following locations: breast, colorectal, lung, pancreas Received prior chemotherapy for colorectal cancer Participants with non-colorectal, non-gastric, or non-pancreatic cancer should have confirmed CEA expression in tumor tissue. CEA expression should be centrally confirmed. For colorectal cancer (CRC) participants, CEA assessment should be performed but the result is not required for participant selection. A pathologically documented colorectal cancer that: Have a primary diagnosis of non-metastatic colorectal cancer and have had surgery and are now receiving adjuvant chemotherapy Diagnosis of any stage I - IV colorectal cancer or recurrent colorectal cancer (Arm ) Treat patients diagnosed with breast, prostate or colorectal cancer