Must have BOTH estrogen receptor (ER) and progesterone receptor (PR)-positive tumors and BOTH must be >= 26% positive; alternatively, if ER and PR are determined by Allred score, the score needs to be 5 or higher Patients must have had estrogen receptor (ER) analysis performed on the primary breast tumor before neoadjuvant therapy according to current American Society of Clinical Oncology (ASCO)/College of American Pathologists (CAP) guideline recommendations for hormone receptor testing; if negative for ER, assessment of progesterone receptor (PgR) must also be performed according to current ASCO/CAP guideline recommendations for hormone receptor testing Patients must have estrogen and/or progesterone receptor positive histologically confirmed stage I-III adenocarcinoma of the breast Invasive breast cancer is estrogen receptor (ER) positive with an Allred score of 6, 7 or 8 by local institution standard protocol; if an Allred score is not reported on the diagnostic pathology report, ER positivity in > 66% cells is eligible; if ER positivity is =< 66%, the staining intensity (weak, intermediate, strong) is needed to calculate the Allred score to determine eligibility Patients with the following types of histologically documented solid tumors:* Estrogen receptor (ER) positive (+)/progesterone receptor (PR)+, ER+/PR negative (-), or ER-/PR+ breast cancer* Gynecologic tumors (endometrial, ovarian, uterine, fallopian tube, peritoneal, etc.)* Desmoid tumors* Tumors that are ER+ or PR+ by immunohistochemistry (including low-level expression) such as non-small cell lung, colorectal, and prostate Patients enrolled based on tumor ER/PR status must have ER/PR status confirmed by the Laboratory of Pathology, National Institutes of Health (NIH); ER/PR status will be determined on a metastatic site, if possible; otherwise, the original site or available tissue will be acceptable * For patients who underwent initial surgery and received adjuvant chemotherapy** Triple negative breast cancer (TNBC) patients must have been axillary node-positive (>= pN1, any tumor size) or axillary node negative (pN0) with invasive primary tumor pathological size > 2 cm (>= pT2)** Estrogen receptor (ER) and/or progesterone receptor (PgR) positive/human epidermal growth factor receptor (HER) 2 negative patients must have had >= 4 pathologically confirmed positive lymph nodes* For patients who underwent neoadjuvant chemotherapy followed by surgery** TNBC patients must have residual invasive breast cancer in the breast and/or resected lymph nodes (non-pathological complete response [pCR])** ER and/or PgR positive/HER2 negative patients must have residual invasive cancer in the breast and/or the resected lymph nodes (non-pCR) AND a clinical pathologic stage (CPS) & estrogen receptor status nuclear grade (EG) score >= 3 Estrogen receptor (ER) and/or progesterone receptor (PR) positive histologically confirmed adenocarcinoma of the breast with staining of >= 1% cells will be considered positive; receptor status may be based on any time during treatment prior to study randomization, and from any site (i.e. primary, recurrent, or metastatic) The tumor must have estrogen receptor (ER)-and progesterone receptor (PgR)-status assessed using current American Society of Clinical Oncology (ASCO)/College of American Pathologists (CAP) guidelines; patients are eligible if the tumor staining meets one of the following criteria:* ER-negative and PgR- negative by ASCO/CAP guidelines, OR* ER or PgR stains are positive in 1-9% of cells and neither is positive in >= 10% of cells Primary tumor and/or metastatic site must be ER+ and may be progesterone-receptor positive (PgR+) or progesterone-receptor negative (PgR-) by IHC; patients with a history of an estrogen-receptor negative (ER-) primary tumor and a documented ER+ metastatic site are eligible Primary and/or metastatic breast tumor must be negative for over-expression of estrogen and progesterone receptors; patients with weak estrogen receptor and/or progesterone receptor expression (< 10% on immunohistochemistry [IHC]) will be eligible Patients must have histologically confirmed, metastatic or unresectable malignancy of the following types: (a) non-small cell lung cancer (NSCLC), (b) triple-negative breast cancer (TNBC; defined by estrogen receptor [ER] < 1%, progesterone receptor [PR] < 1% and HER2 1+ or less by immunohistochemistry [IHC]; if HER-2 expression is 2+, a negative fluorescence in situ hybridization [FISH] testing is required) (c) pancreatic adenocarcinoma (PDAC), or (d) small cell lung cancer (SCLC) Patients must also meet at least one of the following criteria:* Triple negative: histologically confirmed primary and/or metastatic site that is estrogen receptor (ER)-negative (=< 1%), progesterone receptor (PR)-negative (=< 1%), and HER2negative* BRCA mutation: previously confirmed deleterious breast cancer 1, early onset (BRCA1) or breast cancer 2, early onset (BRCA2) germline mutation or suspected deleterious BRCA1 or BRCA2 germline mutation if the classification being used is the 5-tier classification; documentation of germline test results are required Eligible tumor-node-metastasis (TNM) stages include:* Estrogen receptor (ER) and progesterone receptor (PR) negative (defined as < 1% staining for ER and PR by IHC): T2 or T3 N0, T0-3N1-3** Note: Patients with T1, N1mi disease are NOT eligible* ER and/or PR positive (defined as >= 1% staining for ER and/or PR on IHC): T0-3N1-3 or T3N0** Note: Patients with T0N0, T1N0 and T1, N1mi disease are NOT eligible* ER and/or PR positive (defined as >= 1% staining for ER and/or PR on IHC): T0-3N1-3 or T3N0** Note: Patients with T0N0, T1N0, T2N0 and T1-2, N1mi disease are NOT eligible* The eligibility of neo-adjuvant subjects can be assessed on the basis of clinical (c)TNM or pathologic (yp)TNM; the same eligible TNM combinations apply; patients may be eligible if they meet eligibility requirements at either time point, as long as they do not have T4 disease prior to therapy Histologic documentation of women or men with node positive, HER2 negative, anatomic stage II or III breast carcinoma and high risk node negative (defined as estrogen receptor [ER] and progesterone receptor [PR] negative and tumor size > 2 cm) within one year of diagnosis and free of recurrence; patients with pN1mic are eligible; if neoadjuvant therapy was received, either initial clinical stage (determined by physical and or radiologic examination) or post-operative pathologic stage can be used for eligibility purposes, with the higher stage determining eligibility; histologic documentation of node positivity is required; bilateral breast cancers are allowed, as long as both cancers are HER2 negative and at least one of the cancers meets eligibility Any ER/progesterone receptor (PgR) status allowed Patients must have histologically documented unresectable stage III or IV triple negative breast cancer (TNBC) and a known BRCA 1/2 mutation present; TNBC patients must be Her-2 negative, estrogen receptor (ER) negative (defined as less than 3% ER by immunohistochemistry [IHC]) and progesterone receptor (PR) negative breast cancer (defined as less than 3% PR staining by IHC) Estrogen receptor (ER) and progesterone receptor (PR) < Allred score of 3 or =< 5% positive staining cells in the invasive component of the tumor (provided the patient is being treated as triple negative breast cancer) Patients must have had ER analysis performed on the primary breast tumor collected prior to neoadjuvant therapy according to current American Society of Clinical Oncology (ASCO)/College of American Pathologists (CAP) guideline recommendations for hormone receptor testing; if negative for ER, assessment of progesterone receptor (PgR) must also be performed according to current ASCO/CAP Guideline Recommendations for hormone receptor testing