Patients with known optic nerve and/or retinal involvement are not eligible; patients who are presenting with visual disturbances should have an ophthalmologic exam and, if indicated, a magnetic resonance imaging (MRI) to determine optic nerve or retinal involvement Optic nerve sheath meningioma, spinal or other extracranial meningioma, multiple meningiomas, hemangiopericytoma History of metastases to brain stem, midbrain, pons, or medulla, or within millimeter (mm) of the optic apparatus (optic nerves and chiasm) An optic pathway glioma History of metastases to brain stem, midbrain, pons, or medulla, or within millimeter (mm) of the optic apparatus (optic nerves and chiasm); or leptomeningeal metastatic disease; or intracranial hemorrhage Additional Exclusion Criteria: May not have a NF related tumor such as optic pathway glioma or malignant peripheral nerve sheath tumor, which requires treatment with chemotherapy or surgery Known ophthalmologic conditions, such as:\r\n* Current or past history of central serous retinopathy\r\n* Current or past history of retinal vein occlusion\r\n* Known intraocular pressure (IOP) > mmHg (or ULN adjusted by age) or uncontrolled glaucoma (irrespective of IOP); patients with known glaucoma and increased IOP who do not have meaningful vision (light perception only or no light perception) and are not experiencing pain related to the glaucoma, may be eligible after discussion with the study chair\r\n* Subjects with any other significant abnormality on ophthalmic examination should be discussed with the study chair for potential eligibility\r\n* Ophthalmological findings secondary to long-standing optic pathway glioma (such as visual loss, optic nerve pallor or strabismus) or long-standing orbito-temporal PN (such as visual loss, strabismus) will NOT be considered a significant abnormality for the purposes of the study Tumors of all regions of the CNS, with appropriate histology are eligible for study; however patients with SEGA or tumors intrinsic to the optic nerve and involvement of the optic nerve that cannot be biopsied/resected are eligible without histological confirmation Patient must have one of the following: \r\n* For stratum : non NF- associated low grade glioma (LGG) (other than pilocytic astrocytoma or optic pathway glioma) \r\n* For stratum or : non NF-, non-optic pathway pilocytic astrocytoma; note: all patients with non NF- associated optic pathway glioma with or without tissue must be enrolled on stratum Patients with sporadic (non NF- associated), histologically diagnosed progressive, recurrent or refractory non-optic pathway pilocytic astrocytoma who have pre- treatment tumor tissue available for BRAF analysis Patients with progressive, recurrent or refractory optic pathway glioma, with or without pre-treatment tumor tissue Patients with histologically diagnosed progressive, recurrent or refractory non NF- associated LGG (other than pilocytic astrocytoma or optic pathway glioma); these patients must have BRAF aberrations as documented by the Lindeman and Ligon Labs at Brigham and Womens Hospital using the same procedures Patients will be assigned to one of strata prior to enrollment; all BRAF assessments used for stratification below must be done at the Lindeman and Ligon Labs at Brigham and Womens Hospital using the same procedures as described in this protocol; assessments for both BRAF V^E mutation and BRAF KIAA fusion are required for patients who will enroll on strata , and \r\n* Stratum : patients with non NF- associated progressive, recurrent or refractory pilocytic astrocytoma with pre-trial tumor material available and with a BRAF aberration i.e. BRAFV^E mutation and/or BRAF KIAA fusion as determined by IHC and FISH, respectively; patients with optic pathway glioma are excluded\r\n* Stratum : patients with non NF- associated progressive, recurrent or refractory pilocytic astrocytoma with pre-trial tumor material available and without a BRAF aberration i.e. BRAF^VE mutation and/or BRAF KIAA fusion as determined by IHC and FISH, respectively; patients with optic pathway glioma are excluded\r\n* Stratum : patients with neuro-fibromatosis (NF-) associated progressive, recurrent or refractory low grade glioma (World Health Organization [WHO] grade I & II), with or without tissue\r\n* Stratum *: patients with non-NF associated progressive, recurrent or refractory optic pathway glioma with or without tissue available for BRAF evaluation\r\n* Stratum : patients with non NF- associated progressive, recurrent or refractory low grade glioma other than pilocytic astrocytoma or optic pathway glioma with a documented BRAF aberration identified in pre-trial tumor material\r\n* Stratum : patients with non-NF- associated progressive, recurrent or refractory low grade glioma (other than optic pathway glioma [OPG]) with tissue available for BRAF analyses who cannot be classified into stratum , or due to inadequate tissue quality, assay failure, etc\r\n**Clarification: Stratum was specifically designed for patients with hypothalamic/optic pathway gliomas; the intent is that if there is any optic chiasm invasion regardless of where the tumor is originating from (chiasm vs. hypothalamus vs. other location), the patient should be enrolled on Stratum , regardless of whether the tumor has been biopsied or not; obviously, there are some tumors that include part of the hypothalamus and clearly do NOT include the chiasm at all; in these situations, and if the tumor is a biopsy proven pilocytic astrocytoma, these patients should be enrolled on Stratum or (depending upon BRAF status) Fiber optic exam with laryngopharyngoscopy (mirror and/or fiber optic and/or direct procedure) within weeks prior to registration Ophthalmological conditions as follows:\r\n* Current or past history of retinal pigment epithelial detachment (RPED)/central serous retinopathy (CSR) or retinal vein occlusion\r\n* Intraocular pressure (IOP) > mmHg or uncontrolled glaucoma (irrespective of IOP)\r\n* Evidence of optic glioma, malignant glioma, malignant peripheral nerve sheath tumor,\r\n* Ophthalmological findings secondary to long-standing optic pathway glioma (such as visual loss, optic nerve pallor or strabismus) or long-standing orbito-temporal PN (such as visual loss, strabismus) will NOT be considered a significant abnormality for the purposes of the study\r\n* Subjects with any other significant abnormality on ophthalmic examination (performed by an ophthalmologist) should be discussed with the study chair for potential eligibility Papilledema or other active optic nerve disorder Metastases within mm of the optic apparatus May not have a NF related tumor such as optic pathway glioma or malignant peripheral nerve sheath tumor, which requires treatment with chemotherapy, radiation, or surgery Known ophthalmologic conditions, such as:\r\n* Current or past history of retinal pigment epithelial detachment (RPED)/central serous retinopathy (CSR)\r\n* Current or past history of retinal vein occlusion\r\n* Known intraocular pressure (IOP) > mmHg (or ULN adjusted by age) or uncontrolled glaucoma (irrespective of IOP); patients with controlled glaucoma and increased IOP who do not have meaningful vision (light perception only or no light perception) may be eligible after discussion with the study chair\r\n* Subjects with any other significant abnormality on ophthalmic examination (performed by an ophthalmologist) should be discussed with the study chair for potential eligibility \r\n* Ophthalmological findings secondary to long-standing optic pathway glioma (such as visual loss, optic nerve pallor or strabismus) or long-standing orbito-temporal PN (such as visual loss, strabismus) will NOT be considered a significant abnormality for the purposes of the study Lesion(s) involving the brainstem, optic chiasm or optic nerve(s) Patients with a known history or prior diagnosis of optic neuritis or other immunologic or inflammatory disease affecting the central nervous system Metastases in the brain stem, midbrain, pons, medulla, or within mm of the optic apparatus (optic nerves, chiasm and optic tracts) Previous pathologic confirmation of a tumor treated with radiation to the brain completed at least months prior to the start of planned reirradiation, except for patients with tumors that are routinely diagnosed without biopsy, including germinoma and optic pathway glioma; patients with a history of cranial irradiation for leukemia are eligible Patients with a known history or prior diagnosis of optic neuritis or other immunologic or inflammatory disease affecting the central nervous system Patients with a known history or prior diagnosis of optic neuritis or other immunologic or inflammatory disease affecting the central nervous system Adjacent tumor location to optic apparatus or brainstem, precluding achievement of meaningful dose with SRS Lesion located in anatomic regions that are not amenable to SRS, including the brain stem, optic apparatus, or eloquent cortex At the time of planning, unable to deliver Gray (Gy) or less to optic nerve/chiasm Unable to deliver gray (Gy) or less to optic nerve/chiasm (typically mm or more separation of the tumor from the optic nerve/chiasm is necessary) Patients with active optic nerve and/or retinal involvement are not eligible; patients who are presenting with visual disturbances should have an ophthalmologic exam and, if indicated, a magnetic resonance imaging (MRI) to assess optic nerve or retinal involvement Patients with known ophthalmologic conditions, such as:\r\n* Current or past history of central serous retinopathy\r\n* Current or past history of retinal vein occlusion\r\n* Known intraocular pressure (IOP) > mmHg (or upper limit of normal [ULN] adjusted by age) or uncontrolled glaucoma (irrespective of IOP); patients with controlled glaucoma and increased IOP who do not have meaningful vision (light perception only or no light perception) may be eligible after discussion with the study chair\r\n* Subjects with any other significant abnormality on ophthalmic examination (performed by an ophthalmologist) should be discussed with the study chair for potential eligibility\r\n* Ophthalmological findings secondary to long-standing optic pathway glioma (such as visual loss, optic nerve pallor or strabismus) or long-standing orbito-temporal plexiform neurofibroma (PN) (such as visual loss, strabismus) will NOT be considered a significant abnormality for the purposes of the study Recurrent tumors near the brainstem and optic chiasm must not have received prior radiation therapy Metastases to brain stem, midbrain, pons, or medulla or within mm of the optic apparatus (optic nerves and chiasm) No metastases to brain stem, midbrain, pons, or medulla or within mm of the optic apparatus (optic nerves and chiasm) Recurrent tumors near the brainstem and optic chiasm must not have received prior radiation therapy Patients with compression of the optic chiasm due to a macroadenoma or patients at high risk of compression of the optic chiasm (tumor within mm of optic chiasm). Subjects with glaucoma, intraocular pressure > mmHg, or any other significant abnormality on ophthalmic examination (performed by an ophthalmologist); ophthalmological findings secondary to long-standing optic pathway glioma such as optic nerve pallor or strabismus will NOT be considered significant for the purposes of the study Prior history of non-arterial ischemic optic retinopathy Patients must have a pilocytic astrocytoma or optic pathway glioma that has relapsed, progressed, or become refractory to conventional therapy; patients with neurofibromatosis (NF-) are eligible Brain metastasis that is located =< mm of the optic chiasm or within the brainstem Evidence of an optic glioma requiring treatment Participants with leptomeningeal disease; metastases to the brain stem, midbrain, pons, medulla, or within millimeters (mm) of the optic apparatus (optic nerves and chiasm) Clinical or neuroimaging evidence of extraocular disease or orbital optic nerve involvement Patients with a known history of NF (Neurofibromatosis Type ) and either \r\n* A history of a tumor of the central nervous system (astrocytoma or optic glioma), or \r\n* A malignant peripheral nerve sheath tumor with a complete remission of < year are not eligible Subjects with NF- associated optic pathway tumors are excluded if they are actively receiving therapy for the optic pathway tumor or do not meet criteria for PN or malignant solid tumor. Subjects with NF- actively receiving therapy for the optic pathway tumor. Evidence of an optic glioma, malignant glioma, malignant peripheral nerve sheath tumor, or other cancer requiring treatment with chemotherapy or radiation therapy Ophthalmologic conditions:\r\n* Current or past history of central serous retinopathy\r\n* Current or past history of retinal vein occlusion\r\n* Known intraocular pressure (IOP) > mmHg (or ULN adjusted by age) or uncontrolled glaucoma (irrespective of IOP); patients with known glaucoma and increased IOP who do not have meaningful vision (light perception only or no light perception) and are not experiencing pain related to the glaucoma may be eligible after discussion with the study chair\r\n* Subjects with any other significant abnormality on ophthalmic examination should be discussed with the study chair for potential eligibility\r\n* Ophthalmological findings secondary to long-standing optic pathway glioma (such as visual loss, optic nerve pallor or strabismus) or long-standing orbito-temporal PN (such as visual loss, strabismus) will NOT be considered a significant abnormality for the purposes of the study No metastases to brain stem, midbrain, pons, or medulla or within mm of the optic apparatus (optic nerves and chiasm) History of MPNST (malignant peripheral nerve sheath tumor) or any malignancy other than asymptomatic and stable optic nerve glioma Optic nerve glioma that has resulted in precocious puberty or visual impairment of any degree Active or untreated central nervous system (CNS) metastases as determined by computed tomography (CT) or magnetic resonance imaging evaluation during screening (within days before CD) and prior radiographic assessments. Participants with radiographically stable, asymptomatic previously irradiated lesions are eligible provided participant is >= weeks beyond completion of cranial irradiation and >= weeks off of corticosteroid therapy. Participants with metastases to the brain stem, midbrain, pons, medulla, or within millimeter (mm) of the optic apparatus (optic nerves and chiasm) are completely excluded Patients who have previously experienced non-arteritic ischemic optic neuropathy (NAION) NF-associated optic pathway glioma (OPG) will be defined as radiographic evidence of glioma along the optic nerve, chiasm, tract or radiation in a child with NF Family history of Leber Hereditary Optic Neuropathy, Autosomal Dominant Optic Atrophy, Late-Onset Retinal Degeneration, Familial Dysautonomia or other hereditary mitochondrial disease, unless the causative mutation(s) in the family have been determined and the participant has tested negative for the mutation(s). Known prior or current retinal or optic nerve disease (eg, Retinitis Pigmentosa, diabetic retinopathy, optic neuritis) based on screening ophthalmology assessment for eligibility.