Data Provenance and Processing Details

Data sources

• ODE assessment group reports for 2024-25 (ELA, Math, Science).
  • Assessment Group Reports landing page
  • ELA section - base workbook data/raw/ORSchoolTestData.xlsx (archived locally Jan. 12, 2026)
  • Math section - base workbook data/raw/ORSchoolTestDataMath.xlsx (archived locally Jan. 14, 2026)
  • Science section - base workbook data/raw/ORSchoolTestDataScience.xlsx (archived locally Jan. 14, 2026; byte-identical to data/raw/pagr_schools_science_tot_studentgroups_2425.xlsx)
  • pagr_schools_ela_tot_raceethnicity_2425.xlsx (archived locally Feb. 1, 2026)
  • pagr_schools_ela_tot_gnd_2425.xlsx (archived locally Mar. 15, 2026)
  • pagr_schools_ela_tot_ext_swa_swd_2425.xlsx (archived locally Mar. 15, 2026)
  • pagr_schools_ela_tot_elp_ine_mig_tag_2425.xlsx (archived locally Mar. 15, 2026)
  • pagr_schools_ela_tot_sep_hom_mlc_sfc_2425.xlsx (archived locally Mar. 15, 2026)
  • pagr_schools_math_tot_raceethnicity_2425.xlsx (archived locally Feb. 1, 2026)
  • pagr_schools_math_tot_gnd_2425.xlsx (archived locally Mar. 15, 2026)
  • pagr_schools_math_tot_ext_swa_swd_2425.xlsx (archived locally Mar. 15, 2026)
  • pagr_schools_math_tot_elp_ine_mig_tag_2425.xlsx (archived locally Mar. 15, 2026)
  • pagr_schools_math_tot_sep_hom_mlc_sfc_2425.xlsx (archived locally Mar. 15, 2026)
  • pagr_schools_science_tot_studentgroups_2425.xlsx (archived locally Jan. 14, 2026)
  • pagr_schools_science_raceethnicity_2425.xlsx (archived locally Feb. 1, 2026)
• ODE Regular Attenders file for attendance/absence context (2024-25).
  • regularattenders_report_2425.xlsx (archived locally Feb. 5, 2026)
• ODE class-size workbook (2024-25) and ODE school-level spending metadata (2023-24).
  • class_size_report_20242025.xlsx (archived locally Feb. 7, 2026)
  • School Level Expenditure Report_23-24.csv (archived locally Feb. 7, 2026)
• ODE report card media aggregate (school-level context percentages).
  • ODE Report Card Media page
  • Local raw file: data/raw/AAGmediaSchoolsAggregate.csv
• U.S. Census geographies and ACS 5-year neighborhood context metrics (income, education, poverty, labor).
• NCES EDGE locale classifications (city/suburb/town/rural).

How the data is processed

• Assessment workbook families are parsed into CSV and rows with numeric participant counts are retained for analysis.
• For ELA, Math, and Science, the base subject workbooks are combined with supplemental 2024-25 assessment-group workbooks; exact overlapping row keys are treated as validation checks, only genuinely new subgroup rows are appended, and shared total-population rows remained unchanged wherever they were present in that merge. For Science, the local base workbook is byte-identical to the archived total/student-groups workbook, and the current processed/dashboard Science row universe also includes the separate archived race/ethnicity workbook.
• School addresses are added, then geocoded to Census tract geography using Census batch geocoding with fallback passes.
• Tract-level ACS context variables are appended and then propagated across subject datasets using school-level keys.
• Attendance, class size, spending, school type, virtual status, curated choice/immersion flags, and locale labels are merged from separate source files.
• ODE media aggregate school percentages are merged by (District ID, School ID), with unique-school-ID fallback checks and sample name validation.
• Quality checks and merge logs are maintained to support reproducibility and external review.

Reviewed school-location corrections (August 2026)

A targeted address audit compared processed school locations with ODE and NCES records, current district evidence, Google geocoding, and official Census coordinate-to-geography results. Seventeen reviewed corrections were then applied identically to the ELA, Math, and Science datasets. Fifteen schools changed Census tract and block group; in all, corrected location context was attached to 947 assessment rows.

• The correction was limited to address, coordinates, Census geography, and associated ACS neighborhood fields.
• Achievement results, Scored Performance Denominator, participant counts, attendance, staffing, spending, and school-type fields were not changed.
• An automated validator reconstructed the exact pre-correction files and verified their frozen SHA-256 hashes; it also confirmed that the published site copies match the corrected core data.
• The most visible reporting change is that one, rather than two, schools in the least-advantaged fifth reaches top-fifth ELA or Math performance. Fruitdale Elementary is the remaining case; Crooked River Elementary now falls in the middle 60 percent after its neighborhood assignment was corrected.
• The statewide prediction results barely moved: repeated school-holdout full-context R2 changed from 0.4949 to 0.4959, and the level, grade, and subject rankings did not change.
• The live dashboard's simpler scored-student-weighted BA-proficiency correlations moved modestly across the complete sweep (ELA: 0.6001 to 0.5961; Math: 0.6187 to 0.6148; Science: 0.5727 to 0.5719). These descriptive correlations are not held-out prediction R2 values, and their direction and interpretation did not change.

A follow-on closure pass resolved the remaining three material cross-subject location conflicts and rebuilt the NCES locale crosswalk from the complete ELA, Math, and Science school union. The rebuilt crosswalk covers all 1,178 schools with no unmatched, low-confidence, or ambiguous records. Four previously accepted matches that pointed to the wrong school were replaced with reviewed identities; one related location review identified a real campus and tract change for Cascade Heights Public Charter School.

• The final audit found zero material cross-subject location conflicts and zero unresolved NCES tract discrepancies among ordinary physical schools.
• The follow-on closure corrections themselves changed no Two Academic Worlds summary cell, advantage-group membership, residual-candidate tier, dashboard BA-proficiency correlation, or published prediction statistic; repeated school-holdout full-context R2 remained 0.495870.
• For virtual and alternative programs, NCES locale describes the program's administrative location, not necessarily the residential environment of enrolled students.

The canonical correction ledger, corrected-geography ACS lookup, before-and-after field audit, and consequence tables are retained with the reproducible project files.

ODE media aggregate enrichment (2024-25 datasets)

The following school-level percentage fields were added to the 2024-25 processed ELA, Math, and Science datasets: Students with Disabilities, Students Experiencing Poverty, Required Childhood Vaccinations, Average Teacher Retention Rate, and Experienced Teachers. College Going 2022-23 was also merged but has narrower grade-span relevance.

• Primary merge key: (District ID, School ID).
• Fallback key: unique School ID when district pairing is unavailable.
• Verification: duplicate/conflict checks, parse-anomaly checks, and random sample name checks.
• Scope: applied to 2024-25 datasets only (not the 2018-19 math comparison file).

Coverage results from source-vs-processed audit

A direct comparison of ODE source rows against processed files found no loss of rows where participant counts are numeric. Coverage gaps come from ODE suppression/no-test markers in source files. The table below focuses on total-population school/grade coverage; the current source-lineage validator also checks the full source row universe, including Science race/ethnicity rows.

Known deficiencies (current datasets)

Small but non-random gaps remain in contextual fields. The largest source is suppressed assessment rows and unmatched attendance rows.

Attendance counts above mean schools with at least one unmatched attendance row, not schools with no attendance data. Fully missing attendance is much smaller (ELA: 4 schools, Math: 3, Science: 5), and virtual/online programs account for a sizable share of attendance-gap schools (ELA: 32/105, Math: 27/96, Science: 47/132), so statewide patterns are generally less sensitive than raw counts suggest. NCES locale coverage is now complete across the three subject files. The remaining missing-address schools are disproportionately virtual, online, or other special programs; without a physical address they cannot be linked to tract SES and are excluded from SES-association analyses.

For ODE media aggregate fields, missingness is low for most columns, but College Going 2022-23 is intentionally sparse and high-school-oriented. It should be interpreted as a specialized indicator, not a universal school-level context field.

Important disclosure on timing alignment

School type, virtual status, and spending metadata currently come from ODE's 2023-24 school-level spending file, while assessment outcomes are from 2024-25. This is a known temporal mismatch and will be replaced when a synchronized 2024-25 spending metadata file is available.

Separately, the dashboard includes a small curated flag for schools whose enrollment design does not represent a normal neighborhood attendance area, such as language-immersion lottery schools or no-boundary schools of choice. The current lookup is maintained in data/lookup/special_enrollment_schools.csv and is applied separately from ODE's charter and virtual-school metadata.