How Good Are Today’s AI Weather Forecasts Over the Northeastern United States?

Abstract: Recent advances in artificial intelligence (AI) have rapidly expanded the number of operational weather forecasting systems available to users, including both locally executable AI models and provider-issued operational AI forecast products. Despite this rapid evolution, independent evaluations of these forecasting systems within a common verification framework remain limited. This study evaluates the performance of state-of-the-art operational AI and numerical weather prediction (NWP) systems over the Northeastern United States during the summer of 2026. The comparison includes locally executed AI models, operational AI forecast products from multiple providers, and established operational NWP systems. All forecasts are standardized to a common regional framework and evaluated using consistent verification metrics for near-surface temperature and related atmospheric variables. Seasonal forecast performance is assessed across the summer, with particular emphasis on the major Northeastern U.S. heatwave of early July 2026, enabling comparisons of forecast accuracy, event onset, intensity, persistence, lead-time behavior, and systematic biases across forecasting systems. In addition to forecast skill, the evaluation documents forecast provenance, initialization source, and operational configuration to support transparent and reproducible comparisons. The results provide insight into the strengths, limitations, and systematic biases of contemporary operational forecasting systems and establish a reproducible approach for evaluating the rapidly evolving ecosystem of AI weather prediction technologies.