Abstract: New urbanization is a vital force affecting the green and sustainable development of agriculture, particularly in the grain industry. Exploring the impact and mechanism of the green total factor productivity of grains is significant for ensuring China’s food security and achieving high-quality agricultural development. This study analyzes panel data from 31 provincial-level administrative regions in China (excluding Hong Kong, Macao, and Taiwan) from 2007 to 2021. It employs the SBM-GML index model and entropy method to estimate the grain green total factor productivity and the comprehensive development level of new urbanization in each province. A panel regression model was constructed to empirically analyze the impact of new urbanization on grain green total factor productivity. Mediating and regulating effect models were used to examine the mediating effect of technological innovation on the influence of new urbanization on grain green total factor productivity and the moderating effect of rural human capital. The results showed that: 1) The average annual grain green total factor productivity of China at the national level was 1.0082, indicating that China’s grain green total factor productivity had a certain degree of improvement and showed a fluctuating trend. The decomposition results of grain green total factor productivity showed that the average green technology progress and efficiency index values were >1; the improvement in green technology progress was superior to green technology efficiency, indicating technological progress was the leading factor in increasing grain green total factor productivity. At the provincial level, the average grain green total factor productivity of all provinces except Guangxi was >1, indicating that all provinces except Guangxi improved to a certain extent. The level of new urbanization continues to improve, and the differences between provinces are narrowing. 2) New urbanization had a significant promoting effect on grain green total factor productivity. This conclusion remains valid after applying the systematic GMM method and IV-2SLS model to alleviate the endogeneity problem and conduct a series of robustness tests: the impact of new urbanization on grain green total factor productivity has evident regional heterogeneity and operating scale heterogeneity; new urbanization can significantly promote grain green total factor productivity in major grain-producing, production, and marketing balance, and large-scale operation areas, but has no significant effect on grain major marketing or small-scale operation areas. 3) New urbanization can promote green total factor productivity through technological innovation, and the mediating effect of technological innovation is 54.43%. The regulatory effect of rural human, health human, educational human, and migration human capital positively moderates new urbanization, promoting grain green total factor productivity growth. It is suggested to gradually and steadily promote the construction of new urbanization, give full play to its technological innovation effect, avoid negative impacts on green grain production in new urbanization, and consider the cultivation of rural human capital to promote the high-quality development of China’s grain industry.