Generative AI and Its Uses in Education: Foundational Controversies and Responding Strategies

Fengchun Miao’s article appeared in Open Education Research, volume 30, issue 1 of 2024, as the second interpretation of UNESCO’s Guidance for generative AI in education and research. This edition follows the original full-text scope, including three figures, twenty-seven references and the original English metadata, with structured sections and accessible text tables. Technical judgments, statistics and policy analysis are read in the context of writing in 2023; corrections to names, figures and legal descriptions are documented in the editorial notes.

Fengchun Miao — affiliations 1 and 2

1. National Engineering Research Center of Cyberlearning and Intelligent Technology, Beijing Normal University.

2. UNESCO Headquarters.

Abstract: This is the second article in a series interpreting UNESCO’s Guidance for generative AI in education and research. It examines fundamental controversies arising from generative AI and its educational applications. Analysis of technical limitations across its working processes reveals concerns about worsening data poverty, technological opacity and gaps in domestic governance, unauthorized collection of training data, inexplicable model architectures, foundation models’ understanding of the real world, pollution of the internet by generated content, the projection of dominant values, and the spread of unlawful deepfakes. The paper argues that generative AI can profoundly affect educational values including equality, inclusion, learner agency, values and linguistic and cultural diversity, and pluralism in knowledge construction.

These values should form the starting point for assessing its educational suitability. Policymakers and practitioners should follow a pathway of “prioritizing regulation, ensuring inclusion, and guiding applications”, strengthening system-wide regulation and enforcement to establish educational generative-AI ecosystems that are safe, trustworthy, independently controllable and locally appropriate. Capacity development and practical guidance should then support appropriate educational use.

Keywords: generative artificial intelligence; content processing; sources of training datasets; equity, inclusion, and linguistic and cultural diversity.

Introduction

In November 2022, OpenAI released ChatGPT, opening a new phase in generative AI’s transition from research to commercial and public use. The initial ChatGPT was based on the GPT-3.5 series; the original paper’s label “ChatGPT-3” is corrected here using the developer’s launch information, as explained in the editorial notes. The paper groups the impacts and regulatory responses during roughly the first year after launch into four areas.

1. Monopoly and diversity. The paper describes OpenAI, Google and Meta as establishing vertically dominant positions spanning foundation models, network infrastructure and the processing of text, images, audio and video. Other major companies and open-model communities were pursuing open foundation models, a wider choice of platforms, and greater linguistic and cultural diversity through competing research and ecosystem development.

2. Application and substitution. Generative AI was spreading rapidly through commercial activities, accelerating the automation of jobs in affected industries.

3. Controversy and governance. Safety and ethical concerns were moving from theoretical worries into actual legal cases. Confidence in transformative potential stood against concerns about human consequences, accelerating targeted legislative action in countries and jurisdictions including China, the United States and the European Union.

4. Vision and reality. Rapidly emerging generative AI was disrupting and changing education and other social services, while a substantial gap remained between these developments and reasoned assessments of educational suitability and practical usefulness for local students, especially minors. Public views on its possible safety and ethical consequences remained divided.

UNESCO’s Guidance for generative AI in education and research, released in September 2023 and referred to below as the Guidance (Miao, 2023), traced the technology’s working principles and systematically identified eight fundamental controversies. It examined their implications for educational use and proposed public-governance responses, educational policies and a practical framework for human–machine interaction appropriate to the people involved.

As the second interpretation of the Guidance, this study systematically examines the controversies surrounding generative AI and education, identifying governance responses and practical recommendations according to their causes and the actors responsible. Its interpretation rests on three interrelated assumptions:

First, AI innovation, safety and trustworthiness, and inclusive and equal social application should not form a trilemma. These three aims should advance together and reinforce one another.

Second, the study sees the most direct and profound threats as those to educational equality and inclusion, learner agency, values and linguistic and cultural diversity, and pluralism in knowledge construction. These core values should therefore be the starting point for examining generative AI’s educational suitability.

Third, educational applications should follow the logic of “prioritizing regulation, ensuring inclusion, and guiding applications”.

I. Technical origins of the controversies

Discussion of controversies in educational generative AI must be grounded in how the technology works, its limitations, and its observable and potential social effects.

1. Working principles, training-data sources and language distribution

The Guidance defines generative AI in terms of AI’s simulation of the symbolic systems through which human thought is represented: it automatically produces content in response to prompts expressed through those systems. Its abilities in pattern recognition and content production were becoming more powerful across text, speech, sound, images, video and computer code. In the paper’s account, text generation uses artificial neural networks and Transformer-based models commonly described as large language models: deep-learning systems trained on information collected from web pages, social-media conversations and other online media.

Generative pretrained models for text or speech learn syntactic patterns in training datasets and acquire the ability to respond to prompts by repeatedly applying learned patterns through cycles of training, testing and optimization. The principal stages are:

1. Divide a prompt into tokens that the AI can process, and pass them to a generative pretrained Transformer. “Token” is used here in place of the original Chinese text’s inaccurate term “byte”.

2. From linguistic patterns identified in training data, predict the probability of particular words or phrases in context. Statistical predictions combine words and syntactic relationships into coherent responses and predict the words or phrases most likely to follow.

3. Convert the predicted words or phrases into readable text or intelligible speech. Guardrails filter clearly unlawful or otherwise inappropriate outputs, while further processing improves human-like syntax and comprehensibility. The process repeats until a complete response has been produced.

The paper explains image and music generation primarily through generative adversarial networks (GANs), which can be combined with variational autoencoders. It also notes diffusion models as another generative approach used for images. A GAN has two competing components: a generator and a discriminator. In response to a prompt, the generator identifies combinations of image or musical elements and produces an initial image or musical segment. The discriminator compares it with real examples. The generator then adjusts its parameters in response to that comparison. Repeated training over many iterations makes generated images or music increasingly realistic.

A generative pretrained Transformer’s capabilities depend on its architecture, training methods, the quality and quantity of pretraining data, and model parameters. Parameters are numerical values that shape how neural networks process inputs and produce outputs, encoding features learned during training. Their definition and scale affect performance. GPT-3 used approximately 175 billion parameters. The original paper repeats an unconfirmed claim that GPT-4 had “1.8 trillion”; the official GPT-4 technical report did not disclose model size, as the editorial notes explain. The paper’s contemporary model landscape included OpenAI’s ChatGPT products; Stanford’s Alpaca, fine-tuned from Meta’s LLaMA; Meta’s Llama models; Google’s Bard, then based on PaLM 2; and the multimodal Gemini.

Training sources included crawled web pages, social-media conversations, online-library books and encyclopedia content. For GPT-3, the paper cites an estimated corpus of about 1 TB (Thompson, 2023), comprising approximately 61.75% Common Crawl material collected from the internet since 2012; 18.86% Reddit posts and discussions with more than three upvotes; 15.9% books from Library Genesis and Smashwords; and 3.49% Wikipedia. These are the paper’s historical source estimates, not a description of current ChatGPT. The paper emphasizes the predominance of US and European languages in then-dominant models: in the cited GPT-3 corpus, English accounted for about 92.65%, other European languages for over 5%, and Chinese for under 0.1%.

For Meta’s Llama 2, English was 89.7%, Chinese 0.13%, and the other leading fifteen languages largely unchanged (Touvron et al., 2023). Rapid growth in data and parameters also demanded greater high-performance computing. For 2012–2019, the paper cites a three-to-four-month doubling period in computing used to train major AI models (Stanford University, 2019).

2. Integrated advances in content processing and potential limits of the technological category

The study describes integrated advances across core and supporting technologies: combined deep-learning methods, optimized architectures, refinement of parameter sets on the scale of hundreds of billions, dynamic accumulation of cross-platform training data, and growing computational capacity for vast datasets. Together, these advances could produce a “runaway effect”. First, progress in processing and presenting information across symbolic systems expands technological capability and accelerates work on AI chips, supercomputing and data-processing models. The foundational character of these advances explains the widespread adoption of the term “foundation models”, proposed by Stanford researchers in 2021 (Bommasani, 2021).

Second, generative AI was prompting changes to browsers, search engines and digital infrastructure, becoming an important control point for national data security and personal privacy, and creating pressure to update regulation and security systems. Third, it offers productivity tools to economic and social activities whose direct or indirect aims include producing or synthesizing content, with potentially large changes to production. The author argues, however, that its ability to improve efficiency and transform sectors such as education, whose purposes extend beyond content production, could be substantially limited.

Examining the technical approaches known at the time, the author characterizes neural-network achievements as statistical curve fitting, distinct from human reasoning that combines temporal, spatial and causal relationships (Pearl and Mackenzie, 2018). If deep learning were the appropriate computational route to human-like problem solving, continuing advances might eventually produce a runaway effect beyond content processing: a proposed singularity at which AI surpasses human intelligence and develops along a path relatively independent of human control. Yet the author argues that the then-known principles of foundation models appeared to remain within content synthesis, processing through symbolic representations and format conversion, rather than a technical route to human understanding.

In that interpretation, they risk a “category mistake” (Bishop, 2021). Whether the observed advances represented a leap in performance before such a categorical bottleneck remained an open question.

II. Fundamental controversies surrounding generative AI

The paper understands these controversies as human concerns arising from human–machine interaction and examines them along technical and humanistic dimensions. The technical dimension follows the AI-system lifecycle: data production and storage; access to and control over data and data-processing equipment; the observable effects of data- and algorithm-based predictions and decisions; and embodied AI, including intelligent interfaces and devices. The humanistic dimension encompasses individual existence, social interaction, national governance and relationships with environments and ecosystems. Its interconnected levels include individuals, groups interacting with one another, relationships organized as sovereign states, and interactions between people and their surroundings (Miao, 2022).

Before large-scale adoption, users should examine training-data collection and use, basic architectures and pattern-based outputs, assessing immediate threats and potential effects on individuals, societies and countries.

1. Extracting data productivity: worsening data and digital poverty

The author regards China’s 2022 Opinions on building basic systems for data to give fuller play to data as a factor of production, issued by the CPC Central Committee and State Council, as a legal marker of an era of data property and rights protection. From this perspective, collecting individual and business data without payment and profiting from models trained on it brings formerly less visible controversies over cross-border and cross-sector extraction of data value to the forefront.

Accessing and applying quality data, and continually producing quality online and transformed data, have become important conditions for national economic development and individuals’ digital opportunities. Countries lacking data access, data-extraction capabilities or supercomputing, and individuals unable to pay for data applications, face “data poverty” (Marwala, 2023). The author argues that training foundation models on freely collected data and then charging for their services can intensify this poverty. Three requirements underpin such production: innovation in AI architectures and training methods, very large datasets, and supercomputing capacity. In the period discussed, the author saw only the United States, China, the EU and a small number of very large technology companies as simultaneously possessing these conditions.

Data-poor countries were falling further behind and being excluded from core foundation-model research. Wider adoption accelerates data generation and technical iteration among leaders, reinforcing the digital-divide cycle.

To reverse this cycle, countries should identify and address weaknesses throughout data production. The proposed “classification of causes of data poverty” includes internet coverage, population-wide digital literacy, affordable data traffic, AI talent and innovation incentives, access to capable AI chips and supercomputing, and the ability to train local models using trustworthy domestic or international data. A further question is whether international measures such as a data-use tax should redistribute surplus value when transnational providers collect substantial data from low-income countries. Defining and tracing the volume used, calculating surplus value and tax standards, and balancing innovation with the basic interests of data-poor populations would remain difficult frontier issues in international data governance.

2. Governance in service jurisdictions: loss of domestic control

Cross-border generative-AI services should fall under the governance of the jurisdictions they serve, but technical opacity creates difficulties. First, dominant providers may withhold basic transparency information from independent academic bodies or resist academic assessment (Bommasani, 2023). Second, intellectual-property protections in providers’ home countries, principally the United States in the account presented, can restrict access to foundational technologies in user countries, complicating scrutiny of systems and applications (Lin, 2023). Third, despite calls for a pause in development and alignment with public governance, capital-driven iteration moves much faster than the drafting of regulations, placing authorities at a substantial technical disadvantage when addressing legal and ethical concerns.

The paper describes several levels of institutional absence or delay. First, as of July 2023, it reports that 137 countries had data-privacy frameworks, leaving nearly a third without basic privacy laws (UNCTAD, 2023). Second, about 67 countries had national AI strategies before the Guidance; Rwanda and the Dominican Republic announced strategies afterwards. Third, it reports that only about 40 countries had AI-ethics policies by July 2023. Fourth, the author identifies China, the United States and the EU as the jurisdictions that had then examined intellectual-property protection for synthetic works and issued clear management positions. China had published the Interim Measures for the Management of Generative Artificial Intelligence Services before the Guidance.

The US issued its executive order on safe, secure and trustworthy AI in late October 2023 (The White House, 2023). On 9 December 2023, European Parliament and Council negotiators reached a provisional political agreement on the AI Act. The formal Act belongs to 2024; the original reference to “passage” in December 2023 must not be read as formal adoption and entry into force at that time, as the editorial notes explain.

3. Copyright in pretraining data: using content without permission

The collection and use of training datasets raises major disputes over permission to use individuals’ and organizations’ data and copyrighted content.

Models are trained on crawled text, sound, code, images and other internet materials. The paper argues that major models frequently collected such material without prior permission, creating intellectual-property disputes and litigation, including allegations under data-protection legislation such as the GDPR (European Union, 2016). It highlights the news industry: in late October 2023, the US News/Media Alliance, representing more than 2,200 news publishers, alleged that ChatGPT had used millions of paid news articles and reports as training material without rights-holders’ permission, and sought removal of allegedly unlawfully collected content from Common Crawl through legal processes (Robertson, 2023). Such practices also raise the right to erasure, through which data holders may seek deletion of data collected without permission.

Once personal data has influenced a trained model, the author sees substantial difficulty in removing that influence. The original uses absolute expressions such as “no possibility of unlearning” and “impossible”; these are qualified here as part of the contemporary argument. The cited research discusses machine unlearning, model editing and other approaches, as the editorial notes explain. The problem includes eliminating learned responses reflecting a data holder’s views and linguistic or cultural characteristics (Zhang, 2023).

4. Architectural explainability: outputs from opaque models

The paper describes neural-network architectures as black boxes, with difficulties explaining node and layer configurations, parameter definitions and computations. Recent advances amplify these concerns. Even if the overall approach, including algorithms, is partly explainable, it is difficult to account for particular parameters and their relative contributions to an output. Large foundation models use extensive parameter sets and weights to recognize patterns and produce responses, making it difficult to explain why a particular model generates a particular answer. The author expects growing scale and architectural complexity to deepen this problem.

Technical limits on explainability interact with providers’ reluctance to disclose relevant information, obstructing regulators and independent researchers assessing intentional risks and unintended harms. Stanford’s Foundation Model Transparency Index (Bommasani, 2023) evaluates important technical elements of leading models. The paper refers to a comparative assessment of approximately ten major models and reproduces selected indicators for three prominent examples in Table 1. Its argument is that opaque models and hard-to-detect risks make errors difficult to trace and limit transparent assessment and prevention. It cites expert recommendations against using generative AI for high-risk tasks.

5. Understanding: semantics and the real world

Generative AI recognizes contextual patterns probabilistically and produces text according to learned syntax. The author links limited semantic understanding to hallucinations involving factual, historical and scientific errors. A historical Vectara leaderboard hosted on GitHub, last updated on 1 November 2023, reported task-specific hallucination rates of 3%–3.5% for GPT models, 5.1%–5.9% for Llama models and 12.1% for Google PaLM. These concerned document-summary faithfulness, not general question-answering accuracy. The paper further conjectures that error rates in Chinese and other languages would be higher, but that leaderboard does not establish such a cross-language comparison. Students without secure foundational knowledge may therefore learn from uncertain content during independent chats.

The author argues that such systems should not be treated as reliable sources of teaching content. He also distinguishes syntactic pattern processing from understanding real-world objects and relationships, human and social relationships, and people’s relationships with objects and technologies. Scientific discovery, he notes, depends on observing the world, experimentation and reasoning, while generative AI primarily synthesizes prior literature. In his account, new knowledge requires human agency using AI as assistance. The original further draws an inference about “knowledge” from copyright discussions; copyright protection and the definition of knowledge should not be conflated.

Drawing on Candelon (2023), the paper also questions models’ ability to produce targeted, innovative solutions to complex real-world challenges or make social value judgments. It concludes that they should not independently guide complex knowledge learning or ill-structured problem solving without human teachers.

These limitations, the author argues, constrain positive educational transformation in foundational content, higher-order thinking, complex problem solving, values and human development. Generative AI may exert influence partly by challenging learning outcomes and assessment from the outside: automated content processing and harder-to-detect outsourcing weaken the value of low-level synthesis and production as central outcomes. A declining share of basic spelling and syntax exercises, literature-summary reports, slide production and elementary art products in formative and low-stakes assessment would push systems to reconsider their learning priorities and assessment methods.

6. Generated-information pollution: synthetic content on the internet

The paper identifies two forms of pollution arising from the production and dissemination of generative-AI content.

First, models may reproduce harmful material learned from the internet, including misinformation, discrimination and hate speech, then spread it again online. Such repeated circulation can harm learners of different ages in ways that may be difficult to reverse.

Second, an internet increasingly filled with synthetic material affects the training of subsequent models. Quality deep learning depends on diverse human-produced data and varied forms of expression from which different patterns can be learned. When machine-generated content is widely recirculated, later models may train on previous models’ outputs. In the process described, high-probability events become overrepresented and low-probability events underrepresented, gradually erasing the statistical long tail. That tail helps preserve accuracy and diversity. Its loss can reinforce already learned patterns, producing increasingly homogeneous outputs that resemble synthetic material rather than reality, increasing errors and potentially causing model collapse.

This presents further challenges to obtaining high-quality internet training data (Lutkevich, 2023).

7. Projected values: homogenization and linguistic and cultural suppression

The projection of similar values through generated outputs can suppress digitally disadvantaged groups, pluralism in educational knowledge construction and the expression of different viewpoints.

When a string appears frequently in training data, a Transformer is more likely to reproduce it and related sentences. The paper argues that common beliefs and dominant media perspectives in US and European datasets can become model “standard answers”, strengthening already powerful values and linguistic or cultural conventions through a feedback loop. Uncritical adoption without examining training-data origins may reinforce cultural stereotypes. It cites Omiye (2023), whose repeated tests of ChatGPT/GPT-3.5, GPT-4, Bard and Anthropic Claude found biased answers reflecting racial stereotypes in questions about lung capacity, estimated glomerular filtration rate, skin thickness and brain size. These are examples of harmful model bias, not validated biological claims.

Conversely, data-poor and marginalized groups have fewer online traces and little representation in training datasets, leaving their values and linguistic and cultural practices poorly recognized. The author argues that locally relevant models supporting diversity are important for indigenous languages and cultures and the legitimate interests of disadvantaged groups (Miao, 2023).

Overreliance on generative AI for “standard answers” and solutions may also narrow ideas and weaken diverse innovation. In the Boston Consulting Group study cited, involving more than 750 participants, the ideas produced with GPT-4 were reported to be 41% less diverse than those produced without it. Participants often showed little inclination to increase diversity after receiving model suggestions. In the same cited research, 70% expressed concern that long-term use of generative AI for answers could erode human creativity (Candelon, 2023). These findings belong to that study’s setting.

8. Unlawful deepfakes: easier creation and dissemination

Generative AI lowers skill and cost barriers to producing unlawful deepfakes while making detection harder, facilitating their creation and circulation.

The paper discusses how people without journalism, music or video-production skills can use synthetic content to impersonate real people, spread false news or hate, defame others, manipulate existing images and video, or pursue other unlawful ends. The author contrasts uncertain benefits for solving public-interest problems with the accessibility of tools for malicious purposes. The cited 2023 State of Deepfakes report (Home Security Heroes, now Security Hero) asserted that a sixty-second explicit deepfake could be generated from one clear face image in under twenty-five minutes at no charge. The report analysed 95,820 videos and described an increase of 550% over 2019; the original paper’s “nearly one million new videos” and “5.5 times” are corrected against that source.

These figures belong to the report’s survey, rather than a verified worldwide census or a current total. It reported that 98% of surveyed deepfake videos were pornographic and 99% of the people targeted in that category were women.

III. Recommendations for regulation

To address these controversies while developing educational potential, the Guidance proposes human-centred principles for AI development and use. Its pathway of “prioritizing regulation, ensuring inclusion, and guiding applications” aims at ethical, safe, trustworthy, equitable, inclusive and meaningful use.

1. A human-centred approach to AI governance and application

Since 2019, UNESCO has advocated human-centred AI research and application, elaborated through documents including the Recommendation on the Ethics of Artificial Intelligence (cited as UNESCO, 2022), the Beijing Consensus on Artificial Intelligence and Education (UNESCO, 2019), and AI and education: Guidance for policy-makers (cited as Miao, 2022). In the account presented, governance and use should protect fundamental rights, dignity and cultural diversity, alongside a bio-centred understanding of harmonious relationships among people, environments and ecosystems. Development should serve people and strengthen capabilities for effective human–machine cooperation.

Throughout design, development, use and iteration, human agency should remain central, with trustworthiness, appropriateness to people and purposes, explainability, human control and human accountability (Miao, 2022).

2. A governance roadmap for generative AI

The Guidance recommends whole-of-government, collaborative governance that specifies the rights, powers, responsibilities and obligations of actors across sectors and industries.

2.1 Negotiate international instruments, including cross-border data-protection rules

Mutually recognized data-protection rules and cross-border governance mechanisms are proposed as prerequisites for addressing jurisdictional gaps. The GDPR provides an example of a legal framework for governing cross-border data services.

2.2 Establish whole-of-government AI strategies and cross-sector ethical governance

A whole-of-government strategy can coordinate AI governance across sectors and departments. Ethical-governance mechanisms should clarify national data sovereignty; institutional and individual data rights and privacy; lawful relationships in data production; core principles for AI development and use; and the distribution of surplus value from AI products built using public or other people’s data.

2.3 Develop rules specifically addressing generative AI

China published the Interim Measures for the Management of Generative Artificial Intelligence Services in July 2023. The author described them as the world’s first formal rules for jurisdictional governance of generative-AI services, a comparative judgment made in the original historical context. He proposed refining their legal and ethical provisions using the Guidance and contemporary European and US initiatives. The following points were especially relevant to education:

1. Classify technical safety risks and regulate by risk level. The paper’s account of the then-discussed European AI Act draft distinguishes unacceptable risks, including technologies manipulating minors into dependency, emotion recognition and certain intelligence or behaviour prediction; high risks; limited risks; and minimal risks. It describes corresponding approaches ranging from prohibitions on development and marketing, through intensive supervision, review and oversight, to industry self-regulation. Education was identified as an area for close scrutiny. These are the categories discussed by the historical text, not a statement of every provision of the subsequently adopted Act.

2. Set age safeguards for independent generative-AI chat. Because chat models may generate material unsuitable for minors, the Guidance recommends thirteen as a lower threshold for independent use, while considering a stricter threshold of sixteen. As services move from platform-based chats to apps on personal devices, regulators and adults may find independent use harder to supervise. The author calls for assessment of these risks and clearer responsibilities for supervising minors’ use. The original paragraph’s final reference to “adults” is corrected contextually to “minors”. Thirteen and sixteen are discussed recommendations, not a universal legal chat age.

3. Clarify other actors’ responsibilities. The paper contrasts China’s provider-focused Measures with the broader institutional responsibilities described in the US executive order of 2023. It proposes duties for institutions procuring and deploying AI systems: jointly checking data, tools and content services for legality, and helping monitor changing effects on users, particularly disadvantaged groups. With institutional responsibilities clarified, individual users’ duties should also be considered, including awareness and skills for recognizing threats and protecting themselves, and knowledge and skills for lawful use of tools and generated material.

4. Clarify identification, copyright and permitted uses of generated content. China’s Measures require providers to label generated images and video in accordance with the rules governing deep-synthesis internet information services. The original also attributes AI-content labelling to the GDPR; data-protection law and AI-transparency legislation must be distinguished, as the editorial notes explain. The author argues that the three contemporary frameworks discussed did not resolve all questions about legitimate use of generated products. Outstanding issues included limiting unlawful deepfakes; defining and protecting paying users’ rights to generated material; and distinguishing AI-assisted literature synthesis within human-led research from cheating that violates academic ethics or teaching rules.

IV. Policy and practical recommendations for educational use

The Guidance offers detailed recommendations for effective educational applications. The paper highlights the following:

1. Ensure inclusion, equity, and linguistic and cultural diversity

The relationship between generative AI and human-centred principles has four implications. First, equitable access and opportunities are prerequisites for advancing educational inclusion: people should have opportunities regardless of gender, ethnicity, ability, socioeconomic status or fixed residence. Countries should use the proposed classification of data-poverty causes to identify excluded groups and address specific gaps. Second, dedicated funding should support tools for lifelong learners of different abilities and ages, including people with special learning needs. Third, evaluation should examine bias in data sources, preprocessing, model design and outputs, including gender bias, stereotypes about particular groups, discrimination against marginalized people and hate speech.

Fourth, indicators for linguistic and cultural diversity should support minority and indigenous languages through pretraining data, architectures and training methods, preventing their exclusion or the introduction of biased filtering and postprocessing.

2. Protect human agency

As generative AI becomes more capable and partly automates elementary content processing, educational use may weaken agency. Easy production of essays or basic artworks can encourage dependence on externally generated content, reducing opportunities to exercise the mind and develop foundational knowledge and skills. Protecting and strengthening agency should therefore be central to design and adoption: AI may challenge and extend human thought but should not usurp it. The Guidance proposes seven safeguards. First, explain what student data is collected, how it is used and its possible effects on educational and social life. Second, protect intrinsic motivation and human autonomy in decisions and actions in AI-supported teaching and learning.

Third, preserve opportunities to develop cognition and social skills through observing reality, empirical methods such as experiments and discussion, and logical reasoning. Fourth, ensure social interaction and engagement with human-created works, preventing overdependence or addiction. Fifth, consult researchers, teachers and students before large-scale adoption. Sixth, encourage questioning of technical methods, accuracy, embedded values and effects on pedagogy. Seventh, teachers and students should retain responsibility for decisions made with AI assistance.

3. Review and monitor educational AI tools and develop locally appropriate foundation models

Ethics by design and a lifecycle approach should guide development and deployment, protecting learners, teachers and educational relationships from foreseeable technical or ethical risks and unsuitable tools. For existing systems, the Guidance recommends review, admission and continuing monitoring. Ethical checks should examine bias reduction, especially gender bias, and representative linguistic and cultural data. Before approval, systems should avoid foreseeable harm, demonstrate educational value, suit learners’ ages and abilities, and align with institutional principles concerning knowledge, skills, intended outcomes and values. Data use and service authorization require understandable risk information, appropriate guardianship arrangements where applicable, age-appropriate communication, accessibility and valid consent.

Disability alone does not determine legal capacity; the editorial notes explain the correction of the original’s broad categorization using Article 12 of the Convention on the Rights of Persons with Disabilities. Reviews should also check for deepfakes, false news, hate speech and other harmful or unlawful outputs, and examine environmental costs, including electricity and water used in model training, in the local context.

The paper sees shortcomings in the cultural and linguistic fit of then-dominant models beyond the United States and Europe. While respecting lawful international competition, it recommends active development to improve local suitability: support safe, controllable domestic foundation models with appropriate rights arrangements; develop and iteratively test educational generative pretrained Transformers, or EdGPT models, based on local values, linguistic and cultural diversity and curriculum standards; and create incentives for platforms and plugins supporting inquiry and varied learning choices. The aim is an ecosystem in which foundation models and downstream educational applications develop together around local languages, cultures and curricula.

4. Develop teachers’ and students’ AI competencies

AI competencies, especially for school students, are essential for safe, ethical and meaningful use. The paper reports that by mid-2022 only about fifteen countries had nationally endorsed school AI curricula (UNESCO, 2022), making basic AI literacy for all increasingly urgent. The student framework then under development organized competencies by dimensions and levels, combining classroom and extracurricular learning. Its dimensions were a human-centred AI mindset, AI ethics, foundational techniques and applications, and AI-system design. Levels of understanding, application and creation described knowledge, skills and attitudes. Table 2 retains this historical draft.

UNESCO was also developing a teacher framework to guide national standards and training for appropriate use, shown in Table 3 as the original historical draft. Its five dimensions were a human-centred AI mindset, AI ethics, foundations and applications, integration of AI and pedagogy, and AI-supported professional development. Three levels—acquisition, deepening and creation—described outcomes for teachers with different starting points. Acquisition was intended for all teachers after training, including those without prior AI preparation and those in poorly resourced settings. Deepening targeted teachers with intermediate prior knowledge and experience. Creation aimed at expert-level performance among teachers with substantial AI knowledge and application experience.

5. Encourage creative teaching, plural knowledge construction and diverse expression

Education sustains civilization and its continuing development through linguistic and cultural diversity, individual meaning-making and varied expression. The author argues that reproducing training-data owners’ worldviews may suppress alternative perspectives and threaten this mission. Generative AI should therefore not be treated as an authoritative source of knowledge in any subject. Teachers and students should critically assess outputs, recognizing their usefulness for rapid information retrieval, literature searches and format conversion while expecting unreliable content. Teaching should suit the learners involved, encouraging them to evaluate projected values and cultural assumptions and to use literature synthesis and data processing for inquiry.

Students also need sufficient opportunities for trial and error without AI, empirical experiments and observation of the real world.

6. Examine long-term educational effects across disciplines and sectors

Although this study traces known processes and limitations, rapidly evolving foundation models and downstream applications interact with highly varied, uncertain educational situations. Their educational significance should not be assessed narrowly from existing defects or an exclusively sceptical standpoint. The Guidance calls for collaboration among AI researchers and providers, educational theorists, cognitive-neuroscience and other researchers, teachers, students and parents. Together they should examine long-term effects on knowledge production and learning, copyright and research, curricula and assessment, human cooperation and social development, then carefully reconsider curriculum frameworks, educational objectives and examinations and make appropriate system-level adjustments.

Assessing the educational potential of a major technological advance in content processing should not begin and end with commercial claims or productivity gains in content creation. The starting point remains human-centred principles. Educators should examine controversies across the full AI lifecycle and address implications for equity, inclusion, values and linguistic and cultural diversity. In the author’s conclusion, this is necessary to prevent individuals and communities, as educational participants and beneficiaries, from becoming experimental subjects or servants of commercially driven AI.

References

1. Bishop, J. M.(2021). Artificial Intelligence is stupid and causal reasoning won't fix it[J]. Frontier in Psychology, 2021(11):1-18.
2. Bommasani, R., Hudson, D. A., Adeli, E., et al (2021). On the opportunities and risks of foundation models[DB/OL]. [2023-10-16]. https://crfm.stanford.edu/report.html.
3. Bommasani, R., Klyman, K., Longpre, S. et al.(2023). The foundation model transparency index.[DB/OL]. [2023-10-10]. https://arxiv.org/abs/2310.12941.
4. Candelon, F., Krayer, L., Rajendran,S. et al.(2023). How people can create and destroy value with generative AI[DB/OL]. [2023.10-12]. https://www.bcg.com/publications/2023/how-people-create-and-destroy-value-with-gen-ai.
5. European Parliament(2023). MEPs ready to negotiate first-ever rules for safe and transparent Al. [DB/OL. [2023-10-18]. https://digital-strategy.ec.europa.eu/en/policies/european-approach-artificial-intelligence.
6. European Union(2016). General Data Protection Regulation[DB/OL].[2023-08-09]. http://data.europa.eu/eli/reg/2016/679/oj.
7. GitHub(2023). Hallucination Leaderboard. [DB/OL]. [2023-11-08]. https://github.com/vectara/hallucination-leaderboard.
8. Home Security Hero(2023). State of deepfakes: Realities,threats, and impact [DB/OL].[2023-11-11]. https://www.homesecurity-heroes.com/state-of-deepfakes/#key-findings.
9. Lin, B.(2023). Al is generating security risks faster than companies can keep pp.[DB/OL].[2023-08-25]. https://www.wsj.com/articles/ai-is-generating-security-risks-faster-thancompanies-can-keep-up-a2bdedd4.
10. Lutkevich, B.(2023). Model collapse explained: How synthetic training data breaks AI [DB/OL].[2023-10-28]. https://www.techtarget.com/whatis/feature/Model-collapse-explained-How-synthetic-training-data-breaks-AI.
11. Marwala, T(2023). Algorithm Bias: Synthetic Data Should Be Option of Last Resort When Training Al Systems.[DB/OL]. [2023-07-31].https://unu.edu/article/algorithm-bias-synthetic-data-should-be-option-last-resort-when-training-ai-systems.
12. Miao, F.(2022). Al and education: Guidance for policymakers|M]. Paris, UNESCO.
13. Miao, F.et al(2023). Guidance on generative Al in education and research [M]. Paris: UNESCO 2023.
14. 苗逢春(2022). 教育人工智能伦理的解析与治理: 人工智能伦理问题建议书的教育解读[J].中国电化教育,2022(6):22-36.
15. 苗逢春(2023). 数字文明变局中的教育数字化转型[J].电化教育研究，(2):47-63,91.
16. Omiye, J. A., Lester, J.C., Spichak, S. et al.(2023). Large language models propagate race-based medicine[J]. Digital Medicine, 2023(6):195:1-4.
17. Pearl, J.,& Mackenzie, D.(2018). The book of why: The new science of cause and effect [M]. NY: Basic Books.
18. Robertson, K.(2023). News group says A. I. Chatbots heavily rely on news content [EB/OL]. The New York Times. https://www.nytimes.com/2023/10/31/business/media/news-artificial-intelligence-chatbots.html.
19. Stanford University(2019). The 2019 Al Index Report[DB/OL].[2023-1018]. https://hai.stanford.edu/ai-index-2019.
20. The White House(2023). Executive order on the safe, secure, and trustworthy development and use of artificial intelligence [OL] The White House. https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/.
21. Thompson, A. D.(2023). Contents of GPT-3 & the pile v1[EB/OL].[2023-1018]. https://lifearchitect.ai/models/#gpt-3-top-10.
22. Touvron H, Martin, L, Stone, K et al.(2023) Llama 2: Open foundation and fine-tuned chat models[DB/OL][2023-10-16]. https://ai.meta.com/research/publications/llama-2-open-foundation-and-fine-tuned-chat-models.
23. UNCTAD(2023). Data protection and privacy legislation worldwide [DB/OL].[2023-07-18]. https://unctad.org/page/data-protection-and-privacy-legislation-worldwide.
24. UNESCO(2019). Beijing consensus on artificial intelligence and education[S].Paris, UNESCO.
25. UNESCO(2022). Recommendation on the Ethics of Artificial Intelligence[S]. Paris, UNESCO.
26. Zhang, D., Finckenberg-Broman, P., Hoang, T., et al(2023). Right to be forgotten in the era of large language models: Implications, challenges, and solutions [DB/OL]. arXiv: 2307.03941v3 [cs.CY]22 Sep 2023.
27. 中共中央、国务院(2022). 关于构建数据基础制度更好发挥数据要素作用的意见 [EB/OL].[2023-10-23]. https://www.gov.cn/zhengce/2022-12/19/content_5732695.htm.

Original journal English metadata (archive)

Generative AI and its Uses in Education: Foundational Controversies and Responding Strategies

MIAO Fengchun 1,2

1. National Engineering Laboratory for Cyberlearning and Intelligent Technology, Faculty of Education, Beijing Normal University,

2. Headquarters of UNESCO

Abstract: The paper is the second edition of a serials of interpretative articles on the Guidance on generative AI in education and research released by UNESCO, and the main aim of this edition is to examine the controversies around generative AI and its use in education. Based on an analysis of how it works, the development and deployment of generative AI have triggered the following controversies: worsening digital poverties, outpacing domestic governance, the collection of pre-training data without consent, the unexplainability of the architecture, the foundational models can't under the real world, the outputs of generative AI is polluting the internet, the outputs are projecting dominant values, amplifying the creation and dissemination of illegal deepfakes. Generative AI poses direct and most profound threats on the core values of education including inclusion, equity, cultural and linguistic diversities, pluralism in knowledge construction and expression, and therefore, those core values should be upheld as the logic base when examining the proportionality of generative AI to service purposes of education. Policy-makers and practitioners should follow the logic roadmap of "prioritizing regulation, ensuring inclusion, and guiding applications" to enhance the system-wide governance regulations in order to ensure trustable, self-automatic, and locally relevant eco-systems of generative AI for education, followed by developing capacities and guidance to steer proper uses of generative AI in education.

Keywords: generative AI；content processing across symbolic representations of human thinking；sources of pre-training datasets；equity, inclusion, cultural and linguistic diversities

Editorial notes and verification sources

1. Publication dates: the journal identifies February 2024, volume 30, issue 1, pages 4–15, DOI 10.13966/j.cnki.kfjyyj.2024.01.001. The site’s original archive date, 27 January 2024, is retained separately. Original paper; DOI; Journal issue
2. Technical names: the ChatGPT released on 30 November 2022 initially used the GPT-3.5 series. A token is not necessarily a byte. Stanford developed Alpaca by fine-tuning Meta’s LLaMA 7B; the original incorrectly attributed Alpaca to Meta. These names are corrected. ChatGPT launch; Stanford Alpaca
3. Model scale and training data: the original “1.8 trillion” claim for GPT-4 was not confirmed in the official technical report, which did not disclose model size. GPT-3 corpus estimates, source and language proportions, and 2012–2019 compute figures remain attributed historical material, not descriptions of initial ChatGPT or subsequent models. GPT-4 technical report
4. Historical regulation: 9 December 2023 brought a provisional EU political agreement; the formal AI Act is 2024 legislation. China’s full title is Interim Measures for the Management of Generative Artificial Intelligence Services, announced on 13 July 2023 and effective from 15 August. Proposed revisions and comparisons with the US order remain historical analysis. EU agreement, 2023; EU AI Act, 2024; CAC original
5. Age and labelling: thirteen and sixteen are recommendations discussed by the text, not universal legal chat ages. GDPR Article 8 concerns children’s consent to data processing under specified conditions. The original paragraph’s final “adults” is corrected to “minors”. Labelling or transparency requirements in Article 12 of China’s Measures and Article 50 of the EU’s 2024 AI Act should be distinguished from GDPR data-protection rules. GDPR; CAC original; EU AI Act, 2024
6. Unlearning: the original’s claim of impossibility is too absolute. Its cited Zhang et al. version of 22 September 2023 discusses differential privacy, machine unlearning, model editing and prompt engineering. The problem of removing data influence is retained, alongside these technical approaches and their implementation challenges. Cited 2023 study
7. Hallucination rates: the quoted percentages concern Vectara’s historical summary-faithfulness task. The 1 November 2023 version used 831 documents summarized by all compared models. These are not general question-answering error rates and do not establish Chinese-language error rates. A fixed historical commit is linked rather than the changing current leaderboard. Historical Vectara leaderboard
8. Deepfake statistics: the cited industry report analysed 95,820 videos and described growth of 550% since 2019. The original “nearly one million new videos” conflicts with that source; “5.5 times” should not be equated with percentage growth. The report’s 98% and 99% findings retain their survey attribution and are not presented as a global census or current figures. Deepfake report, 2023
9. Disability and authorization: the original broadly grouped minors and people with disabilities as lacking full “criminal capacity”, which is not a valid general authorization rule. This edition distinguishes applicable guardianship, age-appropriate information, accessibility, supported decision making and valid consent. CRPD Article 12 recognizes equal legal capacity and the support and safeguards needed to exercise it. Convention on the Rights of Persons with Disabilities
10. Frameworks and research judgments: Tables 2 and 3 preserve the drafts under development when the paper was written; subsequent 2024 frameworks are covered separately. Claims about understanding, knowledge, copyright and creativity are the author’s contemporary arguments. Statistics and comparative findings belong to the scopes of their original studies. Original paper

Source: Miao, Fengchun (2024), Generative AI and its Uses in Education: Foundational Controversies and Responding Strategies, Open Education Research, 30(1), 4–15. Affiliations and original English metadata are retained from the archive. Relevant rights remain with the journal and authors.

Related research and resources

Further reading: UNESCO guidance on generative AI; the first paper on educational applicability; coverage of the 2024 teacher and student AI frameworks; special and inclusive education resources.