Where Does AI's Knowledge Come From? Understanding the Sources, Controversies, and What You Need to Know About Training Data
Why does AI seem to know everything? Whose articles has it read? Has it even been trained on my own posts? Understanding the origins and controversies of training data will help you use AI more effectively.
AI knows everything from astronomy to geography, and the first natural question is: where does its knowledge come from, and who taught it so much? The answer involves the biggest resource struggle in the entire AI industry and is related to every piece of text you and I post online.
Conclusion first
The knowledge of large language models mainly comes from: large-scale crawling of public web pages, books and papers, code repositories, licensed purchased content (news, forums), and later human annotation and feedback. The content you publicly post online may have already been included in the training data of some model. The copyright and ethical controversy surrounding this issue is still being litigated and regulated globally.
Main sources at a glance
| Source | Content | Controversy level |
|---|---|---|
| Web crawling (e.g., Common Crawl) | Blogs, forums, news, Wikipedia | High——most without consent |
| Books and academic papers | Main source of in-depth knowledge | High——multiple copyright infringement lawsuits ongoing |
| Open-source code | Code from platforms like GitHub | Medium——disputes over license terms |
| Licensed content | News groups, Reddit, and other signed suppliers | Low——paid for |
| Human feedback (RLHF) | Evaluations by annotators | Low——but labor conditions are under scrutiny |
Why the uproar?
The core controversy in one sentence: AI companies use all of humanity's creations to train profitable products, but the creators don't receive a penny. The New York Times is suing OpenAI, record companies are suing AI music platforms, and artists are collectively suing image generation companies... Courts in various countries are deciding whether "using public content to train AI" constitutes fair use. Meanwhile, more and more media outlets are choosing to "charge if they can't win," directly licensing their content to AI companies. The outcome of this battle will determine the future business model for creative works.
Has my data been used for training too?
Likely: your public blog posts, forum comments, and public social media posts are all within the crawling range. As for your conversations with AI: most services default to using them for model improvement, but they all provide opt-out options (e.g., ChatGPT can be set to not use conversations for training, and enterprise versions default to not using them for training). Don't post sensitive information to AI, and close the settings that should be closed——for details, see Is it safe to give AI my data.
Practical implications for users
Understanding the data source will help you better understand AI's temperament: it excels at topics commonly written about online (technology, English content), but is weak in niche, local, or newly emerging knowledge (which is why it will talk nonsense with a straight face); its perspectives reflect the biases in its training data. Knowing what it was trained on will help you know when to doubt it.
Frequently Asked Questions
Will AI use my conversations with it for training purposes?
Most consumer-grade services may use conversations to improve models by default, but they usually offer an opt-out option; enterprise versions typically do not use conversations for training by default. It's recommended to avoid entering sensitive information and to check your privacy settings.
Is it legal to train AI using publicly available content?
The issue remains unresolved globally, with ongoing lawsuits and legislation: the US has several landmark cases pending, the EU requires disclosure of training data, and Japan has a relatively lenient approach. The current trend is moving towards obtaining licenses and paying fees.