khoji parkAI · Menlo Park, California · 2022
Meta’s Science-Writing Demo Was Withdrawn After Two Days
Meta AI released a large language model trained on scientific literature before withdrawing it amid controversy.
Read in slides1. Research Division Announcement
Meta AI, the research division of Meta Platforms, developed a large language model named Galactica designed specifically for generating scientific text. Researchers trained the model on an extensive corpus of scientific papers, reference materials, and knowledge bases with the goal of helping users navigate information overload. On November 15, 2022, the division launched a public online demo to showcase the system's capabilities. Developers highlighted the model's performance on technical tasks such as mathematical reasoning and LaTeX equation formatting.
The model outperformed several existing language models on specialized benchmarks.
2. Early Technical Claims
According to the accompanying research paper, the model achieved high scores on technical knowledge probes and reasoning evaluations. Proponents argued that large language models could serve as novel interfaces for science by organizing complex literature and answering detailed queries. Engineers suggested the system could function as an alternative to traditional search engines for retrieving technical data. The research team open-sourced portions of the model to benefit the broader academic community.
Initial technical reception among machine learning researchers was mixed.
3. Immediate Criticisms
People testing Galactica shared outputs containing fabricated references and inaccurate scientific statements. The generated answers could include the formatting of academic writing without a real supporting publication. Michael Black was among the researchers who publicly criticized the demonstration. His examples added to the records users were posting of what the public interface produced in response to their prompts.
The criticism concerned the answers being generated by the live demonstration.
4. Internal Disagreements
Meta’s researchers and outside critics responded publicly to one another while the demonstration remained online. Yann LeCun defended the research release, while critics continued sharing problematic outputs. These were public statements about an accessible model, not a disclosed record of a private corporate meeting. The team then changed which parts of the release remained available.
The public demonstration was taken offline shortly after launch.
5. Model Withdrawal
Facing mounting controversy regarding false and misleading content generation, Meta officials decided to withdraw the public demo. The online demonstration was taken offline just three days after its initial launch on November 18, 2022. Observers praised the research team for acting in good faith and responding quickly to community concerns. The rapid withdrawal underscored the challenges major technology companies faced when releasing generative models directly to the general public.
Access to the live web demo was permanently terminated.
6. Technical Limitations
Industry analysts evaluated the incident to understand why large language models frequently invent false information. Engineers explained that standard models are trained to maximize the statistical likelihood of text in their training sets rather than verify factual accuracy. A model optimized purely for plausible language generation will naturally produce fabricated facts that sound authoritative. This maximum-likelihood training objective remains fundamentally misaligned with the goal of providing verified factual information.
Alternative training architectures optimizing for truthfulness remain under research.
7. Community Reflections
Prominent AI educators and researchers published post-mortem analyses examining the lessons learned from the rapid release and withdrawal cycle. Commentators emphasized the importance of conducting limited-access trials and thorough safety assessments before deploying powerful natural language tools. The incident joined a series of early rollouts where technology companies encountered unexpected social friction from toxic or biased model behaviors. Researchers stressed the need for mutual respect and constructive discourse within the artificial intelligence community.
Future product releases required more rigorous pre-deployment evaluation phases.
8. Subsequent Developments
Meta withdrew the public Galactica demonstration in November 2022 after users circulated examples of false or invented scientific content. The associated research paper and model work were separate from that public interface. Removing the demo did not turn its generated references into real papers or findings. The release and withdrawal occurred within days, while the examples already captured by users remained available for examination.
The public demo was withdrawn after its short release.