Academics hypothesize, generate data, make sense of it and then communicate it. If AI can help to generate, mine, and refine knowledge faster than human researchers, what does the future of academia look like? The answer lies not in replacing human intellect but in enhancing it, creating a collaborative synergy between AI and human researchers that will define the next era of scientific progress. I’ve been playing around with chatGPT, Google Gemini and Claude.ai to see how well they all do at creating academic papers from datasets.
AI can also serve as a tool to aid humans in data extraction from many papers. Consider a scenario where AI synthesizes information from hundreds of studies to create a refined dataset. That dataset then feeds back into the system, sparking new research papers.
This cycle—dataset to paper, paper to knowledge extraction, knowledge to new datasets—propels an accelerating loop of discovery. Instead of a linear research pipeline, AI enables a continuous, self-improving knowledge ecosystem.
From data to papers
I looked for interesting datasets on Figshare. The criteria was a) that I knew they would be re-usable as they had been cited several times. And b) the files were relatively small (<100MB) so as not to hit the limits of the common AI tools.
This one fit the bill:
Rivers, American (2019). American Rivers Dam Removal Database. figshare. Dataset. https://doi.org/10.6084/m9.figshare.5234068.v12
From there I asked Claude 3.7 Sonnet “Based on the attached files, can you create a full length academic paper with an abstract, methods results, discussion and references”. Followed by “Can you convert the whole paper to latex so I can copy and paste it into Overleaf?”
The resulting paper needs a little tweaking in the layout of the results and graphs, but other than that, has done a great job.
Papers to new data/knowledge
A single paper is just the beginning. The real challenge is synthesizing knowledge from the ever-growing volume of research. This is where specialized knowledge extraction tools become crucial. How do we effectively mine this knowledge? This is where ReadCube shines. ReadCube helps researchers manage and discover scholarly literature, but its real power lies in its knowledge extraction capabilities. Imagine ReadCube as a powerful filter, sifting through countless pages to extract the nuggets of wisdom.
Tools like ReadCube can then analyze vast collections of papers, uncovering patterns and relationships that human researchers might miss. This process involves:
- Text and citation mining: AI can analyze papers to identify emerging trends, inconsistencies, or knowledge gaps.
- Automatic synthesis: AI can compare findings across thousands of studies, synthesizing insights into new, high-level conclusions.
- Hypothesis generation: By recognizing correlations between disparate research areas, AI can propose new research directions.
The Flywheel Effect: How the Cycle Accelerates
The true magic happens when this extracted knowledge becomes the input for the next iteration. Each cycle follows this pattern:
- Raw data is processed by AI to generate initial research outputs
- Knowledge extraction tools mine these outputs for higher-order insights
- These insights form a new, refined dataset
- AI processes this refined dataset, generating more precise analyses
- The cycle continues, with each rotation producing more valuable knowledge
With each turn of this flywheel, the insights become more refined, more interconnected, and more actionable. The initial analyses might focus on direct correlations in the data, while later iterations can explore complex causal relationships, predict future trends, or suggest optimal intervention strategies.
This AI-driven, data-to-knowledge cycle represents a paradigm shift in research. Imagine the possibilities in fields like medicine, climate science, and economics. We’re moving towards a future where AI and human researchers work in synergy, pushing the boundaries of discovery. Rather than replacing researchers, AI acts as a force multiplier, enabling deeper faster knowledge generation.
