Building a Collaborative Research and Writing System

This is a more advanced example of using CrewAI with multiple agents working together to create a collaborative research and writing system.

Introduction

In this chapter, we'll explore an advanced example of using CrewAI to create a collaborative research and writing system. This system is designed to research, summarize, and write a comprehensive paper on any given topic.

This example is also available in GitHub repository that we used in the previous example

Understanding the Crew

The crew in this example consists of three specialized agents working together in a sequential flow to produce a high-quality research paper. Let's break down what the crew does:

1. It takes user input for a research topic.
2. It employs three agents: a Researcher, a Summarizer, and a Writer.
3. The agents work sequentially to research, summarize, and write a paper on the given topic.
4. The output is a comprehensive, well-structured research paper saved as a markdown file.

Agent Collaboration and Sequential Flow

1. Researcher Agent

The Researcher is the first agent in the sequence. Its primary responsibilities are:

• Researching the latest trends about the given topic.
• Ensuring the information retrieved is not older than one month.
• Producing a markdown file containing 10 links to articles about the topic, including titles and snippets.

The Researcher uses a search tool (SerperDevTool) to find relevant and up-to-date information.

2. Summarizer Agent

The Summarizer takes the output from the Researcher and performs the following tasks:

• Visiting each article link provided by the Researcher.
• Extracting relevant text from each article.
• Summarizing the extracted text in 3 to 5 sentences.
• Adding numbered citations to each article.
• Formatting references in APA version 7 style.

The Summarizer uses a web scraping tool (ScrapeWebsiteTool) to extract information from the provided links.

3. Writer Agent

The Writer is the final agent in the sequence. It takes the summarized content from the Summarizer and:

• Creates a new, engaging title for the paper.
• Combines all the summaries into a long, coherent narrative.
• Includes citation numbers for each reference.
• Groups all the APA style references into a numbered list.

The Writer's goal is to produce a comprehensive and high-quality paper based solely on the content provided by the Summarizer.

Sequential Workflow

The agents work together in the following sequence:

1. The Researcher gathers information and produces a markdown file with links and snippets.
2. The Summarizer takes this file, visits each link, summarizes the content, and creates a new markdown file with summaries and references.
3. The Writer uses the Summarizer's output to craft a final, cohesive research paper.

Each agent depends on the output of the previous agent, creating a clear, sequential workflow.

Starting the Crew

To start the crew, we follow these steps:

1. Create the crew object with all three agents and their respective tasks.
2. Get user input for the research topic.
3. Kickoff the crew with the provided topic.
4. The result is printed and saved in markdown files at each stage of the process.

Example: "Grok-2 by xAI"

Let's walk through an example using "Grok-2 by xAI" as the input topic:

1. The Researcher would find 10 recent articles about Grok-2 by xAI and create a markdown file with links and snippets.
2. The Summarizer would then visit each link, summarize the content, and create a new markdown file with concise summaries and properly formatted references.
3. Finally, the Writer would take this summarized information and craft a comprehensive research paper about Grok-2 by xAI, including an engaging title, coherent narrative, and properly cited references.

The final output would be a well-structured, informative paper about Grok-2 by xAI, saved as a markdown file.

Example Output

And example output file can be found here:
xAI's Grok-2: Revolutionizing AI Capabilities with Enhanced Chat, Coding, and Reasoning

Conclusion

This CrewAI example demonstrates a powerful, automated system for conducting research, summarizing findings, and producing high-quality papers on any given topic. By leveraging the strengths of specialized agents working in a sequential flow, it showcases the potential of AI-driven collaborative systems in academic and professional research contexts.