Building an AI Agent for managing meetings

6 min read2 days ago

An agent that summarises meeting minutes, discovers assignees & creates Jira tickets

In today’s fast-paced digital world, meetings remain one of the most widely used collaboration tools, both in personal and professional settings. However, the sheer number of meetings that professionals receive daily can be overwhelming. According to a report by Notta.ai, the The majority (83.13%) of employees spend up to one-third of their workweek in meetings. This deluge of information often leads to missed deadlines, overlooked tasks, and decreased productivity. Enter the AI agent — a smart, automated assistant designed to summarize meetings, extract action items, assign tasks, and even schedule follow-ups or create tickets. In this blog, we’ll explore the need for such an agent, its importance, and how it can revolutionize the way we manage our workflows.

The Solution: An AI Agent for Meeting Management

An AI agent designed to summarize meetings, extract action items, assign tasks, and automate follow-ups can address the challenges mentioned above. Here’s how:

Meeting notes summarization

The AI agent can quickly analyze the content of a meeting and generate a concise summary, highlighting the key points. This saves time and ensures that the recipient gets the gist of the meeting without having to read through the entire message.

For example, instead of reading a long email thread about a project update, the agent can provide a summary like: “Project X is on track. Key deliverables: Design phase completed, development to start next week. Action items: Review the design document by Friday.”

Action Item Extraction

The agent can identify specific tasks or action items mentioned in the meeting and list them clearly. This ensures that nothing falls through the cracks.

For instance, if a summary says, “Please review the report and share your feedback by EOD,” the agent can extract the action item: “Review the report and provide feedback by the end of the day.”

Assignee Identification

The agent can also identify the person responsible for each action item, either by analyzing the transcript content or by using predefined rules. This eliminates confusion and ensures accountability.

For example, if a meeting summary says, “John, please handle the client meeting, and Sarah, prepare the presentation,” the agent can assign the tasks accordingly.

Automated Follow-Ups

The agent can schedule follow-up meetings or reminders based on the action items. For example, if a task requires a follow-up meeting in a week, the agent can automatically add it to the calendar.

It can also send reminders to the assignees to ensure that tasks are completed on time.

Ticket Creation

For tasks that require tracking, the agent can create tickets in project management tools like Jira, Trello, or Asana. This ensures that all tasks are documented and tracked systematically.

Implementation

Step 1: Define the Scope and Requirements

Objective: Clearly define what the AI agent should do:

Summarize the meeting.
Extract action items and assignees.
Schedule follow-up meetings (e.g., using Google Calendar or Outlook).
Create tickets (e.g., in Jira or Trello).
Input: Meeting content (meeting transcript).
Output: Summary, action items, assignees, and automated tasks (meetings or tickets).

Step 2: Choose the Right Tools and Frameworks

Natural Language Processing (NLP): Use pre-trained models like OpenAI’s GPT, Hugging Face Transformers, or Google’s BERT for text summarization and entity extraction.

APIs for Automation:

Calendar APIs (Google Calendar, Outlook) for scheduling meetings.
Ticketing APIs (Jira, Trello) for creating tickets.

Programming Language: Python is a good choice for its rich ecosystem of AI and automation libraries.

Framework: Use a framework like LangChain or Rasa to build the agent.

Step 3: Build the Core AI Components

Transcript Summarization:

We can use an NLP model to generate a concise summary of the transcript.

Example: Fine-tune GPT or use a pre-trained summarization model from Hugging Face.

from transformers import pipeline
summarizer = pipeline("summarization")
summary = summarizer(transcript_text, max_length=130, min_length=30, do_sample=False)

Action Item and Assignee Extraction:

We can use Named Entity Recognition (NER) or fine-tune a model to identify action items and assignees.

Example: Use spaCy or a custom-trained model.

import spacy
nlp = spacy.load("en_core_web_sm")
doc = nlp(transcript_text)
for ent in doc.ents:
    if ent.label_ == "PERSON":  # Assignee
        print(f"Assignee: {ent.text}")
    if ent.label_ == "TASK":    # Action item
        print(f"Action Item: {ent.text}")

Step 4: Integrate with Automation Tools

Scheduling Follow-ups:

We can use Use calendar APIs to create events.

Example: Google Calendar API.

from google.oauth2 import service_account
from googleapiclient.discovery import build

credentials = service_account.Credentials.from_service_account_file('credentials.json')
service = build('calendar', 'v3', credentials=credentials)
event = {
    'summary': 'Follow-up Meeting',
    'start': {'dateTime': '2023-10-30T09:00:00-07:00'},
    'end': {'dateTime': '2023-10-30T09:30:00-07:00'},
}
service.events().insert(calendarId='primary', body=event).execute()

Creating Tickets:

We can use popular ticketing system APIs to create tasks. Example: Jira API.

from jira import JIRA
jira = JIRA(server="https://your-jira-instance.com", basic_auth=("username", "password"))
issue = jira.create_issue(project="PROJ", summary="Action Item", description=action_item, issuetype={"name": "Task"})

Step 5: Deploy and Test the Agent

We can use Flask or FastAPI to create a REST API for the agent.

from fastapi import FastAPI
app = FastAPI()

@app.post("/process-transcript")
def process_meeting(transcript_text: str):
    summary = summarizer(transcript_text)
    action_items = extract_action_items(transcript_text)
    schedule_follow_up()
    create_ticket(action_items)
    return {"summary": summary, "action_items": action_items}

Iterate and improve the model based on feedback.

Here is the Python project for starters

Project Structure

meeting_ai_agent/
│
├── main.py # Main script to run the agent
├── transcript_processor.py # Handles transcript processing (summarization, action items)
├── automation.py # Handles scheduling and ticket creation
├── requirements.txt # List of dependencies
└── credentials/ # Folder for API credentials
 ├── google_credentials.json
 └── jira_credentials.json

transformers==4.30.0
spacy==3.5.0
google-api-python-client==2.85.0
google-auth-oauthlib==1.0.0
google-auth-httplib2==0.1.0
jira==3.4.1
python-dotenv==1.0.0

Install Dependencies

> pip install -r requirements.txt
> python -m spacy download en_core_web_sm

# transcript_processor.py

from transformers import pipeline
import spacy

class TranscriptProcessor:
    def __init__(self):
        # Load summarization pipeline
        self.summarizer = pipeline("summarization")
        # Load spaCy model for NER
        self.nlp = spacy.load("en_core_web_sm")

    def summarize_meeting(self, transcript_text):
        """Summarize the meeting content."""
        summary = self.summarizer(transcript_text, max_length=130, min_length=30, do_sample=False)
        return summary[0]['summary_text']

    def extract_action_items(self, transcript_text):
        """Extract action items and assignees from the transcript."""
        doc = self.nlp(transcript_text)
        action_items = []
        assignees = []

        # Extract entities (customize based on your needs)
        for ent in doc.ents:
            if ent.label_ == "PERSON":  # Assignee
                assignees.append(ent.text)
            if "please" in ent.text.lower() or "action" in ent.text.lower():  # Action item
                action_items.append(ent.text)

        return action_items, assignees

# automation.py
from google.oauth2 import service_account
from googleapiclient.discovery import build
from jira import JIRA
import datetime

class Automation:
    def __init__(self):
        # Load Google Calendar credentials
        self.google_credentials = service_account.Credentials.from_service_account_file(
            'credentials/google_credentials.json',
            scopes=['https://www.googleapis.com/auth/calendar']
        )
        self.calendar_service = build('calendar', 'v3', credentials=self.google_credentials)

        # Load Jira credentials
        self.jira_options = {'server': 'https://your-jira-instance.com'}
        self.jira = JIRA(options=self.jira_options, basic_auth=('username', 'password'))

    def schedule_follow_up(self, summary, start_time):
        """Schedule a follow-up meeting using Google Calendar."""
        event = {
            'summary': summary,
            'start': {'dateTime': start_time, 'timeZone': 'UTC'},
            'end': {'dateTime': (datetime.datetime.now() + datetime.timedelta(hours=1)).isoformat(), 'timeZone': 'UTC'},
        }
        event = self.calendar_service.events().insert(calendarId='primary', body=event).execute()
        return event.get('htmlLink')

    def create_jira_ticket(self, summary, description):
        """Create a Jira ticket."""
        issue = self.jira.create_issue(
            project="PROJ",
            summary=summary,
            description=description,
            issuetype={"name": "Task"}
        )
        return issue.key

# main.py
from summarize_meeting import TranscriptProcessor
from automation import Automation

def main():
    # Sample meeting text
    transcript_text = """
    Hi Team,
    Please review the design document and provide feedback by EOD tomorrow. John, can you handle the client meeting on Friday? 
    Sarah, please prepare the presentation slides. Let's schedule a follow-up meeting next week to discuss the project status.
    Thanks,
    Alice
    """

    # Process the transcript
    processor = TranscriptProcessor()
    summary = processor.summarize_meeting(transcript_text)
    action_items, assignees = processor.extract_action_items(transcript_text)

    print("Meeting Summary:", summary)
    print("Action Items:", action_items)
    print("Assignees:", assignees)

    # Automate follow-ups and ticket creation
    automation = Automation()

    # Schedule a follow-up meeting
    follow_up_link = automation.schedule_follow_up(
        summary="Project Follow-Up",
        start_time=(datetime.datetime.now() + datetime.timedelta(days=7)).isoformat()
    )
    print("Follow-up Meeting Scheduled:", follow_up_link)

    # Create Jira tickets for action items
    for item, assignee in zip(action_items, assignees):
        ticket_key = automation.create_jira_ticket(
            summary=item,
            description=f"Assigned to {assignee}. Please complete by EOD."
        )
        print(f"Jira Ticket Created: {ticket_key}")

if __name__ == "__main__":
    main()

A sample run

Next Enhancements

User Interface: Build a simple UI for users to interact with the agent.
Error Handling: Add robust error handling for API failures or ambiguous text.
Feedback Loop: Allow users to correct summaries or action items to improve the model over time.

Conclusion

As AI technology continues to evolve, the possibilities for such agents are endless, and their importance will only continue to grow. So, if you haven’t already, it’s time to embrace the future & AI agents.