Multimodal WhatsApp agent

A voice message.A reply.

Start from a flow with text, audio, image and PDF inputs. Adapt the instructions and test the formats you will handle.

Start free See it in action

Requires a plan with Flows, configured connections and a test execution.

A person checks an order on their phone while a colleague packs parcels in the shop.

A task in your business

Audio: “What time can I collect the order?”

PulpoChatReply on WhatsApp

Audio: “What time can I collect the order?”

WhatsAppReceive the audio
OpenAIPrepare the reply
WhatsAppChoose text or audio

Reply on WhatsApp

You can collect it Monday to Friday, from 10 am to 6 pm.

Start from a prepared flow

Adapt connections and data

Test before activating

From the start to the result.

You included the collection hours in the agent's instructions.

An audio message arrives from a customer

Multimodal WhatsApp agent

Audio: “What time can I collect the order?”

You included the collection hours in the agent's instructions.

PulpoChatReply on WhatsApp

Audio: “What time can I collect the order?”

WhatsAppReceive the audio
The audio branch downloads and transcribes the message.
OpenAIPrepare the reply
The agent replies using the configured instructions.
WhatsAppChoose text or audio
The output condition routes the reply through the prepared branch.

Reply on WhatsApp

You can collect it Monday to Friday, from 10 am to 6 pm.

The example shows the audio branch. Check text, image and PDF separately.

Explore the template steps

The diagram shows the catalog's nodes and connections. Review their fields and credentials inside the Flows editor.

View the original catalog description

*Create your Custom Business AI Agent that speaks, sees, listens and replies to your customers.* #### 🚀 What this workflow does 1. **Receives any inbound WhatsApp message** via a **PulpoChat Trigger** 2. **Detects the medium** – text, voice note, image or document (PDF) 3. **Processes accordingly** - **Text** → straight to the AI brain - **Voice notes** → download ➜ **Whisper** transcription - **Images** → download ➜ **GPT-4o Vision** analysis - **PDFs only** → download ➜ text extraction 4. Feeds the cleaned input + short-term **memory buffer (20 turns)** to an **OpenAI Chat Agent** (GPT-4o-mini by default) 5. Sends the answer back through PulpoChat: - **If the user sent audio, the bot replies in audio** (OpenAI TTS ➜ saves mp3 to Google Drive ➜ returns the public link). - Otherwise, returns plain text. 6. Gracefully rejects anything that isn’t text, image, audio or a PDF (“Sorry, you can only send …”) *Result:* a polite, context-aware concierge that can read your contract, describe your cat photo, or summarize a 3-minute rant into a single line—without ever leaving WhatsApp. #### 🧩 Key components | Node | Purpose | |------|---------| | **PulpoChat Trigger / PulpoChat** | Receive & send WhatsApp messages | | **Switch → “Input type”** | Routes to Text / Audio / Image / Document branches | | **HTTP Request** | Securely downloads media from PulpoChat | | **OpenAI Whisper** | Turns voice notes into text | | **GPT-4o Vision** | Describes images in detail | | **Extract From File** | Converts PDFs to text | | **LangChain Agent** | Central brain with custom system prompt | | **Memory Buffer Window** | Keeps the last 20 turns per chat | | **OpenAI TTS (“Generate Audio Response”)** | Converts answers to speech (voice “nova”) | | **Google Drive (Upload + Delete)** | Stores the mp3, grabs a share link, cleans up | (Sticky notes in the canvas label the four media lanes so future-you won’t get lost.) #### 🛠️ Prerequisites - **PulpoChat** device + API key - **OpenAI** API key (chat, whisper, TTS, vision) - **Google Drive** OAuth credentials (for audio replies) #### 💡 Ideas & extensions - Pipe extracted conversation data into HubSpot or Airtable. - Replace GPT-4o with your on-prem model ➜ just swap the Chat node. - Add a **Sentiment** node to auto-escalate angry customers. - Expand document branch to Word, PowerPoint or spreadsheets. #### ⚖️ Limits & best-practice nudges - Only PDFs are accepted for now; other file types trigger a polite rejection. - The workflow rate-limits itself by design (single execution per message), but you may want extra guards if you point it at a large audience. - Delete Google Drive files after sending (already included) to keep storage costs clean. - Remember WhatsApp’s 24-hour customer-initiated window. #### 🏁 Ready, set, automate! Import → Hit **Active**. Your WhatsApp number just became a futuristic, multimodal AI agent. Enjoy the peace and quiet while it handles the chatter. 😉

The flow is prepared. You provide the configuration.

Choose “Use template”
Open Flows in your account and choose this template from the catalog. «AI Agent Bot that understands Text, Audio, Image and Documents»
Prepare its connections
Complete credentials, data and instructions for your business task.
Test the result
Run a representative case and check each step before activating the flow.

Before you start

A Pulpo account and plan with Flows available.
OpenAI and Google Drive credentials for the branches that use them.
Instructions, supported formats and the audio reply condition reviewed.

Start free See plans and terms

Before using this template.

What should I check in this flow?

The catalog title identifies this variant. It does not imply a complete course or guaranteed understanding of every file.

What happens after I register?

Open Flows and choose “Use template” in the catalog. Registration opens your account; you will still need to configure and test the flow.

Can I change the steps or stop the flow?

You can edit the nodes and deactivate the flow in the editor. Repeat the tests when you change instructions, connections or data.

Can external connections have separate costs?

Yes. AI models and connected services may require their own account, permissions and usage fees. Review each template connection.

Choose the task you need.

See every template