Lebotfrancais/tavily

  # Tavily Integration

Tavily is a specialized web search tool designed for LLMs, offering enhanced capabilities for finding and extracting information from the web.

## Features

This integration allows your Botpress bots to:

- **Web Search**: Search the web with queries and retrieve relevant results
- **Content Extraction**: Extract content from specific URLs
- **AI-Generated Answers**: Optionally generate concise answers based on search results
- **Web Crawling**: Intelligently crawl websites starting from a base URL (beta feature)
- **Site Mapping**: Map website structures to discover available pages (beta feature)

## Setup

To use this integration, you'll need a Tavily API key:

1. Sign up for a free account at [tavily.com](https://tavily.com)
2. Get your API key from the dashboard (API keys start with `tvly-`)
3. Add your API key to the integration configuration in Botpress

## Actions

### Search

Search the web for information on a specific query.

**Input Parameters**:
- `query` (required): The search query to run
- `searchDepth`: "basic" (default) or "advanced" - determines search quality
- `topic`: "general" (default) or "news" - determines search category
- `maxResults`: Number of results to return (1-20, default: 5)
- `includeAnswer`: Generate an AI answer from search results (true/false/"basic"/"advanced")
- `timeRange`: Filter results by time (day/week/month/year)

**Output**:
- `results`: Array of search results, each with title, URL, content and relevance score
- `answer`: AI-generated answer (if requested)
- `query`: The original search query

### Extract

Extract content from specific URLs.

**Input Parameters**:
- `urls` (required): Array of URLs to extract content from (max 20)
- `extractDepth`: "basic" (default) or "advanced" - determines extraction quality
- `includeImages`: Include images in results (true/false)

**Output**:
- `results`: Array of successful extractions with URL and content
- `failed_results`: Array of URLs that couldn't be processed

### Crawl (Beta)

Crawl a website intelligently starting from a base URL.

> Note: The crawl feature is currently in invite-only beta. Contact [email protected] to request access.

**Input Parameters**:
- `url` (required): The root URL to begin the crawl
- `maxDepth`: Max depth of the crawl (default: 1)
- `maxBreadth`: Max number of links to follow per page (default: 20)
- `limit`: Total number of links to process before stopping (default: 50)
- `query`: Natural language instructions for the crawler
- `selectPaths`: Regex patterns to select specific URL paths
- `selectDomains`: Regex patterns to select specific domains
- `allowExternal`: Whether to return links from external domains (default: false)
- `includeImages`: Whether to extract image URLs from crawled pages (default: false)
- `categories`: Filter URLs by predefined categories (e.g., "Documentation", "Blog")
- `extractDepth`: Depth of content extraction ("basic"/"advanced", default: "basic")

**Output**:
- `baseUrl`: The URL you started the crawl from
- `results`: Array of crawled pages with URL, content, and images
- `responseTime`: The crawl response time

### Map (Beta)

Map a website's structure to discover available pages.

> Note: The map feature is currently in invite-only beta. Contact [email protected] to request access.

**Input Parameters**:
- `url` (required): The root URL to begin the mapping
- `maxDepth`: Max depth of the mapping (default: 1)
- `maxBreadth`: Max number of links to follow per page (default: 20)
- `limit`: Total number of links to process before stopping (default: 50)
- `query`: Natural language instructions for the mapper
- `selectPaths`: Regex patterns to select specific URL paths
- `selectDomains`: Regex patterns to select specific domains
- `allowExternal`: Whether to return links from external domains (default: false)
- `categories`: Filter URLs by predefined categories (e.g., "Documentation", "Blog")

**Output**:
- `baseUrl`: The URL you started the mapping from
- `results`: Array of discovered URLs
- `responseTime`: The mapping response time

## Examples

### Search example

```javascript
// Search for information about climate change
const searchResult = await integration.actions.search({
  query: "latest developments in climate change",
  maxResults: 3,
  includeAnswer: true
});

// Use the results or answer in your bot's response
const answer = searchResult.answer || "I couldn't find a summary, but here are some results.";
```

### Extract example

```javascript
// Extract content from specific URLs
const extractResult = await integration.actions.extract({
  urls: ["https://www.example.com/article", "https://www.example.com/blog"]
});

// Process the extracted content
const content = extractResult.results.map(r => r.raw_content).join("\n\n");
```

### Crawl example

```javascript
// Crawl a documentation website with specific focus
const crawlResult = await integration.actions.crawl({
  url: "https://docs.example.com",
  query: "API reference documentation",
  maxDepth: 2,
  selectPaths: ["/api/.*", "/reference/.*"]
});

// Process the crawled content
const apiDocs = crawlResult.results.map(p => ({ 
  url: p.url, 
  content: p.rawContent 
}));
```

### Map example

```javascript
// Map a website structure 
const mapResult = await integration.actions.map({
  url: "https://www.example.com",
  maxDepth: 3,
  categories: ["Documentation", "Blog"]
});

// Get all discovered URLs
const siteMap = mapResult.results;
```

## Credits

This integration uses the [Tavily API](https://tavily.com). You get 1,000 free API credits each month, after which usage is billed according to Tavily's pricing.