Agent Analytics

​

Intro

• AI bots & agents regularly visit websites to collect, update, and verify information not only for the training of LLMs, but also to power real-time AI search and live answers.
• This crawling behavior determines which content is available and ingested into AI chat assistants and how your brand and products are represented in AI-generated responses.
• Importantly, AI bots behave differently than the Google web crawler. Most do not run JavaScript or maintain cookies or session state. If your site is JS heavy or not server-side rendered then there’s a risk that your content is not being ingested correctly.

​

What’s included in Agent Analytics?

1. Identify which types of AI agents, bots & crawlers are accessing your site and how often they visit
2. See which specific pages & topics are being crawled or overlooked
3. Detect patterns, trends, or technical issues that could impact your AI visibility
4. Understand how LLMs and AI chatbot apps are building knowledge about your brand, products and services from your website

​

Implementation

1. Vercel Log Drain (recommended). No code required direct integration with your server logs from Vercel (Netlify, GCP and AWS coming soon).
2. Cloudflare Worker (recommended). Deploy a lightweight worker via the Cloudflare dashboard.
3. Wordpress plugin. Integrate directly with your Wordpress site, no code required.
4. Next.js SDK. Lightweight, easy to install Next.js JavaScript middleware package.
5. HTTP API Endpoint. Send event data manually from any backend via an HTTP POST request.

​

Get Your Website Key

​

Installation Steps

Vercel

1

Add new 'Custom HTTP endpoint' drain

• In Vercel navigate to Project Settings and select Drains.
• Add a new Custom HTTP endpoint drain.

2

Select "Logs" as data to drain

3

Configure the drain

• Drain Name: GPTrends Bot Analytics
• Projects: Your website project
• Sources: Select all besides builds
• Environment: Production
• Sampling Rate: 100%

4

Set the destination

Add the following POST endpoint URL with your GPTrends API Key appended after ?websiteKey=

Copy

https://api.gptrends.io/v1/intake/vercel-log-drain?websiteKey=[YOUR_GPTRENDS_KEY_HERE]

You can find your GPTrends API Key in the Bot Analytics section of the platform, either in the set up flow or under “Manage Tracking”

5

Select 'Create Drain' to save and finish

Need help? Feel free to grab some time with us here and we’ll be happy to walk you through the set up process

Cloudflare

1

Get Worker Code from GitHub

Navigate to the Cloudflare Worker repository in your browser.

2

Copy Worker Code

• Locate and click on the src/index.ts file in the repository
• Click the Raw button to view the raw code
• Select all the code and copy

3

Access Cloudflare Dashboard

Navigate to Cloudflare Dashboard and select Workers & Pages from the sidebar.

4

Create New Worker

• Click Create Application > Create Worker
• Give your worker a name like gptrends-analytics-worker
• Click Deploy to create the worker with default code

5

Edit Worker Code

• Click Edit Code button in your newly created worker
• Delete all the default code in the editor
• Paste your copied code from GitHub
• Click Save and Deploy

6

Configure Website Key

• In your worker dashboard, navigate to Settings > Variables
• Under Environment Variables, click Add variable
• Set Variable name to: SITELINE_WEBSITE_KEY
• Set Value to your GPTrends website key
• Check Encrypt to keep your key secure
• Click Save and Deploy

7

Link Worker to Your Website

• In the Cloudflare dashboard, select your domain from the Websites section
• Navigate to Workers Routes (found under Workers in the sidebar)
• Click Add Route
• Set Route to: *yourdomain.com/* (replace with your actual domain)
• Set Worker to: gptrends-analytics-worker (or whatever you named it)
• Click Save

8

Verify Installation

Visit your website and check that requests are being processed. You can view logs in Workers & Pages > Your Worker > Logs to confirm the worker is tracking visits.

Need help? Feel free to grab some time with us here and we’ll be happy to walk you through the set up process

Wordpress

See the Agent Analytics Wordpress plugin page for more info.

​

Prerequisites

Before starting, make sure you have:

• Administrator access to your WordPress site
• The possibility to install plugins on your WordPress installation

1

Open the Plugins panel

Open your WordPress website’s admin panel: https://<yourwebsite>.com/wp-adminIn the left sidebar, navigate to: Plugins

2

Search for GPTrends

Search for GPTrends in the Plugins search bar

3

Install the Plugin

Click Install Now to install the plugin.

4

Activate the Plugin

Back in the Plugins section, you should see GPTrends listed.Click Activate under the plugin’s title.

5

Configure the Plugin

Finally, in the left sidebar, click Settings then GPTrends to open the plugin’s settings page.

1. Leave Enable Tracking checked: this allows AI agents, bots and crawlers to be tracked. Unchecking this will pause tracking.
2. Paste your website’s GPTrends API key. You can get this from the GPTrends dashboard, under Agent Analytics > Manage Configuration
3. Leave API Endpoing empty (only needed for testing). Finally, click Save Changes to apply.

6

Recommended: Enable Auto-updates

To keep your integration up to date, we strongly recommend enabling auto-updates from the Plugins page

Next.js

1

Step 1

Install the GPTrends tracking package:

Copy

npm install @gptrends/track

2

Step 2

Update your middleware.ts file:

Copy

import { NextResponse } from "next/server";
import type { NextRequest } from "next/server";
import { GPTrendsTrackerNextJS } from "@gptrends/track/nextjs";

const gptrendsTrack = GPTrendsTrackerNextJS({
  websiteKey: "gptrends_secret_...", // Replace with your website key
});

export async function middleware(request: NextRequest) {
  gptrendsTrack(request);
  return NextResponse.next();
}

export const config = {
  matcher: [
    "/robots.txt",
    "/sitemap.xml",
    "/((?!api|_next/static|_next/image|favicon.ico).*)",
  ],
};

Need help? Feel free to grab some time with us here and we’ll be happy to walk you through the set up process

HTTP API

Track bot visits from any backend with a simple GET request:

Copy

GET <https://hffpucuigqemqhzokpvw.supabase.co/functions/v1/track-drain>

Required Parameters:

• url: Full URL accessed
• userAgent: Bot’s User-Agent header
• ref: Referrer header (empty if none)
• ip: Visitor IP
• websiteKey: Your GPTrends key

Example:

Copy

curl "<https://hffpucuigqemqhzokpvw.supabase.co/functions/v1/track-drain?url=https://yourwebsite.com/&userAgent=Mozilla/5.0%20(compatible;%20ChatGPT-User/1.0)&ref=&ip=1.2.3.4&websiteKey=gptrends_secret_your_key_here>"

Notes:

• Only track key pages (blogs, landing pages, robots.txt, sitemap.xml)
• Skip static assets
• Use fire-and-forget requests for zero impact on site speed
• URL-encode all parameters

Need help? Feel free to grab some time with us here and we’ll be happy to walk you through the set up process

​

Security, Privacy & Performance

• No Personal Identifying Information (PII) from users is stored or logged
• The Cloudflare Worker and Vercel Log Drain approaches run outside of the run path and have no impact on your website performance
• The Next.js package and Wordpress plugin run in a fire-and-forget mode inside middleware; requests are not awaited, so it has no impact on site performance or page load times
• All errors are safely caught, logged, and discarded — ensuring your site runs exactly as before, with zero interference to end users