Appearance
Indexing Dashboard
FreshThe Indexing Dashboard gives you a complete view of how Google indexes your website. It uses the Google URL Inspection API to check each page individually, tracks status changes over time, and shows you exactly which pages are indexed, which aren't, and why.
The dashboard has three tabs:
- Overview - High-level indexing status, trends, and recent movements
- Visual Diagnostics - Charts that expose structural site issues (velocity, directory health, crawl budget, internal links, funnel)
- Log Analyzer - A combined data table merging GSC data, internal links, and server log hits with actionable insights
Figure: Indexing Dashboard with three tabs: Overview, Visual Diagnostics, and Log Analyzer
Getting Started
To use the Indexing Dashboard, you need a Google Search Console property connected to SEO Utils with at least one sitemap added.
Connect a GSC Property
If you haven't already, connect your Google Search Console property using either a Google OAuth Token or a Google Service Account. See the Google Search Console setup guide for instructions.
Add a Sitemap (if needed)
When you connect a GSC property, SEO Utils automatically detects and adds sitemaps from Google Search Console. If sitemaps were found, you can skip this step.
If no sitemaps were detected, or you want to add a different sitemap, navigate to Sitemaps from the property dropdown and add your sitemap URL (e.g., https://yoursite.com/sitemap.xml). SEO Utils will automatically fetch URLs from your sitemap daily.
Figure: Add a sitemap
Open the Indexing Dashboard
Select your property from the dropdown, then click Indexing in the navigation menu. The dashboard will begin collecting data automatically.
WARNING
On first visit, the dashboard shows "Based on 0/N URLs inspected" because no URLs have been inspected yet. Click Run Initial Scan in the top-right to start the first inspection batch, or wait for the automatic inspection to run (every 4 hours).
Shared Controls
These controls appear at the top of the dashboard and apply across all three tabs:
Date Filter
The date dropdown in the top-right controls the time range for the chart, table impressions/clicks, movements, funnel, and velocity data.
Available presets: 7 days, 14 days, 30 days, 2 months, 3 months, 6 months, 12 months. The default is 30 days.
Figure: Date range filter and the settings icon.
Sitemap Errors Banner
If any sitemap has a fetch error (e.g., HTTP 404, timeout), a dismissible banner appears at the top showing the errors. This is visible on all tabs.
Settings
Click the cog icon in the top-right to open the Indexing Settings modal. See the Settings section below for details.
Tab 1: Overview
The Overview tab shows your high-level indexing status, historical trends, and recent status changes.
Figure: Overview tab with status tabs, trend chart, and pages table
Status Tabs & Chart
Three tabs at the top - All, Indexed, Not indexed - filter both the chart and pages table. Use the Show % toggle to switch between counts and percentages.
When you select Not indexed, clickable sub-status chips appear (e.g., "Crawled - currently not indexed", "Not found (404)") to drill down by reason. Select multiple chips to combine filters.
Figure: Sub-status chips filter the chart and table by specific indexing reason
The Indexing Overview Chart shows a stacked bar trend over time - green for indexed, orange/multi-color for not indexed sub-statuses.
Pages Table
Figure: Pages table with filters, bulk actions, and per-URL indexing details
Shows every URL from your sitemap with columns for URL, Clicks, Impressions, Status (coverage state badge), Last Crawl, Rich Results, and Last Inspection.
Filters: Page URL search, Status, Content Group, and "Pages at risk of de-indexing" (indexed pages not crawled in 90+ days). Click Advanced for Clicks/Impressions range filters and Last Crawl date picker.
Actions: Click ... on any row to Check Index (re-inspect via API) or Request Indexing (submit to Google). Select multiple rows for bulk Check Index or Submit Index.
Recent Movements
Figure: Recent Movements tracking status changes across coverage, robots, and canonical fields
Tracks indexing status changes over time across three tabs:
- All Movements - Every status change with a Type column (Coverage, Indexing, Robots, Canonical)
- Indexing Changes - Only coverage and indexing state changes
- Recently Published & Not Indexed - New URLs still not indexed within the date range
Tab 2: Visual Diagnostics
The Visual Diagnostics tab shows five charts that help you identify structural indexing problems at a glance. Data is loaded only when you navigate to this tab.
Internal Link Crawler
At the top of the Diagnostics tab, a Crawler Progress Card shows the status of the internal link crawler:
- Not run yet - "Internal link crawler has not been run yet." with a Run Link Crawler button
- Running - "Crawling... X/Y URLs" with live progress updates
- Completed - "Last crawl: Apr 5, 2026 - 2000 URLs"
Click Run Link Crawler to start a manual crawl. The crawler uses headless Chrome to visit each sitemap URL, extract all internal <a> links, and store the link graph (anchor text, rel attributes, follow/nofollow). This data powers the scatter plot and the inlink counts in the Log Analyzer table.
Figure: Crawler progress card showing last crawl date and Run Link Crawler button
INFO
The crawler processes up to 2,000 URLs per cycle, prioritizing uncrawled URLs first. For larger sites, the full site is covered over multiple weekly cycles. You can configure the crawl rate and schedule in Settings.
Indexing Velocity
This chart answers: "Is Google getting faster or slower at indexing my new content?"
Figure: Indexing Velocity chart showing median time-to-index with P90 band over weeks
- Blue line - Median (P50) time-to-index in days
- Red shaded area - 90th percentile (P90) band, showing how long the slowest 10% of URLs take
The X-axis shows weeks. Each data point represents all URLs published that week and how long they took to reach "Submitted and indexed" status.
INFO
This is a forward-looking metric. It only tracks URLs discovered after you set up SEO Utils, so the data won't include historical pages that existed before your first sitemap sync. New users will see an empty state: "Awaiting new content. Velocity tracking begins when you publish new URLs to your sitemap."
Directory Health
Horizontal stacked bars showing the indexed vs. not-indexed ratio for each subdirectory of your site.
Figure: Directory Health chart showing indexed percentage per directory with green/red stacked bars
- Green = indexed percentage
- Red = not-indexed percentage
- Shows the top 10 directories by URL count, with an "Other" bucket for the rest
- Root-level pages (like
/about,/contact) are grouped into the/bucket
Use the Group By dropdown (top-right of the chart) to switch between:
- Directory Path (Auto) - Groups by first URL path segment (default)
- Content Groups - Groups by the content groups you've created in the Insights tab (using URL filters or manual page selection)
Crawl Budget by Directory
A bar chart showing how Googlebot allocates its crawl budget across your site's directories.
Figure: Crawl Budget chart showing Googlebot hit distribution by directory
This chart is only available when you have a log report linked to this domain. If no log report is linked, you'll see: "Link a log report to see crawl budget allocation by directory."
TIP
The "aha moment": Compare the Directory Health chart with the Crawl Budget chart side by side. If /search/ is only 15% indexed but consumes 75% of all Googlebot hits, you've found a crawl trap that needs fixing.
Internal Links vs. Indexation
A scatter plot that shows the correlation between internal link count and indexation success.
Figure: Scatter plot showing internal links vs indexation with two horizontal bands
- X-axis - Number of internal inlinks (from the local crawler)
- Y-axis - Two bands: "Indexed" (top) and "Not Indexed" (bottom)
- Green dots = indexed URLs, Red dots = not-indexed URLs
- Dot sizes are uniform (no bubble chart)
You'll typically see orphan pages (low inlinks) clustered in the "Not Indexed" band, visually proving that pages need internal links to get indexed.
When log data is available, a Hide Zero-Hit Pages toggle appears to filter out URLs that Googlebot hasn't visited.
WARNING
This chart requires the internal link crawler to have run. If the crawler hasn't been run yet, you'll see: "Run the internal link crawler to see this chart."
Indexing Funnel
A horizontal bar chart showing where your URLs currently sit in the indexing pipeline. Each bar's width is proportional to the total sitemap count, so you can instantly see the distribution.
Figure: Indexing Funnel showing distribution of URLs across pipeline stages with color-coded bars
| Stage | Color | What it means |
|---|---|---|
| In Sitemap | Gray | Total URLs from your sitemap (baseline - always 100%) |
| Discovered | Amber | URLs Google knows about but hasn't crawled yet - may need better internal linking |
| Crawled | Red | URLs Google crawled but rejected - content quality or technical issues |
| Indexed | Green | Successfully indexed and eligible to appear in search |
| Impressions | Violet | Indexed pages that received at least 1 impression in the selected period |
Each bar shows the count and percentage of total. Hover over any bar for a description of what the stage means.
Intentional Exclusions are listed below the bars as a text summary:
- Blocked by robots.txt - URLs your robots.txt prevents Google from crawling
- Excluded (noindex / duplicate canonical) - URLs intentionally excluded from the index
INFO
A large red "Crawled" bar relative to the green "Indexed" bar means Google is visiting your pages but rejecting the content. This is usually the most actionable insight - check those pages for thin content, duplicate issues, or soft 404s.
Tab 3: Log Analyzer
The Log Analyzer tab combines data from three sources into a single actionable table:
- GSC URL Inspection API - Coverage state for each URL
- Internal Link Crawler - Inlink count per URL
- Server Log Files - Googlebot hit counts from your server logs
Figure: Log Analyzer table showing URL, GSC Status, Inlinks, Log Hits, and Actionable Insight columns
Columns
| Column | Source | Description |
|---|---|---|
| URL | Any | The page path (click to open in browser) |
| GSC Status | URL Inspection API | Coverage state badge. Shows "Not in Sitemap" for URLs found only in logs |
| Inlinks | Local Crawler | Internal link count. Shows "-" if crawler hasn't run |
| Log Hits (30d) | Server Logs | Googlebot hit count. Shows "-" if no log report linked |
| Actionable Insight | Computed | Color-coded badge based on the rules below |
Actionable Insights
Each URL gets an automatically computed insight based on its data:
| Insight | Color | Condition |
|---|---|---|
| Healthy | Green | URL is indexed |
| High Crawl Waste | Red | High log hits + not indexed (Googlebot keeps visiting but Google won't index) |
| Low Crawl Waste | Orange | Moderate log hits + not indexed |
| Orphan Page | Amber | Few or zero internal links + not indexed (needs internal links) |
| Not in Sitemap | Purple | URL found in server logs but not in your sitemap |
The thresholds for "high" and "low" crawl waste and the orphan inlink threshold are configurable in Settings.
Filters
- URL search - Text search across all URLs
- GSC Status - Faceted filter (select one or more coverage states, or "Not in Sitemap")
- Insight - Faceted filter (Healthy, High Crawl Waste, Orphan, etc.)
- Inlinks - Range filter (from/to)
- Log Hits - Range filter (from/to)
CSV Export
Click the Export CSV button to download the full merged dataset as a CSV file. The export includes all columns and all rows (not just the current page).
INFO
The Log Analyzer table works with whatever data is available. If you haven't run the crawler, the Inlinks column shows "-". If no log report is linked, the Log Hits column shows "-". The table is still useful with just GSC data alone.
Settings
Click the cog icon in the top-right to open the Indexing Settings modal. The settings are organized into four sections.
Figure: Indexing Settings modal with auto-submit, normalization, thresholds, and crawler settings
Auto Submit for Indexing
| Setting | Description |
|---|---|
| Auto Submit for Indexing | When enabled, automatically submits not-indexed URLs to Google via the Indexing API. URL inspection always runs regardless of this setting. |
| Resubmission Cooldown | Minimum days between resubmission attempts for the same URL. Default: 7 days. Only shown when auto-submit is on. |
URL Normalization
These settings control how URLs are normalized for matching across GSC data, server logs, and crawler results.
| Setting | Description |
|---|---|
| Trailing Slash | How to handle trailing slashes: Follow Sitemap (default - strips trailing slashes), Always Add, or Always Strip |
| Keep Parameters | Query parameters to preserve (e.g., p, page, id, lang). Type a parameter name and press Enter to add it as a tag. All other parameters (including UTM tags) are stripped. |
The Recalculate Normalized Paths button recomputes all normalized paths when you change these settings. This affects how URLs are joined across the three data sources.
Actionable Insight Thresholds
Configure the sensitivity of the Log Analyzer's insight engine:
| Setting | Default | Description |
|---|---|---|
| High Crawl Waste (hits >) | 20 | Log hits above this trigger "High Crawl Waste" for not-indexed pages |
| Low Crawl Waste (hits >) | 5 | Log hits above this trigger "Low Crawl Waste" for not-indexed pages |
| Orphan Page (inlinks <) | 2 | Pages with fewer inlinks than this are flagged as orphan pages |
Internal Link Crawler Settings
| Setting | Default | Description |
|---|---|---|
| Requests per Second | 1 | How fast the crawler visits your pages. Keep low to avoid overwhelming your server. |
| Max Chrome Tabs | 5 | Number of concurrent headless Chrome tabs. Each uses ~50-100MB RAM. |
| Auto-Crawl Schedule | Weekly | How often the crawler runs automatically: Weekly, Bi-weekly, Monthly, or Manual Only |
WARNING
The crawler uses headless Chrome to render pages (capturing JavaScript-rendered links). At the default rate of 1 request/second with 5 tabs, it processes ~300 pages/minute. A 2,000-URL batch takes about 7 minutes.
How It Works Behind the Scenes
URL Inspection
SEO Utils automatically inspects your URLs using the Google URL Inspection API every 4 hours. URLs are prioritized in this order:
- Never inspected - New URLs from sitemap
- Content updated - URLs where the sitemap lastmod changed
- Not indexed - URLs that aren't indexed yet
- Indexed - Re-checked on a rotating basis (lowest priority)
INFO
Google enforces a limit of 2,000 URL Inspection API requests per day per property. For sites with more URLs, SEO Utils automatically spreads inspections across multiple days.
URL Normalization Pipeline
To join data from GSC (full URLs), server logs (request paths only), and the crawler (full URLs), all URLs pass through a normalization pipeline:
- Strip protocol and hostname →
/blog/post - Standardize trailing slashes (based on your setting)
- Remove tracking parameters (UTM, gclid, fbclid, etc.)
- Keep only configured structural parameters
- Lowercase the path
This produces a normalized_path used for exact-match joins across all three data sources.
Googlebot Verification
When importing server logs, SEO Utils verifies Googlebot requests using Google's officially published IP ranges (CIDR blocks). This prevents spoofed Googlebot user agents from polluting your data. IP ranges are refreshed daily from Google's public endpoint.
Troubleshooting
Chart shows "No chart data available"
The chart requires at least one inspection batch to complete. Either:
- Click Run Initial Scan in the top-right to start an immediate inspection
- Wait for the automatic inspection cycle (runs every 4 hours)
Chart data is generated from inspection results, so URLs must be inspected before the chart has data to display.
Dashboard shows "Based on 0/N URLs inspected"
This means URLs have been discovered from your sitemap but haven't been inspected yet. This is normal on first setup. Click Run Initial Scan or wait for the automatic cycle.
"No sitemaps found" error
Your property doesn't have a sitemap configured in SEO Utils. Navigate to Sitemaps from the property navigation and add your sitemap URL.
Velocity chart is empty
The velocity chart only tracks URLs discovered after you set up SEO Utils. If all your URLs were already in the sitemap when you first connected, the chart has no data to show. Publish new content to your sitemap and the chart will populate as those URLs get indexed.
Scatter plot says "Run the internal link crawler"
The Internal Links vs. Indexation chart requires crawl data. Go to the Visual Diagnostics tab and click Run Link Crawler at the top. The scatter plot will populate once the crawl completes.
Crawl Budget chart says "Link a log report"
The Crawl Budget chart requires server log data. Create a log report in the Log Analyzer section of the sidebar, import your server logs, and make sure the report's domain matches your GSC property domain.
Log Analyzer columns show "-"
A dash means no data is available from that source:
- Inlinks showing "-" → Run the internal link crawler
- Log Hits showing "-" → Link a log report with server log data for this domain
The table still works with partial data - you don't need all three sources to use it.
Inspection seems slow
Each URL requires a network call to Google's API. Large sites with hundreds or thousands of URLs may take several minutes to complete a full inspection cycle. This is normal.
Quota exceeded - inspections stopped
Google limits URL Inspection API to 2,000 requests per day per property. When the quota is reached, remaining URLs are scheduled for the next day. The dashboard shows partial data based on what was inspected.
Clicks and Impressions show 0 for all URLs
Clicks and impressions come from Google Search Console performance data, which requires the GSC data sync to be running. Make sure your property has synced data (check the Performance page). The indexing dashboard joins with the performance data for the selected date range.