Feature: Query String Rules

[GeekInTheNorth]

Feature: Query String Rules

Background

As a website owner, I want to configure rules that are triggered by the presence of specific query string parameters so that I can alter search engine behaviour for URL variants without requiring a code deployment.

Many websites expose multiple variations of the same page through query string parameters. Whilst these URLs may be useful to visitors, they are often undesirable for search engine indexing as they can result in duplicate content and unnecessary crawl budget consumption.

Examples include:

/listing-page/?category=123&page=1
/listing-page/?preview=true
/listing-page/?print=true
/listing-page/?sort=name
/listing-page/?view=grid

whereas URLs such as:

/listing-page/?page=1

may still be considered valid for indexing.

Rather than requiring developers to implement custom logic, administrators should be able to configure query string rules within Stott Robots Handler.

---

User Story

As a CMS Administrator

I want to configure Query String Rules that execute configurable actions when specific query string parameters are present

So that I can control how search engines interact with specific URL variants without requiring an application deployment.

---

Acceptance Criteria

Scenario 1 - Matching Query String

Given Query String Rules are enabled
And the request contains a configured query string parameter
When the page is requested
Then the configured actions are executed.

---

Scenario 2 - No Matching Query String

Given Query String Rules are enabled
And the request does not contain a configured query string parameter
When the page is requested
Then no Query String Rule actions are executed
And existing Robots Handler behaviour remains unchanged.

---

Scenario 3 - Rules Disabled

Given Query String Rules are disabled
When any page is requested
Then existing Robots Handler behaviour remains unchanged.

---

Scenario 4 - Robots Override Action

Given a Query String Rule contains the Override Robots action
When the request matches the rule
Then the configured robots directives are applied to the Robots meta tag
And the configured robots directives are applied to the X-Robots-Tag response header.

---

Scenario 5 - Link Processing Action

Given a Query String Rule contains the Add rel="nofollow" to qualifying links action
And an <a> element is rendered
And the destination URL contains a configured query string parameter
Then the rendered link contains a rel="nofollow" attribute.

---

Example Rules

Parameter	Value	Directive
page	*	noindex,follow
preview	*	noindex,nofollow
print	true	noindex,follow
search	*	noindex,follow

Configuration

Add a new administration screen named Query String Rules.

Global Settings

• Enable / Disable Query String Rules

Rules

Administrators can:

• Create rules
• Edit rules
• Delete rules
• Enable / Disable rules
• Reorder rule priority

Each rule contains:

General

• Name
• Enabled

Matching

• One or more Query String Parameter Names

  • Extensible list
  • Parameter names only (no '=' or values)
  • Validation to ensure valid parameter names
  • At least one parameter is required

Actions

A rule can execute one or more actions.

Override Robots

When enabled:

• Override the Robots meta tag
• Override the X-Robots-Tag response header

Administrators can configure:

• No Index
• No Follow

Add rel="nofollow" to Qualifying Links

When enabled:

• Any generated <a> element whose destination contains a matching query string parameter will automatically receive a nofollow relationship.

---

Processing Order

The existing robots override pipeline becomes:

1. Environment Robots Override
2. Query String Rules
3. Page-specific Robots configuration
4. Default behaviour

This ensures Query String Rules naturally extend the existing override behaviour.

---

Implementation Notes

Storage

• Store configuration in DDS.
• Continue to use the existing Repository / Service pattern.
• Cache configuration to minimise DDS lookups.

MetaRobotsTagHelper

Existing:

• If an Environment Robots Override exists, return the configured environment directives.

New:

• If a matching Query String Rule contains the Override Robots action, return the configured robots directives.

Existing:

• If no robots directives are configured, remove the meta tag.

RobotsHeaderMiddleware

Existing:

• If an Environment Robots Override exists, return the configured X-Robots-Tag.

New:

• If a matching Query String Rule contains the Override Robots action, return the configured X-Robots-Tag.

Link Processing

When rendering links:

• Detect whether the destination URL contains a configured query string parameter.

• If a matching Query String Rule contains the Add rel="nofollow" action:

  • Add nofollow to the rel attribute.
  • Preserve any existing rel values.
  • Do not add duplicate values.

---

Version 1 Scope

The initial release supports matching on:

• Presence of one or more query string parameter names.

Future enhancements may include:

• Parameter value matching
• Wildcard parameter names (e.g. utm_*)
• Regular expression matching
• Additional actions beyond robots handling

The underlying design should therefore separate matching from actions, allowing new actions to be introduced without redesigning the feature.

[GeekInTheNorth]

Questions are raised as to the use of this at a global level. But this may be better page level rule that is visible at the admin level