Analysis of Post-Login and SEM URLs Impacted by Meta Robots Directives on Core Web Vitals

Issue to Resolve:

We are analyzing the impact of meta robots directives (noindex, nofollow) on post-login and SEM (Search Engine Marketing) URLs. The primary concern is that these directives might be causing certain pages to appear blank in our data, suggesting that only origin data is being captured rather than specific page data. Despite these directives, we’ve noticed instances where Google considers these pages for measuring Core Web Vitals (CWV) scores, which are crucial for the overall performance and user experience of the site.

Impact on Overall Site Performance:

Presence of noindex and nofollow:

• • Many URLs have the noindex directive set to “Yes”, indicating they should not be indexed by search engines. This can prevent these pages from appearing in search results, which is typically desired for post-login or SEM pages.
  • The nofollow attribute appears on a few pages, instructing search engines not to follow links on these pages. This can impact the flow of link equity across the site.

Robots.txt and Accessibility:

• • Most pages with noindex also have their robots.txt directives set to “Allowed”. This means that while these pages are accessible to crawlers, they are instructed not to be indexed. This setup can help control what content is visible in search engines without completely blocking crawler access.

Performance Metrics (INP):

• • Several URLs show blank values for Mobile INP and Desktop INP, indicating that only origin-level data is being captured for these pages. This can be due to the noindex directive affecting how data is collected.
  • A few URLs have specific performance data:
    • https://www.example.co.uk/lp/mpclpz01.aspx has a Mobile INP of 45, indicating good performance.
    • https://www.example.co.uk/lp/mprukzlp01.aspx has a Desktop INP of 108.
    • https://www.example.co.uk/cvcheck has a Desktop INP of 212.

Conclusion:

Blank Values and Origin Data:

• • Blank values indicate that specific page-level data is not being captured, leading to reliance on origin-level data. This can obscure the true performance metrics of individual pages.

Pages with Performance Data:

• • Only a subset of pages has specific performance metrics, suggesting potential gaps in data collection or inconsistencies in reporting.

Impact of noindex:

• • Despite the noindex directive, some pages are still considered in Core Web Vitals assessments by Google. This indicates that these pages can influence the overall CWV scores, which are critical for SEO and user experience.

Recommendations:

Investigate Data Collection Issues:

• • Review and enhance the data collection processes to ensure specific page-level data is captured accurately for all URLs, especially those marked with noindex.

Verify Meta Directives Implementation:

• • Confirm the correct implementation of noindex and nofollow directives and understand their impact on data collection and reporting.

Monitor and Optimize Performance Metrics:

• • Regularly monitor performance metrics for all important pages, including those with noindex, to ensure they meet desired performance standards and contribute positively to Core Web Vitals.

By addressing these issues, we can improve the accuracy of our data, better understand the performance of individual pages, and ensure that our site meets the performance standards required for an optimal user experience and favorable Core Web Vitals scores.