Project

General

Profile

Actions

Bug #13568

closed
MV MV

Bot agents can recursively call main page

Bug #13568: Bot agents can recursively call main page

Added by Matthias Van Ceulebroeck over 1 year ago. Updated over 1 year ago.

Status:
Closed
Priority:
Normal
Target version:
Start date:
02/24/2025
Due date:
% Done:

100%

Estimated time:

Description

In case of dynamic resources being present on the page, this is requested from the application using the application's entrypoint, with request=resource&resource={id}.
Of course, by that point the application has already been shut down for bots. This will cause a new WApplication being spun up, to serve the non-existing resource.
In some cases, this will result in a 404, but if the entrypoint would match an WApplication path, that can cause a whole page being served.

Many bots will then iteratively go over the response again, and again find those resources, leading to them spamming a server with requests.
It has been decided that bots will get 404 responses in case of dynamic resources. They are still able to request static resources.

MV Updated by Matthias Van Ceulebroeck over 1 year ago Actions #1

  • Status changed from InProgress to Review
  • Assignee deleted (Matthias Van Ceulebroeck)

This implementation was not changed, rather for the widget gallery specifically a robots.txt file was added.

MV Updated by Matthias Van Ceulebroeck over 1 year ago Actions #2

  • Status changed from Review to Implemented @Emweb
  • Assignee set to Matthias Van Ceulebroeck
  • % Done changed from 0 to 100

MV Updated by Matthias Van Ceulebroeck over 1 year ago Actions #3

  • Status changed from Implemented @Emweb to Implemented @Test

MV Updated by Matthias Van Ceulebroeck over 1 year ago Actions #4

  • Status changed from Implemented @Test to Closed
Actions

Also available in: PDF Atom