Miss Islington (bot)
3b0a3c4738
[3.14] gh-138907: Support RFC 9309 in robotparser (GH-138908) (GH-149374)
...
* empty lines are always ignored instead of separating groups
* the "user-agent" line after a rule starts a new group
* groups matching the same user agent are now merged
* the rule with the longest match wins instead of the first matching rule
* in case of equal matches, the “Allow” rule wins over “Disallow”
* special characters “$” and “*” are now supported in rules
* prefer full match for user agent
(cherry picked from commit bc285e5832 )
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com >
2026-05-04 18:28:54 +00:00
..
2026-03-24 00:13:09 +00:00
2026-02-10 11:42:26 +00:00
2026-01-26 12:14:39 +00:00
2026-04-21 21:56:19 +05:30
2026-03-31 16:14:12 +02:00
2026-02-13 08:30:22 +01:00
2026-03-10 18:07:17 +00:00
2026-04-23 15:57:55 +02:00
2026-04-25 15:57:33 +00:00
2026-04-30 18:13:52 -04:00
2026-04-23 19:12:19 +00:00
2026-02-16 10:37:46 -05:00
2026-02-16 13:44:49 +01:00
2026-02-21 16:04:31 +01:00
2026-04-12 18:13:14 -07:00
2026-04-23 19:32:29 +03:00
2026-01-19 11:50:54 +00:00
2026-01-15 13:52:39 +01:00
2026-01-07 17:06:53 +01:00
2026-04-07 21:37:46 +00:00
2026-03-29 15:15:05 +02:00
2026-03-12 02:41:14 +02:00
2026-01-09 16:54:49 +00:00
2026-02-21 14:24:35 +02:00
2026-03-12 02:41:14 +02:00
2026-03-20 03:06:03 +00:00
2026-03-25 16:23:21 +00:00
2026-04-04 17:13:13 +01:00
2026-01-12 11:44:12 +02:00
2026-04-12 10:08:23 -07:00
2026-02-11 17:33:51 +02:00
2026-03-17 10:55:15 +00:00
2026-03-22 14:41:38 +00:00
2026-04-07 06:19:32 +00:00
2026-01-13 00:13:41 +00:00
2026-01-21 17:47:27 +00:00
2026-05-02 09:30:16 +00:00
2026-03-06 21:34:09 +01:00
2026-03-22 15:47:37 +00:00
2026-03-22 15:47:37 +00:00
2026-02-18 13:29:34 +00:00
2026-02-03 17:18:49 +08:00
2026-04-23 13:47:52 +00:00
2026-02-17 02:53:51 +00:00
2026-02-03 11:38:50 +01:00
2026-01-23 19:24:06 +00:00
2026-05-04 10:10:47 +00:00
2026-02-28 19:28:14 +00:00
2026-04-29 14:01:45 +00:00
2026-03-22 15:47:37 +00:00
2026-03-10 21:07:38 -07:00
2026-04-27 21:00:46 +01:00
2026-03-17 10:55:15 +00:00
2026-03-26 00:42:27 +00:00
2026-03-15 22:55:08 +00:00
2026-01-10 12:24:04 +05:30
2026-03-18 13:47:13 +01:00
2025-12-25 13:41:10 +02:00
2026-02-19 23:13:43 +00:00
2026-03-10 17:08:00 +00:00
2026-04-13 22:59:47 +00:00
2026-01-10 12:21:37 +05:30
2026-01-10 12:05:42 +00:00
2026-04-11 22:46:26 +00:00
2026-03-22 14:41:38 +00:00
2026-03-12 10:45:31 +01:00
2026-03-31 08:20:00 +00:00
2026-03-02 19:57:10 +00:00
2026-02-10 15:30:05 +01:00
2026-03-27 21:13:17 +00:00
2026-03-23 23:31:32 +01:00
2026-03-30 16:14:58 +00:00
2026-03-05 19:31:06 +00:00
2026-01-09 17:32:23 +00:00
2026-03-03 16:40:23 +01:00
2026-04-29 13:04:11 +03:00
2026-01-15 04:16:46 +00:00
2026-02-24 02:52:57 +00:00
2026-04-23 15:05:09 +02:00
2026-04-21 10:49:37 +03:00
2026-01-09 16:54:49 +00:00
2026-04-06 18:23:08 +03:00
2026-03-06 10:22:29 +00:00
2026-01-16 13:34:38 -05:00
2026-03-09 20:26:52 +00:00
2026-01-25 17:11:45 +00:00
2026-04-18 12:35:51 +00:00
2026-03-26 00:42:27 +00:00
2026-01-15 16:02:46 +00:00
2026-05-01 17:43:59 +02:00
2025-12-24 13:25:43 +00:00
2026-03-02 18:33:47 +00:00
2026-04-25 13:53:42 +01:00
2026-02-14 12:09:00 +00:00
2026-01-11 12:37:00 +00:00
2026-01-12 09:08:52 +00:00
2026-02-21 16:04:31 +01:00
2026-02-27 13:11:40 +00:00
2026-03-25 10:35:31 +01:00
2026-04-14 10:12:42 +00:00
2026-01-20 16:35:30 -08:00
2026-04-29 12:56:44 +03:00
2026-05-04 18:28:54 +00:00
2026-05-02 02:54:02 +00:00
2026-01-15 09:06:21 +00:00
2026-04-27 22:55:02 +03:00
2026-04-08 18:14:15 -04:00
2026-03-06 21:34:09 +01:00
2026-02-26 23:01:12 +00:00
2026-04-14 12:41:37 +08:00
2026-01-20 12:47:21 +00:00
2026-03-26 11:38:42 +00:00
2026-03-16 14:38:15 +05:30
2026-04-26 04:29:47 +00:00
2026-03-29 04:55:22 +00:00
2026-01-22 11:24:24 +00:00
2026-04-04 03:58:20 +00:00
2026-03-17 10:51:19 +01:00
2026-01-16 17:04:33 +00:00
2026-02-24 11:32:12 +00:00
2026-01-09 15:34:39 +00:00
2026-01-08 13:31:18 +00:00
2026-04-09 15:17:55 +00:00
2026-03-11 16:11:38 +01:00
2026-04-15 21:52:43 -07:00
2026-04-06 12:22:33 +00:00
2026-02-24 21:13:08 +00:00
2026-03-14 10:35:41 +00:00
2026-04-27 09:54:56 -07:00
2026-02-16 14:25:43 +02:00
2026-02-25 00:27:09 +02:00
2026-02-19 23:13:43 +00:00
2026-01-23 13:45:00 +02:00
2026-03-22 15:47:37 +00:00
2026-03-22 15:47:37 +00:00
2026-03-22 15:47:37 +00:00
2026-04-29 17:00:13 +02:00
2026-03-05 14:59:05 +00:00
2026-04-29 12:59:46 +03:00
2026-03-06 12:48:44 +00:00
2026-05-02 16:29:49 +00:00
2026-02-03 10:34:16 +00:00
2026-02-03 10:34:16 +00:00