[search] content.py diverges significantly between services/search/ and src/search/ — security risk #784

New issue

Closed

opened 2026-06-03 00:32:14 +02:00 by sleepy · 0 comments

sleepy commented

2026-06-03 00:32:14 +02:00

Owner

Problem

services/search/content.py (360 lines) and src/search/content.py (402 lines) have 160 lines of diff, making this the most divergent pair in the duplicated search modules.

Key differences:

src/ version adds import copy and uses List[] type hints (older style) vs list[] (newer)
_is_private_host() has different logic ordering and the src/ version includes additional host checks (localhost, metadata.google.internal, metadata)
_resolve_hostname_ips() uses different iteration patterns
_get_public_url() has a different signature (keyword-only args in src/)
src/ version has an extra _extract_og_image() function

This means web content fetching behaves differently depending on which code path is taken.

Risk

Security-sensitive URL validation (SSRF protection) differs between the two copies. The src/ version has stricter checks.

Fix

Merge the security improvements from src/ into services/ (canonical location)
Delete the src/search/ duplicate
See #771 for the broader dedup task

## Problem `services/search/content.py` (360 lines) and `src/search/content.py` (402 lines) have 160 lines of diff, making this the most divergent pair in the duplicated search modules. Key differences: - `src/` version adds `import copy` and uses `List[]` type hints (older style) vs `list[]` (newer) - `_is_private_host()` has different logic ordering and the `src/` version includes additional host checks (localhost, metadata.google.internal, metadata) - `_resolve_hostname_ips()` uses different iteration patterns - `_get_public_url()` has a different signature (keyword-only args in `src/`) - `src/` version has an extra `_extract_og_image()` function This means web content fetching behaves differently depending on which code path is taken. ## Risk Security-sensitive URL validation (SSRF protection) differs between the two copies. The `src/` version has stricter checks. ## Fix 1. Merge the security improvements from `src/` into `services/` (canonical location) 2. Delete the `src/search/` duplicate 3. See #771 for the broader dedup task

sleepy referenced this issue from a commit

2026-06-03 21:04:47 +02:00

[search] Reconcile diverged content.py (#784)

sleepy referenced this issue

2026-06-03 21:05:26 +02:00

[search] Reconcile diverged content.py (#784) #845

sleepy closed this issue

2026-06-03 21:05:48 +02:00

sleepy referenced this issue

2026-06-03 22:25:14 +02:00

[search] Reconcile duplicate search implementations (#771) #856