Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-127794: Validate header name according to rfc-5322 #127820

Open
wants to merge 75 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
901a91c
gh-127794: Validate header name according rfc-5322 and allow only pri…
srinivasreddy Dec 11, 2024
5be0eaa
Add test case
srinivasreddy Dec 11, 2024
6ae6273
Add some more tests
srinivasreddy Dec 11, 2024
a7d1c0c
Fix indentation
srinivasreddy Dec 11, 2024
d04c9a2
Add news entry
srinivasreddy Dec 11, 2024
bcab963
Merge branch 'main' into gh-127794
srinivasreddy Dec 11, 2024
53bdb4f
gh-127794: Create a separate function for validating header
srinivasreddy Dec 13, 2024
31e4f3e
gh-127794: Coverge all the validations into a single regex
srinivasreddy Dec 13, 2024
fae3664
gh-127794: Revert the changes
srinivasreddy Dec 13, 2024
026f35b
gh-127794: Remove one variable
srinivasreddy Dec 13, 2024
8f6f6c3
gh-127794: Update tests
srinivasreddy Dec 13, 2024
ce18c9b
Address review comments
srinivasreddy Dec 17, 2024
aaa8879
Address review comments
srinivasreddy Dec 17, 2024
fb73967
Merge branch 'main' into gh-127794
srinivasreddy Dec 17, 2024
1de4e4c
Merge branch 'main' into gh-127794
srinivasreddy Dec 17, 2024
b29f1aa
Merge branch 'main' into gh-127794
srinivasreddy Dec 17, 2024
075a44a
Add some more tests
srinivasreddy Dec 17, 2024
d1b07c5
Fix the function name
srinivasreddy Dec 17, 2024
8129d13
Add test case for invalid_header_names
srinivasreddy Dec 17, 2024
5e371cc
Update tests
srinivasreddy Dec 17, 2024
84d2aec
Remove redundant regular expression
srinivasreddy Dec 18, 2024
ebd47a4
Update the regex per review comments
srinivasreddy Dec 18, 2024
541f59b
Update test cases
srinivasreddy Dec 18, 2024
3ac53da
Improve blurb message
srinivasreddy Dec 18, 2024
4882f44
Fix failing test
srinivasreddy Dec 18, 2024
9726c21
Update parameter name
srinivasreddy Dec 18, 2024
d6549a0
Merge branch 'main' into gh-127794
srinivasreddy Dec 18, 2024
44985a4
Update Misc/NEWS.d/next/Library/2024-12-11-17-44-36.gh-issue-127794.V…
srinivasreddy Dec 23, 2024
c83b6bf
gh-115999: Enable BINARY_SUBSCR_GETITEM for free-threaded build (gh-1…
corona10 Dec 19, 2024
d0256e6
gh-128069: brew link --overwrite tcl-tk@8 to prevent conflict with Gi…
vstinner Dec 19, 2024
6f3f0d4
gh-128013: fix data race in `PyUnicode_AsUTF8AndSize` on free-threadi…
kumaraditya303 Dec 19, 2024
efdc31f
gh-122706: fix docs for asyncio ssl sockets (#128092)
kumaraditya303 Dec 19, 2024
dd5fcc0
gh-127688: Add `SCHED_DEADLINE` and `SCHED_NORMAL` constants to `os` …
rruuaanng Dec 19, 2024
2fade09
gh-128083: Fix macro redefinition warning in clinic. (GH-127950)
ZeroIntensity Dec 19, 2024
e521e71
gh-127951: Add build option to enable pystats on Windows (GH-127952)
eendebakpt Dec 19, 2024
6853c1c
gh-128008: Add `PyWeakref_IsDead()` (GH-128009)
colesbury Dec 19, 2024
07eb113
GH-122548: Implement branch taken and not taken events for sys.monito…
markshannon Dec 19, 2024
6e55b1b
gh-115999: Specialize `STORE_ATTR` in free-threaded builds. (gh-127838)
nascheme Dec 19, 2024
d28ddba
gh-128062: Fix the font size and shortcut display of the turtledemo m…
Xiaokang2022 Dec 19, 2024
08c1491
gh-127274: Defer nested methods (#128012)
mpage Dec 19, 2024
356306e
gh-128080: remove unnecessary `__init__` method from Enum (GH-128081)
tungol Dec 19, 2024
9c87b55
gh-112328: Document EnumDict in docs and release notes (GH-121720)
u2rafi Dec 19, 2024
9afb5d8
gh-128058: Fix test_builtin ImmortalTests (#128068)
vstinner Dec 20, 2024
a548261
gh-128030: Avoid error from PyModule_GetFilenameObject for non-module…
hauntsaninja Dec 20, 2024
35fd9ed
GH-122548: Correct magic number comment (GH-128115)
markshannon Dec 20, 2024
77175aa
gh-109959: Log the current directory in test_glob.test_selflink() (#1…
vstinner Dec 20, 2024
e803dea
gh-128116: Skip test_socket VSOCK testStream() on PermissionError (#1…
vstinner Dec 20, 2024
952fc1f
gh-127946: Use a critical section for `CFuncPtr` attributes (GH-128109)
ZeroIntensity Dec 20, 2024
6ef3119
gh-127295: ctypes: Switch field accessors to fixed-width integers (GH…
encukou Dec 20, 2024
4aa1b13
GH-127705: Add debug mode for `_PyStackRef`s inspired by HPy debug mo…
markshannon Dec 20, 2024
867ddb4
Python Tutorial typo fix (#128077)
shallow-beach Dec 20, 2024
349b5ec
gh-128049: Fix type confusion bug with the return value of a custom E…
Nico-Posada Dec 20, 2024
8d4f62d
gh-112328: Make EnumDict usable on its own and document it (GH-123669)
encukou Dec 20, 2024
37e6e64
GH-127807: pathlib ABCs: remove `PurePathBase._raw_paths` (#127883)
barneygale Dec 22, 2024
3e44493
GH-127807: pathlib ABCs: remove a few private attributes (#127851)
barneygale Dec 22, 2024
6aba8e0
GH-127807: pathlib ABCs: move private copying methods to dedicated cl…
barneygale Dec 22, 2024
405782e
gh-126664: revert: Use `else` instead of `finally` in docs explaining…
gpshead Dec 22, 2024
2ad5c8f
gh-127949: fix resource warnings in `test_tasks.py` (#128172)
graingert Dec 22, 2024
6331c05
gh-119786: Fix typos in `InternalDocs/interpreter.md` (#128174)
WolframAlph Dec 22, 2024
a813023
gh-100384: Error on `unguarded-availability` in macOS builds (#128155)
zanieb Dec 22, 2024
b7bfacb
gh-126180: Remove getopt and optparse deprecation notices (GH-126227)
ncoghlan Dec 23, 2024
3ce49be
Address review comments
srinivasreddy Dec 23, 2024
7038404
Merge branch 'main' into gh-127794
srinivasreddy Dec 23, 2024
32dc6c8
Update test case
srinivasreddy Dec 23, 2024
151e146
Fix test failures
srinivasreddy Dec 23, 2024
810f9cb
Update Lib/test/test_email/test_email.py
srinivasreddy Dec 31, 2024
079dda7
Address review comments
srinivasreddy Dec 31, 2024
f69394e
Merge branch 'main' into gh-127794
srinivasreddy Dec 31, 2024
e45165f
Order imports as per case sensitive alphabetical order
srinivasreddy Jan 2, 2025
8233e92
Remove the extraneous line
srinivasreddy Jan 2, 2025
2550011
Move the validate_header_name up
srinivasreddy Jan 2, 2025
31edff1
Merge branch 'main' into gh-127794
srinivasreddy Jan 2, 2025
56a77e9
Add RFC section
srinivasreddy Jan 2, 2025
f2bbb92
Update Misc/NEWS.d/next/Library/2024-12-11-17-44-36.gh-issue-127794.V…
srinivasreddy Jan 6, 2025
98e5920
Merge branch 'main' into gh-127794
srinivasreddy Jan 8, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions Lib/email/_policybase.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
"""

import abc
import re
from email import header
from email import charset as _charset
from email.utils import _has_surrogates
Expand All @@ -14,6 +15,14 @@
'compat32',
]

# equivalent to pattern re.compile("[!-9;-~]+$")
valid_header_name_re = re.compile("[\041-\071\073-\176]+$")
srinivasreddy marked this conversation as resolved.
Show resolved Hide resolved

def validate_header_name(name):
bitdancer marked this conversation as resolved.
Show resolved Hide resolved
# Validate header name according to RFC 5322
if not valid_header_name_re.match(name):
raise ValueError(
f"Header field name contains invalid characters: {name!r}")

class _PolicyBase:

Expand Down Expand Up @@ -314,6 +323,7 @@ def header_store_parse(self, name, value):
"""+
The name and value are returned unmodified.
"""
validate_header_name(name)
return (name, value)

def header_fetch_parse(self, name, value):
Expand Down
9 changes: 8 additions & 1 deletion Lib/email/policy.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,13 @@

import re
import sys
from email._policybase import Policy, Compat32, compat32, _extend_docstrings
from email._policybase import (
Compat32,
Policy,
_extend_docstrings,
compat32,
validate_header_name
)
from email.utils import _has_surrogates
from email.headerregistry import HeaderRegistry as HeaderRegistry
from email.contentmanager import raw_data_manager
Expand Down Expand Up @@ -138,6 +144,7 @@ def header_store_parse(self, name, value):
CR or LF characters.

"""
validate_header_name(name)
if hasattr(value, 'name') and value.name.lower() == name.lower():
return (name, value)
if isinstance(value, str) and len(value.splitlines())>1:
Expand Down
24 changes: 24 additions & 0 deletions Lib/test/test_email/test_email.py
Original file line number Diff line number Diff line change
Expand Up @@ -728,6 +728,30 @@ def test_nonascii_add_header_with_tspecial(self):
"attachment; filename*=utf-8''Fu%C3%9Fballer%20%5Bfilename%5D.ppt",
msg['Content-Disposition'])

def test_invalid_header_names(self):
invalid_headers = [
('Invalid Header', 'contains space'),
('Tab\tHeader', 'contains tab'),
('Colon:Header', 'contains colon'),
('', 'Empty name'),
(' LeadingSpace', 'starts with space'),
('TrailingSpace ', 'ends with space'),
('Header\x7F', 'Non-ASCII character'),
('Header\x80', 'Extended ASCII'),
]
srinivasreddy marked this conversation as resolved.
Show resolved Hide resolved
for policy in (email.policy.default, email.policy.compat32):
for setter in (Message.__setitem__, Message.add_header):
for name, value in invalid_headers:
self.do_test_invalid_header_names(policy, setter, name, value)

def do_test_invalid_header_names(self, policy, setter, name, value):
with self.subTest(policy=policy, setter=setter, name=name, value=value):
message = Message(policy=policy)
pattern = r'(?i)(?=.*invalid)(?=.*header)(?=.*name)'
with self.assertRaisesRegex(ValueError, pattern) as cm:
setter(message, name, value)
self.assertIn(f"{name!r}", str(cm.exception))

srinivasreddy marked this conversation as resolved.
Show resolved Hide resolved
picnixz marked this conversation as resolved.
Show resolved Hide resolved
def test_binary_quopri_payload(self):
for charset in ('latin-1', 'ascii'):
msg = Message()
Expand Down
24 changes: 24 additions & 0 deletions Lib/test/test_email/test_message.py
Original file line number Diff line number Diff line change
Expand Up @@ -1004,6 +1004,30 @@ def test_folding_with_long_nospace_http_policy_1(self):
parsed_msg = message_from_bytes(m.as_bytes(), policy=policy.default)
self.assertEqual(parsed_msg['Message-ID'], m['Message-ID'])

def test_invalid_header_names(self):
invalid_headers = [
('Invalid Header', 'contains space'),
('Tab\tHeader', 'contains tab'),
('Colon:Header', 'contains colon'),
('', 'Empty name'),
(' LeadingSpace', 'starts with space'),
('TrailingSpace ', 'ends with space'),
('Header\x7F', 'Non-ASCII character'),
('Header\x80', 'Extended ASCII'),
]
for email_policy in (policy.default, policy.compat32):
for setter in (EmailMessage.__setitem__, EmailMessage.add_header):
for name, value in invalid_headers:
self.do_test_invalid_header_names(email_policy, setter, name, value)

def do_test_invalid_header_names(self, policy, setter, name, value):
with self.subTest(policy=policy, setter=setter, name=name, value=value):
message = EmailMessage(policy=policy)
pattern = r'(?i)(?=.*invalid)(?=.*header)(?=.*name)'
with self.assertRaisesRegex(ValueError, pattern) as cm:
setter(message, name, value)
self.assertIn(f"{name!r}", str(cm.exception))

def test_get_body_malformed(self):
"""test for bpo-42892"""
msg = textwrap.dedent("""\
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
The :meth:`email.message.Message.add_header` method now validates header
field names according to :rfc:`RFC 5322, Section 3.6.8 <5322#section-3.6.8>`
and raises a :exc:`ValueError` if they contain invalid characters. A similar
argument applies if headers are added through :meth:`Message.__setitem__
<object.__setitem__>`.
Loading