randr: Check for overflow in RRChangeProviderProperty()

A client might send a request causing an integer overflow when computing
the total size to allocate in RRChangeProviderProperty().

To avoid the issue, check that total length in bytes won't exceed the
maximum integer value.

CVE-2025-49180

This issue was discovered by Nils Emmerich <nemmerich@ernw.de> and
reported by Julian Suleder via ERNW Vulnerability Disclosure.

Signed-off-by: Olivier Fourdan <ofourdan@redhat.com>
Reviewed-by: Peter Hutterer <peter.hutterer@who-t.net>
Part-of: <https://gitlab.freedesktop.org/xorg/xserver/-/merge_requests/2024>
This commit is contained in:
Olivier Fourdan 2025-05-20 15:18:19 +02:00 committed by Enrico Weigelt
parent da5f8d197f
commit 948630fa42

View File

@ -166,7 +166,8 @@ RRChangeProviderProperty(RRProviderPtr provider, Atom property, Atom type,
if (mode == PropModeReplace || len > 0) {
void *new_data = NULL, *old_data = NULL;
if (total_len > MAXINT / size_in_bytes)
return BadValue;
total_size = total_len * size_in_bytes;
new_value.data = calloc(1, total_size);
if (!new_value.data && total_size) {