modesetting: Fix front_bo leak at drmmode_xf86crtc_resize on XRandR rotation

Since the introduction of "modesetting: Remove unnecessary fb addition from
drmmode_xf86crtc_resize" the fb_id isn't initialited at
drmmode_xf86crtc_resize.

Rotate operation of XRandR uses rotate_bo. So in this case the fb_id
associated to the front_bo is not initialized at drmmode_set_mode_major.
So fd_id remains 0.

As every call to drmmode_xf86crtc_resize allocates a new front_bo we should
destroy unconditionally the old_front_bo if operation success. So we free
the allocated GBM handles.

This avoids crashing xserver with a OOM in the RPI4 1Gb at 4k resolution
after 3 series xrandr rotations from normal to left and vice versa reported at
https://github.com/raspberrypi/firmware/issues/1345

Signed-off-by: Jose Maria Casanova Crespo <jmcasanova@igalia.com>
Reviewed-by: Keith Packard <keithp@keithp.com>
Closes: https://gitlab.freedesktop.org/xorg/xserver/-/issues/1024
Fixes: 8774532121 "modesetting: Remove unnecessary fb addition from
       drmmode_xf86crtc_resize"
This commit is contained in:
Jose Maria Casanova Crespo 2020-05-15 18:33:11 +02:00 committed by José Casanova Crespo
parent b670527429
commit 73480f172a

View File

@ -3233,10 +3233,10 @@ drmmode_xf86crtc_resize(ScrnInfoPtr scrn, int width, int height)
crtc->rotation, crtc->x, crtc->y); crtc->rotation, crtc->x, crtc->y);
} }
if (old_fb_id) { if (old_fb_id)
drmModeRmFB(drmmode->fd, old_fb_id); drmModeRmFB(drmmode->fd, old_fb_id);
drmmode_bo_destroy(drmmode, &old_front); drmmode_bo_destroy(drmmode, &old_front);
}
return TRUE; return TRUE;