On Mon, Oct 24, 2022 at 08:48:15AM -0700, Rob Clark wrote: > On Mon, Oct 24, 2022 at 5:43 AM <maxime@xxxxxxxxxx> wrote: > > > > Hi, > > > > I've discussing the idea for the past year to add an IGT test suite that > > all well-behaved KMS drivers must pass. > > > > The main idea behind it comes from v4l2-compliance and cec-compliance, > > that are being used to validate that the drivers are sane. > > > > We should probably start building up the test list, and eventually > > mandate that all tests pass for all the new KMS drivers we would merge > > in the kernel, and be run by KCi or similar. > > Let's get https://patchwork.freedesktop.org/patch/502641/ merged > first, that already gives us a mechanism similar to what we use in > mesa to track pass/fail/flake > > Beyond that, I think some of the igt tests need to get more stable > before we could consider a "mustpass" list. The kms_lease tests seem > to fail on msm due to bad assumptions in the test about which CRTCs > primary planes can attach to. The legacy-cursor crc tests seem a bit > racy (there was a patch posted for that, not sure if it landed yet), > etc. I think the safest set to start with would be pure uapi validation stuff. Anything that interactics with real world hardware is a much tougher cookie. > > The best thing to do is actually start running CI and tracking xfails > and flakes ;-) > > BR, > -R > > > I did a first pass to create a draft of such a test-suite, which would > > contain: > > > > igt@core_auth@basic-auth > > igt@core_auth@getclient-master-drop > > igt@core_auth@getclient-simple > > igt@core_auth@many-magics > > igt@core_getclient > > igt@core_getstats > > igt@core_getversion > > igt@core_hotunplug@hotrebind-lateclose > > igt@core_hotunplug@hotunbind-rebind > > igt@core_hotunplug@unbind-rebind > > igt@core_setmaster > > igt@core_setmaster_vs_auth > > igt@device_reset@unbind-reset-rebind > > igt@drm_read > > igt@dumb_buffer > > igt@fbdev > > igt@feature_discovery@display > > igt@kms_3d > > igt@kms_addfb_basic > > igt@kms_async_flips > > igt@kms_color > > igt@kms_concurrent > > igt@kms_cursor_crc > > igt@kms_cursor_edge_walk > > igt@kms_cursor_legacy@basic-busy-flip-before-cursor > > igt@kms_cursor_legacy@basic-flip-after-cursor > > igt@kms_cursor_legacy@basic-flip-after-cursor > > igt@kms_display_modes > > igt@kms_dither > > igt@kms_dp_aux_dev > > igt@kms_flip@basic-flip-vs-dpms > > igt@kms_flip@basic-flip-vs-modeset > > igt@kms_flip@basic-flip-vs-wf_vblank > > igt@kms_flip@basic-plain-flip > > igt@kms_flip_event_leak@basic > > igt@kms_force_connector_basic@force-connector-state > > igt@kms_force_connector_basic@force-edid > > igt@kms_force_connector_basic@force-load-detect > > igt@kms_force_connector_basic@prune-stale-modes > > igt@kms_getfb > > igt@kms_hdmi_inject > > igt@kms_hdr > > igt@kms_invalid_mode > > igt@kms_lease > > igt@kms_panel_fitting > > igt@kms_pipe_crc_basic > > igt@kms_plane_alpha_blend > > igt@kms_plane > > igt@kms_plane_cursor > > igt@kms_plane_lowres > > igt@kms_plane_multiple > > igt@kms_plane_scaling > > igt@kms_prop_blob > > igt@kms_properties > > igt@kms_rmfb > > igt@kms_scaling_modes > > igt@kms_sequence > > igt@kms_setmode > > igt@kms_sysfs_edid_timing > > igt@kms_tv_load_detect > > igt@kms_universal_plane > > igt@kms_vblank > > igt@kms_vrr > > igt@kms_writeback > > > > Most of them are skipped on vc4 right now, but I could see that some of > > them fail already (kms_rmfb, core_hotunplug), so it proves to be useful > > already. > > > > What do you think? Is there some more tests needed, or did I include > > some tests that shouldn't have been there? > > > > Thanks! > > Maxime -- Ville Syrjälä Intel
