Skip to content

C
criu
Commit 9db8bd8c authored by Pavel Emelyanov's avatar Pavel Emelyanov
Browse files
Options

zdtm.py: Rework tests list generation 

Hand-made list of tests is not great. As Christopher suggested we can find
all executables in a directory and treat them as tests. This idea is good,
but requires a little bit more trickery.

First, some executables in directory are not tests, e.g. these are per-test
scripts, which should be skipped. Next, not all tests in zdtm can be right
now run in continuous manner (they fail), but we want to run them manually.
To fix that such tests are marked with 'noauto' flag in the description.

So we have the test test descriptor file, which states in which file to
look for tests (executable) and which of them to exclude. Maybe more, can
be added on demand.

And, finally, the per-test description goes in a ${test}.desc file in native
python eval/repr-able format. If the file is absent a default description
is used.
Signed-off-by: 's avatarPavel Emelyanov <xemul@parallels.com>
parent 526b7826
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 156 additions and 365 deletions
test/zdtm.desc 0 → 100644
View file @ 9db8bd8c
{ 'dir': 'zdtm/live/', 'exclude': [ 'static/route_rules', 'static/criu-rtc.so' ] }
test/zdtm.list deleted 100644 → 0
View file @ 526b7826
static/pipe00:
static/pipe01:
static/pipe02:
static/busyloop00:
static/cwd00:
static/cwd01:
static/cwd02:
static/env00:
static/maps00:
static/maps01:
flags: suid
flavor: h ns
static/maps02:
static/maps04:
static/maps05:
static/mlock_setuid:
flags: suid
flavor: h ns
static/maps_file_prot:
static/mprotect00:
static/mtime_mmap:
static/sleeping00:
static/write_read00:
static/write_read01:
static/write_read02:
static/write_read10:
static/wait00:
static/vdso00:
static/sched_prio00:
flags: suid
flavor: h ns
static/sched_policy00:
flags: suid
flavor: h ns
static/file_shared:
static/file_append:
static/timers:
static/posix_timers:
static/futex:
static/futex-rl:
static/xids00:
static/groups:
flags: suid
static/pthread00:
static/pthread01:
static/umask00:
streaming/pipe_loop00:
streaming/pipe_shared00:
transition/file_read:
static/sockets00:
flags: suid
static/sockets01:
static/sockets02:
static/sock_opts00:
flags: suid
static/sock_opts01:
flags: suid
static/sockets_spair:
static/sockets_dgram:
static/socket_dgram_data:
static/socket_queues:
static/deleted_unix_sock:
static/sk-unix-unconn:
static/sk-unix-rel:
static/pid00:
flags: suid
static/pstree:
static/caps00:
flags: suid
static/cmdlinenv00:
flags: suid
static/socket_listen:
static/socket_listen6:
static/packet_sock:
flags: suid
static/packet_sock_mmap:
flags: suid
static/socket_udp:
static/sock_filter:
static/socket6_udp:
static/socket_udplite:
static/selfexe00:
static/link10:
static/unlink_fstat00:
static/unlink_fstat01:
static/unlink_fstat02:
static/unlink_fstat03:
opts: --link-remap
static/unlink_mmap00:
static/unlink_mmap01:
static/unlink_mmap02:
static/rmdir_open:
static/eventfs00:
static/signalfd00:
static/inotify00:
opts: --link-remap
static/inotify_irmap:
flags: suid
static/fanotify00:
flags: suid
flavor: h ns
static/unbound_sock:
static/fifo-rowo-pair:
static/fifo-ghost:
static/fifo:
static/fifo_wronly:
static/fifo_ro:
static/unlink_fifo:
static/unlink_fifo_wronly:
static/zombie00:
static/rlimits00:
transition/fork:
transition/fork2:
transition/thread-bomb:
static/pty00:
static/pty01:
static/pty04:
static/tty02:
static/tty03:
static/console:
flags: suid
flavor: h ns
static/vt:
flags: suid
flavor: h ns
static/child_opened_proc:
static/cow01:
flags: suid
flavor: h ns
static/pdeath_sig:
static/fdt_shared:
static/file_locks00:
flags: excl
opts: --file-locks
static/file_locks01:
flags: excl
opts: --file-locks
static/file_locks02:
flags: excl
opts: --file-locks
static/file_locks03:
flags: excl
opts: --file-locks
static/file_locks04:
flags: excl
opts: --file-locks
static/file_locks05:
flags: excl
opts: --file-locks
static/sigpending:
static/sigaltstack:
static/sk-netlink:
flags: suid
static/proc-self:
static/grow_map:
static/grow_map02:
static/grow_map03:
static/stopped:
static/chroot:
flags: suid
static/chroot-file:
flags: suid
static/rtc:
flags: suid crlib
flavor: h
transition/maps007:
flags: suid
static/dumpable01:
static/dumpable02:
flavor: h ns
static/deleted_dev:
flags: suid
flavor: h ns
static/fpu00:
arch: x86_64
static/fpu01:
arch: x86_64
static/mmx00:
arch: x86_64
static/sse00:
arch: x86_64
static/sse20:
arch: x86_64
static/vdso01:
arch: x86_64
static/vsx:
arch: ppc64le
static/file_fown:
flavor: h
static/socket-ext:
flavor: h
opts: --ext-unix-sk
static/socket-tcp:
flavor: h
opts: --tcp-established
static/socket-tcp6:
flavor: h
opts: --tcp-established
streaming/socket-tcp:
flavor: h
opts: --tcp-established
streaming/socket-tcp6:
flavor: h
opts: --tcp-established
static/socket-tcpbuf:
flavor: h
opts: --tcp-established
static/socket-tcpbuf-local:
flavor: h
opts: --tcp-established
static/socket-tcpbuf6:
flavor: h
opts: --tcp-established
static/pty03:
flavor: h
static/mountpoints:
flags: suid
flavor: h
static/utsname:
flavor: h
static/ipc_namespace:
flavor: h
static/shm:
flavor: h
static/msgque:
flavor: h
static/sem:
flavor: h
transition/ipc:
flavor: h
static/netns-nf:
flavor: h
static/netns:
flavor: h
static/cgroup00:
flags: suid
flavor: h
opts: --manage-cgroups
static/cgroup01:
flags: suid
flavor: h
opts: --manage-cgroups
static/cgroup02:
flags: suid
flavor: h
opts: --manage-cgroups --cgroup-root /newroot --cgroup-root name=zdtmtst:/zdtmtstroot
static/remap_dead_pid:
flavor: h
static/poll:
flavor: h
static/apparmor:
flags: suid
flavor: h ns
static/different_creds:
flags: suid crfail
flavor: h
static/aio00:
feature: aio_remap
static/timerfd:
feature: timerfd
static/session00:
flavor: ns uns
static/session01:
flavor: ns uns
static/tempfs:
flags: suid
flavor: ns uns
static/tempfs_ro:
flags: suid
flavor: ns
static/mnt_ro_bind:
flags: suid
flavor: ns uns
static/mount_paths:
flags: suid
flavor: ns uns
static/bind-mount:
flags: suid
flavor: ns uns
static/netns-dev:
flags: suid
flavor: ns uns
static/mnt_ext_auto:
flavor: ns uns
feature: mntid
opts: --ext-mount-map auto --enable-external-sharing
static/mnt_ext_master:
flavor: ns uns
feature: mntid
opts: --ext-mount-map auto --enable-external-masters
static/mntns_open:
flags: suid
flavor: ns uns
feature: mntid
static/mntns_link_remap:
flags: suid
flavor: ns
feature: mntid
opts: --link-remap
static/mntns_link_ghost:
flags: suid
flavor: ns
feature: mntid
static/mntns_shared_bind:
flags: suid
flavor: ns uns
feature: mntid
static/mntns_shared_bind02:
flags: suid
flavor: ns uns
feature: mntid
static/mntns_root_bind:
flags: suid
flavor: ns uns
feature: mntid
static/mntns_deleted:
flags: suid
flavor: ns uns
feature: mntid
static/tun:
flags: suid
flavor: ns uns
feature: tun
static/seccomp_strict:
flags: suid
flavor: h
feature: seccomp_suspend
static/clean_mntns:
flags: suid
flavor: ns
test/zdtm.py
View file @ 9db8bd8c
......@@ -30,9 +30,6 @@ def traceit(f, e, a):
return traceit
# Descriptor for abstract test not in list
default_test={ }
# Root dir for ns and uns flavors. All tests
# sit in the same dir
zdtm_root = None
......@@ -176,15 +173,8 @@ class zdtm_test:
self.__flavor = flavor
self.auto_reap = True
@staticmethod
def __zdtm_path(name, typ):
return os.path.join("zdtm/live/", name + typ)
def __getpath(self, typ = ''):
return self.__zdtm_path(self.__name, typ)
def __make_action(self, act, env = None, root = None):
tpath = self.__getpath('.' + act)
tpath = self.__name + '.' + act
s_args = ['make', '--no-print-directory', \
'-C', os.path.dirname(tpath), \
os.path.basename(tpath)]
......@@ -197,16 +187,16 @@ class zdtm_test:
def __pidfile(self):
if self.__flavor.ns:
return self.__getpath('.init.pid')
return self.__name + '.init.pid'
else:
return self.__getpath('.pid')
return self.__name + '.pid'
def __wait_task_die(self):
wait_pid_die(int(self.__pid), self.__name)
def start(self):
env = {}
self.__flavor.init(self.__getpath())
self.__flavor.init(self.__name)
print "Start test"
......@@ -220,7 +210,7 @@ class zdtm_test:
if self.__flavor.ns:
env['ZDTM_NEWNS'] = "1"
env['ZDTM_PIDFILE'] = os.path.realpath(self.__getpath('.init.pid'))
env['ZDTM_PIDFILE'] = os.path.realpath(self.__name + '.init.pid')
env['ZDTM_ROOT'] = self.__flavor.root
if self.__flavor.uns:
......@@ -244,7 +234,7 @@ class zdtm_test:
print "Stop test"
self.kill(signal.SIGTERM)
res = tail(self.__getpath('.out'))
res = tail(self.__name + '.out')
if not 'PASS' in res.split():
raise test_fail_exc("result check")
......@@ -276,18 +266,9 @@ class zdtm_test:
def print_output(self):
print "Test output: " + "=" * 32
print open(self.__getpath('.out')).read()
print open(self.__name + '.out').read()
print " <<< " + "=" * 32
@staticmethod
def checkskip(name):
chs = zdtm_test.__zdtm_path(name, ".checkskip")
if os.access(chs, os.X_OK):
ch = subprocess.Popen([chs])
return ch.wait() == 0 and False or True
return False
#
# CRIU when launched using CLI
......@@ -481,14 +462,52 @@ class launcher:
if self.__fail:
sys.exit(1)
def run_tests(opts, tlist):
def all_tests(opts):
desc = eval(open('zdtm.desc').read())
lst = subprocess.Popen(['find', desc['dir'], '-type', 'f', '-executable' ], \
stdout = subprocess.PIPE)
excl = map(lambda x: os.path.join(desc['dir'], x), desc['exclude'])
tlist = filter(lambda x: \
not x.endswith('.checkskip') and \
not x.endswith('.hook') and \
not x in excl, \
map(lambda x: x.strip(), lst.stdout.readlines()) \
)
lst.wait()
return tlist
# Descriptor for abstract test not in list
default_test={ }
def get_test_desc(tname):
d_path = tname + '.desc'
if os.access(d_path, os.F_OK):
return eval(open(d_path).read())
return default_test
def self_checkskip(tname):
chs = tname + '.checkskip'
if os.access(chs, os.X_OK):
ch = subprocess.Popen([chs])
return ch.wait() == 0 and False or True
return False
def run_tests(opts):
excl = None
features = {}
if opts['all']:
torun = tlist
torun = all_tests(opts)
run_all = True
elif opts['test']:
torun = opts['test']
run_all = False
else:
print "Specify test with -t <name> or -a"
return
......@@ -506,11 +525,15 @@ def run_tests(opts, tlist):
print "Skipping %s (exclude)" % t
continue
tdesc = tlist.get(t, default_test) or default_test
tdesc = get_test_desc(t)
if tdesc.get('arch', arch) != arch:
print "Skipping %s (arch %s)" % (t, tdesc['arch'])
continue
if run_all and test_flag(tdesc, 'noauto'):
print "Skipping test %s (manual run only)" % t
continue
feat = tdesc.get('feature', None)
if feat:
if not features.has_key(feat):
......@@ -521,7 +544,7 @@ def run_tests(opts, tlist):
print "Skipping %s (no %s feature)" % (t, feat)
continue
if zdtm_test.checkskip(t):
if self_checkskip(t):
print "Skipping %s (self)" % t
continue
......@@ -534,9 +557,10 @@ def run_tests(opts, tlist):
finally:
l.finish()
def list_tests(opts, tlist):
for t in tlist:
print t
def list_tests(opts):
tlist = all_tests(opts)
print '\n'.join(tlist)
#
# main() starts here
......@@ -582,9 +606,8 @@ lp = sp.add_parser("list", help = "List tests")
lp.set_defaults(action = list_tests)
opts = vars(p.parse_args())
tlist = yaml.load(open("zdtm.list"))
if opts['debug']:
sys.settrace(traceit)
opts['action'](opts, tlist)
opts['action'](opts)
test/zdtm/live/static/aio00.desc 0 → 100644
View file @ 9db8bd8c
{'feature': 'aio_remap'}
test/zdtm/live/static/apparmor.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'h ns', 'flags': 'suid'}
test/zdtm/live/static/bind-mount.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'ns uns', 'flags': 'suid'}
test/zdtm/live/static/cgroup00.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'h', 'flags': 'suid', 'opts': '--manage-cgroups'}
test/zdtm/live/static/cgroup01.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'h', 'flags': 'suid', 'opts': '--manage-cgroups'}
test/zdtm/live/static/cgroup02.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'h', 'flags': 'suid', 'opts': '--manage-cgroups --cgroup-root /newroot --cgroup-root name=zdtmtst:/zdtmtstroot'}
test/zdtm/live/static/clean_mntns.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'ns', 'flags': 'suid'}
test/zdtm/live/static/console.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'h ns', 'flags': 'suid'}
test/zdtm/live/static/cow01.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'h ns', 'flags': 'suid'}
test/zdtm/live/static/deleted_dev.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'h ns', 'flags': 'suid'}
test/zdtm/live/static/different_creds.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'h', 'flags': 'suid crfail'}
test/zdtm/live/static/fanotify00.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'h ns', 'flags': 'suid'}
test/zdtm/live/static/file_locks00.desc 0 → 100644
View file @ 9db8bd8c
{'flags': 'excl', 'opts': '--file-locks'}
test/zdtm/live/static/file_locks01.desc 0 → 100644
View file @ 9db8bd8c
{'flags': 'excl', 'opts': '--file-locks'}
test/zdtm/live/static/file_locks02.desc 0 → 100644
View file @ 9db8bd8c
{'flags': 'excl', 'opts': '--file-locks'}
test/zdtm/live/static/file_locks03.desc 0 → 100644
View file @ 9db8bd8c
{'flags': 'excl', 'opts': '--file-locks'}
test/zdtm/live/static/file_locks04.desc 0 → 100644
View file @ 9db8bd8c
{'flags': 'excl', 'opts': '--file-locks'}
test/zdtm/live/static/file_locks05.desc 0 → 100644
View file @ 9db8bd8c
{'flags': 'excl', 'opts': '--file-locks'}
test/zdtm/live/static/maps01.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'h ns', 'flags': 'suid'}
test/zdtm/live/static/mlock_setuid.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'h ns', 'flags': 'suid'}
test/zdtm/live/static/mnt_ext_auto.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'ns uns', 'feature': 'mntid', 'opts': '--ext-mount-map auto --enable-external-sharing'}
test/zdtm/live/static/mnt_ext_master.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'ns uns', 'feature': 'mntid', 'opts': '--ext-mount-map auto --enable-external-masters'}
test/zdtm/live/static/mnt_ro_bind.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'ns uns', 'flags': 'suid'}
test/zdtm/live/static/mntns_deleted.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'ns uns', 'flags': 'suid', 'feature': 'mntid'}
test/zdtm/live/static/mntns_link_ghost.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'ns', 'flags': 'suid', 'feature': 'mntid'}
test/zdtm/live/static/mntns_link_remap.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'ns', 'flags': 'suid', 'feature': 'mntid', 'opts': '--link-remap'}
test/zdtm/live/static/mntns_open.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'ns uns', 'flags': 'suid', 'feature': 'mntid'}
test/zdtm/live/static/mntns_root_bind.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'ns uns', 'flags': 'suid', 'feature': 'mntid'}
test/zdtm/live/static/mntns_shared_bind.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'ns uns', 'flags': 'suid', 'feature': 'mntid'}
test/zdtm/live/static/mntns_shared_bind02.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'ns uns', 'flags': 'suid', 'feature': 'mntid'}
test/zdtm/live/static/mount_paths.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'ns uns', 'flags': 'suid'}
test/zdtm/live/static/mountpoints.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'h', 'flags': 'suid'}
test/zdtm/live/static/netns-dev.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'ns uns', 'flags': 'suid'}
test/zdtm/live/static/rtc.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'h', 'flags': 'suid crlib'}
test/zdtm/live/static/sched_policy00.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'h ns', 'flags': 'suid'}
test/zdtm/live/static/sched_prio00.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'h ns', 'flags': 'suid'}
test/zdtm/live/static/seccomp_strict.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'h', 'flags': 'suid', 'feature': 'seccomp_suspend'}
test/zdtm/live/static/socket-ext.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'h', 'opts': '--ext-unix-sk'}
test/zdtm/live/static/socket-tcp.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'h', 'opts': '--tcp-established'}
test/zdtm/live/static/socket-tcp6.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'h', 'opts': '--tcp-established'}
test/zdtm/live/static/socket-tcpbuf-local.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'h', 'opts': '--tcp-established'}
test/zdtm/live/static/socket-tcpbuf.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'h', 'opts': '--tcp-established'}
test/zdtm/live/static/socket-tcpbuf6.desc 0 → 100644
View file @ 9db8bd8c
{'flavor': 'h', 'opts': '--tcp-established'}
test/zdtm/live/static/sse20.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/static/tempfs.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/static/tempfs_ro.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/static/timerfd.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/static/tty00.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/static/tun.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/static/unhashed_proc.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/static/unlink_fstat03.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/static/unlink_largefile.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/static/uptime_grow.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/static/utsname.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/static/vdso01.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/static/vsx.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/static/vt.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/streaming/fifo_dyn.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/streaming/fifo_loop.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/streaming/file_aio.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/streaming/netlink00.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/streaming/socket-tcp.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/streaming/socket-tcp6.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/streaming/socket_loop00.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/streaming/unix_sock.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/transition/epoll.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/transition/ipc.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/transition/maps007.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
test/zdtm/live/transition/ptrace.desc 0 → 100644
View file @ 9db8bd8c
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment