A static test has to be stopped, because zdtm.py compares file descriptors
before and after c/r.
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

Commit d9486bd7 ("arm/pie/build: do not produce relocatable parasite
object") removed -r from LDFLAGS used to compile criu pie. This
functionality somehow never made it to criu-dev, and was also lost
in master then compel was ported to it.

Make it work with compel.

Unfortunately it was not as simple as I initially thought, as -r flag
to ld was built into nmk. This patch removes it, and adds it to all
places that need intermediate linking.

Cc: Dmitry Safonov <dsafonov@virtuozzo.com>
Cc: Cyrill Gorcunov <gorcunov@openvz.org>
Acked-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

The objective is to only do parasite code linking once -- when we link
parasite objects with compel plugin(s). So, let's use ar (rather than
ld) here. This way we'll have a single ld invocation with the proper
flags (from compel ldflags) etc.

There are two tricks in doing it:

1. The order of objects while linking is important. Therefore, compel
   plugins should be the last to add to ld command line.

2. Somehow ld doesn't want to include parasite-head.o in the output
   (probably because no one else references it), so we have to force
   it in with the modification to our linker scripts.

NB: compel makefiles are still a big mess, but I'll get there.
Acked-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

When performing static linking, the order of objects is important
(when the linker finds an undefined reference, it looks forward,
never back). Therefore, sorting objects breaks things.

Required for the next patch.

Cc: Cyrill Gorcunov <gorcunov@openvz.org>
Acked-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

1. Remove .FORCE, it's not used.

2. Consolidate CFLAGS stripping into a single line.
Acked-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

The way criu/pie/Makefile is currently written, ld is run twice:

 1. link $(NAME-obj-y) objects to NAME.built-in.o

 2. link NAME.built-in.o, pie.lib.a, and compel plugins
    to NAME.built-in.bin.o (with compel ldflags and linker script)

There is absolutely no need for such two-stage linking, but it was OK.
It is not OK now, as "compel ldflags" for ARM doesn't need -r, and we
can't run the first stage with -r and the second stage without it.

So, let's simplify linking using a single ld invocation. This is my
third attempt in doing it, I think I nailed it this time -- it is now
clean and (relatively) simple.

While at it:
 - fix compel linker script dependency (it was not working);
 - rearrange the Makefile so variables goes first, then rules;
 - remove a comment about mount implementation in restorer.

NOTE that compel is called with ./ prefix so the file paths it prints
are also prefixed with ./, which is needed for objectify macro to ignore
those.
Acked-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

1. Remove useless LDFLAGS -- dynamic linker is not used here.

2. Remove $(iquotes) -- there are none defined here.

3. Consolidate filter-out statements into a single one.
Acked-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

This is an addition to commit 3a7e027b ("criu: pie -- Switch to use
compel shipped lds scripts").
Acked-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

There are too many directories in the include path.

Let's remove this one, and refer to the blobs with pie/ prefix.
Acked-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

This is a missing hunk from commit 85b04c8b "Makefiles: nuke $(SRC_DIR)"
which has somehow escaped from my attention.
Acked-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

It can be passed as link to real object.
Signed-off-by: Stanislav Kinsburskiy <skinsbursky@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

Usually we print the PID we wait for, let's do the same here.
Probably just a typo.

Cc: Kirill Tkhai <ktkhai@virtuozzo.com>
Reviewed-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

I guess this is a followup to commit 59e80d4f.

1 s/determinate/figure out/ (there is no "determinate" verb in English)

2 moved 'with' to the end

Cc: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

These helpers are valuable and can be used outside.
Signed-off-by: Stanislav Kinsburskiy <skinsbursky@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

Before dump:

438 437 0:51 /mtest /zdtm/static/mnt_ext_manual.test rw,relatime - tmpfs zdtm_auto_ext_mnt rw
439 437 0:51 /mtest /zdtm/static/mnt_ext_manual_private_shared_bind.test rw,relatime shared:144 - tmpfs zdtm_auto_ext_mnt rw
440 437 0:51 /mtest /zdtm/static/mnt_ext_manual_bind.test rw,relatime shared:144 - tmpfs zdtm_auto_ext_mnt rw
441 437 0:51 /mtest /zdtm/static/mnt_ext_manual_slave_shared_bind.test rw,relatime shared:145 master:144 - tmpfs zdtm_auto_ext_mnt rw
442 437 0:51 /mtest /zdtm/static/mnt_ext_manual_slave_bind.test rw,relatime master:145 - tmpfs zdtm_auto_ext_mnt rw

>From log: mount mnt_ext_manual_slave_bind is restored as slave of
mnt_ext_manual_slave_shared_bind, which is restored as slave of
mnt_ext_manual_private_shared_bind, which is restored as shared of
mnt_ext_manual_bind, which is restored as private bind of external,
that is exactly the same way mnt_is_external() looks for external.

v2: fix test to cover all cases in mnt_is_external()
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

If container has external bindmount given to criu through
--ext-mount-map option by admin, container user can bindmount
subdirs of these external bindmount to somewhere else inside
container creating secondary external bindmounts. Criu we will
fail to restore them as having unreachable sharing. But we can
restore secondary external bindmounts bindmounting them from
primary external bindmount.

v2: s/external_bind/mnt_is_external/, make mnt_is_external bool,
do mnt_is_external without recursion
v3: add debug message on propagate_mount when bind is set
v5: remove hunk of v4 which goes to previous patch, so same as v3
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

need it to check if we can bindmount from external mount

note: when migrating from criu with patch to criu without,
external mount mapping won't work, we do not support it.

v2: s/real_root/ext_real_root/
v4: add comment
v5: use ext_key field for mapping, put NO_ROOT_MOUNT in root
for old externals for which we do not have it.
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

"return" and "exit" are mixed in this function, and this is wrong.
Must be "return" only, because its callers don't want exit.
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

mi->external is always false in these places
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

s/find_widest_shared/find_wider_shared/
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

"make uninstall" is supposed to remove all the files that
"make install" (with the same arguments) have created.
This is a test to check that.

PS ideally, "make uninstall" should also remove any empty directories,
but let's not care about it for now.
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

As we compile-test non-x86_64 architectures under qemu emulation,
it works pretty slow.
Dmitry Safonov suggested, and Andrey Vagin initially implemented
supporting ccache for such builds. This patch is based heavily
on Andrey's work -- all the bugs added are purely mine though.

Performance results: in an ideal environment (two builds of the same
code, one with cold (empty) ccache, another with the hot one)
I saw compile time improvements of 4x to 5x, and total test run time
improvement up to 2x to 2.5x. In layman terms, the complete test run
that was taking more than 50 minutes now takes about 25!

Notes on handling .ccache directory:

1. Before running docker build, .ccache directory (saved in between
   runs by Travis) is moved to criu source code root, from where it
   is copied by docker together with criu sources.

2. In Dockerfile, .ccache gets moved to /tmp, and CCACHE_DIR
   is set accordingly.

3. After running docker build, .ccache is copied out from docker
   container back to the host (from where it is saved by Travis).

Ccache envorinment notes:

1. CCACHE_NOCOMPRESS is needed because otherwise tons of time is spent
   on compression/decompression (big performance hit under qemu).

2. CCACHE_CPP2 is required with clang, see detailed explanation at
   http://petereisentraut.blogspot.com/2011/09/ccache-and-clang-part-2.html
   The logic of setting CCACHE_CPP2 in Dockerfile is somewhat fancy;
   unfortunately I was not able to come up with a simpler approach.

Misc:

1. Travis runs "ccache -s" after the build is completed. A call to
   "ccache -s" is called to Dockerfile before make, so one can
   compare before/after numbers.

2. make invocations are surrounded by "date" calls so one can get the
   compilation times.

Changes in v2:
 - consolidate Dockerfile statements (ENV, RUN)
 - single object make test is no longer commented out
 - simplify "make mrproper" test
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Reviewed-by: Dmitry Safonov <dsafonov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

For compatibility with ccache (so it can cache the compilation results),
let's compile and link separately. For this, we have to
 - disable the implicit make rules
 - write the explicit ones

While at it, do use the "silent make" stuff that is already here

FIXME: figure out if it helps to speed up ccache build
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

For now, it's done for local builds only (i.e. no per-arch Docker
builds yet). The reason is, it's easier to play with ccache when
the compiles are (relatively) fast.

Performance: there is 2x to 3x improvement in build speeds for
ideal cases (same code, hot cache), but the absolute savings
are negligible (example: 7 seconds instead of 23).

Note that ccache is not compatible with gcov, so we only enable it
for non-gcov build (which happens to be the one with clang).
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

Unify alpine and non-alpine builds. The only difference is foreign
arch builds need some preparation -- separate that to a dependency.

Unfortunately we can't use wildcard targets ("%: ") as non-wildcard
ones are prevaling. Therefore, a somewhat ugly hack to generate
$arch: Dockerfile.$arch dependency is added.

While at it:
 - rename DB_ARGS to DB_CC
 - mark clean as phony
 - make "all" really run all the builds we can

Nice side effect: autocompletion ("make -C scripts/build <TAB>")
now works!
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

As pointed out by Andrey, arch clang tests are using gcc regardless
of CLANG=1 set in travis environment.

Frankly, I do not understand how it worked before (while being
pretty sure it worked!), but here is a way to fix it.
Reported-by: Andrey Vagin <avagin@virtuozzo.com>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

This is generated by clang-3.8:
> compel/arch/ppc64/src/lib/infect.c:31:20: error: unused function
> '__check_code_syscall' [-Werror,-Wunused-function]
> static inline void __check_code_syscall(void)
>                    ^

The fix is the same as in commit 3ea2fd78.
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

clang-3.8 complains:

> criu/arch/arm/bitops.S:5:2: error: invalid instruction
>  strneb r1, [ip] @ assert word-aligned
>  ^

Apparently (see [1]) this is some old asm syntax, which, I guess,
was deliberately dropped from clang.

[1] https://sourceware.org/ml/libc-ports/2013-03/msg00095.html

Cc: Kirill Tkhai <ktkhai@virtuozzo.com>
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Reviewed-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

If using the '--keep-going' option, zdtm prints out an overview how many
tests were run, failed and skipped. This would also be useful to know if
it did not fail. This patch changes the output like this:

 ################## ALL TEST(S) PASSED (TOTAL 297/SKIPPED 36) ###################

or in the case of a failure it is unchanged:

 ################### 2 TEST(S) FAILED (TOTAL 297/SKIPPED 34) ####################
  * zdtm/static/sched_policy00(ns)
  * zdtm/static/cgroup02(h)
 ##################################### FAIL #####################################
Signed-off-by: Adrian Reber <areber@redhat.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

decode_flav() was doing 'if i in flavors:' where 'i' was an integer but
the keys from the flavors dict are strings 'h', 'ns' and 'uns'.
Signed-off-by: Adrian Reber <areber@redhat.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

When infecting victim we construct sigframe to
be able to self-rectore it in case if something
goes wrong. But in case is a targer been using
alternative stack for signal handling it will
be missed in sigframe since we don't fetch it.

Thus add fetching sas on infection stage and
put it into signal frame early.
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

We need to leave this part of restorer in restored task in case
of inserted redirected calls. Jump trampolines from old vdso lead
here - it would be painful to land nowhere.
Signed-off-by: Dmitry Safonov <dsafonov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

Reworked this code a little and it becomes more readable.
Drop those macroses under CONFIG_X86_64 define and just use
boolean `compat_vdso' to check whether insert 64 or 32-bit jmp.
Signed-off-by: Dmitry Safonov <dsafonov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

It's the same on all archs (as it's called from generic code),
no need to redeclare it. <parasite-vdso.h> is included in all
per-arch headers. Drop vdso_symtable forward declaration as
<parasite-vdso.h> includes <util-vdso.h> header which defines
the structure.
Signed-off-by: Dmitry Safonov <dsafonov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

When checkpointing applications with really big
memory slab (like in our vz7 test with 920G of
memory) the int type get cutted, we should use
long int instead, just like we do in other code
pieces.

Otherwise get (on vz7's criu, which s sharing the code)

 | pie: 756: Daemon waits for command
 | (01.193097) Wait for ack 12 on daemon socket
 | (01.193112) Fetched ack: 12 12 0
 | (01.193164) 988065 fdinfo 0: pos:                0 flags:           100002/0
 | (01.193201) fdinfo: type: 0xb flags: 0100002/0 pos:        0 fd: 0
 | (01.193279) 988065 fdinfo 1: pos:                0 flags:           100002/0
 | (01.193307) fdinfo: type: 0xb flags: 0100002/0 pos:        0 fd: 1
 | (01.193341) 988065 fdinfo 2: pos:                0 flags:           100002/0
 | (01.193365) fdinfo: type: 0xb flags: 0100002/0 pos:        0 fd: 2
 | (01.193375) ----------------------------------------
 | (01.193405) Error (criu/parasite-syscall.c:243): BUG at criu/parasite-syscall.c:243
 | pie: 756: Error (criu/pie/parasite.c:676): Trimmed message received (1>
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

Supporting shutdown for dgram sockets (udp and udplite)
is simple -- just fetch the state from diag module and
record it in the image, then upon socket creation restore
this state.
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

To restore fsnotify's watches on files we need to find paths for each
of them using handle we have in /proc/<pid>/fdinfo/<fsnotifyfd>.
These handle is valid to open the file with open_by_handle_at if
you have mount fd where the file lays. So we try open_by_handle_at
for all possible mount fds we have.

But we can not do so for 'file' bind-mounts, as the way we open
mount fd opens file instead and can hang on fifos or fail on sockets.
(see check_open_handle->open_handle->open_mount code path, imagine
lookup_mnt_sdev() found 'file' bind-mount, open_mount() failed(hanged)
in __open_mountpoint() and if irmap_lookup() also was not successful the
whole dump fails too)

So if we have file bindmount of fifo file, and we restore some
inotify on other file on other mount with same s_dev, we hang forever
on open.

So just skip non-directory mounts from inotify search we will find
path for them on other mount(e.g. non-bindmount) with same s_dev.

v2: remove isdir hashing, improve commit message
v3: make lookup_mnt_sdev handle only nondir mounts, add comment,
move more expensive notdir_mountpoint check after s_dev
v4: inverse notdir_mountpoint to be mnt_is_dir, now on error
in mnt_is_dir mount is also skipped
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

External mounts are provided by an user and CRIU doesn't
need to restore their content.

https://github.com/xemul/criu/issues/299

Reported-by: Stéphane Graber
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>