- 02 Apr, 2017 17 commits
-
-
Kir Kolyshkin authored
As we compile-test non-x86_64 architectures under qemu emulation, it works pretty slow. Dmitry Safonov suggested, and Andrey Vagin initially implemented supporting ccache for such builds. This patch is based heavily on Andrey's work -- all the bugs added are purely mine though. Performance results: in an ideal environment (two builds of the same code, one with cold (empty) ccache, another with the hot one) I saw compile time improvements of 4x to 5x, and total test run time improvement up to 2x to 2.5x. In layman terms, the complete test run that was taking more than 50 minutes now takes about 25! Notes on handling .ccache directory: 1. Before running docker build, .ccache directory (saved in between runs by Travis) is moved to criu source code root, from where it is copied by docker together with criu sources. 2. In Dockerfile, .ccache gets moved to /tmp, and CCACHE_DIR is set accordingly. 3. After running docker build, .ccache is copied out from docker container back to the host (from where it is saved by Travis). Ccache envorinment notes: 1. CCACHE_NOCOMPRESS is needed because otherwise tons of time is spent on compression/decompression (big performance hit under qemu). 2. CCACHE_CPP2 is required with clang, see detailed explanation at http://petereisentraut.blogspot.com/2011/09/ccache-and-clang-part-2.html The logic of setting CCACHE_CPP2 in Dockerfile is somewhat fancy; unfortunately I was not able to come up with a simpler approach. Misc: 1. Travis runs "ccache -s" after the build is completed. A call to "ccache -s" is called to Dockerfile before make, so one can compare before/after numbers. 2. make invocations are surrounded by "date" calls so one can get the compilation times. Changes in v2: - consolidate Dockerfile statements (ENV, RUN) - single object make test is no longer commented out - simplify "make mrproper" test Signed-off-by:
Kir Kolyshkin <kir@openvz.org> Reviewed-by:
Dmitry Safonov <dsafonov@virtuozzo.com> Signed-off-by:
Andrei Vagin <avagin@virtuozzo.com>
-
Kir Kolyshkin authored
For compatibility with ccache (so it can cache the compilation results), let's compile and link separately. For this, we have to - disable the implicit make rules - write the explicit ones While at it, do use the "silent make" stuff that is already here FIXME: figure out if it helps to speed up ccache build Signed-off-by:
-
Kir Kolyshkin authored
For now, it's done for local builds only (i.e. no per-arch Docker builds yet). The reason is, it's easier to play with ccache when the compiles are (relatively) fast. Performance: there is 2x to 3x improvement in build speeds for ideal cases (same code, hot cache), but the absolute savings are negligible (example: 7 seconds instead of 23). Note that ccache is not compatible with gcov, so we only enable it for non-gcov build (which happens to be the one with clang). Signed-off-by:
-
Kir Kolyshkin authored
Unify alpine and non-alpine builds. The only difference is foreign arch builds need some preparation -- separate that to a dependency. Unfortunately we can't use wildcard targets ("%: ") as non-wildcard ones are prevaling. Therefore, a somewhat ugly hack to generate $arch: Dockerfile.$arch dependency is added. While at it: - rename DB_ARGS to DB_CC - mark clean as phony - make "all" really run all the builds we can Nice side effect: autocompletion ("make -C scripts/build <TAB>") now works! Signed-off-by: -
Kir Kolyshkin authored
As pointed out by Andrey, arch clang tests are using gcc regardless of CLANG=1 set in travis environment. Frankly, I do not understand how it worked before (while being pretty sure it worked!), but here is a way to fix it. Reported-by:
-
Kir Kolyshkin authored
This is generated by clang-3.8: > compel/arch/ppc64/src/lib/infect.c:31:20: error: unused function > '__check_code_syscall' [-Werror,-Wunused-function] > static inline void __check_code_syscall(void) > ^ The fix is the same as in commit 3ea2fd78. Signed-off-by:
-
Kir Kolyshkin authored
clang-3.8 complains: > criu/arch/arm/bitops.S:5:2: error: invalid instruction > strneb r1, [ip] @ assert word-aligned > ^ Apparently (see [1]) this is some old asm syntax, which, I guess, was deliberately dropped from clang. [1] https://sourceware.org/ml/libc-ports/2013-03/msg00095.html Cc: Kirill Tkhai <ktkhai@virtuozzo.com> Signed-off-by:
Kirill Tkhai <ktkhai@virtuozzo.com> Signed-off-by:
-
Adrian Reber authored
If using the '--keep-going' option, zdtm prints out an overview how many tests were run, failed and skipped. This would also be useful to know if it did not fail. This patch changes the output like this: ################## ALL TEST(S) PASSED (TOTAL 297/SKIPPED 36) ################### or in the case of a failure it is unchanged: ################### 2 TEST(S) FAILED (TOTAL 297/SKIPPED 34) #################### * zdtm/static/sched_policy00(ns) * zdtm/static/cgroup02(h) ##################################### FAIL ##################################### Signed-off-by:
Adrian Reber <areber@redhat.com> Signed-off-by:
-
Adrian Reber authored
decode_flav() was doing 'if i in flavors:' where 'i' was an integer but the keys from the flavors dict are strings 'h', 'ns' and 'uns'. Signed-off-by:
-
Cyrill Gorcunov authored
When infecting victim we construct sigframe to be able to self-rectore it in case if something goes wrong. But in case is a targer been using alternative stack for signal handling it will be missed in sigframe since we don't fetch it. Thus add fetching sas on infection stage and put it into signal frame early. Signed-off-by:
Cyrill Gorcunov <gorcunov@openvz.org> Signed-off-by:
-
Dmitry Safonov authored
We need to leave this part of restorer in restored task in case of inserted redirected calls. Jump trampolines from old vdso lead here - it would be painful to land nowhere. Signed-off-by:
-
Dmitry Safonov authored
Reworked this code a little and it becomes more readable. Drop those macroses under CONFIG_X86_64 define and just use boolean `compat_vdso' to check whether insert 64 or 32-bit jmp. Signed-off-by:
-
Dmitry Safonov authored
It's the same on all archs (as it's called from generic code), no need to redeclare it. <parasite-vdso.h> is included in all per-arch headers. Drop vdso_symtable forward declaration as <parasite-vdso.h> includes <util-vdso.h> header which defines the structure. Signed-off-by:
-
Cyrill Gorcunov authored
When checkpointing applications with really big memory slab (like in our vz7 test with 920G of memory) the int type get cutted, we should use long int instead, just like we do in other code pieces. Otherwise get (on vz7's criu, which s sharing the code) | pie: 756: Daemon waits for command | (01.193097) Wait for ack 12 on daemon socket | (01.193112) Fetched ack: 12 12 0 | (01.193164) 988065 fdinfo 0: pos: 0 flags: 100002/0 | (01.193201) fdinfo: type: 0xb flags: 0100002/0 pos: 0 fd: 0 | (01.193279) 988065 fdinfo 1: pos: 0 flags: 100002/0 | (01.193307) fdinfo: type: 0xb flags: 0100002/0 pos: 0 fd: 1 | (01.193341) 988065 fdinfo 2: pos: 0 flags: 100002/0 | (01.193365) fdinfo: type: 0xb flags: 0100002/0 pos: 0 fd: 2 | (01.193375) ---------------------------------------- | (01.193405) Error (criu/parasite-syscall.c:243): BUG at criu/parasite-syscall.c:243 | pie: 756: Error (criu/pie/parasite.c:676): Trimmed message received (1> Signed-off-by:
-
Cyrill Gorcunov authored
Signed-off-by:
-
Cyrill Gorcunov authored
Supporting shutdown for dgram sockets (udp and udplite) is simple -- just fetch the state from diag module and record it in the image, then upon socket creation restore this state. Signed-off-by:
-
Pavel Tikhomirov authored
To restore fsnotify's watches on files we need to find paths for each of them using handle we have in /proc/<pid>/fdinfo/<fsnotifyfd>. These handle is valid to open the file with open_by_handle_at if you have mount fd where the file lays. So we try open_by_handle_at for all possible mount fds we have. But we can not do so for 'file' bind-mounts, as the way we open mount fd opens file instead and can hang on fifos or fail on sockets. (see check_open_handle->open_handle->open_mount code path, imagine lookup_mnt_sdev() found 'file' bind-mount, open_mount() failed(hanged) in __open_mountpoint() and if irmap_lookup() also was not successful the whole dump fails too) So if we have file bindmount of fifo file, and we restore some inotify on other file on other mount with same s_dev, we hang forever on open. So just skip non-directory mounts from inotify search we will find path for them on other mount(e.g. non-bindmount) with same s_dev. v2: remove isdir hashing, improve commit message v3: make lookup_mnt_sdev handle only nondir mounts, add comment, move more expensive notdir_mountpoint check after s_dev v4: inverse notdir_mountpoint to be mnt_is_dir, now on error in mnt_is_dir mount is also skipped Signed-off-by:
Pavel Tikhomirov <ptikhomirov@virtuozzo.com> Signed-off-by:
-
- 27 Mar, 2017 2 commits
-
-
Andrei Vagin authored
External mounts are provided by an user and CRIU doesn't need to restore their content. https://github.com/xemul/criu/issues/299 Reported-by: Stéphane Graber Signed-off-by:
-
Andrei Vagin authored
Currently we collect mounts to clean up a mount namespace, but it isn't required when we are going to call pivot_root. https://github.com/docker/docker/issues/31663Signed-off-by:
-
- 15 Mar, 2017 21 commits
-
-
Pavel Emelyanov authored
An example, that steals stderr descriptor from victim task. Signed-off-by:
Pavel Emelyanov <xemul@virtuozzo.com> Signed-off-by:
-
Pavel Emelyanov authored
The same for libcompel.so user. Signed-off-by:
-
Pavel Emelyanov authored
This is just export by reasonable name of the existing code, that sends and receives FDs via compel RPC socket. v2: Rebase on recent criu-dev Fix parallel build Signed-off-by:
-
Dmitry Safonov authored
CID 73371 (#1 of 1): Big parameter passed by value (PASS_BY_VALUE) pass_by_value: Passing parameter regs of type user_regs_struct_t (size 224 bytes) by value. Suggesting to do this until compel is released and API is cut in stone. Signed-off-by:
-
Dmitry Safonov authored
Found by Coverity error: > CID 172193 (#1 of 1): Bad bit shift operation (BAD_SHIFT) > 1. large_shift: In expression 1 << sig % 64, left shifting > by more than 31 bits has undefined behavior. The shift amount, > sig % 64, is as much as 63. That is: 1. yes, UB 2. while adding a signal to mask, this has flushed all other signals from mask. Signed-off-by:
-
Dmitry Safonov authored
The purpose is to unblock previously blocked SIGCHLD, not all the signals. IIRC, this block-unblock dancing is about a race where SIGCHLD may come during switching to restorer blob, where CRIU handler function is already unmapped - which will result in segmentation violation if SIGCHLD wasn't blocked. Fixes:3a61c38a ("restore: block sigchld to remap task_entries") Cc: Andrei Vagin <avagin@virtuozzo.com> Signed-off-by:
-
Kir Kolyshkin authored
The statement like #include <compel/plugins/plugin-std.h> looks a bit tautological. I think the single "plugins" word is enough: #include <compel/plugins/std.h> Reviewed-by:
-
Kir Kolyshkin authored
First, for building compel plugins, we already have "-I compel/include/uapi" in ccflags and asflags, so there is no need to add "-iquote include/uapi". Second, let's refer to compel plugin uapi includes in a uniform way, choosing the same way the external code does, i.e. #include <compel/...>. Third, in a few cases simplify #include statements by including compel/plugins/plugin-std.h instead of a number of plugins/std/*.h files. Reviewed-by:
-
Kir Kolyshkin authored
This header should have everything we have in std/ Reviewed-by:
-
Kir Kolyshkin authored
* install 'cp' can copy recursively, create directories, and even dereference symlinks. Everything we have in uapi/ is to be installed. NOTE we can't use -r for compel includes, as there is some extra stuff in there we don't want to take with us (in particular, plugins/ and 'compel -> .' symlinks). * uninstall rm -rf everything under compel includedir While at it, fix some minor things here and there. Reviewed-by:
-
Kir Kolyshkin authored
It is not used anywhere, so unless someone has any plans, let's kill it. Reviewed-by:
-
Kir Kolyshkin authored
A slash after a directory was missing in a number of places, that resulted in "make uninstall" leaving a number of files behind. Signed-off-by:
-
Kir Kolyshkin authored
GNU make tries to rebuild any makefiles it uses. While in general it's a good idea (and it is used e.g. in autoconf-based builds), in our case it is not necessary, as all the makefiles are static. More to say, as we have a few "match anything" rules for subdirectories, Makefiles in these subdirs are also matching these rules, which leads to excessive (re)building while a particular makefile is needed. Protect such Makefiles with explicit (or pattern) rules, so make knows it should do nothing to rebuild those. Signed-off-by:
-
Kir Kolyshkin authored
Apparently @true was used as a rule for any target which should have a non-empty rule doing nothing. For this, there's a "<target>: ;" syntax, let's use this and eliminate an unnecessary fork/exec :) Signed-off-by:
-
Kir Kolyshkin authored
Mostly this is done in order to not repeat the recursive clean lines twice. Signed-off-by:
-
Kir Kolyshkin authored
Signed-off-by:
-
Kir Kolyshkin authored
As all builds are done from top source dir, there is no need to have SRC_DIR. Reviewed-by:
-
Kir Kolyshkin authored
Our whole system of Makefiles are integrated from top to bottom, meaning: 1. The paths in sub-makefiles are relative to the top source dir. 2. Sub-makefiles are executed via make $(build)=<SUBDIR> <TARGET> For some reason, makefiles under lib/ are the exclusion. Let's fix it. Side effect: you can now build any individual target under lib/, for example, "make lib/c/libcriu.so" works. [v2: use the .FORCE, thanks to dsafonov@] Signed-off-by:
-
Kir Kolyshkin authored
As compel is an external tool now, let's treat it as such -- remove the hardcoded paths to compel files, they are now provided via compel CLI. Details: 1. The compel std plugin dependency is dropped from the intermediate criu/Makefile. It was there in order to make sure plugin is built before pie, but the top level Makefile and Makefile.compel take care about it. 2. The compel std plugin logic is simplified in criu/pie/Makefile. 3. The compel linker script dependency is kept as-is for now (except the linker script file name no longer appears in the list of objects to link). 4. Linking of compel std plugin is moved to a second linking stage in order to simplify the Makefile. Side note: we can actually avoid running the linker twice! 5. The compel binary dependency is dropped from criu/pie/Makefile as this is already there in top-level Makefile (look for criu-deps). [v2: reworked after dsafonov@ comments; keep most compel deps] Signed-off-by:
-
Dmitry Safonov authored
To check that jump trampolines to rt-vdso works. Acked-by:
-
Dmitry Safonov authored
Acked-by:
-
