if final size of all pages images is zero than dedup works

use :
bash test/zdtm.sh --auto-dedup
bash test/zdtm.sh --auto-dedup static/maps04
bash test/zdtm.sh -P -i 3 --auto-dedup -t transition/maps007

changes: aplicable for all tests, -ad changed to --auto-dedup,
simplify, check all private pages images, check shmem images
and go with shmem patch set.
Acked-by: Andrew Vagin <avagin@parallels.com>
Signed-off-by: Tikhomirov Pavel <snorcht@gmail.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

in restore_shmem_content() use open_page_read() to open images
Signed-off-by: Tikhomirov Pavel <snorcht@gmail.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

Signed-off-by: Tikhomirov Pavel <snorcht@gmail.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

During the time some files become obsolete and might be missing
in checkpoint image set, but to keep backward compatibility we
still trying to open them, which might print out error like

 | Unable to open 'path-to-file'

and confuse a reader why criu prints error but continue working.

To eliminate this problem O_OPT flag has been introduced in
commit 16b56920, which suppress error message priting
if the flag is set.

Now start using O_OPT in the following functions

 - open_irmap_cache: irmap cache is relatively new optional feature

 - prepare_rlimits, open_signal_image, restore_file_locks,
   prepare_fd_pid, prepare_mm_pid, collect_image: all these
   helpers are trying to open image files which can be missing.
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

On PI machine we've got

 |   CC	   protobuf.o
 | pstree.c: In function ‘core_entry_alloc’:
 | pstree.c:36:10: error: ‘RLIM_NLIMITS’ undeclared (first use in this function)

due to old kernel headers. Note I've dropped off
BUG_ON here to localize all things in pstree code,
no need to sprinkle constants.
Reported-by: Pavel Emelyanov <xemul@parallels.com>
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

No all distros (Rpi) provide O_PATH definition,
so include fcntl.h here thus we don't hit compilation
problem like

 |  CC       image.o
 | image.c: In function ‘open_image_at’:
 | image.c:187:29: error: ‘O_PATH’ undeclared (first use in this function)
Reported-by: Pavel Emelyanov <xemul@parallels.com>
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

The array element is RlimitEntry properly initialized,
no need in additional memcpy-s and size-checks.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

On Thu, Mar 13, 2014 at 02:30:50PM +0400, Cyrill Gorcunov wrote:
>
> This image is deprecated now so move it out of
> _CR_FD_TASK thus we won't be even generating it
> on the dump.
>
> Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>

Updated

>From cb9c3953beac7d42de80635e7a6e537cc867c479 Mon Sep 17 00:00:00 2001
From: Cyrill Gorcunov <gorcunov@openvz.org>
Date: Thu, 13 Mar 2014 14:24:50 +0400
Subject: [PATCH 7/7] rlimit: Move CR_FD_RLIMIT out of _CR_FD_TASK

This image is deprecated now so move it out of
_CR_FD_TASK thus we won't be even generating it
on the dump.
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

We're using new image format, but old image file
is still generated. This will be addressed in
next patch.
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

To save backward compatibility try to read
data from old image if Core entry doesn't
has rlimits bound.
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

Note the restore remains as is for a while, it'll
be addressed later.
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

On Thu, Mar 13, 2014 at 04:20:37PM +0400, Pavel Emelyanov wrote:
>
> Would you rework this patch on top of my recent
> "allocate Core in on xmalloc call" one?

Attached.

>From c2233d4fafce30c4e7214a1a7ab3677824a30d75 Mon Sep 17 00:00:00 2001
From: Cyrill Gorcunov <gorcunov@openvz.org>
Date: Thu, 13 Mar 2014 16:57:14 +0400
Subject: [PATCH] rlimit: Allocate and free appropriate Core entry
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

We will carry rlimits inside Core itself instead of separate image.
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

This allows us to distinguish the situation where image
to be opened is missing but optional, thus no error message
should be printed.
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

arch/x86/crtools.c: In function ‘arch_alloc_thread_info’:
arch/x86/crtools.c:271:6: error: ‘with_xsave’ may be used uninitialized in this function [-Werror=uninitialized]

Actually the with_xsave is with_fpu dependant, but some gccs
can't guess that fact :\
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Acked-by: Cyrill Gorcunov <gorcunov@openvz.org>

For example opening the FIFO blocks until the other end is opened also.

We can use O_PATH to avoid sleeping in the open() call.
Signed-off-by: Andrey Vagin <avagin@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

It's going to be used for restoring namespaces. For example we need to
enumirate the ns_ids list for restoring mount namespaces.
Signed-off-by: Andrey Vagin <avagin@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

The implementation of bit operations for ARM isn't actually
architecture-specific so it would rather be shared with
the upcoming port for AArch64 that won't provide optimized
implementation of bit operations.
Signed-off-by: Alexander Kartashov <alekskartashov@parallels.com>
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

Documentation: remove the linker command line switch --static from the final step described in the cross-compilation guide

Static linking of the binary criu isn't supported any more.
Reported-by: riya khanna <riyakhanna1983@gmail.com>
Signed-off-by: Alexander Kartashov <alekskartashov@parallels.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

Signed-off-by: Alexander Kartashov <alekskartashov@parallels.com
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

This patch should simplify the routine interface change
in the following patch.
Signed-off-by: Alexander Kartashov <alekskartashov@parallels.com>
Reviewed-by: Christopher Covington <cov@codeaurora.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

pre_dump_one_fanotify calls for parse_fdinfo_pid_s where
fsn_params mut not be NULL, otherwise we get nil dereference.

Fix it by passing a real variable instead.
Reported-by: Pavel Tikhomirov <snorcht@gmail.com>
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

Here is a regresion test cae for

commit 1934e1a9
Author: Andrey Vagin <avagin@openvz.org>
Date:   Tue Feb 18 16:53:00 2014 +0400

    posix-timer: take into account that sival_ptr can be NULL (v3)
Signed-off-by: Andrey Vagin <avagin@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

The pie.lds.S.in needs two minor changes to work with the gold (/usr/bin/gold)
linker. These changes are compatible with /usr/bin/ld and make linker script
more portable.

The first change is adding a comma before /DISCARD/ so that the grammar
won't be ambiguous. A Otherwise, gold treats it as a part of the assignment
and would generate a syntax error about the "unexpected ':'".

The second change is moving initialization of __export_parasite_args
to inside the SECTIONS command because it references the dot symbol.
Otherwise, gold would generate the error "invalid reference to dot
symbol outside of SECTIONS clause.
Signed-off-by: Saied Kazemi <saied@google.com>
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

Fix for the previous patch :\
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

On this fs path can be resolved via proc, so even if
we're asked to do force-irmap, try to go via regular
resolve anyway.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

when decide that data is no longer needed, there are two cases:
-if data neighbours previous block of "no needed" data, extend bunch
block(it holds begining and size of concequent "no needed" data) by
length of curent block and go next.
-if data not neighbours bunch block(or bunch block size will be bigger
than MAX_BUNCH_SIZE), than we punch bunch block and set bunch block
to curent block.

in the end make cleanup to punch last bunch block.

changes in v1:
punch_hole takes whole page_read
make restriction more precise
Signed-off-by: Tikhomirov Pavel <snorcht@gmail.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

Otherwise we migh propagate previous vfi status
to vmas which actually don't match.

 | (00.005471) 0x2b79227d6000-0x2b79227d8000 (8K) prot 0x5 flags 0x22 off 0 reg vdso ap  shmid: 0
 | (00.005473) 0x2b79227d8000-0x2b79227da000 (8K) prot 0x3 flags 0x22 off 0 reg vdso ap  shmid: 0
 | (00.005475) 0x2b79227f1000-0x2b79227f2000 (4K) prot 0x3 flags 0x22 off 0 reg vdso ap  shmid: 0
 | (00.005476) 0x2b79227f2000-0x2b79227f4000 (8K) prot 0x3 flags 0x22 off 0 reg vdso ap  shmid: 0
Tested-by: Pavel Tikhomirov <snorcht@gmail.com>
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

No need to check opts.use_page_server, because in write_pagemap_loc
write is local anyway. it appeared that on page-server when saving
pages to images opts.use_page_server=true and no auto-dedup started,
so images were not null as expected.
Signed-off-by: Tikhomirov Pavel <snorcht@gmail.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

The s_dev value we see in irmap is get from proc file and is
raw kernel dev_t. We compare this value to the device get from
stat syscall, which is old-style dev_t (with less bits for minor).

Thus, need to convert kernel dev_t to stat's dev_t for proper
comparison.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

When migrating container with copying its FS, the inode numbers
and thus their handles wil change. This will make the restore of
inotify/fanotify fail, since they do it via fhandles.

We've already faced the problems with fsnotifies on NFS -- they
don't work there. To address this an irmap cache is created on
pre-dump, so to resolve the issue with changed inodes during
migration, we can force the irmap cache build.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>

There's a standard way of doing this -- the MSG_WAITALL flag that
works even for TCP sockets.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>