Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

In case of success, we want to be silent when on default log level.

This is a time-honored UNIX tradition, who we are to break it?
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

Surprisingly, but

offsetof(inotify_wd_entry, node) == offsetof(fanotify_mark_entry, node)

even though both structures have node field in the middle and the
fields sitting above one are totally different.

So the code works, but ... we're risking :)
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

If the ghost file is too big, it might make sence to try seeking
for holes in it, thus reducing the image size.

We've seen this once for tmpfs files in issue #230.
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

We don't need gf there, but will need more than just mode.
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

For convenient simultaneous run.
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

GCC isn't happy if we use %rbp as register for local variable
with -ggdb3 option.
Which resulted in the following build error for `make DEBUG=1`:
> In file included from criu/arch/x86/crtools.c:10:0:
> criu/arch/x86/include/asm/compat.h: In function ‘do_full_int80’:
> criu/arch/x86/include/asm/compat.h:50:1: error: bp cannot be used in asm here

Fix it by saving/restoring %rbp around 32-bit syscall manually.
Just while at it - add a comment about r8-r11 clobbers.
Reported-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Dmitry Safonov <dsafonov@virtuozzo.com>
Acked-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

There are 2 test cases:
1. Connected socket should be restored in the closed state
2. Listening socket state should not change after restore
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Signed-off-by: Eugene Batalov <eabatalov89@gmail.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

New restore option 'tcp-close' was introduced. It restores all connected
TCP sockets in TCP_CLOSE state. Here we consider tcp sockets in
TCP_ESTABLISHED, TCP_FIN_WAIT2, TCP_FIN_WAIT1, TCP_CLOSE_WAIT,
TCP_LAST_ACK, TCP_CLOSING, TCP_SYN_SENT states as connected sockets.
This is consistent with current CRIU usage of these states. Thus this
option doesn't affect sockets with original states of TCP_LISTEN and
TCP_CLOSE.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Signed-off-by: Eugene Batalov <eabatalov89@gmail.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

Pack repeated actions in one. Also, fixed the "return -1"
from the middle of restore_task_with_children(),
while cleanup is need.

v2: goto err in case of error instead of return -1.
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

This is a hot-fix with a regression fix and an urgent
support for the latest-n-greatest kernel API change.
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>

Currently this is lsm.c's static variable, but since kdat
is now cached (and uncached) this value stays zero (no lsm)
if the cache file gets loaded, which is obviously wrong and
breaks the restore all the time on lsm-enabled hosts.

https://github.com/xemul/criu/issues/323Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

This is to remove the function pointer and have only "type"
variable left.
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

We need to keep the host LSM mode on kerndat (next patches),
at the same time the --lsm-profile option needs to correspond
to it.

So split the option handling into two parts -- first keep it
as is, next -- check for kerndat correspondance.
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

https://github.com/xemul/criu/issues/324Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

"With the recent kernel changes criu should never look outside of start-end
region reported by /proc/maps; and restore doesn't even need to know if a
GROWSDOWN region will actually grow or not, because (iiuc) you do not need
to auto-grow the stack vma during restore, criu re-creates the whole vma
with the same length using MAP_FIXED and it should never write below the
addr returned by mmap(MAP_FIXED)" // Oleg
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

If the guard page is not reported in show_map_vma we should
not ajust vma address neither we should call unmap_guard_pages
in restorer.

https://github.com/xemul/criu/issues/322Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

In vanilla kernel commit 1be7107fbe18eed3e319a6c3e83c78254b693acb
show_map_vma() no longer report PAGE_SIZE. Detect it with
simple test and remember in kdat settings.
Suggested-by: Oleg Nesterov <oleg@redhat.com>
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

This is the no-new-features release :) We have several bugfixes,
memory restore optimization and a little bit more.
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>

The list in question is not used for fles nowadays.
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

This case is legacy, tfds are merged into epoll entry, but
to make it working we have separate list of tfds and extra
code in ->open callback.

Keep the legacy code in one place.
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

Current collect helper frees the pb entry if there's
zero priv_size on cinfo. For files we'll have zero
priv_size (as entries will be collected by sub-cinfos),
while the entry in question should NOT be freed.
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

Rather than do open/close to reset pagemap, just update it's state.
Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
Acked-by: Adrian Reber <areber@redhat.com>
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>

For the upcoming userfaultfd integration the skip_pages functionality is
required to find the userfaultfd requested pages.
Signed-off-by: Adrian Reber <areber@redhat.com>
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>

Previous patch (5a1e1aac) tried to minimize the amount of
open()s called when mmap()ing the files. Unfortunatley, there
was a mistake and wrong flags were compared which resulted in
the whole optimization working randomly (typically not
working).

Fixing the flags comparison revealed another problem. The
patch in question correllated with the 03e8c417 one, which
caused some vmas to be opened and mmaped much later than the
premap. When hitting the situation when vmas sharing their
fds are partially premapped and partially not, the whole
vm_open sharing became broken in multiple places -- either
needed fd was not opened, or the not needed left un-closed.

To fix this the context, that tracks whether the fd should
be shared or not, should be moved from collect stage to
the real opening loop. In this case we need to explicitly
know which vmas _may_ share fds (file private and shared)
with each other, so the sharing knowledge becomes spread
between open_filemap() and its callers. Oh, well...
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

On real apps it's typical to have sequences ov VMAs with
absolutely the same file mapped. We've seen this dump-time
and fixed multiple openings of map_files links with the
file_borrowed flag.

Restore situation is the same -- the vm_open() call in many
cases re-open the same path with the same flags. This slows
things down.

To fix this -- chain VMAs with mapped files to each other
and only the first one opens the file and only the last
one closes it.

✓ travis-ci: success for mem: Do not re-open files for mappings when not required
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>

In this routine we'll need to compare fdflags, so to
avoid double if-s, let's calculate and set fdflags early.

✓ travis-ci: success for mem: Do not re-open files for mappings when not required
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>

When a vma we restore doesn't have any pages in pagemaps there's
not need to enforce PROT_WRITE bit on it.

This only applies to non-premmaped vmas.
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

Performance experiments show, that we spend (relatively) a lot of time
mremap-ing areas from premap area into their proper places. This time
depends on the task being restored, but for those with many vmas this
can be up to 20%.

The thing is that premapping is only needed to restore cow pages since
we don't have any API in the kernel to share a page between two or more
anonymous vmas. For non-cowing areas we map mmap() them directly in
place. But for such cases we'll also need to restore the page's contents
also from the pie code.

Doing the whole page-read code from PIE is way too complex (for now), so
the proposal is to optimize the case when we have a single local pagemap
layer. This is what pr.pieok boolean stands for.

v2:
* Fixed ARM compiling (vma addresses formatting)
* Unused tail of premapped area was left in task after restore
* Preadv-ing pages in restorer context worked on corrupted iovs
  due to mistakes in pointer arithmetics
* AIO mapping skipped at premap wasn't mapped in pie
* Growsdown VMAs should sometimes (when they are "guarded" by
  previous VMA and guard page's contents cannot be restored in
  place) be premmaped
* Always premmap for lazy-pages restore
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

Next patch will stop premapping some private vmas. In particular -- those,
that are not COW-ed with anyone. To make this work we need to distinguish
vmas that are not cowed with anyone from those cowed with children only.
Currently both have vma->parent pointer set to NULL, so for former let's
introduce the special mark.
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

Inherited VMAs don't need the descriptor to work with.
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>