shmem: use mincore page status bit with anon shared mem dumping

It turned out that anon shmem can have pages with non zero content
and with both PME_PRESENT and PME_SWAP bits unset in all its vmas
in the whole ps tree.
Such case is reproduced in issue #209:
1. Dump ps tree with anon shmem filled using datagen.
2. Restore ps tree. anon shmem content is restored
   in open_shmem(). fd is created for it and it is
   unmapped from restorer process.
3. anon shmem vma is mapped in restore_mapping() of pie restorer context.
   anon shmem content is already initialized to non zero content
   but restored process doesn't touch its newly mapped vma.
4. Run CRIU dump again. All the pages of anon shmem vmas have
   PME_PRESENT and PME_SWAP bits unset and we don't put
   vma pages to dump.

So if we filter anon shmem pages using PME_PRESENT and PME_SWAP bits
the same way as we do it for anon private mem then we have a bug.
PME_PRESENT and PME_SWAP bits work for anon private mem because
at least one process would restore content of private anon vma
in its own address space thus PME bits will be set and pages
will be damped.

We can't just stop using PME_PRESENT and PME_SWAP bits and dump all
non soft dirty and non zero pfn pages. In this case each 1Gb of
mapped and not used anon shmem vma will go to dump. This is too bad.

To fix the bug in this patch we use mincore bits to finally
understand should we dump page or not. mincore bits show page
usage status better because mincore performs deeper checking of
internal in-kernel state. PME bits filling is based only on
process page table.

Using mincore has a drawback. It doesn't work when page is in swap.
But it's ok for now because mincore was used before we started using
PME bits. Also mincore doesn't break page changes tracking
functionality for anon shmem that we have now.

This bug can be fixed in another way. For example we can make anon shmem
restoration work similar to anon private mem restoration.
But this fix looks much harder to implement.
Signed-off-by: Eugene Batalov <eabatalov89@gmail.com>
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>

criu/shmem.c

@@ -605,6 +605,7 @@ static int dump_one_shmem(struct shmem_info *si)
 	struct page_pipe *pp;
 	struct page_xfer xfer;
 	int err, ret = -1, fd;
+	unsigned char *mc_map = NULL;
 	void *addr = NULL;
 	unsigned long pfn, nrpages;
 
@@ -623,6 +624,14 @@ static int dump_one_shmem(struct shmem_info *si)
 	}
 
 	nrpages = (si->size + PAGE_SIZE - 1) / PAGE_SIZE;
+	mc_map = xmalloc(nrpages * sizeof(*mc_map));
+	if (!mc_map)
+		goto err_unmap;
+	/* We can't rely only on PME bits for anon shmem */
+	err = mincore(addr, si->size, mc_map);
+	if (err)
+		goto err_unmap;
+
 	pp = create_page_pipe((nrpages + 1) / 2, NULL, PP_CHUNK_MODE);
 	if (!pp)
 		goto err_unmap;
@@ -636,7 +645,7 @@ static int dump_one_shmem(struct shmem_info *si)
 		unsigned long pgaddr;
 
 		pgstate = get_pstate(si->pstate_map, pfn);
-		if (pgstate == PST_DONT_DUMP)
+		if ((pgstate == PST_DONT_DUMP) && !(mc_map[pfn] & PAGE_RSS))
 			continue;
 
 		pgaddr = (unsigned long)addr + pfn * PAGE_SIZE;
@@ -665,6 +674,7 @@ err_pp:
 err_unmap:
 	munmap(addr,  si->size);
 err:
+	xfree(mc_map);
 	return ret;
 }