For dumping threads we execute parasite code before we collect the
floating point registers with get_task_regs().

The following describes the code path were this is done:

 dump_task_threads()
     for (i = 0; i < item->nr_threads; i++)
         dump_task_thread()
           parasite_dump_thread_seized()
             compel_prepare_thread()
                prepare_thread()

                   ### Get general purpose registers ###
                   ptrace_get_regs(ctx->regs)

             ### parasite code is executed in thread ###
             compel_run_in_thread(tctl, PARASITE_CMD_DUMP_THREAD)

             compel_get_thread_regs()

                 ### Get FP and VX registers ###
                 get_task_regs()

Since on s390 gcc uses floating point registers to cache general
purpose without the -msoft-float option the floating point
registers would have been clobbered after compel_run_in_thread().

With this patch we first save all thread registers with task_get_regs() and
then call the parasite code.

We introduce a new function arch_set_thread_regs() that restores the saved
registers for all threads. This is necessary for criu dump --leave-running,
--check-only, and error handling.

Pre-dump does not require to save the registers for the threads because
because only the main thread (task) is used for parsite code execution.

The above changes allow us to use all register sets in the parasite
code - therefore we can remove -msoft-float on s390.
Reviewed-by: Michael Holzheu <holzheu@linux.vnet.ibm.com>
Signed-off-by: Alice Frosi <alice@linux.vnet.ibm.com>
Reviewed-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>