The solution for enabling of ptrace and PTRACE_ATTACH in Docker Containers


For the last two years we have been using Docker containerization solutions for the needs of development, CI and production environments. All current products for the customers are being developed to be run in Docker.

We have a very positive experience about that solution. However, Docker changes a lot, from version to version it introduces new amazing stuff like Compose, Swarm, additional security features, which, sometimes, lead to errors in containers that was considered as stable.

Thus, with great amazement we have discovered that a container which was developed and tested 3 months ago is no longer functional in a new Docker version. The container is not a typical thing because it uses strace to analyze a process behaviour to diagnose deviant behaviour of it.

Today our developers started an integration process of that container in an application and found that it doesn’t work anymore and the reason is connected to ptrace which is reported by strace like:

strace: attach: ptrace(PTRACE_ATTACH, ...): Operation not permitted Could not attach to process.  If your uid matches the uid of the target 
process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try again as the root user.  For more details, see /etc/sysctl.d/10-ptrace.conf

You probably wonder, the case is quite rare — strace in Docker… Er, but there are some similar questions in the internet which are more or less address the same problem. So, what’s the reason and how to tackle with it? There are several things.

Seccomp Mechanism

In distros with new Linux operating system kernel there is modern kernel mechanism — seccomp, which enables Docker blocking of system calls inside a container. The Docker documentation about this feature is located at official Docker documentation page. As one can discover there, ptrace is blocked. Thin seccomp configuration can be implemented by overriding the default configuration file by using docker run command line argument –security-opt seccomp=/path/to/file.json. Our container is launched in a protected environment, so we disabled it completely by setting --security-opt seccomp=unconfined.

Yama Mechanism

Strace shown us an clue about other possible ptrace limitation — procfs pseudofile /proc/sys/kernel/yama/ptrace_scope, which introduces additional ptrace limitation capabilities. Let’s take a look at the kernel docs and discover that the file content is presented by digits 0-3 which in case define ptrace capabilities:

The sysctl settings (writable only with CAP_SYS_PTRACE) are:

0 - classic ptrace permissions: a process can PTRACE_ATTACH to any other
    process running under the same uid, as long as it is dumpable (i.e.
    did not transition uids, start privileged, or have called
    prctl(PR_SET_DUMPABLE...) already). Similarly, PTRACE_TRACEME is

1 - restricted ptrace: a process must have a predefined relationship
    with the inferior it wants to call PTRACE_ATTACH on. By default,
    this relationship is that of only its descendants when the above
    classic criteria is also met. To change the relationship, an
    inferior can call prctl(PR_SET_PTRACER, debugger, ...) to declare
    an allowed debugger PID to call PTRACE_ATTACH on the inferior.
    Using PTRACE_TRACEME is unchanged.

2 - admin-only attach: only processes with CAP_SYS_PTRACE may use ptrace
    with PTRACE_ATTACH, or through children calling PTRACE_TRACEME.

3 - no attach: no processes may use ptrace with PTRACE_ATTACH nor via
    PTRACE_TRACEME. Once set, this sysctl value cannot be changed.

By executing following command:

docker exec -it <container> cat /proc/sys/kernel/yama/ptrace_scope

We found that in our case file has value of “1” set, and since our processes were executed with the same UID but not in the parent-child hierarchy, Yama blocked PTRACE_ATTACH.

We tried to put “0” value into /proc/sys/kernel/yama/ptrace_scope but failed because we got “/proc is read only” message.

Privileged Execution Mode

We overcame that “read-only procfs” limitation by running our container in “privileged” execution mode setting “–privileged” docker run flag. We also put “0” to /proc/sys/kernel/yama/ptrace_scope by modification of application bootstrap code.

echo 0 > /proc/sys/kernel/yama/ptrace_scope

All above means allowed us to solve the problem. The example solution can be found in our web ssh proxy container.