removing empty cgroups and other problems

We had to clean up some left over cgroups after another set of
experiments with LXC.The guys doing it encountered problems, as the
logic is the opposite of what you expect from your experience on a
normal unix filesystem.

Specifically the problem happens when you have a nested cgroup – for example /cgroup/foo/bar/

The problem happens because this is a virtual filesystem with its own rules. 

If you create a cgroup (in this case we’ll just mount the net_cls cgroup because its smaller)

# mount -t cgroup /dev/cgroup /cgroup -o net_cls 
# cd /cgroup
# ls
cgroup.procs net_cls.classid notify_on_release release_agent tasks
# mkdir foo
# ls
cgroup.procs foo net_cls.classid notify_on_release release_agent tasks
# ls foo
cgroup.procs net_cls.classid notify_on_release tasks

There are a set of files created to manipulate the cgroup. If the tasks entry is empty, then the cgroup is unused. 

However the files can’t be removed. 

# rm -f *
rm: cannot remove `cgroup.procs': Operation not permitted
rm: cannot remove `foo': Is a directory
rm: cannot remove `net_cls.classid': Operation not permitted
rm: cannot remove `notify_on_release': Operation not permitted
rm: cannot remove `release_agent': Operation not permitted
rm: cannot remove `tasks': Operation not permitted

So here’s where the confusion arose. To remove a directory hierarchy
on a normal file system you should first remove all files from each
subdirectory and then remove the parent directories. 

This is what “rm -fr  directory” will usually do for you. 

However, as the files cannot be removed, that approach will fail in
this virtual filesystem. “rm -rf” is the wrong approach.  

Instead, you need to remove just the cgroup directories. If you want
to do this recursively here’s the magic incantation (we first make a
sub-cgroup to give it more than one level of depth);

# mkdir -p foo/bar
# ls foo/
bar/ cgroup.procs net_cls.classid notify_on_release tasks
# find foo -depth -type d -print -exec rmdir {} ;
# ls
cgroup.procs net_cls.classid notify_on_release release_agent tasks

 As the files aren’t real, they don’t need to be removed – we
only need to remove the cgroup directories. The “-depth” argument tells
find to do a depth first recursion, so we remove sub cgroups

If that fails, then you’ll most likely have a task occupying one of the cgroups somewhere.

 You can find it like this;

# mkdir -p foo/bar
# echo $$ > foo/bar/tasks
# find foo -depth -type d -print -exec cat {}/tasks ;

And to remove those tasks from the subcgroup add them into the top level. 

# echo $$ > tasks
# find foo -depth -type d -print -exec cat {}/tasks ;

The last gotcha with cgroups relates to mounting subsets of functionality.

 If you have created a cgroup, and then attempt to mount or
remount /dev/cgroup with different cgroup subsystems active, the mount
will fail with a rather unhelpful error message “mount: /dev/cgroup
already mounted or /cgroup busy” 

 The answer is to remove the cgroups you have created, so that
there are no cgroups on the system, and then you can mount with
different subsystems active.  

 As in the following example;

# umount /cgroup
# mount -t cgroup /dev/cgroup /cgroup -o net_cls,cpuacct
mount: /dev/cgroup already mounted or /cgroup busy
# mount -t cgroup /dev/cgroup /cgroup -o net_cls
# find /cgroup/foo -depth -type d -print -exec rmdir {} ;
# umount /cgroup
# mount -t cgroup /dev/cgroup /cgroup -o net_cls,cpuacct
# ls /cgroup
cgroup.procs cpuacct.usage net_cls.classid release_agent
cpuacct.stat cpuacct.usage_percpu notify_on_release tasks

Which is probably why fedora mounts them all separately. Even though that looks messy.

Any opinions, news, research, analyses, prices or other information ("information") contained on this Blog, constitutes marketing communication and it has not been prepared in accordance with legal requirements designed to promote the independence of investment research. Further, the information contained within this Blog does not contain (and should not be construed as containing) investment advice or an investment recommendation, or an offer of, or solicitation for, a transaction in any financial instrument. LMAX Group has not verified the accuracy or basis-in-fact of any claim or statement made by any third parties as comments for every Blog entry.

LMAX Group will not accept liability for any loss or damage, including without limitation to, any loss of profit, which may arise directly or indirectly from use of or reliance on such information. No representation or warranty is given as to the accuracy or completeness of the above information. While the produced information was obtained from sources deemed to be reliable, LMAX Group does not provide any guarantees about the reliability of such sources. Consequently any person acting on it does so entirely at his or her own risk. It is not a place to slander, use unacceptable language or to promote LMAX Group or any other FX and CFD provider and any such postings, excessive or unjust comments and attacks will not be allowed and will be removed from the site immediately.