All of lore.kernel.org
 help / color / mirror / Atom feed
From: Bandan Das <bsd@redhat.com>
To: tj@kernel.org
Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
	mst@redhat.com, jiangshanlai@gmail.com, RAPOPORT@il.ibm.com
Subject: [RFC PATCH 0/4] cgroup aware workqueues
Date: Fri, 18 Mar 2016 18:14:47 -0400	[thread overview]
Message-ID: <1458339291-4093-1-git-send-email-bsd@redhat.com> (raw)

At Linuxcon last year, based on our presentation "vhost: sharing is better" [1],
we had briefly discussed the idea of cgroup aware workqueues with Tejun. The
following patches are a result of the discussion. They are in no way complete in
that the changes are for unbounded workqueues only, but I just wanted to present my
unfinished work as RFC and get some feedback.

1/4 and 3/4 are simple cgroup changes and add a helper function.
2/4 is the main implementation.
4/4 changes vhost to use workqueues with support for cgroups.

Accounting:
When servicing a userspace task A attached to cgroup X, for cgroup
awareness, a worker thread could attach to all cgroups
of the task which it is servicing. This patch does it for unbound
workqueues which means all tasks that are bound to certain cgroups
could potentially be serviced by the same worker thread. However,
the same technique could be applicable to bounded workqueues as
well.

Example:
vhost creates a worker thread when invoked for a kvm guest. Since,
the guest is a normal process, the kernel thread servicing it should be
attached to the vm process' cgroups.

Design:

The fundamental addition is a cgroup aware worker pool and as stated above,
for the unbounded case only.

These changes don't populate the "numa awareness" fields/attrs and
unlike unbounded numa worker pools, cgroup worker pools are created
on demand. Every work request could potentially have a new cgroup
aware pool created for it based on the combination of cgroups it's attached
to. However, workqueues themselves are incognizant of the actual cgroups -
they rely on the cgroups provided helper functions either for 1. a match
of all the cgroups or 2. to attach a worker thread to all cgroups of
a userspace task. We do maintain a list of cgroup aware pools so that
when a new request comes in and a suitable worker pool needs to be
found, we search the list first before creating a new one. A worker
pool also stores a a list of all "task owners" - a list of processes
that we are serving currently.

Testing:
Create some qemu processes and attaching them to different
cgroups. Verifying that new worker pools are created for tasks
that are attached to different cgroups (and reuse for the ones
that belong to the same).

Some simple performace testing using netperf below. Although, these
numbers shouldn't be dependent on these patches. The cgroup attach
and match functions are not in hot paths for general usage which
is what this test does.

Netperf:
Two guests running netperf in parallel.
    	   	       	          Without patches                  With patches
   
TCP_STREAM (10^6 bits/second)         975.45			     978.88	
TCP_RR (Trans/second)		      20121			     18820.82
UDP_STREAM (10^6 bits/second)	      1287.82		  	     1184.5
UDP_RR (Trans/second)		      20766.72			     19667.08
Time a 4G iso download		      2m 33 seconds		     3m 02 seconds

Todo:
What about bounded workqueues ?
What happens when cgroups of a running process changes ?
sysfs variables
Sanity check the flush and destroy path.
More extensive testing
Can we optimize the search/match/attach functions ?
Better performance numbers ? (although the onese above don't look bad)

[1] http://events.linuxfoundation.org/sites/events/files/slides/kvm_forum_2015_vhost_sharing_is_better.pdf

Bandan Das (4):
  cgroup: Introduce a function to compare two tasks
  workqueue: introduce support for attaching to cgroups
  cgroup: use spin_lock_irq for cgroup match and attach fns
  vhost: use workqueues for the works

 drivers/vhost/vhost.c       | 103 ++++++++++++++++++---
 drivers/vhost/vhost.h       |   2 +
 include/linux/cgroup.h      |   1 +
 include/linux/workqueue.h   |   2 +
 kernel/cgroup.c             |  40 ++++++++-
 kernel/workqueue.c          | 212 +++++++++++++++++++++++++++++++++++++++++---
 kernel/workqueue_internal.h |   4 +
 7 files changed, 335 insertions(+), 29 deletions(-)

-- 
2.5.0

             reply	other threads:[~2016-03-18 22:15 UTC|newest]

Thread overview: 34+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-03-18 22:14 Bandan Das [this message]
2016-03-18 22:14 ` [RFC PATCH 1/4] cgroup: Introduce a function to compare two tasks Bandan Das
2016-03-18 22:14 ` [RFC PATCH 2/4] workqueue: introduce support for attaching to cgroups Bandan Das
2016-03-18 22:14 ` [RFC PATCH 3/4] cgroup: use spin_lock_irq for cgroup match and attach fns Bandan Das
2016-03-18 22:14 ` [RFC PATCH 4/4] vhost: use workqueues for the works Bandan Das
2016-03-20 18:10 ` [RFC PATCH 0/4] cgroup aware workqueues Tejun Heo
2016-03-21 17:35   ` Bandan Das
2016-03-21  7:58 ` Michael Rapoport
2016-03-21  7:58 ` Michael Rapoport
2016-03-21 17:43   ` Bandan Das
2016-03-22  7:12     ` vhost threading model (was: Re: [RFC PATCH 0/4] cgroup aware workqueues) Michael Rapoport
2016-03-22 19:00       ` vhost threading model Bandan Das
2016-03-23 11:13         ` Michael Rapoport
2016-03-23 11:13         ` Michael Rapoport
2016-03-22  7:12     ` vhost threading model (was: Re: [RFC PATCH 0/4] cgroup aware workqueues) Michael Rapoport
2016-03-21  8:29 ` [RFC PATCH 0/4] cgroup aware workqueues Christian Borntraeger
2016-03-21 17:49   ` Bandan Das
     [not found] ` <201603210758.u2L7wiY9003907@d06av07.portsmouth.uk.ibm.com>
2016-03-30 17:04   ` Tejun Heo
2016-03-31  6:17     ` Michael Rapoport
2016-03-31  6:17     ` Michael Rapoport
2016-03-31 17:14       ` Tejun Heo
2016-03-31 18:45         ` Bandan Das
2016-04-03 10:43           ` Michael Rapoport
2016-04-03 10:43           ` Michael Rapoport
     [not found]           ` <201604031043.u33AhpSF023771@d06av06.portsmouth.uk.ibm.com>
2016-04-04 17:00             ` Bandan Das
2016-04-03 10:43         ` Michael Rapoport
2016-04-03 10:43         ` Michael Rapoport
2016-05-27  9:22         ` Michael Rapoport
2016-05-27  9:22         ` Michael Rapoport
2016-05-27 14:17           ` Tejun Heo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1458339291-4093-1-git-send-email-bsd@redhat.com \
    --to=bsd@redhat.com \
    --cc=RAPOPORT@il.ibm.com \
    --cc=jiangshanlai@gmail.com \
    --cc=kvm@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mst@redhat.com \
    --cc=tj@kernel.org \
    --subject='Re: [RFC PATCH 0/4] cgroup aware workqueues' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.