hello
I will be greatful for any criticism.
In short, i offload a processor and assign it a service.
raz
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Service Oriented Scheduler
MSC Concluding Assignment
Table of Content
1. Document Content ..........................3
2. Background ................................3
3. Purpose....................................4
4. Importance.................................4
5. Description................................5
5.1 Processor Offload Architecture..........5
6. Acknowledgements...........................6
7. Appendix.
7.1 hotplug patch
7.2 SOS frame work
7.2 timer code
7. Bibliography ...............................6
1. Document Content
This document is a proposal of my concluding assignment to the MSC.
2. Background
In today's computer world, we find that most processors have several
embedded cores, virtualization
and hyper-threading. Most programmers do not really use these powerful
features and let the
operating system do the work.
At most, a programmer will bound an application to a certain processor
or assign an interrupt to a
different processor. At the end, we get system busy in maintaining
tasks across processors, balancing
interrupts, replacing TLBs and DTLBs, and worst of all, spin locks
across processors in vein. I argue
that in some cases, part of this behavior is due to fact the multipled
core operating system is not
service oriented but system oriented. There is no easy way to assign a
processor to do a distinct
service, undisturbed, accurate, and fast as long as the processor is
an active part of an operating
system.
3. Purpose
The purpose of this assignment is to create a platform for services.
For example, assume a firewall is
being attacked; the Linux operating system will generate endless
number of interrupts and softirqs to
analyze the traffic and throw out bad packets. This is on the expense
of "good" packets. Have you
ever tried to "ssh" to an attacked machine?
What if we can simply do the packet analysis outside the operating
system, without interrupts and still
fast enough?
Why not assign a core to do only "firewalling"? Or just routing? Maybe
assign it as an ultra accurate
timer? Since a well defined set of schedulers will available for
receiving packets, or serving timers,
the OS will not be bothered.
Technologically speaking, I am referring to the Linux kernel ability
to virtually hot plug a (SMT)
processor but instead of letting it wonder in endless "halts", assign
it a service.
4. Importance
PROS
Money
First, it saves money. Why buy expensive offloading cards when you can
do with commodity
hardware?
Resources
SOS can be assigned to use somewhat esoteric system resources (SMT and
VT) available in
the processor to do some non esoteric tasks.
Accuracy
A programmer may rely on the fact that only his code will be run,
without disturbance
(offloaded core runs in NMI mode).
Security
SOS is designed to assign service to a single resource. This makes the
load be contained
inside SOS. Example for that is a firewall contained inside a single core.
RAS
Remote Access Services: SOS can be used as remote access server (such
as a simple telnet
server) inside the machine but outside the kernel. This can be useful
incases for getting
operating statistics and monitoring an offended system.
Speed-Up
[4] Amdhal Law for liner processors speedup is truly achievable.
CONS
SOS cannot run user space processes.
SOS cannot release memory nor allocate one.
SOS does not perform context switch. SOS has a single context.
SOS does support [1] IPMI, so SOS cannot use vmalloc'd memory. This is
quite a problem since drivers are vmalloc'd.
5. Description
This assignment is practical, not theoretical. I will supply a set of
patches, a driver and configuration
tool. I will write it for the Linux kernel and then I will provide
some benchmarks comparing the current
technology versus the service oriented technology. My hypothesis is
that in some cases, it is best to
use SOS scheduler.
SOS scheduler uses SMP, SMT and if possible over VT.
5.1 Processor Offload Architecture
<<image here>>
6. Acknowledgements
Jeff Roberson, who came with this idea of offloading a whole core, and
showed me that it is possible.
7. Appendix.
7.1 hotplug patch
diff -X dontdiff -urN kernel-centos/arch/i386/kernel/process.c
linux-2.6.18.x86_64-bitband/arch/i386/kernel/process.c
--- kernel-centos/arch/i386/kernel/process.c 2007-10-25
17:28:52.000000000 +0200
+++ linux-2.6.18.x86_64-bitband/arch/i386/kernel/process.c
2007-10-11 10:00:49.000000000 +0200
@@ -138,6 +138,9 @@
}
#ifdef CONFIG_HOTPLUG_CPU
+void (*hotplug_cpu_dead)(void);
+EXPORT_SYMBOL(hotplug_cpu_dead);
+
#include <asm/nmi.h>
/* We don't actually take CPU down, just spin without interrupts. */
static inline void play_dead(void)
@@ -153,6 +156,8 @@
* With physical CPU hotplug, we should halt the cpu
*/
local_irq_disable();
+ if (hotplug_cpu_dead)
+ hotplug_cpu_dead();
while (1)
halt();
}
diff -X dontdiff -urN kernel-centos/arch/i386/kernel/smpboot.c
linux-2.6.18.x86_64-bitband/arch/i386/kernel/smpboot.c
--- kernel-centos/arch/i386/kernel/smpboot.c 2007-10-25
17:28:58.000000000 +0200
+++ linux-2.6.18.x86_64-bitband/arch/i386/kernel/smpboot.c
2007-11-05 15:58:41.000000000 +0200
@@ -1401,8 +1401,6 @@
/* They ack this in play_dead by setting CPU_DEAD */
if (per_cpu(cpu_state, cpu) == CPU_DEAD) {
printk ("CPU %d is now offline\n", cpu);
- if (1 == num_online_cpus())
- alternatives_smp_switch(0);
return;
}
msleep(100);
diff -X dontdiff -urN kernel-centos/arch/i386/kernel/traps.c
linux-2.6.18.x86_64-bitband/arch/i386/kernel/traps.c
--- kernel-centos/arch/i386/kernel/traps.c 2007-10-25
17:29:02.000000000 +0200
+++ linux-2.6.18.x86_64-bitband/arch/i386/kernel/traps.c
2007-11-05 14:45:18.000000000 +0200
@@ -271,6 +271,7 @@
printk(" ");
show_stack_log_lvl(task, NULL, esp, "");
}
+EXPORT_SYMBOL(show_stack);
/*
* The architecture-independent dump_stack generator
diff -X dontdiff -urN kernel-centos/arch/x86_64/kernel/process.c
linux-2.6.18.x86_64-bitband/arch/x86_64/kernel/process.c
--- kernel-centos/arch/x86_64/kernel/process.c 2007-10-25
17:29:01.000000000 +0200
+++ linux-2.6.18.x86_64-bitband/arch/x86_64/kernel/process.c
2007-10-11 10:00:49.000000000 +0200
@@ -172,6 +172,8 @@
EXPORT_SYMBOL_GPL(cpu_idle_wait);
#ifdef CONFIG_HOTPLUG_CPU
+void (*hotplug_cpu_dead)(void);
+EXPORT_SYMBOL(hotplug_cpu_dead);
DECLARE_PER_CPU(int, cpu_state);
#include <asm/nmi.h>
@@ -185,6 +187,8 @@
__get_cpu_var(cpu_state) = CPU_DEAD;
local_irq_disable();
+ if (hotplug_cpu_dead)
+ hotplug_cpu_dead();
while (1)
halt();
}
diff -X dontdiff -urN kernel-centos/arch/x86_64/kernel/setup.c
linux-2.6.18.x86_64-bitband/arch/x86_64/kernel/setup.c
--- kernel-centos/arch/x86_64/kernel/setup.c 2007-10-25
17:29:02.000000000 +0200
+++ linux-2.6.18.x86_64-bitband/arch/x86_64/kernel/setup.c
2007-10-29 14:20:24.000000000 +0200
@@ -522,6 +522,7 @@
void __init setup_arch(char **cmdline_p)
{
printk(KERN_INFO "Command line: %s\n", saved_command_line);
+ printk(KERN_INFO "SOS 1.0 %s\n",__DATE__);
ROOT_DEV = old_decode_dev(ORIG_ROOT_DEV);
screen_info = SCREEN_INFO;
diff -X dontdiff -urN kernel-centos/arch/x86_64/kernel/smpboot.c
linux-2.6.18.x86_64-bitband/arch/x86_64/kernel/smpboot.c
--- kernel-centos/arch/x86_64/kernel/smpboot.c 2007-10-25
17:28:50.000000000 +0200
+++ linux-2.6.18.x86_64-bitband/arch/x86_64/kernel/smpboot.c
2007-11-05 15:58:22.000000000 +0200
@@ -1265,8 +1265,6 @@
/* They ack this in play_dead by setting CPU_DEAD */
if (per_cpu(cpu_state, cpu) == CPU_DEAD) {
printk ("CPU %d is now offline\n", cpu);
- if (1 == num_online_cpus())
- alternatives_smp_switch(0);
return;
}
msleep(100);
diff -X dontdiff -urN kernel-centos/arch/x86_64/kernel/traps.c
linux-2.6.18.x86_64-bitband/arch/x86_64/kernel/traps.c
--- kernel-centos/arch/x86_64/kernel/traps.c 2007-10-25
17:29:02.000000000 +0200
+++ linux-2.6.18.x86_64-bitband/arch/x86_64/kernel/traps.c
2007-11-05 14:44:53.000000000 +0200
@@ -413,6 +413,7 @@
_show_stack(tsk, NULL, rsp);
}
+EXPORT_SYMBOL(show_stack);
/*
* The architecture-independent dump_stack generator
*/
diff -ruN -X vanilla/Documentation/dontdiff
linux-2.6.18.x86_64/kernel/cpu.c
linux-2.6.18.x86_64-bitband/kernel/cpu.c
--- linux-2.6.18.x86_64/kernel/cpu.c 2007-10-08 12:05:31.000000000 +0200
+++ linux-2.6.18.x86_64-bitband/kernel/cpu.c 2007-10-08
11:24:55.000000000 +0200
@@ -195,6 +195,9 @@
mutex_unlock(&cpu_add_remove_lock);
return err;
}
+
+EXPORT_SYMBOL(cpu_down);
+
#endif /*CONFIG_HOTPLUG_CPU*/
/* Requires cpu_add_remove_lock to be held */
@@ -247,6 +250,8 @@
return err;
}
+EXPORT_SYMBOL(cpu_up);
+
#ifdef CONFIG_SUSPEND_SMP
static cpumask_t frozen_cpus;
diff -ruN -X vanilla/Documentation/dontdiff
linux-2.6.18.x86_64/kernel/kallsyms.c
linux-2.6.18.x86_64-bitband/kernel/kallsyms.c
--- linux-2.6.18.x86_64/kernel/kallsyms.c 2007-10-08
12:05:30.000000000 +0200
+++ linux-2.6.18.x86_64-bitband/kernel/kallsyms.c 2007-10-08
11:26:49.000000000 +0200
@@ -155,6 +155,8 @@
return module_kallsyms_lookup_name(name);
}
+EXPORT_SYMBOL(kallsyms_lookup_name);
+
/*
* Lookup an address
* - modname is set to NULL if it's in the kernel
7.2 SOS frame work
see attached tarball
7.2 timer code
see attached tarball
8. Bibliiography
1. "Understanding the Linux Kernel". Daniel P.Bovet & Marco Cesati.
3-rd edition.
2. "What every programmer should know about memory" . By Ulrich
Drepper. Red Hat, In LWN articles
3. LXR.
4. Computer Architecture , Michal J Flynn.
--
Raz
--
To unsubscribe from this list: send an email with
"unsubscribe kernelnewbies" to ecartis@xxxxxxxxxxxx
Please read the FAQ at http://kernelnewbies.org/FAQ
