Skip to content
Snippets Groups Projects
Commit 336fba97 authored by Peter Gerwinski's avatar Peter Gerwinski
Browse files

Echtzeit

parent e7521c21
No related branches found
No related tags found
No related merge requests found
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
<html>
<head>
<meta http-equiv="Content-Type"
content="text/html; charset=iso-8859-1">
<meta name="Template"
content="C:\Program Files\Microsoft Office\Office\html.dot">
<meta name="GENERATOR" content="Microsoft FrontPage 6.0">
<title>What really happened on Mars? -- Authoritative Account</title>
</head>
<body bgcolor="#FFFFFF" link="#0000FF" vlink="#800080">
<p><font size="2">From: Glenn E Reeves[SMTP:</font><a
href="mailto:Glenn.E.Reeves@jpl.nasa.gov" onClick="stc(this, 1)"><font size="2">Glenn.E.Reeves@jpl.nasa.gov</font></a><font
size="2">]<br>
Sent: Monday, December 15, 1997 10:28 AM<br>
To: jack@telerobotics.jpl.nasa.gov;
mishkin@telerobotics.jpl.nasa.gov; rmanning@mail1.jpl.nasa.gov;
miked@wrs.com; Mike Jones; gregg@wrs.com; dlre@chevron.com;
David.Cummings@andrew.com; pky@appsig.com;
karl_schneider@radixtek.com; lisa@wrs.com; dave@wrs.com; John
Krumm; bilzabub@pclink.com; mike_conrad@atk.com;
elf@wavemark.com; mlisa@erols.com; Thomas Blumer;
shep@sunne.East.Sun.com; kmarks@xionics.com;
frederick_hallock@lotus.com; athomas@xenergy.com;
andy@harlequin.com; don@wavemark.com; bostic@bostic.com;
samiller@red.primextech.com; higgins@daffy.fnal.gov;
intersci@smof.demon.co.uk; kay@rsvl.unisys.com; glen@qnx.com;
nev@bostic.com; jharris@mail1.jpl.nasa.gov; Richard A Volpe;
Richard V Welch; Henry W Stone; Allen R Sirota; Kim P Gostelow;
Robert D Rasmussen; David J Eisenman; Daniel E Erickson; Udo
Wehmeier; Peter R Gluck; David E Smyth; Robert C Barry; Steven A
Stolper; Alfred D Schoepke; David C Gruel; Guy S Beutelschies;
Robert M Manning; Friedrich A Krug; Samuel H Zingales; Steven M
Collins; J Morgan Parker<br>
Subject: Re: [Fwd: FW: What really happened on Mars?]</font></p>
<h2><font face="Arial">What really happened on Mars ? </font></h2>
<p><font size="2">By now most of you have read Mike&#146;s
(mbj@microsoft.com) summary of Dave Wilner&#146;s comments given
at the IEEE Real-Time Systems Symposium. I don&#146;t know Mike
and I didn&#146;t attend the symposium (though I really wish I
had now) and I have not talked to Dave Wilner since before the
talk. However, I did lead the software team for the Mars
Pathfinder spacecraft. So, instead of trying to find out what was
said I will just tell you what happened. You can make your own
judgments.</font></p>
<p><font size="2">I sent this message out to everyone who was a
recipient of Mike&#146;s original that I had an email address
for. Please pass it on to anyone you sent the first one to. Mike,
I hope you will post this wherever you posted the original.</font></p>
<p><font size="2">Since I want to make sure the problem is
clearly understood I need to step through each of the areas which
contributed to the problem.</font></p>
<h2><font size="4" face="Arial"><b>THE HARDWARE</b></font></h2>
<p><font size="2">The simplified view of the Mars Pathfinder
hardware architecture looks like this. A single CPU controls the
spacecraft. It resides on a VME bus which also contains interface
cards for the radio, the camera, and an interface to a 1553 bus.
The 1553 bus connects to two places : The &quot;cruise
stage&quot; part of the spacecraft and the &quot;lander&quot;
part of the spacecraft. The hardware on the cruise part of the
spacecraft controls thrusters, valves, a sun sensor, and a star
scanner. The hardware on the lander part provides an interface to
accelerometers, a radar altimeter, and an instrument for
meteorological science known as the ASI/MET. The hardware which
we used to interface to the 1553 bus (at both ends) was inherited
from the Cassini spacecraft. This hardware came with a specific
paradigm for its usage : the software will schedule activity at
an 8 Hz rate. This **feature** dictated the architecture of the
software which controls both the 1553 bus and the devices
attached to it. </font></p>
<h2><font size="4" face="Arial"><b>THE SOFTWARE ARCHITECTURE</b></font></h2>
<p><font size="2">The software to control the 1553 bus and the
attached instruments was implemented as two tasks. The first task
controlled the setup of transactions on the 1553 bus (called the
bus scheduler or bc_sched task) and the second task handled the
collection of the transaction results i.e. the data. The second
task is referred to as the bc_dist (for distribution) task. A
typical timeline for the bus activity for a single cycle is shown
below. It is not to scale. This cycle was constantly repeated.</font></p>
<pre><font size="2"> |&lt; ------------- .125 seconds ------------------------&gt;|</font></pre>
<pre><font size="2"> |&lt;***************| |********| |**&gt;|</font></pre>
<pre><font size="2" face="Courier New"> |&lt;- bc_dist active -&gt;| bc_sched active
</font><font size="2"> |&lt; -bus active -&gt;| |&lt;-&gt;|</font></pre>
<pre><font size="2" face="Courier New">----|----------------|--------------------|--------|---|---|-------
</font><font size="2"> t1 t2 t3 t4 t5 t1</font></pre>
<p><font size="2">The *** are periods when tasks other than the
ones listed are executing. Yes, there is some idle time.</font></p>
<p><font size="2">t1 - bus hardware starts via hardware control
on the 8 Hz boundary. The transactions for the this cycle had
been set up by the previous execution of the bc_sched task.<br>
t2 - 1553 traffic is complete and the bc_dist task is awakened.<br>
t3 - bc_dist task has completed all of the data distribution<br>
t4 - bc_sched task is awakened to setup transactions for the next
cycle<br>
t5 - bc_sched activity is complete</font></p>
<p><font size="2">The bc_sched and bc_dist tasks check each cycle
to be sure that the other had completed its execution. The
bc_sched task is the highest priority task in the system (except
for the vxWorks &quot;tExec&quot; task). The bc_dist is third
highest (a task controlling the entry and landing is second). All
of the tasks which perform other spacecraft functions are lower.
Science functions, such as imaging, image compression, and the
ASI/MET task are still lower.</font></p>
<p><font size="2">Data is collected from devices connected to the
1553 bus only when they are powered. Most of the tasks in the
system that access the information collected over the 1553 do so
via a double buffered shared memory mechanism into which the
bc_dist task places the latest data. The exception to this is the
ASI/MET task which is delivered its information via an
interprocess communication mechanism (IPC). The IPC mechanism
uses the vxWorks pipe() facility. Tasks wait on one or more IPC
&quot;queues&quot; for messages to arrive. Tasks use the select()
mechanism to wait for message arrival. Multiple queues are used
when both high and lower priority messages are required. Most of
the IPC traffic in the system is not for the delivery of
real-time data. However, again, the exception to this is the use
of the IPC mechanism with the ASI/MET task. The cause of the
reset on Mars was in the use and configuration of the IPC
mechanism.</font></p>
<h2><font size="4" face="Arial"><b>THE FAILURE</b></font></h2>
<p><font size="2">The failure was identified by the spacecraft as
a failure of the bc_dist task to complete its execution before
the bc_sched task started. The reaction to this by the spacecraft
was to reset the computer. This reset reinitializes all of the
hardware and software. It also terminates the execution of the
current ground commanded activities. No science or engineering
data is lost that has already been collected (the data in RAM is
recovered so long as power is not lost). However, the remainder
of the activities for that day were not accomplished until the
next day.</font></p>
<p><font size="2">The failure turned out to be a case of priority
inversion (how we discovered this and how we fixed it are covered
later). The higher priority bc_dist task was blocked by the much
lower priority ASI/MET task that was holding a shared resource.
The ASI/MET task had acquired this resource and then been
preempted by several of the medium priority tasks. When the
bc_sched task was activated, to setup the transactions for the
next 1553 bus cycle, it detected that the bc_dist task had not
completed its execution. The resource that caused this problem
was a mutual exclusion semaphore used within the select()
mechanism to control access to the list of file descriptors that
the select() mechanism was to wait on.</font></p>
<p><font size="2">The select mechanism creates a mutual exclusion
semaphore to protect the &quot;wait list&quot; of file
descriptors for those devices which support select. The vxWorks
pipe() mechanism is such a device and the IPC mechanism we used
is based on using pipes. The ASI/MET task had called select,
which had called pipeIoctl(), which had called selNodeAdd(),
which was in the process of giving the mutex semaphore. The ASI/
MET task was preempted and semGive() was not completed. Several
medium priority tasks ran until the bc_dist task was activated.
The bc_dist task attempted to send the newest ASI/MET data via
the IPC mechanism which called pipeWrite(). pipeWrite() blocked,
taking the mutex semaphore. More of the medium priority tasks
ran, still not allowing the ASI/MET task to run, until the
bc_sched task was awakened. At that point, the bc_sched task
determined that the bc_dist task had not completed its cycle (a
hard deadline in the system) and declared the error that
initiated the reset.</font></p>
<h2><font size="4" face="Arial"><b>HOW WE FOUND IT</b></font></h2>
<p><font size="2">The software that flies on Mars Pathfinder has
several debug features within it that are used in the lab but are
not used on the flight spacecraft (not used because some of them
produce more information than we can send back to Earth). These
features were not &quot;fortuitously&quot; left enabled but
remain in the software by design. We strongly believe in the
&quot;test what you fly and fly what you test&quot; philosophy.</font></p>
<p><font size="2">One of these tools is a trace/log facility
which was originally developed to find a bug in an early version
of the vxWorks port (Wind River ported vxWorks to the RS6000
processor for us for this mission). This trace/log facility was
built by David Cummings who was one of the software engineers on
the task. Lisa Stanley, of Wind River, took this facility and
instrumented the pipe services, msgQ services, interrupt
handling, select services, and the tExec task. The facility
initializes at startup and continues to collect data (in ring
buffers) until told to stop. The facility produces a voluminous
dump of information when asked. </font></p>
<p><font size="2">After the problem occurred on Mars we did run
the same set of activities over and over again in the lab. The
bc_sched was already coded so as to stop the trace/log collection
and dump the data (even though we knew we could not get the dump
in flight) for this error. So, when we went into the lab to test
it we did not have to change the software.</font></p>
<p><font size="2">In less that 18 hours we were able to cause the
problem to occur. Once we were able to reproduce the failure the
priority inversion problem was obvious.</font></p>
<h2><font size="4" face="Arial"><b>HOW WAS THE PROBLEM CORRECTED</b></font></h2>
<p><font size="2">Once we understood the problem the fix appeared obvious :
change the creation flags for the semaphore so as to enable priority inheritance. The Wind River folks, for many
of their services, supply global configuration variables for
parameters such as the &quot;options&quot; parameter for the semMCreate used by the select service (although this is not
documented and those who do not have vxWorks source code or have
not studied the source code might be unaware of this feature).
However, the fix is not so obvious for several reasons :</font></p>
<blockquote>
<font size="2">1) The code for this is in the selectLib()
and is common for all device creations. When you change
this global variable all of the select semaphores created
after that point will be created with the new options.
There was no easy way in our initialization logic to only
modify the semaphore associated with the pipe used for
bc_dist task to ASI/MET task communications.</font>
<p><font size="2">2) If we make this change, and it is
applied on a global basis, how will this change the
behavior of the rest of the system ?</font></p>
<p><font size="2">3) The priority inheritance option was
deliberately left out by Wind River in the default selectLib() service for
optimum performance. How will performance degrade if we turn priority
inheritance on
?</font></p>
<p><font size="2">4) Was there some intrinsic behavior of the select
mechanism itself that would change if priority inheritance was enabled ?</font></p>
</blockquote>
<p><font size="2">We did end up modifying the global variable to
include the priority inheritance option. This corrected the problem. We
asked Wind River to analyze the potential impacts for (3) and
(4). They concluded that the performance impact would be minimal
and that the behavior of select() would not change so long as
there was always only one task waiting for any particular file
descriptor. This is true in our system. I believe that the debate
at Wind River still continues on whether the priority inheritance
option should be on as the default. For (1) and (2) the change
did alter the characteristics of all of the select semaphores. We
concluded, both by analysis and test, that there was no adverse
behavior. We tested the system extensively before we changed the
software on the spacecraft.</font></p>
<h2><font size="4" face="Arial"><b>HOW WE CHANGED THE SOFTWARE ON
THE SPACECRAFT</b></font></h2>
<p><font size="2">No, we did not use the vxWorks shell to change
the software (although the shell is usable on the spacecraft).
The process of &quot;patching&quot; the software on the
spacecraft is a specialized process. It involves sending the
differences between what you have onboard and what you want (and
have on Earth) to the spacecraft. Custom software on the
spacecraft (with a whole bunch of validation) modifies the
onboard copy. If you want more info you can send me email.</font></p>
<h2><font size="4" face="Arial"><b>WHY DIDN&#146;T WE CATCH IT
BEFORE LAUNCH ?</b></font></h2>
<p><font size="2">The problem would only manifest itself when
ASI/MET data was being collected and intermediate tasks were
heavily loaded. Our before launch testing was limited to the
&quot;best case&quot; high data rates and science activities. The
fact that data rates from the surface were higher than
anticipated and the amount of science activities proportionally
greater served to aggravate the problem. We did not expect nor
test the &quot;better than we could have ever imagined&quot;
case.</font></p>
<h2><font size="4" face="Arial"><b>HUMAN NATURE, DEADLINE
PRESSURES</b></font></h2>
<p><font size="2">We did see the problem before landing but could
not get it to repeat when we tried to track it down. It was not
forgotten nor was it deemed unimportant. Yes, we were
concentrating heavily on the entry and landing software. Yes, we
considered this problem lower priority. Yes, we would have liked
to have everything perfect before landing. However, I don&#146;t
see any problem here other than we ran out of time to get the
lower priority issues completed.</font></p>
<p><font size="2">We did have one other thing on our side; we
knew how robust our system was because that is the way we
designed it.</font></p>
<p><font size="2">We knew that if this problem occurred we would
reset. We built in mechanisms to recover the current activity so
that there would be no interruptions in the science data
(although this wasn&#146;t used until later in the landed
mission). We built in the ability (and tested it) to go through
multiple resets while we were going through the Martian
atmosphere. We designed the software to recover from radiation
induced errors in the memory or the processor. The spacecraft
would have even done a 60 day mission on its own, including
deploying the rover, if the radio receiver had broken when we
landed. There are a large number of safeguards in the system to
ensure robust, continued operation in the event of a failure of
this type. These safeguards allowed us to designate problems of
this nature as lower priority.</font></p>
<p><font size="2">We had our priorities right.</font></p>
<h2><font size="4" face="Arial"><b>ANALYSIS AND LESSONS</b></font></h2>
<p><font size="2">Did we (the JPL team) make an error in assuming
how the select/pipe mechanism would work ? Yes, probably. But
there was no conscious decision to not have the priority inheritance option enabled. We just missed it. There are several other
places in the flight software where similar protection is
required for critical data structures and the semaphores do have
priority inversion protection. A good lesson when you fly COTS
stuff - make sure you know how it works.</font></p>
<p><font size="2">Mike is quite correct in saying that we could
not have figured this out **ever** if we did not have the tools
to give us the insight. We built many of the tools into the
software for exactly this type of problem. We always planned to
leave them in. In fact, the shell (and the stdout stream) were
very useful the entire mission. If you want more detail send me a
note.</font></p>
<h2><font size="4" face="Arial"><b>SETTING THE RECORD STRAIGHT</b></font></h2>
<p><font size="2">First, I want to make sure that everyone
understands how I feel in regard to Wind River. These folks did a
fantastic job for us. They were enthusiastic and supported us
when we came to them and asked them to do an affordable port of
vxWorks. They delivered the alpha version in 3 months. When we
had a problem they put some of the brightest engineers I have
ever worked with on the problem. Our communication with them was
fantastic. If they had not done such a professional job the Mars
Pathfinder mission would not have been the success that it is.</font></p>
<p><font size="2">Second, Dave Wilner did talk to me about this
problem before he gave his talk. I could not find my notes where
I had detailed the description of the problem. So, I winged it
and I sure did get it wrong. Sorry Dave.</font></p>
<h2><font size="4" face="Arial"><b>ACKNOWLEDGMENTS</b></font></h2>
<p><font size="2">First, thanks to Mike for writing a very nice
description of the talk. I think I have had probably 400 people
send me copies. You gave me the push to write the part of the
Mars Pathfinder End-of-Mission report that I had been
procrastinating doing.</font></p>
<p><font size="2">A special thanks to Steve Stolper for helping
me do this.</font></p>
<p><font size="2">The biggest thanks should go to the software
team that I had the privilege of leading and whose expertise
allowed us to succeed :</font></p>
<blockquote>
<p><font size="2">Pam Yoshioka<br>
Dave Cummings<br>
Don Meyer<br>
Karl Schneider<br>
Greg Welz<br>
Rick Achatz<br>
Kim Gostelow<br>
Dave Smyth<br>
Steve Stolper</font> </p>
</blockquote>
<p><font size="2">Also ,</font></p>
<blockquote>
<p><font size="2">Miguel San Martin<br>
Sam Sirlin<br>
Brian Lazara (WRS)<br>
Mike Deliman (WRS)<br>
Lisa Stanley (WRS)</font> </p>
</blockquote>
<p><font size="2">---------------------</font></p>
<p><font size="2">Glenn Reeves<br>
Mars Pathfinder Flight Software Cognizant Engineer<br>
</font><a href="mailto:glenn.e.reeves@jpl.nasa.gov" onClick="stc(this, 2)"><font
size="2">glenn.e.reeves@jpl.nasa.gov</font></a></p>
<hr>
<p><font size="2">Related Links:</font></p>
<ul>
<li><a href="Mars_Pathfinder.html" onClick="stc(this, 3)"><font size="2">Mike Jones'
initial &quot;What really happened on Mars?&quot; message</font></a></li>
<li><a href="./" onClick="stc(this, 4)"><font size="2">Top-level &quot;What really
happened on Mars?&quot; page</font></a></li>
<li><a href="../" onClick="stc(this, 5)"><font size="2">Mike Jones' home page</font></a></li>
</ul>
<script type="text/javascript">
function stc(e,linkIndex){if(document.images){var linkText;if(navigator.appName.toLowerCase()=="microsoft internet explorer"){linkText=e.innerText}else{linkText=e.textContent}if(linkText==""){if(e.firstChild){var firstChild=e.firstChild.nodeName.toUpperCase();if(firstChild=="IMG"){linkText="Image: "+getName(e.firstChild.getAttribute('src'))}}else{var nodeName=e.nodeName.toUpperCase();if(nodeName=="AREA"){linkText="ImageMap: "+e.href}}}if(linkText==""){linkText=e.href}(new Image()).src="/a/i/stg.gif?f="+escape(document.location.href)+"&t="+escape(e.href)+"&i="+linkIndex+"&n="+escape(trimString(linkText))}}function getName(s){if(s.lastIndexOf('/')>=0){return(s.substring(s.lastIndexOf('/')+1,s.length))}else{return(s)}}function trimString(s){return(s.replace(/^\s*/,"").replace(/\s*$/,""))}
</script>
</body>
</html>
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
<html>
<head>
<meta http-equiv="Content-Type"
content="text/html; charset=iso-8859-1">
<meta name="GENERATOR" content="Microsoft FrontPage 2.0">
<title>What really happened on Mars?</title>
</head>
<body bgcolor="#FFFFFF">
<p><font size="2" face="Courier New">From: Mike Jones &lt;</font><a
href="mailto:mbj@microsoft.com" onClick="stc(this, 1)"><font size="2" face="Courier New">mbj@microsoft.com</font></a><font
size="2" face="Courier New">&gt;<br>
Sent: Sunday, December 07, 1997 6:47 PM<br>
Subject: What really happened on Mars?</font></p>
<p><font size="2">The Mars Pathfinder mission was widely
proclaimed as &quot;flawless&quot; in the early days after its
July 4<sup>th</sup>, 1997 landing on the Martian surface.
Successes included its unconventional &quot;landing&quot; --
bouncing onto the Martian surface surrounded by airbags,
deploying the Sojourner rover, and gathering and transmitting
voluminous data back to Earth, including the panoramic pictures
that were such a hit on the Web. But a few days into the mission,
not long after Pathfinder started gathering meteorological data,
the spacecraft began experiencing total system resets, each
resulting in losses of data. The press reported these failures in
terms such as &quot;software glitches&quot; and &quot;the
computer was trying to do too many things at once&quot;.</font></p>
<p><font size="2">This week at the IEEE Real-Time Systems
Symposium I heard a fascinating keynote address by David Wilner,
Chief Technical Officer of Wind River Systems. Wind River makes
VxWorks, the real-time embedded systems kernel that was used in
the Mars Pathfinder mission. In his talk, he explained in detail
the actual software problems that caused the total system resets
of the Pathfinder spacecraft, how they were diagnosed, and how
they were solved. I wanted to share his story with each of you.</font></p>
<p><font size="2">VxWorks provides preemptive priority scheduling
of threads. Tasks on the Pathfinder spacecraft were executed as
threads with priorities that were assigned in the usual manner
reflecting the relative urgency of these tasks.</font></p>
<p><font size="2">Pathfinder contained an &quot;information
bus&quot;, which you can think of as a shared memory area used
for passing information between different components of the
spacecraft. A bus management task ran frequently with high
priority to move certain kinds of data in and out of the
information bus. Access to the bus was synchronized with mutual
exclusion locks (mutexes).</font></p>
<p><font size="2">The meteorological data gathering task ran as
an infrequent, low priority thread, and used the information bus
to publish its data. When publishing its data, it would acquire a
mutex, do writes to the bus, and release the mutex. If an
interrupt caused the information bus thread to be scheduled while
this mutex was held, and if the information bus thread then
attempted to acquire this same mutex in order to retrieve
published data, this would cause it to block on the mutex,
waiting until the meteorological thread released the mutex before
it could continue. The spacecraft also contained a communications
task that ran with medium priority.</font></p>
<p><font size="2">Most of the time this combination worked fine.
However, very infrequently it was possible for an interrupt to
occur that caused the (medium priority) communications task to be
scheduled during the short interval while the (high priority)
information bus thread was blocked waiting for the (low priority)
meteorological data thread. In this case, the long-running
communications task, having higher priority than the
meteorological task, would prevent it from running, consequently
preventing the blocked information bus task from running. After
some time had passed, a watchdog timer would go off, notice that
the data bus task had not been executed for some time, conclude
that something had gone drastically wrong, and initiate a total
system reset.</font></p>
<p><font size="2">This scenario is a classic case of priority
inversion.</font></p>
<h2><font size="4" face="Arial"><b>HOW WAS THIS DEBUGGED?</b></font></h2>
<p><font size="2">VxWorks can be run in a mode where it records a
total trace of all interesting system events, including context
switches, uses of synchronization objects, and interrupts. After
the failure, JPL engineers spent hours and hours running the
system on the exact spacecraft replica in their lab with tracing
turned on, attempting to replicate the precise conditions under
which they believed that the reset occurred. Early in the
morning, after all but one engineer had gone home, the engineer
finally reproduced a system reset on the replica. Analysis of the
trace revealed the priority inversion.</font></p>
<h2><font size="4" face="Arial"><b>HOW WAS THE PROBLEM CORRECTED?</b></font></h2>
<p><font size="2">When created, a VxWorks mutex object accepts a
boolean parameter that indicates whether priority inheritance
should be performed by the mutex. The mutex in question had been
initialized with the parameter off; had it been on, the
low-priority meteorological thread would have inherited the
priority of the high-priority data bus thread blocked on it while
it held the mutex, causing it be scheduled with higher priority
than the medium-priority communications task, thus preventing the
priority inversion. Once diagnosed, it was clear to the JPL
engineers that using priority inheritance would prevent the
resets they were seeing.</font></p>
<p><font size="2">VxWorks contains a C language interpreter
intended to allow developers to type in C expressions and
functions to be executed on the fly during system debugging. The
JPL engineers fortuitously decided to launch the spacecraft with
this feature still enabled. By coding convention, the
initialization parameter for the mutex in question (and those for
two others which could have caused the same problem) were stored
in global variables, whose addresses were in symbol tables also
included in the launch software, and available to the C
interpreter. A short C program was uploaded to the spacecraft,
which when interpreted, changed the values of these variables
from FALSE to TRUE. No more system resets occurred.</font></p>
<h2><font size="4" face="Arial"><b>ANALYSIS AND LESSONS</b></font></h2>
<p><font size="2">First and foremost, diagnosing this problem as
a black box would have been impossible. Only detailed traces of
actual system behavior enabled the faulty execution sequence to
be captured and identified.</font></p>
<p><font size="2">Secondly, leaving the &quot;debugging&quot;
facilities in the system saved the day. Without the ability to
modify the system in the field, the problem could not have been
corrected.</font></p>
<p><font size="2">Finally, the engineer's initial analysis that
&quot;the data bus task executes very frequently and is
time-critical -- we shouldn't spend the extra time in it to
perform priority inheritance&quot; was exactly wrong. It is
precisely in such time critical and important situations where
correctness is essential, even at some additional performance
cost.</font></p>
<h2><font size="4" face="Arial"><b>HUMAN NATURE, DEADLINE
PRESSURES</b></font></h2>
<p><font size="2">David told us that the JPL engineers later
confessed that one or two system resets had occurred in their
months of pre-flight testing. They had never been reproducible or
explainable, and so the engineers, in a very human-nature
response of denial, decided that they probably weren't important,
using the rationale &quot;it was probably caused by a hardware
glitch&quot;.</font></p>
<p><font size="2">Part of it too was the engineers' focus. They
were extremely focused on ensuring the quality and flawless
operation of the landing software. Should it have failed, the
mission would have been lost. It is entirely understandable for
the engineers to discount occasional glitches in the
less-critical land-mission software, particularly given that a
spacecraft reset was a viable recovery strategy at that phase of
the mission.</font></p>
<h2><font size="4" face="Arial"><b>THE IMPORTANCE OF GOOD
THEORY/ALGORITHMS</b></font></h2>
<p><font size="2">David also said that some of the real heroes of
the situation were some people from CMU who had published a paper
he'd heard presented many years ago who first identified the
priority inversion problem and proposed the solution. He
apologized for not remembering the precise details of the paper
or who wrote it. Bringing things full circle, it turns out that
the three authors of this result were all in the room, and at the
end of the talk were encouraged by the program chair to stand and
be acknowledged. They were Lui Sha, John Lehoczky, and Raj
Rajkumar. When was the last time you saw a room of people cheer a
group of computer science theorists for their significant
practical contribution to advancing human knowledge? :-) It was
quite a moment.</font></p>
<h2><font size="4" face="Arial"><b>POSTLUDE</b></font></h2>
<p><font size="2">For the record, the paper was:</font></p>
<p><font size="2">L. Sha, R. Rajkumar, and J. P. Lehoczky.
Priority Inheritance Protocols: An Approach to Real-Time
Synchronization. In IEEE Transactions on Computers, vol. 39, pp.
1175-1185, Sep. 1990.</font></p>
<hr>
<p><font size="2">This note was widely circulated after I sent it
to a few friends in the systems community. Among other places, it
appeared in Peter G. Neumann's moderated </font><a
href="http://www.csl.sri.com/risksinfo.html" onClick="stc(this, 2)"><font size="2">Risks
Forum</font></a><font size="2"> (comp.risks) on Tuesday, 9
December 1997 in issue RISKS-19.49.</font></p>
<p><font size="2">Be sure to also read the </font><a
href="Authoritative_Account.html" onClick="stc(this, 3)"><font size="2">follow-up
message from Glenn Reeves of JPL</font></a><font size="2">, who
led the software team for the Mars Pathfinder spacecraft.</font></p>
<p><font size="2">Related Links:</font></p>
<ul>
<li><a href="Authoritative_Account.html" onClick="stc(this, 4)"><font size="2">Follow-up
message from Glenn Reeves of JPL</font></a><font size="2">,
who led the software team for the Mars Pathfinder
spacecraft</font></li>
<li><a href="Priority_Inversion.html" onClick="stc(this, 5)"><font size="2">Notes on
history of Priority Inversion</font></a></li>
<li><a href="./" onClick="stc(this, 6)"><font size="2">Top-level &quot;What really
happened on Mars?&quot; page</font></a></li>
<li><a href="../" onClick="stc(this, 7)"><font size="2">Mike Jones' home page</font></a></li>
</ul>
<script type="text/javascript">
function stc(e,linkIndex){if(document.images){var linkText;if(navigator.appName.toLowerCase()=="microsoft internet explorer"){linkText=e.innerText}else{linkText=e.textContent}if(linkText==""){if(e.firstChild){var firstChild=e.firstChild.nodeName.toUpperCase();if(firstChild=="IMG"){linkText="Image: "+getName(e.firstChild.getAttribute('src'))}}else{var nodeName=e.nodeName.toUpperCase();if(nodeName=="AREA"){linkText="ImageMap: "+e.href}}}if(linkText==""){linkText=e.href}(new Image()).src="/a/i/stg.gif?f="+escape(document.location.href)+"&t="+escape(e.href)+"&i="+linkIndex+"&n="+escape(trimString(linkText))}}function getName(s){if(s.lastIndexOf('/')>=0){return(s.substring(s.lastIndexOf('/')+1,s.length))}else{return(s)}}function trimString(s){return(s.replace(/^\s*/,"").replace(/\s*$/,""))}
</script>
</body>
</html>
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
<html>
<head>
<meta http-equiv="Content-Type"
content="text/html; charset=iso-8859-1">
<meta name="GENERATOR" content="Microsoft FrontPage 2.0">
<title>What really happened on Mars?</title>
</head>
<body bgcolor="#FFFFFF">
<h2>What really happened on Mars?</h2>
<p><font size="3">The Mars Pathfinder mission was widely
proclaimed as &quot;flawless&quot; in the early days after its
July 4<sup>th</sup>, 1997 landing on the Martian surface.
Successes included its unconventional
&quot;landing&quot;&#151;bouncing onto the Martian surface
surrounded by airbags, deploying the Sojourner rover, and
gathering and transmitting voluminous data back to Earth,
including the panoramic pictures that were such a hit on the Web.
But a few days into the mission, not long after Pathfinder
started gathering meteorological data, the spacecraft began
experiencing system resets. The press reported these failures in
terms such as &quot;software glitches&quot; and &quot;the
computer was trying to do too many things at once&quot;.</font></p>
<p>On December 3<font size="3"><sup>rd</sup></font>, 1997 I
attended a fascinating talk by David Wilner, Chief Technical
Officer of Wind River Systems, maker of VxWorks, the real-time
embedded systems kernel used in the Mars Pathfinder mission, who
explained the software flaw. I sent a description of his talk
entitled &quot;<a href="Mars_Pathfinder.html" onClick="stc(this, 1)">What really
happened on Mars?</a>&quot; to a few friends in the systems
community, after which it was widely circulated. Among other
places, it appeared in Peter G. Neumann's moderated Risks Forum
(comp.risks) on Tuesday, 9 December 1997 in issue RISKS-19.49.</p>
<p>On December 15<font size="3"><sup>th</sup></font> I was
honored to receive <a href="Authoritative_Account.html" onClick="stc(this, 2)">this
detailed first-hand account</a> of &quot;What really happened on
Mars?&quot; from Glenn Reeves of JPL, who led the software team
for the Mars Pathfinder spacecraft. It contains both a far more
detailed and accurate description of the problem than I sent and
many valuable insights into what factors, both at design time and
mission time, enabled Pathfinder to be such a stunning success,
overcoming this one flaw, and continuing with its mission. I
highly recommend his account.</p>
<p>One other footnote. Since I sent my original message, a number
of people replied saying that the priority inversion problem and
solutions to it were known much prior to Sha et al.'s 1990
article. More history on priority inversion is available <a
href="Priority_Inversion.html" onClick="stc(this, 3)">here</a>.</p>
<p>Finally, I'd like to state publicly that I greatly admire what
the Pathfinder team accomplished. While my account starts by
describing a software flaw, it also describes a just few of the
many, many things that were done right, enabling the flaw to be
tolerated, identified, and fixed in the field. Pathfinder,
including its software systems, is a tremendous engineering and
scientific success story. I know we'll be learning things from it
for many years to come. My hope is that these accounts will help
others to learn from and emulate their successes.</p>
<blockquote>
<blockquote>
<p>Michael B. Jones<br>
Redmond, Washington -- December 16<font size="3"><sup>th</sup></font>,
1997</p>
</blockquote>
</blockquote>
<hr>
<p><font size="2">Related Links:</font></p>
<ul>
<li><a href="Mars_Pathfinder.html" onClick="stc(this, 4)"><font size="2">Mike Jones'
initial &quot;What really happened on Mars?&quot; message</font></a></li>
<li><a href="Authoritative_Account.html" onClick="stc(this, 5)"><font size="2">Follow-up
message from Glenn Reeves of JPL</font></a><font size="2">,
who led the software team for the Mars Pathfinder
spacecraft</font></li>
<li><a href="Priority_Inversion.html" onClick="stc(this, 6)"><font size="2">Notes on
history of Priority Inversion</font></a></li>
<li><a href="http://www.csl.sri.com/risksinfo.html" onClick="stc(this, 7)"><font
size="2">Risks Forum information</font></a></li>
<li><a href="http://cs-www.bu.edu/pub/ieee-rts/rtss97/" onClick="stc(this, 8)"><font
size="2">18th IEEE Real-Time Systems Symposium </font></a></li>
<li><a href="http://mpfwww.jpl.nasa.gov/" onClick="stc(this, 9)"><font size="2">Jet
Propulsion Lab Pathfinder site</font></a></li>
<li><a href="http://www.windriver.com/" onClick="stc(this, 10)"><font size="2">Wind
River Systems</font></a></li>
<li><a href="../" onClick="stc(this, 11)"><font size="2">Mike Jones' home page</font></a></li>
</ul>
<p><font size="2">Last modified December 16, 1997.</font></p>
<script type="text/javascript">
function stc(e,linkIndex){if(document.images){var linkText;if(navigator.appName.toLowerCase()=="microsoft internet explorer"){linkText=e.innerText}else{linkText=e.textContent}if(linkText==""){if(e.firstChild){var firstChild=e.firstChild.nodeName.toUpperCase();if(firstChild=="IMG"){linkText="Image: "+getName(e.firstChild.getAttribute('src'))}}else{var nodeName=e.nodeName.toUpperCase();if(nodeName=="AREA"){linkText="ImageMap: "+e.href}}}if(linkText==""){linkText=e.href}(new Image()).src="/a/i/stg.gif?f="+escape(document.location.href)+"&t="+escape(e.href)+"&i="+linkIndex+"&n="+escape(trimString(linkText))}}function getName(s){if(s.lastIndexOf('/')>=0){return(s.substring(s.lastIndexOf('/')+1,s.length))}else{return(s)}}function trimString(s){return(s.replace(/^\s*/,"").replace(/\s*$/,""))}
</script>
</body>
</html>
File added
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- Created with Inkscape (http://www.inkscape.org/) -->
<svg
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:cc="http://creativecommons.org/ns#"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:svg="http://www.w3.org/2000/svg"
xmlns="http://www.w3.org/2000/svg"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
version="1.0"
width="250"
height="370"
id="svg2"
inkscape:version="0.48.2 r9819"
sodipodi:docname="Operating_system_placement-de.svg">
<title
id="title3174">Wie wirken Benutzer - Anwendung - Betriebssystem - Hardware zusammen</title>
<sodipodi:namedview
pagecolor="#ffffff"
bordercolor="#666666"
borderopacity="1"
objecttolerance="10"
gridtolerance="10"
guidetolerance="10"
inkscape:pageopacity="0"
inkscape:pageshadow="2"
inkscape:window-width="715"
inkscape:window-height="415"
id="namedview52"
showgrid="false"
inkscape:zoom="0.63783784"
inkscape:cx="-149.36441"
inkscape:cy="314.95344"
inkscape:window-x="140"
inkscape:window-y="28"
inkscape:window-maximized="0"
inkscape:current-layer="svg2" />
<defs
id="defs4">
<linearGradient
id="linearGradient3265">
<stop
id="stop3267"
style="stop-color:#ffffff;stop-opacity:1"
offset="0" />
<stop
id="stop3269"
style="stop-color:#ffffff;stop-opacity:0"
offset="1" />
</linearGradient>
<linearGradient
id="linearGradient3251">
<stop
id="stop3253"
style="stop-color:#f5f5f5;stop-opacity:0.96078432"
offset="0" />
<stop
id="stop3255"
style="stop-color:#f5f5f5;stop-opacity:0"
offset="1" />
</linearGradient>
<linearGradient
x1="359.61432"
y1="717.89703"
x2="359.61432"
y2="901.36926"
id="linearGradient3435"
xlink:href="#linearGradient3251"
gradientUnits="userSpaceOnUse"
gradientTransform="matrix(0.1095366,0,0,0.1095366,339.65613,441.60027)" />
<linearGradient
x1="359.61432"
y1="717.89703"
x2="359.61432"
y2="901.36926"
id="linearGradient3437"
xlink:href="#linearGradient3251"
gradientUnits="userSpaceOnUse"
gradientTransform="matrix(-0.1095366,0,0,-0.1095366,423.21886,550.24347)" />
<linearGradient
x1="468.71078"
y1="197.77313"
x2="468.71078"
y2="366.96603"
id="linearGradient3439"
xlink:href="#linearGradient3265"
gradientUnits="userSpaceOnUse"
gradientTransform="matrix(0.1095366,0,0,0.1095366,339.65613,441.60027)" />
<linearGradient
x1="468.71078"
y1="197.77313"
x2="468.71078"
y2="366.96603"
id="linearGradient3441"
xlink:href="#linearGradient3265"
gradientUnits="userSpaceOnUse"
gradientTransform="matrix(0.1095366,0,0,0.1095366,339.65613,470.07979)" />
<linearGradient
x1="468.71078"
y1="197.77313"
x2="468.71078"
y2="366.96603"
id="linearGradient2526"
xlink:href="#linearGradient3265"
gradientUnits="userSpaceOnUse"
gradientTransform="matrix(0.1095366,0,0,0.1095366,339.65613,470.07979)" />
<linearGradient
x1="468.71078"
y1="197.77313"
x2="468.71078"
y2="366.96603"
id="linearGradient2529"
xlink:href="#linearGradient3265"
gradientUnits="userSpaceOnUse"
gradientTransform="matrix(0.3617814,0,0,0.3617814,336.00319,543.94452)" />
<linearGradient
x1="468.71078"
y1="197.77313"
x2="468.71078"
y2="366.96603"
id="linearGradient2534"
xlink:href="#linearGradient3265"
gradientUnits="userSpaceOnUse"
gradientTransform="matrix(0.1095366,0,0,0.1095366,339.65613,441.60027)" />
<linearGradient
x1="468.71078"
y1="197.77313"
x2="468.71078"
y2="366.96603"
id="linearGradient2537"
xlink:href="#linearGradient3265"
gradientUnits="userSpaceOnUse"
gradientTransform="matrix(0.3617814,0,0,0.3617814,336.00319,449.88135)" />
<linearGradient
x1="359.61432"
y1="717.89703"
x2="359.61432"
y2="901.36926"
id="linearGradient2542"
xlink:href="#linearGradient3251"
gradientUnits="userSpaceOnUse"
gradientTransform="matrix(-0.1095366,0,0,-0.1095366,423.21886,550.24347)" />
<linearGradient
x1="359.61432"
y1="717.89703"
x2="359.61432"
y2="901.36926"
id="linearGradient2545"
xlink:href="#linearGradient3251"
gradientUnits="userSpaceOnUse"
gradientTransform="matrix(-0.3617814,0,0,-0.3617814,611.99715,808.71197)" />
<linearGradient
x1="359.61432"
y1="717.89703"
x2="359.61432"
y2="901.36926"
id="linearGradient2550"
xlink:href="#linearGradient3251"
gradientUnits="userSpaceOnUse"
gradientTransform="matrix(0.1095366,0,0,0.1095366,339.65613,441.60027)" />
<linearGradient
x1="359.61432"
y1="717.89703"
x2="359.61432"
y2="901.36926"
id="linearGradient2553"
xlink:href="#linearGradient3251"
gradientUnits="userSpaceOnUse"
gradientTransform="matrix(0.3617814,0,0,0.3617814,336.00319,449.88135)" />
<filter
color-interpolation-filters="sRGB"
id="filter3290">
<feGaussianBlur
id="feGaussianBlur3292"
stdDeviation="4.2491226" />
</filter>
</defs>
<metadata
id="metadata7">
<rdf:RDF>
<cc:Work
rdf:about="">
<dc:format>image/svg+xml</dc:format>
<dc:type
rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
<dc:title>Wie wirken Benutzer - Anwendung - Betriebssystem - Hardware zusammen</dc:title>
<dc:creator>
<cc:Agent>
<dc:title>GGShinobi</dc:title>
</cc:Agent>
</dc:creator>
<dc:language>de</dc:language>
<dc:date>2012-01-21</dc:date>
<dc:relation>http://commons.wikimedia.org/wiki/File:Operating_system_placement.svg</dc:relation>
<dc:subject>
<rdf:Bag>
<rdf:li>Betriebssystem</rdf:li>
<rdf:li>Hardware</rdf:li>
<rdf:li>Anwendung</rdf:li>
<rdf:li>Benutzer</rdf:li>
</rdf:Bag>
</dc:subject>
<cc:license
rdf:resource="http://creativecommons.org/licenses/by-sa/3.0/" />
</cc:Work>
<cc:License
rdf:about="http://creativecommons.org/licenses/by-sa/3.0/">
<cc:permits
rdf:resource="http://creativecommons.org/ns#Reproduction" />
<cc:permits
rdf:resource="http://creativecommons.org/ns#Distribution" />
<cc:requires
rdf:resource="http://creativecommons.org/ns#Notice" />
<cc:requires
rdf:resource="http://creativecommons.org/ns#Attribution" />
<cc:permits
rdf:resource="http://creativecommons.org/ns#DerivativeWorks" />
<cc:requires
rdf:resource="http://creativecommons.org/ns#ShareAlike" />
</cc:License>
</rdf:RDF>
</metadata>
<g
transform="translate(-347.94326,-443.56851)"
id="layer1">
<g
transform="translate(-101.05693,-0.99383)"
id="g3276"
style="filter:url(#filter3290)">
<path
d="m 530.40553,710.17177 -23.94542,12.15358 9.28196,0 0,17.3203 -44.58954,0 c -4.00855,0 -7.23565,3.22707 -7.23565,7.23562 l 0,47.55165 c 0,4.00852 3.2271,7.23562 7.23565,7.23562 l 205.69531,0 c 4.00855,0 7.23562,-3.2271 7.23566,-7.23562 l 0,-47.55165 c 0,-4.00855 -3.22711,-7.23562 -7.23566,-7.23562 l -131.42838,0 0,-17.3203 9.28196,0 -24.29589,-12.15358 z"
id="path3337"
style="fill:#000000;fill-opacity:1;stroke:#000000;stroke-width:1.0853442;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:0.96078431;stroke-dasharray:none" />
<path
d="m 471.15253,457.45614 c -4.00855,0 -7.23565,3.2271 -7.23565,7.23562 l 0,47.55165 c 0,4.00855 3.2271,7.23562 7.23565,7.23562 l 131.41708,0 0,17.24116 -9.28195,0 23.94542,12.14228 24.29586,-12.14228 -9.28196,0 0,-17.24116 44.60086,0 c 4.00855,0 7.23562,-3.22707 7.23566,-7.23562 l 0,-47.55165 c 0,-4.00852 -3.22711,-7.23562 -7.23566,-7.23562 l -205.69531,0 z"
id="path3347"
style="fill:#000000;fill-opacity:0.96078431;stroke:#000000;stroke-width:1.0853442;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:0.96078431;stroke-dasharray:none" />
<path
d="m 530.40553,522.04543 -23.94542,12.15361 9.28196,0 0,17.32027 -44.58954,0 c -4.00855,0 -7.23565,3.2271 -7.23565,7.23562 l 0,47.55165 c 0,4.00855 3.2271,7.23562 7.23565,7.23562 l 131.41708,0 0,17.24116 -9.28195,0 23.94542,12.14228 24.29586,-12.14228 -9.28196,0 0,-17.24116 44.60086,0 c 4.00855,0 7.23562,-3.2271 7.23566,-7.23562 l 0,-47.55165 c 0,-4.00852 -3.22711,-7.23562 -7.23566,-7.23562 l -131.42838,0 0,-17.32027 9.28196,0 -24.29589,-12.15361 z"
id="path3357"
style="fill:#000000;fill-opacity:1;stroke:#000000;stroke-width:1.0853442;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:0.96078431;stroke-dasharray:none" />
<path
d="m 530.40553,616.1086 -23.94542,12.15358 9.28196,0 0,17.3203 -44.58954,0 c -4.00855,0 -7.23565,3.2271 -7.23565,7.23562 l 0,47.55165 c 0,4.00855 3.2271,7.23562 7.23565,7.23562 l 131.41708,0 0,17.24116 -9.28195,0 23.94542,12.14228 24.29586,-12.14228 -9.28196,0 0,-17.24116 44.60086,0 c 4.00855,0 7.23562,-3.2271 7.23566,-7.23562 l 0,-47.55165 c 0,-4.00852 -3.22711,-7.23562 -7.23566,-7.23562 l -131.42838,0 0,-17.3203 9.28196,0 -24.29589,-12.15358 z"
id="path3367"
style="fill:#000000;fill-opacity:1;stroke:#000000;stroke-width:1.0853442;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:0.96078431;stroke-dasharray:none" />
</g>
<g
transform="translate(78.94307,-0.72815)"
id="g2444">
<g
transform="translate(-80,0)"
id="g2438">
<path
d="m 430.40553,710.17177 -23.94542,12.15358 9.28196,0 0,17.3203 -44.58954,0 c -4.00855,0 -7.23565,3.22707 -7.23565,7.23562 l 0,47.55165 c 0,4.00852 3.2271,7.23562 7.23565,7.23562 l 205.69531,0 c 4.00855,0 7.23562,-3.2271 7.23566,-7.23562 l 0,-47.55165 c 0,-4.00855 -3.22711,-7.23562 -7.23566,-7.23562 l -131.42838,0 0,-17.3203 9.28196,0 -24.29589,-12.15358 z"
id="rect2383"
style="fill:#800000;fill-opacity:1;stroke:#000000;stroke-width:1.0853442;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:0.96078431;stroke-dasharray:none" />
<path
d="m 371.15253,457.45614 c -4.00855,0 -7.23565,3.2271 -7.23565,7.23562 l 0,47.55165 c 0,4.00855 3.2271,7.23562 7.23565,7.23562 l 131.41708,0 0,17.24116 -9.28195,0 23.94542,12.14228 24.29586,-12.14228 -9.28196,0 0,-17.24116 44.60086,0 c 4.00855,0 7.23562,-3.22707 7.23566,-7.23562 l 0,-47.55165 c 0,-4.00852 -3.22711,-7.23562 -7.23566,-7.23562 l -205.69531,0 z"
id="rect3223"
style="fill:#ff6600;fill-opacity:0.96078431;stroke:#000000;stroke-width:1.0853442;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:0.96078431;stroke-dasharray:none" />
<path
d="m 430.40553,522.04543 -23.94542,12.15361 9.28196,0 0,17.32027 -44.58954,0 c -4.00855,0 -7.23565,3.2271 -7.23565,7.23562 l 0,47.55165 c 0,4.00855 3.2271,7.23562 7.23565,7.23562 l 131.41708,0 0,17.24116 -9.28195,0 23.94542,12.14228 24.29586,-12.14228 -9.28196,0 0,-17.24116 44.60086,0 c 4.00855,0 7.23562,-3.2271 7.23566,-7.23562 l 0,-47.55165 c 0,-4.00852 -3.22711,-7.23562 -7.23566,-7.23562 l -131.42838,0 0,-17.32027 9.28196,0 -24.29589,-12.15361 z"
id="rect2389"
style="fill:#5aa02c;fill-opacity:1;stroke:#000000;stroke-width:1.0853442;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:0.96078431;stroke-dasharray:none" />
<path
d="m 430.40553,616.1086 -23.94542,12.15358 9.28196,0 0,17.3203 -44.58954,0 c -4.00855,0 -7.23565,3.2271 -7.23565,7.23562 l 0,47.55165 c 0,4.00855 3.2271,7.23562 7.23565,7.23562 l 131.41708,0 0,17.24116 -9.28195,0 23.94542,12.14228 24.29586,-12.14228 -9.28196,0 0,-17.24116 44.60086,0 c 4.00855,0 7.23562,-3.2271 7.23566,-7.23562 l 0,-47.55165 c 0,-4.00852 -3.22711,-7.23562 -7.23566,-7.23562 l -131.42838,0 0,-17.3203 9.28196,0 -24.29589,-12.15358 z"
id="rect2385"
style="fill:#2c5aa0;fill-opacity:1;stroke:#000000;stroke-width:1.0853442;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:0.96078431;stroke-dasharray:none" />
</g>
<g
transform="translate(-80,0)"
id="g2574">
<g
id="g2568">
<path
d="m 430.40553,710.17177 -23.94542,12.15358 9.28196,0 0,17.3203 -44.58954,0 c -4.00855,0 -7.23565,3.22707 -7.23565,7.23562 l 0,47.55165 c 0,4.00852 3.2271,7.23562 7.23565,7.23562 l 205.69531,0 c 4.00855,0 7.23562,-3.2271 7.23566,-7.23562 l 0,-47.55165 c 0,-4.00855 -3.22711,-7.23562 -7.23566,-7.23562 l -131.42838,0 0,-17.3203 9.28196,0 -24.29589,-12.15358 z"
id="path3249"
style="fill:url(#linearGradient2553);fill-opacity:1;stroke:#000000;stroke-width:1.0853442;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:0.96078431;stroke-dasharray:none" />
<path
d="m 517.59484,548.42154 23.94539,-12.15357 -9.28192,0 0,-17.3203 44.58953,0 c 4.00855,0 7.23566,-3.22707 7.23566,-7.23562 l 0,-47.55165 c 0,-4.00855 -3.22711,-7.23562 -7.23566,-7.23562 l -205.69531,0 c -4.00855,0 -7.23565,3.2271 -7.23565,7.23562 l 0,47.55165 c 0,4.00855 3.2271,7.23562 7.23565,7.23562 l 131.42838,0 0,17.3203 -9.28196,0 24.29589,12.15357 z"
id="path3259"
style="fill:url(#linearGradient2545);fill-opacity:1;stroke:#000000;stroke-width:1.0853442;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:0.96078431;stroke-dasharray:none" />
<path
d="m 430.40553,522.04543 -23.94542,12.15361 9.28196,0 0,17.32027 -44.58954,0 c -4.00855,0 -7.23565,3.2271 -7.23565,7.23562 l 0,47.55165 c 0,4.00855 3.2271,7.23562 7.23565,7.23562 l 131.41708,0 0,17.24116 -9.28195,0 23.94542,12.14228 24.29586,-12.14228 -9.28196,0 0,-17.24116 44.60086,0 c 4.00855,0 7.23562,-3.2271 7.23566,-7.23562 l 0,-47.55165 c 0,-4.00852 -3.22711,-7.23562 -7.23566,-7.23562 l -131.42838,0 0,-17.32027 9.28196,0 -24.29589,-12.15361 z"
id="path3263"
style="fill:url(#linearGradient2537);fill-opacity:1;stroke:#000000;stroke-width:1.0853442;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:0.96078431;stroke-dasharray:none" />
<path
d="m 430.40553,616.1086 -23.94542,12.15358 9.28196,0 0,17.3203 -44.58954,0 c -4.00855,0 -7.23565,3.2271 -7.23565,7.23562 l 0,47.55165 c 0,4.00855 3.2271,7.23562 7.23565,7.23562 l 131.41708,0 0,17.24116 -9.28195,0 23.94542,12.14228 24.29586,-12.14228 -9.28196,0 0,-17.24116 44.60086,0 c 4.00855,0 7.23562,-3.2271 7.23566,-7.23562 l 0,-47.55165 c 0,-4.00852 -3.22711,-7.23562 -7.23566,-7.23562 l -131.42838,0 0,-17.3203 9.28196,0 -24.29589,-12.15358 z"
id="path3273"
style="fill:url(#linearGradient2529);fill-opacity:1;stroke:#000000;stroke-width:1.0853442;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:0.96078431;stroke-dasharray:none" />
</g>
<g
id="g2558">
<text
x="403.91736"
y="684.25793"
id="text2395"
xml:space="preserve"
style="font-size:20px;font-style:normal;font-weight:normal;fill:#ffffff;fill-opacity:1;stroke:none;font-family:Liberation Sans"><tspan
x="403.91736"
y="684.25793"
id="tspan2397">Betriebssystem</tspan></text>
<text
x="472.45721"
y="496.13147"
id="text3225"
xml:space="preserve"
style="font-size:20px;font-style:normal;font-weight:normal;text-align:center;text-anchor:middle;fill:#ffffff;fill-opacity:1;stroke:none;font-family:Liberation Sans"><tspan
x="472.45721"
y="496.13147"
id="tspan3227">Benutzer</tspan></text>
<text
x="473.96112"
y="590.19464"
id="text2403"
xml:space="preserve"
style="font-size:20px;font-style:normal;font-weight:normal;text-align:center;text-anchor:middle;fill:#ffffff;fill-opacity:1;stroke:none;font-family:Liberation Sans"><tspan
x="473.96112"
y="590.19464"
id="tspan2405">Anwendung</tspan></text>
<text
x="430.23065"
y="778.32098"
id="text2391"
xml:space="preserve"
style="font-size:20px;font-style:normal;font-weight:normal;fill:#ffffff;fill-opacity:1;stroke:none;font-family:Liberation Sans"><tspan
x="430.23065"
y="778.32098"
id="tspan2393">Hardware</tspan></text>
</g>
</g>
</g>
</g>
</svg>
src/os-layers-5.jpg

53.2 KiB

This diff is collapsed.
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment