Wed Blender: Stereo and Computational Photography / Videography for Cultural Preservation

[Stephen Guerin]

/* Wed night. Thanks to Steve Smith for organizing! */

TITLE: Stereo & Computational Photography for Cultural Preservation

LOCATION: SF Complex, 624 Agua Fria, Santa Fe

DATE:  Wednesday 3/26  6:00 PM - 8:00 PM

You are invited to a kickoff Wed Blender, an informal evening of
presentation/discussion surrounding the application of Stereo and Computational
Photography/Videography. Jason Ordaz of the School of Advanced Research will
present some of his work and techniques in stereo photography for Cultural
Preservation.

We invite related short presentations (10 minutes?) from anyone/everyone with
something to project.

1) Maxwell Museum project - Confirmed
    Catharine Baudoin - Maxwell -  http://www.unm.edu/~maxwell/
    Tim Thomas - UNM HPC  http://www.hpc.unm.edu Confirmed
    Lakshman Prasad  - LANL ISIS project

2) LAVA3D/WorldScape - Confirmed
    Steve Smith http://www.lava3d.net Confirmed
    Dave Modl  LAVA/WorldScape Confirmed
    Rick Kirk   LAVA
    Pete Rogina http://www.wscapeinc.com Confirmed

3) School of Advanced (formerly American) Research - Confirmed
    Jason Ordaz - http://southwestcrossroads.org/ Confirmed
    ???

4) LANLViz/ISIS/Genie - http://isis.lanl.gov/
    Lakshman Prasad
    Kim Edlund
    Reid Porter
    Laura Monroe Confirmed
    Birch Hayes
    Paul Weber
    Dave Hite
    Bob Gislason

5) Santa Fe complex  - http://www.santafecomplex.org
    Stephen Guerin http://www.redfish.com Confirmed
    Shawn Barr Confirmed
    Simon Mihalek Confirmed
    Don Begley Confirmed
    Ed Angel http://artslab.unm.edu
    Roy Wroth
    Steve Smith - http://www.lava3d.net Confirmed
    ???

6) State
    Lea Harris
    Mimi Roberts
    ???
   

[Phil Henshaw-2]

The question is about when there are lots of uncontested resources at first
vs. when things have to switch to negotiating the use of contested
resources.   In the latter case users can eek out a fraction more by
learning to coordinate their independent complex systems, or just do time
sharing, or do some improbable transformative synergy to move the problem to
another scale.   In the former case unlimited resources and no negotiation
means life is simple.

Is it possible that observing a blow-up of complexity might signal that
independent parts of a system might be running into each other after
exhausting the free resources available to both?   It seems that being
forced to negotiate with each other over contested ones might do that.   Is
there a computation analog?




Phil Henshaw                       ????.?? ? `?.????
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
680 Ft. Washington Ave
NY NY 10040                      
tel: [phone]                
e-mail: sy at synapse9.com          
explorations: www.synapse9.com    
-- "it's not finding what people say interesting, but finding what's
interesting in what they say" --

[Marcus G. Daniels]

Phil Henshaw wrote:
> The question is about when there are lots of uncontested resources at first
> vs. when things have to switch to negotiating the use of contested
> resources.   In the latter case users can eek out a fraction more by
> learning to coordinate their independent complex systems, or just do time
> sharing, or do some improbable transformative synergy to move the problem to
> another scale.   In the former case unlimited resources and no negotiation
> means life is simple.
Think of each operating system as a line at the grocery store.   Even if
one of the checkers is slow or a customer can't find her wallet in her
purse, or there is someone buying booze that needs an approval from a
manager, there can be another queue without that problem.   That doesn't
necessarily help any given individual who's already committed to a line,
but in aggregate it does help everyone to have more lines.  There's also
the possibility of super-linear speedups (or synergies).  For example,
cash-only lines.

Marcus

[Phil Henshaw-2]

So a bus, in functional terms, is a 'resource' that never runs into limits
of the kind where users are forced to learn about each other's complex needs
in order to figure our how to get the last little drop of capacity out?
That is, leaving aside the transformational 'synergy' of having everyone in
line fall in love and forget about their shopping... among the other kinds
of choices I had in mind.  :-)

Phil

> -----Original Message-----
> From: friam-bounces at redfish.com [mailto:friam-bounces at redfish.com] On
> Behalf Of Marcus G. Daniels
> Sent: Thursday, March 27, 2008 1:01 AM
> To: The Friday Morning Applied Complexity Coffee Group
> Cc: 'Diegert, Carl F'
> Subject: Re: [FRIAM] can you have 4 operating systems on one buss?
>
> Phil Henshaw wrote:
> > The question is about when there are lots of uncontested resources at
> first
> > vs. when things have to switch to negotiating the use of contested
> > resources.   In the latter case users can eek out a fraction more by
> > learning to coordinate their independent complex systems, or just do
> time
> > sharing, or do some improbable transformative synergy to move the
> problem to
> > another scale.   In the former case unlimited resources and no
> negotiation
> > means life is simple.
> Think of each operating system as a line at the grocery store.   Even
> if
> one of the checkers is slow or a customer can't find her wallet in her
> purse, or there is someone buying booze that needs an approval from a
> manager, there can be another queue without that problem.   That
> doesn't
> necessarily help any given individual who's already committed to a
> line,
> but in aggregate it does help everyone to have more lines.  There's
> also
> the possibility of super-linear speedups (or synergies).  For example,
> cash-only lines.
>
> Marcus
>
> ============================================================
> FRIAM Applied Complexity Group listserv
> Meets Fridays 9a-11:30 at cafe at St. John's College
> lectures, archives, unsubscribe, maps at http://www.friam.org

[Marcus G. Daniels]

Phil Henshaw wrote:
> So a bus, in functional terms, is a 'resource' that never runs into limits
> of the kind where users are forced to learn about each other's complex needs
> in order to figure our how to get the last little drop of capacity out?
> That is, leaving aside the transformational 'synergy' of having everyone in
> line fall in love and forget about their shopping... among the other kinds
> of choices I had in mind.  :-)
>  
A bus is a potential bottleneck.  Typically a bus runs a slower rate
than a processor.   Similarly the things connected to the bus and the
memory are also slower.   So there's a fast resource that can be divided
up in a more sensible way.  Imagine a super-fast Mighty Mouse clerk,
that runs from station to station, operating a hundred times the speed
of a typical customer,.  Such a clerk could soak up all of the latency
introduced by the users of the customer and various sorts of exceptional
conditions.   That's basically analogous to massive hyperthreading on a
computer.

You might be thinking of ways to defeat these grocery store queues.  
For example, what happens when 10 customers come along each with 50
Costco flatbed carts and block all of the lines (supposing there are 10
lines)?    Then everyone else has to wait.   That can be fixed if the
clerks can quickly push off all of a customer's purchases to the side
and start a new checkout on their register.   That 's what
virtualization software/hardware does, or at a higher level, or
system-level checkpointing.    In these everyday cases, I can't see how
it makes any sense to model other users.   The system architecture
should be able to cope.

Anyway, the latter is an example where there can be the perception there
is a limited resource, but really there is not.   Users may not be able
to make good use of peak speed anyway, e.g. having Mighty Mouse as their
individual clerk.

Marcus

[Phil Henshaw-2]

Marcus,
You suggest "You might be thinking of ways to defeat these grocery store
queues", but I'm actually looking for ways to help people recognize when
their systems are running into terminal congestion.  Terminal congestion
strongly appears to be what we're doing to the resources of the earth by
continuing to increase investment in increasingly scattered, smaller and
diminishing pools of opportunity that result in greater side effects.  

There are a lot of people working on the world resource & global warming
conflicts and sustaining our increasingly threatened ecological systems.
They're mostly still quite comfortable believing in our marvelous talent for
inventing increasingly complicated solutions for increasingly complicated
problems.   They're not discussing at all where that's headed though, as if
micro-managing nature was something we knew how to do without question.   If
you look at it straight, though, our increasingly complicated solutions are
consistently producing bigger crises than they solve, with no 'hump' in
sight for the mounting complexity to get over with an extra push of effort.
That's because the symptom is a one directional progression from freedom to
diminishing returns to erupting complexity to conflict, in just about
everything people do, all at once. We're not reading it as a change in our
environment though.

You suggest that when a computer buss is congested it has a self-protection
mechanism that has the authority to put off users requests indefinitely.
That's sort of a central control mechanism for dealing with independent
users that were not smart enough to share the limited resource on their own.
If the independent users were to learn enough about each other's needs they
might learn ways to cooperate and make better use of the limited shared
resource.  For example, they might save up low priority tasks for off peak
times, and so both avoid arbitrary central control and improve the overall
efficiency of the bus as resource.  I guess the question is, does anyone
have a term for the limit of that, the limit of creative collaboration
between independent users sharing a limited resource?  I'd call that the
"edge of chaos" if I didn't know someone else had already used that for
something rather different...  It's the natural line you cross where
independent users exhaust their ability to use a limited resource
cooperatively, that triggers either central control or conflict.

Anything come to mind?


Phil

>
> Phil Henshaw wrote:
> > So a bus, in functional terms, is a 'resource' that never runs into
> limits
> > of the kind where users are forced to learn about each other's
> complex needs
> > in order to figure our how to get the last little drop of capacity
> out?
> > That is, leaving aside the transformational 'synergy' of having
> everyone in
> > line fall in love and forget about their shopping... among the other
> kinds
> > of choices I had in mind.  :-)
> >
> A bus is a potential bottleneck.  Typically a bus runs a slower rate
> than a processor.   Similarly the things connected to the bus and the
> memory are also slower.   So there's a fast resource that can be
> divided
> up in a more sensible way.  Imagine a super-fast Mighty Mouse clerk,
> that runs from station to station, operating a hundred times the speed
> of a typical customer,.  Such a clerk could soak up all of the latency
> introduced by the users of the customer and various sorts of
> exceptional
> conditions.   That's basically analogous to massive hyperthreading on a
> computer.
>
> You might be thinking of ways to defeat these grocery store queues.
> For example, what happens when 10 customers come along each with 50
> Costco flatbed carts and block all of the lines (supposing there are 10
> lines)?    Then everyone else has to wait.   That can be fixed if the
> clerks can quickly push off all of a customer's purchases to the side
> and start a new checkout on their register.   That 's what
> virtualization software/hardware does, or at a higher level, or
> system-level checkpointing.    In these everyday cases, I can't see how
> it makes any sense to model other users.   The system architecture
> should be able to cope.
>
> Anyway, the latter is an example where there can be the perception
> there
> is a limited resource, but really there is not.   Users may not be able
> to make good use of peak speed anyway, e.g. having Mighty Mouse as
> their
> individual clerk.
>
> Marcus
>
> ============================================================
> FRIAM Applied Complexity Group listserv
> Meets Fridays 9a-11:30 at cafe at St. John's College
> lectures, archives, unsubscribe, maps at http://www.friam.org

[Marcus G. Daniels]

Phil Henshaw wrote:
> That's sort of a central control mechanism for dealing with independent
> users that were not smart enough to share the limited resource on their own.
> If the independent users were to learn enough about each other's needs they
> might learn ways to cooperate and make better use of the limited shared
> resource.
It's not that they are not smart enough to figure out what the resource
is and how to share it.   It's that in this case the real failure would
be ongoing haphazard negotiation by users, which is clumsy and poorly
informed and its realization is usually not the primary problem they are
interested in solving.  Better to design an automated load balancing
algorithm and leave that work to a fast and patient computer.   The
identification of general principles of what constitutes fair use (e.g.
equal access to memory and cycles and known turnaround time), is the
social/organizational question, and it's separate from the implementation.

So my question in response to yours, in the context of the subject
line,  was:  "Is there really a resource under contention?"
Or is it just a venue for someone to interleave themselves as a
controller and make themselves more important than they ought to be.  
Lots of people have vested interests in existing inefficiencies, the
management of conflict, and the facilitation of people who would rather
not think.

Marcus

[Roger Critchlow-2]

Back to the original question, and taking "bus" in a more general way,
ethernet has the properties that Phil is looking for:  the resource is
limited, the users allocate and share by each pursuing a local rule, and the
whole thing melts down when it gets overloaded.  The solutions proposed to
solve the melt down, such as token ring and ATM, mostly involve a less
anarchic sharing algorithm.  Yet the most successful solution to the melt
down has been to increase the size of the shared resource.

So the history of shared wire networking, the last 30 years, gives you a
case study in engineering design responding to a particular resource
contention problem and how the economics of it all worked out.

-- rec --

On Thu, Mar 27, 2008 at 10:54 PM, Marcus G. Daniels <marcus at snoutfarm.com>
wrote:

> Phil Henshaw wrote:
> > That's sort of a central control mechanism for dealing with independent
> > users that were not smart enough to share the limited resource on their
> own.
> > If the independent users were to learn enough about each other's needs
> they
> > might learn ways to cooperate and make better use of the limited shared
> > resource.
> It's not that they are not smart enough to figure out what the resource
> is and how to share it.   It's that in this case the real failure would
> be ongoing haphazard negotiation by users, which is clumsy and poorly
> informed and its realization is usually not the primary problem they are
> interested in solving.  Better to design an automated load balancing
> algorithm and leave that work to a fast and patient computer.   The
> identification of general principles of what constitutes fair use (e.g.
> equal access to memory and cycles and known turnaround time), is the
> social/organizational question, and it's separate from the implementation.
>
> So my question in response to yours, in the context of the subject
> line,  was:  "Is there really a resource under contention?"
> Or is it just a venue for someone to interleave themselves as a
> controller and make themselves more important than they ought to be.
> Lots of people have vested interests in existing inefficiencies, the
> management of conflict, and the facilitation of people who would rather
> not think.
>
> Marcus
>
>
> ============================================================
> FRIAM Applied Complexity Group listserv
> Meets Fridays 9a-11:30 at cafe at St. John's College
> lectures, archives, unsubscribe, maps at http://www.friam.org
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://redfish.com/pipermail/friam_redfish.com/attachments/20080328/eb3fab9f/attachment.html 

[Phil Henshaw-2]

Well, the problem with the management solution for natural resource sharing is there's no one in charge, and actually no position from which anyone could be.  So beyond learning creative ways for independent users to share, the only option to avoid unexpected eruptions of conflict would be to have a way of seeing the line of conflict coming.  People seem unaware that there are good long range indicators in the complexity of negotiations for watching that...

Phil.
Sent from my Verizon Wireless BlackBerry

-----Original Message-----
From: "Marcus G. Daniels" <[hidden email]>

Date: Thu, 27 Mar 2008 22:54:51
To:The Friday Morning Applied Complexity Coffee Group <friam at redfish.com>
Cc:"'Diegert, Carl F'" <diegert at sandia.gov>
Subject: Re: [FRIAM] can you have 4 operating systems on one buss?


Phil Henshaw wrote:
> That's sort of a central control mechanism for dealing with independent
> users that were not smart enough to share the limited resource on their own.
> If the independent users were to learn enough about each other's needs they
> might learn ways to cooperate and make better use of the limited shared
> resource.
It's not that they are not smart enough to figure out what the resource
is and how to share it.   It's that in this case the real failure would
be ongoing haphazard negotiation by users, which is clumsy and poorly
informed and its realization is usually not the primary problem they are
interested in solving.  Better to design an automated load balancing
algorithm and leave that work to a fast and patient computer.   The
identification of general principles of what constitutes fair use (e.g.
equal access to memory and cycles and known turnaround time), is the
social/organizational question, and it's separate from the implementation.

So my question in response to yours, in the context of the subject
line,  was:  "Is there really a resource under contention?"
Or is it just a venue for someone to interleave themselves as a
controller and make themselves more important than they ought to be.  
Lots of people have vested interests in existing inefficiencies, the
management of conflict, and the facilitation of people who would rather
not think.

Marcus


============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
lectures, archives, unsubscribe, maps at http://www.friam.org

[Marcus G. Daniels]

Roger Critchlow wrote:
> The solutions proposed to solve the melt down, such as token ring and
> ATM, mostly involve a less anarchic sharing algorithm.  Yet the most
> successful solution to the melt down has been to increase the size of
> the shared resource.
Well, the fix is by in large switching (e.g. stop lights and area
specific transports), not just a single faster shared medium.  
Switching puts an upper limit on the impact of any one user on the network.

Marcus

[Phil Henshaw-2]

Yes, increasing the available resource to relieve conflict has been the norm for centuries.  Now that since nearly anyone's taking of more resources is increasingly robbing and disrupting other users, has sort of become the main source of conflict on earth.. The negotiations are is crossing the line to conflict.  So I figure we need more Earths or more understanding on what's happening and of how to stay out of trouble in our new environment.

That information appears limitless, but is still a function of physical packet flow', means it lives in both worlds, and since info systems have multiple users some of the behavior of open environs for independent systems seem to be displayed.

Phil
Sent from my Verizon Wireless BlackBerry

-----Original Message-----
From: "Roger Critchlow" <[hidden email]>

Date: Fri, 28 Mar 2008 09:55:57
To:"The Friday Morning Applied Complexity Coffee Group" <friam at redfish.com>
Cc:"Diegert, Carl F" <diegert at sandia.gov>
Subject: Re: [FRIAM] can you have 4 operating systems on one buss?


Back to the original question, and taking "bus" in a more general way, ethernet has the properties that Phil is looking for:? the resource is limited, the users allocate and share by each pursuing a local rule, and the whole thing melts down when it gets overloaded.? The solutions proposed to solve the melt down, such as token ring and ATM, mostly involve a less anarchic sharing algorithm.? Yet the most successful solution to the melt down has been to increase the size of the shared resource.?
 
So the history of shared wire networking, the last 30 years, gives you a case study in engineering design responding to a particular resource contention problem and how the economics of it all worked out.

-- rec --
 

On Thu, Mar 27, 2008 at 10:54 PM, Marcus G. Daniels <marcus at snoutfarm.com <mailto:marcus at snoutfarm.com> > wrote:
 
Phil Henshaw wrote:
 > That's sort of a central control mechanism for dealing with independent
 > users that were not smart enough to share the limited resource on their own.
 > If the independent users were to learn enough about each other's needs they
 > might learn ways to cooperate and make better use of the limited shared
 > resource.
 It's not that they are not smart enough to figure out what the resource
 is and how to share it. ? It's that in this case the real failure would
 be ongoing haphazard negotiation by users, which is clumsy and poorly
 informed and its realization is usually not the primary problem they are
 interested in solving. ?Better to design an automated load balancing
 algorithm and leave that work to a fast and patient computer. ? The
 identification of general principles of what constitutes fair use (e.g.
 equal access to memory and cycles and known turnaround time), is the
 social/organizational question, and it's separate from the implementation.
 
 So my question in response to yours, in the context of the subject
 line, ?was: ?"Is there really a resource under contention?"
 Or is it just a venue for someone to interleave themselves as a
 controller and make themselves more important than they ought to be.
 Lots of people have vested interests in existing inefficiencies, the
 management of conflict, and the facilitation of people who would rather
 not think.
 



 Marcus
 
 
 ============================================================
 FRIAM Applied Complexity Group listserv
 Meets Fridays 9a-11:30 at cafe at St. John's College
 lectures, archives, unsubscribe, maps at http://www.friam.org <http://www.friam.org>
 
 ============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
lectures, archives, unsubscribe, maps at http://www.friam.org

[Marcus G. Daniels]

sy at synapse9.com wrote:
> Now that since nearly anyone's taking of more resources is increasingly robbing and disrupting other users, has sort of become the main source of conflict on earth..
>  
If someone wants to copy a real big amount of stuff from one node of a
cluster to another (there can be tens or thousands of these nodes), the
switch can connect these two nodes.  All other transfers in the system
can be going on without notice of this.   To the extent other people
want to deal with those two nodes, the switch can fairly divide down the
bandwidth between those people.   This will typically be a small
fraction of the total capacity of the system or the network.    
Furthermore, on a typical large cluster, there will be a parallel
filesystem with many independent block devices and very low latency
switches.   If I have 100 nodes all writing at once to 100 different
block devices and there is a effectively a different wire from the node
to the drive, then there is no contention.

If a hundred users all want to do this, with their respective
entitlements, and from different nodes, then at some point you run out
of gas.   But a hundred users rarely if ever all want to do this.   This
is a pretty standard assumption of many kinds of telecommunication systems.

So, neither wire networking nor bus use is usable for your analogy.  
The reason is that these resources can be managed by a secure executive
process that divides up the work fairly.   Systems that don't do this
are non-critical systems.

Marcus

[Phil Henshaw-2]

Marcus,
I think the boundary conditions of the problem include both the variable of
system design and control, and that of the independent behaviors of the
users.  The question is what each of those contributes.  With computer
networks you can't do without both, of course, but you can consider what the
options are for each independent of the other.  Then both may learn to make
a combined system work better.

You say " ...resources can be managed by a secure executive process that
divides up the work fairly.   Systems that don't do this are non-critical
systems."   That is generally true for computer networks.   Playing God and
deciding what is fair is the practical thing in that circumstance, since the
system emerged as a controlled system to start with.  

The network manager might be really 'out to lunch' some times though, and
the users needed to share the resource without that global view and central
control.  What could they accomplish just between themselves, is the
question.   They'd have virtually none of the information the manager uses
and none of the control.   If left to themselves, how would they do it?  

I think they'd develop usage signals of various kinds, that communicate
things like 'here I come' or 'now I'm done'.  That would help optimize the
use of the resource without making the users talk to each other to figure
out and mesh each other's needs.  There are signals that convey these kinds
of messages in natural systems, like usage growth and decay patterns, which
telegraph what will follow in considerable detail if you look into the
derivative rates.  That forecasting ability then allows responses before
conflict arises.  If users did that then everyone could get more out of the
shared resource without dropping chains from overloading the buffers, or
having to talk to each other.

To me there are several things here that are closely analogous to the
problem of optimizing the interaction between users of less well defined
resources.  For independent users of resources in open environments there's
no 'God' person worth their salt as controller.  Not knowing how to do
without that role seems to have become a primary problem for the economies
and all our complex shared uses of the earth in general.

Does that make sense?

Phil

> sy at synapse9.com wrote:
> > Now that since nearly anyone's taking of more resources is
> increasingly robbing and disrupting other users, has sort of become the
> main source of conflict on earth..
> >
> If someone wants to copy a real big amount of stuff from one node of a
> cluster to another (there can be tens or thousands of these nodes), the
> switch can connect these two nodes.  All other transfers in the system
> can be going on without notice of this.   To the extent other people
> want to deal with those two nodes, the switch can fairly divide down the
> bandwidth between those people.   This will typically be a small
> fraction of the total capacity of the system or the network.
> Furthermore, on a typical large cluster, there will be a parallel
> filesystem with many independent block devices and very low latency
> switches.   If I have 100 nodes all writing at once to 100 different
> block devices and there is a effectively a different wire from the node
> to the drive, then there is no contention.
>
> If a hundred users all want to do this, with their respective
> entitlements, and from different nodes, then at some point you run out
> of gas.   But a hundred users rarely if ever all want to do this.
> This is a pretty standard assumption of many kinds of telecommunication
> systems.
>
> So, neither wire networking nor bus use is usable for your analogy.
> The reason is that these resources can be managed by a secure executive
> process that divides up the work fairly.   Systems that don't do this
> are non-critical systems.

[ph]

>
> Marcus
>
>
> ============================================================
> FRIAM Applied Complexity Group listserv
> Meets Fridays 9a-11:30 at cafe at St. John's College
> lectures, archives, unsubscribe, maps at http://www.friam.org

[Marcus G. Daniels]

Phil Henshaw wrote:
> The network manager might be really 'out to lunch' some times though, and
> the users needed to share the resource without that global view and central
> control.  What could they accomplish just between themselves, is the
> question.
That's a reasonable question to think about in some other context, but
it simply doesn't need to apply here.

It is unfortunately the case that many computer systems are not
configured and maintained to the level I am describing.

In these situations, users may be forced to do load balancing in some
semi-negotiated way, but that's because of a Bad Situation, not because
of technical necessity.  Bring the Bad Situation to light and get it fixed.
> They'd have virtually none of the information the manager uses
> and none of the control.   If left to themselves, how would they do it?  
>  
The operating system, queuing system, or network switch firmware can do
that just fine.  

If any of this software is not up to the job, it's not a structural
constraint of the universe, it's just a bug.  Bugs should be fixed.
Network managers usually don't fix bugs themselves, but instead decide
to change vendors.  That has the same sort of effect.

Marcus

[Phil Henshaw-2]

Marcus,

I guess I'm not being clear.  I'm trying to compare the use central managed
solutions and user negotiated solutions in this fairly simple problem to
develop a way of discussing the more complicated situations where efficient
and fair central resource management is not possible.  For lots of things
central control is going to work well and be naturally more efficient.  

Without a central operator different users connected to a bus would need
some way of telling what the load on it in the near future would be, in
order to be ready to use it when it wasn't busy.   Would there be any way
for users to sense that other than to sense increase or decrease in
electrical load on the bus somehow?  In open systems the usual way to tell
if something else is using a common resource is finding disturbances around
it, and signs of depletion in what's available for yourself.  Bees might
skip flowers that have been recently visited, for example.  

Phil

>
> Phil Henshaw wrote:
> > The network manager might be really 'out to lunch' some times though,
> and
> > the users needed to share the resource without that global view and
> central
> > control.  What could they accomplish just between themselves, is the
> > question.
> That's a reasonable question to think about in some other context, but
> it simply doesn't need to apply here.
>
> It is unfortunately the case that many computer systems are not
> configured and maintained to the level I am describing.
>
> In these situations, users may be forced to do load balancing in some
> semi-negotiated way, but that's because of a Bad Situation, not because
> of technical necessity.  Bring the Bad Situation to light and get it
> fixed.
> > They'd have virtually none of the information the manager uses
> > and none of the control.   If left to themselves, how would they do
> it?
> >
> The operating system, queuing system, or network switch firmware can do
> that just fine.
>
> If any of this software is not up to the job, it's not a structural
> constraint of the universe, it's just a bug.  Bugs should be fixed.
> Network managers usually don't fix bugs themselves, but instead decide
> to change vendors.  That has the same sort of effect.
>
> Marcus
>
> ============================================================
> FRIAM Applied Complexity Group listserv
> Meets Fridays 9a-11:30 at cafe at St. John's College
> lectures, archives, unsubscribe, maps at http://www.friam.org

[Hugh Trenchard]

I might as well throw this example into the fray, which may cover a few of
your bases, Phil, though I'll happily stand corrected if they are not on
target.

The only complex system I can claim any sort of
slightly-more-than-superficial understanding is that of bicycle pelotons.
As I've mentioned in previous posts, a bicycle peloton is a group of
cyclists who ride within drafting range of each other (except for the riders
facing the wind), who thereby reduce their energy output by drafting. A
peloton is a very good example of resource optimization, since it easily
demonstrated that a peloton can travel faster and farther than an individual
cyclist on his or her own.

In high-level bicycle races, the range between the riders' ability is fairly
narrow (I've compiled some figures which show the range to be about 17
percent).  The range is narrowed further by drafting, and I've also compiled
figures which show that the range is narrowed to an average of about 4%
between first and last place finishers in pelotons (as compared to 17%
between first and last place finishers in individual time trials, which is
where the first figure of 17% above comes from), and there are frequent race
situations where an entire peloton finishes with the same finishing time.

In any event, if I understand your original inquiry, a peloton is a good
example of the kinds of self-organized resource sharing you are talking
about.  When cyclists set off at the beginning of a race, there is a period
when the speeds are low enough when they have no need to draft one another
to feel comfortable in any position in the peloton and are not expending
energy close to maximum capacity.  However, as speeds increase, a transition
occurs (I argue this is a true phase transition) whereby resource sharing
becomes necessary as cyclists are either in drafting positions or at the
front (most are drafting).

In this phase, a balancing occurs between energy expenditure and optimal
position within the peloton.  Because it is a competitive situation, it is
better to be positioned as close to the front as possible.  As this is a
continuous imperative, rotational movements occur within the peloton, where
riders are moving up and down the peloton, or are caught in "eddies" whereby
they advance for relatively short distances within the peloton, before begin
shifted backward again, and then attempt to move forward again.  These
movements occur while riders attempt to use as little energy as possible to
advance.  So, where there are riders who shift to the outside of the pack
(facing the wind by doing so), other riders will follow in their draft.
This results in a pattern whereby riders advance up the sides for relatively
long stretches, while riders drop back within the peloton, and while within
the peloton there are these smaller-scale eddies.

Another phase transition occurs when the pace shifts up beyond another
threshold, whereby the speeds are too high for there to be continuous
rotational movement within the peloton, and the peloton stretches into a
single line.  This phase, while easily observable, is a precurser to a final
transition where the peloton begins to splinter: individual riders fall off
the back, or separations occur in the line of riders which following riders
cannot bridge, and the peloton splinters.

This last phase is an example of the transition to "conflict" which you were
referring to, if I understand it correctly.  In this situation, every rider
is either in direct competition with the each other, or small groups form
which cooperate internally, but each of which are also in direct conflict as
chasing groups want to reintegrate groups ahead, while groups ahead want to
stay ahead of those behind.

Does this sound a bit like the kind of resource sharing states you were
talking about?

Hugh Trenchard


----- Original Message -----
From: "Phil Henshaw" <[hidden email]>
To: "'The Friday Morning Applied Complexity Coffee Group'"
<friam at redfish.com>
Cc: "'Diegert, Carl F'" <diegert at sandia.gov>
Sent: Saturday, March 29, 2008 9:02 AM
Subject: Re: [FRIAM] can you have 4 operating systems on one buss?

> Marcus,
> I think the boundary conditions of the problem include both the variable
> of
> system design and control, and that of the independent behaviors of the
> users.  The question is what each of those contributes.  With computer
> networks you can't do without both, of course, but you can consider what
> the
> options are for each independent of the other.  Then both may learn to
> make
> a combined system work better.
>
> You say " ...resources can be managed by a secure executive process that
> divides up the work fairly.   Systems that don't do this are non-critical
> systems."   That is generally true for computer networks.   Playing God
> and
> deciding what is fair is the practical thing in that circumstance, since
> the
> system emerged as a controlled system to start with.
>
> The network manager might be really 'out to lunch' some times though, and
> the users needed to share the resource without that global view and
> central
> control.  What could they accomplish just between themselves, is the
> question.   They'd have virtually none of the information the manager uses
> and none of the control.   If left to themselves, how would they do it?
>
> I think they'd develop usage signals of various kinds, that communicate
> things like 'here I come' or 'now I'm done'.  That would help optimize the
> use of the resource without making the users talk to each other to figure
> out and mesh each other's needs.  There are signals that convey these
> kinds
> of messages in natural systems, like usage growth and decay patterns,
> which
> telegraph what will follow in considerable detail if you look into the
> derivative rates.  That forecasting ability then allows responses before
> conflict arises.  If users did that then everyone could get more out of
> the
> shared resource without dropping chains from overloading the buffers, or
> having to talk to each other.
>
> To me there are several things here that are closely analogous to the
> problem of optimizing the interaction between users of less well defined
> resources.  For independent users of resources in open environments
> there's
> no 'God' person worth their salt as controller.  Not knowing how to do
> without that role seems to have become a primary problem for the economies
> and all our complex shared uses of the earth in general.
>
> Does that make sense?
>
> Phil
>
>> sy at synapse9.com wrote:
>> > Now that since nearly anyone's taking of more resources is
>> increasingly robbing and disrupting other users, has sort of become the
>> main source of conflict on earth..
>> >
>> If someone wants to copy a real big amount of stuff from one node of a
>> cluster to another (there can be tens or thousands of these nodes), the
>> switch can connect these two nodes.  All other transfers in the system
>> can be going on without notice of this.   To the extent other people
>> want to deal with those two nodes, the switch can fairly divide down the
>> bandwidth between those people.   This will typically be a small
>> fraction of the total capacity of the system or the network.
>> Furthermore, on a typical large cluster, there will be a parallel
>> filesystem with many independent block devices and very low latency
>> switches.   If I have 100 nodes all writing at once to 100 different
>> block devices and there is a effectively a different wire from the node
>> to the drive, then there is no contention.
>>
>> If a hundred users all want to do this, with their respective
>> entitlements, and from different nodes, then at some point you run out
>> of gas.   But a hundred users rarely if ever all want to do this.
>> This is a pretty standard assumption of many kinds of telecommunication
>> systems.
>>
>> So, neither wire networking nor bus use is usable for your analogy.
>> The reason is that these resources can be managed by a secure executive
>> process that divides up the work fairly.   Systems that don't do this
>> are non-critical systems.
>
> [ph]
>
>>
>> Marcus
>>
>>
>> ============================================================
>> FRIAM Applied Complexity Group listserv
>> Meets Fridays 9a-11:30 at cafe at St. John's College
>> lectures, archives, unsubscribe, maps at http://www.friam.org
>
>
>
>
> ============================================================
> FRIAM Applied Complexity Group listserv
> Meets Fridays 9a-11:30 at cafe at St. John's College
> lectures, archives, unsubscribe, maps at http://www.friam.org 

[Marcus G. Daniels]

Phil Henshaw wrote:
> I'm trying to compare the use central managed
> solutions and user negotiated solutions in this fairly simple problem to
> develop a way of discussing the more complicated situations where efficient
> and fair central resource management is not possible.  For lots of things
> central control is going to work well and be naturally more efficient.  
>  
The "user negotiated solutions" reduce to the question of a shared values.
When shared values can be identified amongst N people, then conceptually
we can replace those N people with a single person that plans a larger
array of work and then again it's a question of scheduling, load
balancing, and optimization.  Normally, though, people know what they
want and are competing to get as much of it as possible.

When there are no shared values, than all that can be done is to
dynamically divide the resource into a virtual resource and use the
strengths of the resource to make up for the weaknesses in the
resource.    It's a design question, whose solution may be central or
distributed in nature but still algorithmic.   Virtualization can
prevent hogging, although the resources will divide in power as more and
more users draw upon it.  In contrast, a political solution requires
trust (or at least policing).  Without trust, there will be
instabilities created when people pretend to have consensus in order to
get preferred access and then soon defect on one another when it
actually comes time to use the thing.
> Without a central operator different users connected to a bus would need
> some way of telling what the load on it in the near future would be, in
> order to be ready to use it when it wasn't busy.
Sure, it's called a scheduler.  For schedulers like are in the kernel of
an average Windows/MacOS X/Linux system one resource is made to look
like many, and during the period that a user sees their resource, there
will tend to be minimal contention for resources, although waste may
still occur due to mismatches in latency between the different
components in the system (as would also occur in the non-virtualized
case).    There are some costs to time slicing, in particular that CPU
caches have to be invalidated on context switches, but the idea is to
run long tasks enough to amortize these costs.

For large scale compute environments, this is taken further to have a
high level scheduler that operates on the time frame of days to
months.    Jobs run through such a system get the full machine for a
long period of time without any friction.  The policies for such a
system are to some extent a subject for negotiation.  In academic
environments, it can be a matter of peer review and politics, i.e.
people write a grants to get access to the queues.  In commercial
environments, scheduling can be market driven, or run at a fixed rate
for CPU time / hour.
>   Would there be any way
> for users to sense that other than to sense increase or decrease in
> electrical load on the bus somehow?  In open systems the usual way to tell
> if something else is using a common resource is finding disturbances around
> it, and signs of depletion in what's available for yourself.  Bees might
> skip flowers that have been recently visited, for example.  
>  
More in the philosophy of ethernet or software transactional memory --
wait for a conflict to occur and then retry..

Marcus

[Phil Henshaw-2]

Hugh,

Yes, that example of the breaking point of the peloton does sound like the
limit of negotiated cooperation for the individual cyclists.  Here everyone
expects the usefulness of the peloton to be abandoned entirely at some
point, and choosing just when to break from it is probably a critical
individual decision.  It's an expected 'line of conflict' determined by when
the individual riders break free from the shelter of the group.

Here the common resource is the relative air-pocket formed by the group, and
the regularity of alternating positions within it.  Maybe that would be
analogous to users sharing a bus and having negotiated some regular habit of
coordinating their uses of it.  An established pattern of sharing is one of
the kinds of independent natural systems I focus on.  Once established some
change could cause it to fall apart and then need to be completely
re-negotiated.  

The canonical example is of a resource that begins with having no limit for
a small community of users with various cooperative habits for exploiting
it.  If their habits constitute a growth system, the users will usually know
only their own individual experience and have no experiential information
about the approach of that limit.  It's not clear what their best source of
information would be about it, or how they would choose what to do at the
limits.  

What kind of information might indicate the approach of common resource
limits?  How would that be different from evidence that other users are
breaking their agreements?   As independent users of natural resources tend
to have less information about, or interest in, each other's particular
needs than, say, cyclists in a peloton, how would they begin to renegotiate
their common habits when circumstances require it?  

Phil


-----Original Message-----
From: Hugh Trenchard [mailto:[hidden email]]
Sent: Saturday, March 29, 2008 6:09 PM
To: sy at synapse9.com; The Friday Morning Applied Complexity Coffee Group
Subject: Re: [FRIAM] can you have 4 operating systems on one buss?

I might as well throw this example into the fray, which may cover a few of
your bases, Phil, though I'll happily stand corrected if they are not on
target.

The only complex system I can claim any sort of
slightly-more-than-superficial understanding is that of bicycle pelotons.
As I've mentioned in previous posts, a bicycle peloton is a group of
cyclists who ride within drafting range of each other (except for the riders

facing the wind), who thereby reduce their energy output by drafting. A
peloton is a very good example of resource optimization, since it easily
demonstrated that a peloton can travel faster and farther than an individual

cyclist on his or her own.

In high-level bicycle races, the range between the riders' ability is fairly

narrow (I've compiled some figures which show the range to be about 17
percent).  The range is narrowed further by drafting, and I've also compiled

figures which show that the range is narrowed to an average of about 4%
between first and last place finishers in pelotons (as compared to 17%
between first and last place finishers in individual time trials, which is
where the first figure of 17% above comes from), and there are frequent race

situations where an entire peloton finishes with the same finishing time.

In any event, if I understand your original inquiry, a peloton is a good
example of the kinds of self-organized resource sharing you are talking
about.  When cyclists set off at the beginning of a race, there is a period
when the speeds are low enough when they have no need to draft one another
to feel comfortable in any position in the peloton and are not expending
energy close to maximum capacity.  However, as speeds increase, a transition

occurs (I argue this is a true phase transition) whereby resource sharing
becomes necessary as cyclists are either in drafting positions or at the
front (most are drafting).

In this phase, a balancing occurs between energy expenditure and optimal
position within the peloton.  Because it is a competitive situation, it is
better to be positioned as close to the front as possible.  As this is a
continuous imperative, rotational movements occur within the peloton, where
riders are moving up and down the peloton, or are caught in "eddies" whereby

they advance for relatively short distances within the peloton, before begin

shifted backward again, and then attempt to move forward again.  These
movements occur while riders attempt to use as little energy as possible to
advance.  So, where there are riders who shift to the outside of the pack
(facing the wind by doing so), other riders will follow in their draft.
This results in a pattern whereby riders advance up the sides for relatively

long stretches, while riders drop back within the peloton, and while within
the peloton there are these smaller-scale eddies.

Another phase transition occurs when the pace shifts up beyond another
threshold, whereby the speeds are too high for there to be continuous
rotational movement within the peloton, and the peloton stretches into a
single line.  This phase, while easily observable, is a precurser to a final

transition where the peloton begins to splinter: individual riders fall off
the back, or separations occur in the line of riders which following riders
cannot bridge, and the peloton splinters.

This last phase is an example of the transition to "conflict" which you were

referring to, if I understand it correctly.  In this situation, every rider
is either in direct competition with the each other, or small groups form
which cooperate internally, but each of which are also in direct conflict as

chasing groups want to reintegrate groups ahead, while groups ahead want to
stay ahead of those behind.

Does this sound a bit like the kind of resource sharing states you were
talking about?

Hugh Trenchard


----- Original Message -----
From: "Phil Henshaw" <[hidden email]>
To: "'The Friday Morning Applied Complexity Coffee Group'"
<friam at redfish.com>
Cc: "'Diegert, Carl F'" <diegert at sandia.gov>
Sent: Saturday, March 29, 2008 9:02 AM
Subject: Re: [FRIAM] can you have 4 operating systems on one buss?

> Marcus,
> I think the boundary conditions of the problem include both the variable
> of
> system design and control, and that of the independent behaviors of the
> users.  The question is what each of those contributes.  With computer
> networks you can't do without both, of course, but you can consider what
> the
> options are for each independent of the other.  Then both may learn to
> make
> a combined system work better.
>
> You say " ...resources can be managed by a secure executive process that
> divides up the work fairly.   Systems that don't do this are non-critical
> systems."   That is generally true for computer networks.   Playing God
> and
> deciding what is fair is the practical thing in that circumstance, since
> the
> system emerged as a controlled system to start with.
>
> The network manager might be really 'out to lunch' some times though, and
> the users needed to share the resource without that global view and
> central
> control.  What could they accomplish just between themselves, is the
> question.   They'd have virtually none of the information the manager uses
> and none of the control.   If left to themselves, how would they do it?
>
> I think they'd develop usage signals of various kinds, that communicate
> things like 'here I come' or 'now I'm done'.  That would help optimize the
> use of the resource without making the users talk to each other to figure
> out and mesh each other's needs.  There are signals that convey these
> kinds
> of messages in natural systems, like usage growth and decay patterns,
> which
> telegraph what will follow in considerable detail if you look into the
> derivative rates.  That forecasting ability then allows responses before
> conflict arises.  If users did that then everyone could get more out of
> the
> shared resource without dropping chains from overloading the buffers, or
> having to talk to each other.
>
> To me there are several things here that are closely analogous to the
> problem of optimizing the interaction between users of less well defined
> resources.  For independent users of resources in open environments
> there's
> no 'God' person worth their salt as controller.  Not knowing how to do
> without that role seems to have become a primary problem for the economies
> and all our complex shared uses of the earth in general.
>
> Does that make sense?
>
> Phil
>
>> sy at synapse9.com wrote:
>> > Now that since nearly anyone's taking of more resources is
>> increasingly robbing and disrupting other users, has sort of become the
>> main source of conflict on earth..
>> >
>> If someone wants to copy a real big amount of stuff from one node of a
>> cluster to another (there can be tens or thousands of these nodes), the
>> switch can connect these two nodes.  All other transfers in the system
>> can be going on without notice of this.   To the extent other people
>> want to deal with those two nodes, the switch can fairly divide down the
>> bandwidth between those people.   This will typically be a small
>> fraction of the total capacity of the system or the network.
>> Furthermore, on a typical large cluster, there will be a parallel
>> filesystem with many independent block devices and very low latency
>> switches.   If I have 100 nodes all writing at once to 100 different
>> block devices and there is a effectively a different wire from the node
>> to the drive, then there is no contention.
>>
>> If a hundred users all want to do this, with their respective
>> entitlements, and from different nodes, then at some point you run out
>> of gas.   But a hundred users rarely if ever all want to do this.
>> This is a pretty standard assumption of many kinds of telecommunication
>> systems.
>>
>> So, neither wire networking nor bus use is usable for your analogy.
>> The reason is that these resources can be managed by a secure executive
>> process that divides up the work fairly.   Systems that don't do this
>> are non-critical systems.
>
> [ph]
>
>>
>> Marcus
>>
>>
>> ============================================================
>> FRIAM Applied Complexity Group listserv
>> Meets Fridays 9a-11:30 at cafe at St. John's College
>> lectures, archives, unsubscribe, maps at http://www.friam.org
>
>
>
>
> ============================================================
> FRIAM Applied Complexity Group listserv
> Meets Fridays 9a-11:30 at cafe at St. John's College
> lectures, archives, unsubscribe, maps at http://www.friam.org 

[Steve Smith]

Folks,

I apologize if I missed this in an earlier part of the thread... these
discussions are so elaborate and rich that I simply find I cannot keep
up with them all front to back.

However... this divergence of discussing bicyclist pelotons which is
segueing into what feels like a discussion of seeking solutions to what
is known as the "Tragedy of the Commons" has gotten my attention.

> The canonical example is of a resource that begins with having no limit for
> a small community of users with various cooperative habits for exploiting
> it.  If their habits constitute a growth system, the users will usually know
> only their own individual experience and have no experiential information
> about the approach of that limit.  It's not clear what their best source of
> information would be about it, or how they would choose what to do at the
> limits.  
>
> What kind of information might indicate the approach of common resource
> limits?  How would that be different from evidence that other users are
> breaking their agreements?   As independent users of natural resources tend
> to have less information about, or interest in, each other's particular
> needs than, say, cyclists in a peloton, how would they begin to renegotiate
> their common habits when circumstances require it?  
>  

As most of us know, there has been a lot of abstract study of this in
Game Theory as well as practical study in economics, political science,
and evolutionary biology.

The bicycle peleton seems to arise fairly directly from "reciprocal
altruism".  While there is some cost to the riders at the head of a
peloton in terms of simple distraction and risk of interference, in
general the only cost they bear is relative to the others who gain an
advantage from an emergent common resource, the air pocket behind them
which is unexploited otherwise.   "reciprocal altruism" is an obvious
response, each member of the peleton being motivated to contribute to
the group as a "windbreaker" in exchange for not being ejected or
ditched from the peleton.  As the end of the race nears, the motivation
to "defect" increases and only those with a shared fate (members of the
same team) are likely to maintain pelotons right up to the last minute.

Phil makes good points about global optimization under local awareness.  
As our actions begin to have longer range consequences and we begin to
exploit a larger commons (global, including earth orbit, Lagrange
points, and the lunar surface soon enough) our awareness of the state of
said commons must be expanded equally.   This also is problematic, as
our awareness must be mediated both technologically and socially (we
must use telescopes, remote sensors, etc. and depend on others to share
their observations and judgements about the condition of the commons).  
When these other devices (mechanical and social) are insinuated between
our perceptual system and the commons in question, we are at risk of
them being miscalibrated and of our innate perceptions not being tuned
to them.  We simply may not understand the implications of what our
instruments are telling us in the first case and in the second case, we
may not trust the agenda of the social constructs between us and what we
are observing (see the long-running arguement over whether climate
change is real or not).

- Steve

[Phil Henshaw-2]

Marcus,
[ph]

>
> Phil Henshaw wrote:
> > I'm trying to compare the use central managed
> > solutions and user negotiated solutions in this fairly simple problem
> to
> > develop a way of discussing the more complicated situations where
> efficient
> > and fair central resource management is not possible.  For lots of
> things
> > central control is going to work well and be naturally more
> efficient.
> >
> The "user negotiated solutions" reduce to the question of a shared values.

[ph] Well, not as I see it.  That would be the central manager's assumption
for setting up rules that are to be made self-consistent, and excluding the
environment's inconsistencies from consideration.  From an inclusive view,
though, the individual users will have different needs, interests,
information and perceptions and that's the general problem.

> When shared values can be identified amongst N people, then
> conceptually
> we can replace those N people with a single person that plans a larger
> array of work and then again it's a question of scheduling, load
> balancing, and optimization.  Normally, though, people know what they
> want and are competing to get as much of it as possible.

[ph] Yes,... when shared values can be identified.  In addition to
individuals not choosing to cooperate as you suggest, there are also lots of
times when the long established shared values become inappropriately agreed
to, because of a change of circumstance.  One present glowing example is the
idea of agricultural resources being 'renewable' and the world's
environmentalists and governments committing themselves to a plan of
increasing non-renewable mining of 'renewables', permanently setting aside
growing areas of the surface of the earth to replace the growing energy uses
that used to be supplied from holes in the earth.  It's then kind of
quintessential mistake that helps us examine the real blind spots in our
thinking about nature.

>
> When there are no shared values, than all that can be done is to
> dynamically divide the resource into a virtual resource and use the
> strengths of the resource to make up for the weaknesses in the
> resource.    It's a design question, whose solution may be central or
> distributed in nature but still algorithmic.   Virtualization can
> prevent hogging, although the resources will divide in power as more
> and more users draw upon it.  

[ph] There are lots of things that virtualization might work for, and that's
a good way of saying it.  It still requires the global "God's eye view" of
things that no one naturally has... though.  Still, there are some things
for which one can set up useful controls based on information sharing if one
has run into a necessity of rationing due to not having seen some new
circumstance coming...  That can be done either with central regulation, or
better with informing free markets about the truth of some long neglected
problem, where knowledge is more out-of-date than usual, so they can catch
up.   An example is the fact that the economies are widely acting to
accelerate the depletion of under priced resources to maintain a growth of
output displays.  That seems to be another kind of quintessential error that
helps us examine the real blind spots in our thinking.  

> In contrast, a political solution requires
> trust (or at least policing).  Without trust, there will be
> instabilities created when people pretend to have consensus in order to
> get preferred access and then soon defect on one another when it
> actually comes time to use the thing.

[ph] yes I agree. I think managing social and economic conflict with
political conflict is nearly a complete waste of time.  Governments also
have a tendency of getting fed up with stalled negotiations and going to war
instead...   I think it would be much better if they devoted their limited
resources to giving people better information and discovering comprehendible
rules that would tend to be self-enforcing.

> > Without a central operator different users connected to a bus would
> need
> > some way of telling what the load on it in the near future would be,
> in
> > order to be ready to use it when it wasn't busy.
> Sure, it's called a scheduler.  For schedulers like are in the kernel
> of
> an average Windows/MacOS X/Linux system one resource is made to look
> like many, and during the period that a user sees their resource, there
> will tend to be minimal contention for resources, although waste may
> still occur due to mismatches in latency between the different
> components in the system (as would also occur in the non-virtualized
> case).    There are some costs to time slicing, in particular that CPU
> caches have to be invalidated on context switches, but the idea is to
> run long tasks enough to amortize these costs.

[ph] The management of users on a controlled resource provides the users
with predictable windows of opportunity for using the resource.  There are
some network schemes I've heard discussed that would let subscription users
have ownership of a certain amount of bandwidth that they could then offer
to the highest bidder.  That makes the resource allocation more liquid and
still not 'managed'.  That's something like the carbon "cap and trade" idea
that seems rather efficient for the big users.  

It still does not answer the basic problem though.  On a network one of the
signals of something wrong, and the system of balances going haywire, is
price wars where none were expected, or the promised bandwidth simply not
materializing.  If you sell a certain amount of bandwidth and don't deliver,
then you have trouble.   If you recall, I pointed out examples of regular
'fishtailing' behavior in the financial markets that is definitely not
supposed to ever occur.  In natural systems it appears a variety of 'signs
of trouble' tend to grow exponentially for long periods of time before whole
systems are bought to a stand-still by them.

> For large scale compute environments, this is taken further to have a
> high level scheduler that operates on the time frame of days to
> months.    Jobs run through such a system get the full machine for a
> long period of time without any friction.  The policies for such a
> system are to some extent a subject for negotiation.  In academic
> environments, it can be a matter of peer review and politics, i.e.
> people write a grants to get access to the queues.  In commercial
> environments, scheduling can be market driven, or run at a fixed rate
> for CPU time / hour.

[ph] It's when core policies turn out to be mistaken that it is hardest to
perceive and adjust them as needed, since the whole negotiation model is
predicated on them.  The world is discovering more and more core policies
that are fundamentally in error though, so we need to start looking for what
we should start looking for.

> >   Would there be any way
> > for users to sense that other than to sense increase or decrease in
> > electrical load on the bus somehow?  In open systems the usual way to
> tell
> > if something else is using a common resource is finding disturbances
> around
> > it, and signs of depletion in what's available for yourself.  Bees
> might
> > skip flowers that have been recently visited, for example.
> >
> More in the philosophy of ethernet or software transactional memory --
> wait for a conflict to occur and then retry..

[ph] Yes, that's the usual solution, that natural systems are not in a big
hurry and seem to work well that way, and can just wait for their usual
opportunity to drift bye.  That's not the trend in the design of the complex
industrial system changes for responding to the growing failures of our
complex industrial systems, though...  

The basic circumstance in which we're operating seems to have changed and we
don't seem to know what kind of information we should be looking for to tell
us what to do.  I think developing measures to indicate where conflict is
blowing up might indicate the location of lines of conflict being crossed
that we were unaware of.   I'm thinking that might be better understood in
terms of the simplified model of users of a controlled system, and help
frame the discussion for the tougher problem for users of uncontrolled
systems.

Phil

>
> Marcus
>
> ============================================================
> FRIAM Applied Complexity Group listserv
> Meets Fridays 9a-11:30 at cafe at St. John's College
> lectures, archives, unsubscribe, maps at http://www.friam.org