network 'hubs' v. 'hives'

[Steve Smith]

>> I'm not sure how Hubs and Hives connect in your analogy.  
>> Airports are
>> Hubs and cities are Hives... if I understand your meaning.    
>>    
>
> Well, the intense 'hive' of relationships between airline attendants at
> an airline 'hub' also counts, though they appear at different levels of
> the airline network organization.
Would you characterize a "hive" (subsystem) as (nearly) fully connected?
>  A lot of the appearance of things has
> to do with how you aggregate your data.
One point in using a rich multi-graph model is that there is no explicit
aggregation, right?

What I've been "noodling" on is how to interrogate this kind of rich
model in such a way as to
tease out "natural" affinities rather than to impose aggregations based
on assumptions which
may not be accurate or relevant to the system.
>   There probably are also a
> number of more regional airline networks that are a 'hive' of sorts that
> then serve 'hubs' of a larger system, like bush flights in Alaska that
> would normally just 'not count' or for which data would be unavailable
> when drawing the shape of the larger network.
>  
I agree/understand that power-law distributions (at least of
node-valency) occurs within a single
aspect of a multi-graph... While I invoked "transportation network" you
responded with "airline
transportation network".

>
>>> What we seem to have in the scale-free design of natural systems
>>> is also new evidence of how nature operates with lots of  
>>>      
>> 'different
>>    
>>> worlds'.
>>>      
>> You are referring to the emergence of heirarchy in these systems
>> (villages eventually connect into networks of villages/widely
>> distributed rurals cluster to form regional centers/villages?)
>>    
>
> Well, not quite.  The inception of the idea was more that what's
> different between a mean distance in a network of 5 degrees of
> separation between people, and 19 degrees when measuring the distance
> between web pages, seems to indicate that the separation between our
> mental worlds is vastly greater than our separations physically.  The
> marvel is that when we walk down the street we're quite unaware that
> many of the worlds of ideas in other people's minds you'd a need to go
> to vast lengths to translate.  The scale free property that yields a
> high degree of connection for the whole network also yields a high
> degree of isolation/independence of the parts.  
>  

Yes, I was responding to the "scale-free design of natural systems"
rather than trying to
attribute the isolation to geographic proximity.  Naturally the
topological implications
of the system is what we are talking about.  It is awkward that most
good examples of
natural systems exhibiting scale-free topology seem to give rise to this
property in the
context their physical embodiment.   I could be wrong about this, but
that is my experience.

It probably says more about what we are able to consider "good examples"
than the natural
occurence of this property.   I'm interested if you or others have
ideas/opinions/references
around this.
>>>    One opportunity that presents is a way to find the
>>> functional boundaries of independent system parts topologically.
>>>      
In the case of my graph-layout of the dynamical systems model of 17
infrastructures, I did
indeed seem to find that functional boundaries corresponded with
significant shifts in scale.

Naturally a model of the components of a fresh water system, for
example,  are highly connected
to eachother compared to those same components and say components of the
financial system, but
within any of the infrastructures, functional units emerged which were
highly interconnected but
only loosely intra-connected to other subsystems.

This "study" was strictly a sideline to what my task was and did not
have the time or the permission
to study this aspect any further than what I've reported here... pretty
anecdotal at best.

>> Yes, this is where I have been noodling mostly.   The form/function
>> duality.   By noticing the structural decomposition of a
>> "system", one
>> can maybe identify the subsystems within a system of systems.
>>    
>
> I started doing that by identifying a system 'form' as growth and then
> using that as a way to aggregate the locally involved network identified
> with the same process, then locating various working parts needed for
> the complex process.

I agree that structure either emerges from/with growth or atrophy
(neural systems, transportation
networks, etc.)
>    I think that's a great method for some things,
> but a statistical measure, a topology of 'hiveness', might be better for
> the vast databases that are prevalent.
Yes, data mining in the context of scale-free(ish?) networks calls for a
more general/analytic method.
>  I've not been able to assemble a
> network modeling and analysis tool kit for myself yet, for several
> reasons, but I can see natural systems comprise networks of physical
> parts that are diversely cross connected on many levels, forming
> self-organizing cells with extensive local interconnection and sparse
> remote connection.
Is it worth "noodling around" the consequences of natural systems being
naturally embodied
in physical space?   Obviously "action at a distance" properties
transcend this but many systems
involve action/coupling which attenuates with distance.
>   Sometimes network maps wouldn't show the 'hive'
> characteristic of natural systems when mapping 'nodes' as categories of
> things, rather than individual physical ones.  If you map internet nodes
> as 'cities', for example, you'll still get the scale free distribution
> of connections, but no 'hives', because of how the data is aggregated.
>  
I"m struggling with this still.   I still don't maybe appreciate what
you mean specifically by
"how the data is aggregated".
>  
>
> I'm not quite following what the network is composed of, or what
> physical system it is embedded in.
>  
The infrastructure networks represent a logical composition of physical
networks which
are geospatially embedded.  A freshwater model consists of items like
"reservoirs", "wells",
"pumping stations", "Chlorination stations", "Primary, secondary,
tertiary piping systems",
 etc.   The relations/connections between these elements are a
consequence of their physical
embedding, as well as their logical...

  The gene ontology is a logical composition of the functional relations
of genes in many
organisms whose physical instantiation defines things about their
function.  Cell-Cell signaling,
for example, might be dominated by processes which require (or at least
favor) proximity.

>  
>>>  
>>> The information boundaries surrounding self-connected parts of whole
>>> systems also seem to define structural limits for the 'world views'
>>> for things looking out from their insides.
>>>      
>> I'm not sure I know what you mean by "structural limits"...
>> in the case
>> of geospatial distributions of humans, the literal spatial separation
>> between "villages" or "cities" tends to attenuate awareness of the
>> "things" on the other end of the connection.   Illustrating (or
>> contradicting that), Santa Fe has(Had) roads named "old pecos trail",
>> "taos highway", "cerillos road" which were descriptive of the
>> community
>> in which you would find yourself eventually if you took that road.
>>    
>
> When in New York everything is interpreted in relation to the 'hive' of
> New York issues, and similarly when up camping in the mountains.  It's
> one of the great pleasures of 'going places' that you get to switch the
> whole context of your thinking for a while.

yes, this is a very familiar experience, thanks for reminding me!
>    The idea of structural
> limits is that either of these 'places' is defined by it's strongly
> inter-connected connections and that that has an interior and edges.
>  
And to reinforce (I hope) your point, the highly-interconnectedness  
needn't be geospatially
correlated as in this example (one hive in the city and another smaller
one in the camping
experience).   An e-mail list like FRIAM may have a certain amount of
geospatial correlation
but that does not dominate... and we are a bit of a "hive" or "hive of
subhives"...   only some
of which are geospatially correlated (those who actually live in Santa
Fe, or those who make
it to coffee, etc.)
> As we go out to the edge of the domain of any 'hive' of connections, I
> think we most often turn back, since what's beyond looks like nothing,
> since it has no role in the hive.
In David Abram's "Spell of the Sensuous", he identifies a "shaman" as
someone who lives
within a culture but who also maintains an awareness of that which is
outside.   The culture
is "blind" to anything outside of it for the most part, reducing the
complexity members of
the culture must deal with.  The "shaman"provides at least one important
service to his/her
community/culture, and that is to keep track of what appears to be over
the "horizon", being
aware of threats and opportunities outside the "hive".
>  When you're out in the woods the idea
> of 'ordering out Chinese' is very far from reality and in the city
> spitting on your knife and rubbing it on your pants before sitting down
> in the dirt for dinner probably wouldn't occur to you either.   How many
> worlds can a mind switch to?  Maybe a large number, and there's just no
> telling what one in active in anyone else's at any given time.  ;-)
>  
Yes, and I find it fascinating all of the triggers that help me remember
where I am, what "world"
I am in.  Many of us experience dropping back into familiar patterns
with our family of origin
(parents, siblings) or perhaps ex-spouses/lovers which we might very
well have thought we
forgot.

Similary, wearing a particular bit of clothing may cue us that we are
"camping" or "dating" or
"working" or "relaxing"...
> I guess it looks like I'm poking around with the absence of complex
> local mesh like regions with their defining circular connections that
> are so prevalent in nature but not prominent in either the discussion or
> displays of most network maps.
>  
I did find in both the infrastructure networks and gene ontology
"lattices" of interconnection.

>>  trucks from TX, MX, CA, AZ arrive with all sorts of
>> produce, sort
>> it out, bring it to the back door of my market, etc... ?  
>>    
>
> Sure, the 'little world' of the shop may have a very special community
> of issues that buzz around it and occupy all the people that pass
> through as consumers, and you don't see where the salad greens come from
> any more than you see what the truck driver does with the cash that
> somehow gets from your pocket into his.   When you treat the shop as a
> 'node' you get an image of the larger scale system structures, but they
> only function because the shop is also a 'hive'.  At the larger scale
> the chatter between the truckers may become a 'hive' of relations that
> itself could then relate to other things as a 'node'.   'Hives' and
> 'nodes' may often be the same thing, looked at differently.
>  

Yes, and this is why I prefer to model complex networks as
multi-hyper-graphs...  subgraphs
can aggregate to be nodes in another graph, edges can have many
properties (as many as there
are edges in some cases) and a good visual analytic interface should
allow one to view these
networks with different levels of aggregation and with different
relations having different "weights"
in the layout.

>  
> I've been trying to explain my observation that the world views of
> people are often exceptionally different, and yet we remain largely
> unaware of it, mostly ignore it in conversation, and are relatively
> uninterested in the deep communication problem it produces.
>  
>> In this, I hear you saying that people partition the graph of
>> their life
>> differently?  That in the above-described method of
>> aggregation, we use
>> different edge parameters to aggregate what we consider to be part of
>> our "hive" and what is only accessible through another hub?  
>>    
>
> Or that the 'hives' of relationships that we are in the middle of, look
> as if they are the whole world, but are really only complete and
> satisfying within themselves, and not actually the whole world.

I agree and refer again to Abrams "Spell of the Sensous" to support that.
>   Thus
> the potential for illusion, and people living in literally different
> worlds of ideas and both having to make allowances for how idiotic the
> other must surely be.   My goal would be to reduce the problem to the
> one you suggest, at least learn some way to recognize what local world
> I'm speaking from and to, avoiding the complete structural disconnect
> problem so there's just the matter of two perspectives of the same
> thing.
>  
In a "simple" network where the edges/relations are roughly of the same
type, the structural
features we are discussing do seem to explain this...   in a more
complex one, the same features
can be explained by the valuation of the relations.   For example, I
have groups of acquaintances
(hives) which I am not a "part of" in the sense that the reason I
associate with the members is
not the reason they associate with eachother, and they use the myopia of
those values to limit
their scope of interest/discourse while I do not generally.   If I
reduce or shift my interest to
what relates them, I might "become" part of the hive.

>> One person's metric of distance might be "how long does it
>> take me to get there?" while another's might be "how much
>> does it cost
>> me in $$ to get there?" while another might consider "how far
>> is that as
>> the proverbial crow flies?",  and another "how much
>> irritation will I go
>> through on the way there?"   The shift from 55 to 70+ freeway speeds
>> made me newly aware of this... instead of driving to Denver
>> on the back
>> roads (at an easy 60-65 mph) vs the freeway at similar
>> speeds, it is now
>> more efficient in time to drive the (longer) freeway at 75-80 vs the
>> back roads at 60-65 still.   The back roads afford better scenery and
>> more entertaining places to stop... but the freeway affords
>> the use of
>> cruise-control and regularly provided stops with name-brand
>> eateries.  
>> Which is "closer"?
>>    
>
> They all sound like good measures to me, so long as you're using the
> same one.
>  

But that is the point.   We may NOT use the same one and therefore your
hub may be my hive
or vice-versa.  

>
>>>   That our individual understandings of 'the universe' develop in
>>> relation to sub-networks having local information horizons in every
>>> direction, it means every 'hive' looks like the 'whole'.
>>>      
>> I do think I know what you are describing here, that the
>> natural scale
>> of human perception, when used as a theshold to the
>> scale-free networks
>> of relations they inhabit, yield a set of "separate worlds"...
>>    
>
> It's forming an idea that scale free networks allow clusters to have
> both tight internal connections with independent behaviors, while also
> participating in larger systems to which the cluster is connected by
> some group of its parts as hubs.

I think I agree explicitly with you on this.
>  The thought is that observers within
> such a cluster, human or other type of 'exploring system', might
> naturally see the cluster as the whole world just because clusters tend
> to have edges in all directions.
I think I see your point better now, that the subjective experience from
"within" a cluster may
enhance the isolation compared to what it appears from a more objective
view?

>
>> Are you also noticing that different people have different
>> qualitative
>> perceptions (value systems) can live in different worlds
>> whilst sharing
>> the same space (physical and logical)?  This is what I want
>> to attribute
>> to self-organized graph layout using different parameters of
>> the edges.
>>    
> But what sort of data would reflect such things?  
>  

Using the transportation example earlier, assigning spring
length/strength to the
relevant edge metric (travel-time, $$, distance,
number-of-interesting-stops, etc.)
will cause different layouts with (possibly) different hives and hubs.  

Someone (like maybe a retired over-the-road trucker) who likes to drive,
has a lot of time,
and maybe the $$ to buy the gas and is a  real sucker for national parks
may see the US as
several "hives" of national parks while an airline employee with liberty
to fly standby for
free and loves art-galleries sees the world very differently.
>
>> Can you rephrase what you mean by "topological 'illusion'"?  
>>    
>
> I'm just referring to the possible appearance that there is nothing
> beyond the 'hive', because when within a 'hive' of connections you the
> strength of it's connections has boundaries in all directions, sort of
> like a 'niche' within which a fall-off in connections is read as the end
> of the world.
I remember as a child being fascinated when I realized that the two
roads out of town (next
closest town being 100 miles in each direction) in fact went somewhere
(besides "on down the road
to various semi-local destinations of interest) even more interesting
(complex) than the town I lived
in.
>  I also think an observant person might notice secret
> exits from the hive, perhaps stopping to talk to the truck driver and
> learning a little something about the world beyond the vegetable shop,
> but that's making a break from the hive.
>  
My parents were one of the few who had cars and knowledge suitable for
visiting the "big city"
and were therefore often conscripted to take people to the "big city
doctor" or even more
rarely but importantly to "the airport" where the individual in question
might fly halfway
across the country to another airport where they would be picked up by a
relative and driven
to an equally small town where they might be comfortable again (in a
similar but different hive).

>  
>> I think I'm
>> "in the same universe" as you on this but naturally not completely.
>>    
>>>    Perhaps the very 'independence' of our world views is further
>>> evidence of how deeply embedded in a larger system they are.
>>>      
>> Perhaps.   Quantitatively large as well as qualitatively.   Not just
>> large graphs of graphs, but multi-graphs wherein the power-law of
>> valency varies over the "type" of edge being considered.  
>> Everyone is
>> one-degree of separation apart by this e-mail list, but more
>> like 2 or
>> even 3 by personal connection.   I have never met you  and perhaps of
>> the other FRIAMers I have met, none of them have met you
>> either (though
>> I suspect Gueren to be a bit of a hub in this regard).  Our
>> "co-citation" distance is probably at least 3 and probably 4 or more.
>>    
>
> The interesting thing about people is we all have different kinds of
> doors we open to each other at different times, lots seeming to have to
> do with finding links between world views we find are built so
> differently some times.  Some of that is personal matters, and that
> everybody makes up their own, but there's also a part that comes from
> people not being aware of networks and that the world inside scale free
> networks seems it might look like a different world from inside every
> part.  An email forum and it's conventions like this, might look rather
> strange to some, where people just throw all kinds of things out and
> occasionally there's a little fast paced flurry that follows, producing
> not so much more than a great accumulating kind of 'compost' to mull
> over it seems.  
>  

This conversation is certainly providing lots of composting material and
some of the
process that helps the decomposition (aerating, mixing, moistening, etc.)