order and disorder

[Phil Henshaw-2]

Steve

>
> >>   This would be why, for example, that jpeg's
> >> simple cosine-model for "predicting" bits in a string
> >> (along a line
> >> of an image) works so well on certain types of images (3D objects
> >> with shaded surfaces..) where run-length encoding did not.
> >
> > Great! A cosine model is a good useful incorrect universal model of
> > shape in data.  There may be better ones, based on the same implied
> > principle that in regions of continuity a small number of
> > points gives
> > you a simple rule for all the points in-between.  The one I use is
> > even more naturalistic and easier to calculate, the rule
> > that the 2nd
> > or 3rd derivative at a point, or something, is the same approached
> > from either direction...  If anyone knows anyone, I'd like
> > to talk to people
> > interested in generalizing this and the related issues.  
> > My math isn't really strong enough.
> Mine is laced with rust... probably in some of the structural
> areas as well as the usual fenders-n-floorpans... so I sympathize.
>
> So, the point you raise here seems to admit the possibility of
> meta-models.
> I offer cosine, you respond with smooth higher-order derivatives as a
> more general model that is more general...
>
> I suspect that this is somehow very relevant to how we
> implicitely model the world... models of models... simple
> rules that work most of the time which are specific examples
> of more general rules which themselves are deriveable by yet
> larger models of "reality".

Well, I'd agree that's part of what the great advances in theory tells
us, that what we can build tends to have layers of design and
application, just like nature, but different.  It has an obvious source
in trying to make theories that fit nature, of course, like a glove to a
hand, the levels of order in nature show through if you do it right.
But I would still like to be able to generalize the math for my concepts
of shape in finite sequences though...

> >
> > Still, isn't the basic question when to make the jump from
> > recognizing
> > patterns in the data to recognizing things in the world?
> > Yes, this leap is perhaps as subtle as "what is life" or "what is
> > intelligence"
> > or "what is consciousness".
>
> >  Huge steps
> > have been made in pattern screening, fingerprints & text
> > searches and
> > other complicated things.  Isn't the 'holy grail' to do the
> > same for
> > complex systems?  What would you look for?  Continuities
> > and breaks,
> > periods when shapes have higher derivatives all of the same
> > sign, etc.
> Our "modern" acknowledgement that there are qualitatively
> more complex systems than can be described (modeled) by
> linear systems of differential
> equations is a good start.  But as you point out, merely knowing what
> it is
> not, doesn't always tell us what it is.   Isn't this really what the
> whole field
> of nonlinear/complexity science is about?  Developing and
> classifying nonlinear models for describing/predicting the
> structure and behaviour of "complex systems"?

But what that consists of has diverse opinion.  Take my concept of
levels of continuity implicit in sequences.  From years of trying to
talk about it it seems 99.99% of scientists might not consider 'shapes'
of that sort to be related to functions in any way, even if they might
well correspond to the non-linear dynamics of observed phenomena far
better than functions do for most complex and emergent phenomena.

> Said that way, it would seem premature of us to want a "universal
> classifier" this early in the game.   It seems like the best we are
> going to
> get is a little (more) low-hanging fruit... models of classes
> of systems which are relatively not-so-complex in the sense
> that their behaviour and structure is well characterized by
> our models.
I'd agree that the question of complexity is bigger than it was first
thought, and that the several systems theory movements that have bit the
dust could be joined by more...  That's been some of the discussion here
recently, i.e. what do we have to show for it?   Some of the 'low lying
fruit' still looks large to me though, something like apples lying
around the size of mountains.  Wonder where THEY came from?

> >
> > ..'relative entropy' sounds a little like the concept of
> > 'random with
> > respect to' the local pattern discontinuities in organizational
> > hierarchies.  The behavior of materials is exactly the
> > larger scales
> > of the behavior of their molecules.  The question is whether the
> > behavior of the whole arises from individually orderly molecules
> > behaving 'randomly with respect to' the whole.
> I'm not sure if this is the same thing.   I think the comparison here
> might
> be to say that one can develop a statistical model of a stream of
> encoded
> bits that is good at predicting the "average frequency of 1's vs 0's"
> over
> some sliding or striding interval.   Like 0.5 ...      or maybe I'm
> missing
> the point?

I thought you were using 'relative entropy' and depending on which of
the overlapping patterns in an image (say a picture showing a triple
exposure of three scenes plus some noise).  I'm handicapped by only
having the roughest idea what people mean by entropy in data, though.
Is it correct to think of it as the resolution of the (chosen) image?

> >>  whatever the Jabberwocky in the
> >> Slivy Toves
> >> that means!).    And it is not clear if these two basis
> >> spaces truly
> >> "cover"
> >> the same territory.
> >>
> >> My variation on your observation is to note that to learn
> >> a language
> >> fully requires learning the culture of the language fully,
> >> which most
> >> (all by definition?) members of a given culture never even
> >> achieve.
> >> We revere the OED because it gives us first-known uses of
> >> words and
> >> their context and so forth... most of us are amazed half
> >> of the time
> >> when we look up a word, to discover it's (apparent) origins and/or
> >> multiply nuanced uses, etc.
> >
> > Yes, the same idea.  Maybe the most useful word for it is 'nuance',
> > those feint and powerful paths of association.  Not much nuance to
> > data! (unless you read between the lines, of course)
> Aha!  Precisely... and isn't that what data analysis is all about in
> some sense?
> We fit a "model" to the data with the intention of developing
> a way to generate
> the "data between the lines".    I share what I hear to be your
> reverence for
> the human ability to pick out a "foreground image" from a few strokes
> of a
> Sume brush, or a whole metaphorical world from a few stanzas of
> poetry...
> and wish (too?) to be able to formalize (despite my rusted
> out maths) at least some of this in math and computer models
> capable of doing some of what the human
> eye/ear/mind/brain/intuition seems so apt at.

I'm kind of skeptical about all the approaches to modeling thought that
I've heard of.  They just don't have the form of nature.  They have the
form of machines.  Most likely the best work on that is going on quietly
in some hidden place...  I think we see the layer upon layer form in the
output of chaotic equations and fractals and go ape thinking that's how
nature does it.   I don't think that's how nature does it.   There's an
emulation problem for rule based models based on measurable properties.
Nature doesn't seem to use either rules or measures.   My usual
'cop-out' is to turn the models around so that, as crude as they may be,
they point to things in nature rather than try to represent them.  That
turn around associates them with and highlights the perfection of the
original, which can be helpful.

> >> these can suggest some negative or positive space which
> >> can suggest
> >> an area or an object which can suggest higher orders of
> >> objects like
> >> an animal or a vehicle or a person which can suggest a
> >> relationship
> >> or a scene (flight or fight!) or ...
> >
> > Sorting out the 'powers of suggestion' in any data is
> > definitely not
> > easy.  The closest information theory would seem to come is
> > with the
> > algorithms, that I have no real understanding of but can
> > see how well
> > they work, for making up rules of association between patterns and
> > then skimming matches from huge sets of alternates.  The
> > one thing in
> > the natural world that seems to do something similar, by a
> > different means
> > perhaps, is human thought.   I don't think thought is
> > either digital or
> > analog, but the outside appearance is that people have a similar
> > amazing
> > facility at word puzzles as Google has on the web, and
> > neither have a
> > proportionate grasp on other kinds of meaning.  Isn't there
> > something
> > similar in the disproportionate performance levels on very similar
> > tasks?
> Yes, it is fascinating.

yea!

> > There are lots of interpretation tasks neither man or machine seems
> > likely to ever master, but another one that might be mastered by
> > either, by different means perhaps, is reading curves from
> > dots.  It's
> > one of those natural navigation tasks, to read as far ahead on the
> > curves as possible to minimize the steering necessary. It's
> > the core
> > problem for 'homing systems', I think, which the world produces in
> > abundance and variety.  Basic thermostats only respond to the set
> > point crossings, above or below, but could reasonably be
> > engineered to
> > respond to the system's implied thermal mass (past
> > responsiveness) and
> > the rate of approach of the set point(implied energy flux), just
> > reading the dynamics of the curve.
> I (like to) believe that "evolution" has solved these problems myriad
> times
> for myriad organisms with myriad approaches.   It is why I tend to put
> a lot of faith in learning classifier systems of various sorts... it
> seems to
> be a good meta-model of how "biology" learns through
> trial-and-error... and how the same problem can be solved so
> many similar yet different or is it different yet similar ways.

That's an interesting model!

> >> One man's noise is another man's signal!?
> > Well, sure.  A man looks at a basket of apples as something
> > to buy and
> > his son looks at it as something to eat!   That discrepancy
> > might not
> > depend of choosing which source noise to ignore, or it might.   They
> > might both be overlooking the dirt and worm holes the mom
> > would notice
> > right off, giving her a much bigger picture, and leading her to
> > quickly scurry the two boys away from that stall in the market!
> Yes, all that... and more.

> >> This "noise" is hugely signal to meteorologists...
> >
> > As you were mentioning before, there are many kinds and layers of
> > signal
> > and noise.  That's maybe my main objection to what I was
> > taught in data
> > analysis, essentially to treat data as if all the pattern you didn't
> > understand was made by the same universal noise generator.  
> > It ain't
> > so.
> Yes, it does seem bogus.  Yet at some trite level, I suppose it is
> trivially
> accurate.   The "Universe" is a huge, universal signal generator, the
> system/phenomena we are studying is a tiny part of that which
> is both somewhat independent and somewhat coupled to the rest
> of the universe. For the purposes of data analysis, we
> (apparently) presume that it is
> independent and that the rest of the universe is "noise".  

And why do we do that?, postulate afresh every morning that there's a
universal noise generator producing the dominant signal in our data?
You comment below on pealing away layer upon layer to look for something
is clear, takes a little work, but assuming that form in your raw data
would be the more reasonable assumption to make.  Where this comes out
most clearly for me is in interpreting time-series.  The most natural
thing in the world is for large complex systems to have several scales
of fluctuation, reflecting the interaction of divergent processes with
the several layers of homeostasis involved.  Do we look for that??  No,
usually it's just all seen as bloody noise!

> In my experiences
> with data analysis, it is not uncommon, however, to model the
> background
> signal to remove it.   Finding a needle in a haystack by removing hay
> from
> the stack until the remaining hay is (for example) similar in size to
> the needle
> we are looking for.   By removing all of the very large
> stalks and the
> very fine
> dust (treating it as signal to recognize it, so we can discard it)
> first, we reduce
> the problem to a more tractable one.

> I appreciate this discussion if only to remind me of how
> pervasive and subtle this problem is.
Yea, I don't think we're doing our job unless once we're done theorizing
how the world 'really works' it looks more natural than it did before.
Are we on the way??...

> - Steve
>
>