..and I notice that a lot of people keep saying, "well, I thought of that a long time ago", and that sort of thing and they keep trying to get recognition, and why bother?
To clarify, I feel that Minsky anticipated Waibel (TDNN), Hopfield (RNN), Hochreiter (LSTM), Herbert (ESN), Et. Al like Newton anticipated Cauchy, Riemann, Weierstrass, Et. Al. That is to say, perhaps if one squints hard enough. I do sympathize with Minsky when he laments the absence of philosophical inquiry during this wet lab era of AI, and I very much enjoy listening to his interviews on YouTube. It would have been a real pleasure to have known him.
I appreciate Glen's comment for orienting the discussion around the phenomena, the networks themselves. It is in this sense that wet lab seems like an apt analogy. Inevitably, it is over these grounds that any meaningful higher-level theory must relate. For instance, linguists ask how is it that children learn from so few examples? Some posit highly specialized and innately given structures (Chomsky) while others look for highly specialized and external social networks (Tomasello). In the field of machine learning something of the same appears to be happening. We are pleased that so much can be encoded informationally in the data, and we look to ways that such information can be encoded as or afforded by the structure in a network. To my mind, the unreasonable effectiveness of data points to a dual quality relative to a kind of impedance matching. That an abundance of errorful and incomplete data sets are better than a few pristine data sets speaks to me of the unreasonable robustness of information, ie., the difficulty of accidentally distilling the informational content away from the data, or thought of another way, being unconcerned that some sample or other will exhibit randomness. Thankfully, richly structured sources produce rich information, and this seems especially so for natural language. Thanks for keeping this ball rolling :)
| Free forum by Nabble | Edit this page |