Friam

Interview with Jeremy Howard

Classic

List

Threaded

9 messages Options

Russ Abbott

Interview with Jeremy Howard

I first heard of Jeremy Howard when he won a Kaggle award. Turns out he's something of a rebel. His current thing is his FastAI work, which he uses to teach Deep Learning to virtually anyone. He doesn't have a PhD, and he resists academic approaches to AI. He is originally from Australia. This is an interview with him on his first visit back in a long time.

-- Russ Abbott
Professor, Computer Science
California State University, Los Angeles

- .... . -..-. . -. -.. -..-. .. ... -..-. .... . .-. .
FRIAM Applied Complexity Group listserv
Zoom Fridays 9:30a-12p Mtn GMT-6 bit.ly/virtualfriam
un/subscribe http://redfish.com/mailman/listinfo/friam_redfish.com
FRIAM-COMIC http://friam-comic.blogspot.com/
archives: http://friam.471366.n2.nabble.com/

jon zingale

Re: Interview with Jeremy Howard

I appreciate Jeremy's spit and elbow grease approach to developing his lab, his youthful heart/naivety, and the emphasis he places on architecture and profiling over analytic bounds. His position mostly focuses on the importance of "getting up, getting out, and getting something" with respect to AI, though something about his enthusiasm and virtue signaling gives me pause. Silicon Valley 2.0 is hyper-obsessed with the ethics of its earlier form, and so there remains something disturbing about white men continuing the pattern of imperialism under the guise of missionary work, a mission to serve the noble savages. This pattern is by no means new and to the extent that his desire to help is as authentically quixotic as he presents, it can likely be remedied with a little self-reflection.

While I continue to hold out for high-level neural network theories, I do very much appreciate the attempts to remove false barriers to entry. One tension I feel, when I take a few steps back, is repeated in the very development of the web and more generally in the euro-centric story of westward expansion. The former conveys the tribulations of a world now burdened with Javascript in which we (in the field) scramble to work out what's next (web assembly?) and determine the meaningful patterns. The latter is the story of opportunity in the wilderness followed by the inevitable harness of law (True Grit). If the goal is authenticity wrt distancing ourselves from Silicon Valley 1.0, I wish to see authentically new narratives and archetypes.

All that said, I am excited about the work Jeremy and his wife are doing and I mostly agree that coding is an essential literacy. --trigger warning-- Even if tomorrow the world's computers were to disappear, we would continue to depend on this literacy.

Sent from the Friam mailing list archive at Nabble.com.

- .... . -..-. . -. -.. -..-. .. ... -..-. .... . .-. .
FRIAM Applied Complexity Group listserv
Zoom Fridays 9:30a-12p Mtn GMT-6 bit.ly/virtualfriam
un/subscribe http://redfish.com/mailman/listinfo/friam_redfish.com
FRIAM-COMIC http://friam-comic.blogspot.com/
archives: http://friam.471366.n2.nabble.com/

Jochen Fromm-5

Re: Interview with Jeremy Howard

Deep learning mostly seems to be the good old back-propagation in feedforward neural networks which is rediscovered every 10 years by a new generation. Plus more data and more servers. The result is reasonable pattern recognition which lacks explainability. As Noah Smith said

Deep learning is basically just a computer saying "I can't quite define it, but I know it when I see it."

https://twitter.com/Noahpinion/status/1361752362969272321

-J.

-------- Original message --------

From: jon zingale <[hidden email]>

Date: 2/24/21 18:13 (GMT+01:00)

To: [hidden email]

Subject: Re: [FRIAM] Interview with Jeremy Howard

Marcus G. Daniels

Re: Interview with Jeremy Howard

In reply to this post by jon zingale

There’s also the possibility of learning a generative model that can sample examples dependent on certain givens. It opens up the possibility of empirical study. One can generate unseen data that may represent deep structure.

From: Friam <[hidden email]> On Behalf Of Jochen Fromm
Sent: Wednesday, February 24, 2021 10:39 AM
To: The Friday Morning Applied Complexity Coffee Group <[hidden email]>
Subject: Re: [FRIAM] Interview with Jeremy Howard

Deep learning is basically just a computer saying "I can't quite define it, but I know it when I see it."

https://twitter.com/Noahpinion/status/1361752362969272321

-J.

-------- Original message --------

From: jon zingale <[hidden email]>

Date: 2/24/21 18:13 (GMT+01:00)

To: [hidden email]

Subject: Re: [FRIAM] Interview with Jeremy Howard

Sent from the Friam mailing list archive at Nabble.com.

jon zingale

Re: Interview with Jeremy Howard

In reply to this post by Jochen Fromm-5

"The result is reasonable pattern recognition which lacks explainability".

I would prefer to say *lacks explanation*. Historically, Minsky
inadvertently pushed the field's academic development underground for a few
decades, but this isn't really enough (IMHO) to establish a pattern of
cycles. While I know it can be annoying to make comparisons between
artificial neural networks and their biological analogs, this need for
*explanation* seems common to both. I am impressed by the progress that is
being made and remain hopeful that more is yet to come. OTOH, there is a lot
of difficult engineering, scientific, mathematical, and philosophical work
still ahead.

--
Sent from: http://friam.471366.n2.nabble.com/

- .... . -..-. . -. -.. -..-. .. ... -..-. .... . .-. .
FRIAM Applied Complexity Group listserv
Zoom Fridays 9:30a-12p Mtn GMT-6 bit.ly/virtualfriam
un/subscribe http://redfish.com/mailman/listinfo/friam_redfish.com
FRIAM-COMIC http://friam-comic.blogspot.com/
archives: http://friam.471366.n2.nabble.com/

gepr

Re: Interview with Jeremy Howard

In reply to this post by Russ Abbott

IDK. I'm a fan of flipping things. And when talking about explainability, especially when we only have the sui generis architecture as an existence proof (the brain-body => general intelligence), I'd suggest the *explanations* are fictions and the machine, the network, is real. So, what we're calling "explainability" is actually "fictionalizability". In that flipped conception, our task is not to fictionalize what the network is doing. It's to de-fictionalize what we think or expect.

Or, i.e., the network is doing the real stuff. Our "explanations" of what the network is doing are nonsense.

On 2/24/21 10:39 AM, Jochen Fromm wrote:
> Deep learning mostly seems to be the good old back-propagation in feedforward neural networks which is rediscovered every 10 years by a new generation. Plus more data and more servers. The result is reasonable pattern recognition which lacks explainability. As Noah Smith said
>
> Deep learning is basically just a computer saying "I can't quite define it, but I know it when I see it."
> https://twitter.com/Noahpinion/status/1361752362969272321

--
↙↙↙ uǝlƃ

- .... . -..-. . -. -.. -..-. .. ... -..-. .... . .-. .
FRIAM Applied Complexity Group listserv
Zoom Fridays 9:30a-12p Mtn GMT-6 bit.ly/virtualfriam
un/subscribe http://redfish.com/mailman/listinfo/friam_redfish.com
FRIAM-COMIC http://friam-comic.blogspot.com/
archives: http://friam.471366.n2.nabble.com/

uǝʃƃ ⊥ glen

Jochen Fromm-5

Re: Interview with Jeremy Howard

In reply to this post by jon zingale

I remember posting on Usenet about 15 or 20 years ago (I think it was about neural networks on comp.ai or so) and then suddenly Marvin Minsky himself replied "look I have done that already in 1960 or 1970). I was impressed to get a response from him, after all he was at MIT and had written "The society of Mind" etc.

I am still impressed by the progress Google has made. If you look at Google Translate it is just amazing to see how good the translations are already. This was unthinkable 20 years ago. I believe the success comes from the amount of data they use in a smart way. Halevy, Norvig and Pereira called it "The Unreasonable Effectiveness of Data"

https://research.google/pubs/pub35179

-J.

-------- Original message --------

From: jon zingale <[hidden email]>

Date: 2/24/21 20:05 (GMT+01:00)

To: [hidden email]

Subject: Re: [FRIAM] Interview with Jeremy Howard

Barry MacKichan

Re: Interview with Jeremy Howard

In reply to this post by Russ Abbott

In poker terms, I’ll meet your 20 years and raise it to 60. When I was a freshman, Harvard had optional non-credit seminars to introduce us to some advanced work. The one I took was on computer science before that was a phrase, much less a department. My project was with Anthony Oettinger and Susumo Kuno on automatic language translation. This was when they were discovering how hard it is — some of the examples I remember are “Couples applying for marriage licenses wearing pedal pushers will be denied licenses” has about 120 grammatical interpretations, and “Time flies” has several (noun ‘time’, verb ‘flies’ or imperative verb ‘time’ and insect subject ‘flies’).

They were still excited about the usefulness of a software technique they called a ‘pushdown store’ for parsing sentences. Today we call it a stack. Chomsky’s transformational grammar work was still very new then.

I’m not surprised that the methods that succeeded in language translation seem to be an end run around the early methods used then.

—Barry

On 25 Feb 2021, at 10:39, Jochen Fromm wrote:

I remember posting on Usenet about 15 or 20 years ago (I think it was about neural networks on comp.ai or so) and then suddenly Marvin Minsky himself replied "look I have done that already in 1960 or 1970). I was impressed to get a response from him, after all he was at MIT and had written "The society of Mind" etc.

I am still impressed by the progress Google has made. If you look at Google Translate it is just amazing to see how good the translations are already. This was unthinkable 20 years ago. I believe the success comes from the amount of data they use in a smart way. Halevy, Norvig and Pereira called it "The Unreasonable Effectiveness of Data"

<https://research.google/pubs/pub35179>

jon zingale

Re: Interview with Jeremy Howard

In reply to this post by Jochen Fromm-5

I hope his comments didn't stymie you and your group from continuing to make progress on whatever idea y'all were expressing on Usenet. If we could only travel back in time and play this later recording for earlier Minsky.

..and I notice that a lot of people keep saying, "well, I thought of that a long time ago", and that sort of thing and they keep trying to get recognition, and why bother?

To clarify, I feel that Minsky anticipated Waibel (TDNN), Hopfield (RNN), Hochreiter (LSTM), Herbert (ESN), Et. Al like Newton anticipated Cauchy, Riemann, Weierstrass, Et. Al. That is to say, perhaps if one squints hard enough. I do sympathize with Minsky when he laments the absence of philosophical inquiry during this wet lab era of AI, and I very much enjoy listening to his interviews on YouTube. It would have been a real pleasure to have known him.

I appreciate Glen's comment for orienting the discussion around the phenomena, the networks themselves. It is in this sense that wet lab seems like an apt analogy. Inevitably, it is over these grounds that any meaningful higher-level theory must relate. For instance, linguists ask how is it that children learn from so few examples? Some posit highly specialized and innately given structures (Chomsky) while others look for highly specialized and external social networks (Tomasello). In the field of machine learning something of the same appears to be happening. We are pleased that so much can be encoded informationally in the data, and we look to ways that such information can be encoded as or afforded by the structure in a network. To my mind, the unreasonable effectiveness of data points to a dual quality relative to a kind of impedance matching. That an abundance of errorful and incomplete data sets are better than a few pristine data sets speaks to me of the unreasonable robustness of information, ie., the difficulty of accidentally distilling the informational content away from the data, or thought of another way, being unconcerned that some sample or other will exhibit randomness. Thankfully, richly structured sources produce rich information, and this seems especially so for natural language. Thanks for keeping this ball rolling :)