mooc for credit?

[Steve Smith]

I'm wondering if we are mixing apples and orangutans here?

1) Some of the discussion here seems to be about equality... making sure that *everyone* has access to good education... probably including the third world, but definitely including those in the US who are living at or near the poverty level.  Those unlikely to get more than very limited access to the system, if only because they are locked into a self-image that doesn't include higher education (or possibly even secondary).

2) Some of the discussion seems to be about viable business models for state and/or private universities.  These are the realities of the evolving (inverting?) public/private system of higher education...   Most of this may not relate to 1) except insomuch as it spills over (free online courses/materials accessible to anyone with Internet Access).

3) Some of the discussion seems to be about how learning works in general... and specifically for relatively privileged (lower-middle-class US/Euro and above) people.

4) Some of the discussion seems to be about how we (and people like *us*) seek, access and engage in learning.   90% of us probably are beyond seeking formal education for anything except expanding our horizons.  As few as 10% of us might actually be pinning a future/career on getting an education.  Many of us have children or  grandchildren who are about to be directly experiencing this system.

5) Many of us have been (or currently are) involved in education... higher and/or secondary level perhaps.  Some are involved in very conventional education while others have experimented with many alternative forms.

I believe that we are in the midst of (perhaps early years) a major social restructuring which will be either based in, or highly correlated with our public education opportunities.   The modern public education system (K-12 and public post-secondary) seems key in establishing and maintaining social and economic norms.  Whether it is through (racial/culturally biased?) standardised testing, or the classroom norms (very strict behaviour standards up through the 60's or 70's followed by many classrooms becoming a cross between a nursery and a prison ward), or a fundamental divide between Sci/Tech and "English Majors", or a focus primarily on Vocational Education (all disciplines focused somewhat on viability in the workplace?).

I find 1), 3) and 4) most interesting:  2) may be a key constraint and 5) is a natural artifact of the nature of this list.   I'm sure I'm missing other aspects of the discussion.

- Steve

PS.  I'd still like to entertain a "tip" system for education where base salaries are lowered and it is presumed that "good service" will yield "good tips".  If it were not awkward, I would happily provide "my share" of the support of a few teachers I had over the decades...  Aside from the natural lag (it would take 12-30 years for my first grade teacher to cash in on her good/bad work with me), I'm sure this is riddled with practical problems.

One of my best former students just left Pixar after 15 years to join Digipen. It will be interesting to talk to him at SIGGRAPH this summer. It's also interesting that he received all his college education in public universities  before their costs became ridiculous.

Let me ask you the following question: Do you think that the student who chooses digipen will get a better CS education that if she went to the University of Washington which has one of the top CS programs? How does the answer depend on the fact that Washington's support for its universities has dropped in half the past couple of years?

Here's another illustrative anecdote about public education in the State of Washington. A very close friends of mine passed away this year while he was president of Lake Washington Technical College, which is a state school that until a couple of years ago was a 2  year TVI school and now also gives B Tech degrees. Because of the state budget cuts, it didn't do him much good to recruit more students from Washington for which he received less than cost of educating them. His innovative solution was to recruit students from China who pay full tuition  to car repair program since China is selling more cars than anyone but lacks trained mechanics to fix them. 

My reading of these examples is that while the digipens (which are the class of the focussed private education field) may do well, it does not represent an overall improvement for US educations, especially for those who lack the financial means to attend the digipens. Nor is it clear that such focussed education is better for most students.

Ed

__________

Ed Angel

Founding Director, Art, Research, Technology and Science Laboratory (ARTS Lab)
Professor Emeritus of Computer Science, University of New Mexico

1017 Sierra Pinon

Santa Fe, NM 87501
505-984-0136 (home)   [hidden email]

505-453-4944 (cell)  http://www.cs.unm.edu/~angel

On Mar 31, 2013, at 11:55 AM, Roger Critchlow wrote:

The rankings at http://www.topuniversities.com/university-rankings are interesting, because I run out of non-US universities that I recognize in the rankings long before I run out of US universities that don't appear in the rankings.  When I visited the site last spring they were listing tuition costs, too.  US education is priced like US health care, insanely more expensive than the rest of the world, 5 to 10 to 20 times more expensive.

What happened to free education?  People figured out how to make a profit from it and maximized the profit at the expense of the education.  Free education had no business model, so some bean counters made one up.  Lots of places still do free education, but not in the USA.

I don't see MOOC's as a replacement for traditional education.  It's a an outreach tool, a recruiting program that finds the people who can apply themselves to a subject and benefit and remain interested in the subject.  It finds them much more efficiently than admitting applicants to a four year program.  Really good students are really rare.  Most of them would never consider applying to a top 20 university.   Most of them would never come to the attention of a top 20 university admissions program.  MOOC's can find those rare students, make them aware of their abilities, and bring them to the attention of the best schools.  If a school can find one Andrew Viterbi equivalent and educate him or her, the consequences are breathtaking.

Of course the students of a MOOC won't learn as much computer graphics as full time students in an on campus course.  But the on campus course students probably won't learn as much computer graphics as the students at Digipen.com taking c++, physics based modeling, linear algebra, and computer graphics for their first year classes.   But the motivated independent learner will probably out strip them all.

-- rec --

On Sun, Mar 31, 2013 at 10:05 AM, Bruce Sherwood <[hidden email]> wrote:

Ed's post is highly cogent, and based on tons of experience. One of his points that I had missed in my own analysis is the key difference between an on-line course taken by on-campus students and remote students who lack the supporting social infrastructure and may be consumed by job and life responsibilities (my mature high school physics teachers were an unusual bunch). That difference may account for the reported success of the on-line intro physics course at Arizona State.

Another point Ed correctly makes about Udacity's CS 101 and computer graphics MOOCs that I too should have made is that both these courses, while interesting experiments, are indeed very far from equaling the breadth and depth of corresponding one-semester university courses.

Bruce

On Sun, Mar 31, 2013 at 9:48 AM, Edward Angel <[hidden email]> wrote:

Dave,

I don't think interesting describes my response to this post. More like disgusted. I would have said outraged but I'm getting too used to seeing nonsense on the web to respond as I used to. Although I agree with most of the points you and Bruce made, I disagree in a couple of important places but more than that I object to the characterization of what is going on in the post and how willing people are to accept some of its statements, most of which are a total misreading of what is going on with universities and MOOCs.

If I were to make a single statement about how to understand what is going on, I'd harken back to Deep Throat and advise people to take his advice: "Follow the money."

It amazes me how many people are willing to see the faculty as the bad guys on the credit issue and not even look deep enough into the issue to see that is not the case for most of them. i've spent over 40 years in academia, a lot of battling administrators and often other faculty about these issues. But with regard to MOOCs, it's hard not to be a little sympathetic to the situation college presidents find themselves in, especially at public institutions. Budgets in states, including California and Washington, have been cut dramatically. Although there is some idealism in universities' support of MOOCs, they are not charitable institutions and other than a few elite universities which can afford to support experiments with MOOCs that provide high level classes for a global audience, the vast majority of universities are struggling to support their own students. From the administration's perspective MOOCs appear as a possible cost cutting measure, one that may be necessary even if quality declines a bit. Most of the faculty who are against MOOCs are fighting to preserve quality. Maybe that's a losing cause but not something they should for which they should be reviled. These issues have been discussed in detail in the Chronicle but the post that you sent ignores the underlying issues.

Let me examine one course in detail that to me shows why granting credit is not justified. The Udacity computer graphics course is being taught by a very good friend of mine, one I have tremendous respect for. I am enjoying the course and am impressed by the quality of the tools that Udacity has made available to him to enhance the presentation.  Nevertheless I doubt that even 1% of the students who finish the course would be able to pass the standard senior/graduate course in Computer Graphics that is taught by most CS departments (most of which use my textbook). If you want to take the view that what we do in academia is irrelevant than I'd estimate that even fewer would pass the certification exam in OpenGL that is being developed by the Khronos Group, the industry group that sets many of the standards including OpenGL, WebGL, and OpenCL. 

I don't think there are necessarily any bad guys here (other than those who intentionally distort the data). Nevertheless, it is totally unclear as to (a) whether there is a business model that makes sense for MOOCs and (b) what happens to students who complete a less than standard course via a MOOC. Is there a benefit to students who complete a beginning programming or graphics course other than to have sparked their interest? If they want to continue, most will be led right back to the system that is having financial problems and looked to MOOCs to get around them.

From what I've seen, the same is true for essentially all the low level MOOCs. The situation is different for  advanced technical courses such as the Stanford Machine Learning course but in the end I suspect that they will also have a minimum impact due to both money issues and to the problems facing non-traditional students other than the ones on this list.

I have been involved with advanced technical courses for non-traditional students since 1967 when as a grad student I taught some graduate computer design courses for USC at Lockheed and other locations around Southern California. The students were desperate for advanced education since the aerospace industry was known to lay off engineers with 10-15 years of experience at the slightest downturn and then hire new graduates as soon as business improved. In spite of their motivation and good preparation, very few of these students could complete a standard course in a semester due to the demands of a full time job and a variety of other life issues. I've confirmed this over the years by teaching the same course on campus and off campus both live and via remote technologies multiple times. The on campus students were always able to get the course done while on the average the off campus students could handle about 1/2 to 2/3 of the course.

In1972, as a junior faculty I taught one of the first remote delivery courses at USC to a similar audience using one way video and two way voice. It was a huge technical advance and provided high level courses all over the LA area. Later USC, Stanford and others, such as the National Technical University, went national with their programs. At UNM I used a variety of methods to reach remote students, including teaching live classes at Las Alamos, using the video system and recently the on-line system. For 30 years at UNM, almost all of my advanced classes were taught to remote students. Under all these systems, very little changed in terms of their effectiveness. None of the methods had a business model that was able to survive changing technologies, competition, and the true delivery costs. 

But more than these factors, are the difficulties of teaching in teaching non-traditional students. For every Owen who is willing to put in all the effort needed to get the most out of a class, there may be 10-100 others who are less prepared, don't have the time and are dealing with their jobs. In all the years, I've been teaching such students, I've had some great successes but I've also had to put in far more effort per student for remote students than I did for on-campus students. I note that many courses at UNM are now taught concurrently both on campus and on line. Many local students choose the on-line versions and are willing to pay an additional $100 delivery fee (which does come close to extra costs for the remote course). But most of these students actually are on campus so can access their cohorts, their instructors and the live lectures if desired. Thus they are actually paying for the extras of being able to not come to campus with its parking issues and to be able to review material on-line which to most is worth the extra $100 fee.  Their performance is very different from that of truly remote students who cannot access the campus.

My final comment is about the bandwagon everyone seems to be jumping on the bandwagon to dump on US colleges and universities. At this point in my life, I've taught in over 20 countries in five continents, including over 100 professional development courses. The reason I and others have been in such demand comes back to the successes of US schools in educating us. So while every other advancing economy is trying emulate the US success, here we are slashing budgets (what every happened to the free college education?),  crapping on ourselves and looking for magic solutions in MOOCs. We have plenty of problems to solve, many that the colleges and universities have helped exacerbate and even greater problems with K-12 education but let's acknowledge where our colleges and universities have gotten us and not be so quick to toss out what we have achieved.

Ed

__________

Ed Angel

Founding Director, Art, Research, Technology and Science Laboratory (ARTS Lab)
Professor Emeritus of Computer Science, University of New Mexico

1017 Sierra Pinon

Santa Fe, NM 87501
<a moz-do-not-send="true" href="tel:[phone]" value="[phone]" target="_blank">505-984-0136 (home)   [hidden email]

<a moz-do-not-send="true" href="tel:[phone]" value="[phone]" target="_blank">505-453-4944 (cell)  http://www.cs.unm.edu/~angel

On Mar 27, 2013, at 9:19 AM, Prof David West wrote:

those discussing MOOCs recently, might find this interesting

http://techcrunch.com/2013/03/22/72-of-professors-who-teach-online-courses-dont-think-their-students-deserve-credit/

davew

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

[Edward Angel]

Of course that's all true but I think you're wiggling out of the essence of the question. Is the student who is honest and sincere about becoming a computer scientist (but perhaps not focussed yet on some of the specific programs at digipen) better off going to digipen over UW? You can average your response over lots of students.

I can even repose it to is the cheater better off going to digipen over UW or is it worth the money to cheat your way through Harvard (probably)?

Ed

__________

Ed Angel

Founding Director, Art, Research, Technology and Science Laboratory (ARTS Lab)
Professor Emeritus of Computer Science, University of New Mexico

1017 Sierra Pinon

Santa Fe, NM 87501
505-984-0136 (home)   [hidden email]

505-453-4944 (cell)  http://www.cs.unm.edu/~angel

On Mar 31, 2013, at 1:52 PM, Roger Critchlow wrote:

On Sun, Mar 31, 2013 at 1:08 PM, Edward Angel <[hidden email]> wrote:

Let me ask you the following question: Do you think that the student who chooses digipen will get a better CS education that if she went to the University of Washington which has one of the top CS programs? How does the answer depend on the fact that Washington's support for its universities has dropped in half the past couple of years?

The student might do better or worse.  The outcomes of education vary greatly between students.  What we count as "a better CS education" varies depending on the student we're looking at.  There's a reliable proportion of undergraduate students who never learn anything for longer than it takes to pass the exam.  Some of them won't even do that if they can figure out how to cheat.  

You can find them at Harvard commencement ceremonies.  After four years at one of the top 5 universities in the world, they believe that the difference between winter and summer is that the earth is closer to the sun in summer.  20-30% of graduating seniors and alumni questioned, if I remember the study correctly.  Smarter than the average bear?

-- rec --

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

[Prof David West]

 

Ed,

 

I am curious where you disagree / what you disagree with.  I see one thing in the post below that is inconsistent with my opinions as stated.  I did not address motivations, faculty, or economics - but would agree with everything you said below in those regards.

 

The only point of potential disagreement - you are far more charitable with regard existing university education (your final paragraph).  While I do not advocate "dumping on" existing higher ed - I do believe that it has become an entirely untenable model - both in terms of economics and in terms of education.

 

MOOCs are NOT the answer!  As far as I can see they perpetuate a model that, in my view, is not working - at least not working in terms of educating people who can think, who are engaged with knowledge, ... (discussion for another time and place).

 

Totally online universities - e.g. Digipen - are not the answer!

 

My answer to your question in a subsequent post:

  - Digipen could probably (opinion again - no hard numbers here) be a better option for 1-5% of the people they enroll - primarily because it can give a mature dedicated student with professional experience a more focused and integrated program of study of immediate professional use. (Note this says nothing about the education they will receive.)

  - UNM would be best for the roughly 20-30 percent of students able to take on-campus courses and interact with peers and faculty on a regular basis. (except in math, where I like the Digipen options over the traditional 8-credits of calculus).

  - I would suspect that Digipen's ten year average drop-out/non-completion rate will be double or triple UNM's.

  - both offer very little, especially given the cost, to a majority of their students beyond a piece of paper that will get them past the HR department of a hiring corporation.

 

In my opinion, on-line has the potential to replace and improve upon the standard 40 hours of lecture, single textbook, lame classroom discussion, homework assignments and exams part of existing higher ed.  But simply moving the standard model to on-line (I would guess more than ninety-percent of current on-line efforts, including Digipen) will not realize that potential.

 

My belief / quest / futile tilting at windmills is focused on a really radical reinvention of education in toto along with the ways such an education is offered/obtained.  (Again, another discussion for a different place and time.)

 

A question - perhaps a way to get some real data into the discussion - do you, Ed, or anyone else on the list have any data about graduation rates for on-line schools, like Full Sail University or Walden?  I think the Feds have started requiring for-profits to start posting this data, but could not find it in a cursory search.

 

davew

 

 

 

 

On Sun, Mar 31, 2013, at 09:48 AM, Edward Angel wrote:

Dave,

 

I don't think interesting describes my response to this post. More like disgusted. I would have said outraged but I'm getting too used to seeing nonsense on the web to respond as I used to. Although I agree with most of the points you and Bruce made, I disagree in a couple of important places but more than that I object to the characterization of what is going on in the post and how willing people are to accept some of its statements, most of which are a total misreading of what is going on with universities and MOOCs.

 

If I were to make a single statement about how to understand what is going on, I'd harken back to Deep Throat and advise people to take his advice: "Follow the money."

 

It amazes me how many people are willing to see the faculty as the bad guys on the credit issue and not even look deep enough into the issue to see that is not the case for most of them. i've spent over 40 years in academia, a lot of battling administrators and often other faculty about these issues. But with regard to MOOCs, it's hard not to be a little sympathetic to the situation college presidents find themselves in, especially at public institutions. Budgets in states, including California and Washington, have been cut dramatically. Although there is some idealism in universities' support of MOOCs, they are not charitable institutions and other than a few elite universities which can afford to support experiments with MOOCs that provide high level classes for a global audience, the vast majority of universities are struggling to support their own students. From the administration's perspective MOOCs appear as a possible cost cutting measure, one that may be necessary even if quality declines a bit. Most of the faculty who are against MOOCs are fighting to preserve quality. Maybe that's a losing cause but not something they should for which they should be reviled. These issues have been discussed in detail in the Chronicle but the post that you sent ignores the underlying issues.

 

Let me examine one course in detail that to me shows why granting credit is not justified. The Udacity computer graphics course is being taught by a very good friend of mine, one I have tremendous respect for. I am enjoying the course and am impressed by the quality of the tools that Udacity has made available to him to enhance the presentation.  Nevertheless I doubt that even 1% of the students who finish the course would be able to pass the standard senior/graduate course in Computer Graphics that is taught by most CS departments (most of which use my textbook). If you want to take the view that what we do in academia is irrelevant than I'd estimate that even fewer would pass the certification exam in OpenGL that is being developed by the Khronos Group, the industry group that sets many of the standards including OpenGL, WebGL, and OpenCL. 

 

I don't think there are necessarily any bad guys here (other than those who intentionally distort the data). Nevertheless, it is totally unclear as to (a) whether there is a business model that makes sense for MOOCs and (b) what happens to students who complete a less than standard course via a MOOC. Is there a benefit to students who complete a beginning programming or graphics course other than to have sparked their interest? If they want to continue, most will be led right back to the system that is having financial problems and looked to MOOCs to get around them.

 

From what I've seen, the same is true for essentially all the low level MOOCs. The situation is different for  advanced technical courses such as the Stanford Machine Learning course but in the end I suspect that they will also have a minimum impact due to both money issues and to the problems facing non-traditional students other than the ones on this list.

 

I have been involved with advanced technical courses for non-traditional students since 1967 when as a grad student I taught some graduate computer design courses for USC at Lockheed and other locations around Southern California. The students were desperate for advanced education since the aerospace industry was known to lay off engineers with 10-15 years of experience at the slightest downturn and then hire new graduates as soon as business improved. In spite of their motivation and good preparation, very few of these students could complete a standard course in a semester due to the demands of a full time job and a variety of other life issues. I've confirmed this over the years by teaching the same course on campus and off campus both live and via remote technologies multiple times. The on campus students were always able to get the course done while on the average the off campus students could handle about 1/2 to 2/3 of the course.

 

In1972, as a junior faculty I taught one of the first remote delivery courses at USC to a similar audience using one way video and two way voice. It was a huge technical advance and provided high level courses all over the LA area. Later USC, Stanford and others, such as the National Technical University, went national with their programs. At UNM I used a variety of methods to reach remote students, including teaching live classes at Las Alamos, using the video system and recently the on-line system. For 30 years at UNM, almost all of my advanced classes were taught to remote students. Under all these systems, very little changed in terms of their effectiveness. None of the methods had a business model that was able to survive changing technologies, competition, and the true delivery costs. 

 

But more than these factors, are the difficulties of teaching in teaching non-traditional students. For every Owen who is willing to put in all the effort needed to get the most out of a class, there may be 10-100 others who are less prepared, don't have the time and are dealing with their jobs. In all the years, I've been teaching such students, I've had some great successes but I've also had to put in far more effort per student for remote students than I did for on-campus students. I note that many courses at UNM are now taught concurrently both on campus and on line. Many local students choose the on-line versions and are willing to pay an additional $100 delivery fee (which does come close to extra costs for the remote course). But most of these students actually are on campus so can access their cohorts, their instructors and the live lectures if desired. Thus they are actually paying for the extras of being able to not come to campus with its parking issues and to be able to review material on-line which to most is worth the extra $100 fee.  Their performance is very different from that of truly remote students who cannot access the campus.

 

My final comment is about the bandwagon everyone seems to be jumping on the bandwagon to dump on US colleges and universities. At this point in my life, I've taught in over 20 countries in five continents, including over 100 professional development courses. The reason I and others have been in such demand comes back to the successes of US schools in educating us. So while every other advancing economy is trying emulate the US success, here we are slashing budgets (what every happened to the free college education?),  crapping on ourselves and looking for magic solutions in MOOCs. We have plenty of problems to solve, many that the colleges and universities have helped exacerbate and even greater problems with K-12 education but let's acknowledge where our colleges and universities have gotten us and not be so quick to toss out what we have achieved.

 

Ed

 

__________

 

Ed Angel

 

Founding Director, Art, Research, Technology and Science Laboratory (ARTS Lab)

Professor Emeritus of Computer Science, University of New Mexico

 

1017 Sierra Pinon

Santa Fe, NM 87501

505-984-0136 (home) [hidden email]

505-453-4944 (cell) http://www.cs.unm.edu/~angel

 

 

On Mar 27, 2013, at 9:19 AM, Prof David West wrote:

 

those discussing MOOCs recently, might find this interesting

 

http://techcrunch.com/2013/03/22/72-of-professors-who-teach-online-courses-dont-think-their-students-deserve-credit/

 

davew

 

============================================================

FRIAM Applied Complexity Group listserv

Meets Fridays 9a-11:30 at cafe at St. John's College

to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

 

 

============================================================

FRIAM Applied Complexity Group listserv

Meets Fridays 9a-11:30 at cafe at St. John's College

to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

[Edward Angel]

Dave,

Actually, I had only one very slight disagreement with you which I kind of forgot about by the end of my email. I have no problem with what you are proposing and should have pointed out how I agreed that one of the requirements of all the possible ways we can improve the system is the commitment by the student to 30-40 hours a week, something that many students can't or are unwilling to do.

The slight disagreement was with respect to "with the exception of elite research universities and 2 year professional / vocational institutions." I'm a great beneficiary of and believer in public high schools, colleges and universities and want to see them improved. I'm disturbed by a tendency all the way up to Obama to emphasize 2 year vocational training. It's an easy way to avoid dealing with the serious problems of public education. It's especially pronounced in NM where our public K-12 system is terrible so we wind up with so many young people with a GED and vocational training who can never achieve their dreams or realize their potential. As inefficient as UNM can be by having low admission standards and teaching classes that can be done at a lower cost in 2 year schools, it does provide an opportunity for many that is in danger of disappearing. Lately, we've noticed it in young people who are now jobless due to the collapse of the construction industry. With a GED and a young family, they are really stuck.

Ed

__________

Ed Angel

Founding Director, Art, Research, Technology and Science Laboratory (ARTS Lab)
Professor Emeritus of Computer Science, University of New Mexico

1017 Sierra Pinon

Santa Fe, NM 87501
505-984-0136 (home)   [hidden email]

505-453-4944 (cell)  http://www.cs.unm.edu/~angel

On Mar 31, 2013, at 3:06 PM, Prof David West wrote:

 

Ed,

 

I am curious where you disagree / what you disagree with.  I see one thing in the post below that is inconsistent with my opinions as stated.  I did not address motivations, faculty, or economics - but would agree with everything you said below in those regards.

 

The only point of potential disagreement - you are far more charitable with regard existing university education (your final paragraph).  While I do not advocate "dumping on" existing higher ed - I do believe that it has become an entirely untenable model - both in terms of economics and in terms of education.

 

MOOCs are NOT the answer!  As far as I can see they perpetuate a model that, in my view, is not working - at least not working in terms of educating people who can think, who are engaged with knowledge, ... (discussion for another time and place).

 

Totally online universities - e.g. Digipen - are not the answer!

 

My answer to your question in a subsequent post:

  - Digipen could probably (opinion again - no hard numbers here) be a better option for 1-5% of the people they enroll - primarily because it can give a mature dedicated student with professional experience a more focused and integrated program of study of immediate professional use. (Note this says nothing about the education they will receive.)

  - UNM would be best for the roughly 20-30 percent of students able to take on-campus courses and interact with peers and faculty on a regular basis. (except in math, where I like the Digipen options over the traditional 8-credits of calculus).

  - I would suspect that Digipen's ten year average drop-out/non-completion rate will be double or triple UNM's.

  - both offer very little, especially given the cost, to a majority of their students beyond a piece of paper that will get them past the HR department of a hiring corporation.

 

In my opinion, on-line has the potential to replace and improve upon the standard 40 hours of lecture, single textbook, lame classroom discussion, homework assignments and exams part of existing higher ed.  But simply moving the standard model to on-line (I would guess more than ninety-percent of current on-line efforts, including Digipen) will not realize that potential.

 

My belief / quest / futile tilting at windmills is focused on a really radical reinvention of education in toto along with the ways such an education is offered/obtained.  (Again, another discussion for a different place and time.)

 

A question - perhaps a way to get some real data into the discussion - do you, Ed, or anyone else on the list have any data about graduation rates for on-line schools, like Full Sail University or Walden?  I think the Feds have started requiring for-profits to start posting this data, but could not find it in a cursory search.

 

davew

 

 

 

 

On Sun, Mar 31, 2013, at 09:48 AM, Edward Angel wrote:

Dave,

 

I don't think interesting describes my response to this post. More like disgusted. I would have said outraged but I'm getting too used to seeing nonsense on the web to respond as I used to. Although I agree with most of the points you and Bruce made, I disagree in a couple of important places but more than that I object to the characterization of what is going on in the post and how willing people are to accept some of its statements, most of which are a total misreading of what is going on with universities and MOOCs.

 

If I were to make a single statement about how to understand what is going on, I'd harken back to Deep Throat and advise people to take his advice: "Follow the money."

 

It amazes me how many people are willing to see the faculty as the bad guys on the credit issue and not even look deep enough into the issue to see that is not the case for most of them. i've spent over 40 years in academia, a lot of battling administrators and often other faculty about these issues. But with regard to MOOCs, it's hard not to be a little sympathetic to the situation college presidents find themselves in, especially at public institutions. Budgets in states, including California and Washington, have been cut dramatically. Although there is some idealism in universities' support of MOOCs, they are not charitable institutions and other than a few elite universities which can afford to support experiments with MOOCs that provide high level classes for a global audience, the vast majority of universities are struggling to support their own students. From the administration's perspective MOOCs appear as a possible cost cutting measure, one that may be necessary even if quality declines a bit. Most of the faculty who are against MOOCs are fighting to preserve quality. Maybe that's a losing cause but not something they should for which they should be reviled. These issues have been discussed in detail in the Chronicle but the post that you sent ignores the underlying issues.

 

Let me examine one course in detail that to me shows why granting credit is not justified. The Udacity computer graphics course is being taught by a very good friend of mine, one I have tremendous respect for. I am enjoying the course and am impressed by the quality of the tools that Udacity has made available to him to enhance the presentation.  Nevertheless I doubt that even 1% of the students who finish the course would be able to pass the standard senior/graduate course in Computer Graphics that is taught by most CS departments (most of which use my textbook). If you want to take the view that what we do in academia is irrelevant than I'd estimate that even fewer would pass the certification exam in OpenGL that is being developed by the Khronos Group, the industry group that sets many of the standards including OpenGL, WebGL, and OpenCL. 

 

I don't think there are necessarily any bad guys here (other than those who intentionally distort the data). Nevertheless, it is totally unclear as to (a) whether there is a business model that makes sense for MOOCs and (b) what happens to students who complete a less than standard course via a MOOC. Is there a benefit to students who complete a beginning programming or graphics course other than to have sparked their interest? If they want to continue, most will be led right back to the system that is having financial problems and looked to MOOCs to get around them.

 

From what I've seen, the same is true for essentially all the low level MOOCs. The situation is different for  advanced technical courses such as the Stanford Machine Learning course but in the end I suspect that they will also have a minimum impact due to both money issues and to the problems facing non-traditional students other than the ones on this list.

 

I have been involved with advanced technical courses for non-traditional students since 1967 when as a grad student I taught some graduate computer design courses for USC at Lockheed and other locations around Southern California. The students were desperate for advanced education since the aerospace industry was known to lay off engineers with 10-15 years of experience at the slightest downturn and then hire new graduates as soon as business improved. In spite of their motivation and good preparation, very few of these students could complete a standard course in a semester due to the demands of a full time job and a variety of other life issues. I've confirmed this over the years by teaching the same course on campus and off campus both live and via remote technologies multiple times. The on campus students were always able to get the course done while on the average the off campus students could handle about 1/2 to 2/3 of the course.

 

In1972, as a junior faculty I taught one of the first remote delivery courses at USC to a similar audience using one way video and two way voice. It was a huge technical advance and provided high level courses all over the LA area. Later USC, Stanford and others, such as the National Technical University, went national with their programs. At UNM I used a variety of methods to reach remote students, including teaching live classes at Las Alamos, using the video system and recently the on-line system. For 30 years at UNM, almost all of my advanced classes were taught to remote students. Under all these systems, very little changed in terms of their effectiveness. None of the methods had a business model that was able to survive changing technologies, competition, and the true delivery costs. 

 

But more than these factors, are the difficulties of teaching in teaching non-traditional students. For every Owen who is willing to put in all the effort needed to get the most out of a class, there may be 10-100 others who are less prepared, don't have the time and are dealing with their jobs. In all the years, I've been teaching such students, I've had some great successes but I've also had to put in far more effort per student for remote students than I did for on-campus students. I note that many courses at UNM are now taught concurrently both on campus and on line. Many local students choose the on-line versions and are willing to pay an additional $100 delivery fee (which does come close to extra costs for the remote course). But most of these students actually are on campus so can access their cohorts, their instructors and the live lectures if desired. Thus they are actually paying for the extras of being able to not come to campus with its parking issues and to be able to review material on-line which to most is worth the extra $100 fee.  Their performance is very different from that of truly remote students who cannot access the campus.

 

My final comment is about the bandwagon everyone seems to be jumping on the bandwagon to dump on US colleges and universities. At this point in my life, I've taught in over 20 countries in five continents, including over 100 professional development courses. The reason I and others have been in such demand comes back to the successes of US schools in educating us. So while every other advancing economy is trying emulate the US success, here we are slashing budgets (what every happened to the free college education?),  crapping on ourselves and looking for magic solutions in MOOCs. We have plenty of problems to solve, many that the colleges and universities have helped exacerbate and even greater problems with K-12 education but let's acknowledge where our colleges and universities have gotten us and not be so quick to toss out what we have achieved.

 

Ed

 

__________

 

Ed Angel

 

Founding Director, Art, Research, Technology and Science Laboratory (ARTS Lab)

Professor Emeritus of Computer Science, University of New Mexico

 

1017 Sierra Pinon

Santa Fe, NM 87501

505-984-0136 (home) [hidden email]

505-453-4944 (cell) http://www.cs.unm.edu/~angel

 

 

On Mar 27, 2013, at 9:19 AM, Prof David West wrote:

 

those discussing MOOCs recently, might find this interesting

 

http://techcrunch.com/2013/03/22/72-of-professors-who-teach-online-courses-dont-think-their-students-deserve-credit/

 

davew

 

============================================================

FRIAM Applied Complexity Group listserv

Meets Fridays 9a-11:30 at cafe at St. John's College

to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

 

 

============================================================

FRIAM Applied Complexity Group listserv

Meets Fridays 9a-11:30 at cafe at St. John's College

to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

[Roger Critchlow-2]

On Sun, Mar 31, 2013 at 2:09 PM, Edward Angel <[hidden email]> wrote:

Of course that's all true but I think you're wiggling out of the essence of the question. Is the student who is honest and sincere about becoming a computer scientist (but perhaps not focussed yet on some of the specific programs at digipen) better off going to digipen over UW? You can average your response over lots of students.

No.  Digipen is for people with clear focus.  U-dub would be the better choice for the honest and sincere computer science candidate.  (By the way, Digipen in Redmond is an in-person school, it shares an office park with Nintendo, students live off campus.)

But this _is_ all oranges and orangutans.  

Are MOOC's credit worthy?  Sure, for some definition of credit.    But probably more useful as a way of finding people who can do the work so they can be enticed into more intensive opportunities.

Are all the credits that traditional universities grant worth anything?  Of course not, some of them are just tokens granted in return for tuition while keeping the juvenile delinquents off the streets for a few years.  Higher education has always been a multi-objective institution, taking revenues where ever and why ever they were offered, providing value how ever possible.  Baby sitting juvenile delinquents has always been part of the bargain.  It would probably be more cost effective to put the juvenile delinquents in the Marines, but the Marines are too smart to take them anymore.

Can MOOC's keep juvenile delinquents off the streets?  Not a chance.  Video games appear to work, at least for a while, but no credentials for video gamers, yet.

-- rec --

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

[Prof David West]

Ed,

 

Cool --- and I think we are generally in agreement in this area as well - again, with the exception of how radical a solution is required.

 

It is my belief that graduate students at Tier One research institutions like UNM are getting a pretty solid education, a professional network, and reasonable career preparation at a fairly reasonable (at least at the public schools) cost-benefit ratio.

 

Students at 2-yr and voc/tech institutions are getting immediate job prep, at the expense of a substantive education, but also at a pretty reasonable cost-benefit ratio.- The lack of substantive education, however, means 2-yr graduates cannot adapt and grow, as your construction industry example shows.  In addition, two-year schools offer very little in terms of creating an educated and responsible citizenry.  2-yrs and an immediate job, is Not the answer!

 

K-12 and the majority of schools between Tier One and community college are severely broken.  In both cases. neither society nor the student is getting a benefit even remotely commensurate with cost.

 

That said, the educational need of the majority of students and the social obligation to meet that need has to be addressed.

 

It would be nice if this could be done via "reform" and "incremental improvement," but, In my opinion, this is not possible - it will take a substantial, radical and revolutionary change.

 

davew

 

 

 

On Sun, Mar 31, 2013, at 03:30 PM, Edward Angel wrote:

Dave,

 

Actually, I had only one very slight disagreement with you which I kind of forgot about by the end of my email. I have no problem with what you are proposing and should have pointed out how I agreed that one of the requirements of all the possible ways we can improve the system is the commitment by the student to 30-40 hours a week, something that many students can't or are unwilling to do.

 

The slight disagreement was with respect to "with the exception of elite research universities and 2 year professional / vocational institutions." I'm a great beneficiary of and believer in public high schools, colleges and universities and want to see them improved. I'm disturbed by a tendency all the way up to Obama to emphasize 2 year vocational training. It's an easy way to avoid dealing with the serious problems of public education. It's especially pronounced in NM where our public K-12 system is terrible so we wind up with so many young people with a GED and vocational training who can never achieve their dreams or realize their potential. As inefficient as UNM can be by having low admission standards and teaching classes that can be done at a lower cost in 2 year schools, it does provide an opportunity for many that is in danger of disappearing. Lately, we've noticed it in young people who are now jobless due to the collapse of the construction industry. With a GED and a young family, they are really stuck.

 

Ed

__________

 

Ed Angel

 

Founding Director, Art, Research, Technology and Science Laboratory (ARTS Lab)

Professor Emeritus of Computer Science, University of New Mexico

 

1017 Sierra Pinon

Santa Fe, NM 87501

505-984-0136 (home) [hidden email]

505-453-4944 (cell) http://www.cs.unm.edu/~angel

 

 

On Mar 31, 2013, at 3:06 PM, Prof David West wrote:

 

 

Ed,

 

I am curious where you disagree / what you disagree with.  I see one thing in the post below that is inconsistent with my opinions as stated.  I did not address motivations, faculty, or economics - but would agree with everything you said below in those regards.

 

The only point of potential disagreement - you are far more charitable with regard existing university education (your final paragraph).  While I do not advocate "dumping on" existing higher ed - I do believe that it has become an entirely untenable model - both in terms of economics and in terms of education.

 

MOOCs are NOT the answer!  As far as I can see they perpetuate a model that, in my view, is not working - at least not working in terms of educating people who can think, who are engaged with knowledge, ... (discussion for another time and place).

 

Totally online universities - e.g. Digipen - are not the answer!

 

My answer to your question in a subsequent post:

  - Digipen could probably (opinion again - no hard numbers here) be a better option for 1-5% of the people they enroll - primarily because it can give a mature dedicated student with professional experience a more focused and integrated program of study of immediate professional use. (Note this says nothing about the education they will receive.)

  - UNM would be best for the roughly 20-30 percent of students able to take on-campus courses and interact with peers and faculty on a regular basis. (except in math, where I like the Digipen options over the traditional 8-credits of calculus).

  - I would suspect that Digipen's ten year average drop-out/non-completion rate will be double or triple UNM's.

  - both offer very little, especially given the cost, to a majority of their students beyond a piece of paper that will get them past the HR department of a hiring corporation.

 

In my opinion, on-line has the potential to replace and improve upon the standard 40 hours of lecture, single textbook, lame classroom discussion, homework assignments and exams part of existing higher ed.  But simply moving the standard model to on-line (I would guess more than ninety-percent of current on-line efforts, including Digipen) will not realize that potential.

 

My belief / quest / futile tilting at windmills is focused on a really radical reinvention of education in toto along with the ways such an education is offered/obtained.  (Again, another discussion for a different place and time.)

 

A question - perhaps a way to get some real data into the discussion - do you, Ed, or anyone else on the list have any data about graduation rates for on-line schools, like Full Sail University or Walden?  I think the Feds have started requiring for-profits to start posting this data, but could not find it in a cursory search.

 

davew

 

 

 

 

On Sun, Mar 31, 2013, at 09:48 AM, Edward Angel wrote:

Dave,

 

I don't think interesting describes my response to this post. More like disgusted. I would have said outraged but I'm getting too used to seeing nonsense on the web to respond as I used to. Although I agree with most of the points you and Bruce made, I disagree in a couple of important places but more than that I object to the characterization of what is going on in the post and how willing people are to accept some of its statements, most of which are a total misreading of what is going on with universities and MOOCs.

 

If I were to make a single statement about how to understand what is going on, I'd harken back to Deep Throat and advise people to take his advice: "Follow the money."

 

It amazes me how many people are willing to see the faculty as the bad guys on the credit issue and not even look deep enough into the issue to see that is not the case for most of them. i've spent over 40 years in academia, a lot of battling administrators and often other faculty about these issues. But with regard to MOOCs, it's hard not to be a little sympathetic to the situation college presidents find themselves in, especially at public institutions. Budgets in states, including California and Washington, have been cut dramatically. Although there is some idealism in universities' support of MOOCs, they are not charitable institutions and other than a few elite universities which can afford to support experiments with MOOCs that provide high level classes for a global audience, the vast majority of universities are struggling to support their own students. From the administration's perspective MOOCs appear as a possible cost cutting measure, one that may be necessary even if quality declines a bit. Most of the faculty who are against MOOCs are fighting to preserve quality. Maybe that's a losing cause but not something they should for which they should be reviled. These issues have been discussed in detail in the Chronicle but the post that you sent ignores the underlying issues.

 

Let me examine one course in detail that to me shows why granting credit is not justified. The Udacity computer graphics course is being taught by a very good friend of mine, one I have tremendous respect for. I am enjoying the course and am impressed by the quality of the tools that Udacity has made available to him to enhance the presentation.  Nevertheless I doubt that even 1% of the students who finish the course would be able to pass the standard senior/graduate course in Computer Graphics that is taught by most CS departments (most of which use my textbook). If you want to take the view that what we do in academia is irrelevant than I'd estimate that even fewer would pass the certification exam in OpenGL that is being developed by the Khronos Group, the industry group that sets many of the standards including OpenGL, WebGL, and OpenCL. 

 

I don't think there are necessarily any bad guys here (other than those who intentionally distort the data). Nevertheless, it is totally unclear as to (a) whether there is a business model that makes sense for MOOCs and (b) what happens to students who complete a less than standard course via a MOOC. Is there a benefit to students who complete a beginning programming or graphics course other than to have sparked their interest? If they want to continue, most will be led right back to the system that is having financial problems and looked to MOOCs to get around them.

 

From what I've seen, the same is true for essentially all the low level MOOCs. The situation is different for  advanced technical courses such as the Stanford Machine Learning course but in the end I suspect that they will also have a minimum impact due to both money issues and to the problems facing non-traditional students other than the ones on this list.

 

I have been involved with advanced technical courses for non-traditional students since 1967 when as a grad student I taught some graduate computer design courses for USC at Lockheed and other locations around Southern California. The students were desperate for advanced education since the aerospace industry was known to lay off engineers with 10-15 years of experience at the slightest downturn and then hire new graduates as soon as business improved. In spite of their motivation and good preparation, very few of these students could complete a standard course in a semester due to the demands of a full time job and a variety of other life issues. I've confirmed this over the years by teaching the same course on campus and off campus both live and via remote technologies multiple times. The on campus students were always able to get the course done while on the average the off campus students could handle about 1/2 to 2/3 of the course.

 

In1972, as a junior faculty I taught one of the first remote delivery courses at USC to a similar audience using one way video and two way voice. It was a huge technical advance and provided high level courses all over the LA area. Later USC, Stanford and others, such as the National Technical University, went national with their programs. At UNM I used a variety of methods to reach remote students, including teaching live classes at Las Alamos, using the video system and recently the on-line system. For 30 years at UNM, almost all of my advanced classes were taught to remote students. Under all these systems, very little changed in terms of their effectiveness. None of the methods had a business model that was able to survive changing technologies, competition, and the true delivery costs. 

 

But more than these factors, are the difficulties of teaching in teaching non-traditional students. For every Owen who is willing to put in all the effort needed to get the most out of a class, there may be 10-100 others who are less prepared, don't have the time and are dealing with their jobs. In all the years, I've been teaching such students, I've had some great successes but I've also had to put in far more effort per student for remote students than I did for on-campus students. I note that many courses at UNM are now taught concurrently both on campus and on line. Many local students choose the on-line versions and are willing to pay an additional $100 delivery fee (which does come close to extra costs for the remote course). But most of these students actually are on campus so can access their cohorts, their instructors and the live lectures if desired. Thus they are actually paying for the extras of being able to not come to campus with its parking issues and to be able to review material on-line which to most is worth the extra $100 fee.  Their performance is very different from that of truly remote students who cannot access the campus.

 

My final comment is about the bandwagon everyone seems to be jumping on the bandwagon to dump on US colleges and universities. At this point in my life, I've taught in over 20 countries in five continents, including over 100 professional development courses. The reason I and others have been in such demand comes back to the successes of US schools in educating us. So while every other advancing economy is trying emulate the US success, here we are slashing budgets (what every happened to the free college education?),  crapping on ourselves and looking for magic solutions in MOOCs. We have plenty of problems to solve, many that the colleges and universities have helped exacerbate and even greater problems with K-12 education but let's acknowledge where our colleges and universities have gotten us and not be so quick to toss out what we have achieved.

 

Ed

 

__________

 

Ed Angel

 

Founding Director, Art, Research, Technology and Science Laboratory (ARTS Lab)

Professor Emeritus of Computer Science, University of New Mexico

 

1017 Sierra Pinon

Santa Fe, NM 87501

505-984-0136 (home) [hidden email]

505-453-4944 (cell) http://www.cs.unm.edu/~angel

 

 

On Mar 27, 2013, at 9:19 AM, Prof David West wrote:

 

those discussing MOOCs recently, might find this interesting

 

http://techcrunch.com/2013/03/22/72-of-professors-who-teach-online-courses-dont-think-their-students-deserve-credit/

 

davew

 

============================================================

FRIAM Applied Complexity Group listserv

Meets Fridays 9a-11:30 at cafe at St. John's College

to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

 

 

============================================================

FRIAM Applied Complexity Group listserv

Meets Fridays 9a-11:30 at cafe at St. John's College

to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================

FRIAM Applied Complexity Group listserv

Meets Fridays 9a-11:30 at cafe at St. John's College

to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

 

============================================================

FRIAM Applied Complexity Group listserv

Meets Fridays 9a-11:30 at cafe at St. John's College

to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

[Bruce Sherwood]

Ruth Chabay makes an important comment about the credit issue, one that hasn't come up in this discussion so far. As I said before, we went all the way through Udacity's CS 101 "course", which was excellent. After Ed correctly pointed out that this and the Udacity computer graphics "course" are not at all equivalent to comparable traditional courses, I agreed. But Ruth points out that CS 101 was only a 6- or 7-week minicourse, and we worked hard at it, spending many hours per week. If CS 101 had the length of an entire semester, the comparison with a traditional course in terms of depth and breadth would be quite different. She argues cogently that we need not think in terms of traditional semester courses; minicourses with small credit might have a lot to recommend them: semesters are very long, and instructors and students get weary. Perhaps the MOOC instructors who said "no" to course credit might have said "yes" if they had been asked something like "Do you think your MOOC should grant 1 credit hour?"

I know from experience that there are plenty of intro CS and physics courses taught at respected universities that are a crock, in which very little learning takes place, and a really good MOOC such as Udacity's CS 101 could be a big improvement. Whether ways could be found to make it possible for students lacking high motivation to benefit from a MOOC is an open question, as is the cheating issue with respect to credit.

Bruce

On Mon, Apr 1, 2013 at 12:30 PM, Prof David West <[hidden email]> wrote:

Ed,

 

Cool --- and I think we are generally in agreement in this area as well - again, with the exception of how radical a solution is required.

 

It is my belief that graduate students at Tier One research institutions like UNM are getting a pretty solid education, a professional network, and reasonable career preparation at a fairly reasonable (at least at the public schools) cost-benefit ratio.

 

Students at 2-yr and voc/tech institutions are getting immediate job prep, at the expense of a substantive education, but also at a pretty reasonable cost-benefit ratio.- The lack of substantive education, however, means 2-yr graduates cannot adapt and grow, as your construction industry example shows.  In addition, two-year schools offer very little in terms of creating an educated and responsible citizenry.  2-yrs and an immediate job, is Not the answer!

 

K-12 and the majority of schools between Tier One and community college are severely broken.  In both cases. neither society nor the student is getting a benefit even remotely commensurate with cost.

 

That said, the educational need of the majority of students and the social obligation to meet that need has to be addressed.

 

It would be nice if this could be done via "reform" and "incremental improvement," but, In my opinion, this is not possible - it will take a substantial, radical and revolutionary change.

 

davew

 

 

 

On Sun, Mar 31, 2013, at 03:30 PM, Edward Angel wrote:

Dave,

 

Actually, I had only one very slight disagreement with you which I kind of forgot about by the end of my email. I have no problem with what you are proposing and should have pointed out how I agreed that one of the requirements of all the possible ways we can improve the system is the commitment by the student to 30-40 hours a week, something that many students can't or are unwilling to do.

 

The slight disagreement was with respect to "with the exception of elite research universities and 2 year professional / vocational institutions." I'm a great beneficiary of and believer in public high schools, colleges and universities and want to see them improved. I'm disturbed by a tendency all the way up to Obama to emphasize 2 year vocational training. It's an easy way to avoid dealing with the serious problems of public education. It's especially pronounced in NM where our public K-12 system is terrible so we wind up with so many young people with a GED and vocational training who can never achieve their dreams or realize their potential. As inefficient as UNM can be by having low admission standards and teaching classes that can be done at a lower cost in 2 year schools, it does provide an opportunity for many that is in danger of disappearing. Lately, we've noticed it in young people who are now jobless due to the collapse of the construction industry. With a GED and a young family, they are really stuck.

 

Ed

__________

 

Ed Angel

 

Founding Director, Art, Research, Technology and Science Laboratory (ARTS Lab)

Professor Emeritus of Computer Science, University of New Mexico

 

1017 Sierra Pinon

Santa Fe, NM 87501

<a href="tel:[phone]" value="[phone]" target="_blank">505-984-0136 (home) [hidden email]

<a href="tel:[phone]" value="[phone]" target="_blank">505-453-4944 (cell) http://www.cs.unm.edu/~angel

 

 

On Mar 31, 2013, at 3:06 PM, Prof David West wrote:

 

 

Ed,

 

I am curious where you disagree / what you disagree with.  I see one thing in the post below that is inconsistent with my opinions as stated.  I did not address motivations, faculty, or economics - but would agree with everything you said below in those regards.

 

The only point of potential disagreement - you are far more charitable with regard existing university education (your final paragraph).  While I do not advocate "dumping on" existing higher ed - I do believe that it has become an entirely untenable model - both in terms of economics and in terms of education.

 

MOOCs are NOT the answer!  As far as I can see they perpetuate a model that, in my view, is not working - at least not working in terms of educating people who can think, who are engaged with knowledge, ... (discussion for another time and place).

 

Totally online universities - e.g. Digipen - are not the answer!

 

My answer to your question in a subsequent post:

  - Digipen could probably (opinion again - no hard numbers here) be a better option for 1-5% of the people they enroll - primarily because it can give a mature dedicated student with professional experience a more focused and integrated program of study of immediate professional use. (Note this says nothing about the education they will receive.)

  - UNM would be best for the roughly 20-30 percent of students able to take on-campus courses and interact with peers and faculty on a regular basis. (except in math, where I like the Digipen options over the traditional 8-credits of calculus).

  - I would suspect that Digipen's ten year average drop-out/non-completion rate will be double or triple UNM's.

  - both offer very little, especially given the cost, to a majority of their students beyond a piece of paper that will get them past the HR department of a hiring corporation.

 

In my opinion, on-line has the potential to replace and improve upon the standard 40 hours of lecture, single textbook, lame classroom discussion, homework assignments and exams part of existing higher ed.  But simply moving the standard model to on-line (I would guess more than ninety-percent of current on-line efforts, including Digipen) will not realize that potential.

 

My belief / quest / futile tilting at windmills is focused on a really radical reinvention of education in toto along with the ways such an education is offered/obtained.  (Again, another discussion for a different place and time.)

 

A question - perhaps a way to get some real data into the discussion - do you, Ed, or anyone else on the list have any data about graduation rates for on-line schools, like Full Sail University or Walden?  I think the Feds have started requiring for-profits to start posting this data, but could not find it in a cursory search.

 

davew

 

 

 

 

On Sun, Mar 31, 2013, at 09:48 AM, Edward Angel wrote:

Dave,

 

I don't think interesting describes my response to this post. More like disgusted. I would have said outraged but I'm getting too used to seeing nonsense on the web to respond as I used to. Although I agree with most of the points you and Bruce made, I disagree in a couple of important places but more than that I object to the characterization of what is going on in the post and how willing people are to accept some of its statements, most of which are a total misreading of what is going on with universities and MOOCs.

 

If I were to make a single statement about how to understand what is going on, I'd harken back to Deep Throat and advise people to take his advice: "Follow the money."

 

It amazes me how many people are willing to see the faculty as the bad guys on the credit issue and not even look deep enough into the issue to see that is not the case for most of them. i've spent over 40 years in academia, a lot of battling administrators and often other faculty about these issues. But with regard to MOOCs, it's hard not to be a little sympathetic to the situation college presidents find themselves in, especially at public institutions. Budgets in states, including California and Washington, have been cut dramatically. Although there is some idealism in universities' support of MOOCs, they are not charitable institutions and other than a few elite universities which can afford to support experiments with MOOCs that provide high level classes for a global audience, the vast majority of universities are struggling to support their own students. From the administration's perspective MOOCs appear as a possible cost cutting measure, one that may be necessary even if quality declines a bit. Most of the faculty who are against MOOCs are fighting to preserve quality. Maybe that's a losing cause but not something they should for which they should be reviled. These issues have been discussed in detail in the Chronicle but the post that you sent ignores the underlying issues.

 

Let me examine one course in detail that to me shows why granting credit is not justified. The Udacity computer graphics course is being taught by a very good friend of mine, one I have tremendous respect for. I am enjoying the course and am impressed by the quality of the tools that Udacity has made available to him to enhance the presentation.  Nevertheless I doubt that even 1% of the students who finish the course would be able to pass the standard senior/graduate course in Computer Graphics that is taught by most CS departments (most of which use my textbook). If you want to take the view that what we do in academia is irrelevant than I'd estimate that even fewer would pass the certification exam in OpenGL that is being developed by the Khronos Group, the industry group that sets many of the standards including OpenGL, WebGL, and OpenCL. 

 

I don't think there are necessarily any bad guys here (other than those who intentionally distort the data). Nevertheless, it is totally unclear as to (a) whether there is a business model that makes sense for MOOCs and (b) what happens to students who complete a less than standard course via a MOOC. Is there a benefit to students who complete a beginning programming or graphics course other than to have sparked their interest? If they want to continue, most will be led right back to the system that is having financial problems and looked to MOOCs to get around them.

 

From what I've seen, the same is true for essentially all the low level MOOCs. The situation is different for  advanced technical courses such as the Stanford Machine Learning course but in the end I suspect that they will also have a minimum impact due to both money issues and to the problems facing non-traditional students other than the ones on this list.

 

I have been involved with advanced technical courses for non-traditional students since 1967 when as a grad student I taught some graduate computer design courses for USC at Lockheed and other locations around Southern California. The students were desperate for advanced education since the aerospace industry was known to lay off engineers with 10-15 years of experience at the slightest downturn and then hire new graduates as soon as business improved. In spite of their motivation and good preparation, very few of these students could complete a standard course in a semester due to the demands of a full time job and a variety of other life issues. I've confirmed this over the years by teaching the same course on campus and off campus both live and via remote technologies multiple times. The on campus students were always able to get the course done while on the average the off campus students could handle about 1/2 to 2/3 of the course.

 

In1972, as a junior faculty I taught one of the first remote delivery courses at USC to a similar audience using one way video and two way voice. It was a huge technical advance and provided high level courses all over the LA area. Later USC, Stanford and others, such as the National Technical University, went national with their programs. At UNM I used a variety of methods to reach remote students, including teaching live classes at Las Alamos, using the video system and recently the on-line system. For 30 years at UNM, almost all of my advanced classes were taught to remote students. Under all these systems, very little changed in terms of their effectiveness. None of the methods had a business model that was able to survive changing technologies, competition, and the true delivery costs. 

 

But more than these factors, are the difficulties of teaching in teaching non-traditional students. For every Owen who is willing to put in all the effort needed to get the most out of a class, there may be 10-100 others who are less prepared, don't have the time and are dealing with their jobs. In all the years, I've been teaching such students, I've had some great successes but I've also had to put in far more effort per student for remote students than I did for on-campus students. I note that many courses at UNM are now taught concurrently both on campus and on line. Many local students choose the on-line versions and are willing to pay an additional $100 delivery fee (which does come close to extra costs for the remote course). But most of these students actually are on campus so can access their cohorts, their instructors and the live lectures if desired. Thus they are actually paying for the extras of being able to not come to campus with its parking issues and to be able to review material on-line which to most is worth the extra $100 fee.  Their performance is very different from that of truly remote students who cannot access the campus.

 

My final comment is about the bandwagon everyone seems to be jumping on the bandwagon to dump on US colleges and universities. At this point in my life, I've taught in over 20 countries in five continents, including over 100 professional development courses. The reason I and others have been in such demand comes back to the successes of US schools in educating us. So while every other advancing economy is trying emulate the US success, here we are slashing budgets (what every happened to the free college education?),  crapping on ourselves and looking for magic solutions in MOOCs. We have plenty of problems to solve, many that the colleges and universities have helped exacerbate and even greater problems with K-12 education but let's acknowledge where our colleges and universities have gotten us and not be so quick to toss out what we have achieved.

 

Ed

 

__________

 

Ed Angel

 

Founding Director, Art, Research, Technology and Science Laboratory (ARTS Lab)

Professor Emeritus of Computer Science, University of New Mexico

 

1017 Sierra Pinon

Santa Fe, NM 87501

<a href="tel:[phone]" value="[phone]" target="_blank">505-984-0136 (home) [hidden email]

<a href="tel:[phone]" value="[phone]" target="_blank">505-453-4944 (cell) http://www.cs.unm.edu/~angel

 

 

On Mar 27, 2013, at 9:19 AM, Prof David West wrote:

 

those discussing MOOCs recently, might find this interesting

 

http://techcrunch.com/2013/03/22/72-of-professors-who-teach-online-courses-dont-think-their-students-deserve-credit/

 

davew

 

============================================================

FRIAM Applied Complexity Group listserv

Meets Fridays 9a-11:30 at cafe at St. John's College

to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

 

 

============================================================

FRIAM Applied Complexity Group listserv

Meets Fridays 9a-11:30 at cafe at St. John's College

to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================

FRIAM Applied Complexity Group listserv

Meets Fridays 9a-11:30 at cafe at St. John's College

to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

 

============================================================

FRIAM Applied Complexity Group listserv

Meets Fridays 9a-11:30 at cafe at St. John's College

to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

[Edward Angel]

That might work for a beginning computer course. I don't think it would work well in other fields. Also, a single credit is rarely of much use to a student and when students try to transfer credits, the usual process requires them to show there is an equivalent course at the college where they want to transfer the credit.

A long time ago I tried as the first UNM Presidential Teaching Follow to put together an array of one credit computer courses that would each me taught in 5 week blocks according to demand and different departments could then meet the needs of their own programs by putting together the right combination of courses. Even though the deans all thought this was a great idea, in the end it went nowhere. There was a no single reason it failed other than the inertia of trying to make a change in a big multi-level system.

Ed

__________

Ed Angel

Founding Director, Art, Research, Technology and Science Laboratory (ARTS Lab)
Professor Emeritus of Computer Science, University of New Mexico

1017 Sierra Pinon

Santa Fe, NM 87501
505-984-0136 (home)   [hidden email]

505-453-4944 (cell)  http://www.cs.unm.edu/~angel

On Apr 1, 2013, at 1:33 PM, Bruce Sherwood wrote:

Ruth Chabay makes an important comment about the credit issue, one that hasn't come up in this discussion so far. As I said before, we went all the way through Udacity's CS 101 "course", which was excellent. After Ed correctly pointed out that this and the Udacity computer graphics "course" are not at all equivalent to comparable traditional courses, I agreed. But Ruth points out that CS 101 was only a 6- or 7-week minicourse, and we worked hard at it, spending many hours per week. If CS 101 had the length of an entire semester, the comparison with a traditional course in terms of depth and breadth would be quite different. She argues cogently that we need not think in terms of traditional semester courses; minicourses with small credit might have a lot to recommend them: semesters are very long, and instructors and students get weary. Perhaps the MOOC instructors who said "no" to course credit might have said "yes" if they had been asked something like "Do you think your MOOC should grant 1 credit hour?"

I know from experience that there are plenty of intro CS and physics courses taught at respected universities that are a crock, in which very little learning takes place, and a really good MOOC such as Udacity's CS 101 could be a big improvement. Whether ways could be found to make it possible for students lacking high motivation to benefit from a MOOC is an open question, as is the cheating issue with respect to credit.

Bruce

On Mon, Apr 1, 2013 at 12:30 PM, Prof David West <[hidden email]> wrote:

Ed,

 

Cool --- and I think we are generally in agreement in this area as well - again, with the exception of how radical a solution is required.

 

It is my belief that graduate students at Tier One research institutions like UNM are getting a pretty solid education, a professional network, and reasonable career preparation at a fairly reasonable (at least at the public schools) cost-benefit ratio.

 

Students at 2-yr and voc/tech institutions are getting immediate job prep, at the expense of a substantive education, but also at a pretty reasonable cost-benefit ratio.- The lack of substantive education, however, means 2-yr graduates cannot adapt and grow, as your construction industry example shows.  In addition, two-year schools offer very little in terms of creating an educated and responsible citizenry.  2-yrs and an immediate job, is Not the answer!

 

K-12 and the majority of schools between Tier One and community college are severely broken.  In both cases. neither society nor the student is getting a benefit even remotely commensurate with cost.

 

That said, the educational need of the majority of students and the social obligation to meet that need has to be addressed.

 

It would be nice if this could be done via "reform" and "incremental improvement," but, In my opinion, this is not possible - it will take a substantial, radical and revolutionary change.

 

davew

 

 

 

On Sun, Mar 31, 2013, at 03:30 PM, Edward Angel wrote:

Dave,

 

Actually, I had only one very slight disagreement with you which I kind of forgot about by the end of my email. I have no problem with what you are proposing and should have pointed out how I agreed that one of the requirements of all the possible ways we can improve the system is the commitment by the student to 30-40 hours a week, something that many students can't or are unwilling to do.

 

The slight disagreement was with respect to "with the exception of elite research universities and 2 year professional / vocational institutions." I'm a great beneficiary of and believer in public high schools, colleges and universities and want to see them improved. I'm disturbed by a tendency all the way up to Obama to emphasize 2 year vocational training. It's an easy way to avoid dealing with the serious problems of public education. It's especially pronounced in NM where our public K-12 system is terrible so we wind up with so many young people with a GED and vocational training who can never achieve their dreams or realize their potential. As inefficient as UNM can be by having low admission standards and teaching classes that can be done at a lower cost in 2 year schools, it does provide an opportunity for many that is in danger of disappearing. Lately, we've noticed it in young people who are now jobless due to the collapse of the construction industry. With a GED and a young family, they are really stuck.

 

Ed

__________

 

Ed Angel

 

Founding Director, Art, Research, Technology and Science Laboratory (ARTS Lab)

Professor Emeritus of Computer Science, University of New Mexico

 

1017 Sierra Pinon

Santa Fe, NM 87501

<a href="tel:[phone]" value="[phone]" target="_blank">505-984-0136 (home) [hidden email]

<a href="tel:[phone]" value="[phone]" target="_blank">505-453-4944 (cell) http://www.cs.unm.edu/~angel

 

 

On Mar 31, 2013, at 3:06 PM, Prof David West wrote:

 

 

Ed,

 

I am curious where you disagree / what you disagree with.  I see one thing in the post below that is inconsistent with my opinions as stated.  I did not address motivations, faculty, or economics - but would agree with everything you said below in those regards.

 

The only point of potential disagreement - you are far more charitable with regard existing university education (your final paragraph).  While I do not advocate "dumping on" existing higher ed - I do believe that it has become an entirely untenable model - both in terms of economics and in terms of education.

 

MOOCs are NOT the answer!  As far as I can see they perpetuate a model that, in my view, is not working - at least not working in terms of educating people who can think, who are engaged with knowledge, ... (discussion for another time and place).

 

Totally online universities - e.g. Digipen - are not the answer!

 

My answer to your question in a subsequent post:

  - Digipen could probably (opinion again - no hard numbers here) be a better option for 1-5% of the people they enroll - primarily because it can give a mature dedicated student with professional experience a more focused and integrated program of study of immediate professional use. (Note this says nothing about the education they will receive.)

  - UNM would be best for the roughly 20-30 percent of students able to take on-campus courses and interact with peers and faculty on a regular basis. (except in math, where I like the Digipen options over the traditional 8-credits of calculus).

  - I would suspect that Digipen's ten year average drop-out/non-completion rate will be double or triple UNM's.

  - both offer very little, especially given the cost, to a majority of their students beyond a piece of paper that will get them past the HR department of a hiring corporation.

 

In my opinion, on-line has the potential to replace and improve upon the standard 40 hours of lecture, single textbook, lame classroom discussion, homework assignments and exams part of existing higher ed.  But simply moving the standard model to on-line (I would guess more than ninety-percent of current on-line efforts, including Digipen) will not realize that potential.

 

My belief / quest / futile tilting at windmills is focused on a really radical reinvention of education in toto along with the ways such an education is offered/obtained.  (Again, another discussion for a different place and time.)

 

A question - perhaps a way to get some real data into the discussion - do you, Ed, or anyone else on the list have any data about graduation rates for on-line schools, like Full Sail University or Walden?  I think the Feds have started requiring for-profits to start posting this data, but could not find it in a cursory search.

 

davew

 

 

 

 

On Sun, Mar 31, 2013, at 09:48 AM, Edward Angel wrote:

Dave,

 

I don't think interesting describes my response to this post. More like disgusted. I would have said outraged but I'm getting too used to seeing nonsense on the web to respond as I used to. Although I agree with most of the points you and Bruce made, I disagree in a couple of important places but more than that I object to the characterization of what is going on in the post and how willing people are to accept some of its statements, most of which are a total misreading of what is going on with universities and MOOCs.

 

If I were to make a single statement about how to understand what is going on, I'd harken back to Deep Throat and advise people to take his advice: "Follow the money."

 

It amazes me how many people are willing to see the faculty as the bad guys on the credit issue and not even look deep enough into the issue to see that is not the case for most of them. i've spent over 40 years in academia, a lot of battling administrators and often other faculty about these issues. But with regard to MOOCs, it's hard not to be a little sympathetic to the situation college presidents find themselves in, especially at public institutions. Budgets in states, including California and Washington, have been cut dramatically. Although there is some idealism in universities' support of MOOCs, they are not charitable institutions and other than a few elite universities which can afford to support experiments with MOOCs that provide high level classes for a global audience, the vast majority of universities are struggling to support their own students. From the administration's perspective MOOCs appear as a possible cost cutting measure, one that may be necessary even if quality declines a bit. Most of the faculty who are against MOOCs are fighting to preserve quality. Maybe that's a losing cause but not something they should for which they should be reviled. These issues have been discussed in detail in the Chronicle but the post that you sent ignores the underlying issues.

 

Let me examine one course in detail that to me shows why granting credit is not justified. The Udacity computer graphics course is being taught by a very good friend of mine, one I have tremendous respect for. I am enjoying the course and am impressed by the quality of the tools that Udacity has made available to him to enhance the presentation.  Nevertheless I doubt that even 1% of the students who finish the course would be able to pass the standard senior/graduate course in Computer Graphics that is taught by most CS departments (most of which use my textbook). If you want to take the view that what we do in academia is irrelevant than I'd estimate that even fewer would pass the certification exam in OpenGL that is being developed by the Khronos Group, the industry group that sets many of the standards including OpenGL, WebGL, and OpenCL. 

 

I don't think there are necessarily any bad guys here (other than those who intentionally distort the data). Nevertheless, it is totally unclear as to (a) whether there is a business model that makes sense for MOOCs and (b) what happens to students who complete a less than standard course via a MOOC. Is there a benefit to students who complete a beginning programming or graphics course other than to have sparked their interest? If they want to continue, most will be led right back to the system that is having financial problems and looked to MOOCs to get around them.

 

From what I've seen, the same is true for essentially all the low level MOOCs. The situation is different for  advanced technical courses such as the Stanford Machine Learning course but in the end I suspect that they will also have a minimum impact due to both money issues and to the problems facing non-traditional students other than the ones on this list.

 

I have been involved with advanced technical courses for non-traditional students since 1967 when as a grad student I taught some graduate computer design courses for USC at Lockheed and other locations around Southern California. The students were desperate for advanced education since the aerospace industry was known to lay off engineers with 10-15 years of experience at the slightest downturn and then hire new graduates as soon as business improved. In spite of their motivation and good preparation, very few of these students could complete a standard course in a semester due to the demands of a full time job and a variety of other life issues. I've confirmed this over the years by teaching the same course on campus and off campus both live and via remote technologies multiple times. The on campus students were always able to get the course done while on the average the off campus students could handle about 1/2 to 2/3 of the course.

 

In1972, as a junior faculty I taught one of the first remote delivery courses at USC to a similar audience using one way video and two way voice. It was a huge technical advance and provided high level courses all over the LA area. Later USC, Stanford and others, such as the National Technical University, went national with their programs. At UNM I used a variety of methods to reach remote students, including teaching live classes at Las Alamos, using the video system and recently the on-line system. For 30 years at UNM, almost all of my advanced classes were taught to remote students. Under all these systems, very little changed in terms of their effectiveness. None of the methods had a business model that was able to survive changing technologies, competition, and the true delivery costs. 

 

But more than these factors, are the difficulties of teaching in teaching non-traditional students. For every Owen who is willing to put in all the effort needed to get the most out of a class, there may be 10-100 others who are less prepared, don't have the time and are dealing with their jobs. In all the years, I've been teaching such students, I've had some great successes but I've also had to put in far more effort per student for remote students than I did for on-campus students. I note that many courses at UNM are now taught concurrently both on campus and on line. Many local students choose the on-line versions and are willing to pay an additional $100 delivery fee (which does come close to extra costs for the remote course). But most of these students actually are on campus so can access their cohorts, their instructors and the live lectures if desired. Thus they are actually paying for the extras of being able to not come to campus with its parking issues and to be able to review material on-line which to most is worth the extra $100 fee.  Their performance is very different from that of truly remote students who cannot access the campus.

 

My final comment is about the bandwagon everyone seems to be jumping on the bandwagon to dump on US colleges and universities. At this point in my life, I've taught in over 20 countries in five continents, including over 100 professional development courses. The reason I and others have been in such demand comes back to the successes of US schools in educating us. So while every other advancing economy is trying emulate the US success, here we are slashing budgets (what every happened to the free college education?),  crapping on ourselves and looking for magic solutions in MOOCs. We have plenty of problems to solve, many that the colleges and universities have helped exacerbate and even greater problems with K-12 education but let's acknowledge where our colleges and universities have gotten us and not be so quick to toss out what we have achieved.

 

Ed

 

__________

 

Ed Angel

 

Founding Director, Art, Research, Technology and Science Laboratory (ARTS Lab)

Professor Emeritus of Computer Science, University of New Mexico

 

1017 Sierra Pinon

Santa Fe, NM 87501

<a href="tel:[phone]" value="[phone]" target="_blank">505-984-0136 (home) [hidden email]

<a href="tel:[phone]" value="[phone]" target="_blank">505-453-4944 (cell) http://www.cs.unm.edu/~angel

 

 

On Mar 27, 2013, at 9:19 AM, Prof David West wrote:

 

those discussing MOOCs recently, might find this interesting

 

http://techcrunch.com/2013/03/22/72-of-professors-who-teach-online-courses-dont-think-their-students-deserve-credit/

 

davew

 

============================================================

FRIAM Applied Complexity Group listserv

Meets Fridays 9a-11:30 at cafe at St. John's College

to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

 

 

============================================================

FRIAM Applied Complexity Group listserv

Meets Fridays 9a-11:30 at cafe at St. John's College

to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================

FRIAM Applied Complexity Group listserv

Meets Fridays 9a-11:30 at cafe at St. John's College

to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

 

============================================================

FRIAM Applied Complexity Group listserv

Meets Fridays 9a-11:30 at cafe at St. John's College

to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com