Intro to Artificial Intelligence....

[naturalist.atheist]

I just signed up for this class. It's been decades since my last class so I hope I can cut it. It is open to all takers and they will grade your homework and exams. However it is not good for Stanford academic credit unless you are a registered Stanford student. But they will email you a certificate with your class standing.

ai-class.com - Introduction to Artificial Intelligence - Fall 2011 (http://www.ai-class.com)

[naturalist.atheist]

Class starts on the tenth. It looks like they moved the site to

ml-class.org

and it is still open for enrollment. The first set of lectures is available online with quizzes. For some reason I am having a heck of time with the first quiz but ace all the others. I can to the math standing on my head but the AI concepts still elude me.

I've posted a request for study group/project partners on the class forum.

I wonder if they will post statistics on class participation as the class proceeds through the material. It looks like they have close to 100,00 students enrolled.

The class also has a twitter feed:@ml_class (http://twitter.com/#!/ml_class/)

I have a sneaking suspicion that this class will be historic and one can only wonder what will come of so many people learning the basics of programming machine intelligence.

Machine Learning: About the class - YouTube

[Dragar]

You've done much coding na?

[naturalist.atheist]

You've done much coding na?

At least 5 million lines over 40 years worth. Wrote my first program on an IBM 370 in 360 assembler in the IBM Boca plant in 1970 when I was 17. But from what I understand there will be more math than coding in this class. Apparently an "algorithm" in machine intelligence is more like a formula which is then "programmed" in MathLab or GNU Octave. If things go well in the class I my try to do some of the assignments in F#.

[Dragar]

You should be fine then! You've done plenty enough I imagine (way more than me!).

Ammembly code is weird. I don't know how anyone codes in that. Part of the course I'm teaching right now involves deciphering the assembly code compared to the Fortran code, and it was pretty esoteric!

Oh, and report back with your findings on this course. :)

[naturalist.atheist]

Will do. Wish me luck.

[ceptimus]

Good luck!

I would be interested in signing up. But I know I don't have the free time to do the necessary work :(

I look forward to your reports! :)

[naturalist.atheist]

There is a meetup for a study group in Tallahassee...

Stanford Machine Learning Course - Tallahassee (Tallahassee, FL) - Meetup (http://www.meetup.com/ml-class-org-Tallahassee/)

[Iacchus]

You've done much coding na?

At least 5 million lines over 40 years worth. Wrote my first program on an IBM 370 in 360 assembler in the IBM Boca plant in 1970 when I was 17.Hey, same here! Except it was at work in Sunnyvale, California, and it was about 10 years later. But no, I haven't written much code since then, until recently, when working on web pages and writing forex trading advisors.

[naturalist.atheist]

You've done much coding na?

At least 5 million lines over 40 years worth. Wrote my first program on an IBM 370 in 360 assembler in the IBM Boca plant in 1970 when I was 17.Hey, same here! Except it was at work in Sunnyvale, California, and it was about 10 years later. But no, I haven't written much code since then, until recently, when working on web pages and writing forex trading advisors.

Hardly the same thing. The 370 was brand new back then, the PC wasn't invented. Networking was just beginning to be developed as a military project and it would be decades before the idea of HTML would occur to anybody. The 370's were massive machines that could cost millions of dollars back in the 70s, when a million dollars was serious scratch. They were not something that you would let just anybody near, let alone a 17 year old. And let alone allow the 17 year old free reign of an entire development floor with dozens of mainframes of all kinds. It's flat out amazing when I think back on it.

It was a very different world.

[Iacchus]

It was a VM 370, although I guess it had been refurbished and they got some sort of deal through IBM. But yeah, the system was massive, and took up a good 30' x 30' room (or more?) with the peripherals and all. It seemed like it had a lot of downtime though, because it was an older machine.

[naturalist.atheist]

A few weeks into the machine learning course and so far so good. I'm learning how to use GNU Octave, which is a kind of freeware version of MathLab. This is a very powerful tool for numerical analysis of all kinds. So far machine learning algorithms look like various wrinkles on linear regression.

[naturalist.atheist]

Who would have thunk it! Character recognition is an application of linear regression.

[naturalist.atheist]

I'm working on the third assignment, and getting a 95.1% accuracy recognizing handwritten digits 0-9 using a simple linear regression technique called Logistic Regression.

All digit images are 20x20 8 bit gray scale images. Each image is converted into a 400x1 vector.

The idea is that for detecting a particular digit like say "2" there is an equation of the form:

f(theta * X)

where theta and X are vectors which in this case would be 401 values (20x20 + 1) in length.

In the case of logistic regression the output desired is something that stays between 0 and 1 and mostly 0 or 1 with intermediate values when transitioning from 0 -> 1.

In this case the sigmoid function is used:

f(z) = 1/(1 + e^z)

The "hypothesis" for the problem would then be stated as:

h(theta, X) = (1/(1 + e^(theta *X))

The trick is finding the magic theta coefficients. This is where the "training" comes in.

A set of 5000 images is supplied with the value each image should have. This is called the training set. The value of each image would be in a 5000 x 1 vector called y. If the model is perfect and the thetas are perfect then for every image in the set:

h(theta, X) - y = 0

Given say 5000 images they are converted into a 5000 x 401 matrix. Each row in the matrix corresponds to a particular image and each column of that row corresponds to a pixel in that image. The extra 1 column is set to 1. This is for the bias coefficient of theta (theta(0)).

However in order to use the "training set" to determine the thetas you need to come up with a "cost function". This should produce a "convex" function with a single global minimum. Such a function will then be amenable to numerical techniques for finding the global minimum such as gradient descent, newtons method and so forth.

The cost function in the case of logistic regression is a somewhat complicated expression:

cost function(theta) = -y * log(h(theta, X)) - (1-y)log(1-h(theta, X))

The gradient of this cost function has the somewhat simpler form:

grad = (h(theta, X) - y) * X

Using one of several possible minimization techniques the "training" set is iteratively used in the gradient of the cost function for every sample in the training set. This results in a set of thetas that can then be used in the hypothesis to take a new 20x20 digit image and determine if it is the digit trained against.

Needless to say I've left out a lot but by using GNU Octave there is very little actual branching and looping involved. It's mostly vector equations and minimization functions.

[naturalist.atheist]

This is a demonstration of the kind of results you can get with a more sophisticated neural network model:

http://yann.lecun.com/exdb/lenet/gifs/asamples.gif

http://yann.lecun.com/exdb/lenet/gifs/a384.gif

http://yann.lecun.com/exdb/lenet/gifs/anoise2.gif

http://yann.lecun.com/exdb/lenet/gifs/anoise4.gif

[naturalist.atheist]

I just finished the three layer neural network assignment. This stuff is very, very cool. To see all the different handwritten numbers just magically get recognized is just way too cool. I could not even begin to think how I would perform such a task using a traditional programming approach. I feel as excited as I did when I wrote my first "Hello World" program in C thirty five years ago.

[naturalist.atheist]

I'm halfway through the course and so far so good. The instructor, Andrew Ng, is doing a fantastic job. If this represents the level of instruction at Stanford then it is no surprise that it is such a well regarded institution. His approach is very practical so I am very hopeful that I will be able to apply what I've learned.

[Dragar]

One of the other PhD students in my year uses neutral networks for photometric redshift fitting of distant galaxies. All this talks of coefficients and training sets is ringing bells now.

She did say they don't use them very much for AI these days though, as they don't seem to work in the way they were hoped they would.

[SR71]

Maybe someone can get adobe to fix PDF so it can sort typewritten letters and numbers. Fun is finding out you didn't get your critical part because it was stock number 386B, not 3868.

[naturalist.atheist]

Dragar, that doesn't correlate with the instructor's take on the success of such techniques. He seems to think that they've been very successful. Especially now that we are well along Moore's curve. He does seem to think that many people are having trouble applying these techniques because they don't know what he is currently teaching us. Certainly, if in principle a brain neuron is as simple as he models it in his class, then theoretically by recreating a similar or equivalent computer neural network, it should be possible to reproduce complex human behavior in a machine. Of course the complexity of human networks is several orders of magnitude greater than what can be accomplished now with even the largest of computers and the goals of machine learning are much less ambitions. Rather than reproduce all the abilities of a functioning human brain they are simply trying to reproduce small subsets of behavior.

A neural network in principle is simple. But training it appears to be the tricky part. It's certainly been difficult training the neural network between my ears.

[naturalist.atheist]

Maybe someone can get adobe to fix PDF so it can sort typewritten letters and numbers. Fun is finding out you didn't get your critical part because it was stock number 386B, not 3868.

That error was probably introduced by the neural network pulling the item from the shelf.

[SR71]

Did the course give a blurb definition of AI?

[naturalist.atheist]

I was taking the AI course but didn't find it very interesting, so I've dropped out. The machine learning class defines a machine learning algorithm as one that gives a computer the ability to learn how to perform a task without being explicitly programmed.

[Dragar]

Dragar, that doesn't correlate with the instructor's take on the success of such techniques. He seems to think that they've been very successful. Especially now that we are well along Moore's curve. He does seem to think that many people are having trouble applying these techniques because they don't know what he is currently teaching us. Certainly, if in principle a brain neuron is as simple as he models it in his class, then theoretically by recreating a similar or equivalent computer neural network, it should be possible to reproduce complex human behavior in a machine. Of course the complexity of human networks is several orders of magnitude greater than what can be accomplished now with even the largest of computers and the goals of machine learning are much less ambitions. Rather than reproduce all the abilities of a functioning human brain they are simply trying to reproduce small subsets of behavior.

A neural network in principle is simple. But training it appears to be the tricky part. It's certainly been difficult training the neural network between my ears.

Odd. Perhaps its the distinction between machine learning (where I've only ever heard success about them) and AI. Perhaps we are using the word AI in different ways.

You're closer to the source, anyway, so it's interesting you're saying they've been very successful.

[naturalist.atheist]

Dragar, that doesn't correlate with the instructor's take on the success of such techniques. He seems to think that they've been very successful. Especially now that we are well along Moore's curve. He does seem to think that many people are having trouble applying these techniques because they don't know what he is currently teaching us. Certainly, if in principle a brain neuron is as simple as he models it in his class, then theoretically by recreating a similar or equivalent computer neural network, it should be possible to reproduce complex human behavior in a machine. Of course the complexity of human networks is several orders of magnitude greater than what can be accomplished now with even the largest of computers and the goals of machine learning are much less ambitions. Rather than reproduce all the abilities of a functioning human brain they are simply trying to reproduce small subsets of behavior.

A neural network in principle is simple. But training it appears to be the tricky part. It's certainly been difficult training the neural network between my ears.

Odd. Perhaps its the distinction between machine learning (where I've only ever heard success about them) and AI. Perhaps we are using the word AI in different ways.

You're closer to the source, anyway, so it's interesting you're saying they've been very successful.

I'm not sure if this is AI but I would consider this progress in AI

Jeopardy! IBM Watson Day 3 (Feb 16, 2011) Part 2/2 - YouTube

And there is autonomous driving which is considered machine learning

Google Self-Driving Car - Part 1: Outside - YouTube

Google autonomous car - YouTube

Now this one is very interesting. It is also an application of machine learning. What makes it interesting is that it can make that helicopter do things that a human could never do. And there was not one line of code written that knew anything about aerodynamics. It would be the way a helicopter bird could fly if there were such a thing. Andrew Ng was involved in this project.

Stanford Autonomous Helicopter - Airshow #1 - YouTube

And then there are the obvious military applications:

http://www.youtube.com/watch?NR=1&v=yDVLUiJfpPw

[But]

Odd. Perhaps its the distinction between machine learning (where I've only ever heard success about them) and AI. Perhaps we are using the word AI in different ways.

You're closer to the source, anyway, so it's interesting you're saying they've been very successful.

I think AI doesn't have well-defined boundaries and at the same time, it's a sexy catch-all term that attracts grant money. It can mean almost anything.

[naturalist.atheist]

Just found what looks to be a good resource for using F# for machine learning.

F# and Data Mining: Support vector machines (SVMs) in F# using Microsoft Solver Foundation (http://fdatamining.blogspot.com/2011/02/support-vector-machines-svms-in-f-using.html)

I'll be giving a talk on machine learning in January for the local .NET architecture group and it would be more PC to do it in F# rather than Octave. So that's what I'm gonna try to do. Perhaps I'll cover octave a little just to show a more targeted DSL (domain specific language) for machine learning.

[BTW, just covered support vector machines in the last lecture. Getting ready to do the assignment.]

[Chatter]

"AI" is so wishy-washy, you might as well just define it by its subfields: automated reasoning, automated planning, agent-based systems, machine learning, natural language processing/generation, machine translation,....

All of these fields may have once arisen from people ambitious to reproduce human-level intelligence, but the goals are usually much more modest and practical these days.

[naturalist.atheist]

Practical is good. It means there is a level of accomplishment that allows a practical approach. For the longest time it was all talk. At least there is something to show.

[SR71]

Ever want to know what earworm song is stuck in your head, but don't know the title, band or lyrics? A cool app would be to hum or sing what you can remember, than have the algorithm record it and search a database for matches.

That would be really hard for a number of reasons, wild guess.

[JoeP]

Shazam (http://www.shazam.com/)

Not sure how well it works with amateur humming or singing attempts but it is phenomenal with quite lo-fi radio clips.

[SR71]

Cool! Thanks JoeP! :thanked:

[Crumb]

There was also one where you tap out the song on your space bar. That worked pretty well.

[JoeP]

Your post is useless without links.

[Crumb]

Bored.com - Tap the rhythm of the song's melody to find music (http://www.bored.com/songtapper/index.html)

[JoeP]

Does it work? Beyond "Thank you please wait a moment while we search"?

[Crumb]

It just did for me.

[naturalist.atheist]

I'm coming to the end of the class. So far so good. I've managed to complete every assignment and extra credit with a perfect score. Recommender systems and anomaly detection are being covered right now. The algorithm he is teaching us for anomaly detection is essentially co-variant Gaussian probability density fitting and then using a cross variant data set to find the best threshold trigger. On recommender systems he is treating them as a linear regression problem where both the parameters and feature set are variables which are then optimized using gradient descent. His examples are the Netflix movie recommender and Amazon product recommender. Perhaps I have the hubris of the spectator but the instructor makes this stuff look easy.

[Dragar]

I'm sure you're just good at code, NA. Also, as you well know, it's always easy when you are shown how it's done.

[naturalist.atheist]

I'm sure you're just good at code, NA. Also, as you well know, it's always easy when you are shown how it's done.

Yes, it is always easy when you are shown how it's done. But there really wasn't that much coding as I am used to it. I think I may have written a total of 4 'for' loops. Octave allows one to write the vector and matrix equations as vectors and matrices. Many equations were expressed in a single line looking pretty much like the equation. If they had been written in a curly bracket language they would have been hundreds of lines of code. IMO Octave was a good choice for the course because the coding did not in any way get in the way of learning what the algorithm does and how and when to use it. There was a small learning curve for Octave but it wasn't that bad. Now that I understand the algorithms if the need arises to express them in C or SQL, I can do it.

In terms of being shown what to do, the instructor, Andrew Ng was top notch.

[naturalist.atheist]

These classes along with Khan Academy is discussed here:

Reinventing Education with Khan Academy and AI Class - YouTube

[naturalist.atheist]

This Sunday at midnight PST the class will be over. I've got everything done and I am wondering how well I did.

Be that as it may, I have enjoyed the course thoroughly and recommend it to anyone with some math background in linear algebra and calculus.

I think this course will be looked back at in the coming decades as a game changer.

[naturalist.atheist]

I'm watching these interesting videos from an ML summer camp.

This talk makes some interesting observations about actual learning and machine learning.

What is cognitive science? - videolectures.net (http://videolectures.net/mlss2010_tenenbaum_csfml/)

[kowalskil]

Natural intelligence is influenced by artificial intelligence, sometimes positively and sometimes negatively. Many influences are not even recognized.

Ludwik Kowalski (see Wikipedia)

[naturalist.atheist]

Yeah, the cognitive scientists were saying that without machine learning they were pretty much at a loss for coming up with ideas about cognition that they could test by experiment.

[naturalist.atheist]

Boy, AI is getting all kinds of attention.

David Brin on "So you want to make gods. Now why would that bother anybody?" - YouTube

[ceptimus]

I'm working on the third assignment, and getting a 95.1% accuracy recognizing handwritten digits 0-9 using a simple linear regression technique called Logistic Regression.

It sounds as though Logistic Regression treats the images as a bunch of individual pixels and computes some tables of numbers. These tables can then be used to 'test' fresh handwritten digits to find which one gives the best 'match' and therefore identify the digit.

If I've got that basically right, then I'm surprised it works as well as it does. The way (I think) humans do the task is not to treat the image as a bunch of pixels but try to identify lines - straight or curved - and then use those for the 'recognition tests'.

I would have thought that this would be a more efficient way for a machine learning algorithm to work too: first identify, say, the most obvious straight lines or curves in an image - stop after finding at most, say, four of them. These lines/curves can then be summarized by just a few parameters each: length, angle, radius, relative position.

Then you'd be looking for a best match based on maybe thirty values rather than the four hundred in a twenty by twenty pixel image.

Of course there is all the extra work of identifying the lines in the first place, but I would expect this would be balanced by having much smaller vectors to work with thereafter.

I intuitively expect that a 'line based' model would be much better at identifying handwriting than a pixel based one.

What approach is used by current handwriting recognition technologies?

[naturalist.atheist]

My understanding is that the state of the art for character recognition is to use either neural networks or support vector machines. Both of these are applications of a kind of linear regression. The support vector machine uses a kernel which may vary from problem to problem but I am told that it is usually some simple distance weighted inverse exponential function. They both treat the problem as a set of pixels and then solve for a set of linear coefficients using a training set.

Now an interesting thing happens when you look at the coefficient space and internal values. It looks just like it is processing the characters by finding edges and patterns.

If you take a closer look at this image I posted earlier at the images in the left column, what you are looking at are intermediate values of internal neural layers comprised of arrays of logistic regression nodes or "neurons"

http://yann.lecun.com/exdb/lenet/gifs/asamples.gif

And this was accomplished without explicitly programming to detect those features. It just came out in the wash.

The professor was trying to explain what he thought was going on. He thought it important that we know two things about how the brain worked.


Many areas of the brain can change to process a new stream of data. So the neurons are restructuring themselves, they are learning. A person can use their hearing to sort of see with their ears if they have lost their sight.

His simplistic portrayal of the brains neural network as being similar to the networks he showed us in class is where each neuron in the brain is connected to many others and the signal from a given neuron has a learned weight. So a given neuron that is getting signals from many other neurons will have a weight for each connection. And when it learns how to process the signal it has learned what weights to apply. These weights correspond to the coefficients in the linear equations.


When a linear regression model, or a neural network or support vector machine is trained it is learning the weights to apply to make it work.

Now I doubt that the brain is learning its coefficients by applying gradient descent or Newtons method but if the simplistic model has any correspondence to the brain then learning in humans has some mechanism where we get training examples and the weights are adjusted till we get it right. In other words neural networks are trained as humans or animals are trained.

And when a logistic regression model, support vector machine or neural network is trained using a training set, the coefficients are mathematically computed to get it right for the training set. And the larger the training set then the better it will generalize (make an accurate prediction for samples not in the training set). Of course a single logistic regression node has limitations on how complex its representation of the image space can be, which is why with a three layer neural network you can get very close to 100% accuracy.

Obviously there is a lot more to it, but it is an approach that I wouldn't have thought would work unless I had seen it myself and tried it myself. I'm still amazed at it and trying to wrap my head around it.

[naturalist.atheist]

For those of you who have Visual Studio installed you might find this code project interesting.

The author reproduces the AT&T results discussed earlier. You can examine the code and play around with it yourself.

Neural Network for Recognition of Handwritten Digits - CodeProject® (http://www.codeproject.com/KB/library/NeuralNetRecognition.aspx)

[naturalist.atheist]

Another set of Machine Learning lectures. These are more at the PhD level.

http://www.cs.cmu.edu/~tom/10701_sp11/lectures.shtml

[naturalist.atheist]

I found these interesting google talks where the speaker, Geoffrey Hinton, implies that the generative neural network he proposes has a mind. Maybe it doesn't have a mind as much as an imagination. The model he uses has some interesting correspondences with the human brain. It can learn a lot just by processing data with no training. And after doing such learning a little training goes a long way. And that it doesn't see so much as imagine what it should be seeing with great fidelity to reality.

The Next Generation of Neural Networks - YouTube

This one covers the same network two years later. It now imagines in video.

Recent Developments in Deep Learning - YouTube

[naturalist.atheist]

Last century was the century of things that make us go and connect. This century is the century of things that think and go and connect all by themselves. AKA Skynet.

A Swarm of Nano Quadrotors - YouTube

[naturalist.atheist]

For around $150 you can buy a neural network on a chip that has more power than 90 of the fastest DSPs and is programmed by training it.

Cognimem Technologies, Inc. (http://www.cognimem.com/products/chips-and-modules/CM1K-Chip/index.html)

Gaze tracking using Cognimem and Freescale i.mx53 Quick Start Board - YouTube

CogniMem Technologies Inc. • Applications: Video Surveillance (http://www.cognimem.com/resources/application-references/Toy-Recognizing-Cue-Cards/index.html)

And this is just the sort of sub-system that would be needed for pedestrian tracking from a robotic car.

CogniMem Technologies Inc. • Resources: People Tracking (http://www.cognimem.com/resources/application-references/people-tracking/index.html)

[ceptimus]

Dear ceptimus,

We're sorry to have to tell you that our Machine Learning course will be delayed further. There have naturally been legal and administrative issues to be sorted out in offering Stanford classes freely to the outside world, and it's just been taking time. We have, however, been able to take advantage of the extra time to debug and improve our course content!

We now expect that the course will start either late in February or early in March. We will let you know as soon as we hear a definite date. We apologize for the lack of communication in recent weeks; we kept hoping we would have a concrete launch date to give you, but that date has kept slipping.

Thanks so much for your patience! We are really sorry for repeatedly making you wait, and for any interference this causes in your schedules. We're as excited and anxious as you are to get started, and we both look forward to your joining us soon in Machine Learning!

Andrew Ng and the ML Course Staff
:waiting:

[naturalist.atheist]

Dear ceptimus,

We're sorry to have to tell you that our Machine Learning course will be delayed further. There have naturally been legal and administrative issues to be sorted out in offering Stanford classes freely to the outside world, and it's just been taking time. We have, however, been able to take advantage of the extra time to debug and improve our course content!

We now expect that the course will start either late in February or early in March. We will let you know as soon as we hear a definite date. We apologize for the lack of communication in recent weeks; we kept hoping we would have a concrete launch date to give you, but that date has kept slipping.

Thanks so much for your patience! We are really sorry for repeatedly making you wait, and for any interference this causes in your schedules. We're as excited and anxious as you are to get started, and we both look forward to your joining us soon in Machine Learning!

Andrew Ng and the ML Course Staff
:waiting:

I just got a similar reply from Probabilistic Graphical Models (pgm-class), Natural Language Processing (nlp-class), and Introduction to Algorithms (algo-class). Interestingly the Software as a Service course is not delayed. Probably because it originates out of UC Berkeley. So I think this is specifically a Stanford problem. When students dropped out of the chalkboard course to take the online course it probably dawned on Stanford that this could hurt them financially. And since the ML course was developed at Stanford with Stanford employees and facilities they want to control it. So while the coursera people are trying to negotiate with Stanford to let them start the classes using previous content they are feverishly trying to recreate the content so that it belongs to coursera. Thrun of Udacity probably saw this coming or said to Stanford to go play with themselves and is recreating his courses from scratch and so has avoided the kerfuffle completely.

Now maybe I am reading too much into this delay but if I'm right then Stanford is missing a huge opportunity to montize all of this without having to go to the trouble of creating the content and distribution. They have the accreditation and could add value to coursera's and Udacity's offerings by offering credits for a fee after validating the students knowledge.

[naturalist.atheist]

Just got this email that spells out what is happening at Stanford:

Dear Technology Entrepreneurship Enrollee,

Unfortunately, the launch of my Technology Entrepreneurship online course has been placed on hold, due to delays surrounding copyright and intellectual property clearance issues. We are working on this and I anticipate providing you with an update within the next few months. Until the course launches, I encourage you to explore the material that is available on Stanford University's Entrepreneurship Corner website (ECorner (http://ecorner.stanford.edu/)). ECorner offers over 2,000 videos and podcasts, featuring entrepreneurial thought leaders. Also, please check my personal website (Chuck Eesley (http://eesley.blogspot.com)) for additional updates.

Sincerely,
Chuck Eesley

I would think that a two month delay is probably what it takes to recreate the content.

Here is something that others might find interesting Humanities Education Serves as a Toolbox for Life | Humanities at Stanford (http://humanexperience.stanford.edu/artofliving#videos) and http://www.youtube.com/watch?v=BqKRcbxq7ck&feature=youtu.be

[PopeyesPappy]

Cognimem Technologies, Inc. (http://www.cognimem.com/products/chips-and-modules/CM1K-Chip/index.html)

About 8 or 9 years ago the tiny company I was with took a DARPA SBIR for a wearable target identification and tracking system into phase II. Our prototype used a neural network consisting of 5 Intel PXA255 embedded processors. One handled the video I/O, one for shapes, one for color, one for movement and a control processor. The system ran RT Linux and utilized an intelligent agent based architecture that made decisions based on weighted maps. It worked fairly well with thermal images but not nearly as well with standard color video. Your ZISC chips might have been a big help. Too bad we didn’t know they were out there back then.

[naturalist.atheist]

Yeah, it's pretty cool how much progress has been made in the last 8 years.

http://www.technologyreview.com/computing/38367/