On the computational basis of learning and cognition: Arguments from LSA
------------------------------------------------------------------------
by Landauer 

Here are some notes and questions I wrote down while reading the
paper.

Its main point is to argue that learning from empirical association,
if done right and writ very large, is capable of much more than often
supposed.

"coincidence, co-occurrence, contingency, or correlation"
vs. experiment?

contiguity vs. similarity (Locke's view).  How do we compute
similarity?

"It is intimately related to object identification and recognition,
whether innate or learned, and to generalization, categorization,
induction, prediction, and inference.": Can we make this concrete?

The paper argues that LSA has achieved success with verbal semantics
and the same should hold for vision.  How can we test this?

Is "words vs. words" data really different from "words vs. contexts"?

Does anyone know about "geons"?  If we want to use dimension reduction
approaches in vision, we'll need something like this.

pg. 16: Sure, memory of a passage isn't word for word, but we'll
remember *some* of the words and phrases.  Clearly, there are multiple
components to representation.

Does LSA's representation qualify as a representation?  Do we have
guidelines on what's needed to be a representation?  What should a
representation be able to do?  How can we evaluate it?

pg. 17: By the way, they didn't really test 66,000 dimensions.

pg. 23, "Surprisingly, the field of information retrieval research has
never developed a technology for comparing the accuracy of machine and
human performance in this task, so we do not know whether the LSA
enhancement meets this objective.":  Really?

Scoring similarity is obviously a very important job of a
representation.  But, it is not sufficient for all tasks.  What do we
require of our representation to allow it to be used for natural
language generation?

pg. 35, Chomsky: "barring miracles ... Children must be basically
acquiring labels for concepts they already have."  Them's fighting
words!

pg. 40: "The worst offender, in my opinion, is explanation by positing
rules."

pg. 47, "In Edelman's terms, the claim is that representation is
representation of similarity, rather than representation of structures
or properties. ...  It yields what animals and humans need,
recognition, identification, generalization, categorization, by
computations on the available evidence: empirical association in space
and time."


CLASS NOTES by Eiman Elnahrawy (edited by me)
===========

When we switched from document representation to vision we changed the
dimensions of the representation matrix from (words X passages) to
(objects X images).  Then, professor Hirsh suggested a 3X3 pixels to
correspond to words rather than the use of objects. He mentioned that
a similar approach is used in foreign languages (other than English)
where 3 characters are used instead of words.  I recall one comment
that these 3 characters may indeed identify some specific words.  An
example was the 3 characters UFA and the word manUFActure.

Then, we discussed an outline for representation.  The topics were as
follows.

- Span levels
- Similarities
- Registration/ Unification, matching (This is how to align two
- objects.  Similarities are then based on the alignment.)
- Task Specifity
- Abstraction
- Generalization Via Hierarchy? (There was a debate about this).
- Compositionality