Some elements of a science of psychology

[Dennis Campbell]

Some Elements and Characteristics of a Science of Psychology

Over my last 50 years or so as an adult, as much by bumbling and persistence as any rational plan, I acquired one undergraduate and two graduate degrees in three different states and was paid for practicing and teaching at various jobs with “psychology” as a part of the title. Out of all that, I formed some professional opinions as to what a science of psychology must include.

It should also be noted that “scientific thinking” as we call it today is a relatively new manner of thinking. Science has evolved into a more formal discipline only over the last two hundred years or so. For most of recorded human history, other forms of thinking have prevailed. Problem solving efforts for most of recorded history have included “divine inspiration,” “revealed truths;” blind guessing, and various beliefs handed down over generations. Many of these still are used. Scientific thinking continues to evolve and change, which bothers many people since “universal truths” are short-lived in science. Most of the technological advances of Western cultures, more than some other cultures to date, has been due to the increase in scientific thinking. The relatively changeless beliefs of some cultures has also resulted in little or no technological advances in those cultures. The same can be said of individuals.

What makes scientific thinking advantageous is that it results in more tangible improvements in human life, such as longevity, health, safety, more control over our physical environment; and a more reliable basis for predicting future events. This kind of problem solving, however, requires that we respond less to our emotions of the moment and more to logical and rational thought. It also requires that we tolerate a greater measure of anxiety and doubt, instead of assuming that what we “know” at the moment is unchanging and eternal. Many people find science difficult, especially when other forms of thinking offer greater assurances of certainty and the absence of troubling doubts.

The following list of assumptions and characteristics reflects some of the elements of science that are ideal goals in the study of psychology. “Ideal” means that sometimes we fall short of meeting those goals, and science has had many examples of theories and propositions that turned out to be seriously flawed. But as long as we subscribe to the idea that nothing is an unchallenged truth, there is at least the chance of continued improvement. By “improvement” I mean greater understanding and predictive accuracy, and greater control over our lives and the things that affect them. “Understanding” means being able to describe “antecedent-consequent event relationships,” and thereby be able to both predict and modify ourselves and our world.

Science, as a last comment here, is fundamentally anti-authoritarian. The dictums of political, religious, financial, social, philosophical and even scientific “authorities” are always questionable and subject to challenge based on objective facts. Science requires that we be assertive and questioning. Passive acceptance of “proclaimed truths” destroys science.

1. Elements of a Science of Psychology

1.1 Skepticism, not blind belief, is a requirement in any science. Ideally, anything and everything that people do is a potential subject for questioning and study. Realistically, psychologists live and work in a society and as people they are influenced by the cultural values of that society. All they can do is try to minimize those biases when it comes to objectively studying behavior.

1.2 Operational definitions. This in turn is defined as the actions by which a concept is made public. Key elements in a theory need to be defined in a way that everyone has the same understanding of what that element means. An operational definition is one way try to assure that others know what a theory proposes. For example, an operational definition of “intelligence” might be the scores on an IQ test; the act, in this case test scores, by which the concept is made public. Some terms, such as “spirit” or “soul” are difficult to operationally define.

1.3 Theory. A theory is a concept or idea that is not right or wrong; it is more or less useful as a basis for making a prediction about the likelihood of future events. For example, “Intelligent Design,” “Creationism,” or “Evolution” are propositions that are to be judged by how well they predict observed events. A theory is only useful if it can generate predictions that can be tested or verified.

1.4 Understanding. In psychology as I understand it, “understanding” refers to being able to specify the antecedent-consequent event relationships of an observed event or behavior. In other words, to what degree are you able to predict consequent (future) events from antecedent (past) events? If it is possible to predict some set of events with a high degree of probability, then it can be argued that we “understand” those events or behavior.

1.5 Probability. Science is a set of probability statements, more than any claimed collection of never-changing facts. Probability is expressed between .00 (no chance) and .99 (something will occur 99% or more of the time). If a probability statement generated by one theory is low, such as for example .01, and one generated by another theory is high, such as .80, then the higher prediction, if verified, suggests that the second theory is more useful.

1.6 Occam’s Razor. This is a proposition proposed by a 17th century Monk with an original religious application that basically says that the simplest theory that accounts for the observed events is preferable to the more complex theory. In science, it is a working assumption that theories should be as minimal as possible if they account for the observed events.

1.7 Epistemology. This is the study of how you know what you claim to know. “Revealed truth,” for instance, has no place in any science since no one can independently verify such a revelation. Because “my father told me it was so,” might be verifiable as to the source of the claimed knowledge, but it leaves the question of how he came to know whatever it is. Science questions the basis of knowledge, but the most important criteria of the validity of some claimed knowledge remains the extent to which that knowledge predicts future events.

1.8 Objective referents. The critical terms of a theory must have objective referents. At some point in the discussion of any theory, there must be terms that can be observed, measured and recorded. Subjective terms, by contrast, cannot be seen and verified by others. A theory must be anchored in empirical data.

1.9 Moral neutrality. A science of the study of behavior might have findings that have moral implications for many, but the study itself is morally neutral. That can be said as well for any science. How scientific results are used by people, or how they like or dislike them, are moral and not scientific issues. A scientist as a person will have moral beliefs that influence his or her behavior as they live in society, but a scientist tries not to allow moral issues to influence the results that the science produces.

1.10 Replicability. Research findings can be replicated by others using the same methodology. The possible of one researcher can be checked by others with a different bias, so that the results are consistent in spite of possible researcher biases and prejudices.

2. Some Questions not appropriate for a Science of Psychology

Given the above assumptions and operating principles, it follows that some questions cannot be answered or even addressed by a psychological science, because of the implications and assumptions of the questions themselves. Some examples:

2.1 What is the nature of Evil? This question has words that require definition, “nature,” and certainly “Evil.” If researchers could agree on an operational definition, or set of them, for “Evil” as well as “nature,” then perhaps the question could be reconsidered, but not as stated.

2.2 Is abortion a bad thing? This is essentially a moral question, a question asking for some values to be elucidated. Science does not determine values, right or wrong, or good or bad; like or dislike, approval or disapproval. Science can offer a definition of abortion and many of the physical and psychological consequences of abortion, but science does not provide a value judgment as to abortion being a moral good or bad.

2.3 Is God real? The term “God” must be defined in operational terms that will more than likely not be acceptable to many. Unless and until such a term can be operationally defined, it cannot be the subject of science.

2.4 Are women superior to men? Assuming “women” and “men” are clearly defined; then the issue becomes the meaning of the word “superior.”

2.5 What was s/he thinking when s/he did that? Thought or thinking is not an objective referent, it is an inferred construct whose only referents are such behaviors as what the person says, writes or does that we consider reflects thinking. That is why psychology is a study of observed behavior, since we cannot directly observe or measure “thinking.”

2.6 Why do people go crazy? It turns out there are perhaps hundreds of operational definitions of “crazy,” many of which describe behaviors quite different from each other. The question as stated cannot be addressed, let alone answered, because the objects of proposed study are so many that antecedent conditions associated with a particular set of behaviors called “Crazy” are not associated with other behaviors also called “crazy.”

2.7 Is homosexuality a choice? Science can describe and define a set of conditions called “homosexuality,” and it can describe known antecedent physical, genetic or psychological events that may have some correlation with the expression of homosexual behaviors, but science cannot deal with the word “choice.” Choice is a term that simply has no useful role in a science, useful as it may be in society, politics, law and religion.

3. Some Characteristics of Science that distinguish it from Philosophy, Religion and Political ideologies

3.1 Science is cumulative, in that it is build on earlier demonstrated propositions or theories to formulate new theories that account for new evidence or observations

3.2 Science is self-critical and self-examining. Given no unquestioned propositions, science is always questioning and changing it’s theories in order to better explain new facts. Many of the premises of political ideologies and religions are essentially unchanged for hundred of years; that relative changelessness is often considered a virtue. Science, because it is self-critical and self-examining, changes dramatically in just a few years or even months.

3.3 Science has a close relationship to applied technologies. As a result, both have significantly improved our longevity, comfort and safety multiple times, as well as increasing our ability to directly affect the world around us. Religion and politics are not as useful in those respects.

3.4 Critical thinking. Science as a discipline and a profession rewards rational, critical thinking and objective accomplishment rather than rewarding adherence to some set of unquestioned beliefs or values.

3.5 Open communication. The practices and methods of science can be followed by anyone willing to learn them; they are not tenets of belief and ritual closely guarded by a select and secretive few elites. Science progresses by a free exchange of information and ideas; it is severely constrained in progress when such an exchange is restricted for political, philosophical or religious reasons.

3.6 Results-based respect. Respect in science is based on the tangible results of any proposition or theory. It is not based on the social status or claimed authority of the person(s) making a claim. The efforts made to arrive at a new understanding can certainly be appreciated, but what counts in the end are the results of that claimed understanding. The same is true in this class. I grade on results, not effort.

3.7 Logic precedes conclusions. In science we observe, describe, think and analyze, form theories or ideas, then we test those conclusions, and maybe then we may have some degree of belief. The conclusions are always subject to further change. In some other forms of problems solving, we believe first, then seek to confirm that belief by selective perceptions and resist or deny contradictory evidence.

3.8 Work and effort. Science requires prolonged, deliberate and constantly questioning work. In this class, assertiveness and questioning are both prerequisites of a good grade, and some acquisition of scientific findings. Some of your viewpoints may be challenged. If so, it is your job to express those viewpoints, as well as to reflect upon them. I cannot acknowledge nor speak to any opinions not expressed in a public manner.

Dennis E. Campbell, Ph.D.

[Angakuk]

Wait a minute! I don't remember signing up to take this class. Is it a required course or an elective?

[beyelzu]

Science, as a last comment here, is fundamentally anti-authoritarian. The dictums of political, religious, financial, social, philosophical and even scientific “authorities” are always questionable and subject to challenge based on objective facts. Science requires that we be assertive and questioning. Passive acceptance of “proclaimed truths” destroys science

Or perhaps, another way of putting it would be that science is incredibly authoritarian, but, rather than investing the authority in some arbitrary figure, person or institution, the authority to declare science invalid resides ultimately in the universe itself.

[beyelzu]

2.7 Is homosexuality a choice? Science can describe and define a set of conditions called “homosexuality,” and it can describe known antecedent physical, genetic or psychological events that may have some correlation with the expression of homosexual behaviors, but science cannot deal with the word “choice.” Choice is a term that simply has no useful role in a science, useful as it may be in society, politics, law and religion.

I don't know if I agree with this. We can certainly meaningfully define choice in some ways that it is commonly use. We could define choice as making a selection of possible outcomes and show how being homosexual is not the same sort of choice as deciding what kind of soda to drink.

I don't think its a good idea to dodge hard or important questions.

1.3 Theory. A theory is a concept or idea that is not right or wrong; it is more or less useful as a basis for making a prediction about the likelihood of future events. For example, “Intelligent Design,” “Creationism,” or “Evolution” are propositions that are to be judged by how well they predict observed events. A theory is only useful if it can generate predictions that can be tested or verified.

I am not a big fan of your definition of theory either. Theories can be falsified at least hypothetically or they aren't a theory. A theory which is falsified could certainly be considered to be wrong or perhaps inexact. Further, I think it is important to underscore that a theory is a framework for understanding a set of facts, that it is robust and has been tested in many ways without be falsified. It is true that it must be able to make predictions, if not forwards than at least backwards ones (the way relativity predicted the aberration in Mercury's orbit)

I

[Dennis Campbell]

Originally Posted by beyelzu

I don't know if I agree with this. We can certainly meaningfully define choice in some ways that it is commonly use. We could define choice as making a selection of possible outcomes and show how being homosexual is not the same sort of choice as deciding what kind of soda to drink.

I don't think its a good idea to dodge hard or important questions.

I am not a big fan of your definition of theory either. Theories can be falsified at least hypothetically or they aren't a theory. A theory which is falsified could certainly be considered to be wrong or perhaps inexact. Further, I think it is important to underscore that a theory is a framework for understanding a set of facts, that it is robust and has been tested in many ways without be falsified. It is true that it must be able to make predictions, if not forwards than at least backwards ones (the way relativity predicted the aberration in Mercury's orbit)

I

Agree with falsifiable, was/am using theory as a conceptual hypothesis that seeks to prove or disprove some relationships. If a theory, e.g. Creationism, cannot predict anything, it is not falsifiable and therefore useless as a tool for science.

This OP was my first lecture in a class in Intro Psych, to provide a basis for future topics' consideration and to set the class tone.

[Dennis Campbell]

Originally Posted by beyelzu

Or perhaps, another way of putting it would be that science is incredibly authoritarian, but, rather than investing the authority in some arbitrary figure, person or institution, the authority to declare science invalid resides ultimately in the universe itself.

That’s an interesting comment I’d not considered before, but it makes sense to me. In the common usage of the term antiauthoritarian it refer to human authority, but a science is authoritarian relative to a set of operating principles as partially exemplified in the OP. I avoid the “science of the mind” title, because to me psychology studies observed behavior, and “mind” is an inferred construct of sometimes questionable use. My intent is to mostly hang around on these science threads, as I’m less interested in the usual endless “flame wars” on religion, etc.

As a point of introduction, I’m retired after 50 years as a practicing clinical psychologist, with a PTE occupation on the side of community college teaching of Intro, social, abnormal, developmental and abnormal psychology. FTE work was in tax-funded social services agencies, hospitals, prisons and private practice.

[beyelzu]

It is a pleasure to officially meet you, Dennis or would you prefer Dr. Campbell? When I eventually get a doctorate, I will no longer have a first name

My name is Grady or beyelzu or bey, I am a nontraditional bio/microbio double major at UGA with a minor in religion. Science is very important to me, and I hope one day to be a college professor and/or do research.

Fwiw, I view the mind as an emergent property of the brain and as such it is going to be something of a squishy concept that is hard to reduce. I am not particularly well versed in the brain. I have at best a rudimentary understanding of neurons. Also I do not envy the kind of restrictions on research that dealing with humans necessarily entails. I can just go out and delete genes at random in H. pylori and see what happens. It is seems so much more difficult when you can't.

I certainly like talking about science. I am interested in both the history of science and the philosophy of it as well, at least to some degree.

I don't mean to be super nitpicky about the definition of theory, but as a biologist I often see it misused and so am particular about it.

We don't have very much in the way of flamewars here at FF, but many of us cut our teeth on such fare at II.

[Dennis Campbell]

Dennis is fine. Only required title in class or the office, neither of which apply here or anymore. Did learn BTW that my education actually began more after the doctorate, that just gave me entre into positions in which more realistic education took place, usually with some pain as always.

[Dennis Campbell]

I have been interested, for many years, in pursuing the description of personality syndromes, apart from the usual DSM types. I’ve come to the position that (1) regardless of etiology, a person once past puberty expresses a quite reliable pattern of behavioral and emotional dispositions that (2) provides a basis for anticipating what they’re more likely to do in future circumstances; (3) aren’t all that subject to change although specific expressions might be modified by education and aging. Psychiatric categories aren’t my main focus, more the kind of patterns suggested by code patterns on the MMPI, PRF, CPI, etc. Some of these patterns, perhaps all of them, have advantages and disadvantages in differing social contexts, and a few are quite problematic in most situations (e.g. Hare’s Psychopath), but most aren’t “abnormal” in any psychiatric sense. The 5-factor model might offer a conceptual basis, but needs refining.

[LadyShea]

From a layperson standpoint, I have found that certain specific personality "types" (controlling, manipulative, narcissists for one exact example) do indeed exist and when you've met one you have a pretty good way to predict the behavior of another. It continues to surprise me that I meet these types repeatedly regardless of location and time, and I wonder why these types seem to have a script, but other people seem much more individualized (to me)? Maybe I have just learned to see that specific pattern of behavior or something.

Anyway, layperson, yeah. Interested though

[Dennis Campbell]

We are to some degree blinded by our own perceptual bias, me included, so I prefer when possible from basis apart from my own mind or views. A work in progress, the development of a comprehensive set of major personality types. I am more than familiar with narcissistic personality disorders, unhappily.

[LadyShea]

From a practical standpoint, I am glad I can identify them as it facilitates avoidance.

[thedoc]

Originally Posted by Dennis Campbell

Dennis is fine. Only required title in class or the office, neither of which apply here or anymore. Did learn BTW that my education actually began more after the doctorate, that just gave me entre into positions in which more realistic education took place, usually with some pain as always.

Interesting parallels here, I attended college for teaching and then taught for 7 years. After that I worked in industry in the fields I taught, so my degree got me those jobs as well. I then found that much of what I was taught and was teaching was BS and the real education started when I started working for real. This has led me to the idea that if a field of study is seperated from the profession, it will diverge over time to accomidate the needs of teaching rather than the needs of that profession. It seems that many fields of medicine have not gone this route, my cardiologist is also a professor at a major university teaching that subject. So I can be reasonably certain that the study of that field of medicine will not stray from the practice of medicine

[thedoc]

Originally Posted by Dennis Campbell

We are to some degree blinded by our own perceptual bias, me included, so I prefer when possible from basis apart from my own mind or views. A work in progress, the development of a comprehensive set of major personality types. I am more than familiar with narcissistic personality disorders, unhappily.

Have you ever read, or seen preformed, 'Equus' by Peter Shaffer?

[Dennis Campbell]

Whatever “system” of personality types evolves, it’ll be defined by the means by which such types are measured; which will themselves evolve and hopefully improve. Whatever, a few areas of human endeavors come to mind that would need to be answered. For example, how does this person characteristically respond when:

1. Faced with some ambiguous situation requiring a decision?
2. When under stress in an unexpected emergency situation?
3. When meeting strangers in an open social group?
4. In relating to others in sexual emotional intimacy?
5. When faced with choices involving delay of gratification for long term goals vs. immediate goals with less delay?
6. When angry?
7. When the social situation calls for dominating others?
8. When having to function for long periods of time alone?
9. When confronted with failure and/or error by oneself

There are any number of such scenarios that could be developed, and the pattern of to those constitutes the syndrome characteristic of that person. The labels, codes, or names are of less importance than the clarity of the answers to such situations. “Sexy” or names evocative of some other attributes should be avoided.

[Qingdai]

1., 2., 5., 6., and 9. are all big problems with my 8 year old. He reacts in outrage and anger, cognitive behavioral psychology has been recommended to me.
Do you have any thoughts or reference materials (ie. case studies) that consider CBT and young children?

[Dennis Campbell]

Originally Posted by LadyShea

From a layperson standpoint, I have found that certain specific personality "types" (controlling, manipulative, narcissists for one exact example) do indeed exist and when you've met one you have a pretty good way to predict the behavior of another. It continues to surprise me that I meet these types repeatedly regardless of location and time, and I wonder why these types seem to have a script, but other people seem much more individualized (to me)? Maybe I have just learned to see that specific pattern of behavior or something.

Anyway, layperson, yeah. Interested though

You’re quite right, one use of having some sense of a personality is to avoid that person (e.g. narcissist, psychopath). Change isn’t really a realistic option. Another use is management, in terms of placement in occupations or MOS. Another might be to then conduct research on possible antecedent causes, but that’s a huge task. As an aside, I have collected MMPI profiles from 10 people and then done it again with 7 of those across 20 years: those profiles were readily identifiable with the earlier ones, with the codes remaining almost unchanged. I suspect that corporations, when considering upper management, or the military when selecting candidates for training (submariners, pilots, SEALS, etc) already do this.

[yguy]

Originally Posted by Dennis Campbell

Skepticism, not blind belief, is a requirement in any science. Ideally, anything and everything that people do is a potential subject for questioning and study.

Well how very handsome. When may we look forward to seeing psychologists question the value of psychology?

A theory is only useful if it can generate predictions that can be tested or verified.

Certainly persuading people to think of themselves as animals who ought to heed the pronouncements of authority figures rather than the insight which distinguishes them from animals does much to further that end.

[lisarea]

I have a real problem with people diagnosing others with psychiatric conditions.

There are a few obvious cases where someone's behavior merits treating them differently. E.g., if someone is talking about being implanted with tracking devices because of political reasons, I'd treat them much more harshly and dismissively than I would someone who was talking about being implanted with tracking devices and also compulsively rhyming.

Most of the time, though, I'd think that simply responding to a behavior is sufficient, without trying to diagnose someone with a condition. My personal diagnosis of 'asshole' is sufficient for my purposes, and if someone is being manipulative, I don't really have a horse in the race of why they're doing it. The behavior itself is sufficient reason for me to cut them off, regardless of why they're acting like that. What does it matter why someone is just, say, acting like a sociopath or something vs. actually being diagnosable as one? Except in very limited cases (that a layperson isn't qualified for), simply recognizing and responding to a recognizable pattern of behavior is all that's needed.

I know one person who has Narcissistic Personality Disorder straight up, and his behaviors are so extreme and so consistent, I really don't think I've ever known anyone else I'd ever lump in with him, even though I've seen other people who sometimes behaved similarly.

Even when it's not actively damaging to diagnose people casually (which it often is), I just don't see that it's ever necessary. And it super-creeps me out when corporations and such use psychological measures for their employees, particularly since they all seem to be administered by creepy hucksters.

[LadyShea]

Just FYI, I carefully avoided a diagnosis and simply stated a group of behaviors/traits I have frequently seen together. I don't care if they have an actual named disorder or not, since I have learned to identify "that type" I can avoid that person and if that's not possible take steps to protect myself and mineown.

Various types of assholes aren't a problem for me, it's that combination of manipulative, controlling, and self absorbed that I really want to keep at a distance (especially if they are also gun nuts). I have found their patterns of behavior similar enough to have actually made accurate predictions about their behavior.

[Dennis Campbell]

Originally Posted by LadyShea

Just FYI, I carefully avoided a diagnosis and simply stated a group of behaviors/traits I have frequently seen together. I don't care if they have an actual named disorder or not, since I have learned to identify "that type" I can avoid that person and if that's not possible take steps to protect myself and mineown.

Various types of assholes aren't a problem for me, it's that combination of manipulative, controlling, and self absorbed that I really want to keep at a distance (especially if they are also gun nuts).

I'm not, as posted earlier here, thinking of "Diagnosis" in the sense of the DSM, but patterns of behavior. It is true that having one trait, say, being passive-dependent, makes it more likely that such person avoids decision-making, avoids being alone and others.

[lisarea]

Originally Posted by LadyShea

Just FYI, I carefully avoided a diagnosis and simply stated a group of behaviors/traits I have frequently seen together. I don't care if they have an actual named disorder or not, since I have learned to identify "that type" I can avoid that person and if that's not possible take steps to protect myself and mineown.

Various types of assholes aren't a problem for me, it's that combination of manipulative, controlling, and self absorbed that I really want to keep at a distance (especially if they are also gun nuts). I have found their patterns of behavior similar enough to have actually made accurate predictions about their behavior.

Yeah, I think that's a really good use. I'm just saying that I think there's a big difference between responding to something someone does vs. responding to what you think they are. It's one thing to steer clear of someone who is being manipulative and disrespectful, and even to see similar behavior patterns in people who exhibit certain behaviors.

That's the main reason I take issue with workplace 'personality tests.' They're not based on performance, but on sloppy and largely unsupported generalizations and predictions, which I suspect are used to skirt equal employment regulations.

[Dennis Campbell]

I am assuming, based on professional experiences and education, that there are a finite number of syndromes that will represent 90+% of the human race. At a guess, somewhere less than 100. That’s again, just an assumption. We all have basically the same range of temperaments based on physiology, we all live on the same planet. One psychiatrist, whose name and book I’ve lost, laid out some 85 personalities, with 15 or so basic types, but that was just arm-chair speculation. If and as we develop from perhaps an instrument such as the MMPI, PRF, CPI etc., or some other procedure a useful and reliable collection of syndromes, that number will be what it is. Someone like Jeffery Dahlmer, who I met, or Casey and other serial killers are quite rare and unique, and fit no DSM category well.

[Dennis Campbell]

That's the main reason I take issue with workplace 'personality tests.' They're not based on performance, but on sloppy and largely unsupported generalizations and predictions, which I suspect are used to skirt equal employment regulations.

Depends on the test, the research behind it and the applications. There are well-based tests wrongly applied, poor tests badly applied and some good procedures well applied that are quite accurate. "Quite accurate" meaning a correspondence statistic between say .75 and .85 from test prediction to outcomes in real life.

By the way, “validity” as I use it applies to the statistical correlation of some test result with some other measureable observation, either concurrent or in the future. That is also expressed between 0 and .99.

Perhaps an actual example would help. In the 1960s Peace Corps candidates were assessed prior to placement in some location. They were subject to (l) a psychiatric interview, (2) a series of psychological tests, and (3) ratings from each other, as they lived and trained together. Those results were correlated with actual outcome measures such a dropout rate, terminations, and complaints. The best predictor was peer group ratings, the 2nd best was tests and the worse was psychiatric interviews. So the “goodness” was defined by those statistical correlations.

[lisarea]

What would be a good test, and how would its results be measured excluding external factors? And what are the goals? To fit people to career paths or to fit career paths to people?

It seems as though you'd need some really broad, long ranging double blind studies to get anywhere close to measuring results; and then you'd have to measure those results against results of simply asking people what their goals and interests are.