In other words, the intellectual community behind assessment believes that the only way that assessment can be valid is within the confines of a specific organizational structure. Fortunately, they are well prepared to field a team of interviewers and questioners who can gather the appropriate data required to make assessment work in your organization. This approach, which precedes the web, is based on a model of the universe that assumes that you build an hypothesis and then test it for validity. 

According to most I/O Psychologists, this is the only way that one can achieve scientific validity. In reality, it is just the cost structure of the companies that offer this sort of assessment. It was a good approach before the web rendered it obsolete. The problem is that the I-O community has not recognized that things have moved on.

Science has always recognized inductive (generalizing from specific data) and deductive (forecasting specifics from general principles) reasoning. Typical 1990s scientific method used deductive logic to suggest hypotheses which were then validated or invalidates through testing. This is how norms are built in the traditional I-O methodology. Unfortunately, this leaves assessment companies busy defending huge upfront costs in their work while more nimble competitors take more pragmatic approaches.

The web, because it generates huge volumes of behavioral data, opens the distinct possibility that assessment can be induced from specific patterns in the behavioral data from a workforce. It is normal for an I-O psychologist to turn their nose at the suggestion that a tool like the Myers-Briggs can be effective (and we wonder what all that jabber actually means). But, and it's a big but, lots of hiring managers use the tool and the web provides a solid mechanism for supporting its usage through data.

The traditional I-O type would claim that a test without local norms has low validity. A pragmatic practitioner, on the other hand,  would love the idea of a ready to use instrument that could be later validated. The swirl of contradictory opinion and opportunity leaves the traditional I-O practitioner in an indefensible position that must be played defensively rather than offensively. Because the convention holds that data must precede utility, the I-O folks have to defend eternally high upfront costs while the competition can move those expenses downstream. It sad to see institutional belief systems that are nearly intentionally designed to produce failure. 

Solving the impending labor crisis depends, in part, on the development of tools that transcend organizational boundaries. As counter-intuitive as it seems today, making job changing easier, while painful as an attrition problem, makes labor more readily available in the aggregate. The time spent looking for work can readily be converted into productive labor if the barriers to job changing are reduced. I-O Psychologists could be at the cutting edge of this trend if their misplaced belief in outmoded methods were reconciled.