I have been part of some interesting discussion threads on LinkedIn touching on the epistemology of skills. What can we know about skills and how can we know it?
A lot of people are somewhat-to-very dissatisfied with the work big-data players are doing to build skills taxonomies based on user-defined skill terms. What is the value of users simply claiming a skill, without any empirical validation of their claim, or even any clear definition of what the skill is?
This is particularly the case for folks from I/O psychology backgrounds, who have spent entire careers working to bring scientific rigour to the way that work is classified and competency evaluated. Some call for the continued pursuit of skills verification and measurement as the ultimate end goal of all work on skills…in spite of the difficulty of achieving this at scale.
As an enthusiastic empiricist I have a huge amount of sympathy with this position. Big-data skills inference is a wilderness where completely unsubstantiated claims are mushed together by unthinking language processing algorithms into disturbingly plausible aggregated data sets. This leaves all sorts of room for inequity, and seems to be taking us back to a place where invalid signals are used for employment decisions. You could easily argue that the noise is drowning out the signal.
What we are seeing with skill inference at scale is the aggregation of ideographic variables (user-defined skills). This a profound violation of the scientific paradigm, where only nomothetic variables can be subject to quantitative analysis. We can only know that a skill term is nomothetic when that skill is observed, verified and defined. Only then does that skill term become an actual skill which can be used for employment decisions.
However...in my role focused on using workforce intelligence for talent acquisition, I see a huge amount of value in skill data inferred from user text sources. It may not represent a ‘correspondent truth’, but that is not the only epistemic position of value. In fact the value of these inferred skills is that they are not empirically verified. Once a skill is clearly defined and verified, it is effectively pinned in place by the science which created it. It can only change as quickly as the epistemic process allows...and given it can take years to get an article published in a journal, or a vocational standard set, that pace is glacial by contemporary standards.
What we see with the aggregation of these user-generated terms is a kind of fuzzy, emerging truth. It allows use to see around corners and see what is coming. And frankly, given how quickly skills are changing, we may never see some emerging skills formalized. For example generative AI prompt writing has become a very widely used skill overnight, and may well disappear again into the rear vision mirror before it is ever formalized into any traditional frameworks.
My Jamesian pragmatic viewpoint is that it really only becomes important to know exactly what you mean by a skill-term when you are actually talking about a specific work context. Organizations have an epistemic role, which is to patrol the borders of their own skill-truth in the areas that are most core for their operations. That is where both the productivity and the equity impacts will be felt, and where efforts need to be directed. That doesn’t mean we don’t need to be aware of the risks of just accepting the output of search algorithms when using tools like Linkedin to create candidate longlists, and in fact this challenge is shaping up to be the focus of my doctoral studies.
But as a skills acquisition consultant, whose job it is to work both inside and outside the organizational boundary, and reconcile the two, I am as happy operating in the skills wilderness as I am helping patrol the boundaries of the skills settlement.