Via Twitter I lately came upon an article on automated personnel selection based on personality traits. Basically the notion of this article was that – for a modern way of personnel selection – we can fully skip judging the potential of people on our own and have it done by AI-based personality tests instead (or more specifically by the author’s own software of course). This is one of the articles I would have actually skipped reading some time ago, as described in my opening post.
During my Ph.D. I delved deep into the world of personality assessment and its application in organizational settings. I thus know that even the smartest and most renowned researchers engaging in this field for decades came to the conclusion that behavior is a function of the person as well as the situation. The ignorance of the respective other position will always lead to an incomplete picture of the complete personality triad of persons, situations, and behaviors. (If you are further interested in this topic, you can find an openly accessible review here). Thus – even before reading, I somehow knew that this article would probably sell personality tests as something they are not and cannot do… and I was right!
However, the article still did a good job for me. I started by looking into the software promoted and found that it seems to be quite successful. Then I started researching the general market on AI-based personality tests for a bit and – wow – the race seems to be quite on here. Thus, I started making up my mind by 1.) compiling a list of popular data sources used to predict personality based on AI, and 2.) reflecting about potential applications of these AI-based personality tests in an organizational context. While the interest in personality tests in an organizational context is nothing new, these novel solutions make it seem like something new and shiny. The aim of this article is to show a more realistic picture of these solutions.
Popular data sources used to predict personality based on AI
Before reflecting on potential organizational applications of AI-based personality tests, I will compile a list of popular data sources used to predict personality based on AI here and will also shortly describe how said AI is used to predict personality.
Digital Traces
AI-based personality tests based on digital traces mainly take a pool of existing data and try to match these with validated personality tests (you can find some examples in the Further Readings section below). The main aim is to reach model fit with these already existing personality tests. This mostly includes matching the Big Five personality model (i.e. Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism) with digital footprints from social media, like e.g. spending records, music preferences, and mobile sensing data. There are even a number of contests regarding the prediction of traits from these digital footprints.
While these models sometimes even now already reach satisfactory fit and might be able to predict personality traits from these digital traces, we have to keep in mind that this is actually all they can do. The results of such personality tests based on digital traces will thus always closely resemble the results of the original test (which is the aim eventually).
Another approach – currently less followed though – is real unsupervised machine learning in unstructured data in order to predict behavior. You can find an openly accessible review of the topic here. This approach is still in its initial stages, but considered a potential future field in order to find new insights into individual differences and their potential effects on behavior. In order to apply these findings in an organizational context however, they might still need theoretical bases and explanations to some extent. Especially how stable these relationships are over time for example, might be of interest in an organizational setting.
Picture-based
There are also companies selling solutions to make personality (and other) judgements based on our photographs. Before we talk about this any further, we might just make a short digression to the history of Physiognomics. While a biological and/or evolutionary perspective on personality is a viable theoretical lens, the thought of an individuals’ face showing their personality can actually be considered (ethically questionable) pseudoscience. For the sake of the complete picture, I will link an article trying to make a scientific contribution with some findings on that here. I am not going to start a lecture on journal quality or significance and effect sizes now (there is a nice episode of Last Week Tonight on that if you want to dig deeper here). But with r = .24 we are talking about roughly above 58% percent accuracy – meaning the results are just a little better than they would have been by using chance. I guess I do not have to explicitly state my opinion on this form of personality judgements to make my point clear here.
Speech-based
Recently also solutions inferring personality traits from language samples have emerged (you can find some examples in the Further Readings section below). Psycholinguistic and prosodic features (like syntactic and lexical complexity, or pitch and timbre) have been used to match and predict the Big Five personality traits. While this is not a new topic, and some researchers have already been suggesting a link between speech and personality for a few decades, the automatization allows for an easier and more explorative way of searching for such connections. First studies even seem to show satisfying matching results in predicting some of the Big Five – but again – the validity and reliablity of these models will have to be analyzed, and theoretical backing is needed here.
Eye-movement
In the field of robotics, studies on personality prediction based on eye-movements have recently emerged. Promising results have been reached in predicting e.g. some of the Big Five personality traits, and also underpinning it with theoretical considerations instead of just finding random variable fit. (You can read an openly accessible study here and also have a look at the respective GitHub library).
Potential applications of AI-based personality tests in an organizational context
Before explicitly looking at the potential usefulness (or sometimes uselessness) of the specific AI solutions, we will need to have a look at how personality in general can be used in an organizational context.
Why might it be useful to know about personality in an organization?
Actually there are more or less two reasons to be interested in personality in the workplace: Predicting future workplace behaviors (i.e. will this individual perform well in general and in a team) and predicting organizational commitment (i.e. will this person stay within the organization). Of course, personality will account for different mechanisms indirectly influencing these final aspects (like motivation, or fit, etc.) – but in the end – these also only are of interest for us in order to find out about future performance and commitment.
The relationship between the Big Five personality traits and general job performance is not an easy one. Only conscientiousness is related to job performance over a different set of jobs. For specific jobs some relationships with other traits could be found, but the general consensus is, that effect sizes are rather small and personality alone cannot really account for much variance in performance differences. (If you are interested, you can find a review here). Moderating mechanisms (like the situation) and mediating mechanisms (like motivation and skills) seem to be more important in predicting job performance – or at least will somehow interact with the Big Five in this regard. While this all is really interesting from a research perspective, for the moment we can just summarize, that you cannot deduce that high values in a trait will definitely lead to certain work behaviors. The same applies to everything related to organizational commitment. Although personality might help to recruit employees with high person/organization-fit, these employees might leave the organization because of reasons having nothing to do with that.
How useful are AI-based personality tests in this endeavor?
As elaborated above, in an organizational context personality tests might especially be applied for recruiting and talent management in order to predict future behavior or tenure. In case you want to adopt AI-based personality tests in your organization, you should actually ask yourself two questions in order to decide on the usefulness of such solutions.
Why is the solution better than a common personality test (which is available for free e.g. here).
This question is especially relevant for the model-fit kind of tests, as they do nothing else than to find satisfactory fit with existing scales. In a recruiting setting having people fill out an additional question battery might not be too big of a problem. Of course, social desirability might be a thing in such a setting. However, it is already known that all candidates will want to look good in a recruitment setting and thus will inflate their answers in a desirable way, but this does not influence the predictive validity of the Big Five (you read further here). Or, to put it bluntly: Since all people somehow lie in personality tests – and especially for answers related to higher conscientiousness and lower neuroticism – it does not really have an impact in the end. However, applying such model-fit kind of solutions might still be interesting for you and of course, you can apply them. You should just keep in mind the low effect sizes of personality tests, the possibility that your candidates will not show enough valid data, and the probability of error based on the respective model fit. Otherwise, you might accidentally eliminate your best candidate.
Although somehow already covered by the last paragraph, some of the model-fit personality tests show a different aspect, which I want to point out additionally. For the eye-movement and speech-recognition tests there are first attempts of applying them in the actual situation. This is without doubt very interesting (and maybe also a bit creepy) from a robotics point of view. Just imagine your future robotic work buddy being able to anticipate your reactions. However, I also see some applications in an organizational context, for example in real conversations, in order to improve the atmosphere and output etc. As elaborated in the beginning – personality becomes interesting when looked at in combination with situations – so that is definitely something I would further observe. Although that does not mean that I would implement the above-mentioned GitHub library in my recruitment process already.
For those tests applying unsupervised machine learning the problem actually is, that they might be onto something really great, while on the other hand, you just cannot really say that by now. There is a vital debate on the use of big data and machine learning in personality psychology (see e.g. here). The problem is, that we cannot even be sure that all of these stable models developed for decades can reliably predict behavior. So I would suggest waiting for more validation and additional theory in terms of such findings, even if they might be really interesting from a science point of view. Especially when deriving person-based decisions – or decisions that have an influence on your organization – from them, I personally would want more certainty and least some theoretical explanations or long-term studies.
What are the potential problems of applying this solution?
The aim of this article is to discuss AI-based personality test developments in regard to their potential organizational applications. Thus, I am not going to dive too deep into societal consequences and ethical considerations for the moment. However, I do not feel comfortable without adding at least some thoughts on that in the end now. So, I guess this needs to be your second question in terms of usefulness.
When applying personality tests in an organizational setting, we should really know what one can and cannot predict based on them (as described above). In order to do so, you should know how the solution does the predictions in order to be able to evaluate its usefulness for your organization. So you should at least be able to check and understand the model behind the algorithm, something also known as algorithmic transparency. And I guess you should also thoroughly reflect a process often described as AI supporting people instead of replacing them, or sometimes also described as AI augmenting human decisions. In order to make good decisions, you should still make up your own mind in the end and also keep it mind that your decision, as well as the AI’s decision can be wrong (although – let’s be honest – that might easier be said than done. I might look at this in a future post, so stay tuned on that 🙂 ).
So – of course – in terms of for example finding out which applicants out of a large pool might be a good fit in general, knowing about their personality might be a nice feature. However, we are talking about real people and real jobs here. And – I promised not to go too deep into ethical considerations here, but allow me this – we already know a lot about the fairness (or rather unfairness) of algorithms. Promoting a product falsely considered interesting to someone based on an algorithm might be one thing, but when it comes to making employment decisions based on such algorithms, this is something else. It also does not really matter here honestly how you derived your variables. In the end, even if we should be confronted with future models predicting behavior with a higher effect size than personality traits, this consequently does not mean that every person showing these traits will definitely show the respective behavior. So deriving person-based decisions from such models will always be discriminating against certain individuals to some extent. However, so are other forms of person-based decisions, which you sometimes have to take in an organization. All I am saying is, that you should keep in mind to act as responsibly as possible.
Further readings (partly openly accessible):
- Alexander, L. & Mulfinger, E. & Oswald, F. (2020). Using Big Data and Machine Learning in Personality Measurement: Opportunities and Challenges. European Journal of Personality. doi:10.1002/per.2305.
- Azucar, Danny & Marengo, Davide & Settanni, Michele. (2018). Predicting the Big 5 personality traits from digital footprints on social media: A meta-analysis. Personality and Individual Differences. 124. 150-159. doi:10.1016/j.paid.2017.12.018.
- Barrick, M. R., & Mount, M. K. (1996). Effects of impression management and self-deception on the predictive validity of personality constructs. Journal of Applied Psychology, 81(3), 261–272. doi:10.1037/0021-9010.81.3.261
- Barrick, M.R., Mount, M.K. and Judge, T.A. (2001), Personality and Performance at the Beginning of the New Millennium: What Do We Know and Where Do We Go Next?. International Journal of Selection and Assessment, 9: 9-30. doi:10.1111/1468-2389.00160
- Bleidorn, W., & Hopwood, C. J. (2019). Using Machine Learning to Advance Personality Assessment and Theory. Personality and Social Psychology Review, 23(2), 190–203. doi:10.1177/1088868318772990
- Funder, D. C. (2008). Persons, situations, and person-situation interactions. In O. P. John, R. W. Robins, & L. A. Pervin (Eds.), Handbook of personality: Theory and research (p. 568–580). The Guilford Press.
- Guidi, A., Gentili, C., Scilingo, E. & Vanello, N. (2019). Analysis of speech features and personality traits. Biomedical Signal Processing and Control. 51. 1-7. doi:10.1016/j.bspc.2019.01.027.
- Hoppe S, Loetscher T, Morey SA and Bulling A (2018) Eye Movements During Everyday Behavior Predict Personality Traits. Front. Hum. Neurosci. 12:105. doi: 10.3389/fnhum.2018.00105
- Orrù G, Monaro M, Conversano C, Gemignani A and Sartori G (2020) Machine Learning in Psychometrics and Psychological Research. Front. Psychol. 10:2970. doi: 10.3389/fpsyg.2019.02970
- Polzehl, Tim & Möller, S. & Metze, Florian. (2010). Automatically Assessing Personality from Speech. IEEE Internet Computing – INTERNET. 134 – 140. doi:10.1109/ICSC.2010.41.
- Schoedel, R., Pargent, F., Au, Q., Völkel, S. T., Schuwerk, T., Bühner, M., and Stachl, C. (2020) To Challenge the Morning Lark and the Night Owl: Using Smartphone Sensing Data to Investigate Day–Night Behaviour Patterns. Eur. J. Pers., 34: 733– 752. doi:10.1002/per.2258
- Stachl C, Pargent F, Hilbert S, et al. Personality Research and Assessment in the Era of Machine Learning. European Journal of Personality. 2020;34(5):613-631. doi:10.1002/per.2257
WorkNotes 