Dr. Kurt Kraiger did his sabbatical with Dr. Eric Surface at ALPS Solutions in Raleigh, NC for research on both training and job analysis.

We interviewed Kurt to get to know how his research intersects with the research ALPS does, and how their partnership will challenge the Learning + Development world.

“You take a learner who doesn’t know a lot and then you say “how are you going to get that knowledge?” They don’t know what they don’t know. So, that’s kind of an interesting problem space, but it’s clearly really applicable to a broader range of training issues.”

Jess: Thanks for taking the time to chat with me today, Kurt. Let’s jump right in, you’re a professor at Colorado State. What do you teach there?

Kurt: I teach in the psych department. I teach a combination of graduate and undergraduate classes. Interestingly I do almost nothing in training at either level. I also teach a little bit of online courses; I teach a capstone course for master’s students.

Jess: Busy guy! As well as teaching you have a business…

Kurt: Correct, JobZology: We are the science behind careers that fit.

Jess: Great, so it seems that you, from the research I have done, equip hiring personnel with the tools they need to find the right individuals for their business. Is that correct?

Kurt: Our principal markets are universities and colleges, career centers, individual colleges or departments. We give them tools they can pass to their students to help the students figure out what they want to be, what they’re going get up in the morning for and get excited to study and what their profession may be. And, then through that we’re hoping to work with schools to have them help us identify who their affiliated employers are so those employers that normally hire their students would get the bonus of not just hiring someone who has the degree in engineering or accounting, but who is also a good fit for the work and for their company culture.

Jess: I see! So …you’re equipping education centers with these tools, and hiring the right people. How do you see that affecting the outcome of training within the organizations?

Kurt: Here’s one example. We did a validation study for one of our clients. They provide about 75% of the background investigations to the US government, so they’re hiring a lot of bright people, sometimes a lot of ex-military, ex-FBI, ex-police people, who kind of dig into people’s past and find out things about them. After hire it’s about paying them – it’s about 4-6 month hire period before they start training. The training’s pretty rigorous, and they’re investing tens of thousands of dollars in these people--and then they quit training. They drop out for several reasons; sometimes it’s too hard, but usually it’s not what they expected. So, what we’re able to do is use our tools to screen individuals and what we’ve found is that the better they fit the job or the culture the more persistent they are in training. They know what they’re getting into and the people who say ‘yeah I would love to do this’ are much more willing to persist and hang in there when the training gets tough.

Jess: That’s exciting that you have research that affirms your work. Well, you know here at ALPS we work with organizations to help them succeed in their trainings by giving them insight into their data. How do you see your research overlapping with ours?

Kurt: My research goes in a couple different directions; Historically one of the first things I was interested in was how do we evaluate training? And I’m not sure that that’s a research question but I’ve always been kind of interested conceptually and practically about the best ways of evaluating training. From working with ALPS over the past couple months I’ve had a lot of opportunities to talk with Eric and there’s a lot of overlap between his thinking, the approach that ALPS takes, and what I think, and one of the things that we keep talking about is a lot of non-overlap between the kind of vision we have for evaluation and what current practice is. But we also know you read all the time that many companies are unhappy with their evaluation programs. They keep doing the same thing over and over again but they’re not particularly happy with the outcomes of doing that evaluation. We’re excited about writing about and talking about how to better evaluate training programs that have impact. And so, that’s been a passion of mine for over 20 years and I think that really aligns with the ALPS approach to evaluation.  

And my other research interest is a learning perspective which I call learning in ill-structured environments. A structured learning environment is when you have an instructor in the front of a room that says here are five objectives in this training program; I’ll tell you when you’ve got content and I’ll tell you when you’re ready for an exam. An ill-structured environment would be any situation where expectations are not particularly well known- it may not be articulated. You could think about an online learning environment, you could think about a mentoring situation, you could think about field training; those are all situations where there may not be clear objectives. The learner has a lot more control over deciding what to study, how to study, and one of the things we know generally in literature is that when you give them that kind of control they usually make a lot of bad decisions. You take someone who doesn’t know a lot and then you say “how are you going to get that knowledge?” They don’t know what they don’t know. So that’s kind of an interesting problem space but it’s clearly really applicable to a broader range of training issues.

Jess: Very interesting, your two areas of interest in research are training evaluation and ill-structured learning events, which both complement the research ALPS has done on training. I’m curious, and this is my last question, but is your research what introduced you to Eric?

Kurt: (laughs) Yeah I think Eric approached me after a talk that I had done at SIOP probably on one of these themes and he was either in graduate school or just out of graduate school and was really interested in some of the same things that I was – the evaluation piece, some of the learning stuff – and so the first time we met he had come up and talked with me about some of the things that I was doing that he liked, and some of the things that he was doing, so yeah it came through that research.

So, my friendship with Eric, apart from the kind of overlap in research we talked about, another common bond is that he started his earlier company much, much before I did. He’s done a lot of work with the iteration of the company now. I think, compared to a typical academic, that I have more insights on running and building a business. Compared to most small business owners he has more insights on kind of the science side. So, I think I’ve appreciated that as an aspect of our relationship; that we kind of were in the same space asking, “how do you do good science but how do you build a business”? He [Eric] has a really unique perspective that I value. And I think there have been times that I’ve been able to help him too with my insights.

Jess: That’s beautiful, and your friendship, with the combination of your expertise, your experience, and passion on the matter have a real potential to grow and steer the training world in a direction where businesses, trainers, educators, and learners can be more efficacious.

Kurt: Exactly, and that potential is what keeps Eric and I passionate about delivering effective results and seeing organizations succeed on many levels of their training.

Dr. Kurt Kraiger + Dr. Eric Surface will be collaborating on many upcoming projects designed to impact the Learning and Development world. Follow them on LinkedIn to stay updated on their projects and research.

Written by: Jessica Perez, communication manager at ALPS Solutions: a company looking to shake up the Learning and Development world through efficacious analytics and knowledge transfer to improve the impact of training + learning.

Viewing Language Assessment through the Lens of Training Transfer

Organizations want to leverage investments in language learning. Language proficiency tests are often used to assess both acquisition and transfer. Research suggests proficiency tests might not provide sufficient information about job performance. Options for selecting and using language assessments in the work context are discussed from a training transfer perspective.