Welcome to the Pipelineers Podcast, episode 103, sponsored by InterSYS Corporation, provider of Poins, the Pipeline Operations Excellence Management System, SCADA, Compliance, and Operations Software for the Pipeline Control Center. Find out more about Poins at intersyscorp.com. The Pipelineers Podcast, where professionals, public heaps, and industry insiders share their knowledge and experience about technology, projects, and pipeline operations. And now your host, Russell Treet.
Thanks for listening to the Pipelineers Podcast. I appreciate you taking the time, and to show that appreciation, we are giving away a customized Yeti Tumblr to one listener each episode. This week, our winner is Tyler Windgard with Southeast Gas. Congratulations, Tyler.
Your Yeti is on its way. To learn how you can win this signature prize pack, stick around until the end of the episode. This week, Charles All Day with Pipeline Performance Group is coming back to talk to us about workload analysis. Charles, welcome back to the Pipelineers Podcast.
Well, I appreciate the opportunity to be with you again, Russell. It's been a while. Yeah, it has. You were last owned in episode 26 when we talked about team training, and we just recently crossed over our 100th episode.
So that means it's been about a year and a half since I had you on the podcast laugh. I'm surprised I've allowed that much time to pass. Well, I appreciate you welcoming back. I look forward to having a discussion.
So I asked you on the talk about workload assessment, and maybe we'll just dive into that. I'll ask, what is workload assessment, and why is it important? Well, workload assessment in the pipeline control room world came about with the control room management regulation. And it became important because companies needed to figure out whether or not controllers had sufficient time to respond to alarms and handle abnormal operation situations while doing all the other tasks required of them.
Maybe you talk a little bit about how many of these studies you've done just to give the listeners a little bit of context. Well, we've done 254, and we've got eight or 10 others in progress right now. And when we first came across this requirement of making sure that you monitor the content and volume of the work being directed to controllers, Dr. Michelle Terranova, one of the principals and I, came up with a method that where we wanted to measure time and task and use the NASA TAS load method, and then compare that to all the different tasks that are being done so that operators would have a measure of whether or not controllers had time to respond to alarms or not.
Right. So the NASA TAS load method, I've heard that talked about, but frankly, I'll be honest with you, I've never actually looked at that. So what is the NASA TAS load method? Back in the 1960s and 1970s, NASA was trying to figure out the workload of astronauts, aerospace pilots, and jobs like that that they were concerned about.
And they tried all kinds of methods. They tried probes and sensors and everything hooked up to people, and that just didn't seem to work. So finally, one of them said, why don't we just ask them about their workload? Ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha What a radical idea, right?
Yeah, no. So they came up with a scale that had six dimensions, mental demand, physical demand, time demand, effort, how hard they're having to work, frustration level, and then their performance satisfaction. So it's been used in all kind of industries all over the world since I came up with the method. And we use a 10 point scale for the NASA TAS load index method.
And the process we use when we go on site to do an onsite workload assessment is that we work with the controllers and we have 10 task categories. And then we also look at the task load index. And so we ask the controllers for every hour of two day shifts and two night shifts to report the amount of time they spend doing discrete tasks, the NASA TAS load index, and controller alertness levels. On the face of it, that sounds kind of abstract.
All right, so let me break it down a little bit to you. Yeah, if you would, that'd be great. So we have these 10 large task categories, operations, in other words, active control task, the amount of time they spend monitoring, whether or not doing active task, for liquids, sampling, and proving task, how much time they spend on log sheet and paperwork, how much time they spend on phone and radio calls, and whom they're talking with, how much time they spend face to face talks, how much time they spend on administrative tasks, which all the other stuff that people have to do in companies. And then we get into the more important areas from a regulatory perspective.
How much time they spend responding to abnormal events, how much time they spend responding to emergency events, and then the 10th category is how much time they get for breaks during the shift. Interesting. So I guess there's kind of two aspects of this. It's kind of helpful the idea of capturing what they're doing, but some of the data that you're gathering here, some of the information going to the complexity of the task, and the frustration experienced in engaging the task.
How do you get to a point where you can analyze that and kind of get to a scientific result? Well, we probably should have got the scientists, Dr. Tarrin, over to participate with us. Maybe that could be a follow-up podcast.
But the way we do it is, like I said, we collect that data from all of the controllers, all of the controllers participate because one of the ways I interpreted the regulation related to workload is and it's the workload directed and expected of each controller. So the method that we use, we want every controller to participate. And so once we get this data every hour from every controller for two day shifts and two night shifts, we've got some statistical processing software. And Dr.
Tarrin overtakes all this data and analyzes it in that software. And we come up with the percentage of time that each controller spends doing those discrete tasks. And we combine all of that data for each controller into the percentage of time, the controllers in that control room spend on those discrete tasks, but also those 10 categories that are named. Right.
And then we take the NASA Task Load Index for a day shift, night shift, weekends, and so forth. And then we conclude that as part of the data. And then we take the controller alertness ratings that we've collected, in other words, from extremely alert to extremely sleepy. And we include that as part of the data too.
And how do you get to the alertness level? Well, yeah, that's a self-measure based on the carolinska alertness or sleepiness scale that was done by an organization in Sweden. And it's a, I think it's got seven points to it from extremely alert to extremely sleepy. And each hour of the shelf, the controller selects one of those ratings.
And then we look at all those ratings and see how many they were across the population. Right. But a lot of this is information provided by the controller, not stuff that you're measuring through instrumentation or that type of thing. It is all information provided by the controller.
Yeah, so it's what I would call a clipboard study. I think a lot of times it's challenging to get consistency when you do it that way. So what it raises for me is what's the challenge of getting to a good result doing this kind of work? Well, it certainly is challenging because you're working with a group of people.
And I know when we first started doing this method, a lot of the companies that we talked with about it said, it was too subjective. They wanted objective data. They only wanted to look at alarm rates, phone calls, all that kind of thing. And so you mentioned clipboard study.
We had one client where we did a clipboard study two years in a row, which is pretty labor intensive, pretty expensive and also intrusive on the individuals who are being measured. Right. So I suggested the third year to this client, why don't we do it both ways? Since you're stuck on the clipboard method, why don't we do it both ways this year and see if these results correlate?
And so we did it both ways in third year and statistically the results correlated. And therefore we were able to move forward with them ever since then using this method that I'm describing for you. At first I had the same thoughts you did. This is not gonna work.
People will overestimate. People won't participate. All that kind of thing. And so as the years went by and we collected more and more data from more and more control rooms of natural gas, liquids, refineries, gas plants, other operations like that, where anybody's sitting at a panel or a skater display doing these tasks, we found that it correlated very well across all of the control rooms that we've done this thing.
So no statistical difference between doing it as a third party with the clipboard versus asking the controllers to self-report? None that we saw and as we collect more and more data, you know, scientists love data, right? Oh, absolutely. That's already talked about anymore is data analytics.
So the more data we've collected, the more assurance we have that is statistically valid. That's very interesting, very interesting. So how frequently do you recommend that somebody do this kind of analysis or study? Well, you've got to do something in each calendar year and I'll take 15 months.
It doesn't specify what you got to do. So the way it works for us is we got clients where we do it every year with them. We got clients where we do it one year and they do something else for a year or two and I have no idea sometimes what they're doing. But then they ask us to come back every third or fourth year and do the method again.
So Charles, that actually kind of answers one of the questions I had, which is can you do this yourself and if you try to do it yourself, what are the challenges? Well, you can do workload measurement yourself because the method we use is not the only method that can be used. I'm aware of companies that use a NASA Taskload Index as part of their workload measurement method. So you can certainly do that.
The other thing you can do is if you wanna use a clipboard method or a self-reporting method, you can do that yourself. I think what we found in working with some companies is their challenge comes from how to analyze all that data and that's what we run across every now and then. Somebody has done something themselves and then they can't take that data and reach a conclusion about whether or not controllers do have time to respond to alarms because that's what the gist of the regulation is for me, do controllers have time to respond to alarms. And so what happens when companies use a method is that they get a bunch of data, objective or subjective or some combination thereof, and then they can't reach a conclusion.
And so now that we've done so many of these, we have benchmarks for what's acceptable amounts of time to spend on those 10 task categories that I talked about. And the benchmark we use is if a control room, their controllers are spending more than 10% of our time responding to alarms and our abnormal operating conditions, they've got a problem they need to address. 10%. That seems to me like a small number.
Well, our benchmark is somewhere around 5.1% responding to abnormal events. Interesting. Because what we found in, I don't know, eight or 10 control rooms, either controllers were having to contend with abnormal operating conditions caused by poor communication equipment and they were spending a lot of time on that. And so one of our recommendations was you need to fix your equipment.
Another place, this doesn't occur very often anymore, but when control room management first came about, a lot of control rooms still had controllers doing a lot of administrative tasks, because guess what? Some of the control room is always occupied. So we can get them to do this stuff. Yeah, it's like every little job that nobody else wants to do that needs to happen overnight, they just throw it into the control room.
Yeah, so that has changed a lot in our experience. Right. I think the other thing that's changed too, at least in my experience is that, when we, before the control room management rule came out, pretty much everybody was in a perpetual alarm flood. Yeah, that's another thing.
Yeah, they had so many alarms coming in that really, and I would also characterize that the mindset was, I'm gonna make sure the controller has situational awareness by making sure he has alarm for that. Versus focusing on how do I build the graphics and how do I display the information in a way that I can see abnormal before I get in the alarm? Yeah, well, the other thing we did as part of our workload assessment method is, we take the alarm data that's provided to us by the client, and we look at the number of alarm occurrences against the metrics that are in API 1167. And then we analyze that too, and we say, well, if your controllers are reporting, they spend an extra percentage of time responding to alarms, and then you're presenting them a number of alarms you're telling us, that are maybe an indication they're not really doing an adequate alarm response.
They just may be doing more of an alarm acknowledgement. Right. So that's another part of the, where we do take the objective data and include that as part of the analysis. That's the kind of thing we're a third party looking in that's seen a lot of control rooms and that's some substantial value, I think.
Well, the other thing we do when we do our first own site, assessment with a company is we do a human factors assessment. We look at the control room environment, the ergonomics, the training program, the procedures, the fatigue management program. We basically look at the whole thing, and we have always have a pack liner consultant and a human factors consultant as part of the team. And we make recommendations for improving those areas.
We capture best practices they have, and then we also as part of the process as a minister, a controller human factors survey to collect data on all of those subjects in our human factors assessment. And so the client gives a separate report, a very lengthy report that contains all these findings, plus it also compares to the benchmarks of other control rooms where we've done these human factors assessments. Yeah, that to me, that's almost a baseline prerequisite before you start doing workload analysis, because the level of complexity of the task and the level of effort to perform the task is impacted by the environment that you're performing the task in. And that's why when we came up with this method years ago, we decided to do both.
And to tell you the truth, the first one of these, I ever did before, before control room management ever came about, I spent two weeks in a control room, day shift, night shift, making observations, interviewing every control room in those two control rooms. And that's how, that's one of the tests we used to see if what we were doing was effective. I wanna ask you a question, and I didn't really tee this up for you, but it's a question that's kind of on my mind. So we have a software tool that we call workload analysis.
And what it does is it monitors the data but you can put into the logbook and the data that's being captured in this data system, and uses that as a way to determine what workload is. I'm curious what you think the value of that would be versus the value of what you're doing in this data. Well, the value of that is it does capture consistently what information is being entered into the system or to the software, and then you can compare that to, I guess, other tasks that controllers are doing. Yeah, I think one of the things that we've seen is that particularly in an environment where the workload is changing, and you tend to see this a lot in midstream, where they're adding wells and they're taking wells off and they're adding systems and systems are maturing.
So the workload changes and kind of moves around the system, if you will, is it gives people an indication of how is the workload changing over time? Yeah, so I think the great advantage is that it's a consistent approach to text place throughout the year, right? Right, exactly. I think you can get that report anytime you want to see it.
And I can do comparisons this week versus last week, this controller versus another controller, this operating area versus another operating area. And I can do more detailed analysis, but one of the prerequisites to do this, right, is you've got to know, well, how much time do I place on these tasks? And that comes from the kind of work that you're doing. So you gotta establish a baseline first.
Yeah, I think one of the things that has occurred to me through the years and seeing different methods and everything, it shouldn't be either or it should probably be both. Right. And there should be something like what we do at some point in time to establish that baseline and to figure out what all these tasks are. And probably then smart companies would take some software like what your company provides or some of the other vendors provides and then use that as an ongoing basis to see what's changing.
Right. So yeah, so I'm thinking this more programmatically and again, this is kind of idealistic, I guess, but if you have a good solid baseline that looks at the level, what is the environment, the human factors analysis? And then I do some kind of baseline study and say here are the key tasks that are occurring. And here's kind of the baseline time required to take care of one of these tasks.
And then you do ongoing capture to see if things are changing over time. And then on some periodic basis, a year or three years, whatever, you do some kind of periodic analysis to see if your baseline has changed. Yes, I think that's worthwhile, particularly for those companies that have people, analysts, within the company that can have time to do that. What you made me think of was we went on site in 2012 to do a control room management audit before FIMSA came and did an audit.
And this was a large control center, had a lot of support staff. And this analyst was showing me how they were measuring workload. And he was telling me he was spending about 12 hours a week analyzing this data. I said, well, buddy, if you've got 12 hours a week to do that, then more power to you.
But most companies don't have time to do that. Yeah, most companies don't have 12 hours in a year to do that. Yeah, so that's the advantage of what we do is it's a one time a year thing that just, and if you do it over time in different seasons, then you've got some outside help, of course, with any outside help, there's additional expense. Yeah.
But anyway, I like the fact that there are software packages that can capture data on an ongoing basis. And it probably could work pretty well as long as either the vendor or the company itself got somebody that can look at it and say, OK, if this is telling me the workload is X, and I can conclude that my controllers have time to respond to alarms and handling the abnormal or emergency situations in my company. In other words, I've got some spare time there that could be used. Yeah, the way we do that is we call, this is maybe a simplistic approach.
But the way we do that is we perform this data capture and do some math, and we calculate, and we can look at this hour by hour or day by day or shift by shift. And we know that there was this much time spent doing work that we tracked, and this much time left over. And that time left over, we call that vigilance time. And what we tried to do is set a target for vigilance time.
And we think a good practice would be 30% to 35%. So the vigilance time would be time spent going through screens, looking at what's happening, understanding what's going on, seeing if you've seen anything that's in that normal condition, and responding to alarms that would be beyond what the normal level of alarm activity is. OK, so that's very interesting, because we would call that same thing the amount of time that controllers have for monitoring. In our benchmark, based on 254 studies is 38.2%.
Ha, ha, ha. Well, I was off about 3%. Dang it, well, 3. Something.
That is interesting, right? It is interesting, because we've never had this conversation before, and it's really interesting to see how that correlates. Yeah, it is. And that's amazing to me.
Yeah, me too. Me too, really. You made me a little nervous there when you're teeing that up. I was like, oh, where's this going?
When you said 35, I thought, I think that's pretty bad to go up close to what we are making for monitoring. Right, right. So that's interesting. That's interesting.
It's just a little bit different, a little bit different, semantic, a little bit different thinking, but kind of coming to the same conclusion. There is one thing that I heard about this week that I'd like to mention. It's OK. Sure.
So, there are companies that use the NASA TASLOD index not related to workload, but they're using it to say whether or not controllers are fatigued or not. And that's two different things to me. If you're going to measure workload, then NASA TASLOD index works fine, and it's an important part of it. If you're going to measure fatigue levels, you need to use some kind of alertness rating or something like that, rather than trying to include the NASA TASLOD index as part of your fatigue management measures.
Yeah, I don't remember the episode. I'd have to look it up. I'll link it up in the show notes for anybody who's interested. But we did an episode with a PhD engineer, a human factors person from Denmark, who had developed a technology that was basically a camera technology and would monitor eye position and face, monitor the face in the eye in particular, and use that to set a fatigue level from a direct monitoring perspective.
We talked about that in some way. I find that kind of technology fascinating. I looked at an eyewear technology many years ago and talked to some control rooms about that and the consistently the answer I got was, yeah, we don't think anybody's actually going to do that. Camera's less intrusive.
But I think it'd be very interesting if you could do a study and figure out how do controllers perform given different levels of complexity, different levels of environment and different levels of fatigue. I don't know if there's an appetite for anybody to fund a study like that, but I think we would learn a lot if you were able to do it. Yes, I think that would be very interesting, Russell, if we could incorporate all of these things into a comprehensive look at control rooms and controllers. I'm not sure we're there yet or with the use of technology, but it would be worthwhile.
Yeah, and I don't know if there's an appetite to fund that type of thing within the industry and nor do I have any idea what level of funding would be required to do it. Yes, I think that's the challenge because I talked to one vendor that supplied equipment like that and it would cost $3,000 a person to have the equipment. I said, I don't know if any pipeline company is going to go for that. Yeah, no doubt, no doubt at all.
Well, look, this has been awesome. I want to ask you one other question, kind of different topic and that's what are you hearing? What's the scope around the industry, around the control room? What is going on that you think people are know about?
Well, I think the most important thing is that PHMSA says they're going to start repeating all the control room management audits in early 2020 and they're getting all their inspection questions together including adding some about cyber security to be used for informational purposes. So if you got a control room management plan, if you've been audited before, you need to get ready for another comprehensive control room management audit. Yeah, that we're hearing exactly the same thing, Charles. What we hear is that PHMSA has been shorter to get back and re-inspect everybody within the next three years.
And I have heard some of our customers are telling us they're getting phone calls and people are gathering information. PHMSA's trying to identify every control room for every OPID that they have in their database. Yeah, I was talking to a control room manager in the Midwest about a month ago and he said they'd already been contacted with possible dates in the first part of 2020 for the auditors to come on site. I guess that means we got work to do.
Happy New Year pipeline control rooms. Yeah, exactly, exactly. Hey, Charles, as always, it's a pleasure. Thank you for coming back.
We need to make sure we don't wait so long but before we do this again. Yeah, no, I'm getting older and older just to tell how much longer I got. Well, that's all we're else going, but sure enough. All right, thank you, my friends.
All right, thank you, Russell. I appreciate it. I hope you enjoyed this week's episode of the Pipelineers Podcast and our conversation with Charles all day. Just a reminder, before you go, you should register to win our customized Pipelineers Podcast Yeti Tumper.
Simply visit pipelineerspodcast.com slash win to enter yourself in the drawing. If you'd like to support the podcast, the best way to do that is to leave us a review in the blessed place to leave us a review is on iTunes. You can find instructions at pipelineerspodcast.com. If you have ideas, questions or topics you'd be interested in, please let me know on the contact us page at pipelineerspodcast.com or you can reach out to me directly on LinkedIn.
Thanks for listening. I'll talk to you next week.