Welcome to the Pipe Liners Podcast, Episode 11. The Pipe Liners Podcast, where professionals, public eeks, and industry insiders share their knowledge and experience about technology, projects, and pipeline operations. And now your host, Russell Treet. Thanks for listening to the Pipe Liners Podcast.
We appreciate you taking the time to listen to this episode and to show our appreciation, we want to let you know about our signature prize back. We are offering a free customized Yeti Tumblr to one listener each episode. So how do you register to win? Simply visit pipe linerspodcast.com slash win.
That's pipe linerspodcast.com slash the word when to enter yourself in the drawing. This is our way of saying thank you for being part of the show. We are excited to announce our latest winner. Congratulations to Travis Freasenheim.
This week we have as our guest Scott Williams with Interscorporation. Scott is the manager of development and SCADA at Interscists. He has 20 years experience building SCADA systems from scratch in multiple different applications and in multiple technologies, all in oil and gas. So Scott, welcome to the Pipe Liners Podcast.
Thank you. It's really good to have you. And for our listeners benefit, we kind of put this up front. How long have you and I been working together?
Oh, let's see. It's been a while. I'd say, well, yeah, certainly over 20 years. You know, my kids are in high school now and the kids weren't around when we first we first got started.
Yeah. So we have been you and I have spent a couple of decades plus doing SCADA and measurement projects. And what we'd like to do today, what I'd like to do today is have a conversation that maybe others that are learning this business could benefit from. And maybe they can learn from all the things that we've done wrong.
Although I don't know we get all that in a single podcast episode. It's just been a fair bit of that. Yeah. So I guess the first question I'd like to ask is, you know, you've been doing this a long time.
So what do you think is most important when you're when you're looking at doing a major upgrade or a change out of a SCADA system? What do you think is the key thing you need to worry about upfront? One of the first things you want to do when you're evaluating your old system, your existing system as part of an upgrade is to get a handle on what you have because just looking at it, it may look like this huge task. When you start digging into it, there's a lot of similar pieces in that entire system.
So one of the first things we'll do is you look at how you want to organize your new system. What we tend to use is a hierarchy based organization. We call it our area of operations. So you may have a West operating region and you may have some facilities out there and you may have this call to gas gathering system.
These all are sub areas within what I say, a system. And you know, at a particular facility, you're going to break that up based on how you operate. So you may have the inlet metering in the in the process. I use a gas gas storage because that's pretty common.
A lot of people are familiar with those. So you have your metering coming into the facility. You'll have your compression processing the product, putting it into the ground. Then you'll have your pressure regulation heaters for when you're taking it back out of the ground and then output metering.
So you may break those up into areas. Then within those areas, then you start looking for common equipment. You know, heaters are a pretty common thing. You're going to have valves all around that facility.
You have the meters at the end of the out and so on. Once you've identified that key equipment, then you start thinking about what are your different variants because you're probably not going to have very many. You know, consider a meter if for the most part, if it's a gas meter, you're going to have pressure temperature and some other flow related field, maybe differential, if you have an orifice meter. If it's liquid, you may have a couple of different fields, but meters are pretty common things regardless of the type of equipment you have.
Valves are another really big one. You're going to have your inner facility. You'll have valves for safeties that'll be sitting at a particular position until you have to execute an ESD or some sort of emergency shutdown and then you're going to open those valves. So those are simple one function valves.
You'll have other valves that control the flow throughout the facility. So you open or close those. And lastly, you'll have your common control valves, how much gas you run into the heaters or your pressure regulators. And so there's three different variants of valves, but after you've got those variants, then it's really a matter of which kind of actual physical valve corresponds to which of those variants, which those models for a valve.
So I guess the question that I'm trying to is we're having this conversation and you and I've had this conversation a lot in-depth, but I'm trying to think about how somebody who maybe is new to this might be thinking about that. And the question that comes up for me is, well, why do you need to organize it that way? I mean, really, it's just a bunch of points, right? You know, I just it's just a point to open a point to close and a point or more that tells me what the position of the valve is.
Why do I got to classify valves by type, like shutdown versus lineup versus flow control? Well, compare a control valve to a just an open close, you know, valve that you use for control flow out through your facility. So your open close valve, let's say you'll have an open command to close command, an open limit switch and a close limit switch, just your very basic valve. For a control valve, you'll have a manual position set point with that.
You'll tend to have an auto manual control and then you'll have whatever process variables that tie into that control. So you'll just just looking at those two, there's this fairly different bits of data associated with each to the root of the question. Why is that important? Well, I can look at my drawings and I know where I have valves and it should be pretty straightforward to identify.
This is a control valve that wins an open close valve. Then when I go look at my data, as I'm sorting through my skated data points, you know, just the bucket of points, I can identify that, okay, here's valve 542. And here's the open limit switch, but I don't have a close limit switch because I've normalized my standard stuff. I know what points I'm expecting to find.
If I don't find those points, then I have to dig deeper and figure out why maybe I didn't even have that point in the first place, which suggests an error in your skated system, or maybe it wasn't an open close valve in the first place. It was a shut down valve. This is the stuff that always drives you crazy when you're trying to, you know, get organized to do a project, right? Is getting an understanding of the data and just because you got a valve on a drawing and you've got points in a PLC, you got to make sense of all that, right?
And to the extent you can get through that quicker, the better. You know, I think the thing that tends to drive me most crazy, you know, when we're doing projects, it's not the, it's not the points that are missing. It's the ones that are there and we don't know where they go. Oh, yeah.
Because you're always worried, is that something that's really important to the process? Is that something that's important operating information? And when you start organizing these things by what they're used to do from an operating perspective, because in a lot of cases, you know, open close valve for a lineup in the PLC, it's the same thing as an emergency shutdown valve. The points are the same, but the use is completely different.
So, you know, the other thing about creating these standard, standard valves by their purpose is now I can start getting the alarming and the alarm rationalization standardized, which, you know, we talked about this in an earlier episode with Matt McClure about, you know, his biggest challenge in standing up control room is always in the area of the alarming and the alarm management. And part of the reason is there's a disconnect or the difference, maybe a better way to say it between people who are building the automation logic and people who are understanding how to operate the process. That's a different view to the data, if you will. So it kind of raises a question.
So in your experience, right, when working with a legacy SCADA system, an older SCADA system, and you're doing an upgrade project, what do you think the typical error rate in the data? And I'm the finding error as a missing point or a misspelling on a point or a point that's assigned to the wrong place. What do you think that the error rate typically is? I'd say if it's a SCADA system that's been around for a while, and it typically is when you don't upgrade or change your SCADA system after a couple of years, you would have to have five or 10 years because it's painful and it takes time.
So with that kind of age, the likelihood that you have an incorrect point is pretty low. However, that doesn't necessarily mean that you don't have extraneous points because that's extremely common. That's just age. I would imagine if you were to study, you could say for every number of years, this SCADA system has been around.
You'll gain this percentage of points that don't do anything anymore. There may be controls that have never been stroked for whatever reason. You know, that's not supposed to happen, but it doesn't mean it doesn't. Yeah, just if I might add, there can be a distinction between the points the controllers are seeing and the entirety of the points that are actually in the SCADA database.
I've seen situations where, you know, it looked like people started to build a meter and didn't finish and then they started to build the same meter and they didn't finish and then they built a meter out and you come back behind them and you're looking at all the points and like, looks like this meter's in here three times. Yes. I've seen that before. The other thing that tends to make a pretty big difference into how your ability to apply consistency for the new system is the nature of your automation.
So if you have a lot of say, fixed function, R2Us, like a rock flow computer, things like that, those tend to have a pretty standard set of points you're going to be looking at and they're the same against across the different rock versions. So if you have a bunch of rock meters, they're probably all right and they're probably all the same and you probably have all the data. But now you go look at PLCs, there are some companies that do a really good job of using consistency in their PLC loads across their different sites. Those guys are sitting pretty for SCADA upgrades because at a very high level, the PLCs are all similarly put together.
Then you have systems that evolved over time. So you have different vendors doing the PLCs and that kind of consistency wasn't enforced. Your risk of finding errors and having errors is much higher in that case. Well, maybe not even error.
I mean, I would want to make a distinction between errors and inconsistencies. Right. Right. So one challenge is an error, but the other challenge is inconsistency.
And it could be inconsistency in naming convention or inconsistency in the way that data is structured or it could be an inconsistency in the way the automation is approached. You know, there's lots of ways that things can get different. Yeah. You'll appreciate it.
I've never started a project like this where people didn't say, Oh, it's all the same. We did it all the same until you start pulling it back and you're looking like, yeah, they're all a blues suit, but they're not on the same size. You know, that kind of thing. I think the other thing that's present for a lot of people doing this kind of stuff is that, you know, if I've got it, if I've got a mature pipeline operation, you know, large transmission, you know, not really, you know, the nature of how it's operating is not changing, then those guys operate.
Their whole thing is risk of verse and optimization. If on the other hand, I'm looking at a midstream operator and they're building facilities and, you know, their ability to get the facility up and running is what's critical then, and I'm doing this with multiple facilities. It's easy for things to diverge. Certainly.
If you have a single facility done first and you can promote a certain consistency or a, you know, here's the model we want to use, that can make a huge difference. Yeah. But if you got multiple sites going up, you're going to have different vendors working on the PLC, those two sites. So keeping everybody on the same page is going to be tricky.
I think Scott, you said it very well that you need to understand what you have. And to the extent you really ring that out in detail up front and you understand what you have, regardless of it's very well organized and it's very consistent or not, you're going to do a better job of executing the project. Let's follow up with this question. So we talked about the need to get the data well organized.
What do you think is the second most important thing? That's a tricky question. This may, you know, kind of follow up the first one, but having the customer resources to help you work through those, that's a definite plus. That's not really what you're asking though.
So I think you're making a good point. And this is one of those areas where you and I kind of come at things from a different perspective. But I think customer involvement, you know, whether you're doing this internally as a skater group for a customer that's pipeline controlled or whether you're an integrator or what, having the user involved up front and through the process, not like full time involved, but you know, so they know what decisions are being made and they understand what's coming is really critical to success. I think the worst thing you can do is, particularly when you're doing an upgrade to something that already exists, the worst thing you can do is not plan for the need to train and facilitate change when you start bringing this up and operating this new system because it's not going to be the same.
Even if you sit down and you're going to try and you're going to copy, you know, it's not going to be the same. The only time it's the same is when I'm just running a software upgrade. Right. And actually that triggered a very good thought for me.
You mentioned having the end user people available makes a big difference. The other thing that will help is when you're defining what screens you actually need. You know, at the point you're building this new skater system, adding screens and doing that is relatively low cost compared to doing it down the road. So think about what the controllers do every day.
Are there screens they have that they never use? Are there screens that combine what was on a couple of different screens in the old system that would make particular tasks, they do a lot easier. So your screens aren't just, this is my system and this is how I've organized it. Your screens also should support what the controllers need to do.
If there's a task that involves what would normally be on a cavern screen and a pressure regular screen and a heater screen, put the key bits on one single screen so they can manage what it is they're trying to accomplish without switching back and forth between the mess of screens. Yeah, exactly. So and to use your gas storage example, you know, gas storage facilities operate basically four ways. They're either injecting into the caverns, they're withdrawing from the caverns or they're wheeling meaning they're moving gas through the header, but they're not really putting gas in or out of the caverns or they're just dormant.
Nothing's going on. So those are the four things, four states they operate in. What I need to see when I'm injecting is different than what I need to see when I'm withdrawing because I'm running different equipment. Likewise, if I'm wheeling what I need to see.
So if you use the ISA 101 model or the high performance HMI model, you talk about level one, two and three screens. What we're talking about here is is building task specific screens so that rather than an operator having to jump between two or three screens to do a task, you put all of it on a single screen so that they can do the task, right? Whatever you do, you won't, you won't something that supports organizing the equipment into a hierarchy. So it makes these kind of things easier and you want to, you want things that are templated or objects so that I take, you know, I take a vow, I take valves and meters and, you know, things and I put that together and that's a meter skid and then I could just take that meter skid and drop it on a screen screen.
When you start doing that, then it gets easier. But the challenge I think is when it starts getting easy to build screens, the gravity will take us to building a whole bunch of screens that nobody's actually going to use. Having conversations with conversations with the guys that are actually running the system and learning what they do day to day or week to week that is painful in the current system because of the screen layout. Just having that conversation with them can help you find the screens that they may not even know they need it.
I love you for it. Yeah, exactly. I think that's another really excellent point you're making Scott because in my experience working, you know, with the controllers, the guys that are actually in front of the screens doing the work, they are a wealth of knowledge, but you kind of have to mine for it. And what I mean by that is what they tell you they want because they don't really have the context of how you build these things may not be that helpful.
But what they know they need that you kind of have to through conversation and working together, kind of pull out is extraordinarily valuable because those guys are going to have ideas about, well, if I could do this and this and this on the same screen, you know, and you also have to talk about maybe some different ways of looking at the data other than just, you know, vows and PNID diagrams and, you know, trend charts. You may want to figure out a way to combine things in a way that maybe they haven't thought about because they've never seen that kind of screen. Having done a number of systems, you can bring ideas like that to the table. You know, in my career, I've seen dozens of SCADA systems.
I doubt that it would be easy for you to find a controller that can say the same. Yeah, it's a good point. Here's the other question I want to ask. And I'm going to kind of tee this up because I think the third most important thing is how you do the factory acceptance and site acceptance testing because if you do that well, you streamline and simplify the entire project.
And if you don't, that's where you're going to get into the project that never ends. So what's required to do an efficient and effective factory acceptance and efficient and effective site acceptance? The factory acceptance, there's a couple of different things that help a lot. Ideally, at your factory acceptance test, you're going to have the availability of at least of one of each type of field equipment so you can actually look at specific screens against specific devices and see how the interactions work.
At another level, you'll want to be able to walk through the screens and see how tasks are performed. The FAT is about does the basic functionality do what it's supposed to do. But it's also an opportunity, you'll say a last look of is there anything here that will make our life difficult or doesn't need to be? Because once you get in the field and you start installing this stuff, it's kind of too late for that.
You don't want to have those kinds of changes come up when you're in the field. All the servers are planted. People are waiting to start using the new system. So the more that you can get in the FAT before you've gone to the field, the better off you're going to be.
And what about the site acceptance? Under the best possible circumstances, you can have the old skated system and the new skated system both running both live at the same time. That promotes a couple of different things. It lets the controllers get a little hands on of saying, OK, this is what I saw, the old one, this is what I saw, the new one.
It's an additional way to prove out your data. So here's what I'm seeing on the old screen. Here's what I'm seeing on the new screen. In addition to all of that, you want people in the field that can manipulate the IO, can manipulate the field equipment so you can prove it gets to your system correctly.
It's your point to point one on one. But that's critical, especially for controls because it's a new system. I mean, all those are different things that are involved in the ICT, the site acceptance test. And you want at least one or two of the key day to day controllers involved in that, not just the skated supervisor, because that's their first chance to see it with their own data and to start the real adoption process.
So one of the things you didn't talk about, and I think is a best practice, is you should also have a site execution plan. So the site execution plan kind of lays out the process to get the equipment installed, to establish communications and to call commissioning complete. And I'm ready to do site acceptance testing. I think where these projects tend to go is that commissioning and site acceptance testing kind of get blended.
They're not really distinct parts of the process and that can cause thrashing. You know, I go out to a site and I try to get everything commissioned and tested. And, you know, I find a problem. You need to do that during the installation side.
Right. Yeah. Absolutely. Yeah.
You know, that site execution plan, which, which details what you're getting set up, you want to be through that. You want your systems in place. You want to work and you want communications to be up. You want to be able to get to that point and say, okay, this should be working now before you start the actual formal, formal testing.
Communication things are going to come up anyway, but you don't want to start there. Let that be an exception rather than a, oh, the comms aren't quite working yet. Just get through all that, get all that done, then start your testing. Yeah.
And I think probably one of the most valuable things or that we do kind of in our processes, we review the site execution plan during the factory acceptance test. If you do a thorough review of the site execution plan right after you kind of are finishing up or as you've wrapped up the fat, it's fresh on everybody's mind. And all these questions about, well, how are we going to, you know, how are we actually going to get two sets of consoles where we can see both of them? How's that going to work?
You know, those kind of nitty gritty details tend to get wrung out more effectively, particularly once you're actually sitting in front of the new one and have a sense of what it is and how it's going to be implemented. Certainly. Well, not to mention what field resources you're going to need on site because doing point to points, you need people to be out there, depending on the size of your system, you may have multiple teams. So having that conversation well in advance, let's the customer plan for that, make sure they have the appropriate people available so you can actually get through, you know, any installation issues as well as site acceptance issues when you're on site.
Well, and I think the other thing that's important to understand is the kind of people you need to do installing and commissioning. And I'm calling commissioning. I've got the comms in and I've got the data lit up and I'm ready to start actually doing site testing. Those skills are actually different than the people that are going to go out and hook up to instruments and range them and, you know, support the point.
And those guys want to move fast is generally the people you have doing that, you're taking them off of other duties. Right. So that's part of why you want to get your everything commissioned and working before you start getting them involved because that's the worst thing. They're out there working on something like, Oh, wait, wait, comms aren't working.
Then we get the comms up to that site. They won't be happy with you. Exactly. So we've been talking almost 30 minutes, I believe, and we really hadn't talked at all about technology, but we've been talking about how to simplify the scale of build.
So really simplifying the scale of build, it appears based on our conversation that it's about the analysis of what you have and the planning and the talking and the communication and the teamwork to get it all done. Yeah. That's pretty accurate. I mean, there's multiple skated products and I know which ones I prefer and I prefer them for technical reasons because they support the way I like to build systems.
But there's more ways than one is to skin a cat. If you could tell Young Scott something important that he needs to know, what would you tell Young Scott? I'd say some of the most important things that make the most difference at the end of the day are how consistent you can be building your system, how close to reality you can get your FAT, your factory acceptance test. If there's always a limitation, you're not going to have all the R2s, all the field equipment, but how close you can get to reality for your FAT and then how rigorous and well-defined your site execution plans and your site acceptance test plans are once if everybody's on board with those before you get to them, your life goes so much easier.
Yeah, I think I would tell Young Russell the same thing, exactly the same thing. You know, and the point you made and we probably I'll explore this a little bit is make the fat as real world as possible. People tend to simplify the fat rather than work to make it as real world as possible. And in my experience, the more real world the fat, the better everything else after the fat, the less real world, the more you're going to be sorting out in the field where everything costs more and takes more time and people get irritated.
Exactly. When actually one other thing that I want to told Scott, talk to the controllers from day one. Oh gosh, yes. Their input from the very, very beginning.
I would tell you, Scott, that at this point in my career, I don't think I'd want to do a project where the people in front of the console are not involved from inception. And I don't mean everybody and all the time, but enough so that they feel that their input is heard and put into the project because ultimately, that's who you're building this for. They're the ones that are going to use it. They're going to determine whether it's a success or a failure.
Yeah. I mean, on any controller team, you're going to have a few guys, sort of the old hands or the, the, the recognized voice of the group. You get them involved. Everybody else will be fine.
Right. I agree with you. I agree with you. And again, we talked about this on an earlier episode with Jeremy Coleman as he was implementing a high performance HMI in his control room.
And he talked about the need to have those guys involved in, you know, how you know whether what you're building is good or not is, you know, do they use it or not? If they use it, it's good. If they don't, it's not pretty simple. So, well, look, I think this has been great.
I know that we could probably unpack this some more, but this is probably a good time to bring it to a close. I'd just like to say thank you very much. I appreciate it. And we'd like to have you back at some point in the future.
It's been fun. I look forward to it. I hope you enjoyed this week's episode of the Pipewiners podcast and the conversation with Scott Williams. I certainly did.
Just a reminder before you go that you should register to win our customized Pipewiners podcast Yeti Tumblr. Simply visit pipewinerspodcast.com slash win. That's pipewinerspodcast.com slash the word win to enter yourself in the drawing. If you have ideas, questions or topics that you'd be interested in, please let us know on the contact us page at pipewinerspodcast.com or reach out to me directly on LinkedIn.
My profile name is Russell Treat. That's our USSEL with one L and treat just like it sounds T-R-E-A-T. Thanks again. And I'll talk to you next week.