In recent decades, governments have increasingly adopted “best value” procurement as a more nuanced approach to purchasing goods and services, compared to the traditional-and still widespread-“low-bid” approach that selects suppliers based on cost alone. Rather than choosing a supplier based on lowest cost, the best value model allows decision makers to select for the best combination of cost, quality and an array of other considerations.
In 2009, the state of Oklahoma began implementing a type of best value procurement known as the Performance Information Procurement System (PIPS), developed nearly two decades ago by Dr. Dean Kashiwagi and promoted by his team at Arizona State University’s Performance Based Studies Research Group (PBSRG). Oklahoma’s Office of Management & Enterprise Services has been working with PBSRG as a research partner on implementing PIPS, which is based on identifying and leveraging the knowledge of experts in the supplier community to provide clients with a plan to accomplish their objectives.
Reason Foundation Director of Government Reform Leonard Gilroy recently interviewed Steve Hagar, Oklahoma’s State Purchasing Director, on the state’s use of best value procurement, the PIPS model, successes seen thus far and much more.
Leonard Gilroy, Reason Foundation: Can you explain the difference between best value procurement and the more traditional low-bid procurement?
Steve Hagar, State Purchasing Director, Oklahoma Office of Management & Enterprise Services: There’s a contrast between a best value process and a more traditional low-bid environment with a specification driven by minimum requirements or qualifications. In a low-bid environment, the state will establish a minimum requirement where we know we will get at least this much, and we may get more than that because the suppliers trying to get our business will probably offer us more than we would expect based on the minimum.
From the supplier perspective though, knowing that it’s a price-based award, they will see the minimum requirement as more of a maximum they can offer without pricing themselves out of the competition. They understand that if they bring their best people to the project and offer everything they can offer, they will price themselves out. So what a low-bid procurement does is cause the supplier to compress their expertise-they’ll opt to use junior people over higher performing people in an environment where they’ll be told what to do by the client. They can just send their junior staff in knowing that the state is going to be managing, directing and controlling them, locking them in at the minimum price and telling them how it’s going to be done.
That’s at the opposite end of the spectrum from a best value environment, where we encourage suppliers to use their best people and tell them that we will pay for their best people because we know that if it’s done right the first time that it will be cheaper in the long run. It allows suppliers to bring in their best people knowing that they will be in charge of the process and won’t be told what to do. There’s nothing more frustrating than for an expert to come into an environment and not be able to use their expertise due to a client that’s very managing and controlling.
We want to create an environment through the best value process that allows them to come in and actually utilize their expertise, so that’s why we work to make sure that the clients-the state agencies-understand that the supplier should lead the way. They’re going to tell us how they’re going to get there, so ask any questions you want but don’t revert back to managing and controlling the supplier, because it will drive an expert crazy.
Gilroy: What is the PIPS system?
Hagar: PIPS, or Performance Information Procurement System, is a version or hybrid of best value procurement, which has been around for awhile and is really nothing more than saying, “we’re going to tell you what we need and you tell us what you’re going to do to achieve our objective.”
Dr. Dean Kashiwagi at Arizona State University, who is the founder of PIPS, took that philosophy and expanded it such that it leverages past performance in the PIPS process to identify expertise. What he’s done in the process is incorporated elements that make it very defendable, leveraging common sense and natural laws.
Dr. Kashiwagi wrote a book that supports the PIPS process called Information Management Theory that is all about human nature and natural laws, and that how an event will turn out is predicated by the initial conditions. If you can understand the initial conditions, you can understand how it’s going to end. So there’s a lot of theory about all of this, but the bottom line is that the selection is relatively clean and easy to understand.
This best value process allows us to robustly identify an expert in the field, and the process itself utilizes what we call “dominant” information-basically, information that everyone can understand. The process has three phases, which are selection, clarification, and management by risk mitigation.
In the selection phase, we ask that submissions from vendors be written in a way that everyone can understand, and it’s based on a six-page project capability submission that the evaluation team reviews. The cost submission is shared with the evaluation team after all evaluations are completed. We really don’t get an idea of how they’re going to achieve the objectives until we get to the next phase when they convey a complete scope of work and we can ask all of the questions we need to and bring our state experts in to do a sanity check. What we want is for the evaluators, who are evaluating blindly to avoid any subliminal bias, to look at what’s submitted based only on what’s on the paper. It’s a very objective way to go about it and allows us to have a selection process that’s very defendable. Of the 20 or so projects we’ve done, we’ve only had three bid protests, which were all denied without even progressing to the second phase of the protest.
After the evaluation teams review the six-page submissions, we bring the vendors in for an interview, which is the last part of the selection process. That’s where we get a sense of who they are, and they’re able to speak candidly. We have the interview with the two or three people who will be working directly with the state and have the expertise to be able to convey to us a potentially complex project in layman’s terms. And if they’re able to lay it out for us that way-with high performance claims and validating metrics-and everyone in the room is able to understand, then they will score a “10.”
The scoring we use is either a “1,” “5,” or “10.” If the evaluators are unsure whether a vendor is a “5” or a “10,” then we encourage them to mark it down as a “5,” which means we really don’t know. “1”‘s don’t happen a lot-that’s when something is demonstrably bad-and we only get “10”‘s about 20% of the time. “10s” are awarded when suppliers give us a high performance claim that includes verifiable performance metrics that can validate the claim.
So it’s a very good way to leverage the metrics organizations have accumulated over their years of service, and if they can convey the great things they’ve done with clients that had similar projects, we’re well down the road in identifying an expert. They’re leveraging their past projects that they’ve encountered and how they’ve successfully navigated through those. We want to see a vendor with continuous improvement. If they can convey to us how they’ve matured as an organization, it’s that much better for them in terms of bringing the client-our state agencies-into a comfort area.
Once we’ve identified that expert, we go into the clarification phase, which lays out what they’re going to provide for us. The clarification phase is a little bit more difficult, because that’s where you get into the roots of the delivery. The supplier effectively writes the statement of work or scope of services, but we write the terms and conditions. So they write the specifications-the “how we’re going to get there”-which is what they present in the clarification phase. It’s a detailed picture of what’s going to happen.
The clarification phase is the first opportunity to get into the nitty gritty details of the vendor’s plan. We’re only dealing with one vendor, so we’re not wading through multiple scopes of work and trying to make basically a subjective opinion as to who’s the best. We allow the expertise to move someone into the clarification phase, and once they reach that phase they’re able to convey to us specifically how they’re going to meet our objectives.
The great thing about this is that when we go to contract award, we’ll have a very detailed plan, we’ll understand what our role is and will have very specific dates and deliverables, the cost structure and the detail of the costs-it’ll all be mapped out. Most importantly, we’ll understand what risks are involved in the delivery that could potentially derail the process. During the clarification phase, the supplier will not only provide a comprehensive assessment of risks that are outside the supplier’s control, but will also provide a mitigation plan to address them should they occur or a plan to keep them from occurring.
Gilroy: After the contract is awarded, you enter the third and final phase: management by risk mitigation. Can you describe that?
Hagar: After a contract award, there is a tendency for people to revert back to their old ways of doing things in terms of telling suppliers what to do, so we encourage and instruct the clients not to do that. Instead of telling a supplier what to do, ask them, “why they are doing that, why wouldn’t you do it this way” and have them explain to you why they are doing things the way they are. But as soon as you start trying to manage, direct and control-which is a no-no in Dr. Kashiwagi’s world, as he just doesn’t believe people can be controlled like that-you are going to bring risk back upon yourself.
In our PIPS process, one of the primary things that I will convey to a vendor is that they are to provide us a scope of work assuming a perfect delivery environment, so they are not to price in any risk in their scope of work. That’s part of the scope minimization process. Not only are they only telling us what we need to know on how they will accomplish their objectives, they’re also removing any risk that would be incorporated into a typical low-bid response. In a traditional response, if the supplier believes there’s about a 10% risk factor, they may bid 15% high to assure their margins remain intact.
Rather than do that, we tell them to eliminate that from their proposal, and we’ll give them a mechanism to come out whole, such that the margin you bid is what you can expect to achieve at the end of the day. That makes it a little bit more comfortable for them to follow us down this path.
The way they come out whole is that during implementation, there’s a component called a weekly risk report that’s designed to manage documented risks after the award is made. The weekly risk report provides transparency to the client on how the project is progressing and is conveyed in terms of metrics. The report also gives the supplier a mechanism to recover from realized risk that can be demonstrated as outside of the supplier’s control.
What we ask is that suppliers give us a heads up on project risks and what the impacts are on costs and delivery. For example, if I was a member of the client team, the report provides the transparency to hold me accountable for a deliverable that’s holding up the project. It gives the supplier a mechanism to recover if a realized risk and which may end up costing additional funds. Conveying that ability to make them whole is key, so that suppliers understand that they’re not at risk by removing risk from their scope.
At face value all of this may seem fairly straightforward, but there’s a lot of thought and theory underneath that allow it to function and to achieve the results that we’ve seen. We’ve had about $121 million worth of contract awards that had achieved about $67 million in cost avoidance, which is a result of the scope minimization process and only buying what we need to achieve the objective, as well as removing risk from the equation. This is possible by the identification and utilization of vendor expertise.
We don’t want to call it cost “savings” because we’re just minimizing scope, not getting something for nothing. There’s actually no negotiation involved in the PIPS process, and the reason is that we’ve asked suppliers to remove risk and minimize scope, so basically all they’re going to propose to use is only what we need to meet our objective. The dollars associated with that are what they are.
If we got to the final stage of the process and then told the suppliers to drop their price 10%, then now they’ve got to cut something out of their deliverables. They’ve got to cut 10% of the work they would have done out. All that does is compromise their ability to do the job well and shift risk back to us. If something falls apart down the line, then they can always come back and say, “well, we had that at the beginning, but when you asked us to cut 10%, we cut that out.” So by negotiating you’re actually bring risk back on the state. We’ve found that about 70% of the time, the best value supplier is also the lowest cost provider because they’re leveraging their expertise and they understand that by removing risk and minimizing scope, they’re able to offer us an aggressive price.
Gilroy: How did you come to embrace PIPS, and what was the transition like?
Hagar: As probably the leading state to embrace PIPS, we didn’t really have others to look to. Originally, our construction and properties administrator saw Dr. Kashiwagi speak at a conference and invited him to give a presentation in Oklahoma that we were brought into. Within the first five minutes I understood the process was about leveraging vendor expertise, so rather than utilizing a very prescriptive specification-how we want a job accomplished-we merely state objectives and let the experts in the industry provide a path to get us where we need to go.
This has proven to be a huge benefit. Experts can minimize our specifications to just what we need to accomplish our objectives, instead of what we think we need based on our limited knowledge. We try to have an understanding of what we’re purchasing, but the reality is that we will never have the expertise on everything we purchase that is available in the industry.
Before embracing PIPS, we had been living and breathing “strategic sourcing” for the four years prior. We were trying to educate our buyers-our central procurement staff-and the experts in whatever commodity that we were dealing with, and we were doing market research to figure out what the trends were and the best time to go to market. The bottom line is that we were just trying to match wits with the experts, trying to come up to speed and understand what they do as well as they do. That’s a difficult position to put yourself in.
After hearing from Dr. Kashiwagi, it became clear that we didn’t have to do that, and we could actually leverage supplier expertise. That’s what struck me, and that’s what typically resonates with the agency heads that I deal with. We’ve found a way to utilize their expertise that allows the supplier to perform more efficiently and effectively. We don’t have a dedicated project manager and the supplier doesn’t have a dedicated project manager that do nothing but communicate how the project is doing. We can eliminate those two people for the most part by leveraging transparency and allowing the supplier to convey their progress through a series of predetermined metrics that give us an idea of where we started and what we’re trying to achieve. The idea is that a client can look at a weekly risk report that includes 8-10 high-level metrics so that we can understand through numbers how they’re progressing toward achieving the goal that we’ve put before them.
We tend to look to the supplier to provide us those metrics that they would use to convey that progress because they’ve done that in the past and know what things will demonstrate their success and progress.
It’s funny-when I do the pre-education for this process, I usually get one of two responses from suppliers. The first response will come from the sales and marketing folks who lean back, roll their eyes, and say, “you’re really not going to do it this way, are you?” The second response we get is from those who’ve always wanted to be able to use their own expertise and get the opportunity to do things the way they know it needs to be done, as opposed to how we think they should be done. Those people get excited about this process; they lean forward and their eyes light up and they ask, “why aren’t more people using this?” A vendor the other day asked, “are other states using this?” And the answer is no, not really. Not a lot of them are embracing it yet. The vendors typically tell us that it’s so refreshing that you would come to us and ask us how we should do this project, rather than telling us how it needs to be done.
From a big picture perspective, when you think about how you’re treating a vendor in a low-bid environment, you’re really not wanting their best people or ideas. You only want their cheapest ideas. You’re really driving down the expertise in the industry, because if you’re not utilizing it, that expertise is just going to go away.
So in my mind, what we’re doing now is the best thing that’s happened to purchasing because we’re still able to control the process, but the actual projects are put into the hands of the ones who’ve done it before repeatedly. Before a contract award, we’ll validate that all of the references a supplier has given us are legitimate. We tell them not to include anything in their submissions that cannot be verified and substantiated by previous clients, or else they’ll be in danger of getting kicked out of the competition.
Gilroy: Given how different it was than what they had been used to, how did the state agency clients respond to the introduction of best value procurement?
Hagar: It’s a huge shift in the way that we’ve traditionally gone to market, which was a difficult thing to overcome. Basically it’s through client education that we’re able to gain buy-in on the process. I typically meet with senior agency management to give them a bird’s eye view of how the process works. Once they understand that we’re leveraging supplier expertise and utilizing their ideas and that it’s a very defendable process, they come on board pretty quickly. The pre-education is critical.
We find that often the clients are initially uncomfortable, given that this process is such a drastic departure from a traditional request for proposals process. Agencies may be uncomfortable with the short responses we require-how can you possibly know that a vendor can meet our objectives through a six-page response? How can we possibly know that they’re going to meet our objective if we’re not going to see how they’re going to do it until the clarification stage? Our interviews are typically 20-30 minutes, and the agency clients often think, “how can you possibly establish someone as an expert in 20 minutes?”
But it turns out that you can establish expertise fairly quickly. Once they get into the interview, the clients understand that someone that can take us to a 35,000-foot view of what they’re going to achieve and explain very succinctly how they’re going to march through the delivery of the objectives, it makes it pretty clear to the client that these folks know what they’re talking about.
By reducing the space that the vendors have in the initial submission and the time they have in the interview, we effectively make their time a premium, so they really have to delete any marketing and sales and cut to the chase of what they’re going to do to accomplish their objectives. That’s typically where the clients start feeling a whole lot better.
Gilroy: What kind of results have you seen thus far since implementing best value procurement?
Hagar: I’ll give a couple of very diverse examples. One of the first projects we did was a lightbulb contract, and it was one that was met with a lot of skepticism in our office. Why would you use best value for a commodity like lightbulbs? What risks do you really have with lightbulbs? And what value added is there that a supplier could possibly provide?
But we went ahead and used this best value process for the lightbulb statewide contract because we didn’t have a good idea of what our actual usage was of the various bulbs. We submitted what we thought that usage was, but one of the reasons the award was made to the vendor of choice was because they did offer a value-added service, which was that they would go around to the various agencies and would do an audit of the lighting fixtures and the bulbs being used and then make us recommendations as to how we might be able to save money. They’ve now done that for a multitude of agencies, finding ways to save through a more efficient utilization of bulbs or changing out fixtures to be able to accommodate more efficient bulbs.
Another interesting thing about that contract is that after they started tracking our usage, they found that one of the bulbs we had contracted had a much higher usage than what we had originally estimated it would be, so they actually came back and lowered their price on that. That may not exactly be a direct outcome of the best value process, but it’s a byproduct in that best value suppliers tend to look out for the best interests of their clients.
In the second example we had a project for the state tax commission, which had a multitude of disparate systems for the sales tax, income tax, tobacco tax-all of these different systems within the tax commission didn’t speak to one another, so they had no visibility of what the other systems were doing. They wanted to create a way to have consistent reporting across all of those platforms. They had a budget of $40 million on that project, and we got a multitude of responses, ranging between $16 million to $66 million. Our process found that the $16 million submission was the one that provided the best value and provided the most expertise. It was a small organization, but they were very successful in getting the work done.
They came in and told the tax commission how they were going to achieve the objectives, and while their price was $16 million, there were still a lot of other value-added services that they could offer for $8 million that were not requirements of the project but that would bring better results, if you will. So the tax commission not only got what they wanted for significantly less than they budgeted for, they also got a lot of value-added services on top of that.
That $24 million was $16 million less than what was originally budgeted, and we call that cost avoidance because it’s not really a savings-it’s just lower than what we originally estimated it was going to cost. If you look at that relative to the other bids, there’s an even bigger cost avoidance.
That project is completed now, and the tax commission became a big fan of the process.
Gilroy: What kinds of challenges do you see in implementing best value procurement and how have you addressed them?
Hagar: The paradigm shift is that while the process is quicker and easier for the clients, and the initial submissions are quicker and easier for the suppliers, it’s quite a bit more work on the procurement end. It is so much different from a traditional request for proposals, where we would receive a specification from a client agency, we would post that as a solicitation and get responses from suppliers, the responses go to the client for evaluation-and we’re completely removed from that evaluation process in the traditional scenario. Then they send back their recommendation, and if we can validate that their numbers add up, we would make the award and our buyers would move on to the next project.
That’s the mentality that we’ve had to address in the best value process. The contracting officer and the facilitator are involved in every phase of the project, from selection to evaluation to verification. All of this is monitored through the procurement office. And the contracting officers are also included in the weekly risk reports, so they watch the progress of the project too.
That’s one of the tough parts of PIPS: maintaining the actual documentation of the project. A buyer that monitors multiple projects will start getting multiple weekly risk reports in, so the projects just don’t go away like they used to. They just hang around and keep multiplying. So now we’re tracking a multitude of best value projects in various stages of delivery, and while it’s always the client’s responsibility to monitor those risk reports and make sure the project is on track, there is quite a bit more work on the procurement side in terms of education, facilitation and staying in lockstep with the client along the way. In a traditional process, the clients are left to do a lot of those things on their own.
That said, once our buyers get comfortable with the process, they tend to become big fans as well. When you work with a supplier that’s an expert, the chances of success are greatly improved.
Gilroy: Even though you have suppliers remove risk from their pricing during the procurement process, is there a threat that risks will materialize during project implementation that could significantly affect the cost of delivery after the fact?
Hagar: There’s always that potential, but I can’t think of any project we’ve done thus far where we’ve seen risks come up that have added to the price of the delivery. I think in part we’ve been lucky, but a lot of it is also because we’re doing relatively straightforward processes, as opposed to something like construction projects where there can be a lot of unforeseen risks that could affect pricing.
I’ll give you an example. We did an end-of-year testing bid where we used a traditional request for proposals and made an award, but there was a minor technicality that we missed-the award was supposed to be approved by the Board of Education before we officially made the award, and we didn’t do that. We got a bid protest, and we acknowledged that they were right and we had to go back out to bid.
The original bid took about nine months, but we had a very short timeframe to do the re-bid, so we used best value to expedite the process. The way it worked out, we ended up with the same vendor we awarded previously, but the substantial difference was in pricing-the vendor reduced their pricing by $4.9 million, on a $29 million dollar the project, by removing risk from the specification.
That’s the idea with the PIPS process. You take out some of the costs from the award that would have been associated with risk. The idea is that if the supplier issues enough of a heads up on the upcoming risks, then you’re able to take steps to mitigate the risk and keep it from impacting either the time or the cost to any great extent. I think that we’ve been very successful at that. You have to be able to identify that risk before the award is made and identify ways to control it in a risk mitigation plan. We all know moving forward that if that risk happens then there’s the plan for how to handle it. That transparency and visibility allow us to eliminate a lot of risk that might occur in a traditional project delivery.
Also, we understand that the client is usually the primary driver of risk, so if we can take the client risk out of the equation-meaning giving suppliers a mechanism to bring to our attention when we’re going awry and are doing things that might prevent them from achieving our objectives-that’s another reason why we haven’t seen risk adding costs.
Gilroy: What lessons learned would you share with peers in other states looking to implement best value procurement?
Hagar: The biggest lesson learned is that you have to have executive support and make sure that your senior leadership is on board and understands in case there’s some pushback from the vendor community in terms of what’s going on.
You also have to have what Dr. Kashiwagi calls “visionaries.” You can break the population into two groups-those that can see into the future and don’t believe in control, and those that believe in management and control because they believe they don’t really have control over their own destiny. Unfortunately, there’s a whole lot more of the folks that believe in management and control than believe in the visionary style.
So it’s important to have someone that understands how the process works and why it works, and understands the theory behind it, so that when questions come up you can address those in a manner that makes sense. There’s a lot of counterintuitive things in the process that make it work. For example, less communication is better, which is a hard thing for a lot of people to understand. But by communicating with metrics, there’s less need to communicate verbally between client and supplier. If you allow the supplier to use their expertise, then we don’t have to make a lot of decisions. Decisions inherently tend to bring risk because they are made when you don’t know where each paths leads. Experts, having been down each path, know which path to take from experience.
We tell vendors right up front, “don’t be afraid to tell us when we are creating risk.” That’s because it’s our tendency as a client-we’ll change specifications, we’ll change delivery times, we’ll just change things because as the client, we can. If the supplier has a mechanism where they can highlight that risk and how it will impact delivery of the project, then they can have a way to address it.
Education is also key. We send our people to Dr. Kashiwagi’s annual best value certification event in Arizona, and our buyers always come away from that with a better idea of what we’re trying to achieve.
The PIPS process is another tool in our toolbox. It’s not going to be right for every situation, but is something on the other side of the spectrum from strategic sourcing, for example. It’s a great tool to have as an option, particularly when you have a project that’s politically sensitive and complex, where we may know what we want to achieve but do not have the expertise to know how to get there. But there’s almost always someone out there that does have that expertise. When we write our specifications, we frame it as our objectives-we don’t tell anyone, “this is what we want you to do.”
The objective of the tax commission project I mentioned earlier was conveyed in one-page; a traditional information technology project of that magnitude would typically be 50 to 100 pages of specifications, and then we would receive 300-page responses from each supplier that we had to wade through. It becomes a very time-consuming and subjective process at that point.
With this redacted amount of information that we use in PIPS, they simply have to say, “we understand your objective, we’ve done this before, and here are the results.” We can process that very quickly, objectively, and effectively.
Steve Hagar is the state purchasing director for the state of Oklahoma. Before taking that role in February 2015, he had nine years of experience serving as deputy director of central purchasing for the state of Oklahoma and has been an instrumental part of the division’s many recent accomplishments. Most notably, he has developed the state’s best value procurement process and holds the only A+ certification in Performance Information Procurement System (PIPS) in the country.
Hagar has an extensive background in both manufacturing and government procurement. During his manufacturing career, he earned certifications from the National Association of Procurement Management (NAPM) and American Production and Inventory Control Society (APICS).
Other articles in Reason Foundation’s Innovators in Action 2015 series are available online here.