Please find below a transcription of the audio portion of John Owen’s session, Simplifying Schedule Risk Analysis Using Microsoft Project Custom Fields, being provided by MPUG for the convenience of our members. You may wish to use this transcript for the purposes of self-paced learning, searching for specific information, and/or performing a quick review of webinar content. There may be exclusions, such as those steps included in product demonstrations. You may watch the recording of this webinar at your convenience.
John Owen: Thank you. Good morning, everybody. Good afternoon, West Coast. And thank you, Melanie. I’m disappointed to hear your Land Rovers are unreliable. I have two, well one has a 140,000 miles on it, the other has a 150,000 miles on it. And the one with a 140,000 miles just made a round trip to Florida from Houston, Texas with no issues. Hopefully we can figure that one out for you. Well, thank you everybody for joining this webinar, we’re just going to talk about some simple ways of performing Schedule Risk Analysis basically just using Microsoft Project with an add in that we sell called Full Monte.
John Owen: What’s the problem that we’re trying to solve with Schedule Risk Analysis really? Well the problem is that the traditional Critical Path Method, which is embodied, it’s an algorithm that’s embodied in all the major project management tools like Microsoft Project, Deltek Open Plan, Oracle Primavera, they all fundamentally use the same algorithm. It’s called Critical Path Method, some people call it longest path and basically you build a model where you have the task with the durations and then the dependencies between those tasks, which basically specifies the order that they need to be performed in. And all of those tools will produce a project finish date, if you like, a forecast or prediction of when that project will be completed.
John Owen: And the problem historically is that date is often not all that realistic. Unfortunately, many projects to the detriment of the people executing them, are delivered late and it’s not usually anything to do with estimating or project execution. It’s just simply the fact that the schedule was unrealistic in the first place and that is because it does not take into account uncertainty. All of the scheduling tools, Microsoft Project, et cetera, they produce a single, what we call a deterministic forecast of when the project or any given task will be completed. It does not take into account uncertainty. And unfortunately everything is affected by uncertainty. How can we take that into consideration?
John Owen: Well, first of all, I want to demonstrate that this is an important factor in our ability to deliver on time. If you look at the top two tasks on the screen, I’ve got two tasks, task A, task B, each of five days and we’re going to assume that they are indeed subject to uncertainty so we’re going to say that either task could finish in as little as four days or take as much as six days. Let’s simulate the execution of the project. We do task A and unfortunately it has slipped to six days so it has pushed out task B. But we’re good project managers, we’ll be using project management techniques like earned value to be made aware of the slippages and we can focus our effort strongly on task B or we may just get lucky and task B finishes in four days and the project is still delivered at the end of the 10th day so we have a success.
John Owen: But take those same two part tasks and put them in parallel. Again, if task A happens to slip to six days, then it pushes out whatever is coming next. Doesn’t matter whether it’s a milestone or just another task and there’s nothing we can do with task B. If task B finishes in four days, then that successor has still being pushed out six days by the one task. And basically what this means is any task or milestone that has more than one predecessor is likely to be delayed. Its chance of being delayed increases effectively as a ratio of the number of predecessors as it has. Now, there are obviously exceptions. If one predecessor is one day and the other is a 100 days, then chances are, you’re not going to see this effect.
John Owen: But many times we have similar duration. They don’t have to be the same duration, but similar duration task with similar uncertainty and the knock on effect of the delay of the successor, even if the uncertainty, as in this example is symmetrical, just as likely to finish a bit early as a bit late, the delay has a knock on effect. And that is one of the primary reasons that projects are surprisingly delivered late. It’s got nothing to do with poor estimating, poor execution or even risks that occurred or did not occur. This merge bias effect is one of the biggest reasons that the project gets delivered late. And just to reiterate in this very simple example, with two identical tasks, with identical uncertainty, we only have a 25% chance of that successor starting on time. It’s as simple as that. 75% chance that task will be delayed. That’s why it’s such a major factor.
John Owen: The purpose of the Schedule Risk Analysis is to consider the uncertainty that we’re putting into the model and come up with a better forecast. Now, many people assume that the purpose of the SRA is to produce a range of dates when the project will be completed. But I don’t see that as the best way of describing what we’re trying to achieve. A better description in my mind is a schedule risk analysis can provide a quantifiable level of confidence in achieving a required delivery date. Basically it’s giving us more information about our chance of delivering on a particular date and that gives us information. I do say can, I don’t say will. As with everything, if there is a flaw in the information you put into the model, then it’s not going to give you a realistic assistance on that forecast. But provided the model is reasonably sound, we are going to get that quantifiable level of confidence, which allows us to make choices.
John Owen: And I want to also draw your attention to the fact that in a best practice schedule, effectively you’re going to have two finishes. You’re going to have the project finish, which I regard as when the work is complete. You’ve got all the tasks representing the things that need to be done to achieve the deliverable and then you have the project finish, which means that deliverable is complete. You will have a second date and sometimes it’s not actually in the schedule, not represented in the schedule, but you will have some kind of promise to your customer, a contract delivery date, a committed date, promise date, people do use different terms, but basically there are two finish dates. There’s the end of the calculation in the scheduling tool and then there’s what you’re committing or promising to do to your customer.
John Owen: The PMI has a term for the difference between those two dates. They call it the schedule margin. It’s effectively a contingency for risk, designed to protect that promise date from uncertainty during the project execution because we’ve already seen, there’s a good chance that that project finish date from Microsoft project is not going to be achieved. We have a buffer, another term for it. I don’t like using the term buffer because then it gets confused with things like the critical chain technique, but this contingency is designed to protect our promise date to the customer. The Full Monty simulation or the Schedule Risk Analysis simulation will show us how confident we can be achieving the two dates. Typically, we’ll only have a small chance of achieving the project finished date calculated by Microsoft Project, but we should have or hope we have a much higher chance of delivering by the promised date.
John Owen: Let’s talk about a little bit about this confidence. I said it would give us a range of confidence in our ability to deliver. And a lot of people assume that that should be a 100% but except in a few extreme cases, it’s not going to be a 100%. Most commercial organizations shoot for somewhere between an 80 and 90% chance of achieving the promise that they made in the contract to the customer. The why 80 and 90%, why not a 100%? Well the trouble is if you commit to a date at a 100% probability of success, that means you can’t use your resources to do anything else before that date has arrived because they need to be held back to ensure that they continue to work on the project if it’s been delayed and that they are available to do so.
John Owen: And that’s basically a cost benefit analysis. Basically compare the cost of not being able to take onboard new work a little bit sooner versus the penalty clauses in the contract. That’s an organizational decision as to what level of confidence that you want to go for. A construction company building an Olympic stadium probably needs to shoot for a 100% because the loss of reputation, not to mention any damages if the stadium is not ready on day one of the Olympics would be catastrophic to the organization, but for most commercial purposes, somewhere between an 80 and 90% confidence of delivering on the promise date is normal.
John Owen: I need to reiterate this fact as well, that schedule quality matters. It has to be logically sequenced. Basically you have to have the entire scope of what you’re planning to do in the schedule and the order which things needs to be done, needs to be clearly identified with predecessor successes. It must not be constrained with things like finish no later than, start no later than. Those are bad constraints to use. It’s okay to use deadline dates in Microsoft Project. It’s okay to use start no earlier than constraints where you are expecting a deliverable from an external source but don’t use any constraint dates which prevent dates moving into the future. Because during the simulation we need to see what is the effect of this uncertainty and the delays they represent by pushing the dates out.
John Owen: In Microsoft Project especially you need to take care if you’re modeling level of effort. It’s a way of representing overhead resources, costs and resources like IT support and so on in the schedule, but make sure that they are not logically linked such that they will be driving or potentially driving the critical path to your deliverables. Make sure you have milestones in the schedule to identify deliverables. It’s a best practice and it’s also necessary with our software, for you to be able to focus what we call a sensitivity analysis that I’ll go into later onto particular deliverables to see the things that are driving those dates.
John Owen: And contrary to some advice you will hear, more detail is always better in your schedule. If you run on a summary schedule, run the Schedule Risk Analysis on a summary schedule, you are not going to see the effect of merge bias. And those I said earlier, it’s a major contributor to unexpected delays in schedules. If you’ve got some risks, you might expect them to occur, but merge bias is the sort of secret killer to your project that will creep in and push the finish date out without you really being aware of it. As much detail as possible in the schedule is a good idea.
John Owen: Talking of project detail and so on. We have a second tool called Schedule Inspector. And as a thank you for all the attendees on this webinar, if you would like to email mpug@barbecana.com, we’ll send you a complimentary license for our Schedule Inspector for Microsoft Project, the latest version, which includes logic and float tracing status.
John Owen: How does Full Monte work? Well, it uses a technique called Monte Carlo Simulation, hence the name. And basically we’re going to simulate the execution of the project thousands of times. The more times the better and for each iteration of that simulation, we’re going to substitute in different durations for the task from within what we call a 3 Point Estimate that you’ve given for the tasks in the schedule 3 Point Estimate effectively Is a best case, most likely worst case estimate of the likely duration for a task when you think it’s going to be executed. You don’t have to go to a lot of effort to force your estimators to give you three point estimates in terms of durations for all the tasks in the schedule.
John Owen: I suggest you focus on critical, near critical and known high risk tasks and for all the other tasks that are not necessarily on the critical path, not a high level of risk associated with them, it’s okay to use generic information. And you’ll see we use percentages inside Full Monte or we can say for those not critical tasks, we can just use percentage and say the duration is likely to be plus or minus 5% around whatever we estimated or it might finish 5% early and 10% late. And you’ll see how we set that up in the product.
John Owen: The benefits of Full Monte, it executes as a Microsoft Tools for Office customization inside Microsoft Project. Basically you’re working in your own comfortable environment. That has a number of benefits, no learning a whole new tool, there’s no exporting and importing of data into third party tools. It’s a single source of the truth. You know you’re running on the latest version of the schedule because that’s what you opened in Microsoft Project. And then the results can be easily shared using standard Microsoft Project views or saved to Project Online, Project Server custom fields and so on. And Full Monte has a very fast simulation engine, which again, plays into the fact that we recommend you run as many simulations as you can.
John Owen: If you’ve got a 5,000 line schedule, maybe you’ll find while you’re working up and experimenting to see what’s causing issues with the schedule, maybe only run a 1,000 simulations, but when you’re getting ready to run your customer reports or your management reports, run 10,000, 50,000, a 100,000 simulations because it will improve the fidelity of the probability histograms that are the ultimate output from the tool. More simulations is always better, but you don’t have to run loads and loads of simulations every single time while you’re working on it. A 100 even, will give you some idea of where the dates are going to play out. I’d recommend more, a 1,000, 5,000, but then when you’re producing your management report, tack a couple of extra zeros, go grab a coffee, go to lunch and come back and see a much prettier graphs.
John Owen: The process is going to be, we’re going to add uncertainty to the model using those range estimates we talked about. We are optionally going to include information about known threats, i.e. risks to the schedule. We’re going to run the Schedule Risk Analysis, i.e. perform that simulation, look at the output. If they’re not acceptable, if we’re not getting the confidence that we want in achieving a particular date or cost on the schedule, then we’re going to have to modify the input and then communicate the outcomes to stakeholders, which as I said, you can do just using your normal Microsoft Project reporting processes because of the way the product runs.
John Owen: Just a quick summary, risk analysis helps produce achievable plans, realistic plans. It often requires changes to the original schedule to achieve an acceptable chance of success. And the primary reason for that is all too often, we see customers, they build a schedule in Microsoft Project, it’s a great schedule, it’s realistic in terms of the estimates of the durations but it doesn’t take into account uncertainty. That gives them a project finished date, they give that to the client and call that the contract date. And of course the problem is it didn’t take into account uncertainty so it’s unlikely to be realistic so they are going to have to go back and if they’ve committed to that date, then they’re going to have to modify their schedule to actually finish earlier so it gives them that schedule margin, that contingency for risk or they’re going to have to go back and renegotiate the contract and push the date out. Start off with those two dates in mind when you’re building your schedule.
John Owen: I do also recommend continuing to run a risk analysis throughout the lifecycle of the project. Quite a lot of people only use it for a proposal and that’s better than not using it but obviously as you enter status information, you need to know that your assumptions are still reasonable and Full Monte is nice because it will prorate the original uncertainty you put into the model so you don’t need to make any changes to the uncertainty unless you have new knowledge that work on task A is not going as well as expected and the new worst case duration is X and you can put that into the model and see what it does to your confidence of delivering on time.
John Owen: Very quickly, my name is John Owen. That’s my email address, jowen@barbecana.com. You can email with questions about the presentation or Schedule Risk Analysis. You can go to our website, www.barbecana.com to request a free trial of either Schedule Inspector or the Full Monte Schedule Risk Analysis. As I said, if you email mpug@barbecana.com, you can get yourself a free license to the Schedule Inspector.
John Owen: What I’d like to do now is to stop using PowerPoint and flip over to actually using the Project product. I’m using Microsoft Project 2019. I’m using the professional edition that is available through Project online. Our product works with custom fields and so on or projects opened from Project Online project server, but it works equally well just locally, which is how I’m going to be using the tool today. We do support Project 2010 through 2019, standard or professional editions and we also support 32 and 64 bit. I’m using the 32 bit version here.
John Owen: I’m going to select to just use my local computer file system rather than Project online. Now, one of the first questions that people ask is, “Well, you said you should put uncertainty onto all the tasks but how do you know what to use?” Well honestly, one of the best ways is to look at what’s happened in the past. I’m going to simply open an old project here, the only caveat with this technique is the project must have had a baseline when it was first created, before it was executed because Microsoft Project has the unfortunate habit of adjusting the duration to match the actual start, actual finish that you put into the tool into Microsoft Project as you progress it, which means you lose sight of the original estimate. Hence, you need a baseline which will show you what the original task duration was before it was executed.
John Owen: I’m going to go to add ins. We appear inside Microsoft Project on the add ins menu. I can launch Full Monte. It’s warning me there are issues with the project. I don’t really care. I’m not going to be running a risk analysis. I just want to be able to do a comparison of what was originally estimated versus the actual for this particular project. I’m just going to say no, and I’m going to go view, open name view, history. There’s a special report inside Full Monte to look at historical behavior on a completed or partially competed project. I’m just going to open the graph at the top level. And there’s good news and bad news in here.
John Owen: Obviously the bad news is that 0.8% of the task, it was actually just one task, slipped over six times. It took 660% of the estimated duration. Now if it was a one minute task or not on the critical path, wouldn’t really matter. In actual fact, when we did the analysis, if we drill down into the tool here, we can find this toss was actually a three week task on the critical path and it has a significant impact, delay impact on the project. Basically, a risk occurred that had not been identified and mitigated. But looking at the core data here, we can see that 33% of tasks finished in 100% of their estimated duration. That means if they were estimated to take 10 days, the actual duration was 10 days. That’s pretty good news. 33% of our estimates were spot on.
John Owen: We can actually see a range down here and surprising number, 3% of the tasks finished in 50%, 52% of the estimated duration. Quite a few tasks we’re finishing earlier than we had originally estimated but unfortunately we do have the other side of the curve here and this is showing that nearly 6% took 1.4 times the estimated duration. I could use this information to suggest with this as backup that for future work of the same type being done by the same people, I could say a realistic range of uncertainty would be a best case of say 50%, most likely of a 100% and then the worst case, I might say is 200%. I can ignore these outliers. We can investigate what they are. They could be data entry errors for all we know. Somebody put in the wrong actual start, actual finish date. They would need investigation, but they don’t represent the core data.
John Owen: The core data is down here. I could use this, as I say 50%, a 100%, worst case 200%. And I could use that as generic uncertainty for a similar type of project that I’m working on a proposal for the future. And that may sound horrifying, but a lot of people are talking well, shall we say the optimistic is 90% of the estimate? And the worst case is a 110, a 115, a 120? When in reality, in this case, we have good evidence that the worst case can be significantly worse than 120% of the estimated duration. You can use the tool to do that analysis on past information.
John Owen: I’m going to close this example now and actually open the project that I’m going to use for the majority of the demonstrations, Full Monte demonstration with risk. It’s a very simple project. It’s designed to fit on my screen. Effectively we’ve got a hardware component, a software component, we’ve got some requirements that need to be worked upon. Then we’re going to integrate the hardware and the software, have a system test completely separately. We’ve got some brochure development going on. Down here, I do have a schedule margin tasks so I’ve actually got in my schedule, I’m saying I’ve got that contingency for risk of eight days. I’ve got my project completion. I’m saying I expect the work to finish on August the 28th, but the committed delivery in the contract, the promise date to the customer is September the 9th. I’ve got that range of dates separated by in this case eight days. Will warn you if you’re using older versions of Full Monte, you do need to zero out the duration on the schedule margin task before you run the analysis but I’m using the very latest version, which will do that automatically for me.
John Owen: Briefly, I want to show you the Schedule Inspector. I can open this to take a look at any issues that it finds in the project. Is looking for things like hard constraints. It’s found I’ve got three inactive tasks. I’m using those to model risk. Down here, I’ve got the hardware integration and test and that’s followed by failure rework. If the test fails, I’ve got a risk built into my schedule. I’ve marked, I’m using Project professional so I’ve marked it as inactive so it doesn’t change the finish dates in Microsoft Project. If you just have the standard version, you can leave it in there as a real task. Scheduling Inspector has done this analysis. I can choose and reset between different industry best practice standards for the tests that you want to run. It’s a very simple tool.
John Owen: I’m actually going to demonstrate a new feature of the new version here so I focused on project complete and I want to do a float analysis. What is driving that project complete inside the schedule? I can just quickly click okay there and I get a tabular report on the various float paths that are driving. I can see the primary path driving my project completion goes down through system test, hardware failure. It’s predominantly following the hardware path, which makes sense, because that is being shown by Microsoft Project as the critical path, the red task of a highlighted critical path. But I can use that information. I told it to save that information to a Microsoft Project custom field. Again, focusing on the use of custom fields in Microsoft Project. And so now I can very graphically see that primary critical path, the next most likely influencer to the critical path, which is a hardware assembly A. Then the third, most likely influencer to the critical path, which is going to be the software code and so on and so forth. We can use custom fields to create custom views inside Microsoft Project.
John Owen: Let’s go back to looking at the uncertainty in the model. I’m actually just going to launch Full Monte at this point. I haven’t really done anything to tell it about uncertainty. I’m just going to open Full Monte. Now by default here, it has applied in my case, plus or minus 25% to all the durations on this schedule. On average, every task is expected to finish in the estimated time but I’ve put a range of plus or minus 25%. Out of the box, the system will give you a plus or minus 10% on the task, but because this is such a small project and I want to emphasize uncertainty, I’ve got mine set to use plus or minus 25%.
John Owen: You’ll notice down here, this schedule margin task, it has appeared and I can see the duration was eight days, but under existence here, the probability of actually occurring has been set to 0% automatically by four months because it knows that schedule margin and it shouldn’t be taken into account. It’s that buffer for uncertainty so it shouldn’t itself be taken into account when we’re understanding the impact of uncertainty. Full Monte has automatically set that to 0%. Well, the quick question then becomes, how? And we have this technique called field mappings, where we can tell Full Monte which Microsoft Project custom fields, going back to the emphasis of the presentation, it’s all about Microsoft Project custom fields and how to use them to do a schedule risk analysis without having to do too much work outside of project.
John Owen: And down here, I have told it, the schedule margin flag is actually custom flag field number seven. Where that is set to yes, in Microsoft Project, Full Monte is going to treat that as schedule margin, which means that it will set it to inactive during the simulations. It will have no part to play in the simulation. The logic will obey but effectively it’s treated as a zero duration task.
John Owen: Moving on, let’s run a risk analysis. I’m going to run 10,000 simulations. Not going to worry about the options at this point. We ran 10,000 simulations of our project using this default plus or minus 25% on all of the tasks in the schedule and let’s go and look at the project completion date. I get this probability distribution histogram for the project completion task. And actually I can see I have a 29% chance of this finishing during the simulations compared to the deterministic 28th of August that Microsoft Project calculated. We’ve got a non-zero chance of achieving this, 29%. On a real project, a lot of customers find that when they’re looking at the end of the work, the project finish shown by the scheduling tool, they often unfortunately have a 0% chance of achieving it. But that’s the purposes of having that buffer between when you’re expecting to finish work and the contract promise date. Got a 29% chance of that, but let’s go and look at the committed delivery, which was protected by the schedule margin.
John Owen: And here I can see based on this simulation, plus or minus 25%, I have a 99% chance of being able to deliver by the 9th of September, which was the date we put into Microsoft Project. Schedule margin was doing its work. It was acting as a buffer for the uncertainty in the schedule and we have a good chance, an excellent chance of delivering by the 9th of September.
John Owen: Just briefly going back to this task, many people when they first see this, assume it must be wrong. Remember, each individual task plus or minus 25% uncertainty. Just as likely to finish a bit early as a bit late. And so in theory, you would expect the project to have a 50% chance of being completed on time, except for merge bias. Merge biases what’s causing this delay. Now it’s not as bad as the 25% example we saw with just the two tasks, but that’s because this is a more complex schedule. The task leading into individual successes where there’s more than one, they’re not exactly the same duration. This is why we’re seeing a slightly different realistic number, rather than that theoretical 25%. But obviously 29% is still lower than we might’ve expected.
John Owen: Let’s close Full Monty and I’m going to tell it to save the results. Again, it’s using those field mappings where I told it I wanted certain results from the analysis stored into Microsoft Project custom fields. And now you’ll see each of the tasks has two bars. I’ve got the original red and blue bars from the CPM analysis by Microsoft Project and then underneath each bar, I’ve got a color coded bar and basically the darker the color of the bar, the more likely it was to be on the critical path. This particular task, somewhere between 50 and 99% of the time it was on the critical path to the delivery. I can see the effect that uncertainty is having. The reason these two here are a different color, they’re less than 50% chance is because that they’re effectively in parallel. The chance of going through any particular task was actually around about 48%. Anyway, we can see that that has pushed out our finish date. Here was the original project finish date on 8/28 and the software is telling me, I have let’s find the date columns.
John Owen: Apparently I deleted those columns. That was daft of me. Oh no, sorry. I’m looking at it right here. Finish percentile, so the 80th percentile or 80% confidence dates are being shown here. Project complete was 8/28, the 80th percentile is 9/3. Interestingly, we’re actually seeing the committed date here, the 80th percentile is 9/3. These two aren’t always the same. It’s just because it’s such a simple model. But remember that this was giving us a 99% chance of achieving the 9/9. Basically 10% of the results here were later than 9/3 but before 9/9.
John Owen: How can we improve on this very simplistic model that we’ve got at the moment? Well, the simplest way, let me bring back my hide some of these columns. The simplest way is to actually just use a text field, to put in your assessment of the risk or your confidence in the duration estimates for particular tasks. For example here, we have the requirements which we’ve estimated to take five days and once we’ve spoken to the team, we’ve determined we’re not very confident we can do it in five days. It may be more. Possibility it might be less, but almost certainly it’s going to be a bit more so we’ve described that as high uncertainty.
John Owen: In actual fact, I’ve defined a dropdown list with some terms that we wanted to use on this project. These terms don’t have to be anything. You could say, high risk, low risk. You can say high uncertainty, low uncertainty. You could say high confidence, low confidence. You can use numbers, one, two, three. Whatever you want to use or are already using in your organization to specify how confident you are in the duration estimates for a particular task, you can put into this column. I’m actually using a custom field page, it’s text 22. I specified the lookup table of values that I wanted people to use. I like using the lookup table because it prevents them accidentally putting in values that the software is not going to later recognize. How do we use this information? Let’s go back to Full Monte.
John Owen: Here under edit manage templates, I can actually define. I’ve got the very high uncertainty. I’m saying it might finish in as little as 75% of the estimated duration. It’s more likely to take a 120% of the estimated duration. Effectively we’re saying, “We don’t have much confidence in the people or the company that estimated this duration so we’re going to factor in another 20% on whatever they say. And then the worst case we’re setting at 200% of whatever they estimated” As I said, very high uncertainty. We don’t have much confidence in the estimates that have been used on this task so we’re giving it quite a dramatic range. We actually apply that information by just simply selecting all the tasks, so go into edit, select all, right click, apply template from text field and choose whichever column you want to use for the uncertainty. And the nice thing about this is you can have different scenarios in different text columns in Microsoft Project.
John Owen: I’m going to use this duration uncertainty. It tells me it’s going to update 16 tasks and it’s now applied that information, the plus or minus 25% that we were seeing has gone away. It’s been replaced by more realistic, albeit still percentage values, being loaded from Microsoft Project and we’re ready to rerun that risk analysis. And let’s go and look at what that’s done to our project completion. Project completion, unfortunately we now have zero chance. None of the simulations indicated that we would finish the project execution by the 28th of August. We’re looking at dates in September all the way up on the S curve here.
John Owen: But what about the committed delivery? What we promised to the customer? Well, the good news is that it’s dropped but it’s still at 90% confidence. And if we’re a typical organization, we may be saying it is acceptable to forecast dates to customers at 80% confidence. That gives us that opportunity to try and execute other projects or assume we can execute other projects using our resources before the committed date to the customer. That’s good news. We still have a reasonable chance, good chance, 90% chance of delivering to the customer on time despite the fact that the finish date, the project completion or work completion, we now have zero chance of finishing by the date shown in Microsoft Project. But that’s fine because we had that schedule margin buffer, which is protecting the committed delivery. And so far, that eight day buffer has been good enough to absorb the delays due to uncertainty.
John Owen: The other thing that we can do, I’m going to save this information, save the results. The dates, these bars have now pushed further out. One striking thing here is because we had more higher uncertainty, a more high uncertainty, more uncertainty. Not speak English today, more uncertainty on the software task compared to the hardware task, which were originally on the critical path, we can now see the darker bars clearly going through the software task. Software is now really driving the completion date for the project. The hardware is now less critical than it was when we first started out this exercise. But I said for known or critical or known high risk items, it is often worth getting the estimators, the subject matter experts to give you discreet three point estimate duration. Don’t use percentages.
John Owen: Using percentages is fine but it leaves you open to criticism where people say, “Well, where do you get those percentages from?” For these known high risk items, critical items, near critical items, you can capture more information. Here, what I’ve done is for anything that was identified as high uncertainty or very high on certainty, I’ve asked the subject matter expert, the estimator to give me discreet duration estimates. In this case, the requirements they’re saying, “Well, it could take as little as four, most likely to take five days, which is what we originally said. But worst case scenario, the customer is slow to answer our questions, it might take eight days.” And we’re going to use a triangular distribution between those, those three points. The choice of the distribution type honestly is less critical to getting sensible results compared to getting a good range of best case, worst case, most likely worst case, regardless of whether you use percentages or discrete durations like this.
John Owen: A lot of people use a triangular distribution, honestly a beta distribution, which I’ve used on my rework task down here is typically more realistic of the actual probability distribution that you’ll see for individual tasks, but a lot of people use triangular because it’s either required in the contract or they’re just more comfortable using it. And there’s statistical reasons for doing that. The actual standard deviation of a triangular distribution is greater than the beta distribution so it effectively builds integrated contingency for risk through the back door, as it were without explicitly stating it. Using a triangular, nobody ever got fired for using a triangular distribution, put it that way. For these high uncertainty and extremely high uncertainty, we’ve put in the discrete values. And the other thing that I’ve done is I’ve used another custom column here to associate risks that we’ve identified with the individual task representing the work required to do them.
John Owen: I have this hardware failure. We do the integration and test, hardware failure rework. It estimates the five days and I am going to associate that with risk and I’ve called it RSK002. The numbering can come straight out of your risk register. How do I use that information? Well, let’s go back to the Full Monte. You’ll see us remembered everything we’ve done up to this point because I told it to save the data changes. And what I’m going to do is go to file import and tell it to load that those discrete duration estimates from duration one, duration three and I’m still using the duration column for the most likely. And then we’ve the discrete risks of being loaded from Text23, which Microsoft Project, we renamed it to Risk. I can go ahead and import, tells me all that worked. I can now see that information. There’s the requirements.
John Owen: It’s now got the triangular distribution. The optimistic is four days, most likely is five days, worst case is eight days. The other tasks which we set using the templates, they’ve still got the percentages but for all of the major rework tasks, we’ve got their discrete duration estimates. You see, I’ve got a lot of uncertainty on the rework and that’s because when we’re building the plan, we don’t actually know what’s going to go wrong. It may be as simple as just changing one wire or it may need a whole new circuit board redesign. A lot of uncertainty on that. And because we linked it to that task, when I go to the existence information over here, I can see it’s linked to RSK002 and that’s actually managed in Full Monte under managed risks. I can see that the RSK002, 5% chance of occurring. That will be used when we run the simulation. Let’s go ahead and run the simulations.
John Owen: We’ve rerun the simulation, taking into account all the information and the cool thing here is so far, I haven’t typed anything into full Monty, so you don’t need to learn how to use a spreadsheet and so on. Although it works in a very similar way to Microsoft Project, there are obviously tiny detail differences. It’s just easier to put stuff into custom fields in Microsoft Project.
John Owen: Let’s go and look at the graph for the project completion, the work complete. We actually, it’s now actually showing a 1% chance of being able to do that. Could be due to the fact that this is ultimately based on random numbers. The zero, the one very similar to one another. If I ran more simulations. I might stop minor changes like that or it could just be that the changes we made to the model have increased that chance. But let’s look at the committed delivery.
John Owen: This is what we promised our customer and this is not such good news because after we’ve made all these changes, we’ve incorporated the discreet uncertainty on those high risk items, we’ve added in the risks that may or may not occur, we only have a 79% chance of delivering to the customer on the date that we promised. Now 79% is real close to 80% so I might run the simulation again, just to see whether does that help? But let’s assume that this is a realistic number. And our corporate standard is to get to 80% confidence in terms of meeting the customer required date, so what was causing that? And that’s where the sensitivity analysis, a lot of people call it a tornado chart because it looks like a tornado, but it truthfully, this is a sensitivity analysis, which is trying to show us which tasks are creating most uncertainty in the selected outcome.
John Owen: Now, right now the selected outcome is project complete so these are the tasks that are creating the most uncertainty. And there are actually two different ways of looking at this. If you look at the purple bars here, I can see that software code A actually has the biggest purple bar. And that means it is consistently creating the most variability in the completion date. It doesn’t necessarily mean it’s responsible for the worst completion date, but it is certainly creating a lot of variability. But here the first three, they’re actually the rework tasks and the purple bar is small because it’s basically saying, they didn’t happen very often so overall in 4% of the time that this task was impacting the outcome. It doesn’t happen very often but when I go and look at the green or red bar over here, I can see when it does happen, it has a major impact on the delivering of the project on time.
John Owen: The split between the green and the red bars here is actually the mean finish or average finish for the project during the simulations, which was actually 9/7.7th of September was the mean finish based on the uncertainty we put into the model. But what this is telling me is when this task either did not occur or happened closest to its most optimistic duration then the project did tend to finish sooner. But when this did occur and occurred at its worst case duration, remember we gave it quite a poor score for its worst case duration, it really pushed out the average finish date for the project. When this occurred and it was at its worst case duration, the mean finish for the project is actually in October, October the 1st. It had a big impact. This is key information to the scheduler, stroke project manager, to understand what’s driving uncertainty in the schedule and look for opportunities to improve it.
John Owen: Here we can see that, can we go back in? Can we find a better process, which doesn’t have the same risk as failure? Can we re-estimate the rework and see if we can narrow down the extra work that it might take to resolve issues? Can we go back into the regular tasks, those that are not risk driven and simply say, “Software code A, we’ve we’ve scheduled it for 19 days. Can we re estimate it? Can we use more resources to do it in less time? Can we use better resources to do it in less time? Can we change the logic so it’s not on the critical path to the deliverable as often?” We’re giving you places to focus your efforts to see if you can improve on getting that 79 back up to 80% confidence of being able to deliver the committed date.
John Owen: I did say that this chart can be focused on anything in the schedule. if I focus the risk analysis, here we have this schedule sensitivity target. I’m actually going to focus it on the software completion and run the analysis. And we can see far fewer things are actually affecting that particular milestone. That’s good news. If I am the manager responsible for the software component of this project, these are the things that I need to focus on. 100% of the time, the code test, the design, the requirements were on the critical path to the completion of this software milestone.
John Owen: Again, we’re seeing the failure rework 5% of the time because it had a probability of existence 5% inherited from the RSK003. 5% of the time it was on the critical path. Again, when it did occur, it had quite a big, large or large impact compared to the uncertainty on the other tasks, which were not risk driven. Cool feature here is we get 2% criticals. We get the percent critical. I can see that 49% of the time software code A was on the critical path to project delivery. But 91% of the time, it was on the critical path to my software completion milestone, which is what I care about. As an individual component manager, I’ll be focused on the percent critical sensitivity. These are the things driving my deliverable date. These date the project manager will be more concerned about what are the key things driving the overall project completion?
John Owen: There’s lots of other features in the product that I don’t have time to go into, conditional and probabilistic branching. Conditional branching is a great way of modeling risk mitigation into your schedule. Correlation where we can see, say that these tasks are all being executed by the same subcontractor. If they perform well, then all the work that they’re doing probably will go well, if they perform badly or their estimates are bad, then all of the tasks will be similarly affected. There are other different features in the product. If you would like more information again, please visit barbecana.com, email me, we can set up a one on one demonstration of the project and answer any further questions that you have. What I’d like to do now is to ask Melanie if there have been any questions and if there have been, I will attempt to answer them. And otherwise I would just thank you for allowing me to talk to you today. Thank you.
Melanie: Yes John, we do. A question from Laura in the audience, “Is schedule margin duration set by the tool? Or was that just a value manually entered?”
John Owen: It was just manually entered so that the eight days that I’m using in this particular model, it was if you like, an estimate created by the project manager based on their experience of the amount of margin that they will need. Now, there are ways that we can estimate the amount of margin that you want. For example, going off script here, so things can always go wrong, but I can use this S curve here to say that if I want to be 80% confident of being able to deliver on time, then the 10th of September in this particular case is the date that I’m looking at. And I can then calculate the difference between the 10th of September and the deterministic finish, the 28th of August. And that difference is the schedule margin that will give me an 80% chance of being able to deliver by the 10th of September.
John Owen: We know in this particular case, that’s actually not good enough. The 79% date was 9th of September. I can do that, but yes, you can use this information to calculate a schedule margin value but we find most customers have some idea of what they want to put in there and oftentimes it is simply the difference between what the schedule they’ve built shows as a project finish date versus what the contract is. And they say, “That’s the schedule margin.” They do the date difference and put that number in a schedule margin and then Full Monte can tell them whether that’s enough.
Melanie: Okay. Thank you. We have one other question here regarding the recording. Can it be shared? We will be sending that out later this afternoon. John, is there anything else you’d like to mention in closing?
John Owen: No, thank you very much. I appreciate the opportunity to talk to you all. I appreciate the time that you’ve given me. Thank you, Laura, for coming up with a question to make me do some work for my living. Thank you everybody and thank you Melanie for hosting.
Melanie: Thank you. John, very many thanks. I will switch over here and pull up the PDU code for all of our attendees and make sure we have the right screen. There we go. A big thank you for our MPUGgers attending our Simplifying Schedule Risk Analysis today. Again, the MPUG PDU code for PMI is mpug071421. We will, as I mentioned earlier, send a link to the recording and a survey later today. It’s always helpful for our experts to get feedback from you. Please feel free to share this session with anyone that you know might be interested in this expertise that John shared with us. It is open to the public so they do not have to be a member to watch this session and recording. And as you consider your next career boosting investment in training, we have a couple things coming up.
Melanie: Project Requirement Management, that’s a three part course. It’s available on demand now. You can see that right on our homepage. August is going to bring us another three part course on building and managing diverse teams with Dr. Lynette Reed. And then September we have yet again, another live three part course, which is going to be exploring VBA. And this is a fun and informative course with our project management icon, Ira Brown. If you’ve experienced Ira, he’s wonderful. And the business user, the Microsoft Project business user will get an in depth understanding of the automation capabilities with the tool and practical illustrations solving the most common challenges with the tool.
Melanie: Again, thank you for joining us today. I’ll leave the chat box open. If there are additional questions I can share those or if there are questions for us and I will also pull up the PDU number again on the screen. Thank you again.
