Stochastic Parrot for PMs?

---

One meme making the rounds is the "Stochastic Parrot" caricature of 'large language models' (LLMs) that underpin all the AI chat and query systems we're experiencing. 

• "Stochastic" meaning something randomly determined; having a random probability distribution or pattern that may be analyzed statistically but may not be predicted precisely.
  
  Stochastic processes or events are different frohm a truly random phenomenom insofar as something truly random will have no predictable pattern or statistically viable distribution of outcomes. 
  
  "White noise" is stochastic; it's probability distribution is "uniform". A blip here and there is random; there is no pattern of outcomes that can be fit to a probability distribution.
  
  
• "Parrot" suggesting that was the LLM is doing is just repeating back what others have said

And so you put those ideas together, and you get the characterization of an LLM as a parrot of ideas that are statistically combined in the most probable configuration. And, you wouldn't be far off with that idea. 

What about the PM application?

PMs are happy with a stochastic parrot outcome of statistically relevant ideas and data from a vast history database. From the very beginning, PMs have been encouraged to keep history; to keep records; and to apply the lessons of history to the next project. And so a LLM becomes a tool for just that purpose. 

As an example here's a "stochastic parrot" taken from ChatGPT 3.5 on the risk matrix, a common tool in project management.

The risk matrix is a commonly used tool in project risk management, but it has several potential limitations, including:

1.     Over-simplification: A risk matrix can reduce complex risk factors into simple numeric values and colors, which can lead to oversimplification of the actual risk.

2.     Lack of consistency: Different people or organizations may use different criteria to assign numeric values to risks, leading to inconsistencies in risk assessment.

3.     False sense of precision: The use of numbers and colors in a risk matrix can give a false sense of precision and accuracy, even though the underlying risk assessment may be subjective and based on limited information.

4.     Lack of context: The risk matrix does not provide enough context or information about the risks being assessed, making it difficult to understand the underlying causes and potential impacts of risks.

5.     Limited scope: The risk matrix only considers risks that have been identified and may not take into account unknown or emerging risks.

6.     Over-reliance: Risk matrices are often used as the sole tool for risk assessment and management, leading to oversimplification of complex risks.

7.     Limited detail: Risk matrices provide limited information about individual risks and often do not include mitigation strategies.

8.     Qualitative vs Quantitative: The use of qualitative information to assign risk scores in a matrix can lead to subjective interpretations and inconsistencies between users.

9.     Inflexibility: Risk matrices are static, making it difficult to reflect changes in risk as a project progresses.

10.  False sense of security: The use of color-coding and other visual aids can create a false sense of security about the level of risk present in a project.

11.  Lack of integration: Risk matrices are often used in isolation, without being integrated into the overall risk management process.

12.  Culture and politics: The way risks are perceived and prioritized can be influenced by organizational culture and political considerations.

---

Like this blog? You'll like my books also! Buy them at any online book retailer!

AI Threat model Framework

---

Somewhat very akin to the conventional "use case" structure that many are familar with, and borrowing the power of a conversational narrative from Agile methods, and sundry psychology systems, Daniel Miessler posits a threat model framework which he says is a target for for policy makers, but I say it's a target for any PMO working in software services. You can read it here.

In his framework, Miessler has these components:

• The ACTOR, which can be an individual, an enterprise, or a system
• The TECHNIQUE, which is the method or process, like hacking, will cause harm.
• The HARM, which is the functional outcome of the harm, like 'misinformation'.
• The IMPACT, which is what happens when the 'harm' reaches its target. One example might be financial loss. 

As PMs we're familair with frameworks and how to apply them. Miessler writes that the objective of his framework is to talk about the AI threat in a converational way that will draw in policymakers. He says this: 

What I propose here is that we find a way to speak about these problems in a clear, conversational way.

Threat Modeling is a great way to do this. It’s a way of taking many different attacks, and possibilities, and possible negative outcomes, and turning them into clear language that people understand.

---

Like this blog? You'll like my books also! Buy them at any online book retailer!

Kano: Utility at work

---

"Customer value", aka "the value proposition", is complicated. 

Books fill the shelves on those topics. 

• What do 'early adopters' value?
• How does age come into play?
• Is economic willingness different from economic capability in the value equation?
• How do culture and relationships figure in the proposition?

All good questions, to be sure. 

But the missing quality is "UTILILTY", specifically "marginal utility." Marginal utility is the added satisfaction a user (customer, or consumer) gets from having one more unit of a product or service. 

Marginal utility separates the "willing" from the "capable", given the capacity to spend. The whole idea of working on the margin begs images of value plots that could be anywhere from linear (the last unit is just as valuable and desired as the next unit) to various shapes of non-linear (the last and next units' values are different)

One might ask: is there a way to map all this stuff so that a picture emerges? Yes. Kano Analysis may help see the bigger picture.

What is Kano analysis?

Kano analysis is a product feature/function evaluation tool that gives visualization to relative merit over time as trends change. The usual presentation is a four-sector grid with trend lines that connect the sectors. 

The grids are defined by the horizontal and vertical scales. Don't take the word 'scale' too seriously; for the most part the scales are non-calibrated, but informed, opinion:

• Vertical: customer attitude, feeling of satisfaction, or other elements of value appeal.'
• Horizontal: some quality (or metric) of the feature/function that's important to the customer.

 

Trends and Utility

The trends plot the utility of changing customer satisfaction (vertical) as a function of product functionality (horizontal). These plots vary from linear to nearly exponentially non-linear.  And, the utility of satisfaction need not maintain only one direction; direction can change, trending up or down, as customer/user attitudes change.

 

Developers use the Kano board with sticky notes to show how feature/function in the form of stories or narratives might play out over time.

 And, we take the trouble to do this because:

• There's only so much investment dollars available; the dollars need to be applied to the best value of the project.
  Presumably that's the "ah-hah!" feature, but the "more is better" is there to keep up with competition; and, some stuff just has to be there because it's commonly expected or need by regulation.
  
• Trends may influence sequencing of iterations and deliveries. Too late, and decay has set in and the market's been missed.
• The horizontal axis may be transparent to the customer/user, but may not be transparent to regulators, support systems, and others concerned with the "ilities". Thus, not only don't forget about it, but actually set aside resources for these 'indifferent' features and functions.
  

How far ahead of the trend can you be and not be too far ahead? Just a rhetorical question to close this out.

---

Like this blog? You'll like my books also! Buy them at any online book retailer!

The tale of the tails

---

Your data analyst comes to you with tales of the tails:

• Yikes! Our tails are fat!
• Wow! Our tails are thin.

What's that about?

If you're into big words, it's about the "kurtosis" of the data, a measure of the distribution of data around the mean or average of a bell-like distribution of probabilities. More or less kurtosis means more or less data, respectively, in the tails of the bell-like distribution.

It's about risk and stability

If you don't care about the big words, but you do care about risk management and volatility or predictability that could affect your project, then here's what that is about:

• Fat Tails: If there's more data in the tails, farther from the mean, then there is correspondingly less data clustered around the mean. Interpret fat tails as meaning there are more frequent outliers and more non-average happenings, meaning more volatility and less predictability than a normal "bell curve" of data points.  
  
  
• Thin tails: Really, just the opposite of the fat tails situation. Thin tails means less data in the tails, and the outliers, such as they are, are many fewer. There is a concentration around the mean that is more prominent than the usual bell curve.
  
  Interpretation: more stability and predictability than even the steady-Eddie bell curve, because most happenings are clustered around a predictable norm. 

Is there an objective metric?

Actually, yes. From math that you don't want to even know about, a normal "bell curve" has a kurtosis of "3". Fat tail distributions have a figure greater than 3; thin tail distributions, less than 3. Note: some analysts normalize everything to "0" +/-, rather than "3" +/-.

Excel formula:

As luck would have it, there is a formula in Excel for figuring the kurtosis of a data set. "KURT" is the formula, and you just show it your data set, and Excel does all the work! But as a PM interested in risk to your project, you just need to know from your analyst: fat, thin, or normal.

---

Like this blog? You'll like my books also! Buy them at any online book retailer!

Balancing Expectations: Sponsor and PM

---

If you follow this blog you've read several references to the project balance sheet. So, is this about accounting? Yes, and no: Yes, it's about a double entry tool to keep track of "mine" and "yours", but no, it's not the accountant's tool used in your father's accounting office.

Take a look at this figure:

What have we got here?

First, the business and the project; but also what's mine -- the project stuff -- and what's yours -- the business stuff. Mine and yours!

First, the left side
On the left side of the balance sheet is the sponsor's investment in the project. Investment need not be all monetized, and it need not be all tangible. Sometimes 'good will' -- the accountant's name for the intangible gut feeling that this thing is worth more than book value (market-valued assets) -- counts for a lot. (Think: sponsor commitment, even when the going gets tough)
'Yours' simply means it's resources and commitments owned and given by others to the project. It's the 'your's side of the balance sheet that's somewhat akin to the right side of the financial balance sheet (money owed to creditors and money invested by owners).

Then, the right side
On the right side is the 'mine' side of the project balance sheet, akin to the left side of the financial accounting sheet (assets of the enterprise). The right side is the project side:

• Estimates and evaluations of the project manager
• Uses for the investment and resources to be entrusted to the project manage -- in effect deliverables and other artifacts, and perhaps some intangibles as well*

All about facts
 And, take note: the left side, the sponsor's side, is the fact-free zone: it's a top down allocation of resources to the vision. It is the ultimate utility expression of the sponsors: what's valuable, and how valuable, even if not entirely objective.

And on the right side, it's all about facts (benchmarks) and estimates (benchmarks applied to project circumstances). It's bottom up.

The gap
Of course, there's the inevitable gap where utility collides with facts and fact-based estimates. The gap is the risk between expectations and capacity-capability. And how large is the gap (risk): only as large as needed to create a balance--that is, a deal with the devil--so that the project can go forward.

 In other words, the gap (risk), shown on the project side, is only as large as it needs to be to close the gap. Usually, it's a matter of negotiation, but once the PMB is set, the risk is the PM's responsibility to manage.

Oops! the PM is the ultimate risk manager.

In a real world example, I had this situation:

• We bid a job competitively in a firm fixed price environment. 
• We offered a price that was equal to our cost; in other words, no fee (profit).  We just wanted to keep the lights on and keep barriers to competition with our customer as high as possible. 
• We won! 
• And, in  the next moment, my general manager said: "Your bonus depends on making 4% net margin".  I had my gap!  (oh yes, I made the margin and the customer was satisfied)

---

* Yes, indeed! Projects produce intangibles, even good will, but it makes for an accounting of project value all the less objective.

---

Like this blog? You'll like my books also! Buy them at any online book retailer!

Threats, vulnerabilities, and Risks

---

Daniel Messiler has an interesting essay about threats, vulnerabilities, and risks that is worth a quick read.

He summarizes this way:

•  A Threat is a negative scenario you want to avoid; damage or loss of an asset; a danger
• A Threat Actor (person, entity, organization, is the agent that makes a Threat happen; threats and threat actors are cause-effect related.
  
  
• A Vulnerability is a weakness that can be exploited in order to attack you; vulnerabilities may enable threats. Security weaknesses in data and communication systems are a common vulnerability.
  
  
• A Risk is a negative scenario you want to avoid, combined with its probability and its impact. Risk and threat the same? No, because a threat is deterministic whereas a risk is probabilistic. 
  
  The difference between a Threat and a Risk is that a Threat is a negative event by itself, where a Risk is the negative event combined with its probability and its impact
  
  

---

Like this blog? You'll like my books also! Buy them at any online book retailer!

Stability! It's a good thing

---

Stability!

It counts for a lot.

It implies -- for behaviors and management decisions -- predictability, reliability, under-control (but not risk-free, of course), coherent narrative, steady-state goals, and a strategy that is understandable to those who have the job of implementing it.

Perhaps you are aware, as many are, that stability requires feedback to effect error correction and trap excesses and blind alleys. 

Ah yes!

We know about feedback.

Open loop systems -- those with outcome but no feedback -- are prone to many uncontrolled and unexpected responses. Who can predict what a stimulus will do to a system that has no feedback? Actually, that's a really tricky task.

So, what about feedback? 

What's to know?

• Timing is everything! Getting the feedback "phased" in time such that it has a correcting effect rather than a destructive effect is vital. The former is generally called "negative feedback" for its corrective nature; the latter is generally called "positive feedback" for its reinforcing rather than corrective nature. And, when its too late, it's generally called ineffective.
  
  
• Amplitude, or strength, or quantity is next: It has to be enough, but not too much. Tricky that! Experimentation and experience are about the only way to handle this one.

What could possibly go wrong?

Actually, a lot can go wrong.

No feedback at all is the worst of the worst: the 'system' is 'open loop', meaning that there are outcomes that perhaps no one (or no thing) are paying attention to. Stuff happens, or is happening, and who knows (or who knew)?

Timing errors are perhaps the next worst errors: if the timing is off, the feedback could be 'positive' rather than 'negative' such that the 'bad stuff' is reinforced rather than damped down. 

Strength errors are usually less onerous: if the strength is off, but the timing is on, then the damping may be too little, but usually you get some favorable effect

Practical project management

Feedback for correcting human performance is familiar to all. Too late and it's ineffective; too much over the top and it's taken the wrong way. So, timing and strength are key

But, the next thing is communication: both verbal and written (email,etc). Closing the loop provides reassurance of the quality and effectiveness of communication. You're just not talking or writing into the wind!

And, of course, in system or process design, loops should never be open. Who knows what could happen.

I should mention:

The study of feedback systems generally falls within what is called 'cybernetics'. As described by sciencedirect.com, MIT mathematician Norbert Wiener defined cybernetics as “the study of control and communication in the animal and the machine." 

From Wikipedia, we learn: The core concept of cybernetics is circular causality or feedback—where the observed outcomes of actions are taken as inputs [ie, feedback] for further action in ways that support the pursuit and maintenance of particular conditions [ie, 'ways that support' requires the correct timing and strength]

---

Like this blog? You'll like my books also! Buy them at any online book retailer!