Thursday, August 31, 2023

Stochastic Parrot for PMs?

One meme making the rounds is the "Stochastic Parrot" caricature of 'large language models' (LLMs) that underpin all the AI chat and query systems we're experiencing. 
  • "Stochastic" meaning something randomly determined; having a random probability distribution or pattern that may be analyzed statistically but may not be predicted precisely.

    Stochastic processes or events are different frohm a truly random phenomenom insofar as something truly random will have no predictable pattern or statistically viable distribution of outcomes. 

    "White noise" is stochastic; it's probability distribution is "uniform". A blip here and there is random; there is no pattern of outcomes that can be fit to a probability distribution.

  • "Parrot" suggesting that was the LLM is doing is just repeating back what others have said
And so you put those ideas together, and you get the characterization of an LLM as a parrot of ideas that are statistically combined in the most probable configuration. And, you wouldn't be far off with that idea. 

What about the PM application?
PMs are happy with a stochastic parrot outcome of statistically relevant ideas and data from a vast history database. From the very beginning, PMs have been encouraged to keep history; to keep records; and to apply the lessons of history to the next project. And so a LLM becomes a tool for just that purpose. 

As an example here's a "stochastic parrot" taken from ChatGPT 3.5 on the risk matrix, a common tool in project management.

The risk matrix is a commonly used tool in project risk management, but it has several potential limitations, including:

1.     Over-simplification: A risk matrix can reduce complex risk factors into simple numeric values and colors, which can lead to oversimplification of the actual risk.

2.     Lack of consistency: Different people or organizations may use different criteria to assign numeric values to risks, leading to inconsistencies in risk assessment.

3.     False sense of precision: The use of numbers and colors in a risk matrix can give a false sense of precision and accuracy, even though the underlying risk assessment may be subjective and based on limited information.

4.     Lack of context: The risk matrix does not provide enough context or information about the risks being assessed, making it difficult to understand the underlying causes and potential impacts of risks.

5.     Limited scope: The risk matrix only considers risks that have been identified and may not take into account unknown or emerging risks.

6.     Over-reliance: Risk matrices are often used as the sole tool for risk assessment and management, leading to oversimplification of complex risks.

7.     Limited detail: Risk matrices provide limited information about individual risks and often do not include mitigation strategies.

8.     Qualitative vs Quantitative: The use of qualitative information to assign risk scores in a matrix can lead to subjective interpretations and inconsistencies between users.

9.     Inflexibility: Risk matrices are static, making it difficult to reflect changes in risk as a project progresses.

10.  False sense of security: The use of color-coding and other visual aids can create a false sense of security about the level of risk present in a project.

11.  Lack of integration: Risk matrices are often used in isolation, without being integrated into the overall risk management process.

12.  Culture and politics: The way risks are perceived and prioritized can be influenced by organizational culture and political considerations.

Like this blog? You'll like my books also! Buy them at any online book retailer!