The Challenge
One of the biggest unsolved opportunities in business software is the proper balance of flexibility and control.
Today, there are two separate worlds. There is the world of the "spreadsheet" and the world of "enterprise software". The world of the spreadsheet is; highly flexible, highly productive, low to no governance. The world of enterprise applications is: highly controlled, highly governed, but low agility.
What is needed is a proper balancing of these two worlds.
Flexibility is a paradox. To design a system effectively, you need to understand the range of flexibility represents a spectrum of control, governance, and adaptiveness. At one extreme lies complete unconstrained freedom, while at the other sits rigid, invariant control. There is no one "right answer" in terms of flexibility; flexibility is not "good" or "bad". Understanding where a system or process falls within this continuum helps highlight the trade-offs.
- Total Fluidity (Freeform; Absolute Anarchy)
- Guided Flexibility (Bounded Autonomy)
- Structured Standardization (Configurable Governance)
- Total Rigidity (Absolute Control)
A well-designed system is not one with maximum flexibility. A well-designed system has exactly the degrees of freedom (e.g. flexibility) necessary to achieve its objectives and no more.
The goal of a purpose-built operational spreadsheet environment is not to eliminate flexibility, but to intentionally engineer it. Rather than exposing infinite capabilities, the system should grant only the specific degrees of freedom required for users to execute their work effectively. Determining this optimal balance requires domain expertise beyond traditional software engineering; it demands the workflow optimization principles of industrial engineering, grounded directly in the operational user's specific goals.
Engineering an operational spreadsheet environment is an exercise in intentional constraint. The goal is not to strip away flexibility, but to carefully scope it; offering users precisely the degrees of freedom their responsibilities demand, and no more. Software engineers build the underlying architecture. Establishing the ideal equilibrium requires an industrial engineering mindset driven by the true intent of the operational user.
Evolution of Software
Historically, software evolution often follows this path:
General Tool >> Critical Use Case Emerges >> Purpose-Built System Appears
I heard once in the 1980s/1990s that Microsoft Excel was the largest development platform in the world. Not for professional software engineers; for "citizen programmers" or "end-users" that understood the tasks that they needed to complete but they either did not have an IT department to go to or the IT department was to slow. So, they built their own systems. I was one of those citizen programmers. Someone used the term "dinker" to describe what I was doing.
Eventually, when an idea was really good, a "roll your own" system gets converted into a category of software. For example, email and a spreadsheet were used in the sales process; then email/spreadsheet system was converted to professionally created CRM software. Or, documents created in a word processor evolved to become a content management system. Or, web page creation software evolved to become an E-commerce platform.
Is the spreadsheet about to go through a similar transformation? How is artificial intelligence impacting the traditional electronic spreadsheet? There are a set, a pattern, of critical use cases that have emerged. All one needs to do is go look at those shadow spreadsheets that exist in an organization. And there are the fragmented set of 800 electronic spreadsheets used by the average Fortune 1000 company to generate its compliance report which is sent to their regulator. This is not "normal". People see it as normal because they see no other existing alternative that works better.
What would "better" look like?
Objective
So, what exactly is the objective? The objective might be something like "create an operational environment for trusted business processes" or the "transform of business information into reliable operational decisions". The system is responsible for ensuring that the information is:
- Correct
- Complete
- Consistent
- Explainable
- Reproducible
- Traceable and trackable
- Auditable (e.g. information provenance)
- Governed
The system makes sure the information has those characteristics before the information is used to make a decision. In this age of artificial intelligence, you will have both humans and machines making use of information.
The spreadsheet becomes a trusted user interface into an enterprises "operational environment". The ERP system is another part of the operational environment. The data warehouses are another part of the operational environment. Or, might it be the case that every interaction with the operational environment becomes an "operational spreadsheet" or a "semantic spreadsheet" or "operational intelligence systems or "controlled operational spreadsheet systems".
Maybe we stop dealing with "data" and start dealing with "information"; my notion of the "digital information organism" or the "information Lego brick". All information is "pre-verified". Or as I have put it, a system of discrete stable units of information.
The Business Opportunity
My personal focus is on accounting, reporting, auditing, and analysis. I, and others, have "packaged" all of this under the notion of Accounting & Audit by Design (A&AD) Framework. This business opportunity for accounting, reporting, auditing, and analysis is absolutely a no brainer.
But the larger opportunity is even bigger than just accounting. Accounting is simply the use case that has (a) the most demanding requirements such as zero tolerance for error and auditability, (b) the most mature set of well understood shared semantics, (c) a deterministic system provided by the double entry bookkeeping mathematical model, (d) pretty much everything is mission critical because the information ultimately makes its way to a regulator that does not tolerate mistakes.
Many other "operational spreadsheets" exist and they share the same characteristics; they have the same "patterns" or use cases. They all tend to be:
- Mission critical
- Frequently updated
- Collaborative in nature (both creation and use)
- Subject to controls (e.g. governance)
- Subject to audit or review
- Used repeatedly
Examples from the accounting, reporting, audit, analysis world and other use cases include:
- Financial close (e.g. what I call the closing book, those 800 spreadsheets used by the typical Fortune 1000 company)
- Budgeting and forecasting (FP&A)
- Risk management
- Regulatory reporting (e.g. the audit bundle)
- Capital planning
- Pricing models
- Project portfolio management
These aren't really spreadsheets. They're operational systems that happen to be implemented in spreadsheets. That distinction is enormous. And the fact that these systems are spreadsheets is problematic.
Industrialization
To industrialize something means to turn it into a routinized, repeatable, scalable, reliable process that can be done the same way every time. That includes:
- Standardization: You define the steps clearly so the work is done the same way each time.
- Repeatability: Anyone (or any machine) can follow the steps and get the same result.
- Scale: You can do it not just once, but hundreds or thousands of times.
- Efficiency: You remove unnecessary variation, waste, or improvisation.
- Transfer from “craft” to “system”: Before industrialization: Work depends on individual skill, judgment, or memory. After industrialization: Work depends on a documented, controlled process.
A simple metaphor. If you cook a meal from memory, that’s craft. If you write a recipe that anyone can follow, that’s standardization. If you build a kitchen that can produce 500 identical meals a day, that’s industrialization.
Why this matters conceptually; industrialization is essentially the move from:
- tacit knowledge understood by an individual to explicit procedure “baked” into a process
- individual variation into a controlled process
- one off performance by a person to routinized, systematized production
The Business Outcome
The opportunity is not about replacing Excel or Google Sheets, or the traditional electronic spreadsheet. Excel and Google Sheets are incredibly useful tool and they will be with us for a long time to come.
The opportunity is to identify the relatively small set of business processes that are simultaneously:
- High value
- Repeated frequently
- Implemented in spreadsheets
- Too important to fail
- Need to be used by both humans and artificial intelligence based agents
Then; build an environment whose degrees of freedom (e.g. flexibility) have been deliberately engineered around the objective of producing trusted, explainable, auditable operational decisions.
If that can be achieved while preserving much of the usability and cognitive simplicity that made electronic spreadsheets successful in the first place, it could represent the closing of one of the largest remaining gaps in business software architecture.
Additional Information:
Comments
Post a Comment