Modeling frameworks: What is a scientific model?
What is a scientific model? Every scientist has an intuitive notion of what constitutes a model in their discipline, and they might give one of the following definitions.
- A system of ordinary differential equations whose variables represent the state of a system \frac{dx}{dt} = f(x)
- A family of probability distributions that sample represent a data generation process y\sim f(x\mid \theta)
- A transfer function representing a linear time invariant system y(t) = h(t)*x(t)
- A mathematical function that computes properties of a system y = f(x)
Each scientific paradigm has a kind of model considered in scope for the discipline (“The Structure of Scientific Revolutions, (Third Edition)” 1997). There is a long history in the philosophy of science in studying the relationship between theory and models (Bailer-Jones 2009). The paradigms form what we call a modeling framework and they have two parts. There is always a language in which to specify the models and a way to give mathematical meaning to the models. In the language of programming language theory, we call these the syntax in which you express programs and the semantics that you use to evaluate them. In mathematical logic, these are often called a theory, which defines the grammar of statements and their axioms, and an interpretation, which give mathematical meaning to the terms and statements of the theory. A logician would call the interpretation a model of the theory (Tarski 1994). Theory \xrightarrow{F} Model The syntactic realm is comprised entirely of formulas that express statements in a logic and rules for manipulating those statements to build proofs within the theory. A logician is interested in building a mathematical model of proofs. A logician is interested in building a mathematical model of proofs. In the early 20th century, there was a movement to axiomatize all of mathematics which lead to the notion of a formal proof as a mathematical object. The study of these formal proofs was called proof theory.
Category Theory is also a type of meta-mathematics, which studies the ways in which mathematical structures recur in different contexts and builds rigorous methods for making analogies between branches of mathematics. Since we want to build computer programs that create and manipulate mathematical models in every area of science, a meta-mathematical tool like category theory seems appropriate.
In AlgebraicJulia, we take this logical paradigm and apply it to scientific modeling frameworks. We define a modeling framework as a generalized algebraic theory along with a structure preserving map into Julia Data Structures and Functions. The GAT provides the logical syntax and rules for manipulating the syntax expressions to perform valid constructions describing large scale systems in terms of their components (Cartmell 1986). GAT \xrightarrow{Instance} Julia For a typical GAT there are Julia types representing the types in the GAT. Terms in the GAT map to values of those types in Julia. The GAT term constructors get mapped to julia functions that construct bigger values from component values. The GATs can have axioms which assert that certain terms are equal. A common axiom is associativity, which allows you to reorder computation by moving parentheses in an expression when it doesn’t change the mathematical meaning.
This component of AlgebraicJulia relates to model specification and system construction. We can accept as input a description of a GAT (in terms of types, type constructors, term constructors, and axioms) and generate a language for formulating systems within the that GAT. A functor from your GAT to Julia Types is called an instance. Instances capture all the information necessary to instantiate terms of your theory as models implemented as Julia data structures.
The structure preserving map from your GAT to your instance is called a GAT homormorphism and generalizes the notion of a group homormorphism, which preserves the structure of a group, namely its associative multiplication with identity. Every group homomorphism associates every element in the domain to an element of the codomain such that the multiplicative structure is preserved. Any homomorphism from a GAT to an Interpretation is a little interpreter for the syntax of your theory. It gives a denotational semantics in terms of Julia types (GAT types) and values (GAT terms). Applying a functor to a GATExpr is basically running a LISP interpreter on a LISP program to transform input to output. Taking the example above you can walk the expression tree recursively to compute by computing F(expr) = F(expr.head)(map(F, expr.args)) at each node in the tree, you interpret each of the arguments and then apply the node operation to the interpretations of the arguments.
Each AlgebraicJulia library implements a modeling framework in this sense. For example in AlgebraicPetri, you can specify a compartmental model of epidemiology using a Catlab syntax for hypergraph categories. Hypergraph categories are a mathematical model of networks and capture the essence of hierarchical interconnection. In AlgebraicPetri, you specify a reaction network by building it up from component reaction networks using a GATExprs in a Hypergraph Category. The interpreter for this language converts your description into a Petri Net, which is a bipartite graph with two kinds of nodes. Petri Nets have a state for every population in your epidemic and a transition for every event that can happen in your epidemic.
The edges in a Petri Net tell you for every transition, which populations are inputs and which populations are outputs. From this description you know everything there is to know about the structure of the disease. There is a functor F from Petri Nets to Differential Equations defined by the law of mass action which converts any Petri Net into a system of coupled ODEs. These ODEs have for every population in the system a variable in the ODE and for every transition in the Petri Net, a set of terms in the ODE vector field. The hypergraph category operations on Petri Nets become superposition of vector fields. Petri \xrightarrow{F} Dynam In this setting a Petri Net forms the syntax for describing a model, which is then interpreted as denoting a specific dynamical system. The SIR model of epidemiology is a classic example whose Petri Net formulation and Dynamical System interpretation is given below.You might have noticed that the drawing of a Petri Net looks is a particular type of graph with some information about reaction rates stored on some of the nodes. In Graphs and C-sets 1 we showed the schema for a graph and discussed how C-sets can represent more exotic notions of graph. Petri Nets are one such exotic notion of graph, with the following indexing category. This indexing represents an initial value problem on a Petri Net, by encoding the species (blue nodes) and reaction (orange nodes) along with directed multiedges between them (inputs and outputs). Each species has an initial concentration and each reaction has a reaction rate. From this data and the principle of mass action kinetics, a differential equation and initial value problem is fully specified. The AlgebraicPetri documentation explores this C-set approach to modeling dynamical systems.
The next step in the development of AlgebraicJulia is algorithms that exploit hierarchy not only in model construction, but also in model simulation. This means building solvers that can understand the language of the GAT and use the axioms of the theory to improve the time to solution of a solver. Most modern solvers for large scale systems will exploit sparsity, which is the fact that not all parts of a system interact with every other part. When engineers design systems, they break them down into subsystems and recursively design those subsystems and combine them. In this way the process of engineering builds sparsely constructed systems. The next step in advanced solvers is to exploit this hierarchical structure to optimize computation.
Applied Category Theory provides hypergraph categories, which are a very general mathematical model of hierarchy that captures the notion of hierarchically interconnected systems of systems. Hypergraph categories are prevalent in many areas of computational science including Matrices, Graphs, Markov Chains, Reaction Networks (Petri Nets), Resistor Circuits. All of these modeling frameworks provide different interpretations (models) of the theory of hypergraph categories. Any model in these paradigms represents a system that can be broken down and studied in terms of its components and the pattern of interconnection between them. That compositionality is essential for building a cumulative understanding which is the foundation of scientific progress.